Omnimaga
Omnimaga => News => Topic started by: DJ Omnimaga on April 10, 2012, 09:10:27 pm
-
Omnimaga has experienced unscheduled downtime from approximately 10:40 PM yesterday to 2:45 PM today. It was caused by a direct denial of service (DDoS) attack on his VPS by hundreds of IP addresses. Although we have no clue about who and where this attack originated from, Omnimaga should hopefully be functional again now, aside from timezone issues.
UPDATE (April 12th 2012) To clarify on the other downtime that happened the day after this got fixed, it was yet another DDoS attack.
-
Yay.
-
So that's why Netham wasn't responding.
I wonder who did it and why.
-
Well netham45 not responding was possibly due to him being out of town. I doubt the DDoS attack had any impact on his ability to get online, since he was most likely connecting from a different IP. Hadn't he been able to get online for a bit just now, however, we would probably have remained unavailable for even longer.
As for who did it, it could possibly be somebody who dislikes Omni and wanted to disrupt its activity/visits flow, or maybe it is related to a recent IRC ban/gline (since a few days ago scripts were tested again on Juju's server, causing downtimes)...
And yeah timezones are messed up on the server side. All posts until downtime started are fine, but today's posts are 6 hours ahead...
-
o.O maybe it was sircmpwn.. or KK (nah.. it couldn't be him, KK i dont think has anything to do with omni)
-
I doubt it, otherwise Cemetech would be affected too. And it could very well be a Youtube troll that absolutely hates calculator gaming and are totally against it that could have launched an attack or something anyway. It's best to not jump to conclusions about who might have done it, since there are so many possibilities, including someone with too much free time on his hand finding a random site to test scripts.
-
bah.
KILL TEH DDOS-ERS!!!!
-
Or maybe someone was tired of losing The Game? O.O
Anyway I guess that ends our 100 post/day streak, especially with the messed up timezones D:
-
umm why? (the timezone stuf)
-
Because all posts from today are dated from 6 hours ahead (old posts are fine, though). Basically in 1.5 hour it switches to tomorrow in board stats.
-
I thought it was my router at first, so I unplugged it and plugged it back in again like three times, then I msged Juju :P
Yay.
lol
<--
-
My first reaction was to go check netham45.org, in case it was just DNS issues, and ask Eeems if he knew if Netham45 had returned and was doing server stuff. However I was a bit worried when I read that Netham45 did not answer back (since he was in vacation).
-
I wondered what was going on, but I'm glad things are up and running again. =)
@Snake: You really shouldn't even suggest things like that. Especially if it is just speculation.
-
Something odd, why is it that when I hover over specific avatars, I get some weird ad thing?
http://dl.dropbox.com/u/10573921/Weird.png (http://dl.dropbox.com/u/10573921/Weird.png)
-
That is weird. I wonder why it happened.
-
Prolly a virus.
-
Something odd, why is it that when I hover over specific avatars, I get some weird ad thing?
http://dl.dropbox.com/u/10573921/Weird.png (http://dl.dropbox.com/u/10573921/Weird.png)
I think you are infected by some virus or something. Otherwise we got hacked.
EDIT: Definitively a virus you got. I tried in Chrome and didn't get those ads you got in it.
-
homer check your chrome addons you have something funny in there im sure
-
Oh, that's why it was down all of the sudden. D:
I saw the OmnomIRC shutting down, and Omni didn't work after that.
-
Now I guess what remains to be solved is the Chrome-wide issue causing people to be unable to load most pages. It seems to be related to the new anti-spam, though.
-
Something odd, why is it that when I hover over specific avatars, I get some weird ad thing?
http://dl.dropbox.com/u/10573921/Weird.png (http://dl.dropbox.com/u/10573921/Weird.png)
Did some looking, is this what you are having? http://wwws.superfish.com/ (http://wwws.superfish.com/)
and if you haven't already looked into it http://forums.mozillazine.org/viewtopic.php?t=1979591&f=38 (http://forums.mozillazine.org/viewtopic.php?t=1979591&f=38)
-
WTF I thought my computer couldnt access the internet yesterday because of this!
-
Ha Ha Chrome is still giving me this,
(https://lh4.googleusercontent.com/-arP92EONnNA/T4T11lD5aiI/AAAAAAAAAPM/bQPcbRGW_dE/s438/screen1.png) :D
-
Lol weird. At least it doesn't show up in different colors like Netham45 a few years ago. :P
Also The real posting stats for today are messed up because of timezones:
date topics posts signups max-online views
2012-04-10 1 57 0 112 5454
2012-04-11 4 71 0 105 9252
Normally it should have been the following:
2012-04-10 5 128 0 112 14706
Might be good to substract the 2012-04-11 stuff above from the stats once timezones are fixed I guess. Normally there were 128 posts today, but the stats shows 57, because when the site came back online, the server went 6 hours ahead. I guess technically we didn't end our 100-post day posting streak, despite being up for only 9 hours today! :)
-
Wow... I thought Netham45 just nuked his server for fun :P Apparently it's something a lot worse...
Is he planning to install some kind of DDoS protection? Also, have we figured out the culprit? Google tells nothing about any 4chan DDoS planning... :P
-
im glad omni is back up
-
WTF I thought my computer couldnt access the internet yesterday because of this!
I always go to http://www.google.com.au (in my case) just to check that it's not just omnimaga that's down.
Wow it must have been bad timing yesterday that I was online in IRC during the time it went down. I believe it went for longer than 3 hours though.
Wow what a fail I just read the times wrong and thought it was 3 hours without actually reading the title topic! Yes 16 hours makes much more sense.
I really hope that this doesn't happen again (though it probably will :P), downtimes like these are not good for Omnimaga
-
Yay!
Omnimaga is not down anymore! I'm relieved.
I don't understand how people can attack a website like Omni... Who could be offended by calculator programmers ?
-
At this we still aren't sure who is attacking. Good point Chockosta.
Also the forum time offset is now 6 hours less! Which I guess it is ok, however eg: UTC+10 is now 8. It'd be good if the time offset were set so that it fit with GMT/UTC time zones.
-
Hmm, I wonder if there is a way Netham could take the offensive...
What I am thinking is that since it was from a bunch of IPs, it came from a bunch of "zombies." (I think that is the term I learned it as-- pretty much, a bunch of computers that can be used normally, but can also be controlled by one person because of a virus). Maybe, if you could find a way to trace back to the roots, you could give them a virus of your own making. Make it destroy their computers, monitors, who knows. I know there used to be a trick where you could cause a few components to overheat and destroy the computer using a virus.
-
I'd love to see that happen but I do believe it would get whoever did it into legal trouble...
-
That'd be funny to see, but some Zombie computers are actual PC's that people don't realize sometimes they have such a virus. :/
It could be destroying private property of individuals who have no involvement, unfortunately. :P
-
That'd be funny to see, but some Zombie computers are actual PC's that people don't realize sometimes they have such a virus. :/
It could be destroying private property of individuals who have no involvement, unfortunately. :P
That's why I said trace it back to the roots instead of to the zombies :D
-
what it DDosed again? all the posts are from yesterday?
-
Yes, it was.
-
install DDos protection
-
is there?
-
Did it cut out again today (apr 11) or is my computer/internet just being stupid. Reason i ask is that I couldn't access the site all day :-\ ???
-
Yeah, today I couldn't access for a while either
-
Yeah, Omnimaga was inaccessible for part of the day.
-
I didn't have it today either. I thought my phones browser was messed up. :/
-
if this continues, it's not good. D:
-
Well, not at all.
-
Writing a virus to counter a virus... not exactly a great idea if you put it that way :P
However, if you know the IPs and the virus, disassemble it, and then figure out a way to control the botnet, you might be looked favorably by authorities (and gain some fame, too!).
Anyway... are the logs of the DDoS IPs going to be made public?
-
I would like to see those ip's
and that would be cool alberthrocks
-
We'll be possibly looking into a new server tomorrow. The issue appears to be on my hosts side (My VPS just randomly loses internet, the bandwidth graphs drop immediately to 0, yet the VPS appears to be running completely normally if I connect through the console login, asides from the lack of network connectivity). I've contacted him about the issue 3 times now and received nothing meaningful in response whatsoever (which for how much I'm paying for the VPS is completely pathetic.)
I moved part of what I have hosted on there (completely unrelated to Omnimaga, although I suspect it's what is getting DoS'd) over to Amazon's EC2 server earlier, and it seems to be a really nice and proper interface that may be a decent alternative.
-
So you think it's not omni that is being ddosed?
-
that makes more sense, i mean who wants to DDoS a cac website??
-
Several people ;)
Anyway, I hope the problem goes away.
-
So you think it's not omni that is being ddosed?
No, I don't think it's Omni.
-
Yeah, keep in mind other websites unrelated to Omni but managed by Netham are hosted on the same server.
And yeah, Amazon EC2 sounds like a good idea to me.
-
Yeah, keep in mind other websites unrelated to Omni but managed by Netham are hosted on the same server.
This. I have a half-dozen or so sites on there.
As far as the DoS goes, it shouldn't have been enough to bring down the server, but the host was crashing (as in, outside my guest OS), and dropping my networking until I rebooted.
-
Something odd, why is it that when I hover over specific avatars, I get some weird ad thing?
http://dl.dropbox.com/u/10573921/Weird.png (http://dl.dropbox.com/u/10573921/Weird.png)
Btw, I figured out what this was. It was a "Read Later" extension I had installed. Odd how that suddenly incorporated the image ad thing :s
-
If the DDoS attack targets Omnimaga in particular and was from someone that really hates us and intends to continue in long terms, do you think it's blockable or will Omni have to continue living at the mercy of those attacks? Also if every IP/hostnames used for the attack are proxies, does it makes it impossible to trace which people(s) initiated the attack? (although it could be a virus installed on people's computer that performs the attack without them noticing).
Also could an exploit in the new anti-spam be what allows those attacks to happen? It seemed strange that the attacks starts happening shortly after the speed throttling anti-spam (what also causes Omni to be unreachable in Chrome for many people) gets added.
-
So you think it's not omni that is being ddosed?
No, I don't think it's Omni.
Oh of course how could I forget, netham45.org was down too at the same time. Therefore it's not a targeted attack on Omni phew. I think the news article can be a bit misleading too then. Maybe we should add that several sites were down too?
-
Is it related to BBaS/BMaC? Back in 2008-09 random people from Steam kept joining our IRC channel to rant at you for hacking or something in Team Fortress 2, and I recall this server is still used for that stuff.
-
The issue appears to be on my hosts side (My VPS just randomly loses internet, the bandwidth graphs drop immediately to 0, yet the VPS appears to be running completely normally if I connect through the console login, asides from the lack of network connectivity). I've contacted him about the issue 3 times now and received nothing meaningful in response whatsoever (which for how much I'm paying for the VPS is completely pathetic.)
(http://vpssd.com/images/netham.png)
Also, that is totally purple, not blue.
-
What kind of random comment is this O.o
-
Actually he has access to the site data since he's CalcGames admin and most likely helping manage Barrett's server (on which Netham45's VPS is hosted). The data above is Netham45's traffic during the DDoS attack.
-
The data above is Netham45's traffic during the DDoS attack.
The alleged attack. It is quite clear that there wasn't. This means that whatever problems he's having is local to his server configuration itself, and there is nothing we can do about it.
-
Looking at the graph clearly shows a huge spike of incomming data before each downtime.
To me this looks like a DDoS.
-
If 600 kbps is enough to knock his server offline, he has it configured wrongly. Also, a DDoS would be a LOT more than 600 kbps. I can manage 2mbps on my home connection. A DDoS by definition has multiple connections involved. It would be a lot higher.
-
It does beg the question though, why *does* the incoming data spike before the crash?
-
Couldn't it just have been a DoS attack, as in coming from one single Internet connection, but spread across multiple IPs to make banning harder? Or maybe a small-scale DDoS attack from two or three people?
Shortly after Omni's downtimes, #omnimaga and #cemetech were hit by spam attacks. #omnimaga-fr and #omnimaga-radio were hit too, meaning the person or group of people who did it either knew the community rather well or was browsing netham45.org folders.
-
Hmm also what does the "totally purple not blue" mean? O.o
-
Yeah I am unsure. Unless he means the site design being purple and Netham was saying it was blue or something.
In any case, however, site loads perfectly now for me and I have not gotten any downtimes ever since. It even loads faster than before the DoS/DDoS/whatever it was. It seems strange that immediately after the DDoS attack issue gets solved, that #omnimaga, #omnimaga-fr, #omnimaga-radio and #cemetech all get hit by massive spam attacks, though, and that the day after, we receive the visit of an unusually obnoxious new member on IRC that I will not name. This tells me someone could have been targetting Netham45 stuff and whatever he thought that was affiliated with him, someone taking revenge against a ban in one of the channels, or some Casio/HP fanboy/calc gaming hater wanting to have fun on TI websites.
-
People on #cemetech keep reporting the site being down a lot, is this supposed DDoS still occurring? Perhaps someone is exploiting a glitch in one of the site projects like OmnomicIRC? Cemetech has had no lag or downtime at all, other than the self-inflicted database load yesterday from me importing all the UnitedTI data.
-
Do they use Chrome?
I know there appears to be an issue accessing the site in Chrome since Netham45 added an anti-spam recently. In my case I had the problem and after a few page loads, it takes like 20-60 seconds for pages to load. In other browsers, such as IE9, Opera and Firefox we have absolutely zero problem accessing the site.
-
People on #cemetech keep reporting the site being down a lot, is this supposed DDoS still occurring? Perhaps someone is exploiting a glitch in one of the site projects like OmnomicIRC? Cemetech has had no lag or downtime at all, other than the self-inflicted database load yesterday from me importing all the UnitedTI data.
I don't go on cemetech that often but every time it's always been up for me with no lag or downtime really at all.
Also the anti-spam is blocking out many ip addresses which means I have to type in a captcha D:
-
Oh Kerm meant people in #cemetech were saying Omnimaga seemed down a lot.
Also when do you have to type the captcha? Is it when posting or when logging in?
-
I was getting 3Mbpsish bursts, and the networking on the host was dying. The stack on the client was fine, and it was simply reporting that no packets were going in/out of the server on the card. I moved the site that was getting DoS'd to a new box, and it works fine using an identical config to the current box, even after nightly DoS's since. I'm not sure what your experience with Windows Server is, but it's not exactly easy to drop the entire networking stack requiring a reboot with it. Also, assuming that 2Mbps of any sort of data provides an equal load is incorrect.
That's not even considering the frequent network outages on the datacenters side. It's commonly gone down for 4+ hours at a time, with frequent half-hour or so periods of downtime/high packet loss almost nightly.
Another issue is the frequent disk lag. I've had multiple days when I was unable to open a program or copy a file due to disk I/O taking seconds to respond.
All in all, your service was an unstable lag-infested piece of crap. I didn't really mind until I started trying to sell stuff off of my server, only to find people bashing my stuff after repeated downtimes due to your end.
As far as you posting graphs of my bandwidth usage, I consider that private data that you were out of line posting. You've even made comments to Barrett in the past about posting bandwidth statistics and information in #tcpa. Highly unprofessional.
-
What was the site by the way? I kinda suspected BBaS/BMaC or other HL2 stuff. That was sources of problems before.
As far as you posting graphs of my bandwidth usage, I consider that private data that you were out of line posting. You've even made comments to Barrett in the past about posting bandwidth statistics and information in #tcpa. Highly unprofessional.
Also Netham45 given who posted the graphs and which people are involved on one of the site they co-hosted (Omnimaga), I am not surprised <_< (no offense intended of course, but I do know Omni and some of us did not have a very good reputation among people involved with #tcpa/CalcGames/etc, which I can understand, though, considering the methods we have used to steal the #omnimaga channel from Nikky on May 18th 2008. :P)
-
The anti-spam measure to register is what ajanata was referring to when he said it's purple (not blue).
Anyway, I'm not going to sit here while you unjustifiably trash my service.
First of all--you contacted me once (not 3 times) about your network going down. I replied twice (after looking at graphs and logs that showed no evidence of a problem on our end) with troubleshooting questions/advice, and you never responded.
2.) You're in for a big surprise if you think you're paying a lot of money for a Windows VPS with the amount of RAM that you have.
3.) The fact that rebooting your VPS fixed your networking virtually proves that it was not something on the host side (this is ignoring the fact that no one else had problems).
4.) Even if it were something on the host side (which there is no evidence of), you can not blame the service for you getting DDoS'd. Shutting off your networking is a perfectly acceptable way to deal with a client getting attacked (even though we did no such thing).
5.) It has *never* gone down for "4+ hours." The longest datacenter outage was roughly an hour or an hour and a half, and this was because of a very serious hardware failure. The longest after that? Maybe 10 minutes (half-hour tops), and this only for people going through Level3. And that hasn't happened at all for about 6 or 8 months. It happens. They fixed it. Saying "It's commonly gone down for 4+ hours at a time" is a flat-out lie.
6.) No one else has had the disk lag problems that you've had, with the exception of one stretch of an hour or two. That should be telling. And what was I doing during that time? Troubleshooting. Helping you. Figuring out what was happening. Telling the client who was responsible (who pays way more than you) that he was causing problems for other people. Researching ways to limit disk I/O (which, by the way, would hurt you far more than anyone else).
7.) Ever heard the story of the boy who cried wolf? I can't count the number of times that you've assumed a problem on my end, only to have it be a problem on your end. It makes it very difficult for me to take things you say seriously--but I still do.
-
Yeah, I always thought something was wrong and keeps being wrong in your configuration, Netham...
-
Could you please at least say what/who you are?
-
Barrett is the guy who hosts the VPS Omni is hosted on, for anyone who doesn't know.
-
What is a VPS server? (sorry for being n00bish)
-
A VPS (Virtual Private Server) is basically a virtual machine. Someone can rent a physical server and create tons of virtual machines on it and sell each one of them as servers.
-
So that pretty much could eat ram and stuff if i get it right x.x
Sounds like a good way to make money.
-
VPS's can be very nice if well setup.
-
Indeed. Even Windows VPSes can be nice if well configured, but I think Windows is easier to misconfigure. (Not that I call Netham a n00b, or that I want to cause a OS war, but I think there's lot of what I (or any good sysadmin) consider bad practices he need to fix.)
-
7.) Ever heard the story of the boy who cried wolf? I can't count the number of times that you've assumed a problem on my end, only to have it be a problem on your end. It makes it very difficult for me to take things you say seriously--but I still do.
I've complained about one issue to you that was actually on my end. I only mistook it for an issue on your end because I've had issues on your end (the heavy Disk I/O lagging me) give the same result multiple times. That one time I contacted you wasn't the only time that it happened. It did for the most part stop afterwards, yes, but that wasn't the only time by a long shot.
As for the network outages, there were 45 minute or so periods DAILY around rougly 3-4 AM where I had no connectivity for months on end that I complained about numerous times.
As for complaining about the DDoS having issues on the network, I just looked and I had two e-mails sent to you, and I know I brought it up when I called you. Also, I'm not saying the hosts network is crashing entirely, I'm saying that the virtual NIC inside your hypervisor was crashing, rendering my VPS useless.
@Juju I would argue that Windows is harder to misconfigure, as it automatically configures stuff like networking and such for you. As far as bad practices, the only thing I can think of that was sub-optimal was having a bit too much running on there because I couldn't afford another server.
-
Oh ok. Hope stuff gets more stable on your side as on the hypervisor's side then.
-
Barrett is the guy who hosts the VPS Omni is hosted on, for anyone who doesn't know.
And Calcgames (calcg.org) founder.
That said I have an history of sending Netham45 PMs and never receiving a response back, so it's very possible that you replied to his e-mail(s) and he didn't notice. Or maybe the opposite happened, since Netham45 have just looked and he actually sent as many as he said.
@Netham45 about Windows the problem is that Micro$$$oft has a long history of producing buggy and bloated OSes that get more and more unstable as you fill them with random softwares and uninstall stuff. It was particularly bad around the Windows XP era, so while Windows may be easier to configure and use for most people, its unreliability at times could possibly explain why you have so much problems with it. When Omni was moved on Barrett's VPS and started having problems such as forcing you to change timestamp settings and other things on SMF, I thought the VPS was just misconfigured for SMF, but when I discovered that it ran IIS a few months ago, then I thought the issues were caused by IIS, directly or indirectly.
-
I can't troubleshoot a problem when I don't get responses to questions, and when you don't follow good-practice suggestions (such as to not have your only DNS server on your VPS). It's not particularly important to me that you all know the details of what's gone on. That being said, I won't put up with someone exaggerating facts (and making them up) when I have gone out of my way time and time again to help this person (especially with huge discounts to be nice and for good PR (oops; that backfired!)), and even losing more money from his usage than he's paying me. In a month or two (preferably less; possibly by Monday at 12:01 am if he chooses not to renew), he will no longer be hosted with us. I don't mind criticism. I do mind slander/libel.
-
That said we should probably remain civil in this thread, since it is located on this forum, after all. Plus, it's your speech (and Andy's) versus Netham45, and given the history between both parties and Netham45, and to a lesser extent, Omnimaga (most notably the #omnimaga takeover by Calcgames botnet), the older users of us tend to take anything that comes from the #tcpa/#calcgames crowd (such as your messages above, along with Andy's) with a grain of salt.
-
I have never been a part of any "channel wars" and do not have prejudice against omnimaga members in any way. I don't even know what reasons there would be for that to be the case. This isn't a debate, and I don't care who you believe. I've said my part (even allowing some of netham's exaggerations to go unchallenged).
-
I haven't gotten any impressions that anyones affiliations with any groups have had any impact what-so-ever with anything.
-
I didn't before, until I learned that Andy was involved. That impression I had was mainly due to the history of bans on Netham45 in #tcpa/#calcgames and some not-so-positive remarks towards Omnimaga and to a lesser extent TI-BASIC programs (which was pretty much all Omnimaga was about until 5 years ago) and most people involved were CG staff or even admin. Of course I did not realize the botnet was not owned by Barrett, though, and that hosting payments were always late (in AspirationHosting case, the site would have closed immediately if I paid one single hour too late, and that until I send the payment :P).
-
The late payments were not an issue in my mind. Yes, they were late, but I didn't care until now.