Omnimaga
Omnimaga => News => Topic started by: Geekboy1011 on September 20, 2013, 02:25:04 pm
-
Today me and Eeems have started a long been wanted process of moving Omnimaga to a new host once again. We have been on amazon web services for a bit over a year now and its just to expensive for us to keep us running. We have started the move over to Ubiquity which has served eeems for over a year hosting eeems.ca. This move will take place over the next 48hours or so, During this time we please ask you to keep in mind the website might seem spotty but it will be back shortly at any of these occasions.
I will keep this post updated with the status of everything:
As of now the post database has been migrated and slowly everything else will be. I will also post the new ip for people with slow dns's once we make that change.
Thank you
~The Management
-
Nice, i hope the new server will be quicker :)
I guess we'll still use cloudflare? :)
Also, i hope the migration will work smoothley :)
EDIT: just noticed all those smileys :)
-
Thanks for the heads-up!
Will we still be able to post/upload attachments during said period of time?
-
Yes we will still use cloudflare. It saves a lot of hdd i/o and other things. And eeems is handling it we are moving to a server with more ram and disk space. We just have less cpu speed. 2.0ghz x2 cores
@pimath as long as the website loads yes you should be able to. attachments I would be cautious of but yes you should be able to
-
what CPU do we currently have in comparison?
-
a single core at 2.0ghz so faster Clock more ram and a faster hdd for less :D and we have a bandwidth cap at like 1tb instead of a metered bandwidth where we pay for what we use
-
2TB bandwidth cap. Not that we even touch 1TB. We will still use cloudflare for security reasons.
The forum will be disabled when I perform the site migration which could take a while since I'll have to copy the files between the servers. The new server shouldn't ever have slowdown due to how powerful it is compared to the current one.
-
Sounds good guys, thanks for the heads up! (｀_´)ゞ
-
Is this why there was a DB downtime last night? (The site got a bit messed up afterward, but posting/viewing topics still worked fine, as well as the OmnomIRC block)
Also nice to hear that you found something more affordable. Do you think the new server will be able to run the default Omnimaga search engine or will you still have to stick to DuckDuckGo?
Another question: How much did the previous host cost approximately every month? I know that Omni server sometimes rivals Ticalc.org in amount of traffic and the site is particularly resource intensive with 360000 posts and lots of attachments, but I was curious.
what CPU do we currently have in comparison?
CPU: 2 MHz Z80
RAM: 0.0022888 MB
Diskspace: 32 KB ROM plus a mod for SD card support (up to 16 GB)
OS: TI-81 ROM version 1.1K
-
its was about 60-80 a month on amazon. we do not know if we can have the proper search engine yet tho we can try it in the future for now lets get it running happily.
and yes the down time was cause of the hdd getting full which is another part of the reason we are moving
-
its was about 60-80 a month on amazon. we do not know if we can have the proper search engine yet tho we can try it in the future for now lets get it running happily.
and yes the down time was cause of the hdd getting full which is another part of the reason we are moving
O.O Darn that is insane! That's like twice what we used to pay I think on Barrett's server, yet we didn,t even have close to Barrett's server performance (since we constantly had to cut down features). How much is the new server?
-
about 25$ a month
-
We may be able to handle the old search method, SMF hasn't been migrated yet so I can't do stress testing.
-
about 25$ a month
That's pretty cool. I tend to not trust small VPS providers that charges like $12 a month since they usually disappear after a year or so or have crappy uptime, but I find the $40 a month that some charges for very basic plans to be too expensive. I never tried Ubiquity, but they have been around for years so I doubt they'll go away anytime soon. The only downside is that way back in 2008-09, people used their servers to host spambots. Hopefully that was fixed, so that this doesn't cause any uptimes issues.
-
We have migrated the actual site to the new server, let us know if you notice anything acting strange.
-
I got redirected to some weird URL (http://www.omnimaga.org/23.82.187.93) for a while but I managed to fix it now. Thanks for all the hard work on the website :)
As for loading time it seems fine for now (posting via quick reply seems faster), but the true test will be from Monday to Friday from 4 PM to 6 PM GMT-5 and when the search engine is enabled again :P
EDIT Attachments work too, so that's good.
-
yay, new server, nive work guys! :D
-
Wow! So much cheaper and so much better :)
The only potential issue I see is what happens when you hit the 2TB cap (if ever XD)
Some people attacking the server may force it to hit the 2TB cap D:
-
hm, http://status.omnimaga.org/api.php?req=counts×tamp=1379763623984 (when going on status.omnimaga.org) gives internel server error
-
hm, http://status.omnimaga.org/api.php?req=counts×tamp=1379763623984 (when going on status.omnimaga.org) gives internel server error
it's possible that this wasn't setup yet. It might have moved, but I don't think status.omnimaga.org, axe.omnimaga.org, asmtools.omnimaga.org and the rest were updated yet.
-
hm, http://status.omnimaga.org/api.php?req=counts×tamp=1379763623984 (when going on status.omnimaga.org) gives internel server error
it's possible that this wasn't setup yet. It might have moved, but I don't think status.omnimaga.org, axe.omnimaga.org, asmtools.omnimaga.org and the rest were updated yet.
That is why i poked it here :P
-
I got redirected to some weird URL (http://www.omnimaga.org/23.82.187.93) for a while but I managed to fix it now. Thanks for all the hard work on the website :)
As for loading time it seems fine for now (posting via quick reply seems faster), but the true test will be from Monday to Friday from 4 PM to 6 PM GMT-5 and when the search engine is enabled again :P
EDIT Attachments work too, so that's good.
Woops, I'll go fix that. That was me trying to get the old server to force you to use the new one. I'll do some testing with the old search engine to see if it will work fine on the new server.
Wow! So much cheaper and so much better :)
The only potential issue I see is what happens when you hit the 2TB cap (if ever XD)
Some people attacking the server may force it to hit the 2TB cap D:
We use cloudflare to mitigate attacks. It would be really hard for them to do that. Plus I'm sure if I talked to customer support and they realized it was attack we wouldn't be charged for it. Also, the 2TB cap is way better then paying per MB. We still would have had the same issue on AWS. With the new server we only pay per MB after we hit the 2TB limit per month. The old server averaged around 60GB/mo.
hm, http://status.omnimaga.org/api.php?req=counts×tamp=1379763623984 (when going on status.omnimaga.org) gives internel server error
it's possible that this wasn't setup yet. It might have moved, but I don't think status.omnimaga.org, axe.omnimaga.org, asmtools.omnimaga.org and the rest were updated yet.
They all were.
EDIT: I think I fixed the status subdomain. No idea how I did though x.x
-
By the way, could you post the monthly requests since 2012? I am curious about how much traffic the site gets nowadays. I know in early 2010 traffic was about 10 times lower than now, but with all the info and help topics on the forums we tend to get many Nspire users seeking for help (not necessarily registering, but I notice in the online list sometimes).
-
http://status.ourl.ca/statsgraph.php is still current.
I can't quite get the full year for 2012 on cloudflare, but here are the analytics.
EDIT: I have no idea what happened to the missing chunk.
EDIT2: Oh and btw, I did a bunch of optimization that should help make the site feel faster.
-
Baidu still crawls the site? o.o
-
It tries to.
Btw, old search is back since our server seems to be able to handle it.
-
I just love how so many threats originate from china XD
And yay for search! :D
-
We get more bots, crawlers and threats than users? lol I didn't know the interwebs were this dangerous ^^
-
Wait did we only get 909060 hits in the last year or is that a monthly average? O.O That doesn't seem right (I know activity dropped in the last two years, but there's no way traffic could have dropped from 6 million non-bot hits in August-Sept 2012 to 909060 for a 6 times longer period (excluding of course the missing stats) ???)
EDIT Nvm, I got confused with file requests. We actually got 8 million requests from real users and 18 millions including bots, excluding of course the missing chunk.
Here are the traffic stats from Sept 2008 to May 2011, by the way (although they exclude OmnomIRC files):
Total Requests
Sep08 54414
Oct08 55652
Nov08 85580
Dec08 79483
Jan09 86728
Feb09 75421
Mar09 85843
Apr09 91038
May09 117290
Jun09 147615
Jul09 145551
Aug09 144507
Sep09 175618
Oct09 169147
Nov09 182796
Dec09 186541
Jan10 268161
Feb10 243358
Mar10 379828
Apr10 503271
May10 541576
Jun10 742455
Jul10 802643
Aug10 882497
Sep10 1282296
Oct10 1866178
Nov10 3505173
Dec10 3270968
Jan11 3609766
Feb11 4944255
Mar11 5003401
Apr11 4833258
May11 4715885
From memory, August or September 2012 was around 9 millions total, with 6 from real users. I don't know if that included Omnom, though. The Feb and Mar 2011 spikes were because of nDoom. I'm betting that December 2012 had insane stats as well.
-
By the way, has a solution been found to the broken posts/topic counts on board index and front page, other than incrementing all post IDs to get rid of negatives? Also, I have found a solution to avoid breaking all old links that would result from incrementing all topic/post IDs to get rid of negative ones: Basically, increment every single topic/post by 100000 and 1000000 or so, then install some SMF mod where if someone tries to access an URL where the topic/posts ID is too low, then it automatically redirects to an URL where they are incremented. For example, http://ourl.ca/19632/361636 would redirect to http://ourl.ca/119632/1361636 .
This would make many people on other sites happy, along with whoever worked hard on tutorials and project lists, and this would reduce the risks of Omnimaga search ranking and traffic being hurt considerably. In addition to that it would still let you upgrade fine in the future.
-
I've been at work all day, I haven't had time to look at it yet.
-
Ok, hopefully fixing it won't be too rough. Although this isn't impacting user experience much, it's possible that certain people might think that we're lying about our total post count or something to get on BigBoards or something lol (like I saw on Invisionfree before).
It's annoying how moving a site from a place to another can cause so much problems. I remember having problems switching TIMGUL from 1and1 to Aspiration too >.< (lots of broken theme CSS and stuff, along with missing attachments). The move to Ubiquity gotta be by far the least rocky one in the last few years, though, considering it almost went unnoticed if we didn't read the news.
I also don't notice slowdowns so far.
-
It's probably the least rocky because it's being handled by a professional ;)
-
Lol yeah. I personally just used shared hosting because I didn't want to risk myself into VPS management since I had no clue what I am doing, plus I did not know tricks to keep a site online while it moves and stuff. I was amazed at how during this move I could still post only seconds before it happens and that the site was already available for use right afterward. In my case it was usually offline for two days for certain people in Australia and such locations.
It's also good that it's being handed by a professional that has at least some free time to do it :P (Netham45 knew some stuff, but he often was away for a week or two, unable to fix issues that could occur right after the move)
-
You sure are a professional Eeems ^^
EDIT: I also noticed that, even though the switch, when doing quick reply with alt+s it sometimes says pretty long 'loading' up there, sometimes it even says that longer than the new-post-bot notices the new post.
-
You sure are a professional Eeems ^^
EDIT: I also noticed that, even though the switch, when doing quick reply with alt+s it sometimes says pretty long 'loading' up there, sometimes it even says that longer than the new-post-bot notices the new post.
Strange, it's snappier for me then it was before.
-
Often it is quicker, but sometimes it is very slow.....but most of the time it is quicker, i can live with it ;)
-
You sure are a professional Eeems ^^
EDIT: I also noticed that, even though the switch, when doing quick reply with alt+s it sometimes says pretty long 'loading' up there, sometimes it even says that longer than the new-post-bot notices the new post.
For me it was the opposite actually. Sometimes on the old server it said Loading, then I saw it appear in OmnomIRC. I had to refresh or wait longer to see my post. Now it's much more rare.
-
DJ_O suggested me to copy/paste my answer to the broken links issue here :
----
Instead of using http://ourl.ca/172700 just use http://www.omnimaga.org/index.php?tttopic=14673.30 and when you encounter http://ourl.ca/172700 make a redirection to http://www.omnimaga.org/index.php?tttopic=14673+n.30 where n is the offset.
Other than that, as Lionel said, you will nuke your google visitors, same for inner topic links, same for crossposting links.
So my idea would work I guess.
-
Thanks for cross-posting. I brought up the issue in the other topic first so the author is aware that he's probably gonna have to edit his Nspire project list (since he asked if people were still using it) due to the ID changes.
I think Eeems idea is to change links from
http://www.omnimaga.org/index.php?topic=-2597.msg-54124#msg-54124
http://ourl.ca/19650/361831
To
http://ourl.ca/2598/54125
http://ourl.ca/22248/415956
Which was the problem, due to what Levak mentionned. My idea would have changed topic IDs to this:
http://ourl.ca/100000/1000000
http://ourl.ca/119650/1361831
In which case it wouldn't be too hard to get any topic ID under 90K to be re-mapped to its 100K counterpart and so on. It's fine if the 2005-08 topics won't redirect, but just as long as the more recent ones does. Else, there is Levak's idea.
-
Thanks for cross-posting. I brought up the issue in the other topic first so the author is aware that he's probably gonna have to edit his Nspire project list (since he asked if people were still using it) due to the ID changes.
I think Eeems idea is to change links from
http://www.omnimaga.org/index.php?topic=-2597.msg-54124#msg-54124
http://ourl.ca/19650/361831
To
http://ourl.ca/2598/54125
http://ourl.ca/22248/415956
Which was the problem, due to what Levak mentionned. My idea would have changed topic IDs to this:
http://ourl.ca/100000/1000000
http://ourl.ca/119650/1361831
In which case it wouldn't be too hard to get any topic ID under 90K to be re-mapped to its 100K counterpart and so on. It's fine if the 2005-08 topics won't redirect, but just as long as the more recent ones does. Else, there is Levak's idea.
I could probably just bump them to the latest topic id and up. This could break links to the old ones, but whatever.
-
Actually, just bumping negative topics up is another solution, but there would be some problems with SMF sorting: When checking topic lists, those topics would appear at the top (I tried on a test board before). Not that this is much of an issue for the New Posts page, since the old threads would disappear within weeks, but for regular topic listing, this could cause confusion about what happened to the newer topics and increased necroposting.
Basically, you would need to write a SMF mod that allows you to sort by date rather than ID. I'm unsure why SMF does that, because other softwares sorts topics and posts in the right date order, regardless of the ID.
-
Speaking of the old posts, will the few posts of 2004 ever be merged?
If it would have to be done manually i would take my time to do it, only because it would be cool IMO to have all of 'em :)
-
Actually that would be cool. They could be added right before the negative IDs. You could always do it manually then ask a staff to edit the IDs, dates, nicknames, etc, but like the Pre-Invisionfree news sub-forum (http://www.omnimaga.org/index.php?board=203.0), it's not guaranteed that you will ever be answered, so the 56 posts might end up remaining as Sorunome posts forever (also the fact that the old HTML news were converted to the current forums would make assigning IDs to the old posts very hard). >.<
EDIT: Regarding the inflated postcounts, something I noticed is that topics are now listed multiple times. For example, check those links:
http://www.omnimaga.org/index.php?board=4.2700
http://www.omnimaga.org/index.php?board=4.0
And there are plenty of empty pages too.
-
(for those who are wondering which 56 posts we are talking about, it are these: http://web.archive.org/web/20040627225157/http://www.yaronet.com/forum.php?s=462 )
(note to remove the webarchive part after clicking on any board)
-
Yeah all sub-forums are still intact, just not the board index.
Also, only two members from back then even have an accout here: Me and Iambian (he was Iambian Zenith back then)