Omnimaga

Omnimaga => News => Topic started by: Geekboy1011 on March 08, 2014, 04:07:19 pm

Title: The Recent Forum/IRC down Time
Post by: Geekboy1011 on March 08, 2014, 04:07:19 pm
Over the last few days there have been a series of downtime on both the irc network and the forum. We apologize for this. Our server manged to have a logging issue where we filled up our disk space and caused MySQL and some other processes to crash. Nickserv information on IRC was lost and thankfully nothing on the forum was lost to these incidents.

What we have done to fix it. We have started implementing local backups of everything to allow us to quickly fix the issues should they arise again. As well as taken a look at the issues with the logs and made the proper config changes to them (Why are rolling logs not the default anyone please?)

 Thank you,
     The Omnimaga Administration Team
Title: Re: The Recent Forum/IRC down Time
Post by: Juju on March 08, 2014, 04:14:56 pm
Glad it works again!
Title: Re: The Recent Forum/IRC down Time
Post by: Sorunome on March 08, 2014, 04:26:53 pm
Good to hear it is working again :)
Title: Re: The Recent Forum/IRC down Time
Post by: DJ Omnimaga on March 08, 2014, 05:03:41 pm
At least I'm glad it's not just the server technical specs being too low to handle OIRC2, SMF 2.0 and Omninet simultaneously. That would have sucked since this would have meant having to host Omnom on Soru or Albert's servers. :P
Title: Re: The Recent Forum/IRC down Time
Post by: Eeems on March 08, 2014, 05:05:19 pm
At least I'm glad it's not just the server technical specs being too low to handle OIRC2, SMF 2.0 and Omninet simultaneously. That would have sucked since this would have meant having to host Omnom on Soru or Albert's servers. :P
Our server specs are fine. More then adequate for everything. It's kinda difficult when a log file grows to 13GB+ in size though.
Title: Re: The Recent Forum/IRC down Time
Post by: Vogtinator on March 08, 2014, 05:56:55 pm
WOW, 13GB is massive. Which log file was it?
On my servers systemd handles logging just fine. You can even set a nice size limit for the syslog.
And for other services which log into own files there's logrotate.
Title: Re: The Recent Forum/IRC down Time
Post by: Eeems on March 08, 2014, 05:58:15 pm
WOW, 13GB is massive. Which log file was it?
On my servers systemd handles logging just fine. You can even set a nice size limit for the syslog.
And for other services which log into own files there's logrotate.
Mysql binary logs.
I've disabled it and we will have to setup something else for backups now then replication.
Title: Re: The Recent Forum/IRC down Time
Post by: Geekboy1011 on March 08, 2014, 05:59:29 pm
Why not just disable it for issue tables? Or is it not that simple?
Title: Re: The Recent Forum/IRC down Time
Post by: Eeems on March 08, 2014, 06:27:10 pm
Why not just disable it for issue tables? Or is it not that simple?
Then we wouldn't be able to backup the important parts (messages)
Title: Re: The Recent Forum/IRC down Time
Post by: Geekboy1011 on March 08, 2014, 06:33:48 pm
It's the forum dB that's having issues?
Title: Re: The Recent Forum/IRC down Time
Post by: Eeems on March 08, 2014, 07:10:47 pm
It's the forum dB that's having issues?
No binary logging was used to backup, and if I was to just disable it for the ones that would cause the filesystem to fill up then we wouldn't be able to properly back up.