Omnimaga

Omnimaga => Discontinued => Our Projects => OmnomIRC Development => Topic started by: Sorunome on March 31, 2013, 09:27:29 pm

Title: New Post Bot
Post by: Sorunome on March 31, 2013, 09:27:29 pm
So, in this topic goes everything related with the new post bot, as OmnomIRC and the new post bot are totally seperate (sorry, it just started bugging me that there were so many new-post-bot related posts in the OmnomIRC thread)
sooo.here we go!
Title: Re: New Post Bot
Post by: DJ Omnimaga on May 08, 2013, 02:37:34 pm
Can we report when the bot isn't reporting posts here? (Such as right now it doesn't work, or almost every other day lately)
Title: Re: New Post Bot
Post by: Sorunome on May 08, 2013, 06:58:11 pm
Yes, you can.
Title: Re: New Post Bot
Post by: DJ Omnimaga on June 10, 2013, 10:36:54 pm
By the way, is the post bot gonna remain even after you move to Germany? It's down right now D:
Title: Re: New Post Bot
Post by: Sorunome on June 14, 2013, 03:02:35 am
I noticed that once I landed, idk why but the thread that is checking new posts crached while the actual bot stayed intact......i just restarted the bot and it worked fine :)
Title: Re: New Post Bot
Post by: DJ Omnimaga on June 14, 2013, 08:02:52 pm
Yeah sometimes the bot is there but it doesn't work. It's kinda like the old one by Netham I guess. :P
Title: Re: New Post Bot
Post by: Darl181 on June 14, 2013, 10:12:13 pm
Speaking of the old bot, what's the formal name of the new one? This board still has "SpyBot45" in its name :P
Title: Re: New Post Bot
Post by: DJ Omnimaga on June 15, 2013, 12:15:24 am
I don't think there is an official name anymore. Ever since it was replaced with Alberth's then Sorunome's, the bot now uses random funny names most of the time, such as Omnom, Satanic, Peaceful, Awesome, etc., because of the way post notifications are posted (using the action command).
Title: Re: New Post Bot
Post by: Sorunome on June 15, 2013, 09:46:39 am
mine /is/ alberths, i only enhanced it and fixed some bugs, but usually its nick is omnom, so i always call it omnom
Title: Re: New Post Bot
Post by: DJ Omnimaga on June 16, 2013, 08:13:25 pm
Ah I see, I thought you had started your own. :P
Title: Re: New Post Bot
Post by: Sorunome on June 17, 2013, 04:17:45 am
Ha, nope, all I manly did was fixing the unicode bug, and then I also added so that it is at the same time a channel-management-bot (auto-op), lol
Title: Re: New Post Bot
Post by: DJ Omnimaga on June 18, 2013, 01:25:19 am
And the bot is down again. It might be due to how slow Omni ran an hour ago or so.
Title: Re: New Post Bot
Post by: Sorunome on June 18, 2013, 03:38:25 am
And the bot is down again. It might be due to how slow Omni ran an hour ago or so.
it is only omnis fault if it posts in the irc channel once every 3 seconds "I died, hope that sorunome sees the bug and fixes it" or something like that, am restarting it, and this thing is a PITA
EDIT: I can't even look into the log file what the mistake is because I had to disable it as the log file grew too big and the bot crached due to no hdd space >.<
Title: Re: New Post Bot
Post by: DJ Omnimaga on June 18, 2013, 03:53:52 am
Nah it didn't say it died IIRC, it just stopped notifying of new posts. As for when it happened, the site had like 9000-41000ms PHP load time.
Title: Re: New Post Bot
Post by: Streetwalrus on June 18, 2013, 03:55:10 am
Wow yeah there are huge lag spikes on Omni recently.
Title: Re: New Post Bot
Post by: Sorunome on June 18, 2013, 03:57:04 am
Did the admins already put the script online to auto-clear the cache every now and then? They said that it'll prolly speed omni up >.>
Title: Re: New Post Bot
Post by: DJ Omnimaga on June 18, 2013, 02:42:54 pm
I don't think so. You might want to email their e-mail address to make sure.
Title: Re: New Post Bot
Post by: Eeems on June 21, 2013, 09:01:08 pm
And the bot is down again. It might be due to how slow Omni ran an hour ago or so.
it is only omnis fault if it posts in the irc channel once every 3 seconds "I died, hope that sorunome sees the bug and fixes it" or something like that, am restarting it, and this thing is a PITA
EDIT: I can't even look into the log file what the mistake is because I had to disable it as the log file grew too big and the bot crached due to no hdd space >.<
Add log rotating/trimming :)
Did the admins already put the script online to auto-clear the cache every now and then? They said that it'll prolly speed omni up >.>
That was done a while ago. There are speed increases due to it. Any slowdown is either google indexing the site or something else like that.
Title: Re: New Post Bot
Post by: DJ Omnimaga on June 21, 2013, 09:19:56 pm
Last night apparently it was a Chinese search engine that did it. But Geekboy seemed unable to ban it (I think the IP range was just too big)
Title: Re: New Post Bot
Post by: Eeems on June 21, 2013, 09:21:59 pm
Last night apparently it was a Chinest search engine that did it. But Geekboy seemed unable to ban it (I think the IP range was just too big)
Again? We've banned it's ip's multiple times before :/
Title: Re: New Post Bot
Post by: DJ Omnimaga on July 27, 2013, 02:43:09 am
It's down again since the past few hours D:
Last night apparently it was a Chinest search engine that did it. But Geekboy seemed unable to ban it (I think the IP range was just too big)
Again? We've banned it's ip's multiple times before :/
It probably changed IPs or something
Title: Re: New Post Bot
Post by: Sorunome on July 27, 2013, 05:51:18 am
It's down again since the past few hours D:
That's why I made this topic: http://ourl.ca/19286
Title: Re: Re: Re: New Post Bot
Post by: DJ Omnimaga on July 27, 2013, 11:59:50 am
It's down again since the past few hours D:
That's why I made this topic: http://ourl.ca/19286
oh I missed the post bot part of that topic and thought you were talking about OmnomIRC yesterday. Probably due to being on my phone. My bad x.x
Title: Re: New Post Bot
Post by: Sorunome on July 28, 2013, 11:32:44 am
So yeah, the bot is back up :D
Title: Re: Re: New Post Bot
Post by: DJ Omnimaga on July 28, 2013, 01:58:32 pm
Good to hear :). Was the problem really what you thought or was another cause found?
Title: Re: New Post Bot
Post by: Sorunome on July 28, 2013, 02:21:08 pm
(posting for if anypony else finds this and is desperatley searching for the solution)
check here: http://ourl.ca/19286/355767
Title: Re: New Post Bot
Post by: Streetwalrus on July 30, 2013, 01:11:31 pm
So I suggested this :
Quote from: #omnimaga
19:07 < Streetwal> also isn't it possible to make omnom use some kind of push system ?
19:07 <@Sorunome> but that would require programming :P
19:07 < Streetwal> like an smf mod that sends it a notification when somepony posts
19:08 < Streetwal> and then it sends a message to irc
19:08 <@Sorunome> but that would require programming :P
19:09 < Streetwal> that'd reduce the load on omni's server a bit and also fix the skip when two ponies posts in a 5secs interval
Title: Re: Re: New Post Bot
Post by: DJ Omnimaga on August 02, 2013, 12:23:46 pm
It used to work similarly, so when someone posted, the board sent a notification to the bot. However, it only worked if OmnomIRC was hosted on the same server as Omnimaga, which is no longer possible without upgrading to 32 cores and 512 GB of RAM.
Title: Re: New Post Bot
Post by: Sorunome on August 02, 2013, 12:34:54 pm
Wait, that means it wasn't an IRC bot back then? Because running an IRC bot on the server is still possible :) But i heard that that checking server-side somehow nommed tons of CPU ???
Title: Re: New Post Bot
Post by: Streetwalrus on August 02, 2013, 01:45:15 pm
Wait, that means it wasn't an IRC bot back then? Because running an IRC bot on the server is still possible :) But i heard that that checking server-side somehow nommed tons of CPU ???
My suggestion was to send a notification from within whatever PHP script adds a post to the DB. That way it's CPU friendly. ;)
Title: Re: New Post Bot
Post by: Sorunome on August 02, 2013, 01:50:11 pm
well, they are send anyways to the db because they are stored (duh).
But yeah, i /could/ write a bot that hooks up with the db but idk how much cpu it would nom
Title: Re: New Post Bot
Post by: DJ Omnimaga on August 02, 2013, 02:24:38 pm
Wait, that means it wasn't an IRC bot back then? Because running an IRC bot on the server is still possible :) But i heard that that checking server-side somehow nommed tons of CPU ???
It was an IRC bot, but it was hosted on the same server as Omni. The problem is that the SMF mod that sent the new post notification to the IRC bot required that both the bot and OmnomIRC are hosted with the forums.

I wonder if it would be possible to send such data over two servers?

Oh and IIRC, another issue with such DB access is that if IRC was down or something else prevented the notification from being sent, it didn't let you post, so if that was revived, it would need a timeout feature implemented (so if after 10 seconds, it didn't respond, then it posts anyway without the New post warning.
Title: Re: New Post Bot
Post by: Eeems on August 02, 2013, 02:28:07 pm
well, they are send anyways to the db because they are stored (duh).
But yeah, i /could/ write a bot that hooks up with the db but idk how much cpu it would nom
It would be so much easier if you just had a node bot acting like a server that sent the post notifications to IRC whenever a call was made by Omnimaga to it. The posting function that stores the data to the database would also make a quick call to the post bot, thus causing push notifications :)
Title: Re: New Post Bot
Post by: DJ Omnimaga on August 02, 2013, 02:33:37 pm
Just one thing, though: If the direct DB access bot is ever revived, make sure to not forget to disable notifications for the staff and private boards (except game walkthroughs), because I remember at first when some private thread appeared in OmnomIRC. It was a little akward >.<. Imagine, for example, if the entire Omni admin team got under NDA with a company or had to keep something secret there, then accidentally leaked the name of a top secret project that way O.O . Plus, I would find it kinda akward as well to see an unreachable topic notification saying "New post by Netham45 in Here's what DJ told juju IRL http://ourl.ca/33934" while chatting O.O
Title: Re: New Post Bot
Post by: TIfanx1999 on August 02, 2013, 02:37:49 pm
Haha, good point. :)
Title: Re: New Post Bot
Post by: Sorunome on August 02, 2013, 05:37:48 pm
why won't we actually somehow embed the new post notification in OIRC?
Title: Re: New Post Bot
Post by: DJ Omnimaga on August 02, 2013, 05:47:49 pm
I wouldn't mind, as long as they also appears on EFnet and Omninet too.

Btw it would kinda make sense to make French and German posts show up in their respective channels, like they used to do. They could still appear in the main channel  as well, though, in case an English person can help on programming help.
Title: Re: New Post Bot
Post by: Sorunome on August 02, 2013, 05:56:48 pm
it would be that on efnet and omninet they appear over the irc user OmnomIRC.
Also, the other chans is actually a good idea.... :)
Title: Re: New Post Bot
Post by: Streetwalrus on August 03, 2013, 02:24:16 am
well, they are send anyways to the db because they are stored (duh).
But yeah, i /could/ write a bot that hooks up with the db but idk how much cpu it would nom
It would be so much easier if you just had a node bot acting like a server that sent the post notifications to IRC whenever a call was made by Omnimaga to it. The posting function that stores the data to the database would also make a quick call to the post bot, thus causing push notifications :)
That's what I keep saying. :P
Title: Re: New Post Bot
Post by: Eeems on August 03, 2013, 05:30:19 am
well, they are send anyways to the db because they are stored (duh).
But yeah, i /could/ write a bot that hooks up with the db but idk how much cpu it would nom
It would be so much easier if you just had a node bot acting like a server that sent the post notifications to IRC whenever a call was made by Omnimaga to it. The posting function that stores the data to the database would also make a quick call to the post bot, thus causing push notifications :)
That's what I keep saying. :P
Thats why I'm reiterating it :P
Title: Re: New Post Bot
Post by: Sorunome on August 03, 2013, 06:00:58 am
well, they are send anyways to the db because they are stored (duh).
But yeah, i /could/ write a bot that hooks up with the db but idk how much cpu it would nom
It would be so much easier if you just had a node bot acting like a server that sent the post notifications to IRC whenever a call was made by Omnimaga to it. The posting function that stores the data to the database would also make a quick call to the post bot, thus causing push notifications :)
That's what I keep saying. :P
Thats why I'm reiterating it :P
That's why i didn't understand it :P
JK, lol, well, but with a real push system, wouldn't that also require adding some extra stuff to the posting.php (or however it is called)
Title: Re: New Post Bot
Post by: Eeems on August 03, 2013, 05:06:24 pm
well, they are send anyways to the db because they are stored (duh).
But yeah, i /could/ write a bot that hooks up with the db but idk how much cpu it would nom
It would be so much easier if you just had a node bot acting like a server that sent the post notifications to IRC whenever a call was made by Omnimaga to it. The posting function that stores the data to the database would also make a quick call to the post bot, thus causing push notifications :)
That's what I keep saying. :P
Thats why I'm reiterating it :P
That's why i didn't understand it :P
JK, lol, well, but with a real push system, wouldn't that also require adding some extra stuff to the posting.php (or however it is called)
Yes, that's what I've been saying! :P
Title: Re: New Post Bot
Post by: Sorunome on August 03, 2013, 05:07:13 pm
well, they are send anyways to the db because they are stored (duh).
But yeah, i /could/ write a bot that hooks up with the db but idk how much cpu it would nom
It would be so much easier if you just had a node bot acting like a server that sent the post notifications to IRC whenever a call was made by Omnimaga to it. The posting function that stores the data to the database would also make a quick call to the post bot, thus causing push notifications :)
That's what I keep saying. :P
Thats why I'm reiterating it :P
That's why i didn't understand it :P
JK, lol, well, but with a real push system, wouldn't that also require adding some extra stuff to the posting.php (or however it is called)
Yes, that's what I've been saying! :P
So, um, do i have admin permission to try around in that area?
Title: Re: New Post Bot
Post by: Eeems on August 03, 2013, 05:17:59 pm
well, they are send anyways to the db because they are stored (duh).
But yeah, i /could/ write a bot that hooks up with the db but idk how much cpu it would nom
It would be so much easier if you just had a node bot acting like a server that sent the post notifications to IRC whenever a call was made by Omnimaga to it. The posting function that stores the data to the database would also make a quick call to the post bot, thus causing push notifications :)
That's what I keep saying. :P
Thats why I'm reiterating it :P
That's why i didn't understand it :P
JK, lol, well, but with a real push system, wouldn't that also require adding some extra stuff to the posting.php (or however it is called)
Yes, that's what I've been saying! :P
So, um, do i have admin permission to try around in that area?
Yes, just do the initial coding on your machine first (so install smf locally and then test it out) before you muck around on Omni itself.
Title: Re: New Post Bot
Post by: Sorunome on August 03, 2013, 05:24:16 pm
well, they are send anyways to the db because they are stored (duh).
But yeah, i /could/ write a bot that hooks up with the db but idk how much cpu it would nom
It would be so much easier if you just had a node bot acting like a server that sent the post notifications to IRC whenever a call was made by Omnimaga to it. The posting function that stores the data to the database would also make a quick call to the post bot, thus causing push notifications :)
That's what I keep saying. :P
Thats why I'm reiterating it :P
That's why i didn't understand it :P
JK, lol, well, but with a real push system, wouldn't that also require adding some extra stuff to the posting.php (or however it is called)
Yes, that's what I've been saying! :P
So, um, do i have admin permission to try around in that area?
Yes, just do the initial coding on your machine first (so install smf locally and then test it out) before you muck around on Omni itself.
Ok. So just that i get it right......when i post on the serverside i do an internal call to the post bot which listens to an internal port on localhost....right?
Title: Re: New Post Bot
Post by: Streetwalrus on August 04, 2013, 12:53:14 am
I guess you're right. ;) Also for it to be true push the bot should not actively fetch the post info on the forum's call, that should be included in the message.
Title: Re: New Post Bot
Post by: Sorunome on September 30, 2013, 10:11:08 am
So, the new bot with push notification is up! :D
Title: Re: New Post Bot
Post by: DJ Omnimaga on September 30, 2013, 12:27:31 pm
Glad to hear. It's tied to the database like the original OmnomIRC v2 bot, right? (it had to be replaced by someone's bot because OmnomIRC had to be moved to a different server)
Title: Re: New Post Bot
Post by: Sorunome on September 30, 2013, 12:31:46 pm
no, this one is even different.

I modified the smf software, when you post a new post it executes now in addition a shell script with the post information. To prevent long loading times i added &> /dev/null & which basically disables output and executes the shell script in the background, so php doesn't care about it anymore.

In the shell script i do a internal call over a port to the irc bot which listens to that port and then posts that information.


No worries, if efnet dies you can still post ;)
Title: Re: New Post Bot
Post by: Streetwalrus on September 30, 2013, 12:49:42 pm
Sounds pretty cool ! :D Since when is it up ?
Title: Re: New Post Bot
Post by: Sorunome on September 30, 2013, 12:50:54 pm
since like 3 houres :)
Title: Re: New Post Bot
Post by: DJ Omnimaga on November 01, 2013, 10:14:40 pm
Ok so yeah, just to confirm, the new post bot doesn't work anymore when we post using Tapatalk. D:
Title: Re: New Post Bot
Post by: Streetwalrus on November 02, 2013, 03:52:47 am
Yeah, I'm helping Soru to debug it. ;)
Title: Re: New Post Bot
Post by: Sorunome on November 02, 2013, 04:18:47 am
The hard part is that i can't test it myself because i don't have tapatalk <_<
Title: Re:
Post by: Streetwalrus on November 02, 2013, 04:31:03 am
OK so THE TEST
Edit : This was a Tapatalk post to test. It looks like it's working now. :D
Title: Re: New Post Bot
Post by: Sorunome on November 02, 2013, 04:33:33 am
/me lost

So yeah, basically the problem was that tapatalk uses its own function to add posts to the database and I only modified the SMF one to push notifications :)
Title: Re: New Post Bot
Post by: Streetwalrus on November 02, 2013, 04:34:13 am
/me lost
/me runs
Title: Re: New Post Bot
Post by: DJ Omnimaga on November 05, 2013, 03:08:49 am
Bug with new post parsing:

[03:05:47]   *   Omnom post by say in Mockups please http://ourl.ca/12844/311972

When I tried posting in http://ourl.ca/15441 , this is what happened.
Title: Re: New Post Bot
Post by: Sorunome on November 05, 2013, 05:26:53 am
Bug with new post parsing:

[03:05:47]   *   Omnom post by say in Mockups please http://ourl.ca/12844/311972

When I tried posting in http://ourl.ca/15441 , this is what happened.
did you use tapatalk or the real website?
Title: Re: Re: New Post Bot
Post by: DJ Omnimaga on November 05, 2013, 10:25:43 am
The site is real?? O.O

Just kidding, I used the website.
Title: Re: New Post Bot
Post by: Sorunome on November 05, 2013, 11:50:13 am
The site is real?? O.O
No, nothing is real. We are in a matrix.
Quote
Just kidding, I used the website.
I got it too, seems like it is a issue in the SMF modification, and i don't really want to screw around with that one D:
Title: Re: New Post Bot
Post by: DJ Omnimaga on March 22, 2014, 04:19:46 pm
Suggestion: When an admin approves a news article, post or topic, you should make the bot notify of it as a new post (even if the post is older)
Title: Re: New Post Bot
Post by: Sorunome on March 22, 2014, 04:50:32 pm
I guess it wouldn't be too hard, but I still need to enable tapatalk new post notification again :P
Title: Re: New Post Bot
Post by: DJ Omnimaga on May 06, 2014, 08:53:23 pm
The bot is down since the last netsplit D:
Title: Re: New Post Bot
Post by: Eeems on May 07, 2014, 12:44:15 am
The bot is down since the last netsplit D:
Please report this via http://ourl.ca/issue next time. Fixed.