Page 5 of 61

Re: Spiders & bots to add to phpbb3

Posted: Mon Jul 28, 2008 6:56 am
by Marcus Wendel
3Di wrote:
Marcus Wendel wrote:PhpBB3 has a nice built in list of bots, but it is not complete so I'm wondering which bots you all have added to your forums, please share in this thread.

So far I've only added one........ snip ......
It's time to resume all of this into the Topic starter, for a better reading... do you agree?
Regards. :)
I agree, but I'm not sure how much information I can include in one single post.

/Marcus

Re: Spiders & bots to add to phpbb3

Posted: Mon Jul 28, 2008 11:16 am
by 3Di
Marcus Wendel wrote:
3Di wrote:
Marcus Wendel wrote:PhpBB3 has a nice built in list of bots, but it is not complete so I'm wondering which bots you all have added to your forums, please share in this thread.

So far I've only added one........ snip ......
It's time to resume all of this into the Topic starter, for a better reading... do you agree?
Regards. :)
I agree, but I'm not sure how much information I can include in one single post.

/Marcus
Maybe (in case) a link to an external web page does the trick. ;)

Re: Spiders & bots to add to phpbb3

Posted: Tue Jul 29, 2008 2:23 pm
by Luchtzak
Suggestion: a sql-query to run in the database so all the bots are added in an instant ;-)

Re: Spiders & bots to add to phpbb3

Posted: Tue Jul 29, 2008 3:40 pm
by Marcus Wendel
Marcus Wendel wrote:Bot name: Twiceler [Bot]
Agent match: Twiceler
Information on the bot: http://cuill.com/twiceler/robot.html
Corrected the name to Cuil [Bot]

/Marcus

Re: Spiders & bots to add to phpbb3

Posted: Tue Jul 29, 2008 3:42 pm
by Marcus Wendel
Luchtzak wrote:Suggestion: a sql-query to run in the database so all the bots are added in an instant ;-)
Raimon posted one earlier in the thread and you can find it updated with the current bots at http://flagsforum.skalman.nu/viewtopic.php?f=2&t=282

/Marcus

Re: Spiders & bots to add to phpbb3

Posted: Tue Jul 29, 2008 4:10 pm
by Marcus Wendel
3Di wrote:
Marcus Wendel wrote:
3Di wrote:
Marcus Wendel wrote:PhpBB3 has a nice built in list of bots, but it is not complete so I'm wondering which bots you all have added to your forums, please share in this thread.

So far I've only added one........ snip ......
It's time to resume all of this into the Topic starter, for a better reading... do you agree?
Regards. :)
I agree, but I'm not sure how much information I can include in one single post.

/Marcus
Maybe (in case) a link to an external web page does the trick. ;)
I'll keep this thread updated with the bots: http://flagsforum.skalman.nu/viewtopic.php?f=2&t=282

/Marcus

Re: Spiders & bots to add to phpbb3

Posted: Tue Jul 29, 2008 5:58 pm
by Marcus Wendel
Two new ones today:

Bot name: Yebol [Bot]
Agent match: YebolBot
Information on the bot: http://tech.groups.yahoo.com/group/yeboltechnology/

Bot name: Microsoft Research [Bot]
Agent match: MSRBOT
Information on the bot: http://research.microsoft.com/research/sv/msrbot/

/Marcus

Re: Spiders & bots to add to phpbb3

Posted: Tue Jul 29, 2008 6:09 pm
by 3Di
Marcus Wendel wrote:
Luchtzak wrote:Suggestion: a sql-query to run in the database so all the bots are added in an instant ;-)
Raimon posted one earlier in the thread and you can find it updated with the current bots at http://flagsforum.skalman.nu/viewtopic.php?f=2&t=282

/Marcus
Tell him to have a look how phpbb3 (3.0.2 checked here) does that job. ;)

Reference file: install_install.php that lies into the install folder (there are also a lot of bots as candidates, there) ;)

Function: function add_bots($mode, $sub)

Regards.

Re: Spiders & bots to add to phpbb3

Posted: Wed Jul 30, 2008 8:06 am
by Eelke
Luchtzak wrote:Suggestion: a sql-query to run in the database so all the bots are added in an instant ;-)
If you want to make it really fancy: a web page that allows you to pick which bots you want to add, which then generates the SQL for your selection :) [Edit: or a PHP script that uses the phpBB funcion, per 3Di]

Re: Spiders & bots to add to phpbb3

Posted: Wed Jul 30, 2008 4:49 pm
by Marcus Wendel
One more today:

Bot name: BoardReader [Bot]
Agent match: BoardReader
Information on the bot: http://boardreader.com/info/robots.htm

/Marcus

Re: Spiders & bots to add to phpbb3

Posted: Wed Jul 30, 2008 10:19 pm
by Darthmat
I got this

Code: Select all

Parse error: syntax error, unexpected T_CONSTANT_ENCAPSED_STRING, expecting ')' in /home/soulrise/public_html/twb/forum/add_bots.php on line 60
You forgot to add comas to almost all of the ones following that line. ;)

Re: Spiders & bots to add to phpbb3

Posted: Thu Jul 31, 2008 5:22 am
by Marcus Wendel
Darthmat wrote:I got this

Code: Select all

Parse error: syntax error, unexpected T_CONSTANT_ENCAPSED_STRING, expecting ')' in /home/soulrise/public_html/twb/forum/add_bots.php on line 60
You forgot to add comas to almost all of the ones following that line. ;)
Sorry about that, I've corrected it.

/Marcus

Re: Spiders & bots to add to phpbb3

Posted: Thu Jul 31, 2008 1:08 pm
by scott_thewspot
Greetings,

I just tried applying this and it seems to work, but I do have some questions.

1. For the SpamBots, you mention you have them disabled. From what I can tell, after the bots are added, they are *all* enabled, including the spambots. There is nothing obvious in your script I can find that would treat them differently when adding them. Am I missing something here?

2. What happens if the script tries to add a bot that already exists in the DB?

It does seem odd that there isn't an "official" bot thread or update system already from the phpbb team themselves, even as part of the patch updates.

Re: Spiders & bots to add to phpbb3

Posted: Thu Jul 31, 2008 1:39 pm
by Marcus Wendel
scott_thewspot,

I've just reposted the script provided by Raimon earlier in the thread and unfortunately I don't know anything about php-programming, I hope is able to answer your questions.

My input here is limited to posting information on new bots I come across on my sites.

/Marcus

Re: Spiders & bots to add to phpbb3

Posted: Fri Aug 01, 2008 5:30 pm
by Marcus Wendel
Three new ones today:

Bot name: Best of the Web [Bot]
Agent match: BOTW Spider
Information on the bot: http://botw.org/

Bot name: Brandwatch [Bot]
Agent match: Jakarta Commons
IP match: 80.82.139
Information on the bot: http://www.brandwatch.net/crawler

Bot name: Kalooga [Bot]
Agent match: Kalooga
Information on the bot: http://www.kalooga.com

/Marcus