Spiders & bots to add to phpBB

Do not post support requests, bug reports or feature requests. Discuss phpBB here. Non-phpBB related discussion goes in General Discussion!
Get Involved
ameeck
Former Team Member
Posts: 6559
Joined: Mon Mar 21, 2005 6:57 pm

Re: Spiders & bots to add to phpbb3

Post by ameeck »

3DI: That will not include the records in the users table.
User avatar
3Di
I've Been Banned!
Posts: 17538
Joined: Mon Apr 04, 2005 11:09 pm
Location: I'm with Ukraine 🇺🇦
Name: Marco
Contact:

Re: Spiders & bots to add to phpbb3

Post by 3Di »

ameeck wrote:3DI: That will not include the records in the users table.
That's enough for my needs (I just need a CSV), it sorts like:
1,1,AdsBot [Google],3,AdsBot-Google,
2,1,Alexa [Bot],4,ia_archiver,
3,1,Alta Vista [Bot],5,Scooter/,
4,1,Ask Jeeves [Bot],6,Ask Jeeves,
5,1,Baidu [Spider],7,Baiduspider+(,
6,1,Exabot [Bot],8,Exabot/,
7,1,FAST Enterprise [Crawler],9,FAST Enterprise Crawler,
8,1,FAST WebCrawler [Crawler],10,FAST-WebCrawler/,
9,1,Francis [Bot],11,http://www.neomo.de/,
10,1,Gigabot [Bot],12,Gigabot/,
11,1,Google Adsense [Bot],13,Mediapartners-Google,
my PHP controller knows how to handle it.

Indeed it is not the right way of how to transfer that content to another forum, though.

Be my guest so.. thank you. :)
🆓 Free support for our extensions also provided here: phpBB Studio
🚀 Looking for a specific feature or alternative option? We will rock you!
Please PM me only to request paid works. Thx. Buy me a coffee -> Image
My development's activity º PhpStorm's proud user º Extensions, Scripts, MOD porting, Update/Upgrades
the.ronin
Registered User
Posts: 184
Joined: Sun Oct 05, 2008 10:53 pm

Re: Spiders & bots to add to phpbb3

Post by the.ronin »

ameeck wrote:3DI: That will not include the records in the users table.
Yeah I figured that out the hard way a long time ago. :lol:
User avatar
3Di
I've Been Banned!
Posts: 17538
Joined: Mon Apr 04, 2005 11:09 pm
Location: I'm with Ukraine 🇺🇦
Name: Marco
Contact:

Re: Spiders & bots to add to phpbb3

Post by 3Di »

the.ronin wrote:
ameeck wrote:3DI: That will not include the records in the users table.
Yeah I figured that out the hard way a long time ago. :lol:
Well, just I need CVS though. :)
🆓 Free support for our extensions also provided here: phpBB Studio
🚀 Looking for a specific feature or alternative option? We will rock you!
Please PM me only to request paid works. Thx. Buy me a coffee -> Image
My development's activity º PhpStorm's proud user º Extensions, Scripts, MOD porting, Update/Upgrades
User avatar
Marcus Wendel
Registered User
Posts: 534
Joined: Sun Mar 10, 2002 5:58 pm
Location: Sweden
Contact:

Re: Spiders & bots to add to phpbb3

Post by Marcus Wendel »

New today:

Bot name: Setooz [Bot]
Agent match: oozbot
Information on the bot: http://www.setooz.com/oozbot.html

/Marcus
User avatar
onehundredandtwo
Registered User
Posts: 1228
Joined: Fri Nov 14, 2008 8:07 am

Re: Spiders & bots to add to phpbb3

Post by onehundredandtwo »

I don't believe this has been added yet:

Bot name: CommandCom [Bot]
Agent match: [email protected]
Information on the bot: http://vlab.authentium.com/ (under construction)
Need help preventing spam? Read Preventing spam in phpBB 3.0.6 and above
User avatar
Marcus Wendel
Registered User
Posts: 534
Joined: Sun Mar 10, 2002 5:58 pm
Location: Sweden
Contact:

Re: Spiders & bots to add to phpbb3

Post by Marcus Wendel »

New today:

Bot name: Soso Image [Bot]
Agent match: Sosoimagespider
Information on the bot: http://help.soso.com/soso-image-spider.htm (Chinese search engine)

/Marcus
the.ronin
Registered User
Posts: 184
Joined: Sun Oct 05, 2008 10:53 pm

Re: Spiders & bots to add to phpbb3

Post by the.ronin »

Bot name: SimilarPages Nutch Crawler [Bot]
Agent match: SimilarPages/Nutch-1.0-dev
Information on the bot: http://www.botsvsbrowsers.com/details/240535/index.html

http://www.similarpages.com is just a logo.
User avatar
Marcus Wendel
Registered User
Posts: 534
Joined: Sun Mar 10, 2002 5:58 pm
Location: Sweden
Contact:

Re: Spiders & bots to add to phpbb3

Post by Marcus Wendel »

New today:

Bot name: WillyFogg [Bot]
Agent match: WillyBot
Information on the bot: http://www.willyfogg.com/willybot.php

/Marcus
User avatar
Marcus Wendel
Registered User
Posts: 534
Joined: Sun Mar 10, 2002 5:58 pm
Location: Sweden
Contact:

Re: Spiders & bots to add to phpbb3

Post by Marcus Wendel »

New today:

Bot name: Topsy [Bot]
Agent match: Butterfly
Information on the bot: http://labs.topsy.com/butterfly.html

/Marcus
superj707
Registered User
Posts: 1136
Joined: Thu Feb 26, 2009 12:20 am

Re: Spiders & bots to add to phpbb3

Post by superj707 »

would like information about how to add Redlink bot

http://www.aboutus.org/AboutUsBot_Invoked_from_Redlink
User avatar
onehundredandtwo
Registered User
Posts: 1228
Joined: Fri Nov 14, 2008 8:07 am

Re: Spiders & bots to add to phpbb3

Post by onehundredandtwo »

superj707 wrote:would like information about how to add Redlink bot

http://www.aboutus.org/AboutUsBot_Invoked_from_Redlink
Bot name: AboutUs [bot]
Agent match: AboutUsBot/
Information on the bot: http://www.aboutus.org/AboutUs:Bot
Need help preventing spam? Read Preventing spam in phpBB 3.0.6 and above
superj707
Registered User
Posts: 1136
Joined: Thu Feb 26, 2009 12:20 am

Re: Spiders & bots to add to phpbb3

Post by superj707 »

onehundredandtwo wrote:
superj707 wrote:would like information about how to add Redlink bot

http://www.aboutus.org/AboutUsBot_Invoked_from_Redlink
Bot name: AboutUs [bot]
Agent match: AboutUsBot/
Information on the bot: http://www.aboutus.org/AboutUs:Bot

thanks !!
User avatar
Marcus Wendel
Registered User
Posts: 534
Joined: Sun Mar 10, 2002 5:58 pm
Location: Sweden
Contact:

Re: Spiders & bots to add to phpbb3

Post by Marcus Wendel »

New today:

Bot name: Paxle [Bot]
Agent match: PaxleFramework
Information on the bot: http://www.paxle.net/en/bot

/Marcus
youngjediknight
Registered User
Posts: 385
Joined: Wed Apr 22, 2009 2:58 pm

Re: Spiders & bots to add to phpbb3

Post by youngjediknight »

Great thread!
Post Reply

Return to “phpBB Discussion”