Spiders & bots to add to phpBB

Do not post support requests, bug reports or feature requests. Discuss phpBB here. Non-phpBB related discussion goes in General Discussion!
Scam Warning
User avatar
ameeck
Former Team Member
Posts: 6559
Joined: Mon Mar 21, 2005 6:57 pm

Re: Spiders & bots to add to phpbb3

Post by ameeck » Mon Mar 02, 2009 9:13 pm

3DI: That will not include the records in the users table.

User avatar
3Di
Former Team Member
Posts: 14093
Joined: Mon Apr 04, 2005 11:09 pm
Location: Milan (IT) Frankfurt (DE)
Name: Marco
Contact:

Re: Spiders & bots to add to phpbb3

Post by 3Di » Mon Mar 02, 2009 9:19 pm

ameeck wrote:3DI: That will not include the records in the users table.
That's enough for my needs (I just need a CSV), it sorts like:
1,1,AdsBot [Google],3,AdsBot-Google,
2,1,Alexa [Bot],4,ia_archiver,
3,1,Alta Vista [Bot],5,Scooter/,
4,1,Ask Jeeves [Bot],6,Ask Jeeves,
5,1,Baidu [Spider],7,Baiduspider+(,
6,1,Exabot [Bot],8,Exabot/,
7,1,FAST Enterprise [Crawler],9,FAST Enterprise Crawler,
8,1,FAST WebCrawler [Crawler],10,FAST-WebCrawler/,
9,1,Francis [Bot],11,http://www.neomo.de/,
10,1,Gigabot [Bot],12,Gigabot/,
11,1,Google Adsense [Bot],13,Mediapartners-Google,
my PHP controller knows how to handle it.

Indeed it is not the right way of how to transfer that content to another forum, though.

Be my guest so.. thank you. :)
Please PM me only to request paid works. Thx.
Want to compensate me for my interest? Donate
My development's activity º PhpStorm's proud user
Extensions, Scripts, MOD porting, Update/Upgrades
👨‍🏫 | Take a tour to | The Studio | 👨‍🏫

the.ronin
Registered User
Posts: 184
Joined: Sun Oct 05, 2008 10:53 pm

Re: Spiders & bots to add to phpbb3

Post by the.ronin » Tue Mar 03, 2009 1:47 am

ameeck wrote:3DI: That will not include the records in the users table.
Yeah I figured that out the hard way a long time ago. :lol:

User avatar
3Di
Former Team Member
Posts: 14093
Joined: Mon Apr 04, 2005 11:09 pm
Location: Milan (IT) Frankfurt (DE)
Name: Marco
Contact:

Re: Spiders & bots to add to phpbb3

Post by 3Di » Tue Mar 03, 2009 2:46 am

the.ronin wrote:
ameeck wrote:3DI: That will not include the records in the users table.
Yeah I figured that out the hard way a long time ago. :lol:
Well, just I need CVS though. :)
Please PM me only to request paid works. Thx.
Want to compensate me for my interest? Donate
My development's activity º PhpStorm's proud user
Extensions, Scripts, MOD porting, Update/Upgrades
👨‍🏫 | Take a tour to | The Studio | 👨‍🏫

User avatar
Marcus Wendel
Registered User
Posts: 534
Joined: Sun Mar 10, 2002 5:58 pm
Location: Sweden
Contact:

Re: Spiders & bots to add to phpbb3

Post by Marcus Wendel » Sat Mar 07, 2009 9:44 pm

New today:

Bot name: Setooz [Bot]
Agent match: oozbot
Information on the bot: http://www.setooz.com/oozbot.html

/Marcus

User avatar
onehundredandtwo
Registered User
Posts: 1228
Joined: Fri Nov 14, 2008 8:07 am

Re: Spiders & bots to add to phpbb3

Post by onehundredandtwo » Sat Mar 21, 2009 2:09 am

I don't believe this has been added yet:

Bot name: CommandCom [Bot]
Agent match: crawly@commandcom.com
Information on the bot: http://vlab.authentium.com/ (under construction)
Need help preventing spam? Read Preventing spam in phpBB 3.0.6 and above

User avatar
Marcus Wendel
Registered User
Posts: 534
Joined: Sun Mar 10, 2002 5:58 pm
Location: Sweden
Contact:

Re: Spiders & bots to add to phpbb3

Post by Marcus Wendel » Sat Mar 21, 2009 3:25 pm

New today:

Bot name: Soso Image [Bot]
Agent match: Sosoimagespider
Information on the bot: http://help.soso.com/soso-image-spider.htm (Chinese search engine)

/Marcus

the.ronin
Registered User
Posts: 184
Joined: Sun Oct 05, 2008 10:53 pm

Re: Spiders & bots to add to phpbb3

Post by the.ronin » Sun Mar 22, 2009 6:18 am

Bot name: SimilarPages Nutch Crawler [Bot]
Agent match: SimilarPages/Nutch-1.0-dev
Information on the bot: http://www.botsvsbrowsers.com/details/240535/index.html

http://www.similarpages.com is just a logo.

User avatar
Marcus Wendel
Registered User
Posts: 534
Joined: Sun Mar 10, 2002 5:58 pm
Location: Sweden
Contact:

Re: Spiders & bots to add to phpbb3

Post by Marcus Wendel » Wed Apr 08, 2009 3:47 pm

New today:

Bot name: WillyFogg [Bot]
Agent match: WillyBot
Information on the bot: http://www.willyfogg.com/willybot.php

/Marcus

User avatar
Marcus Wendel
Registered User
Posts: 534
Joined: Sun Mar 10, 2002 5:58 pm
Location: Sweden
Contact:

Re: Spiders & bots to add to phpbb3

Post by Marcus Wendel » Fri Apr 10, 2009 5:46 pm

New today:

Bot name: Topsy [Bot]
Agent match: Butterfly
Information on the bot: http://labs.topsy.com/butterfly.html

/Marcus

superj707
Registered User
Posts: 1136
Joined: Thu Feb 26, 2009 12:20 am

Re: Spiders & bots to add to phpbb3

Post by superj707 » Tue Apr 14, 2009 2:10 am

would like information about how to add Redlink bot

http://www.aboutus.org/AboutUsBot_Invoked_from_Redlink

User avatar
onehundredandtwo
Registered User
Posts: 1228
Joined: Fri Nov 14, 2008 8:07 am

Re: Spiders & bots to add to phpbb3

Post by onehundredandtwo » Tue Apr 14, 2009 4:45 am

superj707 wrote:would like information about how to add Redlink bot

http://www.aboutus.org/AboutUsBot_Invoked_from_Redlink
Bot name: AboutUs [bot]
Agent match: AboutUsBot/
Information on the bot: http://www.aboutus.org/AboutUs:Bot
Need help preventing spam? Read Preventing spam in phpBB 3.0.6 and above

superj707
Registered User
Posts: 1136
Joined: Thu Feb 26, 2009 12:20 am

Re: Spiders & bots to add to phpbb3

Post by superj707 » Tue Apr 14, 2009 5:40 am

onehundredandtwo wrote:
superj707 wrote:would like information about how to add Redlink bot

http://www.aboutus.org/AboutUsBot_Invoked_from_Redlink
Bot name: AboutUs [bot]
Agent match: AboutUsBot/
Information on the bot: http://www.aboutus.org/AboutUs:Bot

thanks !!

User avatar
Marcus Wendel
Registered User
Posts: 534
Joined: Sun Mar 10, 2002 5:58 pm
Location: Sweden
Contact:

Re: Spiders & bots to add to phpbb3

Post by Marcus Wendel » Sun May 03, 2009 7:18 am

New today:

Bot name: Paxle [Bot]
Agent match: PaxleFramework
Information on the bot: http://www.paxle.net/en/bot

/Marcus

youngjediknight
Registered User
Posts: 385
Joined: Wed Apr 22, 2009 2:58 pm

Re: Spiders & bots to add to phpbb3

Post by youngjediknight » Tue May 19, 2009 11:32 pm

Great thread!

Post Reply

Return to “phpBB Discussion”