Spiders & bots to add to phpBB

Do not post support requests, bug reports or feature requests. Discuss phpBB here. Non-phpBB related discussion goes in General Discussion!
Scam Warning
Pony99CA
Registered User
Posts: 4783
Joined: Thu Sep 30, 2004 3:13 pm
Location: Hollister, CA
Name: Steve
Contact:

Re: Spiders & bots to add to phpBB

Post by Pony99CA » Mon Apr 08, 2013 11:49 pm

techman41973 wrote:I appreciate all of the work that went into these lists and script.
But can anyone explain the advantages of adding this long list of new bots to my phpbb site?
Have all or most of these bots been tested to be beneficial in some way?
I'm not sure how a bot (if perhaps it was a spam bot) could negatively effect my site.
The advantage is that Bots get put into the correct group (and therefore get the correct permissions). Bots not in the Bots list get treated as Guests, which means they see Register and Log-In links, etc.

Steve
Silicon Valley Pocket PC (http://www.svpocketpc.com)
Creator of manage_bots and spoof_user (ask me)
Need hosting for a small forum with full cPanel & MySQL access? Contact me or PM me.

User avatar
@Marcin
Registered User
Posts: 19
Joined: Wed Feb 01, 2012 12:05 pm
Location: Milton Keynes
Name: Marcin
Contact:

Re: Spiders & bots to add to phpBB

Post by @Marcin » Thu May 02, 2013 11:09 am

IP: 65.52.0.135
NewsGatorOnline/2.0 - http://www.newsgator.com

JRD
Registered User
Posts: 2
Joined: Tue Jun 04, 2013 5:04 pm
Location: España
Contact:

Re: Spiders & bots to add to phpBB

Post by JRD » Tue Jun 04, 2013 5:25 pm

Bot name: uMBot-LN/1.0 [Bot]
Agent match: uMBot-LN
Information on the bot: www.ubermetrics-technologies.com

Bot name: ProCogSEOBot/1.0
Agent match: ProCogSEOBot
Information on the bot: www.procog.com
Pásate por la nueva comunidad de jugadores de Juegos como Counter Strike, Minecraft, lol, Lineage, ArcheAge y muchos más!

Schwpz
Registered User
Posts: 335
Joined: Wed May 07, 2003 1:33 pm
Location: Planet Zot
Contact:

Re: Spiders & bots to add to phpBB

Post by Schwpz » Sun Jul 28, 2013 2:42 pm

Bot name: BLEXBot/1.0 [Bot]
Agent match: BLEXBot
Information on the bot: http://webmeup.com/crawler.html


Bot name: A6-Indexer/1.0 [Bot]
Agent match: A6-Indexer
Information on the bot: http://www.a6corp.com/a6-web-scraping-policy/
..:: PlanetZot.com - Your ultimate source for animation! ^^

Danielx64
Registered User
Posts: 1369
Joined: Wed Nov 04, 2009 5:51 am
Location: In a server room in Australia
Name: Daniel
Contact:

Re: Spiders & bots to add to phpBB

Post by Danielx64 » Mon Jul 29, 2013 12:53 am

Q: Is Manage_Bots 5.5 the latest stable version?

Thank-you :)
Please note that I will not be porting any of my mods to phpBB 3.1. Sorry for the inconvenience this may cause.
Image

Peter77sx
Registered User
Posts: 3253
Joined: Wed Nov 09, 2005 2:51 pm

Re: Spiders & bots to add to phpBB

Post by Peter77sx » Mon Jul 29, 2013 9:25 am

*Daniel wrote:Q: Is Manage_Bots 5.5 the latest stable version?

Thank-you :)
There is Manage_Bots 6.0 Beta Test I've not had any problems with it so far. ;)

User avatar
Marcus Wendel
Registered User
Posts: 534
Joined: Sun Mar 10, 2002 5:58 pm
Location: Sweden
Contact:

Re: Spiders & bots to add to phpBB

Post by Marcus Wendel » Sun Aug 11, 2013 5:46 pm

Bot name: Easou [Bot]
Agent match: EasouSpider
User agent string: Mozilla/5.0 (compatible; EasouSpider; +http://www.easou.com/search/spider.html)
Website: http://www.easou.com

/Marcus

User avatar
Marcus Wendel
Registered User
Posts: 534
Joined: Sun Mar 10, 2002 5:58 pm
Location: Sweden
Contact:

Re: Spiders & bots to add to phpBB

Post by Marcus Wendel » Fri Aug 16, 2013 3:02 pm

Bot name: Yunyun [Bot]
Agent match: YRSpider
User agent string: Mozilla/5.0 (compatible; YRSpider; +http://www.yunyun.com/SiteInfo.php?r=about)
Website: http://www.yunyun.com

Schwpz
Registered User
Posts: 335
Joined: Wed May 07, 2003 1:33 pm
Location: Planet Zot
Contact:

Re: Spiders & bots to add to phpBB

Post by Schwpz » Wed Aug 21, 2013 7:44 am

Thanks Marcus!
..:: PlanetZot.com - Your ultimate source for animation! ^^

User avatar
5hocK
Registered User
Posts: 3011
Joined: Wed Nov 23, 2011 7:00 pm
Location: England

Re: Spiders & bots to add to phpBB

Post by 5hocK » Sat Aug 31, 2013 12:42 pm

Bot name: BLEX [Bot]
Agent match: BLEXBot
User agent string: Mozilla/5.0 (compatible; BLEXBot/1.0; +http://webmeup.com/crawler.html)
Website: http://webmeup.com/crawler.html


Edit: Scrub that, searched for BLEX in this topic, found nothing; posted and then saw it a few posts up :P
A thread is pretty much a topic. Stop with the pedantics will ya.

Peter77sx
Registered User
Posts: 3253
Joined: Wed Nov 09, 2005 2:51 pm

Re: Spiders & bots to add to phpBB

Post by Peter77sx » Thu Sep 05, 2013 12:37 am

Bot Name: DuckDuckGo [Bot]
User Agent String:
DuckDuckBot/1.0; (+http://duckduckgo.com/duckduckbot.html)

website: https://duckduckgo.com/duckduckbot

Baraka
Registered User
Posts: 67
Joined: Tue Oct 19, 2010 12:49 pm

Re: Spiders & bots to add to phpBB

Post by Baraka » Fri Sep 06, 2013 2:52 am

"Mozilla/5.0 (compatible; Genieo/1.0 http://www.genieo.com/webfilter.html)"

Not sure if this helps, only info i could see

User avatar
Marcus Wendel
Registered User
Posts: 534
Joined: Sun Mar 10, 2002 5:58 pm
Location: Sweden
Contact:

Re: Spiders & bots to add to phpBB

Post by Marcus Wendel » Sun Sep 08, 2013 9:03 am

Bot name: ADmantX [Bot]
Agent match: ADmantX
User agent string: ADmantX Platform Semantic Analyzer - ADmantX Inc. - www.admantx.com - support@admantx.com
Website: http://www.admantx.com

/Marcus

cangor
Registered User
Posts: 68
Joined: Thu Oct 28, 2010 1:56 pm

Re: Spiders & bots to add to phpBB

Post by cangor » Mon Sep 09, 2013 2:25 pm

Thanks for this script and all people sharing their data!

Will there be an updated script to automatically insert all the bots from this thread or from http://phpbb-bots.blogspot.de/ into the phpbb-database? The last version I found seems to be older than a year:
viewtopic.php?f=64&t=935605&start=705#p13172802

Pony99CA
Registered User
Posts: 4783
Joined: Thu Sep 30, 2004 3:13 pm
Location: Hollister, CA
Name: Steve
Contact:

Re: Spiders & bots to add to phpBB

Post by Pony99CA » Tue Sep 10, 2013 2:35 am

First, I'm glad you like the manage_bots script. Thanks. :)
cangor wrote: Will there be an updated script to automatically insert all the bots from this thread or from http://phpbb-bots.blogspot.de/ into the phpbb-database?
If you want the bots actually added to phpBB's official database, you'll have to take that up with the developers. I believe that Oleg made a post about that previously.

If you want them added to the script, why would I use that site? Nobody has ever mentioned it here before as far as I can recall and there are lots of lists of bots, so I'm not going to scan them all. If somebody wants a bot added to the script, they should post it here (in the format shown above, after ensuring that it's not already in the script).
cangor wrote: The last version I found seems to be older than a year
As I mentioned previously, I've been busy with other things. If I get some free time, I'll add the bots posted here since my last update to the script.

However, I've tried to make adding your own bots to the script fairly easy (if you can read PHP).

Steve
Silicon Valley Pocket PC (http://www.svpocketpc.com)
Creator of manage_bots and spoof_user (ask me)
Need hosting for a small forum with full cPanel & MySQL access? Contact me or PM me.

Post Reply

Return to “phpBB Discussion”