Page 60 of 61

Re: Spiders & bots to add to phpBB

Posted: Fri May 06, 2016 5:46 am
by Binano

Re: Spiders & bots to add to phpBB

Posted: Mon Apr 24, 2017 3:51 am
by thunderchero
sorry to bring up old topic, but does this work on 3.2?

I also plan on adding "hifikabin" Block Bad Bots with .HTACCESS
https://phpbb.hifikabin.me.uk/viewtopic.php?f=6&t=110

thunderchero

Re: Spiders & bots to add to phpBB

Posted: Mon Apr 24, 2017 5:48 am
by pikachuturkey
thunderchero wrote:
Mon Apr 24, 2017 3:51 am
sorry to bring up old topic, but does this work on 3.2?
Yes it works.

Re: Spiders & bots to add to phpBB

Posted: Mon Sep 18, 2017 1:32 pm
by v12mike
I don't know if this Bot list is being maintained, but I spotted 3 active bots that are not in it:

Name / Agent
megaindex [Crawler] / megaindex.com
Qwant [BOT] / Qwantify
British Library [Bot] / bl.uk_lddc_bot

Re: Spiders & bots to add to phpBB

Posted: Wed Nov 01, 2017 4:39 pm
by open-your-eyes
Hi gues!

I would very much apreciated if someone could point me out to the latest script for adding bots to 3.0. I want lots of them. :D
Thanks.

Re: Spiders & bots to add to phpBB

Posted: Wed Nov 01, 2017 4:49 pm
by HiFiKabin
As 3.0.x is no longer supported, I doubt anyone would want to make an up to date script. This post (with the script) was last updated in 2012 viewtopic.php?p=12119145#p12119145

You should also seriously consider upgrading to phpBB 3.2.x

Re: Spiders & bots to add to phpBB

Posted: Thu Oct 18, 2018 12:18 pm
by Forex Station
Pony99CA wrote:
Fri Feb 13, 2015 3:14 am
Puchahawa wrote:That would be a good Idea Steve thanks! Know of any other bad bots to add to the list?
Check the manage_bots code.
Puchahawa wrote: Would it be the right choice to change their default group to the bad bot group after they are added, or would it make more sense to remove them from the bot group?
You could certainly make the default group "Bad Bots". I don't know about removing them from the Bots group, though. I presume that wouldn't break their bot status, but I don't see a good reason to do it -- they are still bots, after all.

Steve
Steve is the man!

Thank you so much for this great tool. I love it so much. Could we get an updated manage_bots.php for 2018? Including DuckDuckGo etc? :)

Re: Spiders & bots to add to phpBB

Posted: Thu Oct 18, 2018 12:25 pm
by open-your-eyes
I am also interested in this.

Re: Spiders & bots to add to phpBB

Posted: Thu Oct 18, 2018 12:30 pm
by </Solidjeuh>
Just search the bots you want on google and add them via ACP.
Like: https://duckduckgo.com/duckduckbot

Re: Spiders & bots to add to phpBB

Posted: Thu Oct 25, 2018 5:22 pm
by orynider
Here is an interesting source of bots https://github.com/monperrus/crawler-us ... gents.json

How about to add the e-mail address to the default 50 bots we have in the database?

Re: Spiders & bots to add to phpBB

Posted: Thu Oct 25, 2018 7:42 pm
by warmweer
orynider wrote:
Thu Oct 25, 2018 5:22 pm
How about to add the e-mail address to the default 50 bots we have in the database?
The problem here is that you'll need the 50 mailaddresses first 8-) .

Re: Spiders & bots to add to phpBB

Posted: Wed Nov 27, 2019 10:34 am
by bidouille
Hello,

This Bot list is still being maintained?

Ecosia search engine seems to be missing.
I don't find any clue on it.

Re: Spiders & bots to add to phpBB

Posted: Wed Nov 27, 2019 10:41 am
by HiFiKabin
I am afraid it has not been updated in years. I only hosted it as a service for everyone. If anyone would like to actively support it, please contact me.

Re: Spiders & bots to add to phpBB

Posted: Wed Nov 27, 2019 10:43 am
by 3Di
orynider wrote:
Thu Oct 25, 2018 5:22 pm
How about to add the e-mail address to the default 50 bots we have in the database?
Are you bound to mass-email them?

Re: Spiders & bots to add to phpBB

Posted: Wed Nov 27, 2019 11:56 am
by AmigoJack
Many years ago I also thought about filling in the e-mail addresses: the profile field was there, so why not using it when the information is available? But since then there was not even one occurance where contacting the crawler support was needed (no, never mass-e-mailing, just looking up the address). And those crawlers I sent e-mailed to proved to never answer, let alone change their erratic software.