About the bots

[bidouille]

Hello folks,

Over the past few months, we've had a lot of visits from Bytedance / Bytespider (TikTok's bot).
It generates up to 600 guest connections and saturates our forum.

How does phpBB treat these bots compared to a normal user?
For example, does a robot's crawling generate fewer requests to the database than a user?

[Mick]

If the bot/spider is correctly entered in the ‘Spiders/Bots’ list phpBB removes certain unnecessary elements from the page that you wouldn't want to index such as profile links, search page etc. If you want to make a section of your board available to guests but do not want bots indexing those pages you can set the permissions to no access for the bots etc.

[bidouille]

Merci beaucoup Mick

[ukautoforums]

I had the same, basically killing the forums. I ended up blocking a load in htaccess file:

BrowserMatchNoCase "libwww-perl" bad_bot
BrowserMatchNoCase "wget" bad_bot
BrowserMatchNoCase "LieBaoFast" bad_bot
BrowserMatchNoCase "Mb2345Browser" bad_bot
BrowserMatchNoCase "zh-CN" bad_bot
BrowserMatchNoCase "MicroMessenger" bad_bot
BrowserMatchNoCase "zh_CN" bad_bot
BrowserMatchNoCase "Kinza" bad_bot
BrowserMatchNoCase "Bytespider" bad_bot
BrowserMatchNoCase "Baiduspider" bad_bot
BrowserMatchNoCase "Sogou" bad_bot
BrowserMatchNoCase "Datanyze" bad_bot
BrowserMatchNoCase "AspiegelBot" bad_bot
BrowserMatchNoCase "adscanner" bad_bot
BrowserMatchNoCase "serpstatbot" bad_bot
BrowserMatchNoCase "spaziodat" bad_bot
BrowserMatchNoCase "undefined" bad_bot
BrowserMatchNoCase "PetalBot" bad_bot
BrowserMatchNoCase "trendictionbot" bad_bot
BrowserMatchNoCase "SEMrushbot" bad_bot
BrowserMatchNoCase "AhrefsBot" bad_bot
Order Deny,Allow
Deny from env=bad_bot