Just a few notes from my point of view and experiences. The individual case decides, as you know.lurttinen wrote: Thu Apr 25, 2024 12:19 pm ...(...)...
"helpful, honest, and harmless AI system" as they say it is.
We can skew their training by blocking access to the data.
Code: Select all
RewriteCond %{HTTP_USER_AGENT} 11A465|Ahrefs|ArchiveBot|Baiduspider|BLEXBot|Bytedance|Bytespider|CCBot|ChatGPT-User|ClaudeBot|Curebot|Daum|Detectify|DotBot|Elisabot|Grapeshot|GPTBot|heritrix|Kinza|LieBaoFast|Linguee|LMY47V|MauiBot|Mb2345Browser|MegaIndex|MetaJobBot|MicroMessenger|MJ12bot|MQQBrowser|PageFreezer|PiplBot|Riddler|Screaming.Frog|Search365bot|SearchBlox|Seekport|SemanticScholarBot|SEOkicks|serpstatbot|Siteimprove.com|Sogou.web.spider|trendictionbot|TurnitinBot|UCBrowser|UptimeRobot|weborama-fetcher|Vagabondo|VelenPublicWebCrawler|YandexBot|YisouSpider [NC]
RewriteRule ^.* - [F,L]
Code: Select all
BrowserMatchNoCase "BLEXBot/1.0" bad_bot
BrowserMatchNoCase "claudebot" bad_bot
BrowserMatchNoCase "ClaudeBot" bad_bot
BrowserMatchNoCase "ClaudeBot/1.0" bad_bot
BrowserMatchNoCase "Elisabot" bad_bot
BrowserMatchNoCase "UptimeRobot" bad_bot
BrowserMatchNoCase "UptimeRobot/1.0" bad_bot
BrowserMatchNoCase "UptimeRobot/2.0" bad_bot
Order Deny,Allow
Deny from env=bad_bot
Code: Select all
User-agent: BLEXBot/1.0
User-agent: CCBot
User-agent: ChatGPT-User
User-agent: ClaudeBot
User-agent: ClaudeBot/1.0
User-agent: Elisabot
User-agent: GPTBot
User-agent: UptimeRobot
User-agent: UptimeRobot/1.0
User-agent: UptimeRobot/2.0
Disallow: /
you can tell by going to view who's online and view guestsxinu-mike wrote: Fri Apr 26, 2024 9:02 pm How can you tell it is something like this?
For the past two weeks, my board has slowed to a crawl.
I have to purge all the sessions to get it to be snappy again.
Is this what is going on?
I am using version 3.3.10.
robots.txt
. Google turns up many websites having issues with Claudebot and taking action via robots.txt. robots.txt
I have added:
Code: Select all
# 20-Apr-24, P_I, Claudebot causing problems for phpBB sites
User-Agent: Claudebot
Disallow: /
User-agent: anthropic-ai
Disallow: /
.htaccess
handling in place for Claudbot as a backup.Instead of having three entries couldn’t you use a wild card like:Talk19Zehn wrote: Fri Apr 26, 2024 1:33 pmCode: Select all
User-agent: UptimeRobot User-agent: UptimeRobot/1.0 User-agent: UptimeRobot/2.0 Disallow: /
Code: Select all
User-agent: UptimeRobot/*
User-Agent: ClaudeBot/1.0
etc.