Claudebot attack

[ivailo95]

I "ban"claude bot, but still is hanging arround my board

Screenshot 2024-05-04 17.27.44.png

Screenshot 2024-05-04 17.31.53.png

i must block it with CF

[elc32955]

I had over 500 instances of ClaudeBot hit my site simultaneously and come close to bringing the shared webserver to it's knees, as well as over 50 instances of the Facebook bots. All at once. The robots.txt mods seem to have taken care of most of that, however I've also taken the step of temporarily removing open read access without an active forum account until these bots get under control. Too many times of having significant web server slowdowns and my site will probably be looking for a new home as it shares server space with commercial profit-making customers for the webhost.

Understand from monitoring some other forums that the current iteration of ClaudeBot is supposedly being hosted by Amazon AWS....

Eric

[Talk19Zehn]

Hello,

My personal thoughts ->
at the moment I have the suspicion that (Amazon ...(...)... compatible; ClaudeBot/1.0 is testing, checking and/or searching for formulations -> AI.
Although there is nothing to buy from (my) sites! I don't run, never run a shopsystem.

AI bots such as ChatGPT or Google AI are able to automatically retrieve content from websites and use it for various purposes. These bots can access content without authorisation and use it for unwanted purposes, which can lead to copyright infringements and other legal problems. Blocking ChatGPT User Agents is important to ensure the security and integrity of your website, conserve resources and defend against potential cyber-attacks. The unauthorised use of your content. This can have a negative impact on the reputation and success of your own website.

Wasting resources and cyber security threats through AI:
Since ChatGPT user agents make automated requests to your website, they can consume bandwidth and server resources. This can cause your website to load slower and affect the user experience of real visitors, especially if multiple bots are accessing your website at the same time or the server is not optimised for high traffic.

That's why I have also entered in the *robots.txt ->

CCBot, ChatGPT-User, GPTBot: *examples (!)

Code: Select all

User-agent: CCBot
User-agent: ChatGPT-User
User-agent: GPTBot
User-agent: ClaudeBot
User-agent: ClaudeBot/1.0
Disallow: /

and the entry in my .htaccess now reads as follows:

Code: Select all

RewriteCond %{HTTP_USER_AGENT} 11A465|Ahrefs|ArchiveBot|Baiduspider|BLEXBot|Bytedance|Bytespider|CCBot|ChatGPT-User|ClaudeBot|Curebot|Daum|Detectify|DotBot|Elisabot|Grapeshot|GPTBot|heritrix|Kinza|LieBaoFast|Linguee|LMY47V|MauiBot|Mb2345Browser|MegaIndex|MetaJobBot|MicroMessenger|MJ12bot|MQQBrowser|PageFreezer|PetalBot|PiplBot|Riddler|Screaming.Frog|Search365bot|SearchBlox|Seekport|SemanticScholarBot|SEOkicks|serpstatbot|Siteimprove.com|Sogou.web.spider|trendictionbot|TurnitinBot|UCBrowser|UptimeRobot|weborama-fetcher|Vagabondo|VelenPublicWebCrawler|YandexBot|YisouSpider [NC]
RewriteRule ^.* - [F,L]

and since today

Code: Select all

RewriteCond %{HTTP_USER_AGENT} ChatGPT|GPT-4|OpenAI [NC]
RewriteRule .* - [F,L]

I will see what happens now. Specialists probably know more than I can do.

Regards

[wintstar]

I would recommend this library on Github:
https://github.com/JayBizzle/Crawler-Detect

It should be very easy to create an extension for phpbb.
https://github.com/JayBizzle/Crawler-De ... awlers.php

This can also be combined with the phpbb database table phpbb_bots. I use this on my website and have written a script that automatically inserts new bots into the library. On my website I use the phpbb function bots from the version 3.0.14. Of course updated to the new php version 8.3

[Mick]

try it out and report back

Some time later . . .

The wildcards seem to have worked, no more Claudebot or derivatives. I did see PetalBot pop up very briefly this morning but it disappeared as quickly as it came, probably reacting to the robots.txt I suppose.

[Talk19Zehn]

Claudebot attack and others

Okay Mick, what I found in a hurry, on the fly.

I have added the -> petalbot <- bot in the robots (lower case letters) and also in the .htaccess.
Language - German -> https://datadome.co/de/learning-center- ... blockiert/

The eye is alert, because thanks for your hint. The extent to which resources are consumed is unclear. The individual case decides on a recurring basis:
advantages / disadvantages

Greetings

Edit: 06 May 2024, 10:39
@Mick,

I´ve changed -> petalbot <- to -> PETALBOT <-

and

I found the following page, which stops this bot and others via the .htaccess file with -> RewriteRule .* - [R=429]

Status Code 429 Too Many Connections / 429 Too Many Requests

Whether it still works in this way and fulfils the purpose that is being sought, I cannot judge exactly. I can't find anything precise on this, especially with regard to up-to-dateness. And at the moment I am not yet clear about the nature of the consequences in the overall appearance of a board. Whether "dangers" could arise that are unintentional ... (!?).

Language German:
https://hyaena.de/blog/2022/05/20/apach ... s-blocken/

See perhaps also if required:

Language German:
https://www.hosttest.de/artikel/too-man ... e-entsteht

https://datadome.co/de/learning-center- ... blockiert/

Many Regards

[Talk19Zehn]

@wintstar and hello, as far as I know, phpBB is not yet officially approved / optimised for PHP 8.3.

An extension would certainly be an option, which probably requires / may require more than initially thought. Difficult basis at the moment ...

Many regards

[lochness]

Thanks for the info. It looks like I was being attacked both by ClaudeBot and Bytespider and the Facebook hit.

[Mick]

Bytespider seems to have become particularly aggressive this last couple of days, hopefully the .htaccess code posted by HiFiKabin will take care of that.

Et voilà - 9 minutes later and Bytespider is no more.

[ivailo95]

Bytespider seems to have become particularly aggressive this last couple of days, hopefully the .htaccess code posted by HiFiKabin will take care of that.

Where is it?

[HiFiKabin]

The Block Bad Bots HTACCESS I have on my extensions board might block it as it is, and/or you can add Claudebot to the list of blocked bots

[timeforhelp1]

The Block Bad Bots HTACCESS I have on my extensions board might block it as it is, and/or you can add Claudebot to the list of blocked bots

How would I add this on my .htaccess?
Copy/paste below everything else?

Also guys I'm am getting absolutely hammered by tencent bot.
And if I'm honest I want to block ALL from China including Huawei which is also hammering my site.

[HiFiKabin]

If your .HTACCESS has not been altered from the one supplied in the phpBB download then just replace it with the one in the zip file (SSL or non SSL as appropriate)

[timeforhelp1]

Thanks.

I'm on cloudflare and my host has just "switched the under attack mode on and redirected all Chinese traffic away from the website".
Asking him to do Singapore now.

Fingers crossed!

[nou nou]

This is crazy - I've been having the exact same issue for the past 5 days and today I've been at it for hours. Tencent from Singapore plus an attempt at a code injection.

Tried setting up CloudFlare today but CF completely breaks my board as soon as I turn it on.

If you're blocking IP ranges, here are the ones I've identified from Tencent:

Code: Select all

		Deny from 43.130.13.0/24
		Deny from 43.133.32.0/19
		Deny from 43.134.0.0/18
		Deny from 43.134.64.0/18
		Deny from 43.134.128.0/18
		Deny from 43.134.128.0/18
		Deny from 43.135.128.0/18
		Deny from 43.153.0.0/18
		Deny from 43.153.0.0/18
		Deny from 43.153.64.0/18
		Deny from 43.156.0.0/18
		Deny from 43.156.64.0/18
		Deny from 43.159.41.0/24
		Deny from 43.159.144.0/24
		Deny from 43.159.146.0/24
		Deny from 101.32.0.0/16
		Deny from 129.226.0.0/16
		Deny from 170.106.0.0/16

and these are Amazonbot ones that don't respect robots.txt:

Code: Select all

		Deny from 3.128.0.0/9
		Deny from 3.224.0.0/12
		Deny from 23.20.0.0/14
		Deny from 47.128.0.0/14
		Deny from 52.0.0.0/10
		Deny from 52.64.0.0/12

With these three individual ones generating a HUGE amount of traffic:

Code: Select all

		Deny from 23.22.35.162
		Deny from 3.224.220.101
		Deny from 52.70.240.171

And this is the one where the code injection attempt came from:

Code: Select all

		Deny from 213.109.202.0/24

Seems the AI hype has all of a sudden discovered the value of forums. I wonder why; surely not because of the enormous collective effort that has gone into curating valuable content

Anyway, hope these can help someone.