[SPLIT] Python/3.10 aiohttp/3.9.3 anyone?

Do not post support requests, bug reports or feature requests. Discuss phpBB here. Non-phpBB related discussion goes in General Discussion!
Scam Warning
User avatar
Mick
Support Team Member
Support Team Member
Posts: 26702
Joined: Fri Aug 29, 2008 9:49 am

Python/3.10 aiohttp/3.9.3 anyone?

Post by Mick »

Has anyone seen “Python/3.10 aiohttp/3.9.3” or similar hanging about and or found an explanation that is understandable in plain language? Plus, has anyone found a way to stop it without blanket denying it’s IP addresses?

Note: I have searched such things as “what is”, “how to stop” etc.‘til I’m blue in the face but all the results are basically gobbledygook. AI is becoming a proper pain. I’ve just read an article on how to create your own bot in ten minutes in python on the python site. They have a library on bot creation so I assume it’s not python as such it’s probably a dweeb buggering about.
  • "The more connected we get the more alone we become" - Kyle Broflovski©
  • "The good news is hell is just the product of a morbid human imagination.
    The bad news is, whatever humans can imagine, they can usually create.
    " - Harmony Cobel©
🇬🇧
deninho32
Registered User
Posts: 227
Joined: Tue May 21, 2019 8:57 am

Re: Claudebot attack

Post by deninho32 »

I've blocked them in my .htaccess yesterday.
phpBB 3.3.7 | PHP Version 7.4.33 | Milk Theme
User avatar
pierredu
Registered User
Posts: 1268
Joined: Thu Nov 01, 2012 8:04 am
Location: Paris (France)

Re: Claudebot attack

Post by pierredu »

I have 28 guest sessions this morning with this User agent.
User avatar
HiFiKabin
Community Team Member
Community Team Member
Posts: 6731
Joined: Wed May 14, 2014 9:10 am
Location: Swearing at the PC, UK
Name: James

Re: Claudebot attack

Post by HiFiKabin »

I have had them hitting my site with up to 200 instances on line at the same time. The only solution I have found (so far) is a wildcard ip block in HTACCESS which stopped them instantly

Code: Select all

Order Allow,Deny
Deny from 173.252.*.*
Deny from 69.171.*.*
Allow from all
Not my preferred way, but it works
User avatar
bonelifer
Community Team Member
Community Team Member
Posts: 3609
Joined: Wed Oct 27, 2004 11:35 pm
Name: William

Re: Python/3.10 aiohttp/3.9.3 anyone?

Post by bonelifer »

Mick wrote: Wed Jun 19, 2024 6:32 am Has anyone seen “Python/3.10 aiohttp/3.9.3” or similar hanging about and or found an explanation that is understandable in plain language? Plus, has anyone found a way to stop it without blanket denying it’s IP addresses?

Note: I have searched such things as “what is”, “how to stop” etc.‘til I’m blue in the face but all the results are basically gobbledygook. AI is becoming a proper pain. I’ve just read an article on how to create your own bot in ten minutes in python on the python site. They have a library on bot creation so I assume it’s not python as such it’s probably a dweeb buggering about.
https://developers.facebook.com/communi ... 100617448/
https://www.reddit.com/r/webdev/comment ... http_does/

If it is from facebook, they supposedly honor the rate limit of robots.txt. Basically someone has shared your site and fb is going out to fetch your resources in order to show pretty things like an image of the page and grab meta data for link information. If it is one of their IP's, aiohttp is however the name of the library, so not necessarily FB. https://docs.aiohttp.org/en/stable/

Something like this htaccess entry(used to get example https://gist.github.com/dvlop/fca36213a ... e038a3bbc1)

Code: Select all

# Start Bad Bot Prevention
<IfModule mod_setenvif.c>
# SetEnvIfNoCase User-Agent ^$ bad_bot
SetEnvIfNoCase User-Agent "^aiohttp.*" bad_bot
<Limit GET POST PUT>
  Order Allow,Deny
  Allow from all
  Deny from env=bad_bot
</Limit>
</IfModule>
# End Bad Bot Prevention
Also could use these examples: https://stackoverflow.com/questions/274 ... n-htaccess
William Jacoby - Community Team
Knowledge Base | phpBB Board Rules | Search Customisation Database
Please don't contact me via PM or email for phpBB support .

phpBB Modders is looking for developers! If you have phpBB experience and want to join us, click here!
User avatar
Mick
Support Team Member
Support Team Member
Posts: 26702
Joined: Fri Aug 29, 2008 9:49 am

Re: Claudebot attack

Post by Mick »

I saw the Reddit thing but was unaware this, whatever it is, was FB, how would a normal none FB user have a clue what it is?
bonelifer wrote: Wed Jun 19, 2024 9:00 amthey supposedly honor the rate limit of robots.txt
Not in the two cases I’ve had dealings with in the last 24 hours.

As it’s FB I have no issue blanket denying it in .htaccess.
  • "The more connected we get the more alone we become" - Kyle Broflovski©
  • "The good news is hell is just the product of a morbid human imagination.
    The bad news is, whatever humans can imagine, they can usually create.
    " - Harmony Cobel©
🇬🇧
deninho32
Registered User
Posts: 227
Joined: Tue May 21, 2019 8:57 am

Re: Claudebot attack

Post by deninho32 »

Adding these 2 (today a new one already) to my .htacces file solved it.

BrowserMatchNoCase "python" bad_bot
BrowserMatchNoCase "CFNetwork" bad_bot

Order Deny,Allow
Deny from env=bad_bot

That Facebook bot is a nasty one. My error log is filled with this:

Code: Select all

[Wed Jun 19 11:28:29.414188 2024] [access_compat:error] [pid 10266:tid 140604168185600] [client 148.72.152.70:42540] AH01797: client denied by server configuration: /home/negentien13/domains/forum.negentiendertien.nl/public_html/viewtopic.php
phpBB 3.3.7 | PHP Version 7.4.33 | Milk Theme
User avatar
Mick
Support Team Member
Support Team Member
Posts: 26702
Joined: Fri Aug 29, 2008 9:49 am

Re: Claudebot attack

Post by Mick »

I’ve just seen another one pop up that I don’t think I’ve ever seen before ows.eu/owler but, to be fair, the Owler site https://openwebsearch.eu/owler/ does have opt out instructions. I would say though, in that case, shouldn’t you have an opt in in the first place?

Owler is supposed to respect robots.txt

Code: Select all

User-agent: Owler
User-agent: GenAI
Disallow: /
Not tried it yet so can’t guarantee it works.
  • "The more connected we get the more alone we become" - Kyle Broflovski©
  • "The good news is hell is just the product of a morbid human imagination.
    The bad news is, whatever humans can imagine, they can usually create.
    " - Harmony Cobel©
🇬🇧
User avatar
invenio
Registered User
Posts: 377
Joined: Wed Dec 09, 2015 1:45 pm
Location: New Hampshire, USA

Re: Claudebot attack

Post by invenio »

Mick wrote: Wed Jun 19, 2024 6:32 amHas anyone seen “Python/3.10 aiohttp/3.9.3” or similar hanging about
Yes, I see this just popped up for my site today. about 100 of them logged in as guests.
deninho32 wrote: Wed Jun 19, 2024 9:28 am Adding these 2 (today a new one already) to my .htacces file solved it.

BrowserMatchNoCase "python" bad_bot
BrowserMatchNoCase "CFNetwork" bad_bot

Order Deny,Allow
Deny from env=bad_bot

That Facebook bot is a nasty one. My error log is filled with this:

Code: Select all

[Wed Jun 19 11:28:29.414188 2024] [access_compat:error] [pid 10266:tid 140604168185600] [client 148.72.152.70:42540] AH01797: client denied by server configuration: /home/negentien13/domains/forum.negentiendertien.nl/public_html/viewtopic.php
This also worked for me as well.
User avatar
HiFiKabin
Community Team Member
Community Team Member
Posts: 6731
Joined: Wed May 14, 2014 9:10 am
Location: Swearing at the PC, UK
Name: James

Python/3.10 aiohttp/3.9.3

Post by HiFiKabin »

Over the past week or so my site has been hit by this bot with upwards of 100 on live at any one time, sometimes hitting the 200 mark.

All usual methods to block the having failed, I put two ip wildcards in my HTACCESS which stopped them dead in their tracks.

Code: Select all

Order Allow,Deny
Deny from 173.252.*.*
Deny from 69.171.*.*
Allow from all
I contacted my hosting company(Kualo) about this and they gave me a list of all relevant ips thay had, but it was not a complete list because as soon as I tried it they were back in again

They are also hitting phpbb.

Screenshots from my site FYI

screenshot_477.jpg

screenshot_478.jpg
You do not have the required permissions to view the files attached to this post.
User avatar
bonelifer
Community Team Member
Community Team Member
Posts: 3609
Joined: Wed Oct 27, 2004 11:35 pm
Name: William

Re: Python/3.10 aiohttp/3.9.3

Post by bonelifer »

You could just block based on aiohttp;
viewtopic.php?p=16018090#p16018090
William Jacoby - Community Team
Knowledge Base | phpBB Board Rules | Search Customisation Database
Please don't contact me via PM or email for phpBB support .

phpBB Modders is looking for developers! If you have phpBB experience and want to join us, click here!
User avatar
HiFiKabin
Community Team Member
Community Team Member
Posts: 6731
Joined: Wed May 14, 2014 9:10 am
Location: Swearing at the PC, UK
Name: James

Re: Python/3.10 aiohttp/3.9.3

Post by HiFiKabin »

... and they were right back in again. I have tried every trick I know (and some other I found on the web) and the wildcard is the only one thats worked for me
Forceflow
Registered User
Posts: 16
Joined: Mon Mar 24, 2008 8:38 pm

Re: Claudebot attack

Post by Forceflow »

Really glad to see I am not the only one suffering here. Bots are becoming more and more an issue. My provider locked my MySQL DB twice because of all the transactions the bots were causing. Not to mention that the webpages loaded very slow as well. As such I highly recommend locking them down on .htaccess level.
Even if they still keep trying for quite some time the load that this generates is very low.
In addition I had to block a big IP block as well:

Code: Select all

Deny from 47.76.0.0/16
Do look out for that one, it hammered my site generating over 150.000 hits (which was 80% of all hits to my page) and coming from seemingly descriptions. (But all nonsensical since it claimed to use really old chrome versions) It's from ALIBABA Cloud

This is my .htaccess:

Code: Select all

BrowserMatchNoCase "libwww-perl" bad_bot
BrowserMatchNoCase "wget" bad_bot
BrowserMatchNoCase "LieBaoFast" bad_bot
BrowserMatchNoCase "Mb2345Browser" bad_bot
BrowserMatchNoCase "zh-CN" bad_bot
BrowserMatchNoCase "MicroMessenger" bad_bot
BrowserMatchNoCase "zh_CN" bad_bot
BrowserMatchNoCase "Kinza" bad_bot
BrowserMatchNoCase "Bytespider" bad_bot
BrowserMatchNoCase "Baiduspider" bad_bot
BrowserMatchNoCase "Sogou" bad_bot
BrowserMatchNoCase "Datanyze" bad_bot
BrowserMatchNoCase "AspiegelBot" bad_bot
BrowserMatchNoCase "adscanner" bad_bot
BrowserMatchNoCase "serpstatbot" bad_bot
BrowserMatchNoCase "spaziodat" bad_bot
BrowserMatchNoCase "undefined" bad_bot
BrowserMatchNoCase "claudebot" bad_bot
BrowserMatchNoCase "facebook" bad_bot
BrowserMatchNoCase "Petalbot" bad_bot
BrowserMatchNoCase "YandexBot" bad_bot
BrowserMatchNoCase "Applebot" bad_bot
BrowserMatchNoCase "aiohttp" bad_bot
Order Deny,Allow
Deny from env=bad_bot
Deny from 47.76.0.0/16
The aiohttp (or python) thing only really turned up after blocking facebook, and the IPs being used from that one appear to all belong to facebook.

Just so annoying, it's not like I have a huge or highly frequented forum and they still manage to kill my page at times. (And that's even though the forum is pretty much empty unless you are an actually registered User)
User avatar
P_I
Community Team Member
Community Team Member
Posts: 2403
Joined: Tue Mar 01, 2011 8:35 pm
Location: Western Canada 🇨🇦

Re: Python/3.10 aiohttp/3.9.3

Post by P_I »

Using .htaccess with

Code: Select all

# 19-Jun-24, P_I, Yet another one that needs to be banished
BrowserMatchNoCase "aiohttp" bad_bot
Stopped them completely on my boards.
Normal people… believe that if it ain’t broke, don’t fix it. Engineers believe that if it ain’t broke, it doesn’t have enough features yet. – Scott Adams
User avatar
HiFiKabin
Community Team Member
Community Team Member
Posts: 6731
Joined: Wed May 14, 2014 9:10 am
Location: Swearing at the PC, UK
Name: James

Re: Python/3.10 aiohttp/3.9.3

Post by HiFiKabin »

That got the little buggers, and much better than my wildcard IP ban. Thanks

(although why I couldn't get it working before is beyond me. I blame the brain/finger interface)

Return to “phpBB Discussion”