[SPLIT] Python/3.10 aiohttp/3.9.3 anyone?

[deninho32]

Another one with 20+ bots.

MediaMathbot

Added to htaccess fixed it.

[invenio]

Sorry if this is a dumb question as I am not that tech savvy, but it seems like the problem is that these bots create dozens of sessions at once. My site was brought to crawl before blocking via the .htaccess file. But the issue wasn't that there was bad bot crawling the site, but rather that there were 150 sessions from that one bot. Is there anything that can restrict each unique individual user/bot to one session max? Wouldn't that solve the problem? Even if the bot was bad, if we could auto-limit it to 1 session max, and say 1 query every 2 seconds,... it would essentially eliminate the server load. No human user would need to make a query more often than a second or two max.

Again, sorry if this is a stupid question.

[Mick]

Maybe it could be done with an extension but I’m of the belief our hosts should be getting more involved, stopping them before they get to the board would be a better idea.

[Talk19Zehn]

In accordance with that notice, I filed a complaint today:

Why does Facebook appear in my server log?

Facebook allows its users to send links to interesting web content to other Facebook users. Part of how this works on the Facebook system involves the temporary display of certain images or details related to the web content, such as the title of the webpage or the embed tag of a video. Our system retrieves this information only after a user provides us with a link. You may have found this page because a Facebook user sent a link from your website to other Facebook users. If you have any questions or concerns about any links or content sent by one of our users, please contact us at [email protected].

https://www.facebook.com/externalhit_uatext.php

Especially since I don't have and have never had a FB account and I don't have any third-party requests for permission! Furthermore, I am being crawled for pages that have not existed since 2017. This would lead to the conclusion that website owners do not maintain their software, including phpBB forums, or that they are outdated and haunting the WWW.

Regards

Edit:
The battle for the metadata etc. is on. This is according to a number of reliable sources.