Bytespider Attack

[CGI1984]

Yesterday we had a small attack from a user agent called Bytespider. I say it was an attack because there were nearly 30 users online at the same time from this same spider at once and all were using very different IP addresses. And all were trying to post something at the same time in various different forums, but none of them were logged in. So at the very least they were consuming site bandwidth and resources and bogging things down.

In the user agent info for this agent it also said https://zhanzhang.toutiao.com for each of its users. So we created a Spider/Robot entry for it in ACP and deactivated the Bytespider user agent and it stopped the attack. I doubt it will be a problem that we deactivated it as I don't imagine any legitimate user agents will be using this same bot name.

Also, not sure what it was trying to do, but it seemed malicious and based on the domain name the source likely traces back to China. Anyway, just mentioning it in case anyone notices a group of these users on their site at once. The following post titled “Blocking aggressive Chinese crawlers/scrapers/bots" also has a number of user comments where they reference the Bytespider user agent: https://www.johnlarge.co.uk/blocking-ag ... pers-bots/

[P_I]

The following post titled “Blocking aggressive Chinese crawlers/scrapers/bots" also has a number of user comments where they reference the Bytespider user agent: https://www.johnlarge.co.uk/blocking-ag ... pers-bots/

My boards got hit pretty hard in late November, early December but the aggressive bots mentioned in the John Large article. Implementing his suggestions completely resolved the problem for us and resource use and traffic returned back to normal.

I've posted this a number of times that I highly recommend anyone seeing this traffic to implement his solution. For those who might not have followed along, there is an updated version from https://www.johnlarge.co.uk/blocking-ag ... ment-11823.

try adding all of this code to the very end of you .htaccess file

Code: Select all

BrowserMatchNoCase "libwww-perl" bad_bot
BrowserMatchNoCase "wget" bad_bot
BrowserMatchNoCase "LieBaoFast" bad_bot
BrowserMatchNoCase "Mb2345Browser" bad_bot
BrowserMatchNoCase "zh-CN" bad_bot
BrowserMatchNoCase "MicroMessenger" bad_bot
BrowserMatchNoCase "zh_CN" bad_bot
BrowserMatchNoCase "Kinza" bad_bot
BrowserMatchNoCase "Bytespider" bad_bot
BrowserMatchNoCase "Baiduspider" bad_bot
BrowserMatchNoCase "Sogou" bad_bot
BrowserMatchNoCase "Datanyze" bad_bot
BrowserMatchNoCase "AspiegelBot" bad_bot
BrowserMatchNoCase "adscanner" bad_bot
BrowserMatchNoCase "serpstatbot" bad_bot
BrowserMatchNoCase "spaziodat" bad_bot
BrowserMatchNoCase "undefined" bad_bot
Order Deny,Allow
Deny from env=bad_bot

I’m looking for the most efficient way of blocking without sacrificing page load speeds & TTFB. This way (so far) seems a little more efficient. If you need to add a line for another user agent it should be self-explanatory – just pick a section of text from the user agent which appears to be unique to that particular bot and add that text to another line in the same format.

[CGI1984]

Thanks for that. Great! I will indeed do what you suggested by using .htaccess to block it as well as the others on the list. I had created a Robots/Spiders entry for it in ACP and deactivated it thinking that it would block this bot, but it didn't. The gunk came back today in full force.

Only question is as follows on implementation of this code in .htaccess: We have a few IP addresses we are already blocking in our .htaccess file so when we add this code to block these bots should it be option 1 or option 2 below? Or something different to make sure the added code to block these bots works correctly in conjunction with the existing IP blocking? Thanks.

Option 1:

Code: Select all

<Limit GET HEAD POST>
order allow,deny
allow from all
deny from 1.2.3.4
</Limit>

BrowserMatchNoCase "libwww-perl" bad_bot
BrowserMatchNoCase "wget" bad_bot
BrowserMatchNoCase "LieBaoFast" bad_bot
BrowserMatchNoCase "Mb2345Browser" bad_bot
BrowserMatchNoCase "zh-CN" bad_bot
BrowserMatchNoCase "MicroMessenger" bad_bot
BrowserMatchNoCase "zh_CN" bad_bot
BrowserMatchNoCase "Kinza" bad_bot
BrowserMatchNoCase "Bytespider" bad_bot
BrowserMatchNoCase "Baiduspider" bad_bot
BrowserMatchNoCase "Sogou" bad_bot
BrowserMatchNoCase "Datanyze" bad_bot
BrowserMatchNoCase "AspiegelBot" bad_bot
BrowserMatchNoCase "adscanner" bad_bot
BrowserMatchNoCase "serpstatbot" bad_bot
BrowserMatchNoCase "spaziodat" bad_bot
BrowserMatchNoCase "undefined" bad_bot
Order Deny,Allow
Deny from env=bad_bot

Option 2:

Code: Select all

<Limit GET HEAD POST>
allow from all
deny from 1.2.3.4
BrowserMatchNoCase "libwww-perl" bad_bot
BrowserMatchNoCase "wget" bad_bot
BrowserMatchNoCase "LieBaoFast" bad_bot
BrowserMatchNoCase "Mb2345Browser" bad_bot
BrowserMatchNoCase "zh-CN" bad_bot
BrowserMatchNoCase "MicroMessenger" bad_bot
BrowserMatchNoCase "zh_CN" bad_bot
BrowserMatchNoCase "Kinza" bad_bot
BrowserMatchNoCase "Bytespider" bad_bot
BrowserMatchNoCase "Baiduspider" bad_bot
BrowserMatchNoCase "Sogou" bad_bot
BrowserMatchNoCase "Datanyze" bad_bot
BrowserMatchNoCase "AspiegelBot" bad_bot
BrowserMatchNoCase "adscanner" bad_bot
BrowserMatchNoCase "serpstatbot" bad_bot
BrowserMatchNoCase "spaziodat" bad_bot
BrowserMatchNoCase "undefined" bad_bot
Order Deny,Allow
Deny from env=bad_bot
</Limit>

[CGI1984]

I just decided to add it to the .htaccess file as follows in conjunction with the IP blocks we have. So far so good and no errors loading the site as far as I can see:

Code: Select all

BrowserMatchNoCase "libwww-perl" bad_bot
BrowserMatchNoCase "wget" bad_bot
BrowserMatchNoCase "LieBaoFast" bad_bot
BrowserMatchNoCase "Mb2345Browser" bad_bot
BrowserMatchNoCase "zh-CN" bad_bot
BrowserMatchNoCase "MicroMessenger" bad_bot
BrowserMatchNoCase "zh_CN" bad_bot
BrowserMatchNoCase "Kinza" bad_bot
BrowserMatchNoCase "Bytespider" bad_bot
BrowserMatchNoCase "Baiduspider" bad_bot
BrowserMatchNoCase "Sogou" bad_bot
BrowserMatchNoCase "Datanyze" bad_bot
BrowserMatchNoCase "AspiegelBot" bad_bot
BrowserMatchNoCase "adscanner" bad_bot
BrowserMatchNoCase "serpstatbot" bad_bot
BrowserMatchNoCase "spaziodat" bad_bot
BrowserMatchNoCase "undefined" bad_bot
Order Deny,Allow
Deny from env=bad_bot

<Limit GET HEAD POST>
order allow,deny
allow from all
deny from 1.2.3.4
</Limit>

[Willoughby_wx]

Perfect, the solutions here really helped me.

What a crazy bot, changing IPs every minute! They appeared about a month ago and I struggled to get rid of it.

Bytespider appears to be from Bytedance, which is Tiktok.

[Dog Cow]

Even WordPress recommends blocking this.

[mrix2000]

So we created a Spider/Robot entry for it in ACP and deactivated the Bytespider user agent and it stopped the attack. I doubt it will be a problem that we deactivated it as I don't imagine any legitimate user agents will be using this same bot name.

Hello I am getting the same problem, can you please let me know what you added in the ACP which will hopefully also resolve my issue too.
Thanks for any help

[foxiedog]

i have also been getting hit by byetspider bots at the rate of thousands per day,
i tried the above code in htaccess, but they were still getting through in large numbers,
i also tried deny from ip blocks from singapore, which was also ineffective in blocking them,
(they appeared to be mainly from singapore via amazonaws.com)

after much searching on the subject i came across this bit of code,

Code: Select all

# DENY ACCESS TO amazonaws.com

# Apache 2.2
Order Allow,Deny
Allow from all
Deny from amazonaws.com

# Apache 2.4+
<RequireAll>
    Require all granted
    Require not host amazonaws.com
</RequireAll>

which so far seems to have done the trick in blocking bytespider. perhaps a little drastic i know, as it blocks all amazonaws traffic from the forum,
but visits from bytespider have dropped to zero now

[Mick]

You can also report them to Amazon AWS, they take action pretty quickly I’m led to believe.

See: viewtopic.php?t=2642376

[-steves-]

I just decided to add it to the .htaccess file as follows in conjunction with the IP blocks we have. So far so good and no errors loading the site as far as

<snip>

This worked for me, thank you

[JLA]

You can also report them to Amazon AWS, they take action pretty quickly I’m led to believe.

See: viewtopic.php?t=2642376

They will not do a thing

[Mick]

Reported it to my hosting company but they didn't know much more that I did. I added entries to robots.txt and also used the robots/spiders entries in phpbb as well as blocking some of the common IP ranges that it was showing. But even then I was often seeing 6 or 7 anonymous entries in "Who's online".

So reported possible abuse to AmazonAWS and slowly the traffic dropped off and now I'm not seeing any (but then I don't spend all my time monitoring who is online!).

I'm assuming they had a runaway web crawler or someone was abusing their services ..