Online zombies

[siân]

There are 661 guest users online

I swear, there are only a few real visitors in Google Analytics.

Also, there are no spiders except Google, Bing, and Amazon.

Code: Select all

User-agent: SemrushBot
Disallow: /

User-agent: AhrefsBot
Disallow: /

User-agent: MJ12bot
Disallow: /

User-agent: Barkrowler
Disallow: /

User-agent: Bytespider
Disallow: /

User-agent: DataForSeoBot
Disallow: /

User-agent: DotBot
Disallow: /

User-agent: YandexBot
Disallow: /

User-agent: Sogou Spider
Disallow: /

User-agent: SeznamBot
Disallow: /

User-agent: MauiBot
Disallow: /

User-agent: Baiduspider
Disallow: /

User-agent: CCBot
Disallow: /

So who are these zombies?

[KevC]

You can google all of the names and it'll tell you what they are. Yandex for example is like Russian google. Seznam is Czech etc etc.

[siân]

I said all of these are banned, only Google, Bing, Amazon are allowed.

[lurttinen]

Every unique IP address counts as one user.
Online counter counts users (IPs) over specified time. Usually 5 minutes, so you had 661 IPs in 5 minutes. (Or hour, or day, or.. however you set it)

If you have banned only those 13 bots, you are missing few thousand.
And then few thousand more that don't obey your rules even if you tell them no.

Goto users online list and click "Show visitors"
You should see many many bot agents.
Then goto ACP and add those agents as bots, so they count as one user and is shown.

[warmweer]

I said all of these are banned, only Google, Bing, Amazon are allowed.

Did you? I can't find where you mentioned that.

[siân]

Also, there are no spiders except Google, Bing, and Amazon.

[skybound]

I got a bunch of these visitors:

Guest
python-requests/2.29.0

They are all listed as 'downloading file' activity

No common IP range. Come from 18.xx 55.xx 13.xx etc

Also numbering about 500+ in a 10 minute window.

Any idea what they are and how to chase them away?

[Steve]

I got a bunch of these visitors:

Guest
python-requests/2.29.0

They are all listed as 'downloading file' activity

No common IP range. Come from 18.xx 55.xx 13.xx etc

Also numbering about 500+ in a 10 minute window.

Any idea what they are and how to chase them away?

Just create a new bot using the agent match.

acp > quick access > spiders/robots

add new bot:

Spiders-Robots.png

[skybound]

Thanks Steve.

[skybound]

Still left with this. Have no real pattern and cant figure if it is real traffic: About another 500 visitors.

[Steve]

Still left with this. Have no real pattern and cant figure if it is real traffic: About another 500 visitors.

Spend some time monitoring the online page, tbh most of them guests look normal to me...

you may need your own topic also.

[SQLnovice]

For us, what you're experiencing really took off in the second week of March. We extended the Who Is Online time period to better see who is attacking us. The attacks come in waves and for us are simply a DDoS attack with wide reaching ramifications. As we close the doors they're coming through, we end up potentially blocking legitimate site traffic. This is why CloudFlare is a big hit with most on here, if you can figure out how to make it work for you. I have not.

Short of that, you might want to sit down to determine what countries your Web site services. Then use .htaccess, assuming your host is or supports Apache, to block traffic from countries you want to sever ties with permanently. Use this code structure to block in the positive (below), where these ARE the countries you wish to block. You can also ALLOW countries in the reverse way, if say you only want a few countries in and for .htaccess to block every thing else no matter what new country it may be. The code is different, much more simple, for that. Just Google how to alter it correctly, so as not to Fail those country code connections. Each country code is separated by the | pipe symbol. This rewrite Fails all connections that match its criteria.

Code: Select all

GeoIPEnable On
<IfModule mod_geoip.c>
  GeoIPEnable On
</IfModule>
<IfModule mod_geoip.c>
RewriteEngine on
RewriteCond %{ENV:GEOIP_COUNTRY_CODE} ^(AE|AF|AG|AL|AM|AO|AR|AS|AT|AZ|BA|BB|BD|BH|BJ|BN|BO|BR|BS|BT|BW|BY|BZ|CD|CF|CG|CI|CK|CI|CM|CN|CO|CR|CU|CV|CW|CX|CZ...WF|WS|YE|YT|ZM|ZW)$
RewriteRule ^(.*)$ - [F]
</IfModule>

Check your host logs for the most prevalent countries hitting your site. Usually there are filters you can use on your host to analyze traffic by "time."