Excessive forum load time

[sopas]

Hello,

For the last couple of months my forum has been running very slow due to a huge increase of "visitors". There are always over 1,000 users online permanently of which almost all are "guests" (only about a dozen are "registered" or "hidden" users). This of course is affecting the loading time of every page which may take as much as 15 seconds now.

When I click on "Who is online" and then "Display guests" I notice that many of those "guests" have IPs from Brazil.

I also went to my server's Control Panel to check "Awstats" and confirmed that my forum is currently getting most of the traffic from Brazil. This is odd since 95% of my visitors used to be from the USA, UK, and some EU countries, but no Brazil at all. In fact, there are no real users from Brazil registered and posting in my forum.

This is how my Awstats look like now:

country-stats01.jpg

... and this is how it used to be only a coulple of months ago:

country-stats02.jpg

So, having identified the problem, I was wondering if someone could tell me the best way to deal with this. How can I block all traffic originating from Brazil from reaching my forum? I would not mind doing that if that fixes the unaceptable loading times of my forum.

Thank you for your time.

[sopas]

By the way, my robots.txt file looks like this:

Code: Select all

User-agent: *
Disallow: /

User-agent: Bingbot
Disallow:

User-agent: Googlebot
Disallow:

Set to allow only Google and Bing bots, and blocking all the rest.
It does not help with traffic originating from Brazil though, so I guess those are "bad bots" who ignore the robots.txt file maybe?

[thecoalman]

No idea what the results are for blank disallow but use allow instead.

https://developers.google.com/search/do ... robots-txt

There is numerous topics over the past few years about the surge in traffic from bots, just do a search. They aren't identifying themselves as bots and don't obey robots.txt. There is different solutions but Cloudflare is the most robust one.

[SQLnovice]

If your host is using Apache, you can try enabling country-code blocking in htaccess. This one works in the positive, matching countries, then Failing all matches. Remove all the two character country codes you don't want blocked. BR is Brazil.

Code: Select all

GeoIPEnable On
<IfModule mod_geoip.c>
  GeoIPEnable On
</IfModule>
<IfModule mod_geoip.c>
RewriteEngine on
RewriteCond %{ENV:GEOIP_COUNTRY_CODE} ^(AE|AF|AG|AL|AM|AO|AR|AS|AT|AZ|BA|BB|BD|BH|BJ|BN|BO|BR|BS|BT|BW|BY|BZ|CD|CF|CG|CI|CK|CI|CM|CN|CO|CU|CV|CW|CX|CZ|DJ|DM|DO|DZ|EC|EE|EG|EH|ER|ET|FJ|FM|FR|GA|GD|GE|GF|GH|GI|GL|GM|GN|GP|GQ|GS|GT|GU|GW|GY|HK|HN|HT|ID|IL|IN|IO|IQ|IR|JM|JO|KE|KG|KH|KI|KM|KN|KP|KR|KW|KY|KZ|LA|LB|LC|LI|LK|LR|LT|LU|LV|LY|MA|MC|MD|ME|MG|MN|MK|ML|MM|MN|MO|MP|MQ|MR|MS|MT|MU|MV|MW|MX|MY|MZ|NA|NC|NE|NG|NI|NP|NR|NU|OM|PA|PE|PF|PG|PH|PK|PL|PM|PN|PR|PS|PW|PY|QA|RE|RO|RS|RU|RW|SA|SB|SD|SG|SI|SK|SL|SM|SN|SO|SR|SS|ST|SU|SV|SX|SY|SZ|TC|TD|TF|TG|TH|TJ|TK|TL|TM|TN|TO|TR|TT|TV|TW|TZ|UA|UG|UK|US|UY|UZ|VA|VC|VE|VG|VN|VU|WF|WS|YE|YT|ZM|ZW)$
RewriteRule ^(.*)$ - [F]
</IfModule>

We use these in conjunction with the above mentioned BOT blocks and IP range blocks, for granularity.

It can also be tailored to work in the negative, allowing only those counties you specify, say if that's a much shorter list and thus blocking all those that aren't listed like a wildcard. You may have to search for what the character(s) are for that, but it might be the inclusion of the ! character, which I think is equal to NOT; meaning NOT Fail those country codes listed. We have a board that uses that method, so different boards have different needs.

Code: Select all

<IfModule mod_geoip.c>
  GeoIPEnable On
</IfModule>
<IfModule mod_geoip.c>
RewriteEngine on
RewriteCond %{ENV:GEOIP_COUNTRY_CODE} ! ^(UK|US)$
RewriteRule ^(.*)$ - [F]
</IfModule>

[Mick]

This may help viewtopic.php?p=16005985#p16005985