Yesterday evening, I started having a problem with our server. Which is, in normal circumstances (even to 10K pageviews per hour) working like a charm.
There was extreme load on the server, about 30 on average. I checked the server's CPU, that was constantly running up to the 100%
Checking my forum stats, I noticed that yesterday we had beaten the most users ever online record: 5023 on 21 Aug 2019 15:56
I continued to look into the stats and found one particular crawler called BLEXBOT was causing the high load, opening more than 800 sessions.
Blocking the crawler from robots.txt didn't suffice as on another site I discovered that the crawler doesn't listen to the order. I contacted the company (webmeup) via mail and Facebook without a reply.
I decided to block a series of IP-addresses from the forums, adding this to the IP block list: 159.138.*.*
Server load and CPU has significantly dropped after the measure.