CPU Usage abuse solution needed

Get help with installation and running phpBB 3.2.x here. Please do not post bug reports, feature requests, or extension related questions here.
Post Reply
semtal
Registered User
Posts: 213
Joined: Thu Aug 16, 2012 9:05 pm

CPU Usage abuse solution needed

Post by semtal »

Support Request Template
What version of phpBB are you using? phpBB 3.2.7
What is your board's URL? www.allscaletrek.com
Who do you host your board with? siteground
How did you install your board? I used the download package from phpBB.com
What is the most recent action performed on your board? Update from a previous version of phpBB3
Is registration required to reproduce this issue? No
Do you have any MODs installed? No
Do you have any extensions installed? Yes
What version of phpBB3 did you update from? phpBB 3.2.5
What extensions do you have installed? I don't know. The developer did it.
What styles do you currently have installed? pro silver
What language(s) is your board currently using? english
Which database type/version are you using? MySQL 5
What is your level of experience? New to PHP but not phpBB
What actions did you take (updating your board; installing a MOD, style or extension; etc.) prior to this problem becoming noticeable? The developer added a social media extension but I don't think it's related.
Please describe your problem. A massive amount of traffic is causing My CPU Usage and Account Executions to double - triple the allowed usage, causing a ''Usage Abuse'' alert from the host, which in turn causes them to shut down my site.

It was suggested that I use a CDN such as Cloudflare but that seemed to work only for a short period of time.

It was also suggested that I check ''logs'' to ban certain malicious IP's. However, I don't know how to do that and would need hired help or step-by-step help from the PHPBB support forum.

Please help!
Generated by SRT Generator
User avatar
janus_zonstraal
Registered User
Posts: 4795
Joined: Sat Aug 30, 2014 1:30 pm

Re: CPU Usage abuse solution needed

Post by janus_zonstraal »

Do you know where to find the access logs on your host pages?

(for the record, thius has nothing to do with phpbb, the solution is the same for every website.)
Sorry! My English is bat ;) !!!
semtal
Registered User
Posts: 213
Joined: Thu Aug 16, 2012 9:05 pm

Re: CPU Usage abuse solution needed

Post by semtal »

Sadly, I don't.
User avatar
david63
Registered User
Posts: 18293
Joined: Thu Dec 19, 2002 8:08 am
Location: Lancashire, UK
Contact:

Re: CPU Usage abuse solution needed

Post by david63 »

You should have an option in your hosting control panel for "Logs" - if you cannot find it you will need to ask your hosts where to find the logs.
David
Remember: You only know what you know and - you don't know what you don't know!
My CDB Contributions | How to install an extension
I will not be accepting translations for any of my extensions in Github - please post any translations in the appropriate topic.
No support requests via PM or email as they will be ignored
User avatar
warmweer
Jr. Extension Validator
Posts: 5325
Joined: Fri Jul 04, 2003 6:34 am
Location: Van Allen Bel ... gium

Re: CPU Usage abuse solution needed

Post by warmweer »

semtal wrote:
Sat Aug 03, 2019 5:58 am
What extensions do you have installed? I don't know. The developer did it.
Surely you have access to the board and can see which extensions are installed/enabled/uptodate?
Spelling is freeware, which means you can use it for free.
On the other hand, it is not open source, which means you cannot change it or publish it in a modified form.
semtal
Registered User
Posts: 213
Joined: Thu Aug 16, 2012 9:05 pm

Re: CPU Usage abuse solution needed

Post by semtal »

The host suggested I add a robots.txt file and ''Use a set of .htaccess rules that will prevent malicious requests towards the website''. Is this needed or does the PHPBB software have something to counter this problem?
david63 wrote:
Sat Aug 03, 2019 8:00 am
They directed me to Raw Access Logs, but I don't know how to search for malicious IP's.
warmweer wrote:
Sat Aug 03, 2019 9:01 am
semtal wrote:
Sat Aug 03, 2019 5:58 am
What extensions do you have installed? I don't know. The developer did it.
Surely you have access to the board and can see which extensions are installed/enabled/uptodate?
I just checked. OneAll Social Login is the only one listed under extensions.
User avatar
david63
Registered User
Posts: 18293
Joined: Thu Dec 19, 2002 8:08 am
Location: Lancashire, UK
Contact:

Re: CPU Usage abuse solution needed

Post by david63 »

semtal wrote:
Sat Aug 03, 2019 10:28 am
The host suggested I add a robots.txt file and ''Use a set of .htaccess rules that will prevent malicious requests towards the website''.
I would suggest that you ask your hosts how that will help! Using either of those methods means that the malicious "users" will still have to access your board thereby using your CPU allowance. The only way to stop them is to stop them before they reach your board either, as your hosts suggests, using the likes of Cloudflare or using a firewall. Be aware though that if you use Cloudflare and it is not configured correctly you may encounter other problems with your board.
semtal wrote:
Sat Aug 03, 2019 10:28 am
They directed me to Raw Access Logs, but I don't know how to search for malicious IP's.
You can usually spot malicious IPs by their frequency - there will be dozens, if not hundreds, of log entries with the same IP address.
David
Remember: You only know what you know and - you don't know what you don't know!
My CDB Contributions | How to install an extension
I will not be accepting translations for any of my extensions in Github - please post any translations in the appropriate topic.
No support requests via PM or email as they will be ignored
User avatar
thecoalman
Community Team Member
Community Team Member
Posts: 4050
Joined: Wed Dec 22, 2004 3:52 am
Location: Pennsylvania, U.S.A.
Contact:

Re: CPU Usage abuse solution needed

Post by thecoalman »

Look in your control panel for Awstats or some other statistical program. It will give you better idea of where the traffic is coming from, that information is gathered from the access logs and processed so it's more understandable.

semtal wrote:
Sat Aug 03, 2019 10:28 am
The host suggested I add a robots.txt file
This is a start but it will only prevent legitimate bots like Google from loading pages. If you can care less about having your site found in search engines you can simply do this. Google and any other search engine that respects this file will not load any pages from your site. Just so it's perfectly clear if you use the following in robots.txt file your site is going to drop out of any search engine index.

Code: Select all

User-agent: *
Disallow: / 
If you want it indexed this will prevent them from loading unnecessary pages and files.

Code: Select all

User-agent: *
Disallow: /adm/
Disallow: /bin/
Disallow: /cache/
Disallow: /config/
Disallow: /files/
Disallow: /includes/
Disallow: /language/
Disallow: /phpbb/
Disallow: /store/
Disallow: /vendor/
Disallow: /common.php
Disallow: /composer.json
Disallow: /composer.lock
Disallow: /config.php
Disallow: /cron.php
Disallow: /faq.php
Disallow: /mcp.php
Disallow: /memberlist.php
Disallow: /posting.php
Disallow: /report.php
Disallow: /search.php
Disallow: /ucp.php
Disallow: /viewonline.php
# Blocks the first page of specific forums you do not want indexed, in this case the forums with the ID 1,2 and 3
Disallow: /viewforum.php?f=1$
Disallow: /viewforum.php?f=2$
Disallow: /viewforum.php?f=1$
# Blocks goth viewtopic and viewforum for pages with parameters for forums and topics in them you do not want indexed.  
Disallow: /*.php?f=1&*
Disallow: /*.php?f=2&*
Disallow: /*.php?f=3&*

Some explanations:

Code: Select all

# Blocks the first page of specific forums you do not want indexed
Disallow: /viewforum.php?f=12$
The $ tells the bot this is the end of the URL, it's not standard but the major bots respect it. You cannot use this:

Code: Select all

Disallow: /coal-forum/viewforum.php?f=1
That will block any forum ID starting with 1 like /viewforum.php?f=12

Code: Select all

# Blocks goth viewtopic and viewforum you do not want indexed.  
Disallow: /*.php?f=1&*
This covers the bulk of requests for forums and topics in them. The one issue is it will not block requests for topic URL's that do not include the f parameter.

and ''Use a set of .htaccess rules that will prevent malicious requests towards the website''.


That's not very helpful, if you have single bot running amok it might work... Cloudlfare allows you to create page rules and if they blocked at Cloudlfare it never reaches the server.. Check your logs for requests to unnecessary pages, for example wp-login is quite popular with malicious bots. Block them using a rule on Cloudlflare.

Is this needed or does the PHPBB software have something to counter this problem?
Using phpBB's bot permissions to block them from forums you do not want indexed can be helpful but they are still going to make requests for the page. The resources consumed are lower but it's still consuming resources, it's best to block them robots.txt since they won't load any pages.
“Results! Why, man, I have gotten a lot of results! I have found several thousand things that won’t work.”

Attributed - Thomas Edison
User avatar
thecoalman
Community Team Member
Community Team Member
Posts: 4050
Joined: Wed Dec 22, 2004 3:52 am
Location: Pennsylvania, U.S.A.
Contact:

Re: CPU Usage abuse solution needed

Post by thecoalman »

david63 wrote:
Sat Aug 03, 2019 10:45 am
Be aware though that if you use Cloudflare and it is not configured correctly you may encounter other problems with your board.
Out of the box it should not cause issues however Cloudflare may actually have no benefit in this situation. By default hey do not cache php files so impact on reducing traffic is going to be minimal. They are only going to cache image files, CSS and javascript.

The other issue is all the traffic coming to your site is going to have Cloudflare IP. They pass a special header for the original IP and there is an extension for phpBB for this however this only works for phpBB.

If you click the link "Who is online" and the IP's are all from Cloudflare then this is a problem. Log files such as your access logs and anything else will show only Cloudflare IP's which of course will make it impossible to determine where the traffic is really coming from. You need to have your host install mod_cloudflare so the orginal IP is passed to applications, logs etc. You cannot install this yourself on shared host.
“Results! Why, man, I have gotten a lot of results! I have found several thousand things that won’t work.”

Attributed - Thomas Edison
semtal
Registered User
Posts: 213
Joined: Thu Aug 16, 2012 9:05 pm

Re: CPU Usage abuse solution needed

Post by semtal »

thecoalman wrote:
Sat Aug 03, 2019 11:34 am
Look in your control panel for Awstats or some other statistical program. It will give you better idea of where the traffic is coming from, that information is gathered from the access logs and processed so it's more understandable.
I had a look at AWStats > Robots Spiders visitors and saw this one but no way of finding the IP. Do you know how I can get this?
Unknown robot (identified by 'bot' followed by a space or one of the following characters _+:,.;/\-)

Hits 240,436
Bandwidth 2.08 GB
Last visit 03 Aug 2019 - 13:10
User avatar
stevemaury
Support Team Member
Support Team Member
Posts: 51529
Joined: Thu Nov 02, 2006 12:21 am
Location: The U.P.
Name: Steve
Contact:

Re: CPU Usage abuse solution needed

Post by stevemaury »

Maybe try disabling that extension for awhile and see what happens?
For REALLY good and VERY inexpensive hosting CLICK HERE

I can stop all your spam. I can upgrade or update your Board. PM or email me. (Paid support)
semtal
Registered User
Posts: 213
Joined: Thu Aug 16, 2012 9:05 pm

Re: CPU Usage abuse solution needed

Post by semtal »

thecoalman wrote:
Sat Aug 03, 2019 11:34 am
semtal wrote:
Sat Aug 03, 2019 10:28 am
The host suggested I add a robots.txt file
This is a start but it will only prevent legitimate bots like Google from loading pages. If you can care less about having your site found in search engines you can simply do this. Google and any other search engine that respects this file will not load any pages from your site. Just so it's perfectly clear if you use the following in robots.txt file your site is going to drop out of any search engine index.

Code: Select all

User-agent: *
Disallow: / 
i added the txt file as suggested above, just to see if it would help. it took every bot out except the bing bot. i have banned bing bot ip's via the forum acp and also the host cp, but this bot still finds a way to wreck havoc. below is from the last 24hrs.

do you have any ideas how to get rid of this one?

Code: Select all

Hits IP Address UserAgent 
----- ------------- ----------------------------------------------------------------------- 
11402 207.46.13.58 Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) 
7574 207.46.13.56 Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) 
7211 157.55.39.58 Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) 
7014 207.46.13.103 Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) 
6866 207.46.13.117 Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) 
5541 157.55.39.177 Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) 
4769 207.46.13.118 Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) 
4382 40.77.167.104 Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) 
4368 40.77.167.166 Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm) 
4309 40.77.167.155 Mozilla/5.0 (compatible; bingbot/2.0; +http://www.bing.com/bingbot.htm)
semtal
Registered User
Posts: 213
Joined: Thu Aug 16, 2012 9:05 pm

Re: CPU Usage abuse solution needed

Post by semtal »

stevemaury wrote:
Sat Aug 03, 2019 5:55 pm
Maybe try disabling that extension for awhile and see what happens?
the extension was added but not activated.
User avatar
thecoalman
Community Team Member
Community Team Member
Posts: 4050
Joined: Wed Dec 22, 2004 3:52 am
Location: Pennsylvania, U.S.A.
Contact:

Re: CPU Usage abuse solution needed

Post by thecoalman »

semtal wrote:
Tue Sep 03, 2019 11:14 am
do you have any ideas how to get rid of this one?
They load that file occasionally, search your logs for robots.txt. Once Bing loads the file thay should stop crawling.

Once again just so it's perfectly clear, you are blocking all legitimate bots with that directive. Google, Bing and other search engines are going to drop your pages from their index and people will not be able to find them when searching.
“Results! Why, man, I have gotten a lot of results! I have found several thousand things that won’t work.”

Attributed - Thomas Edison
semtal
Registered User
Posts: 213
Joined: Thu Aug 16, 2012 9:05 pm

Re: CPU Usage abuse solution needed

Post by semtal »

thecoalman wrote:
Tue Sep 03, 2019 12:05 pm
semtal wrote:
Tue Sep 03, 2019 11:14 am
do you have any ideas how to get rid of this one?
They load that file occasionally, search your logs for robots.txt. Once Bing loads the file thay should stop crawling.

Once again just so it's perfectly clear, you are blocking all legitimate bots with that directive. Google, Bing and other search engines are going to drop your pages from their index and people will not be able to find them when searching.
i wanted to see if it worked first. which it did, with the exception of this bingbot 2.0.

the site is still listed on google. it is just devoid of a description and states the reason.

how would i allow google but no-one else?
Post Reply

Return to “[3.2.x] Support Forum”