size of a forum?

[jeffinnyc]

Hello all. My first post. I hope I am not violating any posting rules by posing the following question:

I am about to be asked to help a friend rescue an old instance of phpBB that is suddenly running very slowly. I don't know quite what I'm getting into but I am a computer professional so I *think* I'm going to discover that it's his ancient customizations that are finally giving up the ghost. (Or maybe his database is running out of room or who knows what....)

But first I have to orient myself and perhaps help my friend calm down a bit. Thus, this very basic question:

The old forum says of itself: Total posts 427341 • Total topics 56511 • Total members 3750

Is any of those numbers extraordinarily large? They don't sound it to me but I'm a beginner here.

Thanks for any guidance.

[thecoalman]

I would categorize that as medium sized forum.

• Check the users online, if you are seeing a lot of them it's likely being inundated with bad bots. This is a common problem recently and certainly would explain something that has suddenly started.
• Open phpMyAdmin in the hosting control panel and optimize the tables.
• Check to make sure the sessions table is not massive, it's typically 1000's of rows but shouldn't be hundreds of thousands of rows.
• Update to the latest version, you can take advantage of the increased performance of php 8.X

[jeffinnyc]

Thanks, thecoalman. I looked around and most things seem pretty normal.

However, another admin noticed two spiders (from AliBaba) chewing up the site and therefore causing the "users" and "sessions" tables to fill with locks. He slew the spiders and blocked their IPs and now the forum is behaving well again.

Now I know what to look for!

[thecoalman]

Try and avoid using phpBB's IP bans. This will consume more resources because the list has to be checked every page load. This is really only useful for temp bans on problematic users in the hopes they do not know about proxies etc.

If you are going to block by IP this can be somewhat useless for malicious bots as the IP's constantly change. If you are going to do this use .htaccess rules instead which will cut down on resources. Ideally use the firewall but you won't have access to that on shared hosting.

Bots that identify themselves are typically well behaved and can be blocked using robots.txt. They will load this file every so often.

Code: Select all

User-agent: Bingbot
Disallow: /

User-agent: Some_other_user_agent
Disallow: /

User-agent: *
Disallow: /path1/for/other/bots/
Disallow: /path2/for/other/bots/

Save and upload to the root of the domain, it's needs to be accessible at example.com/robots.txt. The first two rules would block Bingbot and "Some_other_user_agent". You would have to add those two lines for each useragent. The user agents can be found on the bots website.

The third set is wildcard for bots you want to index the site but limit what they index.

[romans1423]

If your site's been online for a while — and judging by its stats, it has been! — then it's no doubt become a target for bots looking for vulnerabilities. It happens to just about every site, and it entails bots attempting to access a variety of URLs that are known to be part of weaknesses in various software. They do this repeatedly, hitting a variety of URLs, in the hopes that something they do works to give them some sort of access to your site.

You can mitigate a lot of these bad actors by using a tool like Jeff Starr's 8G Firewall, a tool which amounts to a bunch of directives that you add to your site's .htaccess file. It's free, so check it out if you think it might help.

[jeffinnyc]

Thank you both. I am still trying to figure out how I can see the bots arrive on the site. The list of spiders/crawlers seems to only have information that I enter.

Yes, the other Admin did something with .htaccess, not within the ACP.

[thecoalman]

Thank you both. I am still trying to figure out how I can see the bots arrive on the site. The list of spiders/crawlers seems to only have information that I enter.

phpBB uses strings in the user agent to identify bots, Google for example. Malicious bots do not identify themselves and will use a browser user agent. The easiest way to identify them is to look at the Awstats which is going to be available through most hosting control panels. Scroll to the IP section and it will show you the activity by IP.

[romans1423]

An approach I’ve used before is to add something arbitrary to your robots.txt.

User-agent: *
Disallow: /blackhole.html

(If you already have a file named that for some reason, change the above line to some other file.)

Save and upload the file.

Good bots will obey the file. Bad bots will see the URL and visit it anyway. This results in a 404 error.

Check your server’s error logs and everything that hits the blocked file represents a bot you can block.

[P_I]

Thank you both. I am still trying to figure out how I can see the bots arrive on the site. The list of spiders/crawlers seems to only have information that I enter.

Take a look at "Who is online" and click the "Display guests" link. If you have the proper administration privileges that will allow you to see the user-agent strings of the guests and bots.

The following solution has been posted numerous times before but bears repeating for newcomers. Take a read through Blocking aggressive Chinese crawlers/scrapers/bots - John Large .co.uk for a solution that has served my boards quite well.

[thecoalman]

An approach I’ve used before is to add something arbitrary to your robots.txt.

It's called a honeypot and there is various forms of it. Ideally you would have a script that would add the IP to the firewall or .htaccess file blocking it but this list can grow quite quickly so you also need the means to purge it.

[jeffinnyc]

Thank you, folks. Special thank you to P_I -- I never knew I could click on Who Is Online -- so now I can see which bots are arriving, and catalog them in the Spider/Robots list so, at least, they'll use the Bots Group "Anonymous" user.

There are anywhere from 20-80 of them at any moment. The big players, of course, but also two SEO facilitators, an Android-only search engine, and an AI.

Plus a number of agents that are not hitting us with 15 topics simultaneously. How polite! How unusual!

And there's even one that has included a message... they give their URL and then: "please let us know of any problems;"

[JLA]

Hello all. My first post. I hope I am not violating any posting rules by posing the following question:

I am about to be asked to help a friend rescue an old instance of phpBB that is suddenly running very slowly. I don't know quite what I'm getting into but I am a computer professional so I *think* I'm going to discover that it's his ancient customizations that are finally giving up the ghost. (Or maybe his database is running out of room or who knows what....)

But first I have to orient myself and perhaps help my friend calm down a bit. Thus, this very basic question:

The old forum says of itself: Total posts 427341 • Total topics 56511 • Total members 3750

Is any of those numbers extraordinarily large? They don't sound it to me but I'm a beginner here.

Thanks for any guidance.

No, not large.

From our site (phpbb2)
JLA FORUMS Statistics We have a total of 644,603,924 posted articles with 240,244,656 attachments in our forums.
We have a total of 5,435,248 posted photos in our Photo Galleries.