Page 1 of 24

[3.3][RC] Filter by country - version 1.0.16

Posted: Tue May 28, 2019 5:20 pm
by MarkDHamill
This extension allows an administrator to filter traffic by country, allowing in only traffic from selected countries, or disallowing traffic only from selected countries. So if you get a lot of spam traffic from countries like Ukraine, Russia and Belarus, hopefully this will get rid of it. Or if you want to keep all traffic out except from your country, or a defined set of countries, it will do that too. If traffic is prohibited, they will see a message to this effect, but all other links won't work and the page will redirect back to the same message.

The free MaxMind GeoLite2 Country database is used, so it depends on the integrity of its data. The database is downloaded to the /store/phpbbservices/filterbycountry folder. The extension optionally tracks the number of page requests by country that are allowed and restricted. It is all controlled from two modules in the Extensions tab in the ACP.

Image

Other features:
  • You can allow users to login from restricted countries, if you enable the feature. Otherwise guest and bots see no forum content. Only the login link works.
  • Can filter out bots in phpBB's database from statistics.
  • Can log all restricted attempts in the admin log, if enabled. This can result in a very large log!
  • A cron automatically updates the database weekly
  • A test mode is now supported, but use with caution as legitimate traffic may get blocked.
The report is straightforward and allows you to sort by column. You can also filter the report by both relative and absolute time ranges. Only countries that requested pages are shown for brevity. Data is stored to a table, so for a highly trafficked board this table could potentially get huge. If you enable this feature, make sure you have plenty of database space. A lot of shared hosts will limit database space.

Image

To keep your board from locking up, if you don't choose any countries to allow or restrict, all traffic is allowed. So the first thing to do is set the extension's settings correctly.

A standard block message looks like this. Clicking on any link simply returns the same page.

Image

If you want to allow registered users to login from blocked countries, the block message looks like this:

Image

The extension can be downloaded here or from the GitHub 1.0.15 release branch. If downloaded from GitHub, make sure to place the files in /ext/phpbbservices/filterbycountry.

The GitHub for the project is here:

https://github.com/MarkDHamill/filterbycountry

Re: [DEV] Filter by country

Posted: Tue May 28, 2019 9:19 pm
by dmzx
download link does not work, but nice idea i will try it out.

edit:
Oops! That page can’t be found.

Re: [DEV] Filter by country

Posted: Tue May 28, 2019 9:22 pm
by fagbutlil
Working fine on 3.2.7 with php7.2.17 about time some thing like this come out as they say bye bye Russia

Re: [DEV] Filter by country

Posted: Tue May 28, 2019 9:35 pm
by MarkDHamill
The download link is fixed.

There are other ways to block by country. Services like CloudFlare can do this and arguably can do a better job by keeping traffic from ever hitting your site. This consumes a little CPU but probably not too much plus it optionally allows registered users to login from a restricted area while keeping guests and bots out. The standard forum headers and footers are shown, but just a message saying access is blocked. All the links return the same message. The exception is if login is enabled. Registration is not enabled.

Testing has been limited. I suspect that VPNs may trip this up. If the IP is not in the database though there is the option to allow logins anyhow. MaxMind does have an Anonymous IP database. Unfortunately, this product is not free. It's unclear how much it costs but I may integrate it if there is demand and the phpBB does not object because it's not free. They allow certain other extensions, like Cleantalk, that is run as a paid service.

Re: [DEV] Filter by country

Posted: Tue May 28, 2019 10:18 pm
by fagbutlil
I used a vpn to get around it but i guess most of them china and Viagra sellers wont use vpn.

Re: [DEV] Filter by country

Posted: Wed May 29, 2019 7:22 am
by david63
How do you intend maintaining the database Mark? For me, and probably many others, the maintenance of the database would be a critical issue as IP addresses are being added/changed all the time.

To have the database embedded within the extension seems wrong as if this extension is added to the CDB then a new version will have to be created every time the database changes - with the ensuing delay.

In my opinion this type of lookup should use a remote database lookup that is constantly maintained and unfortunately, in my experience, there are very few free ones that stay around for any length of time.

Re: [DEV] Filter by country

Posted: Wed May 29, 2019 8:35 am
by 3Di
david63 wrote:
Wed May 29, 2019 7:22 am
How do you intend maintaining the database Mark?
I read the code, the DB is self-maintained via a cron job every 7 days.

The idea is good, the code needs to be improved. IMHO.

Re: [DEV] Filter by country

Posted: Wed May 29, 2019 8:54 am
by EA117
3Di wrote:
Wed May 29, 2019 8:35 am
david63 wrote:
Wed May 29, 2019 7:22 am
MarkDHamill wrote:
Tue May 28, 2019 5:20 pm
  • A cron automatically updates the database weekly
How do you intend maintaining the database Mark?
I read the code, the DB is self-maintained via a cron job every 7 days.
And further to that, it's once every seven days because MaxMind updates the free version of the database only once every seven days. That's the answer for "how will the database be updated when new IP address ranges are moved and assigned to a different country"; it's because MaxMind is already maintaining that aspect of the database. The extension just has to download the updated database to benefit from those updates.

Re: [DEV] Filter by country

Posted: Wed May 29, 2019 9:07 am
by 3Di
@EA117
Please don't quote me just to reiterate what I've already said in a few words.
This generates unnecessary notifications. Thank you. :)

Re: [DEV] Filter by country

Posted: Wed May 29, 2019 12:08 pm
by MarkDHamill
Yes, the database is updated every seven days. The database is updated by MaxMind on Tuesdays, but not knowing when on Tuesdays and due to timezones, I set it to check on Wednesdays, but only if seven days have elapsed since the last time it was downloaded.

No doubt the code can be better and the features are just a first set. I'm looking for input on how to make it more useful.

Re: [DEV] Filter by country

Posted: Wed May 29, 2019 3:14 pm
by </Solidjeuh>
I have the "cronstatus" ext installed. Since I installed this ext on my test board I always get the massage on the ACP main page that the cron is blocked. It goes away after some time, and then comes back for a few hours...

This is the ext:
https://github.com/BoardTools/cronstatus

Re: [DEV] Filter by country

Posted: Wed May 29, 2019 5:47 pm
by MarkDHamill
Thanks for helping to test the extension. I installed the Cron Status extension too. The state it is showing is "Auto". Not sure what that means but as I have repeatedly installed, deactivated and deleted data for the extension, and it was most recently reinstalled yesterday, I think it means it's waiting on events. It's possible I am doing something wrong programming the cron, but it's similar to what I did for another extension. I suspect it will "wake up" next Wednesday since a week will have elapsed and it's on or after a Wednesday, which is what I programmed.

I did test the cron by changing the value of phpbbservices_filterbycountry_cron_task_last_gc to 0 in the database, purging the cache, going directly to the index, logging out, then logging back in and observed that the directory was back and the .mmdb file was back as well with a current date and time on the file.

Re: [DEV] Filter by country

Posted: Thu May 30, 2019 7:00 pm
by MarkDHamill
BTW, here is what the blocked message should look like. There are two variations depending on whether logging in from restricted countries is allowed. It should be noted that no forum content is shown and clicking on any link should return the same message.

Image
Image

Re: [DEV] Filter by country

Posted: Fri May 31, 2019 2:11 am
by bubbathegimp
I just installed the extension. I only have a few users, but seem to get spammed weekly by Chinese spammers wanting to sell Nike etc...

I had recently used web.config to block the IP's , But speed on my board was taking a hit, so I decided to try this..

Not sure if I did it right. The App could really use a easy to understand section that shows if a country has been "Blocked", along with
the ability to make changes if necessary

Re: [DEV] Filter by country

Posted: Fri May 31, 2019 2:19 am
by MarkDHamill
It's got it already, but has to be enabled in settings for the extension.
  • Turn on statistics in settings to start collecting statistics on countries blocked and allowed
  • Turn on logging to start putting admin log entries every time an IP is blocked.