[CDB] Filter by country - version 1.0.19

A place for Extension Authors to post and receive feedback on Extensions still in development. No Extensions within this forum should be used within a live environment!
Ideas Centre
Forum rules
READ: phpBB.com Board-Wide Rules and Regulations

IMPORTANT: Extensions Development rules

IMPORTANT FOR NEEDED EVENTS!!!
If you need an event for your extension please read this for the steps to follow to request the event(s)
User avatar
MarkDHamill
Registered User
Posts: 4933
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA

[CDB] Filter by country - version 1.0.19

Post by MarkDHamill »

This extension allows an administrator to filter traffic by country, allowing in only traffic from selected countries, or disallowing traffic only from selected countries. So if you get a lot of spam traffic from countries like Ukraine, Russia and Belarus, hopefully this will get rid of it. Or if you want to keep all traffic out except from your country, or a defined set of countries, it will do that too. If traffic is prohibited, they will see a message to this effect, but all other links won't work and the page will redirect back to the same message.

The free MaxMind GeoLite2 Country database is used, so it depends on the integrity of its data. The database is downloaded to the /store/phpbbservices/filterbycountry folder. The extension optionally tracks the number of page requests by country that are allowed and restricted. It is all controlled from two modules in the Extensions tab in the ACP.

Image

Other features:
  • You can allow users to login from restricted countries, if you enable the feature. Otherwise guest and bots see no forum content. Only the login link works.
  • Can filter out bots in phpBB's database from statistics.
  • Can log all restricted attempts in the admin log, if enabled. This can result in a very large log!
  • A cron automatically updates the database weekly
  • A test mode is now supported, but use with caution as legitimate traffic may get blocked.
  • If an IP address is blocked and you do not allow out of country logins, you can optionally have it redirect to a URL you provide.
The report is straightforward and allows you to sort by column. You can also filter the report by both relative and absolute time ranges. Only countries that requested pages are shown for brevity. Data is stored to a table, so for a highly trafficked board this table could potentially get huge. If you enable this feature, make sure you have plenty of database space. A lot of shared hosts will limit database space.

Image

To keep your board from locking up, if you don't choose any countries to allow or restrict, all traffic is allowed. So the first thing to do is set the extension's settings correctly.

A standard block message looks like this. Clicking on any link simply returns the same page.

Image

If you want to allow registered users to login from blocked countries, the block message looks like this:

Image

The extension can be downloaded here or from the GitHub 1.0.19 release branch. If downloaded from GitHub, make sure to place the files in /ext/phpbbservices/filterbycountry.

The GitHub for the project is here:

https://github.com/MarkDHamill/filterbycountry
Last edited by MarkDHamill on Sat Dec 19, 2020 2:57 pm, edited 36 times in total.
Need phpBB services or a phpBB consultant? I offer most phpBB services. Getting lost managing phpBB? Buy my book, Mastering phpBB Administration. Covers through phpBB 3.3.7. eBook and paper versions available.
User avatar
dmzx
Registered User
Posts: 853
Joined: Fri Sep 05, 2014 6:48 am
Location: The Netherlands

Re: [DEV] Filter by country

Post by dmzx »

download link does not work, but nice idea i will try it out.

edit:
Oops! That page can’t be found.
Contact me on dmzx-web.net
Buy me a beer Donate and try binance.com or try CentOS Web Panel
fagbutlil
I've Been Banned!
Posts: 77
Joined: Wed Mar 07, 2018 10:56 pm

Re: [DEV] Filter by country

Post by fagbutlil »

Working fine on 3.2.7 with php7.2.17 about time some thing like this come out as they say bye bye Russia
User avatar
MarkDHamill
Registered User
Posts: 4933
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA

Re: [DEV] Filter by country

Post by MarkDHamill »

The download link is fixed.

There are other ways to block by country. Services like CloudFlare can do this and arguably can do a better job by keeping traffic from ever hitting your site. This consumes a little CPU but probably not too much plus it optionally allows registered users to login from a restricted area while keeping guests and bots out. The standard forum headers and footers are shown, but just a message saying access is blocked. All the links return the same message. The exception is if login is enabled. Registration is not enabled.

Testing has been limited. I suspect that VPNs may trip this up. If the IP is not in the database though there is the option to allow logins anyhow. MaxMind does have an Anonymous IP database. Unfortunately, this product is not free. It's unclear how much it costs but I may integrate it if there is demand and the phpBB does not object because it's not free. They allow certain other extensions, like Cleantalk, that is run as a paid service.
Need phpBB services or a phpBB consultant? I offer most phpBB services. Getting lost managing phpBB? Buy my book, Mastering phpBB Administration. Covers through phpBB 3.3.7. eBook and paper versions available.
fagbutlil
I've Been Banned!
Posts: 77
Joined: Wed Mar 07, 2018 10:56 pm

Re: [DEV] Filter by country

Post by fagbutlil »

I used a vpn to get around it but i guess most of them china and Viagra sellers wont use vpn.
User avatar
david63
Registered User
Posts: 20646
Joined: Thu Dec 19, 2002 8:08 am

Re: [DEV] Filter by country

Post by david63 »

How do you intend maintaining the database Mark? For me, and probably many others, the maintenance of the database would be a critical issue as IP addresses are being added/changed all the time.

To have the database embedded within the extension seems wrong as if this extension is added to the CDB then a new version will have to be created every time the database changes - with the ensuing delay.

In my opinion this type of lookup should use a remote database lookup that is constantly maintained and unfortunately, in my experience, there are very few free ones that stay around for any length of time.
David
Remember: You only know what you know and - you don't know what you don't know!

I now no longer support any of my extensions but they will start to become available here
User avatar
3Di
I've Been Banned!
Posts: 17538
Joined: Mon Apr 04, 2005 11:09 pm
Location: I'm with Ukraine 🇺🇦
Name: Marco

Re: [DEV] Filter by country

Post by 3Di »

david63 wrote: Wed May 29, 2019 7:22 am How do you intend maintaining the database Mark?
I read the code, the DB is self-maintained via a cron job every 7 days.

The idea is good, the code needs to be improved. IMHO.
🆓 Free support for our extensions also provided here: phpBB Studio
🚀 Looking for a specific feature or alternative option? We will rock you!
Please PM me only to request paid works. Thx. Buy me a coffee -> Image
My development's activity º PhpStorm's proud user º Extensions, Scripts, MOD porting, Update/Upgrades
User avatar
EA117
Registered User
Posts: 2173
Joined: Wed Aug 15, 2018 3:23 am

Re: [DEV] Filter by country

Post by EA117 »

3Di wrote: Wed May 29, 2019 8:35 am
david63 wrote: Wed May 29, 2019 7:22 am
MarkDHamill wrote: Tue May 28, 2019 5:20 pm
  • A cron automatically updates the database weekly
How do you intend maintaining the database Mark?
I read the code, the DB is self-maintained via a cron job every 7 days.
And further to that, it's once every seven days because MaxMind updates the free version of the database only once every seven days. That's the answer for "how will the database be updated when new IP address ranges are moved and assigned to a different country"; it's because MaxMind is already maintaining that aspect of the database. The extension just has to download the updated database to benefit from those updates.
User avatar
3Di
I've Been Banned!
Posts: 17538
Joined: Mon Apr 04, 2005 11:09 pm
Location: I'm with Ukraine 🇺🇦
Name: Marco

Re: [DEV] Filter by country

Post by 3Di »

@EA117
Please don't quote me just to reiterate what I've already said in a few words.
This generates unnecessary notifications. Thank you. :)
🆓 Free support for our extensions also provided here: phpBB Studio
🚀 Looking for a specific feature or alternative option? We will rock you!
Please PM me only to request paid works. Thx. Buy me a coffee -> Image
My development's activity º PhpStorm's proud user º Extensions, Scripts, MOD porting, Update/Upgrades
User avatar
MarkDHamill
Registered User
Posts: 4933
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA

Re: [DEV] Filter by country

Post by MarkDHamill »

Yes, the database is updated every seven days. The database is updated by MaxMind on Tuesdays, but not knowing when on Tuesdays and due to timezones, I set it to check on Wednesdays, but only if seven days have elapsed since the last time it was downloaded.

No doubt the code can be better and the features are just a first set. I'm looking for input on how to make it more useful.
Need phpBB services or a phpBB consultant? I offer most phpBB services. Getting lost managing phpBB? Buy my book, Mastering phpBB Administration. Covers through phpBB 3.3.7. eBook and paper versions available.
User avatar
</Solidjeuh>
Registered User
Posts: 1788
Joined: Tue Mar 29, 2016 3:45 am
Location: Aalst (Belgium)
Name: Andy Dm

Re: [DEV] Filter by country

Post by </Solidjeuh> »

I have the "cronstatus" ext installed. Since I installed this ext on my test board I always get the massage on the ACP main page that the cron is blocked. It goes away after some time, and then comes back for a few hours...

This is the ext:
https://github.com/BoardTools/cronstatus
User avatar
MarkDHamill
Registered User
Posts: 4933
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA

Re: [DEV] Filter by country

Post by MarkDHamill »

Thanks for helping to test the extension. I installed the Cron Status extension too. The state it is showing is "Auto". Not sure what that means but as I have repeatedly installed, deactivated and deleted data for the extension, and it was most recently reinstalled yesterday, I think it means it's waiting on events. It's possible I am doing something wrong programming the cron, but it's similar to what I did for another extension. I suspect it will "wake up" next Wednesday since a week will have elapsed and it's on or after a Wednesday, which is what I programmed.

I did test the cron by changing the value of phpbbservices_filterbycountry_cron_task_last_gc to 0 in the database, purging the cache, going directly to the index, logging out, then logging back in and observed that the directory was back and the .mmdb file was back as well with a current date and time on the file.
Need phpBB services or a phpBB consultant? I offer most phpBB services. Getting lost managing phpBB? Buy my book, Mastering phpBB Administration. Covers through phpBB 3.3.7. eBook and paper versions available.
User avatar
MarkDHamill
Registered User
Posts: 4933
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA

Re: [DEV] Filter by country

Post by MarkDHamill »

BTW, here is what the blocked message should look like. There are two variations depending on whether logging in from restricted countries is allowed. It should be noted that no forum content is shown and clicking on any link should return the same message.

Image
Image
Need phpBB services or a phpBB consultant? I offer most phpBB services. Getting lost managing phpBB? Buy my book, Mastering phpBB Administration. Covers through phpBB 3.3.7. eBook and paper versions available.
User avatar
bubbathegimp
Registered User
Posts: 192
Joined: Tue Sep 25, 2018 8:02 pm
Location: Off Plumb
Name: Bubba

Re: [DEV] Filter by country

Post by bubbathegimp »

I just installed the extension. I only have a few users, but seem to get spammed weekly by Chinese spammers wanting to sell Nike etc...

I had recently used web.config to block the IP's , But speed on my board was taking a hit, so I decided to try this..

Not sure if I did it right. The App could really use a easy to understand section that shows if a country has been "Blocked", along with
the ability to make changes if necessary
If at first you don't succeed, Sky diving is not for you....
Ymmv
User avatar
MarkDHamill
Registered User
Posts: 4933
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA

Re: [DEV] Filter by country

Post by MarkDHamill »

It's got it already, but has to be enabled in settings for the extension.
  • Turn on statistics in settings to start collecting statistics on countries blocked and allowed
  • Turn on logging to start putting admin log entries every time an IP is blocked.
Need phpBB services or a phpBB consultant? I offer most phpBB services. Getting lost managing phpBB? Buy my book, Mastering phpBB Administration. Covers through phpBB 3.3.7. eBook and paper versions available.

Return to “Extensions in Development”