[BETA] Filter by country - version 1.0.8

A place for Extension Authors to post and receive feedback on Extensions still in development. No Extensions within this forum should be used within a live environment!
Suggested Hosts
Forum rules
READ: phpBB.com Board-Wide Rules and Regulations

IMPORTANT: Extensions Development rules

IMPORTANT FOR NEEDED EVENTS!!!
If you need an event for your extension please read this for the steps to follow to request the event(s)
Post Reply
User avatar
MarkDHamill
Registered User
Posts: 3942
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

[BETA] Filter by country - version 1.0.8

Post by MarkDHamill » Tue May 28, 2019 5:20 pm

This extension allows an administrator to filter traffic by country, allowing in only traffic from selected countries, or disallowing traffic only from selected countries. So if you get a lot of spam traffic from countries like Ukraine, Russia and Belarus, hopefully this will get rid of it. Or if you want to keep all traffic out except from your country, or a defined set of countries, it will do that too. If traffic is prohibited, they will see a message to this effect, but all other links won't work and the page will redirect back to the same message.

The free MaxMind GeoLite2 Country database is used, so it depends on the integrity of its data. The database is downloaded to the /store/phpbbservices/filterbycountry folder. The extension optionally tracks the number of page requests by country that are allowed and restricted. It is all controlled from two modules in the Extensions tab in the ACP.

Image

Other features:
  • You can allow users to login from restricted countries, if you enable the feature. Otherwise guest and bots see no forum content. Only the login link works.
  • Can filter out bots in phpBB's database from statistics.
  • Can log all restricted attempts in the admin log, if enabled. This can result in a very large log!
  • A cron automatically updates the database weekly
The report is straightforward and allows you to sort by column. You can also filter the report by both relative and absolute time ranges. Only countries that requested pages are shown for brevity. Data is stored to a table, so for a highly trafficked board this table could potentially get huge. If you enable this feature, make sure you have plenty of database space. A lot of shared hosts will limit database space.

Image

To keep your board from locking up, if you don't choose any countries to allow or restrict, all traffic is allowed. So the first thing to do is set the extension's settings correctly.

A standard block message looks like this. Clicking on any link simply returns the same page.

Image

If you want to allow registered users to login from blocked countries, the block message looks like this:

Image

The extension can be downloaded here:

The GitHub for the project is here:

https://github.com/MarkDHamill/filterbycountry
Last edited by MarkDHamill on Wed Oct 16, 2019 3:55 pm, edited 24 times in total.
Get the latest versions of my Digests and Smartfeed extensions.
Need phpBB services or a phpBB consultant? I offer most phpBB services.

User avatar
dmzx
Registered User
Posts: 780
Joined: Fri Sep 05, 2014 6:48 am
Location: The Netherlands
Contact:

Re: [DEV] Filter by country

Post by dmzx » Tue May 28, 2019 9:19 pm

download link does not work, but nice idea i will try it out.

edit:
Oops! That page can’t be found.
Contact me on dmzx-web.net
 Need a Host or buy me a beer Donate and try binance.com

fagbutlil
I've Been Banned!
Posts: 77
Joined: Wed Mar 07, 2018 10:56 pm

Re: [DEV] Filter by country

Post by fagbutlil » Tue May 28, 2019 9:22 pm

Working fine on 3.2.7 with php7.2.17 about time some thing like this come out as they say bye bye Russia

User avatar
MarkDHamill
Registered User
Posts: 3942
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Re: [DEV] Filter by country

Post by MarkDHamill » Tue May 28, 2019 9:35 pm

The download link is fixed.

There are other ways to block by country. Services like CloudFlare can do this and arguably can do a better job by keeping traffic from ever hitting your site. This consumes a little CPU but probably not too much plus it optionally allows registered users to login from a restricted area while keeping guests and bots out. The standard forum headers and footers are shown, but just a message saying access is blocked. All the links return the same message. The exception is if login is enabled. Registration is not enabled.

Testing has been limited. I suspect that VPNs may trip this up. If the IP is not in the database though there is the option to allow logins anyhow. MaxMind does have an Anonymous IP database. Unfortunately, this product is not free. It's unclear how much it costs but I may integrate it if there is demand and the phpBB does not object because it's not free. They allow certain other extensions, like Cleantalk, that is run as a paid service.
Get the latest versions of my Digests and Smartfeed extensions.
Need phpBB services or a phpBB consultant? I offer most phpBB services.

fagbutlil
I've Been Banned!
Posts: 77
Joined: Wed Mar 07, 2018 10:56 pm

Re: [DEV] Filter by country

Post by fagbutlil » Tue May 28, 2019 10:18 pm

I used a vpn to get around it but i guess most of them china and Viagra sellers wont use vpn.

User avatar
david63
Registered User
Posts: 16704
Joined: Thu Dec 19, 2002 8:08 am
Location: Lancashire, UK
Name: David Wood
Contact:

Re: [DEV] Filter by country

Post by david63 » Wed May 29, 2019 7:22 am

How do you intend maintaining the database Mark? For me, and probably many others, the maintenance of the database would be a critical issue as IP addresses are being added/changed all the time.

To have the database embedded within the extension seems wrong as if this extension is added to the CDB then a new version will have to be created every time the database changes - with the ensuing delay.

In my opinion this type of lookup should use a remote database lookup that is constantly maintained and unfortunately, in my experience, there are very few free ones that stay around for any length of time.
David
Remember: You only know what you know and - you don't know what you don't know!
My CDB Contributions | How to install an extension
I will not be accepting translations for any of my extensions in Github - please post any translations in the appropriate topic.
No support requests via PM or email as they will be ignored

User avatar
3Di
Former Team Member
Posts: 14379
Joined: Mon Apr 04, 2005 11:09 pm
Location: Milan (IT) Frankfurt (DE)
Name: Marco
Contact:

Re: [DEV] Filter by country

Post by 3Di » Wed May 29, 2019 8:35 am

david63 wrote:
Wed May 29, 2019 7:22 am
How do you intend maintaining the database Mark?
I read the code, the DB is self-maintained via a cron job every 7 days.

The idea is good, the code needs to be improved. IMHO.
Please PM me only to request paid works. Thx.
Want to compensate me for my interest? Donate
My development's activity º PhpStorm's proud user
Extensions, Scripts, MOD porting, Update/Upgrades
👨‍🏫 | Take a tour to | The Studio | 👨‍🏫

User avatar
EA117
Registered User
Posts: 1069
Joined: Wed Aug 15, 2018 3:23 am
Contact:

Re: [DEV] Filter by country

Post by EA117 » Wed May 29, 2019 8:54 am

3Di wrote:
Wed May 29, 2019 8:35 am
david63 wrote:
Wed May 29, 2019 7:22 am
MarkDHamill wrote:
Tue May 28, 2019 5:20 pm
  • A cron automatically updates the database weekly
How do you intend maintaining the database Mark?
I read the code, the DB is self-maintained via a cron job every 7 days.
And further to that, it's once every seven days because MaxMind updates the free version of the database only once every seven days. That's the answer for "how will the database be updated when new IP address ranges are moved and assigned to a different country"; it's because MaxMind is already maintaining that aspect of the database. The extension just has to download the updated database to benefit from those updates.

User avatar
3Di
Former Team Member
Posts: 14379
Joined: Mon Apr 04, 2005 11:09 pm
Location: Milan (IT) Frankfurt (DE)
Name: Marco
Contact:

Re: [DEV] Filter by country

Post by 3Di » Wed May 29, 2019 9:07 am

@EA117
Please don't quote me just to reiterate what I've already said in a few words.
This generates unnecessary notifications. Thank you. :)
Please PM me only to request paid works. Thx.
Want to compensate me for my interest? Donate
My development's activity º PhpStorm's proud user
Extensions, Scripts, MOD porting, Update/Upgrades
👨‍🏫 | Take a tour to | The Studio | 👨‍🏫

User avatar
MarkDHamill
Registered User
Posts: 3942
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Re: [DEV] Filter by country

Post by MarkDHamill » Wed May 29, 2019 12:08 pm

Yes, the database is updated every seven days. The database is updated by MaxMind on Tuesdays, but not knowing when on Tuesdays and due to timezones, I set it to check on Wednesdays, but only if seven days have elapsed since the last time it was downloaded.

No doubt the code can be better and the features are just a first set. I'm looking for input on how to make it more useful.
Get the latest versions of my Digests and Smartfeed extensions.
Need phpBB services or a phpBB consultant? I offer most phpBB services.

User avatar
</Solidjeuh>
Registered User
Posts: 1732
Joined: Tue Mar 29, 2016 3:45 am
Location: Aalst (Belgium)
Name: Andy Dm
Contact:

Re: [DEV] Filter by country

Post by </Solidjeuh> » Wed May 29, 2019 3:14 pm

I have the "cronstatus" ext installed. Since I installed this ext on my test board I always get the massage on the ACP main page that the cron is blocked. It goes away after some time, and then comes back for a few hours...

This is the ext:
https://github.com/BoardTools/cronstatus

User avatar
MarkDHamill
Registered User
Posts: 3942
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Re: [DEV] Filter by country

Post by MarkDHamill » Wed May 29, 2019 5:47 pm

Thanks for helping to test the extension. I installed the Cron Status extension too. The state it is showing is "Auto". Not sure what that means but as I have repeatedly installed, deactivated and deleted data for the extension, and it was most recently reinstalled yesterday, I think it means it's waiting on events. It's possible I am doing something wrong programming the cron, but it's similar to what I did for another extension. I suspect it will "wake up" next Wednesday since a week will have elapsed and it's on or after a Wednesday, which is what I programmed.

I did test the cron by changing the value of phpbbservices_filterbycountry_cron_task_last_gc to 0 in the database, purging the cache, going directly to the index, logging out, then logging back in and observed that the directory was back and the .mmdb file was back as well with a current date and time on the file.
Get the latest versions of my Digests and Smartfeed extensions.
Need phpBB services or a phpBB consultant? I offer most phpBB services.

User avatar
MarkDHamill
Registered User
Posts: 3942
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Re: [DEV] Filter by country

Post by MarkDHamill » Thu May 30, 2019 7:00 pm

BTW, here is what the blocked message should look like. There are two variations depending on whether logging in from restricted countries is allowed. It should be noted that no forum content is shown and clicking on any link should return the same message.

Image
Image
Get the latest versions of my Digests and Smartfeed extensions.
Need phpBB services or a phpBB consultant? I offer most phpBB services.

bubbathegimp
Registered User
Posts: 39
Joined: Tue Sep 25, 2018 8:02 pm
Location: Bronston
Name: Robert Anderson
Contact:

Re: [DEV] Filter by country

Post by bubbathegimp » Fri May 31, 2019 2:11 am

I just installed the extension. I only have a few users, but seem to get spammed weekly by Chinese spammers wanting to sell Nike etc...

I had recently used web.config to block the IP's , But speed on my board was taking a hit, so I decided to try this..

Not sure if I did it right. The App could really use a easy to understand section that shows if a country has been "Blocked", along with
the ability to make changes if necessary
73 De
Bubba

User avatar
MarkDHamill
Registered User
Posts: 3942
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Re: [DEV] Filter by country

Post by MarkDHamill » Fri May 31, 2019 2:19 am

It's got it already, but has to be enabled in settings for the extension.
  • Turn on statistics in settings to start collecting statistics on countries blocked and allowed
  • Turn on logging to start putting admin log entries every time an IP is blocked.
Get the latest versions of my Digests and Smartfeed extensions.
Need phpBB services or a phpBB consultant? I offer most phpBB services.

Post Reply

Return to “Extensions in Development”