[BETA] Filter by country - version 1.0.8

A place for Extension Authors to post and receive feedback on Extensions still in development. No Extensions within this forum should be used within a live environment!
Ideas Centre
Forum rules
READ: phpBB.com Board-Wide Rules and Regulations

IMPORTANT: Extensions Development rules

IMPORTANT FOR NEEDED EVENTS!!!
If you need an event for your extension please read this for the steps to follow to request the event(s)
User avatar
</Solidjeuh>
Registered User
Posts: 1716
Joined: Tue Mar 29, 2016 3:45 am
Location: Aalst (Belgium)
Name: Andy Dm
Contact:

Re: [ALPHA] Filter by country - version 1.0.1

Post by </Solidjeuh> » Wed Jun 05, 2019 9:03 pm

Then I don't understand why it's not being blocked here :shock:

User avatar
MarkDHamill
Registered User
Posts: 3929
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Re: [ALPHA] Filter by country - version 1.0.1

Post by MarkDHamill » Wed Jun 05, 2019 9:06 pm

In my case, I hard coded your IP. You might want to check your assumptions. After line 162 in /event/main_listener.php:

Code: Select all

		$user_ip = $this->request->server('REMOTE_ADDR');    // Fetch the user's actual IP address.
add

Code: Select all

		echo $user_ip;
And see what appears on the screen.

Also make sure your settings are identical to mine for each use case.
Get the latest versions of my Digests and Smartfeed extensions.
Need phpBB services or a phpBB consultant? I offer most phpBB services.

User avatar
MarkDHamill
Registered User
Posts: 3929
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

[BETA] Filter by country - version 1.0.2 released

Post by MarkDHamill » Sun Jun 16, 2019 9:33 pm

A new beta version is out. Some changes of note:
  • The settings page looks a bit different. In particular, a setting has been renamed "Allow VPN services" because it more accurately describes what happens if an IP is not found in the MaxMind database ... it is most likely access from a VPN. MaxMind sells a separate database of VPN IPs. They weren't willing to give me a free copy for my development, so I decided not to try to integrate it. Most wouldn't want to buy it anyhow.
  • Fixed a bug that looked for jQuery in another extension instead of this one (my Selective mass emails extension)
  • In the event of a corrupt database, an email will be sent to all the founders warning them of the problem. The extension is also disabled so that board traffic is not brought to a halt in this event.
The reporting interface hasn't changed.

I'm confident that the logic for filtering by country code is working correctly based on my testing.

You can download the latest version by going to the first post.
Get the latest versions of my Digests and Smartfeed extensions.
Need phpBB services or a phpBB consultant? I offer most phpBB services.

User avatar
</Solidjeuh>
Registered User
Posts: 1716
Joined: Tue Mar 29, 2016 3:45 am
Location: Aalst (Belgium)
Name: Andy Dm
Contact:

Re: [BETA] Filter by country - version 1.0.2

Post by </Solidjeuh> » Sun Jun 16, 2019 11:31 pm

My problems seems to be resolved with the new update.
I also noticed that there was only 1 statistic showing, only the "United states".
Now with the update everything shows again.
Belgium has already 3734 hits in 3 hours, but I guess that's normal?
That will be a lot in a few months/years... Will this not raise the database table too much?

User avatar
MarkDHamill
Registered User
Posts: 3929
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Re: [BETA] Filter by country - version 1.0.2

Post by MarkDHamill » Mon Jun 17, 2019 1:22 am

The statistics can be erased by turning them off.

I think there may be some double counting going on, page refreshes or something similar. The details are in the phpbb_fbc_stats table if you want to look at them. This table does not collect the IP address, but if you have logging turned on these will appear in the admin log.
Last edited by MarkDHamill on Thu Jul 04, 2019 9:52 pm, edited 1 time in total.
Get the latest versions of my Digests and Smartfeed extensions.
Need phpBB services or a phpBB consultant? I offer most phpBB services.

User avatar
</Solidjeuh>
Registered User
Posts: 1716
Joined: Tue Mar 29, 2016 3:45 am
Location: Aalst (Belgium)
Name: Andy Dm
Contact:

Re: [BETA] Filter by country - version 1.0.2

Post by </Solidjeuh> » Mon Jun 17, 2019 9:51 am

MarkDHamill wrote:
Mon Jun 17, 2019 1:22 am
The statistics can be erased by during them off.

I think there may be some double counting going on, page refreshes or something similar. The details are in the phpbb_fbc_stats table if you want to look at them. This table does not collect the IP address, but if you have logging turned on these will appear in the admin log.
After disabling the stats, and re-enable them it seems be be good now. Belgium has 186 hits in 7 hours, Germany 700, United States 779 ... Does that look more "normal"?

User avatar
EA117
Registered User
Posts: 940
Joined: Wed Aug 15, 2018 3:23 am
Contact:

Re: [BETA] Filter by country - version 1.0.2

Post by EA117 » Mon Jun 17, 2019 11:33 am

Does the fact this is hooked into the user_setup event mean it will be re-evaluated at each page access, or is there something that keeps it from being quite that frequent? This just came to mind when </Solidjeuh> asked about database access.

Since the MaxMind database is local, it's not like "there will be a new decision if we load and query the database again." At least not until the MaxMind database is updated "on Wednesday", and/or unless the administrator makes a configuration change as to which countries are excluded. As compared to if an online version of the database was being queried, and we would have no idea if our next query will get the same answer or not.

Which made me wonder, would there be a performance benefit to tying this to "session creation" or similar, rather than "user setup". Or perhaps "keep checking during user_setup", but checking a decision which has been "cached", either in the session info or in your own private cache. Such that if the MaxMind database was already checked for the IP address in the user's session, and there hasn't been a configuration change or database update since then, the cached decision continues to be used rather than making a new decision.

Probably not describing anything you hadn't already considered or can see additional challenges for; it just came to mind on the performance question. I have no idea what kind of performance or resource impact is created by the MaxMind database loading and reader they provided, to guess at how significant or insignificant the savings would be from caching.

User avatar
MarkDHamill
Registered User
Posts: 3929
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Re: [BETA] Filter by country - version 1.0.2

Post by MarkDHamill » Mon Jun 17, 2019 12:06 pm

This is a good analysis. It may be that I am not using the right event. I doubt it hurts to recheck, as I am sure the call to MaxMind's database is quick. If the IP is not in the database, I believe you get an automatic refresh in the form of a page with the error message. A lot of it may be that.
Get the latest versions of my Digests and Smartfeed extensions.
Need phpBB services or a phpBB consultant? I offer most phpBB services.

User avatar
</Solidjeuh>
Registered User
Posts: 1716
Joined: Tue Mar 29, 2016 3:45 am
Location: Aalst (Belgium)
Name: Andy Dm
Contact:

Re: [BETA] Filter by country - version 1.0.2

Post by </Solidjeuh> » Mon Jun 17, 2019 1:52 pm

There are country's with special characters like "Oekraïne", "België", but those characters break in the stats. I see: Oekraïne. My translated file should be good, it's UNIX UTF-8.

User avatar
MarkDHamill
Registered User
Posts: 3929
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Re: [BETA] Filter by country - version 1.0.2

Post by MarkDHamill » Mon Jun 17, 2019 3:23 pm

In your language file, I see:

https://github.com/Solidjeuh/filterbyco ... common.php for Belgium:

<option value="BE">Belgie</option>

but I do see it for Ukraine:

<option value="UA">Oekraïne</option>

I installed the Dutch language pack. On the statistics, things look normal:

Image
Get the latest versions of my Digests and Smartfeed extensions.
Need phpBB services or a phpBB consultant? I offer most phpBB services.

User avatar
</Solidjeuh>
Registered User
Posts: 1716
Joined: Tue Mar 29, 2016 3:45 am
Location: Aalst (Belgium)
Name: Andy Dm
Contact:

Re: [BETA] Filter by country - version 1.0.2

Post by </Solidjeuh> » Mon Jun 17, 2019 3:29 pm

Yes I changed all the ë to e because of that problem in the NL language. Gonna change it back if you can fix that .. ? :D

User avatar
MarkDHamill
Registered User
Posts: 3929
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Re: [BETA] Filter by country - version 1.0.2

Post by MarkDHamill » Mon Jun 17, 2019 3:41 pm

I could apply the change to my GitHub master if you publish it.

When I copy and paste it I see the error you described.

The question is what's causing it? The page asserts UTF-8 content:

<meta charset="utf-8">

This data comes from your language file. I use phpStorm for my editor. I can choose any encoding I want. The English file encoding is ISO 8859-1. I tried UTF-8 and it made no difference. There are some special characters in my English version but I haven't tried to use them to see if they render correctly.

Is there a file encoding expert on the thread?
Get the latest versions of my Digests and Smartfeed extensions.
Need phpBB services or a phpBB consultant? I offer most phpBB services.

User avatar
canonknipser
Registered User
Posts: 2086
Joined: Thu Sep 08, 2011 4:16 am
Location: Germany
Name: Frank Jakobs
Contact:

Re: [BETA] Filter by country - version 1.0.2

Post by canonknipser » Mon Jun 17, 2019 4:00 pm

everything nowadays should be utf-8 in phpBB, don't use ISO 8859-1 for file contents. It is too limited
Greetings, Frank
phpbb.de support team member
English is not my native language - no support via PM or mail
New arrival - Extensions and scripts for phpBB

User avatar
MarkDHamill
Registered User
Posts: 3929
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Re: [BETA] Filter by country - version 1.0.2

Post by MarkDHamill » Mon Jun 17, 2019 4:03 pm

I haven't changed any defaults in phpStorm. I would think it would default to UTF-8 encoding.
Get the latest versions of my Digests and Smartfeed extensions.
Need phpBB services or a phpBB consultant? I offer most phpBB services.

User avatar
canonknipser
Registered User
Posts: 2086
Joined: Thu Sep 08, 2011 4:16 am
Location: Germany
Name: Frank Jakobs
Contact:

Re: [BETA] Filter by country - version 1.0.2

Post by canonknipser » Mon Jun 17, 2019 4:22 pm

I guess you problem is you are a bit lazy ;)
You already have a form field option list in you language file, and read it in html mode:
https://github.com/MarkDHamill/filterby ... #L293-L294 Maybe this drops the national characters

Afterwards, you put the translated values in an array without the html part: https://github.com/MarkDHamill/filterby ... #L295-L306

I guess working the other way round having a php array in your language file and create the option set when you build the field should work as intended (like the timezone handling in core: https://github.com/phpbb/phpbb/blob/102 ... L985-L1445)
Greetings, Frank
phpbb.de support team member
English is not my native language - no support via PM or mail
New arrival - Extensions and scripts for phpBB

Post Reply

Return to “Extensions in Development”