[CDB] Digests 3.2.11

A place for Extension Authors to post and receive feedback on Extensions still in development. No Extensions within this forum should be used within a live environment!
Scam Warning
Forum rules
READ: phpBB.com Board-Wide Rules and Regulations

IMPORTANT: Extensions Development rules

IMPORTANT FOR NEEDED EVENTS!!!
If you need an event for your extension please read this for the steps to follow to request the event(s)
Locked
steveosonic
Registered User
Posts: 13
Joined: Thu Mar 16, 2017 9:54 am

Re: [RC] Digests 3.2.0-RC7

Post by steveosonic » Sun Mar 19, 2017 9:14 am

Hi Mark, thank you for putting together this add on.

I have a new forum, installed digest 3.2 RC7, each time the digest tries to run in the error log I see this:
Error while creating image
Sun 19 Mar, 2017 8:36 am Error while creating image
» Error in [ROOT]/phpbb/user.php on line 599: DateTime::setTimezone() expects parameter 1 to be DateTimeZone, null given
Sun 19 Mar, 2017 8:36 am Error while creating image
» Error in [ROOT]/phpbb/user.php on line 599: DateTime::setTimezone() expects parameter 1 to be DateTimeZone, null given
Sun 19 Mar, 2017 8:36 am Error while creating image
» Error in [ROOT]/phpbb/user.php on line 599: DateTime::setTimezone() expects parameter 1 to be DateTimeZone, null given
Sun 19 Mar, 2017 8:36 am Error while creating image
» Error in [ROOT]/phpbb/user.php on line 599: DateTime::setTimezone() expects parameter 1 to be DateTimeZone, null given
Sun 19 Mar, 2017 8:36 am Error while creating image
» Error in [ROOT]/phpbb/user.php on line 599: DateTime::setTimezone() expects parameter 1 to be DateTimeZone, null given
Sun 19 Mar, 2017 8:36 am Error while creating image
» Error in [ROOT]/phpbb/user.php on line 599: DateTime::setTimezone() expects parameter 1 to be DateTimeZone, null given
The Admin logs shows that Digests are stuck on 16th March:
Sun 19 Mar, 2017 8:36 am Regular (phpBB) cron run of the mailer invoked
Sun 19 Mar, 2017 8:36 am Running digests for 2017-03-16 06 UTC
Sun 19 Mar, 2017 6:13 am Regular (phpBB) cron run of the mailer invoked
Sun 19 Mar, 2017 6:13 am Running digests for 2017-03-16 06 UTC
Sun 19 Mar, 2017 12:01 am Regular (phpBB) cron run of the mailer invoked
Sun 19 Mar, 2017 12:01 am Running digests for 2017-03-16 06 UTC
I've had a look in the user table for the database and there is nothing unusual for timezone. My users are mostly UTC / London timezone. I have a custom date format though this may not be the issue?

Any clues on where to look would be gratefully received.

Many thanks,
Steve

User avatar
MarkDHamill
Registered User
Posts: 3956
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Re: [RC] Digests 3.2.0-RC7

Post by MarkDHamill » Mon Mar 20, 2017 4:08 pm

Steve, this is probably due to being an admin when logged in and not having a timezone set. It might also be some users where their default timezone is not set. They should not be able to subscribe to a digest without having a timezone set. Perhaps you did a mass subscribe?

Assuming your table prefix is phpbb_ this will set all user_timezone columns where it is blank to GMT and should make the problem go away. Use phpMyAdmin or similar tool:

Code: Select all

update phpbb_users set user_timezone = 'UTC' WHERE user_timezone = '' 
Change the above to a SELECT statement to see the extent of the problem.

A list of valid values for the field can be found here:

http://php.net/manual/en/timezones.php
Last edited by MarkDHamill on Mon Mar 20, 2017 4:53 pm, edited 1 time in total.
Get the latest versions of my Digests and Smartfeed extensions.
Need phpBB services or a phpBB consultant? I offer most phpBB services.

User avatar
MarkDHamill
Registered User
Posts: 3956
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Re: [RC] Digests 3.2.0-RC7

Post by MarkDHamill » Mon Mar 20, 2017 4:12 pm

richey wrote:
Sat Mar 11, 2017 2:07 pm
p.s. the whole database doesn't contain "acp_digests" (except for the log table which contains that the module was removed), which makes it even more strange ...
Richey, you shouldn't be getting the message if the module ACP_CAT_DIGESTS does not exist in the modules table. Is this still the case?
Get the latest versions of my Digests and Smartfeed extensions.
Need phpBB services or a phpBB consultant? I offer most phpBB services.

steveosonic
Registered User
Posts: 13
Joined: Thu Mar 16, 2017 9:54 am

Re: [RC] Digests 3.2.0-RC7

Post by steveosonic » Mon Mar 20, 2017 7:50 pm

MarkDHamill wrote:
Mon Mar 20, 2017 4:08 pm
Steve, this is probably due to being an admin when logged in and not having a timezone set. It might also be some users where their default timezone is not set. They should not be able to subscribe to a digest without having a timezone set. Perhaps you did a mass subscribe?

Assuming your table prefix is phpbb_ this will set all user_timezone columns where it is blank to GMT and should make the problem go away. Use phpMyAdmin or similar tool:

Code: Select all

update phpbb_users set user_timezone = 'UTC' WHERE user_timezone = '' 
Change the above to a SELECT statement to see the extent of the problem.

A list of valid values for the field can be found here:

http://php.net/manual/en/timezones.php
Hi Mark, thank you for taking the time to reply, and for replying so quickly.

I did take a look at the users table, and all of the users had a timezone set already. I have reset all of the users to UTC, and also reset the date/time format which I'd moved to a custom value. The prior timezones were either UTC, or Europe/London which are valid values.

I am at a bit of a loss, although my database uses a custom prefix, taking a look at your code, there are no suspects to be seen (I am not a java or php coder, though can read them if not 'speak' them :lol: )

Indeed I did use the Mass Subscribe feature, but checking the user table, there was nothing amiss. The user who did 'have a word' with me about her inbox being blown up was a user that hadn't visited the forum though had signed up, this may or may not be a useful clue.

I'm at a bit of a loss, and will slowly re-subscribe folks until either there is success, or I hit another issue. I'll report back in a few days or so with an update.

Thank you again.

User avatar
MarkDHamill
Registered User
Posts: 3956
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Re: [RC] Digests 3.2.0-RC7

Post by MarkDHamill » Mon Mar 20, 2017 9:14 pm

It's puzzling. I need to look at my mass subscribe code to see what I do if anything with the user_timezone field. Most likely I don't touch it at all but maybe I should check it and if it is empty put in the board default.

I can't say what caused your problem but my extension builds on top of phpBB's architecture, which builds on top of the features in the supported versions of PHP. Most likely you had a lot of invalid date/time strings in the column that was triggering the error.

You might want to send a mass email to your users saying they should check/change their board timezone in the User Control Panel. They should then check the digest send hour as it depends on their board timezone setting.
Get the latest versions of my Digests and Smartfeed extensions.
Need phpBB services or a phpBB consultant? I offer most phpBB services.

User avatar
MarkDHamill
Registered User
Posts: 3956
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Re: [RC] Digests 3.2.0-RC7

Post by MarkDHamill » Wed Mar 22, 2017 12:12 am

Also, to get digests unstuck (so they don't deliver old digests) there is an easy option:

ACP > Extensions > Digests > Reset mailer
Get the latest versions of my Digests and Smartfeed extensions.
Need phpBB services or a phpBB consultant? I offer most phpBB services.

steveosonic
Registered User
Posts: 13
Joined: Thu Mar 16, 2017 9:54 am

Re: [RC] Digests 3.2.0-RC7

Post by steveosonic » Fri Mar 24, 2017 8:51 am

Hi Mark

I've upgraded to PHP 7.1, the forum was on something ancient like 5.4 or 5,3, I have discovered an additional error code (sorry).
24 Mar 2017, 08:38 Error while creating image
» Error in [ROOT]/ext/phpbbservices/digests/cron/task/digests.php on line 1595: array_multisort(): Argument #5 is expected to be an array or a sort flag
24 Mar 2017, 08:38 Error while creating image
» Error in [ROOT]/phpbb/user.php on line 599: DateTime::setTimezone() expects parameter 1 to be DateTimeZone, null given
24 Mar 2017, 08:38 Error while creating image
» Error in [ROOT]/ext/phpbbservices/digests/cron/task/digests.php on line 1595: array_multisort(): Argument #5 is expected to be an array or a sort flag
24 Mar 2017, 08:38 Error while creating image
» Error in [ROOT]/phpbb/user.php on line 599: DateTime::setTimezone() expects parameter 1 to be DateTimeZone, null given
I have again been through the user table, everyone has been set to UTC including the 'anonymous' account and the bots.

Do you have any suggestions on where I should look now? Many thanks, Steve

User avatar
3Di
Former Team Member
Posts: 14485
Joined: Mon Apr 04, 2005 11:09 pm
Location: Milan (IT) Frankfurt (DE)
Name: Marco
Contact:

Re: [RC] Digests 3.2.0-RC7

Post by 3Di » Fri Mar 24, 2017 9:07 am

@ mark

Have a look at this post of mine (7 months ago)
viewtopic.php?p=14521981#p14521981

The relevant part is, now:
There are big issues here and there, timezones. chron, migrations and all of the rest.

Seriously, you should have used features/code/functions (i.e.: timezones) already present in 3.1 instead of porting at all of the costs a previous code "old fashioned" that's now ugly, forced, pragmatic and not really working as it should, flawlessy.
Please PM me only to request paid works. Thx.
Want to compensate me for my interest? Donate
My development's activity º PhpStorm's proud user
Extensions, Scripts, MOD porting, Update/Upgrades
✒️ Black Friday 2019 @ The Studio ▪️◾️

User avatar
MarkDHamill
Registered User
Posts: 3956
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Re: [RC] Digests 3.2.0-RC7

Post by MarkDHamill » Fri Mar 24, 2017 2:16 pm

steveosonic wrote:
Fri Mar 24, 2017 8:51 am
Hi Mark

I've upgraded to PHP 7.1, the forum was on something ancient like 5.4 or 5,3, I have discovered an additional error code (sorry).
24 Mar 2017, 08:38 Error while creating image
» Error in [ROOT]/ext/phpbbservices/digests/cron/task/digests.php on line 1595: array_multisort(): Argument #5 is expected to be an array or a sort flag
24 Mar 2017, 08:38 Error while creating image
» Error in [ROOT]/phpbb/user.php on line 599: DateTime::setTimezone() expects parameter 1 to be DateTimeZone, null given
24 Mar 2017, 08:38 Error while creating image
» Error in [ROOT]/ext/phpbbservices/digests/cron/task/digests.php on line 1595: array_multisort(): Argument #5 is expected to be an array or a sort flag
24 Mar 2017, 08:38 Error while creating image
» Error in [ROOT]/phpbb/user.php on line 599: DateTime::setTimezone() expects parameter 1 to be DateTimeZone, null given
I have again been through the user table, everyone has been set to UTC including the 'anonymous' account and the bots.

Do you have any suggestions on where I should look now? Many thanks, Steve
So the following query returns no rows?

Code: Select all

SELECT * FROM `phpbb_users` WHERE user_timezone = '' 
In my case the Anonymous user matches this criteria, but that user should be filtered out of the list of users that can receive digests.

My guess is that there are one or more invalid date time strings. You can see a list of what you have with this query:

Code: Select all

SELECT distinct user_timezone FROM `phpbb_users` order by 1 
then compare it to the list of recognized user timezones referenced in a previous post. Look for missing underscores in these string, ex: America/Los Angeles is wrong but America/Los_Angeles is right.

By default the user_timezone column in the phpbb_users table does not allow nulls, so I think the error message is looking for null strings.

I don't see any changes in how array_multisort works in PHP 7 so I doubt the issue is there.
Get the latest versions of my Digests and Smartfeed extensions.
Need phpBB services or a phpBB consultant? I offer most phpBB services.

User avatar
MarkDHamill
Registered User
Posts: 3956
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Re: [RC] Digests 3.2.0-RC7

Post by MarkDHamill » Fri Mar 24, 2017 2:21 pm

3Di wrote:
Fri Mar 24, 2017 9:07 am
@ mark

Have a look at this post of mine (7 months ago)
viewtopic.php?p=14521981#p14521981

The relevant part is, now:
There are big issues here and there, timezones. chron, migrations and all of the rest.

Seriously, you should have used features/code/functions (i.e.: timezones) already present in 3.1 instead of porting at all of the costs a previous code "old fashioned" that's now ugly, forced, pragmatic and not really working as it should, flawlessy.
I agree that in retrospect starting from scratch may have been a better approach. I found the 3.1 architecture quite daunting. I think I am past the migrations and chron issues. As I get better with the 3.1 architecture I may undertake a reengineering project. As the extension is complex though it is quite daunting and time consuming.

Using PhpStorm was certain eye opening as it allows many mistakes to at least be found, such as variables never used. It does a lot to help minimize bugs. Some bugs though are inevitable.
Get the latest versions of my Digests and Smartfeed extensions.
Need phpBB services or a phpBB consultant? I offer most phpBB services.

whocarez
Registered User
Posts: 144
Joined: Sat Apr 18, 2009 9:19 pm

Re: [RC] Digests 3.2.0-RC7

Post by whocarez » Sat Apr 01, 2017 12:51 pm

Hello Mark,
I'm not sure, but is your extension save to use with enabled daylight saving time?
I have a weekly digest and it is send on Saturdays. Last week it was ok, but today I extensions send the digest again and again.
user_digest_send_hour_gmt is 9
phpbbservices_digests_time_zone is 2, but with 0 it is the same.

The only difference to last week is, that we switched to the daylight saving time last Sunday.
I'm using the last version for phpbb 3.1 from github with PHP 5.4.45-0+deb7u8

User avatar
MarkDHamill
Registered User
Posts: 3956
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Re: [RC] Digests 3.2.0-RC7

Post by MarkDHamill » Sat Apr 01, 2017 4:51 pm

Digests are sent based on a GMT value stored in the database for each subscriber. It does not change automatically when a timezone goes to DST. Users can change the hour to get the digest in the user control panel.
Get the latest versions of my Digests and Smartfeed extensions.
Need phpBB services or a phpBB consultant? I offer most phpBB services.

steveosonic
Registered User
Posts: 13
Joined: Thu Mar 16, 2017 9:54 am

Re: [RC] Digests 3.2.0-RC7

Post by steveosonic » Sun Apr 02, 2017 8:13 am

whocarez wrote:
Sat Apr 01, 2017 12:51 pm
Hello Mark,
I'm not sure, but is your extension save to use with enabled daylight saving time?
I have a weekly digest and it is send on Saturdays. Last week it was ok, but today I extensions send the digest again and again.
user_digest_send_hour_gmt is 9
phpbbservices_digests_time_zone is 2, but with 0 it is the same.

The only difference to last week is, that we switched to the daylight saving time last Sunday.
I'm using the last version for phpbb 3.1 from github with PHP 5.4.45-0+deb7u8
Hi whocarez

If you go into "ACP / Maintenance / Error Log" what error logs do you see? Are you seeing the same error messages as I posted earlier in this thread or are you seeing different messages?

steveosonic
Registered User
Posts: 13
Joined: Thu Mar 16, 2017 9:54 am

Re: [RC] Digests 3.2.0-RC7

Post by steveosonic » Sun Apr 02, 2017 8:15 am

MarkDHamill wrote:
Fri Mar 24, 2017 2:16 pm
steveosonic wrote:
Fri Mar 24, 2017 8:51 am
Hi Mark

I've upgraded to PHP 7.1, the forum was on something ancient like 5.4 or 5,3, I have discovered an additional error code (sorry).
24 Mar 2017, 08:38 Error while creating image
» Error in [ROOT]/ext/phpbbservices/digests/cron/task/digests.php on line 1595: array_multisort(): Argument #5 is expected to be an array or a sort flag
24 Mar 2017, 08:38 Error while creating image
» Error in [ROOT]/phpbb/user.php on line 599: DateTime::setTimezone() expects parameter 1 to be DateTimeZone, null given
24 Mar 2017, 08:38 Error while creating image
» Error in [ROOT]/ext/phpbbservices/digests/cron/task/digests.php on line 1595: array_multisort(): Argument #5 is expected to be an array or a sort flag
24 Mar 2017, 08:38 Error while creating image
» Error in [ROOT]/phpbb/user.php on line 599: DateTime::setTimezone() expects parameter 1 to be DateTimeZone, null given
I have again been through the user table, everyone has been set to UTC including the 'anonymous' account and the bots.

Do you have any suggestions on where I should look now? Many thanks, Steve
So the following query returns no rows?

Code: Select all

SELECT * FROM `phpbb_users` WHERE user_timezone = '' 
In my case the Anonymous user matches this criteria, but that user should be filtered out of the list of users that can receive digests.

My guess is that there are one or more invalid date time strings. You can see a list of what you have with this query:

Code: Select all

SELECT distinct user_timezone FROM `phpbb_users` order by 1 
then compare it to the list of recognized user timezones referenced in a previous post. Look for missing underscores in these string, ex: America/Los Angeles is wrong but America/Los_Angeles is right.

By default the user_timezone column in the phpbb_users table does not allow nulls, so I think the error message is looking for null strings.

I don't see any changes in how array_multisort works in PHP 7 so I doubt the issue is there.
Hi Mark

All of my users in the user table are 'UTC' - I have doubled checked this morning, sorted by the column displayed all of the rows then checked the first row and last row, and confirmed they are all 'UTC'.

Many thanks :)

User avatar
MarkDHamill
Registered User
Posts: 3956
Joined: Fri Aug 02, 2002 12:36 am
Location: Florence, MA USA
Contact:

Re: [RC] Digests 3.2.0-RC7

Post by MarkDHamill » Sun Apr 02, 2017 1:55 pm

steveosonic,

I think the sorting issue is a result of some unpublicized changes to array_multisort in PHP 7. I will need to update my XAMPP to PHP 7 and see if there are any issues. php.net does not show any changes. Basically all posts and related metadata are read into an array and that array is sorted based on how the user likes posts to be presented based on their UCP settings. My sorting logic used to work fine:

Code: Select all

									array_multisort($left_id, SORT_ASC, $right_id, SORT_ASC, $topic_last_post_time, $topic_asc_desc, $post_time, $post_asc_desc, $posts_rowset);
$topic_last_post_time is the problem here as PHP 7 is expecting an array. What I am trying to do is take a relatively simple two dimensional array and sort it based on multiple columns. I may need to integrate some third party library to do this if array_multisort won't anymore.

Digests do not create images so I think something else is causing that problem.

That said, with phpBB 3.2 data is stored differently in the database, in that new posts will be store as HTML, not BBCode. It's still possible that the post will be in BBCode, but phpBB's cron is supposed to slowly convert these over time. I need some test cases to see if this impacts digests. I think the enable_bbcode column is used as a switch.

whocarez, on DST it may be possible to adjust user_digest_send_hour when DST changes. Rules vary depending on the time zone and country but phpBB apparently can detect it. There may be an event I can plug into to change these values when it occurs.
Get the latest versions of my Digests and Smartfeed extensions.
Need phpBB services or a phpBB consultant? I offer most phpBB services.

Locked

Return to “Extensions in Development”