[3.2][RC] Auto db Backup

A place for Extension Authors to post and receive feedback on Extensions still in development. No Extensions within this forum should be used within a live environment!
Scam Warning
Forum rules
READ: phpBB.com Board-Wide Rules and Regulations

IMPORTANT: Extensions Development rules

IMPORTANT FOR NEEDED EVENTS!!!
If you need an event for your extension please read this for the steps to follow to request the event(s)
User avatar
david63
Jr. Extension Validator
Posts: 12818
Joined: Thu Dec 19, 2002 8:08 am
Location: Lancashire, UK
Name: David Wood
Contact:

Re: [3.2][RC] Auto db Backup

Post by david63 » Mon Mar 06, 2017 3:44 pm

The only way you could get it to run at 3am would be to reset it each day and even then there is no guarantee that it would run at that time.

As you point out it is governed by the way phpBB cron works, which is not ideal, and is only instigated when there is a visit to the site. This is even more compounded by the fact that it will also be dependant on any other cron jobs that are in the queue before this one.

The time for the next backup will be the current time plus the time interval.

As for copying the backup to Google Drive it would be possible for someone to create an extension to do that in the same way that my Dropbox Upload extension copies the backup to Dropbox.
David
Remember: You only know what you know and - you don't know what you don't know!
My CDB Contributions | How to install an extension
I will not be accepting translations for any of my extensions in Github - please post any translations in the appropriate topic.
No support requests via PM or email or as they will be ignored

User avatar
javiexin
Code Contributor
Posts: 1157
Joined: Wed Oct 12, 2011 11:46 pm
Location: Madrid, Spain
Name: Javier
Contact:

Re: [3.2][RC] Auto db Backup

Post by javiexin » Mon Mar 06, 2017 8:33 pm

david63 wrote:
Mon Mar 06, 2017 3:44 pm
The time for the next backup will be the current time plus the time interval.
steveprentice wrote:
Mon Mar 06, 2017 3:33 pm
Is there any way to make the 3am stick there and not update/change? I realise that if the cron doesn't start till 9am, and then running it again at 3am wouldn't be 24 hours after.... but that's fine with me.
If I understood Steve right, what he is asking for is that the time for next backup is NOT current time plus interval, but originally scheduled time plus interval. For what he is saying, I think he understands the way phpbb cron tasks work...

I don't think this is too difficult to change, basically, in the cron task, replace:
$this->config->set('auto_db_backup_last_gc', time(), true); by

Code: Select all

$this->config->set('auto_db_backup_last_gc', $this->config->get('auto_db_backup_last_gc') + $this->config->get('auto_db_backup_gc'), true);
Or you may give an ACP config option, preserve interval or preserve scheduled time.
Regards,
-javiexin

PS: For what is worth, if I had the choice, I would go for a "preserve schedule" option.

User avatar
david63
Jr. Extension Validator
Posts: 12818
Joined: Thu Dec 19, 2002 8:08 am
Location: Lancashire, UK
Name: David Wood
Contact:

Re: [3.2][RC] Auto db Backup

Post by david63 » Mon Mar 06, 2017 9:41 pm

@javiexin

Obviously it would be possible but it is not quite that simple.

Take for example having the interval set at one hour. If there was no activity on the board for two hours then using the "preserved interval" method would result in the next backup being scheduled to be one hour ago, which would effectively mean that the next backup would be ready to run now. If the same thing was to happen (no activity for two hours) then the next backup would be two hours ago, etc.

Also adding into the equation the effect of timezones, DST and the vagaries of phpBB cron then the whole thing would become a mess.
David
Remember: You only know what you know and - you don't know what you don't know!
My CDB Contributions | How to install an extension
I will not be accepting translations for any of my extensions in Github - please post any translations in the appropriate topic.
No support requests via PM or email or as they will be ignored

User avatar
javiexin
Code Contributor
Posts: 1157
Joined: Wed Oct 12, 2011 11:46 pm
Location: Madrid, Spain
Name: Javier
Contact:

Re: [3.2][RC] Auto db Backup

Post by javiexin » Tue Mar 07, 2017 12:04 am

Ok, then something like:

Code: Select all

// Just for clarity
$last_bkp = $this->config->get('auto_db_backup_last_gc');
$this_bkp = time();
$bkp_interval = $this->config->get('auto_db_backup_gc');
$one_hour = ext::seconds;

// Schedule next backup for last schedule + interval, but at least in one hour
$next_bkp = max($last_bkp + $bkp_interval, $this_bkp + $one_hour);

// Now fake current backup time in the config table to get the right scheduled backup time
$this->config->set('auto_db_backup_last_gc', $next_bkp - $bkp_interval, true);
People that set backup frequency so low as one hour should not use "preserve schedule" but keep using the current "preserve interval" schema. Anyhow, the above code does something like that. And it makes sure that there are no backups in less than one hour (the extension minimum).

And timezones and other oddities should be taken care of when setting the date for the next backup, in the admin controller. Here all times are UTC (or at least should be), and you are calculating time intervals: a number of hours is the same regardless of the timezone or dst... Only if you say "4am each day in XXX city" you would have a difference with DST time changes (a "clock day" has 23 or 25 hours these days). But that is not being taken care of (and I don't think it matters). But I am digressing.

The code change is simple enough, I think... I have not tested it, of course, and I might be missing something, but you know the extension and I don't (only looked at it briefly), so you should be able to figure it out better...

Regards,
-javiexin

steveprentice
Registered User
Posts: 5
Joined: Thu Apr 11, 2013 8:37 pm

Re: [3.2][RC] Auto db Backup

Post by steveprentice » Thu Mar 09, 2017 7:35 pm

Thanks guys! Some fab chat on that, really very much appreciated. :)

As a temp workaround, I had taken to injecting the following SQL at the end of the google drive upload task:

UPDATE `phpbb_config`
SET config_value='1489028400'
WHERE config_name='auto_db_backup_last_gc';

That seems to do the trick from what I can see.... but the code you've come up with there looks like a much better option, thanks!

Cheers,

Steve

steveprentice
Registered User
Posts: 5
Joined: Thu Apr 11, 2013 8:37 pm

Re: [3.2][RC] Auto db Backup

Post by steveprentice » Sun Mar 12, 2017 2:48 pm

steveprentice wrote:
Thu Mar 09, 2017 7:35 pm
SET config_value='1489028400'
I'm an idiot. That should be (in my case anyway):

SET config_value=UNIX_TIMESTAMP(DATE_ADD(CURDATE(),INTERVAL '1 3' DAY_HOUR))

Also, anyone looking to automate the backup of the resulting sql file to somewhere (like google drive in my example)... then I'm sure there's more than one way, but what's working well for me is this: https://github.com/prasmussen/gdrive - i.e. gdrive.exe in Windows, which I schedule and use PowerShell script to tell it what files I want to be uploaded to where.

Cheers,

Steve

User avatar
david63
Jr. Extension Validator
Posts: 12818
Joined: Thu Dec 19, 2002 8:08 am
Location: Lancashire, UK
Name: David Wood
Contact:

Re: [3.2][RC] Auto db Backup

Post by david63 » Sun Mar 12, 2017 2:54 pm

steveprentice wrote:
Sun Mar 12, 2017 2:48 pm
nyone looking to automate the backup of the resulting sql file to somewhere (like google drive in my example)... then I'm sure there's more than one way,
There is. There is an event in the Auto Backup that will allow another extension to access the backup file when it is created and do whatever you want with it - such as my Dropbox Upload extensions that uploads the autobackup file to Dropbox.

If you wanted to upload automatically to Google Drive then it is only a matter of creating an extension to use the Google Drive API.
David
Remember: You only know what you know and - you don't know what you don't know!
My CDB Contributions | How to install an extension
I will not be accepting translations for any of my extensions in Github - please post any translations in the appropriate topic.
No support requests via PM or email or as they will be ignored

User avatar
wads24
Registered User
Posts: 341
Joined: Fri Jun 10, 2005 4:44 am

Re: [3.2][RC] Auto db Backup

Post by wads24 » Mon Mar 20, 2017 6:40 am

I do not see the settings on the extensions tab. Where can I find the settings in admin panel?

Update. Found it at
ACP -> Maintenance
Thanks in advance for a reply.

User avatar
wads24
Registered User
Posts: 341
Joined: Fri Jun 10, 2005 4:44 am

Re: [3.2][RC] Auto db Backup

Post by wads24 » Mon Mar 20, 2017 6:47 am

What should I select for option that says, "Optimize the database before performing the backup:"?
Thanks in advance for a reply.

User avatar
david63
Jr. Extension Validator
Posts: 12818
Joined: Thu Dec 19, 2002 8:08 am
Location: Lancashire, UK
Name: David Wood
Contact:

Re: [3.2][RC] Auto db Backup

Post by david63 » Mon Mar 20, 2017 6:55 am

wads24 wrote:
Mon Mar 20, 2017 6:47 am
What should I select for option that says, "Optimize the database before performing the backup:"?
Yes or No
David
Remember: You only know what you know and - you don't know what you don't know!
My CDB Contributions | How to install an extension
I will not be accepting translations for any of my extensions in Github - please post any translations in the appropriate topic.
No support requests via PM or email or as they will be ignored

User avatar
wads24
Registered User
Posts: 341
Joined: Fri Jun 10, 2005 4:44 am

Re: [3.2][RC] Auto db Backup

Post by wads24 » Mon Mar 20, 2017 7:01 am

david63 wrote:
Mon Mar 20, 2017 6:55 am
wads24 wrote:
Mon Mar 20, 2017 6:47 am
What should I select for option that says, "Optimize the database before performing the backup:"?
Yes or No
You're funny
Thanks in advance for a reply.

digitaltoast
Registered User
Posts: 97
Joined: Thu Oct 18, 2007 9:33 am

Re: [3.2][RC] Auto db Backup

Post by digitaltoast » Mon Mar 20, 2017 7:11 am

Based on experience, I would select "NO", because it doesn't backup with "YES"!

That said, you should be using InnoDB anyway, which apparently doesn't allow for optimisation like that (someone correct me if wrong)

User avatar
wads24
Registered User
Posts: 341
Joined: Fri Jun 10, 2005 4:44 am

Re: [3.2][RC] Auto db Backup

Post by wads24 » Mon Mar 20, 2017 7:48 am

digitaltoast wrote:
Mon Mar 20, 2017 7:11 am
Based on experience, I would select "NO", because it doesn't backup with "YES"!

That said, you should be using InnoDB anyway, which apparently doesn't allow for optimisation like that (someone correct me if wrong)
Thank you! You were more helpful than the actual developer of the extension!
Thanks in advance for a reply.

User avatar
KhurramMunawar
Registered User
Posts: 512
Joined: Tue Mar 25, 2014 2:20 am
Location: Islamabad, Pakistan
Name: Khurram Munawar
Contact:

Re: [3.2][RC] Auto db Backup

Post by KhurramMunawar » Mon Mar 20, 2017 7:02 pm

Dear David

Is the backup time issue has been resolved or not yet.
Image

User avatar
david63
Jr. Extension Validator
Posts: 12818
Joined: Thu Dec 19, 2002 8:08 am
Location: Lancashire, UK
Name: David Wood
Contact:

Re: [3.2][RC] Auto db Backup

Post by david63 » Mon Mar 20, 2017 9:28 pm

KhurramMunawar wrote:
Mon Mar 20, 2017 7:02 pm
Dear David

Is the backup time issue has been resolved or not yet.
What issue? As far as I am aware the time is working as intended
David
Remember: You only know what you know and - you don't know what you don't know!
My CDB Contributions | How to install an extension
I will not be accepting translations for any of my extensions in Github - please post any translations in the appropriate topic.
No support requests via PM or email or as they will be ignored

Post Reply

Return to “Extensions in Development”

Who is online

Users browsing this forum: dshyne, fliper4o, Froddelaar, Hippie459MN, king577, nou nou, SalazarAG, Yandex [Bot] and 28 guests