I think that it is better to run the scripts from a command prompt or php shell. I beelieve that it is possible on a windows platform.
The script will not save the last ID if it terminates abnormally. Try running the script on the server from a prompt, or maybe add some debug prints to the script to see if it is choking on a particular url.paul_f wrote: ↑Wed Aug 02, 2017 7:52 am I have a problem with the scripts on my server in that the previous finished point isn't saved. Occasionally it does, but it is now causing the server to timeout on the extracting the links process. Even though it has checked post 850k/900k - the saved starting point is 750k. Tried all sorts of max post values with no luck.
Again the script must be terminating abnormally. try running the script from a prompt and report back
Code: Select all
PHP Warning: Cannot modify header information - headers already sent by (output started at /home/3/m/mazda-ogn/www/scripts/extract_external_links.php:73) in /home/3/m/mazda-ogn/www/includes/functions.php on line 1954
<!DOCTYPE html><html dir="ltr"><head><meta charset="utf-8"><meta http-equiv="X-UA-Compatible" content="IE=edge"><title>General Error</title><style type="text/css">
/* <![CDATA[ */
* { margin: 0; padding: 0; } html { font-size: 100%; height: 100%; margin-bottom: 1px; background-color: #E4EDF0; } body { font-family: "Lucida Grande", Verdana, Helvetica, Arial, sans-serif; color: #536482; background: #E4EDF0; font-size: 62.5%; margin: 0; } a:link, a:active, a:visited { color: #006699; text-decoration: none; } a:hover { color: #DD6900; text-decoration: underline; } #wrap { padding: 0 20px 15px 20px; min-width: 615px; } #page-header { text-align: right; height: 40px; } #page-footer { clear: both; font-size: 1em; text-align: center; } .panel { margin: 4px 0; background-color: #FFFFFF; border: solid 1px #A9B8C2; } #errorpage #page-header a { font-weight: bold; line-height: 6em; } #errorpage #content { padding: 10px; } #errorpage #content h1 { line-height: 1.2em; margin-bottom: 0; color: #DF075C; } #errorpage #content div { margin-top: 20px; margin-bottom: 5px; border-bottom: 1px solid #CCCCCC; padding-bottom: 5px; color: #333333; font: bold 1.2em "Lucida Grande", Arial, Helvetica, sans-serif; text-decoration: none; line-height: 120%; text-align: left; }
/* ]]> */
</style></head><body id="errorpage"><div id="wrap"> <div id="page-header"> <a href="../">Return to index page</a> </div> <div id="acp"> <div class="panel"> <div id="content"> <h1>General Error</h1> <div>SQL ERROR [ mysql4 ]<br /><br />User 'mazdaogn' has exceeded the 'max_updates_per_hour' resource (current value: 36000) [1226]<br /><br />An SQL error occurred while fetching this page. Please contact the <a href="mailto:[email protected]">Board Administrator</a> if this problem persists.</div><p>Please notify the board administrator or webmaster: <a href="mailto:[email protected]">[email protected]</a></p> </div> </div> </div> <div id="page-footer"> Powered by <a href="https://www.phpbb.com/">phpBB</a>® Forum Software © phpBB Limited </div></div></body></html>
Code: Select all
User 'mazdaogn' has exceeded the 'max_updates_per_hour' resource (current value: 36000)
Thats what i asked them, but they said it was "stupid high already" so i had to rewrite the script, or stage it.
Code: Select all
download 254 : http://tanetane92.web.infoseek.co.jp/052991.jpg
<b>[phpBB Debug] PHP Warning</b>: in file <b>[ROOT]/scripts/download_external_images.php</b> on line <b>93</b>: <b>Illegal string offset 'info'</b><br />
<b>[phpBB Debug] PHP Warning</b>: in file <b>[ROOT]/scripts/download_external_images.php</b> on line <b>93</b>: <b>Illegal string offset 'http_code'</b><br />
<b>[phpBB Debug] PHP Warning</b>: in file <b>[ROOT]/scripts/download_external_images.php</b> on line <b>94</b>: <b>Illegal string offset 'info'</b><br />
<b>[phpBB Debug] PHP Warning</b>: in file <b>[ROOT]/scripts/download_external_images.php</b> on line <b>94</b>: <b>Illegal string offset 'size_download'</b><br />
status < FAIL
PHP Warning: Cannot modify header information - headers already sent by (output started at /home/3/m/mazda-ogn/www/scripts/download_external_images.php:91) in /home/3/m/mazda-ogn/www/includes/functions.php on line 1954
Thanks for the reply, and also for making this script I will play around with it some more in the comming week and see what happends. But do you have any idea why it could be failing on file download #254 ?v12mike wrote: ↑Fri Aug 04, 2017 7:59 pm I am not 100% sure that I understand the limits of your hosting environment, but the update rate problem should be solved if you reduce the value of $interval in the script to something you are allowed, and then wait for the next period before you rerun the script.
Your permissions problem could be because the directory is being created by the script which will be inheriting user name and rights from the php process, which is probably not your user name. You need to change (or ask your provider to change) the ownership/permissions so that both you and php can see/modify the new directory.
If you take the link for file 254 and paste it into a browser, you will see that it is redirected to a web page (probably because the original file has been deleted). The script correctly sees this as a failure to download, but this should not stop the script running, it should just go to the next url in the database.
Is there a way i can get it to skip this one? Or to start from file 255?v12mike wrote: ↑Fri Aug 04, 2017 8:31 pmIf you take the link for file 254 and paste it into a browser, you will see that it is redirected to a web page (probably because the original file has been deleted). The script correctly sees this as a failure to download, but this should not stop the script running, it should just go to the next url in the database.