I have made some minor updates to my image download scripts and extension.
The operational changes are:
- The link extraction script now (by default) ignores pbpBB V3.1 format image links. This speeds up operation.
- The download files script now (by default) adds the original file extension to the downloaded file. This allows compatibility with .svg files and perhaps some other file types that browsers will not render without the correct filename extension.
- The download script now (by default) adds the filename extension to a previously downloaded file if it was not present.
- The download script now (by default) skips the download of any file for which a http 404 response has been previously received. If the original server couldn't find it then, it is unlikely to find it now...
- The Download script keeps a list of hosts which do not give valid responses to curl requests, and skips any further files located on that host. This list is recreated on each running of the script.
- The download script now (by default) handles http redirects in the script, rather than in curl, as some hosting companies put restrictions on curl redirects.
- The imageredirect extension now handles locally stored image files with or without filename extensions, for compatibility with the current or older versions of the download script
To upgrade to the new version of scripts, It is important to follow the correct sequence:
- First install the new version of the image redirect script
- Then install and run the new version of "extract_external_links.php" If you used default parameters before, they should still be OK
- Then install and run the new version of "download_external_images.php" If you used default parameters before, they should still be OK