Images Hijacked by Photobucket?

Discussion of non-phpBB related topics with other phpBB.com users.
Forum rules
General Discussion is a bonus forum for discussion of non-phpBB related topics with other phpBB.com users. All site rules apply.
Post Reply
v12mike
Registered User
Posts: 143
Joined: Thu Jul 09, 2015 5:03 pm

Re: Images Hijacked by Photobucket?

Post by v12mike » Sun Aug 06, 2017 4:31 pm

Zepticon wrote:
Sun Aug 06, 2017 1:06 pm
v12mike wrote:
Fri Aug 04, 2017 8:31 pm
Zepticon wrote:
Fri Aug 04, 2017 8:04 pm
Thanks for the reply, and also for making this script :) I will play around with it some more in the comming week and see what happends. But do you have any idea why it could be failing on file download #254 ?
If you take the link for file 254 and paste it into a browser, you will see that it is redirected to a web page (probably because the original file has been deleted). The script correctly sees this as a failure to download, but this should not stop the script running, it should just go to the next url in the database.
Is there a way i can get it to skip this one? Or to start from file 255?
URL 254 is unlikely to be your problem, as it has already been failed and skipped. If your download always stops there, it is more likely that it is URL 255 that is the issue. If you edit the database (phpbb_esternal_images table) and for the row with ext_image_id=255 change the status to something non-zero (perhaps 999), the downloader will skip it.

v12mike
Registered User
Posts: 143
Joined: Thu Jul 09, 2015 5:03 pm

Re: Images Hijacked by Photobucket?

Post by v12mike » Sun Aug 06, 2017 4:36 pm

sp3ctre wrote:
Sun Aug 06, 2017 7:45 am
Great job with the scripts. Out of interest how much of a challenge would it be to trawl the database and replace the url's with the local ones? I'm just concerned about relying on a plugin working with future phpbb releases.
I thought about writing a script to do a bulk upload of the files as attachments, but decided that it would be quite a lot of work, and had a higher risk of corrupting the database if something went wrong.

The business-end of the extension (that actually patches the file links) is based on the extension that phpBB themselves use on this forum to patch insecure image URLs, so I don't think that it will get fundamentally broken for a while. If the extension does become unsupportable, the information is retained in the database to do something else later.

My extension does not actually use the added image database tables, they are only used by the scripts, and kept for future insurance purposes.

Zepticon
Registered User
Posts: 8
Joined: Sun Dec 13, 2015 6:44 pm

Re: Images Hijacked by Photobucket?

Post by Zepticon » Sun Aug 06, 2017 4:50 pm

v12mike wrote:
Sun Aug 06, 2017 4:31 pm
If you edit the database (phpbb_esternal_images table) and for the row with ext_image_id=255 change the status to something non-zero (perhaps 999), the downloader will skip it.
I feel that this is an incredible stupid question, but where is that database located?

User avatar
JimA
Community Team Leader
Community Team Leader
Posts: 7046
Joined: Thu Jul 31, 2008 5:54 am
Location: The Netherlands
Name: Jim Mossing Holsteyn
Contact:

Re: Images Hijacked by Photobucket?

Post by JimA » Sun Aug 06, 2017 5:39 pm

As this topic was becoming more and more of a support topic for the script to take images from Photobucket, as opposed to a discussion topic, I've now moved it over to phpBB Custom Coding.
Image Jim Mossing Holsteyn - Community Team Leader
Knowledge Base | Documentation | Board rules

If you're having any questions about the rules/customs of this website, feel free to drop me a PM.

paul_f
Registered User
Posts: 8
Joined: Thu Jun 18, 2015 7:22 pm
Name: Paul
Contact:

Re: Images Hijacked by Photobucket?

Post by paul_f » Sun Aug 06, 2017 5:51 pm

Zepticon wrote:
Sun Aug 06, 2017 4:50 pm
I feel that this is an incredible stupid question, but where is that database located?
It is the forum database, have you got experience in accessing that?

If not, hopefully you have phpmyadmin with your hosting package or can add it?

sp3ctre
Registered User
Posts: 250
Joined: Sun Jul 15, 2007 11:06 am

Re: Images Hijacked by Photobucket?

Post by sp3ctre » Sun Aug 06, 2017 7:44 pm

v12mike wrote:
Sun Aug 06, 2017 4:36 pm
sp3ctre wrote:
Sun Aug 06, 2017 7:45 am
Great job with the scripts. Out of interest how much of a challenge would it be to trawl the database and replace the url's with the local ones? I'm just concerned about relying on a plugin working with future phpbb releases.
I thought about writing a script to do a bulk upload of the files as attachments, but decided that it would be quite a lot of work, and had a higher risk of corrupting the database if something went wrong.

The business-end of the extension (that actually patches the file links) is based on the extension that phpBB themselves use on this forum to patch insecure image URLs, so I don't think that it will get fundamentally broken for a while. If the extension does become unsupportable, the information is retained in the database to do something else later.

My extension does not actually use the added image database tables, they are only used by the scripts, and kept for future insurance purposes.
Interesting... I haven't examined the code so please excuse my ignorance, but as I understand what you are saying is the extension works as follows:

When an external mage is found it is converted to MD5 and looked up in the directory... if found, it is used, if not then the original url is used.

I originally thought it used the database but this method seems even more slick :)

paul_f
Registered User
Posts: 8
Joined: Thu Jun 18, 2015 7:22 pm
Name: Paul
Contact:

Re: Images Hijacked by Photobucket?

Post by paul_f » Sun Aug 06, 2017 8:11 pm

Tip for those who have a Microsoft IIS server. You will need to make a file called web.config in the ext directory where the images are stored. This only works for IIS7.0 and above. IIS6.0 you will need to browse in the IIS manager to the directory and add a MIME type there

Code: Select all

 <?xml version="1.0" encoding="UTF-8"?>
 <configuration>
     <system.webServer>
         <staticContent>
             <mimeMap fileExtension="." mimeType="image/jpeg" />
         </staticContent>
     </system.webServer>
 </configuration>
Last edited by paul_f on Sun Aug 06, 2017 9:18 pm, edited 1 time in total.

v12mike
Registered User
Posts: 143
Joined: Thu Jul 09, 2015 5:03 pm

Re: Images Hijacked by Photobucket?

Post by v12mike » Sun Aug 06, 2017 8:19 pm

sp3ctre wrote:
Sun Aug 06, 2017 7:44 pm
Interesting... I haven't examined the code so please excuse my ignorance, but as I understand what you are saying is the extension works as follows:

When an external mage is found it is converted to MD5 and looked up in the directory... if found, it is used, if not then the original url is used.

I originally thought it used the database but this method seems even more slick :)
That is a fairly good description, I had planned to use database lookups, but this turned out to be quicker to code and has a certain simplicity.

v12mike
Registered User
Posts: 143
Joined: Thu Jul 09, 2015 5:03 pm

Re: Images Hijacked by Photobucket?

Post by v12mike » Sun Aug 06, 2017 8:22 pm

JimA wrote:
Sun Aug 06, 2017 5:39 pm
As this topic was becoming more and more of a support topic for the script to take images from Photobucket, as opposed to a discussion topic, I've now moved it over to phpBB Custom Coding.
A fair call.

Zepticon
Registered User
Posts: 8
Joined: Sun Dec 13, 2015 6:44 pm

Re: Images Hijacked by Photobucket?

Post by Zepticon » Sun Aug 06, 2017 11:29 pm

paul_f wrote:
Sun Aug 06, 2017 5:51 pm
Zepticon wrote:
Sun Aug 06, 2017 4:50 pm
I feel that this is an incredible stupid question, but where is that database located?
It is the forum database, have you got experience in accessing that?
If not, hopefully you have phpmyadmin with your hosting package or can add it?
I have no experience in any of this to be honest. I had someone set up the forum years ago, and i have had to fix and tweak it myself the last years, but nothing on this level.

I dont have phpmyadmin, since according to my hosting service its unsafe if not updated every new release. I could install it myself but its not adviced from their side.

They recommend me to use either a local software like MySQLWorkbench or just regular SSH in a shell. I had SQL workbench installed, and when i connected up to it i managed to find the databases in question. I an also see that the line 254 is never executed by the download script as it is the first without all columns filled out.

Tho i have no idea how to access and edit the database. Preferably i could look at it like an excel sheet but i dont know if thats doable in any program :D

EDIT:
I downloaded SequelPro who let me edit the tables directly in a GUI so i changed out the bad adresses and now its working it seems :)

Zepticon
Registered User
Posts: 8
Joined: Sun Dec 13, 2015 6:44 pm

Re: Images Hijacked by Photobucket?

Post by Zepticon » Mon Aug 07, 2017 12:33 am

v12mike wrote:
Sun Aug 06, 2017 4:31 pm
URL 254 is unlikely to be your problem, as it has already been failed and skipped. If your download always stops there, it is more likely that it is URL 255 that is the issue. If you edit the database (phpbb_esternal_images table) and for the row with ext_image_id=255 change the status to something non-zero (perhaps 999), the downloader will skip it.
It was 254 that was the issue. I am not sure what error logs i can give you, but it seems the script is failing on several adresses as it moves on. If i go in the database and manually change them to something i know is broken, the script just FAIL them and move on. So far i have gotten to line 379 of 31.000, so im hoping there is a fix that can save me from doing this manually.

What i have gotten errors on so far:
http://tanetane92.web.infoseek.co.jp/052991.jpg
http://www.pakfeifer.at/images/626%20GD_6_fertig.jpg
http://www.stud.ntnu.no/~tobiasbu/Mazda ... %20029.jpg

And get this error in the shell:

Code: Select all

<b>[phpBB Debug] PHP Warning</b>: in file <b>[ROOT]/scripts/download_external_images.php</b> on line <b>93</b>: <b>Illegal string offset 'info'</b><br />
<b>[phpBB Debug] PHP Warning</b>: in file <b>[ROOT]/scripts/download_external_images.php</b> on line <b>93</b>: <b>Illegal string offset 'http_code'</b><br />
<b>[phpBB Debug] PHP Warning</b>: in file <b>[ROOT]/scripts/download_external_images.php</b> on line <b>94</b>: <b>Illegal string offset 'info'</b><br />
<b>[phpBB Debug] PHP Warning</b>: in file <b>[ROOT]/scripts/download_external_images.php</b> on line <b>94</b>: <b>Illegal string offset 'size_download'</b><br />
status E FAIL
PHP Warning:  Cannot modify header information - headers already sent by (output started at /home/3/m/mazda-ogn/www/scripts/download_external_images.php:91) in /home/3/m/mazda-ogn/www/includes/functions.php on line 1954

Holger
Registered User
Posts: 1599
Joined: Tue Mar 12, 2002 3:54 pm
Location: Hannover

Re: Images Hijacked by Photobucket?

Post by Holger » Mon Aug 07, 2017 6:18 am

Ummm, all those images linked above are "File not found".
Are you looking for a phpBB Expert? Find a phpBB Expert!
Are you a phpBB Expert? Post your resume!
http://www.phpbbexperts.com

Lyna
Registered User
Posts: 6
Joined: Sun Aug 06, 2017 5:24 pm

Re: Images Hijacked by Photobucket?

Post by Lyna » Mon Aug 07, 2017 7:12 am

Hello
I followed the instructions and have successfully uploaded and completly enabled the extension.
What do i have to do now so that the stupid photobucket images that appear in the place of some smilies are blocked or redirected?

v12mike
Registered User
Posts: 143
Joined: Thu Jul 09, 2015 5:03 pm

Re: Images Hijacked by Photobucket?

Post by v12mike » Mon Aug 07, 2017 8:42 am

Holger wrote:
Mon Aug 07, 2017 6:18 am
Ummm, all those images linked above are "File not found".
That is true, but the script (for everyone else) copes with that by setting the status in the database so that they are skipped next time around.

The problem may be something to do with the version of php or some other environment issue. It seems that there is some problem with accessing the multi-dimensional array in the case of download failure.

I will try to hack the script to avoid that array access, and see what happens.

zepticon, can you pm me your email address so that I can send you a script to try?

v12mike
Registered User
Posts: 143
Joined: Thu Jul 09, 2015 5:03 pm

Re: Images Hijacked by Photobucket?

Post by v12mike » Mon Aug 07, 2017 8:49 am

Lyna wrote:
Mon Aug 07, 2017 7:12 am
Hello
I followed the instructions and have successfully uploaded and completly enabled the extension.
What do i have to do now so that the stupid photobucket images that appear in the place of some smilies are blocked or redirected?
The most likely cause for this is that the download script skips small files, as they are often icons indicating download failure.

If there are only a few that you need, you could manually download them and rename with the MD5 filename.

You could also edit the download script and change

Code: Select all

define('MINIMUM_FILE_SIZE', 3000);
to be something smaller then rerun that script.

Post Reply

Return to “General Discussion”

Who is online

Users browsing this forum: No registered users and 49 guests