Images Hijacked by Photobucket?

Discussion of non-phpBB related topics with other phpBB.com users.
Forum rules
General Discussion is a bonus forum for discussion of non-phpBB related topics with other phpBB.com users. All site rules apply.
User avatar
chanlon1
Registered User
Posts: 228
Joined: Wed Mar 09, 2005 10:01 pm
Location: Belfast, N. Ireland
Contact:

Re: Images Hijacked by Photobucket?

Post by chanlon1 »

I'm beginning to just look into how this extension works etc......what about photobucket links that are in peoples signatures?
Does it get those images and download them etc?
Or is it purely just inline in the actual posts?

also, after running and identifying over 90K links, it returns this error:

Code: Select all

SQL ERROR [ mysql4 ]
Illegal mix of collations (latin1_swedish_ci,IMPLICIT) and (utf8_general_ci,COERCIBLE) for operation 'like' [1267]

thanks
User avatar
chanlon1
Registered User
Posts: 228
Joined: Wed Mar 09, 2005 10:01 pm
Location: Belfast, N. Ireland
Contact:

Re: Images Hijacked by Photobucket?

Post by chanlon1 »

chanlon1 wrote: Fri Sep 01, 2017 4:20 pm
also, after running and identifying over 90K links, it returns this error:

Code: Select all

SQL ERROR [ mysql4 ]
Illegal mix of collations (latin1_swedish_ci,IMPLICIT) and (utf8_general_ci,COERCIBLE) for operation 'like' [1267]

thanks
I think (still to be checked) I may have solved it.
For some reason some of the columns on the new two tables had their collation changed to swedish.
I forced them back to utf8-bin and the scripts seemed to continue ok.
v12mike
Registered User
Posts: 584
Joined: Thu Jul 09, 2015 5:03 pm

Re: Images Hijacked by Photobucket?

Post by v12mike »

chanlon1 wrote: Fri Sep 01, 2017 4:20 pm I'm beginning to just look into how this extension works etc......what about photobucket links that are in peoples signatures?
Does it get those images and download them etc?
Or is it purely just inline in the actual posts?
That is a good question.

The extract_external_links script for identifying external links and adding them to the new database tables (currently) only looks at the body of posts, so images that only appear in sigs or avatars, or added by extensions are not added to the database and thus not downloaded.

The extension that redirects links to the local copy actually looks at all output fields, and thus does (if the image has been downloaded) redirect correctly images in sigs, avatars, etc.

I don't currently plan to rework extract-external-links to add the extra functionality.
User avatar
chanlon1
Registered User
Posts: 228
Joined: Wed Mar 09, 2005 10:01 pm
Location: Belfast, N. Ireland
Contact:

Re: Images Hijacked by Photobucket?

Post by chanlon1 »

v12mike wrote: Fri Sep 01, 2017 8:26 pm
I don't currently plan to rework extract-external-links to add the extra functionality.
I dont blame you...you've done a great job as it is.
Much appreciated.
User avatar
chanlon1
Registered User
Posts: 228
Joined: Wed Mar 09, 2005 10:01 pm
Location: Belfast, N. Ireland
Contact:

Re: Images Hijacked by Photobucket?

Post by chanlon1 »

Got it working now on my test site and it all seems good.

One question......if I have a photo bucket URL in one of my posts.....how do I work out which image in the images/ext folder it is trying to get???

Im not sure how to translate from the URL to the image filename.
v12mike
Registered User
Posts: 584
Joined: Thu Jul 09, 2015 5:03 pm

Re: Images Hijacked by Photobucket?

Post by v12mike »

chanlon1 wrote: Sun Sep 03, 2017 4:00 pm One question......if I have a photo bucket URL in one of my posts.....how do I work out which image in the images/ext folder it is trying to get???

Im not sure how to translate from the URL to the image filename.
The clue is in the line of code:

Code: Select all

$local_file_name = md5("$url");
Any program that will find the md5 hash of a string should produce the same result (or run the php md5 command from the command line).
User avatar
chanlon1
Registered User
Posts: 228
Joined: Wed Mar 09, 2005 10:01 pm
Location: Belfast, N. Ireland
Contact:

Re: Images Hijacked by Photobucket?

Post by chanlon1 »

v12mike wrote: Sun Sep 03, 2017 5:01 pm
chanlon1 wrote: Sun Sep 03, 2017 4:00 pm One question......if I have a photo bucket URL in one of my posts.....how do I work out which image in the images/ext folder it is trying to get???

Im not sure how to translate from the URL to the image filename.
The clue is in the line of code:

Code: Select all

$local_file_name = md5("$url");
Any program that will find the md5 hash of a string should produce the same result (or run the php md5 command from the command line).
Thanks. Was able to put the URL into a md5 generator and get the string and then tie it to the relevant file.

Appreciate the response.
shnazzle
Registered User
Posts: 5
Joined: Wed Sep 13, 2017 3:15 pm

Re: Images Hijacked by Photobucket?

Post by shnazzle »

was passed this link to solve the issue on our long-running phpBB forum.
Very nice bit of kit!

Ran the extract, then the download. All good.

Decided to download a few of the md5-hashed image files it downloaded and test them.
Renamed them; ...none of them worked. All bogus.

Copied the Curl code from the download script and put in a URL as pulled from the output of the download script and sent it to "testfile.jpg". Worked perfectly.

So
1) It's extracting all the images correctly
2) It's going through the photobucket images in the "images_links" table and downloading them
3) The images for which status 200 is found and all is ok, the URL is md5-hashed and created on the drive.

I now have 1.4GB (so far) of garbage.
Am I missing a very simply step?
User avatar
chanlon1
Registered User
Posts: 228
Joined: Wed Mar 09, 2005 10:01 pm
Location: Belfast, N. Ireland
Contact:

Re: Images Hijacked by Photobucket?

Post by chanlon1 »

shnazzle wrote: Wed Sep 13, 2017 3:22 pm was passed this link to solve the issue on our long-running phpBB forum.
Very nice bit of kit!

Ran the extract, then the download. All good.

Decided to download a few of the md5-hashed image files it downloaded and test them.
Renamed them; ...none of them worked. All bogus.

Copied the Curl code from the download script and put in a URL as pulled from the output of the download script and sent it to "testfile.jpg". Worked perfectly.

So
1) It's extracting all the images correctly
2) It's going through the photobucket images in the "images_links" table and downloading them
3) The images for which status 200 is found and all is ok, the URL is md5-hashed and created on the drive.

I now have 1.4GB (so far) of garbage.
Am I missing a very simply step?
Instead of downloading and renaming etc to test the image, have you tried pointing a browser to the exact file without any extension? I.e domain.com/phpbb/images/ext/hdh636?635gdb

Also have you actually enabled the extension in the ACP?
shnazzle
Registered User
Posts: 5
Joined: Wed Sep 13, 2017 3:15 pm

Re: Images Hijacked by Photobucket?

Post by shnazzle »

chanlon1 wrote: Wed Sep 13, 2017 7:34 pm Instead of downloading and renaming etc to test the image, have you tried pointing a browser to the exact file without any extension? I.e domain.com/phpbb/images/ext/hdh636?635gdb

Also have you actually enabled the extension in the ACP?
Yup, the link does open the image. Albeit with a whole lot of popups/overlays. And if I manually curl the link, it pulls back the image as v12 intended.
Just doesn't download the images when run through the download script.

This is pre-extension. This is the pre-requisite work for the extension as per instructions , so no point firing up the proxy just yet as the images don't work :)

I'm at a loss.
User avatar
chanlon1
Registered User
Posts: 228
Joined: Wed Mar 09, 2005 10:01 pm
Location: Belfast, N. Ireland
Contact:

Re: Images Hijacked by Photobucket?

Post by chanlon1 »

I'm not really sure then. Hopefully Mike will come along to help.
My extract and downloads ran fine. 44K images in the folder, and when I switch the extension on and off I can see the images appearing in the posts and not appearing. So I know it's working.

The only thing is changed was the text in the download file and I made it download only PB images.

Have you made sure that the permissions on the images/ext folder are correct? Think they should be 755. I do recall having to fix that.
shnazzle
Registered User
Posts: 5
Joined: Wed Sep 13, 2017 3:15 pm

Re: Images Hijacked by Photobucket?

Post by shnazzle »

I did indeed. At first it wasn't writing anything. Schoolboy error.

I also considered transfer method incorrect on ftp. But I tried 2 "images" in forced binary mode and still no dice.

Only thing I can think is that there's some limitations on the server's PHP settings.

Just odd that it works perfectly on the same curl code if I do it manually
shnazzle
Registered User
Posts: 5
Joined: Wed Sep 13, 2017 3:15 pm

Re: Images Hijacked by Photobucket?

Post by shnazzle »

Mystery (partially) solved.
It's an FTP thing.

Downloaded a few of the images via cpanel and they all work.
v12mike
Registered User
Posts: 584
Joined: Thu Jul 09, 2015 5:03 pm

Re: Images Hijacked by Photobucket?

Post by v12mike »

I presume that you have it sorted now.

Make sure you have your ftp client set to use binary transfer by default (the image files without extensions might be defaulting to ASCII mode).
shnazzle
Registered User
Posts: 5
Joined: Wed Sep 13, 2017 3:15 pm

Re: Images Hijacked by Photobucket?

Post by shnazzle »

v12mike wrote: Sun Sep 17, 2017 7:37 am I presume that you have it sorted now.

Make sure you have your ftp client set to use binary transfer by default (the image files without extensions might be defaulting to ASCII mode).
Post Reply

Return to “General Discussion”