Attempt Resolving Dead Image Links Using WayBackMachine

Looking for an Extension? Have an Extension request? Post your request here for help. (Note: This forum is community supported; while there is an Extensions Development Team, said team does not dedicate itself to handling requests in this forum)
Suggested Hosts
Post Reply
User avatar
EA117
Registered User
Posts: 516
Joined: Wed Aug 15, 2018 3:23 am
Contact:

Attempt Resolving Dead Image Links Using WayBackMachine

Post by EA117 » Fri Oct 26, 2018 5:11 am

While reviewing the user accounts on our forum, I happened to come across a few long-dead links to popular third-party image hosts (e.g. imageshack.com, etc.) that forum members had used to host images as their avatar, or as part of their signature, and/or as part of their posts.

I thought it was neat how when putting the full image URL into archive.org, they would report whether they had the specific image cached. e.g. The image link http://img1.jurko.net/avatar_2288.gif which has been dead 10 or so years can be viewed using https://web.archive.org/web/20171126042 ... r_2288.gif .

Although it would technically work, I wouldn't want to throw unnecessary load onto archive.org servers by literally replacing the dead http://img1.jurko.net/avatar_2288.gif link with https://web.archive.org/web/20171126042 ... r_2288.gif .

But I thought this might be an interesting phpBB extension. To crawl one's own phpBB forum, and for any now-dead image links that are found, see if archive.org happens to have a copy. And if there is an archived copy of the image, upload that archived image to phpBB's own image store, and replace the dead link with the now-local URL reference.

Maybe, this should even have the option to be a "constant" function. Not the archive.org part, but an option for phpBB to process every [img] type reference as soon as it's created and posted. And to upload the image into phpBB's own local image store, even before the image might become unavailable online. And then be able to refer to that local copy if and when the remote image host is no longer available.

Which looks like pretty much the opposite of what most existing extensions or mods out there did, which seem to be all about making it easier to upload attachments directly to third-party host services, rather than your local phpBB board.

Obviously it's opt-in, and would only be attractive to customers who like the idea of exchanging storage space for reliable images that don't disappear at the whim of some third-party advertisement-supported hosting service that goes out of business without notice.

But I would definitely use both aspects: Attempt resolving existing dead links using WayBackMachine, and prevent future dead links by creating "shadow uploads" that can be used if and when the remote host's image starts being unavailable.

User avatar
EA117
Registered User
Posts: 516
Joined: Wed Aug 15, 2018 3:23 am
Contact:

Re: Attempt Resolving Dead Image Links Using WayBackMachine

Post by EA117 » Fri Feb 22, 2019 7:09 am

EA117 wrote:
Fri Oct 26, 2018 5:11 am
Which looks like pretty much the opposite of what most existing extensions or mods out there did, which seem to be all about making it easier to upload attachments directly to third-party host services, rather than your local phpBB board.
Except for v12mike's work, which I was unaware of until now, as discussed in this thread.

Just leaving links to his https://github.com/v12mike/fetch-external-images and https://github.com/v12mike/imageredirect extensions here, for anyone looking for something similar, and maybe also thinking of using these extensions as a starting point to try and achieve the desired functionality.

User avatar
canonknipser
Registered User
Posts: 1932
Joined: Thu Sep 08, 2011 4:16 am
Location: Germany
Name: Frank Jakobs
Contact:

Re: Attempt Resolving Dead Image Links Using WayBackMachine

Post by canonknipser » Fri Feb 22, 2019 7:44 am

A lot people think so, but content available at the internet is not "public domain" in most cases.
Please be aware that copying foreign owned and hosted content to your own server may have severe legal issues, esp. regarding copyright and similar rights.
Greetings, Frank
phpbb.de support team member
English is not my native language - no support via PM or mail
New arrival - Extensions and scripts for phpBB

User avatar
EA117
Registered User
Posts: 516
Joined: Wed Aug 15, 2018 3:23 am
Contact:

Re: Attempt Resolving Dead Image Links Using WayBackMachine

Post by EA117 » Fri Feb 22, 2019 6:34 pm

This is a situation we're already in, too, unless your board is moderating and scrutinizing the copyright of all posted and attached content. Agreed, the solution for "don't allow content to die with the shutdown of some free hosting site" creates "more possibilities of" that situation actually occurring; meaning more cases where the content in question will have to be removed from your own site when requested. But you're already in a non-zero chance of that occurring even if you didn't implement this. And implementing this is what solves "everything else", for which there never was or won't ever be any copyright concern.

KYPREO
Registered User
Posts: 86
Joined: Fri Feb 02, 2018 9:56 am
Contact:

Re: Attempt Resolving Dead Image Links Using WayBackMachine

Post by KYPREO » Mon Feb 25, 2019 10:08 pm

I have done the same thing on my board, manually finding and replacing images with cached versions from the Wayback Machine. Wayback actually has its own API. I put a call out on v12mike's thread about potentially linking the Wayback API to the Fetch Images the extension. This would essentially add another few steps to v12mike's extension to firstly check whether a file is found. If not, the extension would then need to query archive.org and see if it is hosted there, then if it is, download the image and use that when doing the image replacement step.

My post is here: viewtopic.php?f=6&t=2429041&p=14933146#p14933146

Pretty please can someone make this?? :D

As an IP lawyer, I can think of some very good arguments why this practise is perfectly acceptable. If a user posts an embedded hyperlink to an image in which they own copyright (ie they took the photo) on a forum, they grant an implied licence to use and reproduction of that photo for the purposes for which they were originally posted. If the link goes dead, it would be within the scope of the implied licence to reproduce that photo on my server to restore the image as it was originally intended to be displayed.

Moreover, on my forum, users agreed to terms of use which granted an express irrevocable licence for me to use of photos they posted or hyperlinked on the forum. If they don't own the IP, then they grant me an indemnity for any loss suffered through reproduction of that image on the forum. So there's that too.

If it's 3rd party owned IP, that might be a different story. However, let's be honest the practical risk that a person will complain because I restored an image after the link has been dead for 15 years. Especially when they would need to bring an action against me in another country to obtain jurisdiction, and then wouldn't be able to do so because my servers are actually located in a 3rd jurisdiction. not going to happen.

Moreover, many jurisdictions provide fair use exemptions for exactly this kind of purpose. Specifically reproduction of images for the purposes of archival or backups - copying the photo to create a local backup on a server may fall within this defence.
phpBB user since 2002
www.AusRotary.com

Holger
Registered User
Posts: 1711
Joined: Tue Mar 12, 2002 3:54 pm
Location: Hannover

Re: Attempt Resolving Dead Image Links Using WayBackMachine

Post by Holger » Tue Mar 12, 2019 2:54 pm

KYPREO wrote:
Mon Feb 25, 2019 10:08 pm
Moreover, on my forum, users agreed to terms of use which granted an express irrevocable licence for me to use of photos they posted or hyperlinked on the forum. If they don't own the IP, then they grant me an indemnity for any loss suffered through reproduction of that image on the forum. So there's that too.
I dont think that is legally possible ...

KYPREO
Registered User
Posts: 86
Joined: Fri Feb 02, 2018 9:56 am
Contact:

Re: Attempt Resolving Dead Image Links Using WayBackMachine

Post by KYPREO » Tue Mar 12, 2019 11:50 pm

Holger wrote:
Tue Mar 12, 2019 2:54 pm
KYPREO wrote:
Mon Feb 25, 2019 10:08 pm
Moreover, on my forum, users agreed to terms of use which granted an express irrevocable licence for me to use of photos they posted or hyperlinked on the forum. If they don't own the IP, then they grant me an indemnity for any loss suffered through reproduction of that image on the forum. So there's that too.
I dont think that is legally possible ...
Speaking as an IP lawyer, it is legally possible - at least under Australian law and most jurisdictions. I see you are in Hannover. German copyright law has a bit of a unique feature that copyright resides with the creator and cannot be assigned - but this is not the usual situation in most countries and in any event ownership is different to licensing - in Germany, it is still possible to grant irrevocable royalty-free licences use works through acceptance of user terms and conditions.
phpBB user since 2002
www.AusRotary.com

Holger
Registered User
Posts: 1711
Joined: Tue Mar 12, 2002 3:54 pm
Location: Hannover

Re: Attempt Resolving Dead Image Links Using WayBackMachine

Post by Holger » Wed Mar 13, 2019 7:23 am

Nice, thank you for the clarification.

Post Reply

Return to “Extension Requests”