[3.2][3.3][BETA] Image Redirect

[v12mike]

Extension Name: Image Redirect
Author: v12mike

Extension Description: An extension for automatically redirecting links inserted with BBCode [IMG] tags
Extension Version: 2.0.1-b1

Extension Download: https://github.com/v12mike/imageredirec ... 0.1-b1.zip
Source Repository: github:repository: https://github.com/v12mike/imageredirect

Features:
This extension provides 2 main features, which may be used individually or together:

1. Secure image proxy redirection:
   • Eliminates insecure or mixed content browser warnings by redirecting external image links with http:// protocol to a secure proxy server
   • More efficient than the phpBB/sslAssets extension.
2. Local image store redirection:
   • Redirects external image links to a local store of harvested image files

Requirements:

1. Secure image proxy redirection:
   • This version requires phpBB 3.2.x
   • The secure image proxy feature requires a secure proxy server (e.g. a Camo Proxy) which is not part of this extension.
2. Local image store redirection:
   • This version requires phpBB 3.2.x
   • The redirection to a local image store feature requires that image files have been harvested and stored in a suitable format. Scripts for achieving this are available separately (see: viewtopic.php?f=6&t=2429041#p14776436).

[v12mike]

Explanation of the Secure Image Proxy feature::
If a phpBB board is served from a https:// server, all of the locally hosted content will be served over https:// and a browser will report it as a secure site, but any BBCode [IMG] links posted by users pointing to http://... will appear to browsers to be insecure content, in some browsers promoting a security warning dialogue, and in other browsers resulting in the image becoming inaccessible. It has been reported that insecure content has a negative impact on Google SEO.

An accepted solution for fixing mixed external content is to use a SSL proxy server to make the images appear to be secure. Camo is an example of such a proxy.

With this extension installed, when a phpBB page is being loaded by a user, [IMG] links to http://... images are rewritten by the phpBB renderer so that they become https:// links to the ssecure proxy server, with the original link address encoded into the new link. The user's browser then requests the image from the secure proxy which accesses the original hosting server and re-serves the content on-the-fly using the https:// protocol.

As it is assumed that the server hosting the phpBB board is able to support the https:// protocol, any http://... [IMG] links to the server hosting the phpBB board will be directly re-written to access the original server via https:// bypassing the camo proxy.

This phpBB extension does not supply the actual proxy service, you will need to install camo separately outside of phpBB (but perhaps on the same server), or use another SSL proxy service.

Explanation of the Local Image Store feature feature:
Some phpBB forums (particularly those which have been around for several years) have a large number of posts which contain inline images linked using the BBCode [IMG] tag. Over time a proportion of these images suffer "link rot" and disappear from the forum because the image is removed from the original external host, or the original host goes permanently off-line. This can cause significant anguish to users who expect the forum to provide a lasting historical record.

In some cases the images can be re-hosted or uploaded as attachments to the forum, but often this does not happen because the original poster has moved on (in one sense or another) or no copy of the image can be found for re-hosting. We have taken the approach of making a backup copy of all externally-hosted in-line images on our forum, which are saved to a local server directory.

This extension provides the last linkin the chain by redirecting external [IMG] tags to the locally hosted copy. If there is no locally hosted copy, the original link is used.

History of this extension:
This extension started life in 2016 as an attempt to improve the phpBB/sslAssets extension for redirecting http:// image links to a camo proxy server by adding configurability via the ACP. See v12mike/camosslimageproxy. After that extension was declared abandoned, I just used it on my own forums.

Then when Photobucket started hiding images that users had linked to our forums, I wrote some scripts that harvested externally hosted images for local storage. To display these local images, I added an extra mode to my existing extension and renamed it to Image Redirect. A number of forums have to my knowledge been using v1.x of this extension since then.

The original phpBB/sslAssets extension and v1.x of this extension worked by parsing the html output of the phpBB renderer, looking for html code that the renderer would bave produced from a BBCode [IMG] tag. While working on an idea for a different extension, I became more familiar with the s9e parser/renderer introduced into phpBB v3.2 and it became obvious that it would be more efficient to re-implement this extension to hook into the renderer, avoiding the html parsing operation. That rework has become v2.0 of this extension, and I thought it worth publishing it here.

[v12mike]

Installation:
You can install this on any release of phpBB 3.2 by following the steps below:

• In the `ext/` directory of your phpBB board, create a new directory named `v12mike/` (if it does not already exist) and navigate to it
• Create a directory 'imageredirect'.
• Copy the extension files into the directory 'ext/v12mike/imageredirect/'.
• Navigate in the ACP to `Customise -> Manage extensions`.
• Look for `Image Redirect` under the Disabled Extensions list, and click its `Enable` link.

Configuration of Secure Proxy mode:

• Configuration for 'camo' mode:
  This applies if you are using a camo proxy.
  • Navigate in the ACP to 'Extensions -> Image Redirect -> Proxy Settings'.
  • Ensure that 'Camo Mode' is selected and that 'Proxy Mode' is enabled
  • Enter the proxy address without protocol specifier or trailing / e.g. mydomain.com/camo (these are added automatically)
  • Enter the camo API key (as defined in your cam server configuration)
  • Click the "save proxy configuration" button
• Configuration for 'simple' mode:
  This applies if you are using an alternate (not camo) proxy.
  Note that the proxy url generated is of the form: https://my_proxy_domain/prefixoriginal_stripped_urlsuffix, where original_stripped_url is the original image url with the leading http:// stripped off.
  • Navigate in the ACP to 'Extensions -> Image Redirect -> Proxy Settings'.
  • Ensure that 'Simple Mode' is selected and that 'Proxy Mode' is enabled
  • Enter the proxy address myproxydomain/prefix (without protocol specifier) e.g. res.cloudinary.com/abcdefg/image/fetch/http:// Note that the leading http:// will be stripped off of the original image link, so should be added here if needed.
  • Enter any suffix to be appended to the url mysuffix as the camo API key (or leave it blank)
  • Click the "save proxy configuration" button

Configuration of Local Image Store mode:
This applies if you are using a local image store located (or symlinked) somewhere under the board root directory.

• Navigate in the ACP to 'Extensions -> Image Redirect -> Local Image Store Settings'.
• Ensure that 'Local Image Store Mode' is enabled
• Enter the path to your local image store directory (e.g. images/ext/)
• Click the "save local image store settings" button

[v12mike]

Installation of a camo proxy

First ensure that node.js is installed:

Installing nodejs needs one more thing before yum install nodejs

Code: Select all

# yum install -y gcc-c++ make
# curl -sL https://rpm.nodesource.com/setup_13.x | sudo -E bash -

First of all, You need to enable node.js yum repository in your system provided by the Node.js official website. You also need development tools to build native add-ons to be installed on your system.

Code: Select all

yum install nodejs

Install camo in /opt/camo (location not critical)

Code: Select all

cd /opt 
git clone https://github.com/atmos/camo.git
cd camo
npm install

Apache configuration
Add to the conf.d file of each virtual server that will be serving camo:
This causes the camo proxy to be served at /camo/

Code: Select all

# redirection of /camo/ to the local proxy
    RewriteRule ^/camo/(.*)$ balancer://camo/$1 [P,L] 
    <Proxy balancer://camo>
        BalancerMember http://127.0.0.1:8081
    </Proxy>

Startup scripts for camo
For Linux systems using systemd (e.g. centos 7): borrowed from: https://framasphere.org/posts/2410224

create the file /etc/systemd/system/camo.service containing:

Code: Select all

[Unit]
Description=camo
Documentation=https://example.com
After=network.target

[Service]
Type=simple
User=root
WorkingDirectory=/opt/camo/
Environment=CAMO_KEY=UbrUUVz3AFokgH7T
Environment=NODE_TLS_REJECT_UNAUTHORIZED=0
Environment=CAMO_LENGTH_LIMIT=10485760
Environment=CAMO_HEADER_VIA=Camo asset Proxy at frenchcarforum.co.uk
ExecStart=/usr/bin/node /opt/camo/server.js
Restart=always

[Install]
WantedBy=multi-user.target

For Linux systems notsupporting systemd but supporting init.d (e.g. centos 6), the following init.d script can be used:

Code: Select all

#! /bin/bash
#
# An init.d script for running a Node.js process as a service using Forever as
# the process monitor. For more configuration options associated with Forever,
# see: https://github.com/nodejitsu/forever
#
# This was written for Debian distributions such as Ubuntu, but should still
# work on RedHat, Fedora, or other RPM-based distributions, since none of the
# built-in service functions are used. So information is provided for both.
#
### BEGIN INIT INFO
# Provides:             my-application
# Required-Start:       $syslog $remote_fs
# Required-Stop:        $syslog $remote_fs
# Should-Start:         $local_fs
# Should-Stop:          $local_fs
# Default-Start:        2 3 4 5
# Default-Stop:         0 1 6
# Short-Description:    My Application
# Description:          My Application
### END INIT INFO
#
### BEGIN CHKCONFIG INFO
# chkconfig: 2345 55 25
# description: My Application
### END CHKCONFIG INFO
#
# Based on:
# https://gist.github.com/3748766
# https://github.com/hectorcorrea/hectorcorrea.com/blob/master/etc/forever-initd-hectorcorrea.sh
# https://www.exratione.com/2011/07/running-a-nodejs-server-as-a-service-using-forever/
#
# The example environment variables below assume that Node.js is installed by
# building from source with the standard settings.
#
# It should be easy enough to adapt to the paths to be appropriate to a package
# installation, but note that the packages available in the default repositories
# are far behind the times. Most users will be building from source to get a
# suitably recent Node.js version.
#
# An application name to display in echo text.
# NAME="My Application"
# The full path to the directory containing the node and forever binaries.
# NODE_BIN_DIR="/usr/local/node/bin"
# Set the NODE_PATH to the Node.js main node_modules directory.
# NODE_PATH="/usr/local/lib/node_modules"
# The application startup Javascript file path.
# APPLICATION_PATH="/home/user/my-application/start-my-application.js"
# Process ID file path.
# PIDFILE="/var/run/my-application.pid"
# Log file path.
# LOGFILE="/var/log/my-application.log"
# Forever settings to prevent the application spinning if it fails on launch.
# MIN_UPTIME="5000"
# SPIN_SLEEP_TIME="2000"
 
NAME="camo secure image proxy"
#NODE_BIN_DIR=""
#NODE_PATH=""
APPLICATION_PATH="/opt/camo/server.js"
PIDFILE="/var/run/camo.pid"
LOGFILE="/var/log/camo.log"
MIN_UPTIME="5000"
SPIN_SLEEP_TIME="2000"

 
# Add node to the path for situations in which the environment is passed.
#PATH=$NODE_BIN_DIR:$PATH
# Export all environment variables that must be visible for the Node.js
# application process forked by Forever. It will not see any of the other
# variables defined in this script.
#export NODE_PATH=$NODE_PATH
#export PORT=<%= @camo_port %>
#export CAMO_HEADER_VIA=<%= @camo_header_via %>
export CAMO_KEY="UbrUUVz3AFokgH7T"
#export CAMO_LENGTH_LIMIT=<%= @camo_length_limit %>
#export CAMO_LOGGING_ENABLED=<%= @camo_logging %>
#export CAMO_MAX_REDIRECTS=<%= @camo_mx_redirects %>
#export CAMO_SOCKET_TIMEOUT=<%= @camo_timeout %>
#export CAMO_TIMING_ALLOW_ORIGIN=<%= @camo_timing_cors %>
 
start() {
    echo "Starting $NAME"
    # We're calling forever directly without using start-stop-daemon for the
    # sake of simplicity when it comes to environment, and because this way
    # the script will work whether it is executed directly or via the service
    # utility.
    #
    # The minUptime and spinSleepTime settings stop Forever from thrashing if
    # the application fails immediately on launch. This is generally necessary to
    # avoid loading development servers to the point of failure every time
    # someone makes an error in application initialization code, or bringing down
    # production servers the same way if a database or other critical service
    # suddenly becomes inaccessible.
    #
    # The pidfile contains the child process pid, not the forever process pid.
    # We're only using it as a marker for whether or not the process is
    # running.
    #
    # Note that redirecting the output to /dev/null (or anywhere) is necessary
    # to make this script work if provisioning the service via Chef.
    forever \
      --pidFile $PIDFILE \
      -a \
      -l $LOGFILE \
      --minUptime $MIN_UPTIME \
      --spinSleepTime $SPIN_SLEEP_TIME \
      start $APPLICATION_PATH 2>&1 > /dev/null &
    RETVAL=$?
}
 
stop() {
    if [ -f $PIDFILE ]; then
        echo "Shutting down $NAME"
        # Tell Forever to stop the process.
        forever stop $APPLICATION_PATH 2>&1 > /dev/null
        # Get rid of the pidfile, since Forever won't do that.
        rm -f $PIDFILE
        RETVAL=$?
    else
        echo "$NAME is not running."
        RETVAL=0
    fi
}
 
restart() {
    stop
    start
}
 
status() {
    # On Ubuntu this isn't even necessary. To find out whether the service is
    # running, use "service my-application status" which bypasses this script
    # entirely provided you used the service utility to start the process.
    #
    # The commented line below is the obvious way of checking whether or not a
    # process is currently running via Forever, but in recent Forever versions
    # when the service is started during Chef provisioning a dead pipe is left
    # behind somewhere and that causes an EPIPE exception to be thrown.
    # forever list | grep -q "$APPLICATION_PATH"
    #
    # So instead we add an extra layer of indirection with this to bypass that
    # issue.
    echo `forever list` | grep -q "$APPLICATION_PATH"
    if [ "$?" -eq "0" ]; then
        echo "$NAME is running."
        RETVAL=0
    else
        echo "$NAME is not running."
        RETVAL=3
    fi
}
 
case "$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    status)
        status
        ;;
    restart)
        restart
        ;;
    *)
        echo "Usage: {start|stop|status|restart}"
        exit 1
        ;;
esac
exit $RETVAL

[HiFiKabin]

Sounds brilliant Mike. Downloading it now.

[AmigoJack]

The user's browser then requests the image from the secure proxy which accesses the original hosting server and re-serves the content on-the-fly using the https:// protocol.

I understand which problem(s) this extensions wants to resolve and I guess the vast majority of users want/love it. But practising this undermines security and SSL and has happened before and elsewhere - it will just reset the trust level over time back to before HTTPS became popular, because then every website can appear secure and under the hood it's accessing everything unsecured.

To whoever reads this: the concept of using SSL/HTTPS is not to "make the internet browser giving your website a green icon". If your website embeds HTTP resources then better decide between dumping those resources or not serving your website in HTTPS to begin with. Using proxies in this way is dishonest and wrong - the whole idea of internet browsers warning users about mixed content is nowadays broken because of most people eliminating symptoms instead of problems.

[v12mike]

Do you have a issue with the phpBB.com forums using the same technique? That is where I learned the concept.

[edit]
It is worth noting that this extension only modifies <img src=... > urls. This means that a standards compliant browser will only attempt to render the contents of a link as an image. it will not attempt to run any script that is hiding in the linked content.
[/edit]

[lopoto]

You can clearly describe how to configure for - Local Image Store mode:

for dummes please

[v12mike]

You can clearly describe how to configure for - Local Image Store mode:

for dummes please

I think that I would like to keep this topic only for discussion of the Image Redirect extension, not the associated image harvesting scripts, nor the configuration of a camo proxy server.

For the image harvesting scripts and setting up a local image store, seehttps://www.phpbb.com/community/viewtopic.php?p ... #p14776436 which is a fairly long topic, but should answer most of your questions.

For setting up a camo proxy, I will probably start another topic in the general discussion forum.

[RomaamoR]

And how does this extension differ from "Camo SSL Image Proxy Configuration" ?

[v12mike]

And how does this extension differ from "Camo SSL Image Proxy Configuration" ?

That is covered in the second post in this topic. This extension supercedes it.

[AmigoJack]

Do you have a issue with the phpBB.com forums using the same technique?

Yes, of course. When it started I thought Camo was used to anonymize URIs only, along with partly the same disadvantage.

[v12mike]

I just created version 2.0.0-b2 which has a 1 line change to fix a php warning. I would advise updating to this version if you are testing the extension.

[Ger]

The user's browser then requests the image from the secure proxy which accesses the original hosting server and re-serves the content on-the-fly using the https:// protocol.

I understand which problem(s) this extensions wants to resolve and I guess the vast majority of users want/love it. But practising this undermines security and SSL and has happened before and elsewhere - it will just reset the trust level over time back to before HTTPS became popular, because then every website can appear secure and under the hood it's accessing everything unsecured.

To whoever reads this: the concept of using SSL/HTTPS is not to "make the internet browser giving your website a green icon". If your website embeds HTTP resources then better decide between dumping those resources or not serving your website in HTTPS to begin with. Using proxies in this way is dishonest and wrong - the whole idea of internet browsers warning users about mixed content is nowadays broken because of most people eliminating symptoms instead of problems.

This is not entirely true.

Mixed content simply states that your browser is downloading a mix of encrypted and unencrypted sources. When you use a camo proxy like this, you only download encrypted sources. That results in a "green icon" like you write. That's true, but it's only a side-effect. The purpose of SSL is only to encrypt the connection and show information about the host. No more, no less. SSL does NOT mean everything is secure. If you download bogus over SSL, you still get bogus. One can easily load a hidden crypto miner overloading your CPU over SSL.

So when I visit a board about some medical condition and that has images hosted from other sources, with the board being server over SSL while the image host isn't, a MITM could still find out what image I downloaded. With an extension like this one enabled, the MITM could only find out that I'm requesting an image though the proxy without any knowledge about the original source.

That's what this extension is about. Not about that green icon you refer to.

[AmigoJack]

If you download bogus over SSL, you still get bogus. One can easily load a hidden crypto miner overloading your CPU over SSL.

This is so obvious I don't even care for people thinking HTTPS prevents viruses.

With an extension like this one enabled, the MITM could only find out that I'm requesting an image though the proxy without any knowledge about the original source.

With an extension like this no one will ever find out a MITM attack for the HTTP download - you put that into a box with the only goal to serve HTTPS, but neglect analyzing what happened before. That's what's wrong with these kinds of proxies - I as the end user have no chance to see if it happened on the server already, but using a mixed content website (requesting all resources directly) I also have more chances to see what happens.