[ABD] Camo SSL Image Proxy

[v12mike]

Extension Name: Camo SSL Image Proxy
Note that this topic is describing V1.1.2 of the extension. An earlier version V1.0.1 has been around for a while (and is in use on the forum that you are reading now), but had hard-coded configuration, has never had a topic here and has never really been promoted

Authors: Patrick Webster, phpBB Website Team, Michael Cullum, v12mike

Extension Description:For phpBB boards that use the secure (https://) protocol, this extension rewrites image URLs so that all external image links are (as seen by a user) secure. The extension requires a Camo image proxy server (other suitable proxy server).

Extension Version: 1.1.2

phpBB Compatibility:
This extension is compatible with all released versions of phpBB v3.1 and with v3.2.0-rc1.
It has been tested with phpBB v3.1.10 and v3.2.0-rc1.

Screenshots:




Extension Download: https://github.com/v12mike/camosslimage ... master.zip
The download link is currently for my fork of the project, but (assuming that the pull request is accepted) it will be merged into the phpbb master branch.

Background Information:
If a phpBB board is served from a https:// server, it will generally behave well as a secure site, but any image links posted by users as http://... will appear to browsers to be insecure content, in some browsers promoting a security warning dialogue, and in other browsers resulting in the image becoming inaccessible.

A direct solution of converting the image links in the phpBB database is generally impractical, so an accepted solution is to use a SSL proxy to make the images appear to be secure. Camo is an example of such a proxy.

With this extension installed, when a phpBB page is being loaded by a user, links to http://... images are rewritten so that they become https:// links to the camo proxy server, with the original link address encoded into the new link. The user's browser then requests the image from the camo proxy which accesses the original location and re-serves it on-the-fly using the https:// protocol.

Some http://... links (to servers known to directly support https:// access) may be directly re-written to access the original server via https:// bypassing the camo proxy.

This phpBB extension does not supply the actual proxy service, you will need to install camo separately outside of phpBB (but perhaps on the same server), or use another SSL proxy service.

[v12mike]

Pre-requisites for Installation:
You should have configured the web server that is serving your phpBB board to support the https:// protocol. There are plenty of resources on the web for helping with this. I recommend the free SSL certificates from https://letsencrypt.org.

You also should have access to an SSL image proxy server. Depending on your server configuration this may be installed on the same server that runs your forum, or maybe a different server.

The SSL image proxy will usually be a camo proxy. A later post in this topic will give some hints on the installation and configuration of camo proxy, but this is not a support forum for that software.

This extension also supports another mode of operation called 'Simple Mode' which allows operation with a commercial service (cloudinary.com) which can act as an SSL proxy. That service may be useful for testing or for those who cannot, or do not want to, configure their own SSL proxy server. Other commercial or free SSL proxy services may be available.

Installation:
You can install this on the latest release of phpBB 3.1 by following the steps below:

• In the `ext` directory of your phpBB board, create a new directory named `phpbb` (if it does not already exist) and navigate to it
• `git clone [email protected]:phpbb-extensions/camo-ssl-image-proxy.git`
• Navigate in the ACP to `Customise -> Manage extensions`.
• Look for `Camo SSL Image Proxy` under the Disabled Extensions list, and click its `Enable` link.

Configuration for 'Camo Mode':
This applies if you are using a camo proxy.

• Navigate in the ACP to 'Extensions -> Camo SSL Image Proxy -> Configure'.
• Ensure that 'Camo Mode' is selected and that 'Image Proxy Enable' is selected
• Enter the proxy address (without protocol specifier or trailing /) e.g. mydomain.com/camo
• Enter the camo API key (as applicable)

Configuration for 'Simple Mode':
This applies if you are using an alternate proxy.
Note that the proxy url generated is of the form: https://myproxydomain/prefixoriginalstrippedurlmysuffix, where originalstrippedurl is the original image url with the leading http:// stripped off.

• Navigate in the ACP to 'Extensions -> Camo SSL Image Proxy -> Configure'.
• Ensure that 'Simple Mode' is selected and that 'Image Proxy Enable' is selected
• Enter the proxy address myproxydomain/prefix (without protocol specifier) e.g. res.cloudinary.com/abcdefg/image/fetch/http://
• Note that the leading http:// will be stripped off of the original image link, so should be added here if needed.
• Enter any string to be appended to the url mysuffix as the camo API key (or leave it blank)

Configuration of directly mapped domains:
To save load on the proxy server, some domains (which must themselves support the https:// protocol) can be configured so that image files hosted on those domains can be directly fetched by rewriting the image url from http:// to https://

• Navigate in the ACP to 'Extensions -> Camo SSL Image Proxy -> Configure'.
• Add your sites domain(s) to the Directly Mapped Domains list (without protocol specifier or trailing /) e.g. mydomain.com
• Add any other domain known to support https:// (limit 200 entries)

Note that there is a small additional processing overhead for each domain in the list, so it is recommended to only add a few domains that are commonly used for remote hosting of images on your board.

Enable or Disable 'locations':

'Locations' are phpBB template variables which are scanned by this extension to find insecure image urls to remap.

The locations list includes 'core' locations which are part of the phpBB distribution, and always enabled. There are also pre-defined locations for other extensions that are known to this extension. These are disabled by default, but should be enabled if you have the corresponding extension installed.

If a phpBB board is running other extensions (especially those that display avatars), it may be that the standard dataset of this extension needs to be augmented with extra 'locations'. This can be done automatically using 'Training Mode'.

Each location (except for the predefined core locations) may be enabled or disabled. A disabled location is still in the database, but http:// urls in that location will not be remapped. This is mostly used for testing purposes.

Automatically Training the extension for extra 'locations' (recommended):

In 'Training Mode' This extension scans the template data structures (after rewriting known insecure url locations) and if it finds any, it adds them to the database itself. In Training Mode, any disabled locations will be automatically re-enabled when a resulting insecure image url is detected.

Training mode only operates when pages are loaded by users with admin permissions.

As an admin, you will see training mode operate when you will load a page (e.g. for a newly installed extension) and see that it has mixed content warnings. The first loading of the page (in training mode) will train this extension, and a page refresh should show that the viewed page is now secure.

Configuring extra 'locations':
You can manually add extra template locations for this extension to search. You will probably need some developer knowledge and perusal of the extension source code to find the correct values to add for any particular extension. It is probably best to ask here first.
To add a 'location' fill in the 3 fields and click the 'Add Location' button at the right hand enc of the row.

[v12mike]

Camo Proxy Installation

This post contains notes on how I installed the camo proxy itself on Centos 6 / Apache. I only include it here because there are not many tutorials available on camo installation (and most of those assume the installation of Docker or other virtualisation). These instructions should also work with minor changes on other Linux distributions.

1. Ensure that node.js is installed on the server (not described here)
2. Install forever (a utility to ensure that the proxy is restarted should it die for any reason).
3. Install camo
4. Configure the apache virtual server
5. Add an init.d script to control the camo proxy.

Install forever
Install (if necessary) forever.

Code: Select all

npm install forever -g 

Install camo
the camo installation could be in /opt/camo

Code: Select all

cd /opt 
git clone https://github.com/atmos/camo.git
cd camo
npm install

camo can be manually started with

Code: Select all

screen -S camo sudo  CAMO_KEY="<camo-key>" forever server.js

<camo-key> can be any string that you like to use, and is also configured in the phpBB extension ACP.

Apache virtual server configuration
Add to the conf.d file of the virtual server(s) that will be serving camo (no need to edit anything here):

Code: Select all

# redirection of /camo/ to the local proxy
    RewriteRule ^/camo/(.*)$ balancer://camo/$1 [P,L] 
    <Proxy balancer://camo>
        BalancerMember http://127.0.0.1:8081
    </Proxy>

This causes the camo proxy to be served at mydomain/camo/

An init.d script for the camo proxy
remember to customise the camo key and perhaps file locations.

Code: Select all

#! /bin/bash
#
# An init.d script for running a Node.js process as a service using Forever as
# the process monitor. For more configuration options associated with Forever,
# see: https://github.com/nodejitsu/forever
#
# This was written for Debian distributions such as Ubuntu, but should still
# work on RedHat, Fedora, or other RPM-based distributions, since none of the
# built-in service functions are used. So information is provided for both.
#
### BEGIN INIT INFO
# Provides:             my-application
# Required-Start:       $syslog $remote_fs
# Required-Stop:        $syslog $remote_fs
# Should-Start:         $local_fs
# Should-Stop:          $local_fs
# Default-Start:        2 3 4 5
# Default-Stop:         0 1 6
# Short-Description:    My Application
# Description:          My Application
### END INIT INFO
#
### BEGIN CHKCONFIG INFO
# chkconfig: 2345 55 25
# description: My Application
### END CHKCONFIG INFO
#
# Based on:
# https://gist.github.com/3748766
# https://github.com/hectorcorrea/hectorcorrea.com/blob/master/etc/forever-initd-hectorcorrea.sh
# https://www.exratione.com/2011/07/running-a-nodejs-server-as-a-service-using-forever/
#
# The example environment variables below assume that Node.js is installed by
# building from source with the standard settings.
#
# It should be easy enough to adapt to the paths to be appropriate to a package
# installation, but note that the packages available in the default repositories
# are far behind the times. Most users will be building from source to get a
# suitably recent Node.js version.
#
# An application name to display in echo text.
# NAME="My Application"
# The full path to the directory containing the node and forever binaries.
# NODE_BIN_DIR="/usr/local/node/bin"
# Set the NODE_PATH to the Node.js main node_modules directory.
# NODE_PATH="/usr/local/lib/node_modules"
# The application startup Javascript file path.
# APPLICATION_PATH="/home/user/my-application/start-my-application.js"
# Process ID file path.
# PIDFILE="/var/run/my-application.pid"
# Log file path.
# LOGFILE="/var/log/my-application.log"
# Forever settings to prevent the application spinning if it fails on launch.
# MIN_UPTIME="5000"
# SPIN_SLEEP_TIME="2000"
 
NAME="camo secure image proxy"
#NODE_BIN_DIR=""
#NODE_PATH=""
APPLICATION_PATH="/opt/camo/server.js"
PIDFILE="/var/run/camo.pid"
LOGFILE="/var/log/camo.log"
MIN_UPTIME="5000"
SPIN_SLEEP_TIME="2000"

# Add node to the path for situations in which the environment is passed.
#PATH=$NODE_BIN_DIR:$PATH
# Export all environment variables that must be visible for the Node.js
# application process forked by Forever. It will not see any of the other
# variables defined in this script.
#export NODE_PATH=$NODE_PATH
#export PORT=<%= @camo_port %>
#export CAMO_HEADER_VIA=<%= @camo_header_via %>
export CAMO_KEY="MySecretKey..."
#export CAMO_LENGTH_LIMIT=<%= @camo_length_limit %>
#export CAMO_LOGGING_ENABLED=<%= @camo_logging %>
#export CAMO_MAX_REDIRECTS=<%= @camo_mx_redirects %>
#export CAMO_SOCKET_TIMEOUT=<%= @camo_timeout %>
#export CAMO_TIMING_ALLOW_ORIGIN=<%= @camo_timing_cors %>
 
start() {
    echo "Starting $NAME"
    # We're calling forever directly without using start-stop-daemon for the
    # sake of simplicity when it comes to environment, and because this way
    # the script will work whether it is executed directly or via the service
    # utility.
    #
    # The minUptime and spinSleepTime settings stop Forever from thrashing if
    # the application fails immediately on launch. This is generally necessary to
    # avoid loading development servers to the point of failure every time
    # someone makes an error in application initialization code, or bringing down
    # production servers the same way if a database or other critical service
    # suddenly becomes inaccessible.
    #
    # The pidfile contains the child process pid, not the forever process pid.
    # We're only using it as a marker for whether or not the process is
    # running.
    #
    # Note that redirecting the output to /dev/null (or anywhere) is necessary
    # to make this script work if provisioning the service via Chef.
    forever \
      --pidFile $PIDFILE \
      -a \
      -l $LOGFILE \
      --minUptime $MIN_UPTIME \
      --spinSleepTime $SPIN_SLEEP_TIME \
      start $APPLICATION_PATH 2>&1 > /dev/null &
    RETVAL=$?
}
 
stop() {
    if [ -f $PIDFILE ]; then
        echo "Shutting down $NAME"
        # Tell Forever to stop the process.
        forever stop $APPLICATION_PATH 2>&1 > /dev/null
        # Get rid of the pidfile, since Forever won't do that.
        rm -f $PIDFILE
        RETVAL=$?
    else
        echo "$NAME is not running."
        RETVAL=0
    fi
}
 
restart() {
    stop
    start
}
 
status() {
    # On Ubuntu this isn't even necessary. To find out whether the service is
    # running, use "service my-application status" which bypasses this script
    # entirely provided you used the service utility to start the process.
    #
    # The commented line below is the obvious way of checking whether or not a
    # process is currently running via Forever, but in recent Forever versions
    # when the service is started during Chef provisioning a dead pipe is left
    # behind somewhere and that causes an EPIPE exception to be thrown.
    # forever list | grep -q "$APPLICATION_PATH"
    #
    # So instead we add an extra layer of indirection with this to bypass that
    # issue.
    echo `forever list` | grep -q "$APPLICATION_PATH"
    if [ "$?" -eq "0" ]; then
        echo "$NAME is running."
        RETVAL=0
    else
        echo "$NAME is not running."
        RETVAL=3
    fi
}
 
case "$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    status)
        status
        ;;
    restart)
        restart
        ;;
    *)
        echo "Usage: {start|stop|status|restart}"
        exit 1
        ;;
esac
exit $RETVAL

[david63]

... create a new directory named `phpbb` (if it does not already exist)

I may be wrong about this but to the best of my knowledge only extensions created by the phpBB extensions team can use "phpbb" as the vendor name.

[Cin-]

Is this 3.2 compatible?

[v12mike]

... create a new directory named `phpbb` (if it does not already exist)

I may be wrong about this but to the best of my knowledge only extensions created by the phpBB extensions team can use "phpbb" as the vendor name.

This extension was created by the phpbb team. I have added the ACP configuration page and made some other enhancements. I was advised here to update the existing extension.

Is this 3.2 compatible?

Not yet.

[edit] It is now

I have labelled this version of the extension as being "alpha", but it has been in use on a couple of live forums for a while and appears to be quite stable.

[3Di]

I may be wrong about this but to the best of my knowledge only extensions created by the phpBB extensions team can use "phpbb" as the vendor name.

Yes,"phpBB" and "core" aren't allowed as the vendor names. AFAIK.

This case seems to be a special one. v12mike created a PR on github against the phpBB repo of this extension and published here the whole result of his job to get more audience and feedback. Well done by my side.

[v12mike]

Using the camo SSL Image Proxy extension without camo
For phpBB installations where it is difficult to set up a camo proxy (e.g. shared hosting), or just for testing purposes, the extension can be configured to use other proxy services.

I have successfully used https://cloudinary.com (no affiliation) for this purpose. Their free account tier provides enough capacity for testing, and probably enough for ongoing use by a smallish forum.

To configure this extension to use cloudinary as a proxy:

1. go to https://cloudinary.com site and create a free account.
2. on the cloudinary dashboard page near the top find the string called "cloud name" (typically a random string of letters).
3. in the phpBB camoimageproxy extension configuration page, set the mode to 'simple mode'
4. in the phpBB camoimageproxy extension configuration page set the proxy address to be: res.cloudinary.com/abcdefg/image/fetch/http:// where abcdefg is the cloud name found above.

It should then 'just work'.
Cloudinary has the additional benefit that all proxied images are available in the media library page of your account, and you can see statistics for how many images have been proxied.

Testing the Camo SSL Image Proxy extension without SSL support
You can even test the camoimageproxy extension before upgrading you server to support https:// (I don't think that there is any benefit to this configuration on a production server).

Just set up the extension following the instructions in the posts above (for a camo or cloudinary proxy), and check that images are displayed correctly. You can also use a browser to check the url rewriting. right click on an image and select "inspect element" from the menu.

For a board configured to use a camo proxy you should see something like:

Code: Select all

<img src="https://mydomain.com/camo/d7441df8309e2f6f8be183201fc2de56367a3582/687474703a2f2f69636865662d312e626263692e636f2e756b2f6e6577732f3636302f63707370726f6470622f384533462f70726f64756374696f6e2f5f38353835313436335f32656161643336392d343562622d343061302d613263392d3761306166326665663236382e6a7067" class="postimage" alt="Image">

For a board configured to use cloudinary you should see something like:

Code: Select all

 <img src="https://res.cloudinary.com/abcdefg/image/fetch/http://i997.photobucket.com/albums/af93/tcfcfphotos/be1d7160.jpg" class="postimage" alt="Image">

[HiFiKabin]

Is there any change of making http://images.weserv.nl/ work with this?

The image url

http://s21.postimg.org/g21720t9j/image.jpg

has to be change to:-

https://images.weserv.nl/?url=//s21.postimg.org/g21720t9j/image.jpg

[v12mike]

Is there any change of making http://images.weserv.nl/ work with this?

The image url

http://s21.postimg.org/g21720t9j/image.jpg

has to be change to:-

https://images.weserv.nl/?url=//s21.postimg.org/g21720t9j/image.jpg

Yes that works. I just tried it on my test board. https://www.xantiav6.co.uk/forum/viewto ... =2&t=56380

I just added postimg.org to the configured list of directly mapped domains.

[HiFiKabin]

Is there any change of making http://images.weserv.nl/ work with this?

The image url

http://s21.postimg.org/g21720t9j/image.jpg

has to be change to:-

https://images.weserv.nl/?url=//s21.postimg.org/g21720t9j/image.jpg

Yes that works. I just tried it on my test board. https://www.xantiav6.co.uk/forum/viewto ... =2&t=56380

I just added postimg.org to the configured list of directly mapped domains.

You misunderstood my question. http://images.weserv.nl/ is the ssl proxy I would like to use.

[v12mike]

You misunderstood my question. http://images.weserv.nl/ is the ssl proxy I would like to use.

Yes, I misunderstood. The link format that you require is not currently supported. The closest is

Code: Select all

"https://images.weserv.nl/?url=http://s21.postimg.org/g21720t9j/image.jpg"

which (annoyingly) does not work.

There are a couple of options for fixing this. I will put it onto my enhancements list, do you need a rapid fix?

[HiFiKabin]

You misunderstood my question. http://images.weserv.nl/ is the ssl proxy I would like to use.

Yes, I misunderstood. The link format that you require is not currently supported. The closest is

Code: Select all

"https://images.weserv.nl/?url=http://s21.postimg.org/g21720t9j/image.jpg"

which (annoyingly) does not work.

There are a couple of options for fixing this. I will put it onto my enhancements list, do you need a rapid fix?

Not at all. I would just be grateful if you can get it to work.

TiA

[v12mike]

There are a couple of options for fixing this. I will put it onto my enhancements list, do you need a rapid fix?

Not at all. I would just be grateful if you can get it to work.

The difficulty is in deciding whether to fix this by just stripping the 'http' out of the link, which would work for weserve, and could work with cloudinary, or to make the configuration more generic by allowing configuration of a regex, or to have preconfigured buttons for each proxy service so that configuration for known services is foolproof?

[HiFiKabin]

I would guess a preconfigured button would give the most problems to make, but give the least support questions once its made.