Page 2 of 3

Re: Forcing non-www and https redirects correctly

Posted: Wed Aug 14, 2019 5:11 pm
by urzh
One question around this, I set up this redirect on other hosts too (Non-phpBB sites) and ocassionally I'm getting the redirect to "mysite.com/index.php". How can I make it just go to "mysite.com" ?

Re: Forcing non-www and https redirects correctly

Posted: Wed Aug 14, 2019 5:13 pm
by urzh
Yes, I added the following lines in addition to the rewire rules:

Code: Select all

RewriteEngine on
RewriteCond %{HTTPS} !=on [OR]
RewriteCond %{HTTP_HOST} ^www\.funnyinterestingcool\.com$ [NC]
RewriteRule ^(.*)$ https\:\/\/funnyinterestingcool\.com\/$1 [L,R=301]


RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule ^(.*)$ app.php [QSA,L]

Re: Forcing non-www and https redirects correctly

Posted: Wed Aug 14, 2019 5:15 pm
by Mick
Please use the Quote function only when it is needed to make your reply clear and only quote the parts necessary not the whole post every time. Excessive quoting (especially of long messages and/or multiple images) makes reading the post extremely difficult and tiresome.

Re: Forcing non-www and https redirects correctly

Posted: Wed Aug 14, 2019 7:09 pm
by urzh
So for instance, if I type in "mysite.com/333" to redirects to mysite.com/index.php but I want to redirect to mysite.com.

Re: Forcing non-www and https redirects correctly

Posted: Wed Aug 14, 2019 7:15 pm
by EA117
urzh wrote:
Wed Aug 14, 2019 7:09 pm
So for instance, if I type in "mysite.com/333" to redirects to mysite.com/index.php but I want to redirect to mysite.com.
Yikes. I had an answer until you said that. 😝

So there is a non-phpBB site where you have made .htaccess rules, and hitting that site with a URL like http://mysite.com/333 ends up wanting to access http://mysite.com/index.php ?

Then I would have to assume there is a different rule in the .htaccess that is doing that to you. If index.php wasn't in the original URL, it didn't "come out of nowhere." The HTTPS and WWW subdomain rule we've discussed and implemented has no bias towards any resource, and just "repeats whatever path was in the URL, if any." (Which would be /333 in this example.)

In addition to looking at the other rules in the .htaccess of this non-phpBB site, also consider that this could be behavior from an error document configured on the web server. e.g. Maybe a 404.html or 403.html getting triggered as intended by the web server configuration, and the content of one of those documents is what has a redirect for "send this user trying to access an invalid path to the front page." But the document contains a wrong assumption of index.php being the front page of the site.

Re: Forcing non-www and https redirects correctly

Posted: Wed Aug 14, 2019 7:27 pm
by urzh
Hi,
So since implementing the new re-write rules, I'm noticing the index.php hits more in Google Analytics and I got a warning there for the same sites saying it's traffic appearing from multiple sources. I enabled re-directs in my backend and I'm monitoring.

And yes, correct.
EA117 wrote:
Wed Aug 14, 2019 7:15 pm
So there is a non-phpBB site where you have made .htaccess rules, and hitting that site with a URL like http://mysite.com/333 ends up wanting to access http://mysite.com/index.php ?

This is my full .htaccess where I updated it and I'm having the issues. It's from a Joomla site and the rules are at the bottom of the file. I hope this isn't getting out of scope.


Code: Select all

##
# @package    Joomla
# @copyright  Copyright (C) 2005 - 2016 Open Source Matters. All rights reserved.
# @license    GNU General Public License version 2 or later; see LICENSE.txt
##

##
# READ THIS COMPLETELY IF YOU CHOOSE TO USE THIS FILE!
#
# The line 'Options +FollowSymLinks' may cause problems with some server configurations.
# It is required for the use of mod_rewrite, but it may have already been set by your 
# server administrator in a way that disallows changing it in this .htaccess file.
# If using it causes your site to produce an error, comment it out (add # to the 
# beginning of the line), reload your site in your browser and test your sef urls. If 
# they work, then it has been set by your server administrator and you do not need to 
# set it here.
##

## No directory listings
IndexIgnore *

## Can be commented out if causes errors, see notes above.
Options +FollowSymlinks
Options -Indexes

## Mod_rewrite in use.

RewriteEngine On

## Begin - Rewrite rules to block out some common exploits.
# If you experience problems on your site then comment out the operations listed 
# below by adding a # to the beginning of the line.
# This attempts to block the most common type of exploit `attempts` on Joomla!
#
# Block any script trying to base64_encode data within the URL.
RewriteCond %{QUERY_STRING} base64_encode[^(]*\([^)]*\) [OR]
# Block any script that includes a <script> tag in URL.
RewriteCond %{QUERY_STRING} (<|%3C)([^s]*s)+cript.*(>|%3E) [NC,OR]
# Block any script trying to set a PHP GLOBALS variable via URL.
RewriteCond %{QUERY_STRING} GLOBALS(=|\[|\%[0-9A-Z]{0,2}) [OR]
# Block any script trying to modify a _REQUEST variable via URL.
RewriteCond %{QUERY_STRING} _REQUEST(=|\[|\%[0-9A-Z]{0,2})
# Return 403 Forbidden header and show the content of the root homepage
RewriteRule .* index.php [F]
#
## End - Rewrite rules to block out some common exploits.

## Begin - Custom redirects
#
# If you need to redirect some pages, or set a canonical non-www to
# www redirect (or vice versa), place that code here. Ensure those
# redirects use the correct RewriteRule syntax and the [R=301,L] flags.
#
## End - Custom redirects

##
# Uncomment the following line if your webserver's URL
# is not directly related to physical file paths.
# Update Your Joomla! Directory (just / for root).
##

# RewriteBase /

# Extra Security Headers
<IfModule mod_headers.c>
	Header set X-XSS-Protection "1; mode=block"
	Header always append X-Frame-Options SAMEORIGIN
	Header set X-Content-Type-Options nosniff
</IfModule>

## Begin - Joomla! core SEF Section.
#
RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
#
# If the requested path and file is not /index.php and the request
# has not already been internally rewritten to the index.php script
RewriteCond %{REQUEST_URI} !^/index\.php
# and the requested path and file doesn't directly match a physical file
RewriteCond %{REQUEST_FILENAME} !-f
# and the requested path and file doesn't directly match a physical folder
RewriteCond %{REQUEST_FILENAME} !-d
# internally rewrite the request to the index.php script
RewriteRule .* index.php [L]
#
## End - Joomla! core SEF Section.


<Files 403.shtml>

order allow,deny

allow from all

</Files>


ErrorDocument 400 /400.shtml
ErrorDocument 401 /401.shtml
ErrorDocument 403 /403.shtml
ErrorDocument 404 /404.shtml
ErrorDocument 500 /500.shtml

AddOutputFilterByType DEFLATE text/text text/html text/plain text/xml text/css application/x-javascript application/javascript text/javascript


RewriteEngine on
RewriteCond %{HTTPS} !=on [OR]
RewriteCond %{HTTP_HOST} ^www\.becomethesolution\.com$ [NC]
RewriteRule ^(.*)$ https\:\/\/becomethesolution\.com\/$1 [L,R=301]

Re: Forcing non-www and https redirects correctly

Posted: Wed Aug 14, 2019 7:35 pm
by EA117
urzh wrote:
Wed Aug 14, 2019 7:27 pm
This is my full .htaccess where I updated it and I'm having the issues.
Intentionally being done by the ## Begin/End - Joomla! core SEF Section block in that .htaccess. It's has an intention for Joomla similar to what the APP.PHP rule does for phpBB. If the URL wasn't already for index.php, and if the URL doesn't refer to a non-index.php file or path that actually exists, then re-write the URL to be for index.php instead.

So "/333" in the prior example must have been an invalid path, but instead of letting it hit any error document, the Joomla rule in the .htaccess intentionally says "send this guy to the Joomla index.php instead."

If it's a Joomla site -- and the point at which we do go completely out of scope -- then "I have no idea why this results in a bad behavior on your site." If it's a Joomla site, why isn't there an index.php? Or if there isn't supposed to be an index.php, why are the Joomla .htaccess rules written as though there is supposed to be one?

If you have intentionally or manually changed the site's "main" document to be something other than index.php, then maybe all that's needed here is to update the "index.php" reference in this rule, to instead use whatever your actual main page name is.

EDIT: Note that index.php is also the chosen destination of the ## Begin/End - Rewrite rules to block out some common exploits block that appears earlier in the file. This one is reacting to more exotic conditions that are not being triggered by the "/333" example. But, if you determine that you simply needed to change index.php to some other document name, you'll want to make that update in this ## Begin/End - Rewrite rules to block out some common exploits block as well. Just in case this "common exploits" rule ever does engage.

Re: Forcing non-www and https redirects correctly

Posted: Wed Aug 14, 2019 10:22 pm
by urzh
All I know is that this behavior has started since I update to:

Code: Select all

RewriteEngine on
RewriteCond %{HTTPS} !=on [OR]
RewriteCond %{HTTP_HOST} ^www\.funnyinterestingcool\.com$ [NC]
RewriteRule ^(.*)$ https\:\/\/funnyinterestingcool\.com\/$1 [L,R=301]
from:

Code: Select all

RewriteEngine On
RewriteCond %{HTTP_HOST} ^www\.
RewriteCond %{HTTPS}s ^on(s)|off
RewriteCond http%1://%{HTTP_HOST} ^(https?://)(www\.)?(.+)$
RewriteRule ^ %1%3%{REQUEST_URI} [R=301,L]
So im going to tests to see now

Re: Forcing non-www and https redirects correctly

Posted: Thu Aug 15, 2019 12:07 am
by urzh
So it looks like the behavior is right. When someone types in a 404 page that was for instance http://mysite.com/old-page it redirects to mysite.com/index.php. Now if I could just get it to go to mysite.com without the index.php appeneded that would be best.

Re: Forcing non-www and https redirects correctly

Posted: Thu Aug 15, 2019 12:18 am
by urzh
So how can I fix this so that non-404 pages redirect to the HTTPS versions?

Re: Forcing non-www and https redirects correctly

Posted: Thu Aug 15, 2019 3:01 am
by EA117
urzh wrote:
Thu Aug 15, 2019 12:07 am
So it looks like the behavior is right. When someone types in a 404 page that was for instance http://mysite.com/old-page it redirects to mysite.com/index.php. Now if I could just get it to go to mysite.com without the index.php appeneded that would be best.
Okay, I think I understand better now. One problem right now is that the Joomla-created rules are not external redirects. So actually the new HTTPS and WWW subdomain rule needed to go before these existing Joomla-created rules. (Right after the first RewriteEngine On line would be fine, before the start of the ## Begin - Rewrite rules to block out some common exploits block.)

This is going to make it such that if an HTTPS or WWW subdomain correct needs to occur, this correction will happen first, and a redirect will be sent back to the browser to access the requested URL using the correct protocol and/or domain.

Now, once the incoming request has been changed from http://www.mysite.com/333 to https://mysite.com/333 or whatever, the new HTTPS and WWW subdomain rule won't engage against that URL again, because both conditions are already met. And so the .htaccess moves on to applying the existing Joomla rules that have been there all along, which react to the fact that /333 is not a file or directory which actually exists.

I believe the fact that the new HTTPS and WWW subdomain rule was placed at the end of the file, after existing Joomla-defined rules, is what caused new undesired behavior to occur. A successfully Joomla-rule-rewritten index.php URL was probably passing through the .htaccess rules twice, and ended up not hitting the new HTTPS and WWW subdomain rule until it's second pass. Which is how the index.php reference -- which was intended to be part of an internal-only redirect -- then ended up in the browser's visible URL as part of the R=301 redirect.

Re: Forcing non-www and https redirects correctly

Posted: Thu Aug 15, 2019 1:12 pm
by urzh
Hi,
Ok so you think it's the ordering in my .htaccess file. Can you help adjust the rules accordingly then? I think I am at a lost otherwise.

Re: Forcing non-www and https redirects correctly

Posted: Thu Aug 15, 2019 1:52 pm
by EA117
Moving the rule that you added, such that it appears right before the ## Begin - Rewrite rules to block out some common exploits line. The rest of the file remains unchanged:

Code: Select all

##
# @package    Joomla
# @copyright  Copyright (C) 2005 - 2016 Open Source Matters. All rights reserved.
# @license    GNU General Public License version 2 or later; see LICENSE.txt
##

##
# READ THIS COMPLETELY IF YOU CHOOSE TO USE THIS FILE!
#
# The line 'Options +FollowSymLinks' may cause problems with some server configurations.
# It is required for the use of mod_rewrite, but it may have already been set by your 
# server administrator in a way that disallows changing it in this .htaccess file.
# If using it causes your site to produce an error, comment it out (add # to the 
# beginning of the line), reload your site in your browser and test your sef urls. If 
# they work, then it has been set by your server administrator and you do not need to 
# set it here.
##

## No directory listings
IndexIgnore *

## Can be commented out if causes errors, see notes above.
Options +FollowSymlinks
Options -Indexes

## Mod_rewrite in use.

RewriteEngine On

RewriteCond %{HTTPS} !=on [OR]
RewriteCond %{HTTP_HOST} ^www\.becomethesolution\.com$ [NC]
RewriteRule ^(.*)$ https\:\/\/becomethesolution\.com\/$1 [L,R=301]

## Begin - Rewrite rules to block out some common exploits.
# If you experience problems on your site then comment out the operations listed 
# below by adding a # to the beginning of the line.
# This attempts to block the most common type of exploit `attempts` on Joomla!
#
# Block any script trying to base64_encode data within the URL.
RewriteCond %{QUERY_STRING} base64_encode[^(]*\([^)]*\) [OR]
# Block any script that includes a <script> tag in URL.
RewriteCond %{QUERY_STRING} (<|%3C)([^s]*s)+cript.*(>|%3E) [NC,OR]
# Block any script trying to set a PHP GLOBALS variable via URL.
RewriteCond %{QUERY_STRING} GLOBALS(=|\[|\%[0-9A-Z]{0,2}) [OR]
# Block any script trying to modify a _REQUEST variable via URL.
RewriteCond %{QUERY_STRING} _REQUEST(=|\[|\%[0-9A-Z]{0,2})
# Return 403 Forbidden header and show the content of the root homepage
RewriteRule .* index.php [F]
#
## End - Rewrite rules to block out some common exploits.

## Begin - Custom redirects
#
# If you need to redirect some pages, or set a canonical non-www to
# www redirect (or vice versa), place that code here. Ensure those
# redirects use the correct RewriteRule syntax and the [R=301,L] flags.
#
## End - Custom redirects

##
# Uncomment the following line if your webserver's URL
# is not directly related to physical file paths.
# Update Your Joomla! Directory (just / for root).
##

# RewriteBase /

# Extra Security Headers
<IfModule mod_headers.c>
	Header set X-XSS-Protection "1; mode=block"
	Header always append X-Frame-Options SAMEORIGIN
	Header set X-Content-Type-Options nosniff
</IfModule>

## Begin - Joomla! core SEF Section.
#
RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
#
# If the requested path and file is not /index.php and the request
# has not already been internally rewritten to the index.php script
RewriteCond %{REQUEST_URI} !^/index\.php
# and the requested path and file doesn't directly match a physical file
RewriteCond %{REQUEST_FILENAME} !-f
# and the requested path and file doesn't directly match a physical folder
RewriteCond %{REQUEST_FILENAME} !-d
# internally rewrite the request to the index.php script
RewriteRule .* index.php [L]
#
## End - Joomla! core SEF Section.


<Files 403.shtml>

order allow,deny

allow from all

</Files>


ErrorDocument 400 /400.shtml
ErrorDocument 401 /401.shtml
ErrorDocument 403 /403.shtml
ErrorDocument 404 /404.shtml
ErrorDocument 500 /500.shtml

AddOutputFilterByType DEFLATE text/text text/html text/plain text/xml text/css application/x-javascript application/javascript text/javascript


Re: Forcing non-www and https redirects correctly

Posted: Thu Aug 15, 2019 2:03 pm
by urzh
Ah I see what you meant now. Ok, makes sense. I moved it and I'll monitor.

Thank you

Re: Forcing non-www and https redirects correctly

Posted: Thu Aug 15, 2019 6:37 pm
by urzh
About 99% sure this was my issue. Thank you.

I've made the update to my other sites.