Suggested Master .htaccess file

[g1smd]

That's one way of doing it, but do change the path from .* to index\.php to speed the code up.

There's no point checking if that parameter is attached to requests for images and stylesheets etc.

[g1smd]

Reply from the author of the original master file:

I am wondering what you're trying to say... The docs.joomla.org page has ABSOLUTELY NO CHANGE TO MY MASTER .htaccess FILE. All changes commited are to Mandville's .htaccess code which, in my humble opinion, is too invasive as it tries to block everything, including useful stuff, and should only be used by very experienced system administrators who Know What They're Doing™.

Regarding the proposed changes, some of them seem to be changes to support Apache 1.3, a topic I don't really care about. That said, some of the changes above (SQLi code, bad words filtering) are better accomplished in the PHP level. The above sections were merely an example of how it can be done in .htaccess - so I'm not really interested in optimizing.

He's completely missed the corrections to mod_rewrite syntax and the fixing of several rules which did not work at all.

Here's the new file: http://snipt.net/g1smd/joomla-master-ht ... 011-03-24/

Here's the list of changes: http://snipt.net/g1smd/joomla-master-fi ... -23-to-24/

[mandville]

I am not sure which htaccess is "mine". but
As as been said previously.

* Only discussions on the version declared by g1smd should be discussed now. http://snipt.net/g1smd/joomla-master-ht ... 011-03-24/
* if you want him to fix htaccess files for specific urls or circumstances then please negotiate off forum or via the professional services forum

[Webdongle]

Is that proposed master .htaccess file a little large ? Would it slow done the site significantly, especially for those of us who can only afford inexpensive shared Hosting ?

@g1smd
Have you tested it on shared Hosting on a live site or just tested it on dedicated servers ?

Also would a file like that be too complicated for most users ? Many new/inexperienced users have problems with locating and editing RewriteBase as it is !!

What if it caused problems, how would the new/inexperienced deal with that ?

[PhilD]

@Webdongle
"Have you tested it on shared Hosting on a live site or just tested it on dedicated servers ?"
Please read the posts in the thread. Your questions have been answered.

"Also would a file like that be too complicated for most users ?"
Yes

"Would it slow done the site"
Yes - anything a server must do other than serving the request slows a site down.


"Many new/inexperienced users have problems with locating and editing RewriteBase as it is !"
You must STOP confusing what your doing/learning with your own server with about what a commercial host has already set up!

This is standard for pretty much standard code for every *nix host on the web and If you were to look at the official htaccess included with Joomla you WILL see this is the same code as this is where it was copied from!

########## Begin - RewriteEngine enabled
RewriteEngine On
Options +FollowSymLinks
########## End - RewriteEngine enabled

########## Begin - RewriteBase
# Uncomment following line if your webserver's URL
# is not directly related to physical file paths.
# Update Your Joomla! Directory (just / for root)

# RewriteBase /
########## End - RewriteBase

[Webdongle]

@Webdongle
"Have you tested it on shared Hosting on a live site or just tested it on dedicated servers ?"
Please read the posts in the thread. Your questions have been answered.

...

I have been reading the posts but i must have missed it in the 10 pages, sorry I will re read the thread.

@Webdongle
....
"Many new/inexperienced users have problems with locating and editing RewriteBase as it is !"
You must STOP confusing what your doing/learning with your own server with about what a commercial host has already set up!

....

I am not referring to what I am learning about server/.htaccess set up with what a commercial host has set up.

Shared hosting also use the rewrite code in the .htaccess, especially when they provide a Joomla install.

My question was about ... the more that's in the .htaccess then the harder to find what your looking for ? Many newbies don't think of using 'Find' when searching to edit. The words 'needle' and 'haystack' came to mind when looking at it.

Although looking at it again,#RewriteBase / is near the top and well documented so even the newest novice should be able to find it to uncomment.

[sog2012]

Just need to clarify something as I am still not 100% crystal clear on this.

What is the proper way to type this?

RewriteCond %{HTTP_REFERER} !^https?://(www\.)?mywebsite\.com.au [NC]

OR:

RewriteCond %{HTTP_REFERER} !^https?://(www\.)?mywebsite\.com\.au [NC]

[PhilD]

RewriteCond %{HTTP_REFERER} !^https?://(www\.)?mywebsite\.com\.au [NC]

If the line has \. ( as in mysite\.com) then the \ is required to escape the period or the condition will not work.

If the line has mysite.com then no \ is required to escape the period for the condition to work.

Here is the origional line your asking about
RewriteCond %{HTTP_REFERER} !^https?://(www\.)?example\.com [NC]

So this line requires the \. or it won't work.

[sog2012]

RewriteCond %{HTTP_REFERER} !^https?://(www\.)?mywebsite\.com\.au [NC]

Thank you Phil

Is this right too?

RewriteCond %{HTTP_HOST} !^(www\.mywebsite\.com\.au)?$

[g1smd]

Within each pattern, either ALL of the periods are escaped or NONE of the periods are escaped.

There are NO patterns where SOME of the periods are escaped and SOME are not.

[sog2012]

Within each pattern, either ALL of the periods are escaped or NONE of the periods are escaped.

There are NO patterns where SOME of the periods are escaped and SOME are not.

Sorry that made zero sense to me.

Please remember, you are very clever with such terminology, but many of us, including me, are complete noobs

So are these correct or incorrect

RewriteCond %{HTTP_REFERER} !^https?://(www\.)?mywebsite\.com\.au [NC]

RewriteCond %{HTTP_HOST} !^(www\.mywebsite\.com\.au)?$

[mandville]

Despite numerous warnings about taking this off this off topic and asking for help on specific urls, users have continued to do so.
To prevent further abuse of a good intentioned user, this topic will be locked.

[g1smd]

The file has been edited with hundreds of changes.
http://docs.joomla.org/Htaccess_example ... ldid=35925

Hopefully now, this is complete and functional.
http://docs.joomla.org/Htaccess_example ... ldid=48682

This new version now needs further testing.

I will not answer any other questions other than what is in that new code.

[PhilD]

Thanks for updating the thread!

[abatis]

thank you to everyone in this thread. it has saved me what would probably have been weeks of headaches. so thank you very much.

-abatis

[thorosius]

I used the .htaccess file and my maximenu stopped working. Can't figure out which line is responsible. Can anybody help me with this?

[mandville]

AS previously stated, that sort of assistance will not be provided in this thread.
Try maximenu or the extensions forum. Your post is not security related.

[antihack]

Also, if a line in the htaccess file is causing the issue. Then take out 5 lines at a time until it starts to work. That is the only way to figure it out.

[Webdongle]

AS previously stated, that sort of assistance will not be provided in this thread.
Try maximenu or the extensions forum. Your post is not security related.

but

The file has been edited with hundreds of changes.
http://docs.joomla.org/Htaccess_example ... ldid=35925

Hopefully now, this is complete and functional.
http://docs.joomla.org/Htaccess_example ... ldid=48682

This new version now needs further testing.

I will not answer any other questions other than what is in that new code.

And thorosius said he had a problem using it. Does that not come under

This new version now needs further testing.

I used the .htaccess file and my maximenu stopped working. Can't figure out which line is responsible. Can anybody help me with this?

Now I'm confused because thorosius claims the new .htaccess causes problems. So should that not be considered a bug report for the request for further testing ?

[PhilD]

If you take the new(er) htaccess master and apply it to your site as is,then yes it will cause problems. To apply the file to a site involves the knowledge of correctly editing the various parts of the file, eliminating the parts that don't apply to your particular site, and troubleshooting the file in a particular setting. To do this requires at least a basic knowledge of htaccess and server setups.
If someone finds an area or a line that causes issues with a particular extension then report the section or (better) the line that does not work or causes issues with your sites particular setup. This is a benefit to everyone and what the intent of this particular thread is. It is not to post "I tried it and it don't work, or broke xyz, fix it for me."

Now if you were to say, " My setup is this and this area or these lines (or this line) in the master file are causing an error of abc to happen when extension xxx is installed or accessed, can you help?", then that is helpful in solving the issue, providing a better master htaccess file for everyone and doesn't pollute the thread with posts that aren't helpful in solving a particular issue because of lack of information.

[g1smd]

Also, if a line in the htaccess file is causing the issue. Then take out 5 lines at a time until it starts to work. That is the only way to figure it out.

Absolutely not. That looks like pure guesswork that will cause more problems than it solves. None of the rulesets are 5 lines long. Some are only a single line, others much longer.

[leolam]

@ g1smd
With posting the "suggested htaccess master file" (as the topic was renamed by the mods) I for sure never thought it would bring up this mountain of issues discussed in this post. I highly appreciate your timeless efforts and sharing your knowledge with the Joomla community and the time to explain certain issues. It has been very, very valuable and your contribution on this issue should be remembered! I have learned a lot during this discussion and I am quite happy the way it turned out.

Thanks!

Leo

[g1smd]

Thanks. This stuff is difficult enough as it is, but we also took a few wrong turns due to my unfamiliarity with the format of all the various possible types of Joomla URLs. The input in this forum from the various testers was vital to getting the job done.

There are still several issues to overcome. I am updating the Joomla docs page from time to time. Nikosdion also has a Git repository of his changes. We are converging towards common code, but at present there are still several differences between my thinking and his.

[g1smd]

The file has been updated again. Feel free to test it out and report any further problems.

Nikosdion has also incorporated many of the suggested changes into his version, so you might want to test both versions of the file.

http://joomla-master-htaccess.googlecod ... s.txt?r=31
http://code.google.com/p/joomla-master- ... etail?r=31
http://docs.joomla.org/Htaccess_example ... ldid=58141

http://akeeba.assembla.com/code/master- ... 5e51aef6c4

[PierreB]

This is a very serious and advanced users' thread and maybe joomla imbeciles like me should stay out as much as possible, but I have a practical question, perhaps one that will help deal with people as dense as me on relevant technical matters but who want to keep their htaccess file current with regard to the parts of code that they're using...

...so my question is how do people like me follow this thread and the code changes reported in it?
Do these changes represent resolutions of reported problems or are they proposed optimizations that are more possible to cause issues in comparison to previous "stable" - more widely tested code and which is which?
Of course both previous and new code needs testing, nothing is set in stone or given explicit guarantee to work in all cases...

My question stems from my concern for repeatedly bothering and inconveniencing @g1smd on non-www. to www. and index.php to / redirection...a code which seems to have changed in this .htaccess file version too.
The one proposed to me recently was

Code: Select all

########## Begin - 301 Redirect
#
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)*(index|home)\.html?\ HTTP/
RewriteRule ^(([^/]+/)*)(index|home)\.html?$ http://www.example.com/$1 [R=301,L]
#
RewriteCond %{THE_REQUEST} !^POST
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)*index\.php\ HTTP/
RewriteCond %{SERVER_PORT}>s ^(443>(s)|[0-9]+>s)$
RewriteRule ^(([^/]+/)*)index\.php$ http%2://www.example.com/$1 [R=301,L]
#
RewriteCond %{HTTP_HOST} !^(www\.example\.com)?$
RewriteRule (.*) http://www.example.com/$1 [R=301,L]
#
########## End - 301 Redirect

and the new code is

Code: Select all

########## Begin - Redirect index.php to /
## Note: Change example.com to reflect your own domain name
RewriteCond %{THE_REQUEST} !^POST
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /index\.php\ HTTP/
RewriteCond %{SERVER_PORT}>s ^(443>(s)|[0-9]+>s)$
RewriteRule ^index\.php$ http%2://www.example.com/$1 [R=301,L]
## If the above line throws a 500 error, change [R=301,L] to [R,L]
########## End - Redirect index.php to /
 
########## Begin - Redirect non-www to www
RewriteCond %{HTTP_HOST} !^www\. [NC]
RewriteRule ^(.*)$ http://www.%{HTTP_HOST}/$1 [R=301,L]
## If the above throws an HTTP 500 error, swap [R=301,L] with [R,L]
########## End - Redirect non-www to www

...I don't want to be the guy that needs an answer to be provided to him individually while this may have been answered or discussed in other places on the site...

How can we follow this thread without much seeking individual advice (of course individual advice can help other individuals as well, but you get my point) while still keeping current? How can we keep current without disturbing you or the flow of this thread? How do we treat the changes made to the file?

Cheers

[Webdongle]

1. Is non-www. to www. necessary in a Master.htaccess file ?
2. What security does it provide ?
3. How much editing of the code to put in the url would be necessary ?
4. Would such a file be suitable for All users ?(many newbies would have problems and get confused with the \ syntax would they not ?)

[leolam]

@ Webdongle: Answered extensively hy Phil in Joomla 1.6-forum: http://forum.joomla.org/viewtopic.php?f=621&t=601144

Cheers

Leo

[Webdongle]

@leolam
Thanks

-----

IMHO it looks like this 'Suggested Master .htaccess file' is becoming more and more impractical as complete solution. It looks like a good starting point for people to use as a reference which has chunks of code that will be useful to them ?

Is it possible that the 'Suggested Master .htaccess file' is becoming more of an intellectual exercise than a practical file that can be used ?

A big positive is that it is well notated and tidily written. What a fantastic resource it would make if Tutorials were written that described in depth exactly what each part of the code did. Newbies could read the Tutorial and have an understanding of which parts of the code would enhance the security of their site.

[g1smd]

How do people like me follow this thread and the code changes reported in it?

There's no need to follow every change in this thread. Just take a look at the code in the Docs article or in one of the SVN or GIT code repositories from time to time. If you see there is a newer version available, use the various inbuilt DIFF features on those sites to compare the new version on that site with the previous version on that site (the one that you are using that is) - then make those same changes to your own .htaccess file.

Do these changes represent resolutions of reported problems or are they proposed optimizations that are more possible to cause issues in comparison to previous "stable" - more widely tested code and which is which?

The code itself protects against various malicious exploits that hackers have tried against Joomla sites in the past, as well as a few "theoretical" problems. Most of the code was written years ago, but it looks like some of it was written by people who were not all that familiar with mod_rewrite and introduced a number of errors in the code.

The earliest updates by me simply corrected syntax errors in the code, fixed several rules that could never possibly work and changed several more rules to do "exactly" what they should do rather than being "almost" right. They also introduced more efficient patterns (i.e. they run faster) in place of what went before. Some of the changes were guesswork, because at that point I was not fully aware of all of the possible URL formats that could exist within a Joomla site. Testing in this thread uncovered more limitations which were also fixed.

The more recent changes fixed actual errors found when testing as well as incorporating some newer or better ideas within the code. The newer code will always be a "better bet" than the older code.

My question stems from my concern for repeatedly bothering and inconveniencing @g1smd on index.php to / and non-www. to www. redirection... a code which seems to have changed in this .htaccess file version too.

I have a particular way of doing things, but Nikosdion has other ideas. So, when asked "how to redirect non-www to www" (perhaps in a forum) I have a particular codeset that I will post. However, in the "Master .htaccess file", the code is (at present) aligned more the way that Nikosdion does things, and it has some limitations (e.g. it doesn't redirect www requests that have a port number).

How can we keep current without disturbing you or the flow of this thread? How do we treat the changes made to the file?

Use the links in my previous post and pick whether you prefer to use my code in SVN or in Wiki format, or Nikosdion's code in GIT. All repositories have ways to compare any version with any other version. Keep an eye open for newer versions. In general, newer code will usually work better than anything that has gone before.

1. Is non-www. to www. necessary in a Master .htaccess file?

It isn't strictly necessary to include it here, but it is important exactly where in the file the code is placed. As such the file would be incomplete and more "it doesn't work" support questions would be posed in this forum if it were not included and people placed the non-www to www code in the wrong place within the file.

2. What security does it provide?

It provides no security as such. It is merely one cog in a bigger wheel.

3. How much editing of the code to put in the URL would be necessary?

What do you mean? Every mention of example.com must be replaced with the real domain name.

4. Would such a file be suitable for All users?

No. You need a little bit of knowledge to get this working.

(many newbies would have problems and get confused with the \ syntax would they not?)

There are a million Regular Expression tutorials on the web. There are hundreds of thousands of mod_rewrite tutorials on the web. The code is heavily commented. Little more can be done to help people run with this, beyond the things already mentioned.

This is server configuration code. It configures the core inner workings of the webserver. Ignore that fact at your peril.

IMHO it looks like this 'Suggested Master .htaccess file' is becoming more and more impractical as complete solution. It looks like a good starting point for people to use as a reference which has chunks of code that will be useful to them?

Certainly you can use it as a collection of ideas, and pick and choose the ones you want to use. However, it is certainly not becoming "more impractical". The current codebase is more robust and more efficient than ever before. There is no one right way to do any of this stuff, but there are hundreds of wrong ways of doing it. Once you become proficient at this stuff, the subtle differences in code will become apparent.

Is it possible that the 'Suggested Master .htaccess file' is becoming more of an intellectual exercise than a practical file that can be used?

The whole point in me stepping forward to fix this was simply to turn it into a file that actually works and does exactly what it says on the tin. It is close to doing that, but will always require individual tuning per site.

[g1smd]

This is the complete list of changes that I have already made:
http://docs.joomla.org/Htaccess_example ... ldid=35925

This is much more than "tinkering".