How to Remove index.php from urls

[g1smd]

If Google finds a URL on your site that returns 404, you've got less than 24 hours to set up the redirect to some other URL. Google looks at new 404 errors just a couple of times over a period of several days before assigning a much lower crawl rate to that URL. In many cases it can be many weeks or months before the URL is crawled again - meaning the redirect won't be seen for a very long time.

You can add custom logging of 404 errors quite easily. On the error document page add:

Code: Select all

	</body>

<?php $statusCode = '404'; require_once("requestLogger.php"); ?>

</html>

This last line of the error page then interfaces to the logger code shown below.


Your other error pages can also call the logger code.

If you set the statusCode number, you'll get a separate log file per error code:

Code: Select all

<?php $statusCode = '403'; require_once("/somepath/requestLogger.php"); ?>

LOGGER
======

Code: Select all

<?php

#	Error Logging 2012-06-10

#	Set $status value (usually three digits) before including this file.
#	When status is 301, you must also supply $pageType and $newLocation.


	$oldSetting			= ignore_user_abort( TRUE );	// otherwise can screw-up logfile

	if( !empty( $GLOBALS[ '_SERVER' ])) {
		$_SERVER_ARRAY	= '_SERVER';
	} elseif( !empty( $GLOBALS[ 'HTTP_SERVER_VARS' ])) {
		$_SERVER_ARRAY	= 'HTTP_SERVER_VARS';
	} else {
		$_SERVER_ARRAY	= 'GLOBALS';
	}

	$requestHost		= ${$_SERVER_ARRAY}[ 'SERVER_NAME' ];

	if(stristr($requestHost, 'example.com')) {
		if(stristr($requestHost, 'test')) {
			define( '_DIRECTORY', '/var/www/vhosts/example.com/subdomains/test/httpdocs/assets/includes/requestLog/' );
			$site = 'test';
		} else if(stristr($requestHost, 'dev')) {
			define( '_DIRECTORY', '/var/www/vhosts/example.com/subdomains/dev/httpdocs/assets/includes/requestLog/' );
			$site = 'dev';
		} else if(stristr($requestHost, 'www')) {
			define( '_DIRECTORY', '/var/www/vhosts/example.com/httpdocs/assets/includes/requestLog/' );
			$site = 'www';
		} else if(!stristr($requestHost, 'test') && !stristr($requestHost, 'dev') && !stristr($requestHost, 'www')) {
			define( '_DIRECTORY', '/var/www/vhosts/example.com/httpdocs/assets/includes/requestLog/' );
			$site = 'www';
		}
	}


	if ($statusCode == '301' && ISSET($pageType)) {
	define( '_LOGFILE',	'errorlog' . gmdate('-o-W-') . $site . '-' . $statusCode . '-' . $pageType . '.txt' );
	} else {
	define( '_LOGFILE',	'errorlog' . gmdate('-o-W-') . $site . '-' . $statusCode . '.txt' );
	}

#	define( '_LOGFILE',	'errorlog' . gmdate('-o-W-') . $site . '.txt' );  // ALL IN ONE

	define( '_LOGMAXLINES',	'5000' );

	global ${$_SERVER_ARRAY};


	$logFile		= _DIRECTORY . _LOGFILE;

	$datetime		= gmdate( 'Y-m-d H:i:s O' );

	$remoteIP		= ${$_SERVER_ARRAY}[ 'REMOTE_ADDR' ];

	$requestURI		= ${$_SERVER_ARRAY}[ 'REQUEST_URI' ];

	$referer		= ( isset( ${$_SERVER_ARRAY}[ 'HTTP_REFERER' ]))
		? ${$_SERVER_ARRAY}[ 'HTTP_REFERER' ]
		: '<unknown referer>';

	$userAgent		= ( isset( ${$_SERVER_ARRAY}[ 'HTTP_USER_AGENT' ]))
		? ${$_SERVER_ARRAY}[ 'HTTP_USER_AGENT' ]
		: '<unknown user agent>';

	if(preg_match('#(Opera\ [0-9]+\.[0-9]+)#', trim($userAgent), $extracted)) {
		$agent = $extracted[1];
	} elseif(preg_match('#^(Opera[^(\ ]+)#', trim($userAgent), $extracted)) {
		$agent = $extracted[1];
	} elseif(preg_match('#^(Xenu.*)#', trim($userAgent), $extracted)) {
		$agent = $extracted[1];
	} elseif(preg_match('#^(Googlebot-Image.*)#', trim($userAgent), $extracted)) {
		$agent = $extracted[1];
	} elseif(preg_match('#^(facebookexternalhit[^\ ]+)#', trim($userAgent), $extracted)) {
		$agent = $extracted[1];
	} elseif(preg_match('#^(.{4,22})$#', trim($userAgent), $extracted)) {
		$agent = $extracted[1];
	} elseif(preg_match('#compatible;\ ([^;]+);#', $userAgent, $extracted)) {
		$agent = $extracted[1];
	} elseif(preg_match('#compatible;\ ([^/]+/[^\ ;]+)#', $userAgent, $extracted)) {
		$agent = $extracted[1];
	} elseif(preg_match('#(Firefox/[^\ ;]+)#', trim($userAgent), $extracted)) {
		$agent = $extracted[1];
	} elseif(preg_match('#^([^\ ]+\ )+([^\(\)]+)#', trim($userAgent), $extracted)) {
		$agent = $extracted[2];
	} elseif(!stristr($userAgent, 'ompatible') && preg_match('#^([^\ \(]+)\ \(#', $userAgent, $extracted)) {
		$agent = $extracted[1];
	} else {
		$agent = '<see notes>';
	}

	$remoteIP		= str_pad($remoteIP, 15);

	$agent			= str_pad(trim($agent), 25);

	$requestHost	= str_pad($requestHost, 26, " ", STR_PAD_LEFT);

	$requestURI		= str_pad($requestURI, 80);

	if (ISSET($newLocation)) {
	$newLocation	= str_pad($newLocation, 70);
	}

	$referer		= str_pad($referer, 110);

	$userAgent		= str_pad($userAgent, 120);

	if ($statusCode == '301' && ISSET($newLocation)) {
	$logLine		= $datetime . " - " . $remoteIP . " - " . $agent . " - ". $statusCode . " - ". $requestHost . " - ". $requestURI  . " - [". $newLocation . "] - ". $referer . " - ". $userAgent . "\n";
	} else {
	$logLine		= $datetime . " - " . $remoteIP . " - " . $agent . " - ". $statusCode . " - ". $requestHost . " - ". $requestURI . " - ". $referer . " - ". $userAgent . "\n";
	}

	$log			= file( $logFile );		// flock() disabled in some kernels (eg 2.4)

	if( $fp = fopen( $logFile, 'a' )) {		// tiny danger of 2 threads interfering; live with it
		if( count( $log ) >= _LOGMAXLINES ) {	// otherwise grows like Topsy
			fclose( $fp );				// fopen,fclose put close together as possible
			while( count( $log ) >= _LOGMAXLINES ) array_shift( $log );
			array_push( $log, $logLine );
			$logLine	= implode( '', $log );
			$fp		= fopen( $logFile, 'w' );
		}
		fputs( $fp, $logLine );
		fclose( $fp );
	}
	// exit();

	ignore_user_abort( $oldSetting );

?>

The file names of the log files contain year, week number and the HTTP status code being tracked. The log file contains date and time, remote IP address, status code, requested hostname, what was requested, referrer (if known), and details of the User Agent making the request.

[Calem]

Thanks for the replies and the extra info! The combined rewrite/redirect method shown here does work nicely. The only drawback is the loss of authority from lost inbound links (which you won't get back unless you get other sites to update their links). The pagerank loss will be recovered eventually, as g1smd pointed out.

I have been using REDj, it's a great plugin for 1-1 redirects with macros and it actually has an error logger as well. While I didn't have any problems with 404s, since the change I have been getting a bunch of 500 internal errors from /component/content/4?layout pages. I'm thinking they're related to Xmap.

Thanks again for the replies and original code.

[MTBdude]

I struggled with removing /index.php as well on a Joomla 2.5.6 site hosted on BlueHost. I got 404 errors whenever I tried to turn on url rewriting in global configuration. (Yes, SEF was on and .htaccess present.)

I ended up using BlueHost's cPanel Simple Scripts to install a new copy of joomla in a separate directory. THAT installation had no trouble with turning on URL Rewriting. "index.php" was removed from all links and urls and the site wasn't broken. Looking at the htaccess file installed by BlueHost's Simple Scripts, it had this change:

# For security reasons, Option followsymlinks cannot be overridden.
#Options +FollowSymLinks
Options +SymLinksIfOwnerMatch

Copying those lines to the htaccess file in my main joomla installation GOT IT WORKING. No more "index.php" in my urls!

[lip420]

Code: Select all

RewriteCond %{THE_REQUEST} !^POST
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)*index\.php/([^\ ]*)\ HTTP/
RewriteCond %{SERVER_PORT}>s ^(443>(s)|[0-9]+>s)$
RewriteRule ^(([^/]+/)*)index\.php/(.*)$ http%2://www.example.com/$1$2 [R=301,L]

This isn't working for me. It either freezes the page or the links don't work. I am on Joomla 2.5 and I just migrated from 1.5.

[g1smd]

There's a typo in the code. The $1$2 part should be $1$3.

[lip420]

I got that to disappear by turning url rewriting on in global configuration any reason why I still might want to try this again?

[g1smd]

So, the onsite navigation is fixed, but what about users arriving from SERPs and asking for the old URLs?

The redirect is to steer users and bots that ask for the old URLs to make a new request for the new URL rather than be served duplicate content or see a 404 error.

[giordanolh]

Hey guys, wanted to jump in here and get clarification on something from page 2.

So I applied the code that g1smd and logicc had provided, it worked absolutely perfectly!

Calem then said on page 2 to beware of de-ranking by Google after adding this code in the .htaccess file. I want to get clarification on why/how this de-ranking happens??
Will it only happen if the majority of your site's backlinks were historically pointing to 'index.php' pages?
In my case the majority of my site's backlinks are to my homepage (http://www.domain.com)..so will I be fine??


Thanks!

[g1smd]

Google works in mysterious ways, and nothing is certain. There can be drops in ranking and traffic for several weeks to many months.

With most of your redirects pointing at the site root, changes to the site will have less effect, but a good link profile will have a number of links pointing to various internal pages too.

Getting the technical implementation of the site completely right has to be the best long term solution. Even when many of the incoming links initially point to URLs that redirect, eventually you'll reach a point when there are more links pointing to the new URLs than the old ones.

[vegas-kid]

Wow, thanks for the great shares in this thread.

I implemented the redirect but still have a couple issues. I have some crawl errors that are pointing to content that does not exist, so the redirect doesn't help those. Is there a way to just direct those to the home page or should I do something else?

Example:
index.php/clean/46-minneapolis?format=feed&type=rss
index.php/snow/46-minneapolis?format=feed&type=atom

Also, I have "HTML errors" being reported for duplicate content on pages that do exist, but I have no idea how the url got generated or found by google.

Example:
/image-gallery/46-minneapolis ... should just be /image-gallery

Whats the best way to redirect a single page using what I have already implented below. It's my understanding that order is important.

I appreciate all your assistance, please take into account that I am a total noob when replying

########## Begin - Custom redirects
########## Actions : Redir any other domains or nonwww to www. Removes index.php from home and any internal pages

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)*(index|home)\.html?\ HTTP/
RewriteRule ^(([^/]+/)*)(index|home)\.html?$ http://www.domain.com/$1 [R=301,L]

RewriteCond %{THE_REQUEST} !^POST
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)*index\.php\ HTTP/
RewriteCond %{SERVER_PORT}>s ^(443>(s)|[0-9]+>s)$
RewriteRule ^(([^/]+/)*)index\.php$ http%2://www.domain.com/$1 [R=301,L]

# Remove index.php from sub pages e.g. http://www.domain.com/index.php/about --> http://www.domain.com/about
RewriteCond %{THE_REQUEST} !^POST
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)*index\.php/([^\ ]*)\ HTTP/
RewriteCond %{SERVER_PORT}>s ^(443>(s)|[0-9]+>s)$
RewriteRule ^(([^/]+/)*)index\.php/(.*)$ http%2://www.domain.com/$1$3 [R=301,L]

RewriteCond %{HTTP_HOST} !^(www\.domain\.com)?$
RewriteRule (.*) http://www.domain.com/$1 [R=301,L]
#
########## End - 301 Redirect
########## End - Custom redirects

[g1smd]

Sounds like you have some navigation links coded as page-relative links instead of the required root-relative format with a leading slash.

[vegas-kid]

/\ lol /\

G1smd - I think what happened was Google found my site while I was constructing it. Shared the URL in a support forum for the template... Lesson learned.

As you can see above, I redirected the pages being marked as duplicate content and all URLs containing index.php. For the RSS URLs, I disallowed any containing the format=feed. Then I marked the errors as fixed in Webmaster Tools. We will see if they come back next time Google crawls.

[nis4]

Hi,

I have the following problem:

I have a joomla 1.1.15 installation in a subfolder with those non sef urls that are indexed by google.

In the root i have joomla 2.5 with good urls.

I done a redirect from subfolder to root with:

Code: Select all

RewriteRule ^subfolder\?$ "http\:\/\/www\.my\-web\.com\/" [R=301,L]

But that code does not redirect all other pages i.e.

Code: Select all

www.my-web.com/subfolder/index.php?option=com_content&task=view&id...

I am strugling to find a correct solution that would point all those index.php links from my subfolder site to the new root site.

This is how the part of custom redirects in my .htaccess looks like:

Code: Select all

## Begin - Custom redirects
#
RewriteRule ^subfolder\?$ "http\:\/\/www\.my\-web\.com\/" [R=301,L]
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)*index\.html?\ HTTP/
RewriteRule ^(([^/]+/)*)index\.html?$ http://www.my-web.com/$1 [R=301,L]
#
RewriteCond %{THE_REQUEST} !^POST
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)*index\.php\ HTTP/
RewriteCond %{SERVER_PORT}>s ^(443>(s)|[0-9]+>s)$
RewriteRule ^(([^/]+/)*)index\.php$ http%2://www.my-web.com/$1 [R=301,L]
#
RewriteCond %{HTTP_HOST} !^(www\.my-web\.com)?$
RewriteRule (.*) http://www.my-web.com/$1 [R=301,L]
#
## End - Custom redirects

[g1smd]

Code: Select all

RewriteRule ^subfolder\?$ "http\:\/\/www\.my\-web\.com\/" [R=301,L]

That code is completely broken.

The colon, slashes, and hyphen in the target URL should not be escaped. The quotes should not be there.

The $ anchor means that only requests for the bare folder, not any pages within it, will be considered. However, the pattern will never match as \? is for a literal question mark and there will never be a question mark within the path part of a URL. Question mark is a separator between path and query string and can never be matched in a bare RewriteRule.

This redirects all requests beginning example.com/subfolder/

Code: Select all

RewriteRule ^subfolder/  http://www.example.com/ [R=301,L]

However, redirecting multiple URLs to one URL and especially redirecting to the root of a site is a very bad idea. It is a signal of "low technical quality" as far as Google is concerned.

[yesme]

How about change from www.domain.com/en to www.domain.com ? Which means removing the "en" in front of it because it is weird since my root directory does not contained "en" folder.

Somebody could help me?
Thanks in advance...

[g1smd]

Replace the word "subfolder" with "en" using the previous example.

Add (.*) to the end of the RegEx pattern and re-use it as $1 in the target URL.

[yesme]

Hi g1smd,

Thank you for responding my post.

But the problem is, I am a joomla newbie and this is new thing for me to set the regex or money symbol which I did not know the function. Pity me.

I tried to use this code:

Code: Select all

RewriteRule ^en\?$ "http\:\/\/www\.mydomain\.com\/" [R=301,L]

But an error appeared.

[g1smd]

Code: Select all

RewriteRule ^subfolder/  http://www.example.com/ [R=301,L]

Replace the word "subfolder" with "en" using the previous example.

Add (.*) to the end of the RegEx pattern and re-use it as $1 in the target URL.

Code: Select all

RewriteRule ^en/(.*)  http://www.example.com/$1 [R=301,L]

[te_libera]

Here is one solution that my hosting give me:

One line added to the .httaccces

after: RewriteEngine On
been writting: RewriteBase /

RewriteEngine On
RewriteBase /

[setx]

The total elimination of suffix links the search engines will have a better effect?
Which is better option would Adds Suffix to URL?

[475]

Hi, I have hit a major snag when trying to remove the /index.php from my site.

First I changed the info in the htaccess.txt and renamed it .htaccess. The file now reads;

##
# @version $Id: htaccess.txt 21064 2011-04-03 22:12:19Z dextercowley $
# @package Joomla
# @copyright Copyright (C) 2005 - 2010 Open Source Matters. All rights reserved.
# @license http://www.gnu.org/copyleft/gpl.html GNU/GPL
# Joomla! is Free Software
##


#####################################################
# READ THIS COMPLETELY IF YOU CHOOSE TO USE THIS FILE
#
# The line just below this section: 'Options +FollowSymLinks' may cause problems
# with some server configurations. It is required for use of mod_rewrite, but may already
# be set by your server administrator in a way that dissallows changing it in
# your .htaccess file. If using it causes your server to error out, comment it out (add # to
# beginning of line), reload your site in your browser and test your sef url's. If they work,
# it has been set by your server administrator and you do not need it set here.
#
#####################################################

## Can be commented out if causes errors, see notes above.
#Options +FollowSymLinks
Options +SymLinksIfOwnerMatch

#
# mod_rewrite in use

RewriteEngine On

########## Begin - Rewrite rules to block out some common exploits
## If you experience problems on your site block out the operations listed below
## This attempts to block the most common type of exploit `attempts` to Joomla!
#
## Deny access to extension xml files (uncomment out to activate)
#<Files ~ "\.xml$">
#Order allow,deny
#Deny from all
#Satisfy all
#</Files>
## End of deny access to extension xml files
# Block out any script trying to set a mosConfig value through the URL
RewriteCond %{QUERY_STRING} mosConfig_[a-zA-Z_]{1,21}(=|\%3D) [OR]
# Block out any script trying to base64_encode data within the URL
RewriteCond %{QUERY_STRING} base64_encode[^(]*\([^)]*\) [OR]
# Block out any script that includes a <script> tag in URL
RewriteCond %{QUERY_STRING} (<|%3C)([^s]*s)+cript.*(>|%3E) [NC,OR]
# Block out any script trying to set a PHP GLOBALS variable via URL
RewriteCond %{QUERY_STRING} GLOBALS(=|\[|\%[0-9A-Z]{0,2}) [OR]
# Block out any script trying to modify a _REQUEST variable via URL
RewriteCond %{QUERY_STRING} _REQUEST(=|\[|\%[0-9A-Z]{0,2})
# Return 403 Forbidden header and show the content of the root homepage
RewriteRule .* index.php [F]
#
########## End - Rewrite rules to block out some common exploits


########## Begin - Custom redirects
########## Actions : Redir any other domains or nonwww to www. Removes index.php from home and any internal pages

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)*(index|home)\.html?\ HTTP/
RewriteRule ^(([^/]+/)*)(index|home)\.html?$ http://www.domain.com/$1 [R=301,L]

RewriteCond %{THE_REQUEST} !^POST
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)*index\.php\ HTTP/
RewriteCond %{SERVER_PORT}>s ^(443>(s)|[0-9]+>s)$
RewriteRule ^(([^/]+/)*)index\.php$ http%2://www.domain.com/$1 [R=301,L]

# Remove index.php from sub pages e.g. http://www.domain.com/index.php/about --> http://www.domain.com/about
RewriteCond %{THE_REQUEST} !^POST
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)*index\.php/([^\ ]*)\ HTTP/
RewriteCond %{SERVER_PORT}>s ^(443>(s)|[0-9]+>s)$
RewriteRule ^(([^/]+/)*)index\.php/(.*)$ http%2://www.domain.com/$1$3 [R=301,L]

RewriteCond %{HTTP_HOST} !^(www\.domain\.com)?$
RewriteRule (.*) http://www.domain.com/$1 [R=301,L]
#
########## End - 301 Redirect
########## End - Custom redirects

# Uncomment following line if your webserver's URL
# is not directly related to physical file paths.
# Update Your Joomla! Directory (just / for root)

# RewriteBase /


########## Begin - Joomla! core SEF Section
#
RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
#
# If the requested path and file is not /index.php and the request
# has not already been internally rewritten to the index.php script
RewriteCond %{REQUEST_URI} !^/index\.php
# and the request is for root, or for an extensionless URL, or the
# requested URL ends with one of the listed extensions
RewriteCond %{REQUEST_URI} (/[^.]*|\.(php|html?|feed|pdf|raw))$ [NC]
# and the requested path and file doesn't directly match a physical file
RewriteCond %{REQUEST_FILENAME} !-f
# and the requested path and file doesn't directly match a physical folder
RewriteCond %{REQUEST_FILENAME} !-d
# internally rewrite the request to the index.php script
RewriteRule .* index.php [L]
#
########## End - Joomla! core SEF Section

I then checked the SEOfriendly & Rewrite URL radio buttons in the admin area and thought I was done.

Now when I hover over the links on the site, the status bar shows the link with index.php (http://www.domain.com/index.php/about-us). However when I click on the link the address bar shows without the index.php (http://www.domain.com/about-us).

Worse still, I cannot access the Joomla Admin Area!.. When I try to login at http://www.domain.com/administrator/index.php it states "Username and password do not match". However, when I checked thru phpMyAdmin the user name and password I am using matches the ones in the database (password written as MD5).

Could anyone suggest a way out of this mess?.. please.

[LouPhi]

I too found this string via a google search...

I wanted to remove index.php from the urls on my site (I'm using version 2.5.17)... so I went into global config, and selected 'yes' for use URL rewriting....

Went onto my site http://www.palletrecyclingsouthwales.co.uk and clicked on a link - all seemed to be great (the index.php part of the url had disappeared), until I then went to try another link and am now getting a whole string showing me the entire directory path on the server, ie, http://www.palletrecyclingsouthwales.co ... letswanted

How can I stop this from happening??

I also tried adding RedirectMatch 301 ^/index.php/(.*)$ http://www.palletrecyclingsouthwales.co.uk/$1 to my .htaccess file (as this was mentioned in one of the replies on this string) but it made no difference, so for now I have commented it out in the .htaccess file.

Can anyone help me solve this one?

[Per Yngve Berg]

What type of menu item is "palletswanted"?

[LouPhi]

Every menu item/link is linked through the correct item in the 'main menu' there are no direct 'content' links... is this what you meant?

[LouPhi]

They are all 'single article' menu-item-types....

[GODpleasers]

Ok this looks overly complicated. A job for a Joomla expert genius. Would some one please come volunteer to do this for us. It would be appreciated.

[ladydevil]

I have encountered this problem before. Select 1 in 2 way you can cause was:
Use ".htaccess" if your server uses "Apache".
Use "web.config" if your server uses "IIS".

[GODpleasers]

thanks got it closing this topic out now.

[volito]

Hello all! Is this thread closed because I spent lots of time reading through this and I am still LOST! any one click solutions for this headache?

Joomla! 2.5.27
Apache
When I turn on URL rewriting site goes down and defaults to hosting page....

Does this affect SEO ? is it a security risk? if not I'll just deal with it? newbie and seams like a steep learning curve for a DYIer

thanks

nothing?

[peterparker]

i find it too, thank all hepls