The Joomla! Forum ™

Download

Demo


Board index » Joomla! Older Version Support » Joomla! 1.0 » General Questions - 1.0.x

Forum rules


Forum Rules
Absolute Beginner's Guide to Joomla! <-- please read before posting, this means YOU.
Forum Post Assistant - If you are serious about wanting help, you will use this tool to help you post.



Post new topic Reply to topic  Page 1 of 1
  [ 8 posts ] 
  Print view Previous topic | Next topic 
Author Message
vannquish
PostPosted: Wed Mar 07, 2007 3:24 pm 
User avatar
Joomla! Explorer
Joomla! Explorer
Offline

Joined: Tue Feb 21, 2006 11:10 am
Posts: 399
Location: UK
I currently have a website which has been online for several years.

Im currently redesigning it using joomla and want to test it out online for a few weeks.

How do i stop search engine bots picking up my new web pages and indexing them.
Top
 Profile  
 
pe7er
PostPosted: Thu Mar 08, 2007 12:05 am 
User avatar
Joomla! Master
Joomla! Master
Offline

Joined: Thu Aug 18, 2005 8:55 pm
Posts: 16291
Location: Nijmegen, The Netherlands
Change the robots.txt in your site's root. (you'd better back up this file to robots.bak and create a new robots.txt)

Add
Disallow: /

btw: it will only stop search engines from indexing that make use of the Robots Exclusion Standard:
http://en.wikipedia.org/wiki/Robots.txt

_________________
Kind Regards,
Peter Martin, Global Moderator - Community Leadership Team
www.db8.nl - Joomla specialist, Nijmegen, Nederland
Joomla 1.5 Quick Reference Guide: www.db8.nl/en/downloads/misc-downloads/ ... glish.html
Top
 Profile  
 
vsr
PostPosted: Thu Mar 08, 2007 1:56 am 
Joomla! Enthusiast
Joomla! Enthusiast
Offline

Joined: Sun Oct 15, 2006 2:52 am
Posts: 113
Is the Joomla install in a directory and not your root directory?  If so you can use .htaccess to give hem  403 error when they try to access that part of your site.  Use User Agent Blocking



SetEnvIfNoCase User-Agent "^EmailSiphon" bad_bot
SetEnvIfNoCase User-Agent "^EmailWolf" bad_bot
SetEnvIfNoCase User-Agent "^ExtractorPro" bad_bot
SetEnvIfNoCase User-Agent "^CherryPicker" bad_bot
SetEnvIfNoCase User-Agent "^NICErsPRO" bad_bot
SetEnvIfNoCase User-Agent "^Teleport" bad_bot
SetEnvIfNoCase User-Agent "^EmailCollector" bad_bot
SetEnvIfNoCase User-Agent "^LinkWalker" bad_bot
SetEnvIfNoCase User-Agent "^Zeus" bad_bot


Order Allow,Deny
Allow from all
Deny from env=bad_bot
Top
 Profile  
 
mandville
PostPosted: Thu Mar 08, 2007 1:59 am 
User avatar
Joomla! Exemplar
Joomla! Exemplar
Offline

Joined: Mon Mar 20, 2006 1:56 am
Posts: 9841
Location: The Girly Side of Joomla in Sussex
can we add the infamous libperl one to this list and if so - how?

_________________
HU2HY- Poor questions = Poor answer
Un requested Help PM's will be added to the foe list and possibly just deleted
{Community.Connect Administrator }{ Showcase & Security Moderator}
Top
 Profile  
 
vsr
PostPosted: Thu Mar 08, 2007 2:08 am 
Joomla! Enthusiast
Joomla! Enthusiast
Offline

Joined: Sun Oct 15, 2006 2:52 am
Posts: 113
You just need to know the user agent.  Copy one of the user agent lines abve and put it at the top of the list, replace that agent with the one you want to block.

The code above is placed in your .htaccess file.
Top
 Profile  
 
mandville
PostPosted: Thu Mar 08, 2007 2:27 am 
User avatar
Joomla! Exemplar
Joomla! Exemplar
Offline

Joined: Mon Mar 20, 2006 1:56 am
Posts: 9841
Location: The Girly Side of Joomla in Sussex
let me rephrase that -
how can you stop ALL variants of "lib perl" (thje most common skript bot ) from accessing your site

_________________
HU2HY- Poor questions = Poor answer
Un requested Help PM's will be added to the foe list and possibly just deleted
{Community.Connect Administrator }{ Showcase & Security Moderator}
Top
 Profile  
 
snadowitz
PostPosted: Fri Dec 07, 2007 5:15 pm 
User avatar
Joomla! Guru
Joomla! Guru
Offline

Joined: Wed Jan 18, 2006 4:40 am
Posts: 760
mandville wrote:
let me rephrase that -
how can you stop ALL variants of "lib perl" (thje most common skript bot ) from accessing your site



yes - I would be interested to know how to stop libperl too - I just started getting a lot of hits from a particlular IP using libperl to try and hack. I banned the IP in .htaccess for the time being but a more general solution would be greeeeeeat.

_________________
Snadowitz
Current Projects:-
http://www.flamineight.co.uk
http://kentonbar.blog spot.com
Top
 Profile  
 
mandville
PostPosted: Fri Dec 07, 2007 10:40 pm 
User avatar
Joomla! Exemplar
Joomla! Exemplar
Offline

Joined: Mon Mar 20, 2006 1:56 am
Posts: 9841
Location: The Girly Side of Joomla in Sussex
i am sure that if you search my posts, or libwww then you will find the code very similar to that in the first post

_________________
HU2HY- Poor questions = Poor answer
Un requested Help PM's will be added to the foe list and possibly just deleted
{Community.Connect Administrator }{ Showcase & Security Moderator}
Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  Page 1 of 1
  [ 8 posts ] 


Board index » Joomla! Older Version Support » Joomla! 1.0 » General Questions - 1.0.x

Who is online

Users browsing this forum: No registered users and 68 guests

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
Powered by phpBB® Forum Software © phpBB Group