The Joomla! Forum ™

Download

Demo


Board index » Other Forums » The Lounge



Post new topic Reply to topic  Page 1 of 1
  [ 6 posts ] 
  Print view Previous topic | Next topic 
Author Message
Poage
PostPosted: Tue Dec 20, 2011 4:24 am 
Joomla! Apprentice
Joomla! Apprentice

Joined: Wed Dec 31, 2008 12:04 am
Posts: 19
Location: American Canyon, CA
I tried putting a robots.txt file in the root directory of my website--don't think it slowed down the little buggers. My site has been up for about 2 years and we were mucking along just fine until about a month ago--provider is not happy which means I'm not happy!! :(

If this is not where to address this issue--just tell me where to put it in the forum. ???
Top
 Profile  
 
DaleHutch
PostPosted: Fri Dec 23, 2011 3:54 am 
Joomla! Apprentice
Joomla! Apprentice

Joined: Sun Mar 21, 2010 5:43 am
Posts: 20
Location: Denver, CO
I'm in the same boat!

This spider has been hammering my site recently. I'm doing some tests in my .htaccess file to see if I can stop it.
Top
 Profile  
 
mandville
PostPosted: Fri Dec 23, 2011 11:13 am 
User avatar
Joomla! Master
Joomla! Master

Joined: Mon Mar 20, 2006 1:56 am
Posts: 11706
Location: The Girly Side of Joomla in Sussex
check your logs and ban the IP

_________________
HU2HY- Poor questions = Poor answer
Un requested Help PM's will be added to the foe list and possibly just deleted
{Community.Connect Administrator }{ Showcase & Security Moderator}
Top
 Profile  
 
DaleHutch
PostPosted: Mon Dec 26, 2011 6:38 pm 
Joomla! Apprentice
Joomla! Apprentice

Joined: Sun Mar 21, 2010 5:43 am
Posts: 20
Location: Denver, CO
IPs banned....

Looking okay now.

Thanks :)
Top
 Profile  
 
talks44
PostPosted: Wed Dec 28, 2011 3:48 am 
Joomla! Enthusiast
Joomla! Enthusiast

Joined: Wed Dec 28, 2011 12:28 am
Posts: 132
Location: USA
Could you maybe post your robots.txt file? Perhaps you had something incorrectly scripted.

Would be worth checking before you attempt to go through and ban IPs, especially if you're not too certain of what you're doing. A simple mistype can ban entire regions rather than specifics you may be trying to ban
Top
 Profile  
 
Igal Zeifman
PostPosted: Thu Aug 16, 2012 9:47 am 
User avatar
Joomla! Apprentice
Joomla! Apprentice

Joined: Tue May 22, 2012 2:19 pm
Posts: 6
Hi,
Sorry for reviving an old thread but ,just in case some, this is what you need to know about Baidubot:

a.) from many reports I`ve seen, it will ignore robots.txt rules
b.) to stop it you`ll need to set up an IP based deny access rule
c.) contrary to popular belief, Baidubot will use other IPs besides 180.26...

If needed you can find full user-agent list here: Botopedia.org
There is also an IP check function in place, so you can verify any IP you will want to block.

Hope this helps.
Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  Page 1 of 1
  [ 6 posts ] 


Board index » Other Forums » The Lounge

Who is online

Users browsing this forum: No registered users and 32 guests

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
Powered by phpBB® Forum Software © phpBB Group