The Joomla! Forum ™



Forum rules


Forum Rules
Absolute Beginner's Guide to Joomla! <-- please read before posting, this means YOU.
Forum Post Assistant - If you are serious about wanting help, you will use this tool to help you post.



Post new topic Reply to topic  [ 11 posts ] 
Author Message
PostPosted: Sun Aug 26, 2012 2:19 am 
Joomla! Intern
Joomla! Intern

Joined: Thu Jun 28, 2012 5:02 pm
Posts: 68
Hi everyone, all of a sudden, I am getting an error on webmaster tools that googlebot can't access my site. Any ideas on what I need to do? Sorry, i'm new!

Thanks!!


You do not have the required permissions to view the files attached to this post.


Top
 Profile  
 
PostPosted: Sun Aug 26, 2012 6:52 am 
Joomla! Apprentice
Joomla! Apprentice

Joined: Sun Aug 26, 2012 6:12 am
Posts: 12
Location: Jakarta
Show us your robot.txt


Top
 Profile  
 
PostPosted: Sun Aug 26, 2012 2:06 pm 
User avatar
Joomla! Apprentice
Joomla! Apprentice

Joined: Tue May 22, 2012 2:19 pm
Posts: 6
Hi,
My advice is (in the following order):

1.) Check your robots.txt for Disallow rules (i.e. 'Disallow:\' will ban all access)
2.) Check your source for no-index/no-follow tags
3.) If still no luck, go to WebMasterTools and try to "Fetch as Googlebot"
4.) Try to get a hold of your access logs and check for Googlebot user-agents to better understand your issue (when happened and why banned).

You can get full Googlebot user-agent list here (Botopedia.org)


Top
 Profile  
 
PostPosted: Thu Sep 13, 2012 10:02 pm 
Joomla! Intern
Joomla! Intern

Joined: Thu Jun 28, 2012 5:02 pm
Posts: 68
haryono wrote:
Show us your robot.txt


Is this what you mean? (see attached). Thanks!


You do not have the required permissions to view the files attached to this post.


Top
 Profile  
 
PostPosted: Thu Sep 13, 2012 11:08 pm 
Joomla! Intern
Joomla! Intern

Joined: Sat Mar 29, 2008 3:52 pm
Posts: 74
Hello,

This is a known bug in Webmaster Tools. If you fetch as Googlebot Google will come back with a positive fetch. Then, a couple days will go by and you will get the same message. There are some threads on the Google Webmaster forum about this, but Google has not responded. It does not seem to be an issue with most sites and does not impact rankings.


Top
 Profile  
 
PostPosted: Tue Sep 25, 2012 2:40 pm 
Joomla! Intern
Joomla! Intern

Joined: Thu Jun 28, 2012 5:02 pm
Posts: 68
flashgordonweb wrote:
Hello,

This is a known bug in Webmaster Tools. If you fetch as Googlebot Google will come back with a positive fetch. Then, a couple days will go by and you will get the same message. There are some threads on the Google Webmaster forum about this, but Google has not responded. It does not seem to be an issue with most sites and does not impact rankings.


Thanks! I didn't know that. It has effected my site though. I was showing strong, and now nothing. Disappeared.


Top
 Profile  
 
PostPosted: Thu Sep 27, 2012 10:39 pm 
Joomla! Intern
Joomla! Intern

Joined: Thu Jun 28, 2012 5:02 pm
Posts: 68
I just found this post in this forum! Could it be the cause? viewtopic.php?f=618&t=757022


Top
 Profile  
 
PostPosted: Fri Sep 28, 2012 3:03 pm 
Joomla! Intern
Joomla! Intern

Joined: Thu Jun 28, 2012 5:02 pm
Posts: 68
I'm totally confused now! (sorry...new to this ;-). And THANKS for the help!! I have one site that indexes/crawls fine. And the other site has an error message at the top of webmaster tools and no longer shows for searches (but showed strong up until this ocured). I'll also note that the site having this issue is an upgrade to 2.5.7 (this happened after i upgraded), and the site that works fine is not upgraded and is on 2.5.6. :

" Googlebot can't access your site
Aug 9, 2012"

They BOTH have the EXACT robots.txt contents. I'll paste them here:

GOOD SITE (CRAWLED):
# If the Joomla site is installed within a folder such as at
# e.g. http://www.example.com/joomla/ the robots.txt file MUST be
# moved to the site root at e.g. http://www.example.com/robots.txt
# AND the joomla folder name MUST be prefixed to the disallowed
# path, e.g. the Disallow rule for the /administrator/ folder
# MUST be changed to read Disallow: /joomla/administrator/
#
# For more information about the robots.txt standard, see:
# http://www.robotstxt.org/orig.html
#
# For syntax checking, see:
# http://www.sxw.org.uk/computing/robots/check.html

User-agent: *
Disallow: /administrator/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /images/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /libraries/
Disallow: /logs/
Disallow: /media/
Disallow: /modules/
Disallow: /plugins/
Disallow: /templates/
Disallow: /tmp/


BAD SITE (NOT CRAWLED AND DISAPPEARED FROM SEARCH RESULTS AFTER SHOWING STRONG FOR A MONTH--and after I upgraded to 2.5.7):

# If the Joomla site is installed within a folder such as at
# e.g. http://www.example.com/joomla/ the robots.txt file MUST be
# moved to the site root at e.g. http://www.example.com/robots.txt
# AND the joomla folder name MUST be prefixed to the disallowed
# path, e.g. the Disallow rule for the /administrator/ folder
# MUST be changed to read Disallow: /joomla/administrator/
#
# For more information about the robots.txt standard, see:
# http://www.robotstxt.org/orig.html
#
# For syntax checking, see:
# http://www.sxw.org.uk/computing/robots/check.html

User-agent: *
Disallow: /administrator/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /images/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /libraries/
Disallow: /logs/
Disallow: /media/
Disallow: /modules/
Disallow: /plugins/
Disallow: /templates/
Disallow: /tmp/


Let me know your thoughts. Thank you!!! BB


Top
 Profile  
 
PostPosted: Wed Oct 03, 2012 2:00 am 
Joomla! Apprentice
Joomla! Apprentice

Joined: Wed Oct 03, 2012 12:53 am
Posts: 11
Location: Jakarta
How many joomla sites in your host root? how you install them? in directory or use sub domain?

_________________
Try something http://joomblaster.blog.com/ listed in http://jbs.web.id/


Top
 Profile  
 
PostPosted: Wed Oct 03, 2012 4:48 pm 
Joomla! Intern
Joomla! Intern

Joined: Thu Jun 28, 2012 5:02 pm
Posts: 68
dbqt wrote:
How many joomla sites in your host root? how you install them? in directory or use sub domain?


I have 3. I set them up through cloudaccess.net


Top
 Profile  
 
PostPosted: Sat Oct 20, 2012 3:12 am 
Joomla! Apprentice
Joomla! Apprentice

Joined: Fri Sep 28, 2012 6:08 am
Posts: 15
Location: delhi
sometimes google finds it difficult to crawl website because of server problems
the same has been with my websites too, contact your hosting company. however these are temporarily problems.

_________________
http://www.apnatimepass.com


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 11 posts ] 



Who is online

Users browsing this forum: No registered users and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
Powered by phpBB® Forum Software © phpBB Group