Googlebot can't access my site??!
Moderator: General Support Moderators
Forum rules
Forum Rules
Absolute Beginner's Guide to Joomla! <-- please read before posting, this means YOU.
Forum Post Assistant - If you are serious about wanting help, you will use this tool to help you post.
Forum Rules
Absolute Beginner's Guide to Joomla! <-- please read before posting, this means YOU.
Forum Post Assistant - If you are serious about wanting help, you will use this tool to help you post.
-
- Joomla! Intern
- Posts: 68
- Joined: Thu Jun 28, 2012 5:02 pm
Googlebot can't access my site??!
Hi everyone, all of a sudden, I am getting an error on webmaster tools that googlebot can't access my site. Any ideas on what I need to do? Sorry, i'm new!
Thanks!!
Thanks!!
You do not have the required permissions to view the files attached to this post.
-
- Joomla! Apprentice
- Posts: 12
- Joined: Sun Aug 26, 2012 6:12 am
- Location: Jakarta
- Contact:
Re: Googlebot can't access my site??!
Show us your robot.txt
- Igal Zeifman
- Joomla! Apprentice
- Posts: 6
- Joined: Tue May 22, 2012 2:19 pm
- Contact:
Re: Googlebot can't access my site??!
Hi,
My advice is (in the following order):
1.) Check your robots.txt for Disallow rules (i.e. 'Disallow:\' will ban all access)
2.) Check your source for no-index/no-follow tags
3.) If still no luck, go to WebMasterTools and try to "Fetch as Googlebot"
4.) Try to get a hold of your access logs and check for Googlebot user-agents to better understand your issue (when happened and why banned).
You can get full Googlebot user-agent list here (Botopedia.org)
My advice is (in the following order):
1.) Check your robots.txt for Disallow rules (i.e. 'Disallow:\' will ban all access)
2.) Check your source for no-index/no-follow tags
3.) If still no luck, go to WebMasterTools and try to "Fetch as Googlebot"
4.) Try to get a hold of your access logs and check for Googlebot user-agents to better understand your issue (when happened and why banned).
You can get full Googlebot user-agent list here (Botopedia.org)
-
- Joomla! Intern
- Posts: 68
- Joined: Thu Jun 28, 2012 5:02 pm
Re: Googlebot can't access my site??!
Is this what you mean? (see attached). Thanks!haryono wrote:Show us your robot.txt
You do not have the required permissions to view the files attached to this post.
-
- Joomla! Intern
- Posts: 74
- Joined: Sat Mar 29, 2008 3:52 pm
Re: Googlebot can't access my site??!
Hello,
This is a known bug in Webmaster Tools. If you fetch as Googlebot Google will come back with a positive fetch. Then, a couple days will go by and you will get the same message. There are some threads on the Google Webmaster forum about this, but Google has not responded. It does not seem to be an issue with most sites and does not impact rankings.
This is a known bug in Webmaster Tools. If you fetch as Googlebot Google will come back with a positive fetch. Then, a couple days will go by and you will get the same message. There are some threads on the Google Webmaster forum about this, but Google has not responded. It does not seem to be an issue with most sites and does not impact rankings.
-
- Joomla! Intern
- Posts: 68
- Joined: Thu Jun 28, 2012 5:02 pm
Re: Googlebot can't access my site??!
Thanks! I didn't know that. It has effected my site though. I was showing strong, and now nothing. Disappeared.flashgordonweb wrote:Hello,
This is a known bug in Webmaster Tools. If you fetch as Googlebot Google will come back with a positive fetch. Then, a couple days will go by and you will get the same message. There are some threads on the Google Webmaster forum about this, but Google has not responded. It does not seem to be an issue with most sites and does not impact rankings.
-
- Joomla! Intern
- Posts: 68
- Joined: Thu Jun 28, 2012 5:02 pm
Re: Googlebot can't access my site??!
I just found this post in this forum! Could it be the cause? http://forum.joomla.org/viewtopic.php?f=618&t=757022
-
- Joomla! Intern
- Posts: 68
- Joined: Thu Jun 28, 2012 5:02 pm
Re: Googlebot can't access my site??!
I'm totally confused now! (sorry...new to this ;-). And THANKS for the help!! I have one site that indexes/crawls fine. And the other site has an error message at the top of webmaster tools and no longer shows for searches (but showed strong up until this ocured). I'll also note that the site having this issue is an upgrade to 2.5.7 (this happened after i upgraded), and the site that works fine is not upgraded and is on 2.5.6. :
" Googlebot can't access your site
Aug 9, 2012"
They BOTH have the EXACT robots.txt contents. I'll paste them here:
GOOD SITE (CRAWLED):
# If the Joomla site is installed within a folder such as at
# e.g. http://www.example.com/joomla/ the robots.txt file MUST be
# moved to the site root at e.g. http://www.example.com/robots.txt
# AND the joomla folder name MUST be prefixed to the disallowed
# path, e.g. the Disallow rule for the /administrator/ folder
# MUST be changed to read Disallow: /joomla/administrator/
#
# For more information about the robots.txt standard, see:
# http://www.robotstxt.org/orig.html
#
# For syntax checking, see:
# http://www.sxw.org.uk/computing/robots/check.html
User-agent: *
Disallow: /administrator/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /images/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /libraries/
Disallow: /logs/
Disallow: /media/
Disallow: /modules/
Disallow: /plugins/
Disallow: /templates/
Disallow: /tmp/
BAD SITE (NOT CRAWLED AND DISAPPEARED FROM SEARCH RESULTS AFTER SHOWING STRONG FOR A MONTH--and after I upgraded to 2.5.7):
# If the Joomla site is installed within a folder such as at
# e.g. http://www.example.com/joomla/ the robots.txt file MUST be
# moved to the site root at e.g. http://www.example.com/robots.txt
# AND the joomla folder name MUST be prefixed to the disallowed
# path, e.g. the Disallow rule for the /administrator/ folder
# MUST be changed to read Disallow: /joomla/administrator/
#
# For more information about the robots.txt standard, see:
# http://www.robotstxt.org/orig.html
#
# For syntax checking, see:
# http://www.sxw.org.uk/computing/robots/check.html
User-agent: *
Disallow: /administrator/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /images/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /libraries/
Disallow: /logs/
Disallow: /media/
Disallow: /modules/
Disallow: /plugins/
Disallow: /templates/
Disallow: /tmp/
Let me know your thoughts. Thank you!!! BB
" Googlebot can't access your site
Aug 9, 2012"
They BOTH have the EXACT robots.txt contents. I'll paste them here:
GOOD SITE (CRAWLED):
# If the Joomla site is installed within a folder such as at
# e.g. http://www.example.com/joomla/ the robots.txt file MUST be
# moved to the site root at e.g. http://www.example.com/robots.txt
# AND the joomla folder name MUST be prefixed to the disallowed
# path, e.g. the Disallow rule for the /administrator/ folder
# MUST be changed to read Disallow: /joomla/administrator/
#
# For more information about the robots.txt standard, see:
# http://www.robotstxt.org/orig.html
#
# For syntax checking, see:
# http://www.sxw.org.uk/computing/robots/check.html
User-agent: *
Disallow: /administrator/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /images/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /libraries/
Disallow: /logs/
Disallow: /media/
Disallow: /modules/
Disallow: /plugins/
Disallow: /templates/
Disallow: /tmp/
BAD SITE (NOT CRAWLED AND DISAPPEARED FROM SEARCH RESULTS AFTER SHOWING STRONG FOR A MONTH--and after I upgraded to 2.5.7):
# If the Joomla site is installed within a folder such as at
# e.g. http://www.example.com/joomla/ the robots.txt file MUST be
# moved to the site root at e.g. http://www.example.com/robots.txt
# AND the joomla folder name MUST be prefixed to the disallowed
# path, e.g. the Disallow rule for the /administrator/ folder
# MUST be changed to read Disallow: /joomla/administrator/
#
# For more information about the robots.txt standard, see:
# http://www.robotstxt.org/orig.html
#
# For syntax checking, see:
# http://www.sxw.org.uk/computing/robots/check.html
User-agent: *
Disallow: /administrator/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /images/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /libraries/
Disallow: /logs/
Disallow: /media/
Disallow: /modules/
Disallow: /plugins/
Disallow: /templates/
Disallow: /tmp/
Let me know your thoughts. Thank you!!! BB
-
- Joomla! Apprentice
- Posts: 11
- Joined: Wed Oct 03, 2012 12:53 am
- Location: Jakarta
- Contact:
Re: Googlebot can't access my site??!
How many joomla sites in your host root? how you install them? in directory or use sub domain?
Try something http://joomblaster.blog.com/ listed in http://jbs.web.id/
-
- Joomla! Intern
- Posts: 68
- Joined: Thu Jun 28, 2012 5:02 pm
Re: Googlebot can't access my site??!
I have 3. I set them up through cloudaccess.netdbqt wrote:How many joomla sites in your host root? how you install them? in directory or use sub domain?
-
- Joomla! Apprentice
- Posts: 15
- Joined: Fri Sep 28, 2012 6:08 am
- Location: delhi
Re: Googlebot can't access my site??!
sometimes google finds it difficult to crawl website because of server problems
the same has been with my websites too, contact your hosting company. however these are temporarily problems.
the same has been with my websites too, contact your hosting company. however these are temporarily problems.