How to solve Google webmaster toots crawling error

Everything to do with Joomla! 1.5 templates and templating.

Moderator: General Support Moderators

Forum rules
Forum Rules
Absolute Beginner's Guide to Joomla! <-- please read before posting, this means YOU.
Locked
jahir07
Joomla! Apprentice
Joomla! Apprentice
Posts: 19
Joined: Fri Jun 03, 2011 12:26 pm
Contact:

How to solve Google webmaster toots crawling error

Post by jahir07 » Fri Mar 09, 2012 12:00 pm

Fm online radio -->> http://thefmradio.com
Disability cars http://disabilitycars.net

User avatar
Bjorg
Joomla! Ace
Joomla! Ace
Posts: 1557
Joined: Tue Jul 07, 2009 4:45 am

Re: How to solve Google webmaster toots crawling error

Post by Bjorg » Mon Mar 12, 2012 2:44 am

404 error Not found = Bad Link from the Web Site

Restricted by robots.txt = Joomla! server root directory / folder where Joomla! is installed has a file called "robots.txt" which tells search engines what and what not to index. Apparently, the robots.txt is telling either all search engines, or only G00gle, not to index the directory / folders shown upon the server.

500 Error could be server directory / folder file permissions.

All of which may be set using the web hosting service "cpanel", "File Manager" to include editing of the file "robots.txt"

Hope this helps!

jahir07
Joomla! Apprentice
Joomla! Apprentice
Posts: 19
Joined: Fri Jun 03, 2011 12:26 pm
Contact:

Re: How to solve Google webmaster toots crawling error

Post by jahir07 » Mon Mar 12, 2012 5:16 am

Bjorg wrote:404 error Not found = Bad Link from the Web Site

Restricted by robots.txt = Joomla! server root directory / folder where Joomla! is installed has a file called "robots.txt" which tells search engines what and what not to index. Apparently, the robots.txt is telling either all search engines, or only G00gle, not to index the directory / folders shown upon the server.

500 Error could be server directory / folder file permissions.

All of which may be set using the web hosting service "cpanel", "File Manager" to include editing of the file "robots.txt"

Hope this helps!
Here is my robot.txt:
# If the Joomla site is installed within a folder such as at
# e.g. http://www.example.com/joomla/ the robots.txt file MUST be
# moved to the site root at e.g. http://www.example.com/robots.txt
# AND the joomla folder name MUST be prefixed to the disallowed
# path, e.g. the Disallow rule for the /administrator/ folder
# MUST be changed to read Disallow: /joomla/administrator/
#
# For more information about the robots.txt standard, see:
# http://www.robotstxt.org/orig.html
#
# For syntax checking, see:
# http://www.sxw.org.uk/computing/robots/check.html

User-agent: *
Disallow: /administrator/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /images/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /libraries/
Disallow: /logs/
Disallow: /media/
Disallow: /modules/
Disallow: /plugins/
Disallow: /templates/
Disallow: /tmp/

Is this enough? Can you tell me total webmaster guide for joomla?
Fm online radio -->> http://thefmradio.com
Disability cars http://disabilitycars.net

User avatar
Bjorg
Joomla! Ace
Joomla! Ace
Posts: 1557
Joined: Tue Jul 07, 2009 4:45 am

Re: How to solve Google webmaster toots crawling error

Post by Bjorg » Tue Mar 13, 2012 12:50 am

This is basic web site setup that technically has nothing to do with Joomla!

The following should allow a better understanding of web site administration tools:

Below is a single explanation for each of the above categories to the original question:
404 error Not found ‎(2,088) : For Mar 7, 2012
http://www.e-opensources.com/Oracle/Page-7?start=188 << The web site has a dead link that does not point to anything on this web site [remove the link or put something there]
Restricted by robots.txt(350)‎: For Mar 7, 2012
http://www.e-opensources.com/templates/ ... e-save.php << The robots.txt file tells all search engines to:
Disallow: /templates/ << This tell all SERP's to not index, look inside the /public_html/wherejoomlaisinstalled/templates/ folder [if it is desired to allow, then remove this line from the robots.txt file]
It also shows 500 Error and sitemap error << This is an error by the web server where Joomla! is installed and has nothing to do with Joomla! [install the correct Joomla! version, to the correct server mysql and php versions]
Perhaps do a search for each of these errors to better understand each one in detail as this is not a Joomla! specific problem affecting functionality, it is a web masters choice of tools to maintain a web site.

Hope this helps!

jahir07
Joomla! Apprentice
Joomla! Apprentice
Posts: 19
Joined: Fri Jun 03, 2011 12:26 pm
Contact:

Re: How to solve Google webmaster toots crawling error

Post by jahir07 » Fri Mar 16, 2012 4:51 pm

@Bjorg
thanks dude.. I will follow your instruction. Hope this help me to better index in search engine. can You please tell me.. What is the Proper way to seo..

thanks again.
Fm online radio -->> http://thefmradio.com
Disability cars http://disabilitycars.net

User avatar
Bjorg
Joomla! Ace
Joomla! Ace
Posts: 1557
Joined: Tue Jul 07, 2009 4:45 am

Re: How to solve Google webmaster toots crawling error

Post by Bjorg » Sun Mar 18, 2012 4:19 am

jahir07 wrote:@Bjorg
thanks dude.. I will follow your instruction. Hope this help me to better index in search engine. can You please tell me.. What is the Proper way to seo..
thanks again.
I am going through learning SEO myself and would not be of much help. However, I do have some beginner suggestions:

Use words within the URL web site address in title, headings (for large article or content), content and any sidebar modules.

Keep all contents of pages relative to the specific words within the URL.

Ensure the web page reflects an author, description, title and content to search engines.

Check all spelling and grammar (to include meta descriptions and keywords.)

Match meta descriptions to each linked word, phrase or image name.

Stick with either php code or .htaccess hard 301 redirects.

Use robots.txt to keep spiders and SERP's from indexing unnecessary pages to index or follow
(e.g., in my humble opinion such pages may be the registration form page, contact page, etc.)

Not wise to duplicate anyone else’s content even with permission (original images are the exception.)
This includes the same content that is displayed on more than one web site owned by the same person.

Do not spam forums for backlinks, its tacky, undesired and bothersome. Beside, anyone can set all backlines
to no follow which would defeat the initial spamming purpose. Though not proven, SERP’s may use this as an excuse
to ban a domain or set of domains in the future under the digital copyright laws.

FYI:
In the United States the Library of Congress is beginning to archive all Twitter posts up to 140 characters as of March 2006:

http://blogs.loc.gov/loc/2010/04/how-tw ... r-archive/

Hope this helps!

jahir07
Joomla! Apprentice
Joomla! Apprentice
Posts: 19
Joined: Fri Jun 03, 2011 12:26 pm
Contact:

Re: How to solve Google webmaster toots crawling error

Post by jahir07 » Sun Apr 15, 2012 4:36 am

Have you know any plugin which solve this problem? Now shows 3000+ error
:'(

before i was use sef404 plugin. but on that time shows 4000+ error..
I am totally fed-up about this site..
Fm online radio -->> http://thefmradio.com
Disability cars http://disabilitycars.net

User avatar
Bjorg
Joomla! Ace
Joomla! Ace
Posts: 1557
Joined: Tue Jul 07, 2009 4:45 am

Re: How to solve Google webmaster toots crawling error

Post by Bjorg » Sun Apr 15, 2012 6:09 am

Post URL to live site please and explain the "3000-4000" error's as Joomla! "out-of-the-box" is generally error free? Perhaps it is the content or the way in which it was imported into the database and utilized with Joomla! that is the problem?

Hope this helps!

jahir07
Joomla! Apprentice
Joomla! Apprentice
Posts: 19
Joined: Fri Jun 03, 2011 12:26 pm
Contact:

Re: How to solve Google webmaster toots crawling error

Post by jahir07 » Sun Apr 15, 2012 9:57 am

which redirection plugin is nice to solve 404error?

and can i upload all 404error in database? and how to solve next?
Fm online radio -->> http://thefmradio.com
Disability cars http://disabilitycars.net

User avatar
Bjorg
Joomla! Ace
Joomla! Ace
Posts: 1557
Joined: Tue Jul 07, 2009 4:45 am

Re: How to solve Google webmaster toots crawling error

Post by Bjorg » Sun Apr 15, 2012 2:51 pm

jahir07 wrote:Please help me.. I have a site (http://e-opensources.com) based on joomla 2.5. It was 1.6. I upgrade it 2.5. But at the beginning I am facing the following problem--
404 error Not found ‎(2,088) : For Mar 7, 2012
http://www.e-opensources.com/Oracle/Page-7?start=188
http://www.e-opensources.com/Pharmacy/h ... rol-weight
http://www.e-opensources.com/Oracle/Page-19?start=155
http://www.e-opensources.com/Computers/ ... des-manual
http://www.e-opensources.com/Oracle/[b] ... r&start=82 [/b]
http://www.e-opensources.com/Oracle/Page-2?start=188
http://www.e-opensources.com/Oracle/Page-9?start=188
..... etc.
Restricted by robots.txt(350)‎: For Mar 7, 2012
http://www.e-opensources.com/templates/ ... e-save.php
http://www.e-opensources.com/plugins/system/rokbox/
http://www.e-opensources.com/templates/zt_news_iii16/
http://www.e-opensources.com/administrator/
http://www.e-opensources.com/templates/ja_nex/
http://www.e-opensources.com/libraries/gantry
http://www.e-opensources.com/index.php? ... Itemid=201
......etc
It also shows 500 Error and sitemap error.. I solve sitemap error but i can't solve above error.. I have another site http://ebanglanewspaper.com which was created by wordpress.. But that site has no 404 error. Please help me with detail.
For the 404 error, we will use a URL that was given above and assume that the URL is this vary same web site:

http://[email protected]/Pharmacy/h ... rol-weight

The above link results in the following text actually shown in a web browser URL address bar:

http://[email protected]/Pharmacy/h ... weight.php

Then add various suffixes to the end of the URL (e.g., htm, php, etc) to see if the Joomla! “SEO Settings,” “Add suffix to URLs” has a problem with interpretation.

Now the URL address http://[email protected]/Pharmacy/h ... rol-weight may be shortened to the “Category” of “Pharmacy.”

http://www.e-opensources.com/Pharmacy/ (which also gives a 404 error.)

Since, the “Category” of “/Pharmacy/” from the URL does not exist, the “/helps-control-weight/” article title may exist, but the assigned category within the Joomla! “Article Manager” does not select “Pharmacy,” or the category of “Pharmacy” as this was deleted after the above given link was created.

The entire web site may have this problem and if the URL’s were created and then an administrator deleted all the “Sections” or “Categories” then the web site will have many 404 error’s.

It is certain that one of the URL’s provided has been deleted as shown by the 404 error not found.

Hope this helps!

jahir07
Joomla! Apprentice
Joomla! Apprentice
Posts: 19
Joined: Fri Jun 03, 2011 12:26 pm
Contact:

Re: How to solve Google webmaster toots crawling error

Post by jahir07 » Mon Apr 16, 2012 2:35 am

http://e-opensources.com/education/phar ... rol-weight
this is now exist. yah i was edit category. now it is sub category of "eduction". Have you any redirection extension which auto redirect to modified link like wordpress "Redirection" plugin which Manage all your 301 redirects and monitor 404 errors. i use in my wordpress site (http://www.ebanglanewspaper.com) and its well control my 404 error.

Or do u have any extension which omit category just show main url like e-opensources.com/helps-control-weight.php or e-opensources.com/helps-control-weight.html
Fm online radio -->> http://thefmradio.com
Disability cars http://disabilitycars.net

User avatar
Bjorg
Joomla! Ace
Joomla! Ace
Posts: 1557
Joined: Tue Jul 07, 2009 4:45 am

Re: How to solve Google webmaster toots crawling error

Post by Bjorg » Mon Apr 16, 2012 1:15 pm

If this is about a specific Joomla! extension from the JED here at this domain, then perhaps make a new post
within that extensions place within the Joomla! forum for help.

Place in the Joomla! forum for "Extensions for Joomla! 2.5 / Joomla! 1.7":

http://forum.joomla.org/viewforum.php?f=620

Place in the Joomla! forum for "Extensions for Joomla! 1.5":

http://forum.joomla.org/viewforum.php?f=470

Place in the Joomla! forum for extensions pertaining to "Joomla! 1.0":

http://forum.joomla.org/viewforum.php?f=505

or

Place in the Joomla! forum for "Extensions.Joomla.org - Feedback/Information":

http://forum.joomla.org/viewforum.php?f=262

or

Make a new post at the author’s web site for the Joomla! third party extension by either reading the documentation
that came with the extension files, or by searching through the Joomla! Extensions Directory (JED):

http://extensions.joomla.org/


Since this is specific to G00gle Webmaster Tools the perhaps register at that web site domain and ask that question
in the forum for further assistance.

Here is the link to the G00gle Webmaster Group Forum page to allow web masters to ask specific questions about G00gle indexing services:

http://groups.google.com/a/googleproduc ... webmasters

Here is a good starting place for G00gle questions about "FAQ: Crawling, indexing & ranking":

http://sites.google.com/site/webmasterh ... ---ranking

Lastly, please review the forum “Rules” page as it is consistent with many other web sites when posting for assistance: http://forum.joomla.org/viewtopic.php?f=8&t=65 .)

Good luck!

paabhatti91
Joomla! Fledgling
Joomla! Fledgling
Posts: 1
Joined: Wed May 25, 2016 7:36 pm

Re: How to solve Google webmaster toots crawling error

Post by paabhatti91 » Wed May 25, 2016 7:58 pm

i think first you check your robot.txt and then you check your soft 404s and in last you also check sitemapes. and i am using different method on different website so you can also check # Ecouter Radio En Direct and FM Radio Online Streaming


Locked

Return to “Templates for Joomla! 1.5”