The Joomla! Forum ™

Download

Demo


Board index » Joomla! Older Version Support » Joomla! 1.5 » Search Engine Optimization (Joomla! SEO) in Joomla! 1.5

Forum rules


Please click here to view the forum rules



Post new topic Reply to topic  Page 1 of 1
  [ 9 posts ] 
  Print view Previous topic | Next topic 
Author Message
sfpkent
PostPosted: Thu Apr 28, 2011 3:42 am 
User avatar
Joomla! Explorer
Joomla! Explorer

Joined: Wed Jun 07, 2006 10:56 am
Posts: 317
Location: Malaysia
Hi,

Xenu has problem to crawl my site to generate Xenu sitemap.xml. There are a lot of forbidden request (pls refer to screen shot).

May I know how to solve the problem?

Thanks!
Kent


You do not have the required permissions to view the files attached to this post.

_________________
Internet Marketing Consultant and Web Developer
http://humanwebsite.com.my
http://worldwebcreator.com
Top
 Profile  
 
sfpkent
PostPosted: Thu Apr 28, 2011 4:20 am 
User avatar
Joomla! Explorer
Joomla! Explorer

Joined: Wed Jun 07, 2006 10:56 am
Posts: 317
Location: Malaysia
btw, my site is http://humanwebsite.com.my.

I am wondering that if Xenu can't crawl the pages which are forbidden request, does that mean Googlebot can't crawl as well?

Kent

_________________
Internet Marketing Consultant and Web Developer
http://humanwebsite.com.my
http://worldwebcreator.com
Top
 Profile  
 
g1smd
PostPosted: Thu Apr 28, 2011 11:00 am 
Joomla! Guru
Joomla! Guru

Joined: Mon Feb 21, 2011 4:02 pm
Posts: 951
Location: UK
It looks like your site security settings block Xenu.

Here's what Google sees: http://www.google.com/search?num=100&q= ... Newsletter

Google is spidering the calendar and you are presenting Infinite Duplicate Content with thousands of almost blank pages all with the exact same title and same meta description. Heck, the title doesn't even include a date!

_________________
Online since 1995.
Top
 Profile  
 
sfpkent
PostPosted: Thu Apr 28, 2011 11:08 am 
User avatar
Joomla! Explorer
Joomla! Explorer

Joined: Wed Jun 07, 2006 10:56 am
Posts: 317
Location: Malaysia
Is it because of the calendar so Xenu can't crawl those forbidden request pages?

I have removed the calendar.

_________________
Internet Marketing Consultant and Web Developer
http://humanwebsite.com.my
http://worldwebcreator.com
Top
 Profile  
 
sfpkent
PostPosted: Thu Apr 28, 2011 11:17 am 
User avatar
Joomla! Explorer
Joomla! Explorer

Joined: Wed Jun 07, 2006 10:56 am
Posts: 317
Location: Malaysia
I just tried again, but the same thing happen....

I suspect that sh404sef causes the problem.

_________________
Internet Marketing Consultant and Web Developer
http://humanwebsite.com.my
http://worldwebcreator.com
Top
 Profile  
 
g1smd
PostPosted: Thu Apr 28, 2011 11:36 am 
Joomla! Guru
Joomla! Guru

Joined: Mon Feb 21, 2011 4:02 pm
Posts: 951
Location: UK
Something on your site detects the Xenu User Agent and blocks it.

_________________
Online since 1995.
Top
 Profile  
 
sfpkent
PostPosted: Thu Apr 28, 2011 4:50 pm 
User avatar
Joomla! Explorer
Joomla! Explorer

Joined: Wed Jun 07, 2006 10:56 am
Posts: 317
Location: Malaysia
ok, thanks, I am looking Xenu forum, I tried to google it, but it seems there is none. Does xenu has forum?

_________________
Internet Marketing Consultant and Web Developer
http://humanwebsite.com.my
http://worldwebcreator.com
Top
 Profile  
 
Disa192
PostPosted: Mon May 07, 2012 8:23 pm 
Joomla! Fledgling
Joomla! Fledgling

Joined: Mon May 07, 2012 8:16 pm
Posts: 1
sfpkent wrote:
... There are a lot of forbidden request (pls refer to screen shot).

May I know how to solve the problem?

Thanks!
Kent



Hi! Just check your security settings of your sh404SEF component and switch OFF anti-flood control. "Configuration - security - antiflood" (something like this)
Cheers!
Top
 Profile  
 
ashokbabacomp
PostPosted: Wed Mar 20, 2013 8:24 am 
Joomla! Apprentice
Joomla! Apprentice

Joined: Tue Jan 15, 2013 5:51 am
Posts: 17
Thanks, it helped me :p 8)
Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  Page 1 of 1
  [ 9 posts ] 


Board index » Joomla! Older Version Support » Joomla! 1.5 » Search Engine Optimization (Joomla! SEO) in Joomla! 1.5

Who is online

Users browsing this forum: Majestic-12 [Bot] and 7 guests

You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
Powered by phpBB® Forum Software © phpBB Group