Google indexing Issues - Redirect

[jjspelman]

Google Search Console is continuously telling us that pages are not being indexed due to a page redirect. There are no redirects in the Joomla Redirect component, there are no redirects in the .htaccess file (other than the https).



I have tried adding a redirect plugin in the event that the pages were not being sent to www or vice/versa but this didn't work.

The URL is pensacolacarpetandtilecleaning.com

Has anyone experienced anything similar?

Any thoughts appreciated.

[Webdongle]

Yes, when I move or add menu items and don't get google to recrawl the sitemap

[jjspelman]

But I have resubmitted the sitemap many times.

The strange thing is that if I do a "site" search on Google for this domain, ie: "site:pensacolacarpetandtilecleaning.com", the URLs that claim not to be indexed (as shown in the screenshot), show up in that search result. I am at a loss and the client is flipping out.

[Per Yngve Berg]

Mod. Note: Relocated the Topic to the SEO Forum.

You don't redirect to https in the .htaccess file.

Go into Global Configuration and set "Force SSL Entire Site"

[jjspelman]

Correct, I meant www - that is what is in the .htaccess file, not the https.

[Webdongle]

I seem to recall you can tell google to treat www and non www url's as the same web address?

[jjspelman]

I'm not sure about that. I was always under the impression that you had to choose canonical locally.

[shumisha]

Hi

Your screenshot shows 33 pages are concerned by this. It would help a lot if you stated clearly one or a few of these URLs.
When you click on that "33", what are the exact first few URLs where they see a redirect?

[jjspelman]

Hi, thanks for asking. Here are the first ten of them:

https://pensacolacarpetandtilecleaning.com/about-us

https://pensacolacarpetandtilecleaning. ... stretching

https://pensacolacarpetandtilecleaning.com/

https://pensacolacarpetandtilecleaning. ... or-removal

https://pensacolacarpetandtilecleaning. ... ter-damage

https://pensacolacarpetandtilecleaning. ... ter-damage

https://pensacolacarpetandtilecleaning. ... e-cleaning

https://pensacolacarpetandtilecleaning.com/index

https://pensacolacarpetandtilecleaning.com/index.php

https://pensacolacarpetandtilecleaning. ... cal-choice

[shumisha]

Hi

> https://pensacolacarpetandtilecleaning.com/about-us

They are correct, trying to reach this URL, I am redirected to https://WWW.pensacolacarpetandtilecleaning.com/about-us

[jjspelman]

Thank you, this does seem to be the case. But where are they being instructed to try and index those pages? The site maps that have been submitted all contain the www prefix.

I appreciate your input!

[shumisha]

Hi,

The sitemap is not where search engines find links. Its purpose is to tell them which links are more important and which links you "prefer" them to use, in case there's a choice (ie same page has different urls).

For sites with less than 500 pages (per google documentation), a sitemap is not needed.

> But where are they being instructed to try and index those pages?

Somewhere on your site, you have these links. Search engines crawls your pages to find links to explore.
If they had found those links on other websites, they would normally not take them into account due to the redirect (unless the redirect was put in place only recently)

[Webdongle]

https://developers.google.com/search/bl ... ns-of-site

[stutteringp0et]

When I look at your robots.txt file to see your sitemaps, neither one works. If those are the sitemap urls you submitted to Google, they're not working.

[Per Yngve Berg]

The site maps that have been submitted all contain the www prefix.

The redirected URL have WWW with Capital Letters.

[jjspelman]

That is curious. The redirect is written as follows:

Code: Select all

	RewriteEngine On
	RewriteCond %{HTTP_HOST} ^pensacolacarpetandtilecleaning.com [NC] 
	RewriteRule ^(.*)$ http://www.pensacolacarpetandtilecleaning.com/$1 [L,R=301]

The sitemap.xml and html files all have lower case www.

I don't see where WWW would come in?

[shumisha]

Hi,

There's no such thing, the WWW does not exist. I only wrote it that way in my message so that you would notice the difference between with and without the www.

Note that even if you were using capitals (WWW) that would not matter as domain names are case-insensitive - contrary to URLs which are case-sensitiev for Google (but not Bing).

Have you found where Google finds these links without the www, and therefore list them as "not indexed"?

Also, to be clear, these warnings do not matter at all and are not a problem, as long as the same URL with www is indexed normally.

ie: as long as if https://www.pensacolacarpetandtilecleaning.com/about-us, then it's not really a worry that https://pensacolacarpetandtilecleaning.com/about-us is NOT indexed. If both version were indexed, that would be an issue.

Also,as mentioned above by @stutteringp0et, you should remove these broken sitemaps links from your robots.txt. They are not really causing a problem (as they are invalid, no URL is submitted) but just to keep things tidy.

[jjspelman]

Have you found where Google finds these links without the www, and therefore list them as "not indexed"?

I found a few internal links on the site that were linking to the non-www home page but not much else.

Also, I did just delete those lines to the old sitemap files in robots.txt. Somehow I did not see that. Thanks.

[shumisha]

I found a few internal links on the site that were linking to the non-www home page but not much else.

Hmm, is it possible that these links were submitted as part of the now-deleted sitemaps?

[jjspelman]

I don't believe so. The old Jsitemap was pulling from the menu system. The site owner added those links manually in the blog section. So he neglected to include w...

[shumisha]

Hi

The site owner added those links manually in the blog section. So he neglected to include w...

If this happened at a time when there was no redirect from no-www to www, then this would explain that, because using a link to home page without the www would cause Joomla to render all links without the www as well.
So this only becomes odd if the www redirect is only recent or was not working at some point.

Note that this may have happened a long time ago, and they only show in the Search console now. Data in Search console comes and go, and is not updated as quickly as we'd like sometimes.