Google Crawl Error: Blocked due to access forbidden (403)

[john2024]

Hi there
Site = https://gudpal.com
Joomla = Joomla5
Php = 8.1

I submitted my site's sitemap to Google Search Console, which gave a status of "Success". See attached screenshot for details.

After 24 hours, I then checked my site homepage on https://search.google.com/test/rich-results but I am getting Google Crawl Error "Failed: Blocked due to access forbidden (403)". See attached screenshot for details.

I have checked my site and couldn't find anything that's blocking Google Bot. I have also checked our CDN Cloudflare Security and its associated log Events, but couldn't find anything that's logged there in relation to Googlebot.

I am not sure if its got to do anything with mod_security. I see that its installed but don't know how to check its rules.

Secondly, I am not sure if its got to do with permissions on any directories, or with settings under robots.txt.

Any guess why Google Crawl 403 error is coming?

Many thanks!
John

[ltheme]

Hi,
Can you recheck it? I do SEO task, and sometime I also got it, look like it's cache issue from your website.

[john2024]

Hi,
I cleared all cache - in both website and CDN Cloudflare, but still getting the 403 error for any webpage I test on https://search.google.com/test/rich-results

Any other suggestions?

Many thanks!
John

[john2024]

Also, I just checked my site's robots.txt and it also contains the following text:
User-Agent: Googlebot
Allow: /*.js*
Allow: /*.css*
Allow: /*.png*
Allow: /*.jpg*
Allow: /*.gif*

User-agent: *
Disallow: /administrator/
Disallow: /api/
Disallow: /bin/
Disallow: /cache/
Disallow: /cli/
Disallow: /components/
Disallow: /includes/
Disallow: /installation/
Disallow: /language/
Disallow: /layouts/
Disallow: /libraries/
Disallow: /logs/
Disallow: /modules/
Disallow: /plugins/
Disallow: /tmp/

[Per Yngve Berg]

Mod. Note: Relocated topic to the SEO Forum.

[ltheme]

Hi,

I see you enabled DDOS checking process from CL. You can disable it and recheck.
Btw, I tested your domain from Google.com and I already see it's indexed with some links
Try go to google.com
and type:
"site:https://gudpal.com/"

[Per Yngve Berg]

I visited your site and was presented with a captcha. I had to click "I'm not a robot" to get in. That is probably what is preventing Google access.

[CityPop]

Hey there - looks like a couple of people have helped answer this question for you, but this happened to me and I was able to fix it pretty quickly. Review your robots.txt file to ensure that it's not blocking access to critical resources or entire sections of your site. Ensure that there are no directives like "Disallow: /" that might prevent Googlebot from accessing your site.