0

When i test my site in mobile friendly test suggested google it gives error.

Page cannot be reached
This could be because the page is unavailable or blocked by robots.txt

I tried many different things but not able to solve. Any help will be highly appreciated. Thanks

0

It looks like you currency converter is doing something weird with the useragent google bot. When i change my user agent to google bot i get redirected to: http://stage.infinitealoe.com/directory/currency/switch/currency/GBP/uenc/aHR0cDovL3N0YWdlLmluZmluaXRlYWxvZS5jb20,?___store=default

Looks like something to do with cloudflare too

That's where your problem is what I get when my user agent is google bot

| improve this answer | |
  • i removed the above url "stage.infinitealoe.com/directory/currency/switch/currency/GBP/… " still facing the same issue – Arjun Sep 7 '18 at 9:38
  • stilling getting the same error page when my user agent is set to google bot (but not getting redirected). So it looks like cloudflare is blocking googlebot for some reason. I don't know much about cloudfare, but I would guess either magento (or your server) is doing something weird for the googlebot user agent cause cloudfare to think its some sort of DOS attack or maybe some setting in cloudfare is blocking crawling bots... but that's just guesses. – PaddyD Sep 7 '18 at 11:43
  • i have a doubt if googlebot is blocking my site from accessing then shouldn't i face same issue for web. Why am i facing the issue only when i test my site for mobile testing in google. one more thing i am getting 406 in page speed insights of google. – Arjun Sep 8 '18 at 5:31
  • I think you missunderstand, your website/server/cloudfare is blocking googlebot from accessing the website. When I pretend to be googlebot (change my user agent to googlebot) I get that error message. So the mobile testing and speed insights are getting that page I'm getting too (in the screenshot) as they would be using googlebot user agent. You can confirm this too in chrome by changing you user agent with the inspect tool (under network conditions, deselect "select automatically" and select googlebot) – PaddyD Sep 10 '18 at 9:39
  • just to prove the point, I ran your site on this speed site site: webpagetest.org/result/… Which works fine, but when i change the user agent to google bot, webpagetest.org/result/… it gets a 403 error – PaddyD Sep 10 '18 at 11:29

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.