0

Google Search throws a problem with a link that is not being indexing

I search the cause and is for a rule of robotx.txt that block the catalog

Disallow: /catalog/

So since the link path starts with catalog/product/view/id it will disallow it from indexing, but since the page of the product is indexable it will try to do it.

I have a custom url_key for the product, but since I can access it as well through the link www.page.com/catalog/product/view/{product_id} Google will try to index it. How can I solve this conflict?

Improve this question

1 Answer 1

Reset to default
0

Disallow: /catalog/ >> Yes it will Block all urls with /catalog like you mentioned catalog/product/view/id it will disallow it from indexing,

But since the page of the product is indexable it will try to do it. >> NO it will not index if started with /catalog/

The link www.page.com/catalog/product/view/{product_id} >> Google will not index untill and unless you have submitted through some another source medium. And if you want some special urls to be indexed you can create search engine friendly urls for Products in any platforms.

Thanks!

Improve this answer
1
  • Hellos, thank you for your response. I check and the URL was in the sitemap.xml file, so I think that is the reason why Google was trying to index although the robotx.txt file blocks that route. The reason why that url was on the sitemap file and not the url-key was because the url-key was duplicated so when magento generated the sitemap file it will use the default url instead of the url key. I fixed this adding a custom url-key to that specified product, generating a new sitemap file in the admin. But I don't how to tell google that try another index of the website with the new sitema
    – ramon colmenares
    Commented Jul 13, 2022 at 19:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.