So, my robots.txt explicitly says not to crawl any URLs containing a URL Parameter. And that works as expected.

However, Google Search Console is moaning with the following issue:

Indexed, though blocked by robots.txt

Affected Pages 1.88k

I would like to know if I missed a setting in Magento that allows me to set auto-generated pages based on URL Parameters and Page Numbers to NOINDEX, NOFOLLOW whilst leaving the rest of the site set to INDEX, FOLLOW.

If Magento doesn't support this out-the-box, what options do I have?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.