How do I make sure Google doesn't index simple products that are linked to configurable products?

Question

I am getting ready to restructure my product database to use configurable products instead of custom options. I want the configurable products to be indexed by Google, but not the related simple products, since this would create duplicate content. If I have the products set as "Not Visible Individually" will this keep them from being crawled by Google or do I need to do something else?

I read a couple posts from people saying that google was crawling product pages that weren't visible and creating 404 erros so I'm wondering if that was the best practice is.

Answer 1

The standard way to deal with simple products in Magento is to have them set to -"Not Visible Individually". That should be enough.

Answer 2

I realise this is an old ticket, but I found it after trying to searh the answer to another problem.

To answer the question, you will need to add in a module that allows you to "canonicalise" your simple products to your configurable product.

i.e You may want Google to see your simple product, however, you're "canonicalising" - telling Google that there's a more relevent page that it should display instead.

So in Magento, where the product details are for your simple product, you would simply paste in the URL to the Canonical field, so that Google can read that information as it indexes the page.

https://developers.google.com/search/docs/crawling-indexing/canonicalization