A client has a Magento 2.3.2 multistore with 5 stores. For convenience I name them store 1/5. The store has created 5 sitemaps in magentoroot/sitemap/{storename}/sitemap.xml.

The robots.txt I assume are configured under content/design/ -> edit / Search Engine Robots.

The problem: Google is not indexing store 2/5 because the sitemap refers to the sitemap of store 1.

I assume Magento needs to inject the correct sitemap into the robots.txt for each domain. However here it gets curious.

I added a "Disallow: /test/" to the Global SEO settings. This results in store 2/5 also getting this /test/ marker in robots.txt, but not store 1 (which is the main store). It is not taking the settings from the Global/Website 2/5 SEO settings.

My questions.

  1. How can I make sure the right sitemap gets injected in robots.txt
  2. How can I make sure each of these 5 domains gets their own robots.txt

I really appreciate the feedback.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.