A client has a Magento 2.3.2 multistore with 5 stores. For convenience I name them store 1/5. The store has created 5 sitemaps in magentoroot/sitemap/{storename}/sitemap.xml.

The robots.txt I assume are configured under content/design/ -> edit / Search Engine Robots.

The problem: Google is not indexing store 2/5 because the sitemap refers to the sitemap of store 1.

I assume Magento needs to inject the correct sitemap into the robots.txt for each domain. However here it gets curious.

I added a "Disallow: /test/" to the Global SEO settings. This results in store 2/5 also getting this /test/ marker in robots.txt, but not store 1 (which is the main store). It is not taking the settings from the Global/Website 2/5 SEO settings.

My questions.

  1. How can I make sure the right sitemap gets injected in robots.txt
  2. How can I make sure each of these 5 domains gets their own robots.txt

I really appreciate the feedback.

  • I'm also having this issue on Magento ver. 2.3.5-p2
    – Bernard
    Aug 4 at 20:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.