1

I have created a robots.txt and sitemap.xml files using Magento UI but when trying to access www.mydomain.com/robots.txt or www.mydomain.com/sitemap.xml it gives me 404 not found.

I found that robots.txt and sitemap.xml file are created in mydomain/http/, however, the pub directory is set to be my root so moving the file to the pub fixes the issue.

Why Magento created the files in this location? If I want to make changes to the robots.txt file, I can't use the native Magento feature? What is the best solution to this?

I am using NginX and added this to my configuration file which "solves" the issue, however I am not sure if this is the best solution:

# robots
location = /robots.txt {
    alias /var/www/example.com/html/robots.txt;
}

# sitemap
location = /sitemap.xml {
    alias /var/www/example.com/html/sitemap.xml;
}

I am using Magento 2.1.8.

2 Answers 2

1

This seems to be fixed in Magento 2.2.1, see https://github.com/magento/magento2/pull/9094.

2
  • Great! Thanks for getting back :) I hope someone else find this useful :)
    – Greg
    Mar 20, 2018 at 9:48
  • 1
    2.2.9 and I don't see any change in magento's behaviour.
    – omsta
    Nov 19, 2019 at 10:53
0

I am running 2.3.3 and it is not fixed.

The workaround was to add /pub/sitemap.xml in the field and then generate.

Magento backend will show the link with the /pub/ like this: http://example.com/pub/sitemap.xml, which will show 404 since the vhost is pointed to pub.

However, the file is generated inside the pub folder. That means it can be accessed as http://example.com/sitemap.xml.

Hope this helps someone.

Hope this helps.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.