In robots.txt can be only one directive host. How right use directive host in robots.txt in magento multisite configuration with many domains?

1 Answer 1


Create the necessary re-write rule so that any request to the robots.txt file in the domain root is deferred to a custom request handler, for e.g. robots.php in the document root of your Magento installation.

Now, create the request handler robots.php file in the document root of your Magento installation dir:

/home/www-data/public_html/robots.php Add the following code:


// Parse robots file location using server name
$server_name = 
    : 'default';
$robots_file = getcwd() .'/robots/'. $server_name .'/robots.txt';

// Check if robot file exists for current domain, else default
$robots_file = 
    ? $robots_file 
    : getcwd() .'/robots/default/robots.txt';

// Serve robots file
header("Content-Type: text/plain");
header("Content-Length: ". filesize($robots_file));
exit; ?>

Check out this link for more details - How to serve robots.txt file for multi-store Magento setup

  • Its not recommended to only post a link, please write the answer in here
    – Klettseb
    Jul 12, 2016 at 12:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.