when I try to set the robots tags by xml or programatically in _preparelayout, it doesn't work. Instead they are always NOINDEX, FOLLOW. This also happens for categorysearch but I cannot find the code where the robots-tags are changed and actually set.

in local.xml I tried:

<tag_product_list translate="label">
        <label>Tagged Products List</label>
        <!-- Mage_Tag -->
        <reference name="head">
        <action method="setRobots"><value>INDEX,FOLLOW</value></action>

in Tag/Block/Product/result.php I tried

$head = $this->getLayout()->getBlock('head');
$head->setRobots('index, nofollow');

Setting the description works fine. But the robot-tags are always set to NOINDEX,FOLLOW.

Can anyone tell me, where the robot-tags for tag pages or catalogsearch pages are set?


3 Answers 3


Create your custom module to do so.

<?xml version="1.0"?>



class Custom_MetaChange_Model_Observer extends Mage_Core_Model_Abstract {

    public function changeRobots($observer) {

                if ($observer->getEvent()->getAction()->getFullActionName() == 'catalogsearch_result_index') {
                $layout = $observer->getEvent()->getLayout();
                $product_info = $layout->getBlock('head');
                $layout->getUpdate()->addUpdate('<reference name="head"><action method="setRobots"><value>NOINDEX,FOLLOW</value></action></reference>');

Please create your module activation file.
What I think that you are trying to change robots tags after xml has been generated, you need to do it before xml is generated.
Hoping this will help you.

  • So why am I still able to change my description? I updated my original post as I accidently posted the wrong xml-code to change the robot tags. That is definitly before the xml is generated. I'm guessing somewhere deep down the data gets overwritten. I will try your solution later. And finally setup xdebug. Thanks
    – kilsen
    Sep 25, 2014 at 11:39
  • Yes, using xdebug will be give you much idea. please post your findings here also. Sep 25, 2014 at 12:22

The robots tag can be set in your admin panel

System -> Configuration -> Design -> HTML Head -> Default Robots

be sure to check which store view you are on for this setting as the store view will override the default config. Make sure this setting is correct for what you want your store's pages to default to, then update the pages you want to change via XML.

Assuming you want to set these pages to INDEX/FOLLOW edit your local.xml file with the following:

<!-- Advanced Search page -->
  <reference name="head">
    <action method="setRobots"><value>INDEX,FOLLOW</value></action>

<!-- Search Results page -->
<catalogsearch_result_index translate="label">
  <reference name="head">
    <action method="setRobots"><value>INDEX,FOLLOW</value></action>

<!-- Advanced Search page -->
<catalogsearch_advanced_index translate="label">
  <reference name="head">
    <action method="setRobots"><value>INDEX,FOLLOW</value></action>

Next, in the admin go to

System -> Cache Management

and flush your Magento caches. Go to the front end of your website and make sure you clear your local cache also. View source on the page and you should see your updated robots info.

Note: I have not tested the section but the catalog search results pages have been tested and do work. A word of caution, many sites set the catalog search results and advanced search results to NOINDEX, NOFOLLOW to prevent search engine penalties for duplicate content.

  • Thank you for your detailed answer. I tried all that but I just couldn't override those tags. Somehow my local.xml still got overriden by another xml file. Do you know why this could have happened? Yes, I am aware of duplicate content. I added static blocks to my tag pages so I obviously want those pages indexed. I also added layered navigation so I hope this doesn't cause any issues.
    – kilsen
    Sep 25, 2014 at 16:20
  • Magento's XML structure loads local.xml last so it will override any other XML file. However if there is even 1 error in local.xml the whole file won't work. I'd suggest trying it out in pieces to see what does and does not work. Create a back up of your local.xml file and then test the following: Sep 25, 2014 at 19:31
  • <?xml version="1.0"?> <layout version="0.1.0"> <!-- Search Results page --> <catalogsearch_result_index translate="label"> <reference name="head"> <action method="setRobots> <value>NOINDEX,NOFOLLOW</value> </action> </reference> </catalogsearch_result_index> </layout> Sep 25, 2014 at 19:32
  • Save local.xml and clear your Magento caches. Refresh the search results page and see if it took effect. Also, be sure that you are putting local.xml in to the correct package and theme folder. You should see the package and theme settings in the admin System -> Configuration -> Design - Package and Theme sections. Especially check that your Theme section Layout is set to look in the correct theme folder. If local.xml is not in the correct theme folder Magento may not see it. Sep 25, 2014 at 19:33

Okay those robot tags were set by my template (argento). What I don't understand is why my local.xml didn't override it. Shouldn't that be default behaviour? The argento.xml file is located in /default, my current template in /argento with my local.xml . I guess my settings in Design/Themes aren't correct. Has anyone any idea how to make sure local.xml is always on "top"?

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.