Where are robot tags for categorysearch and tag pages set?

Question

when I try to set the robots tags by xml or programatically in _preparelayout, it doesn't work. Instead they are always NOINDEX, FOLLOW. This also happens for categorysearch but I cannot find the code where the robots-tags are changed and actually set.

in local.xml I tried:

<tag_product_list translate="label">
        <label>Tagged Products List</label>
        <!-- Mage_Tag -->
        <reference name="head">
        <action method="setRobots"><value>INDEX,FOLLOW</value></action>
    </reference>
</tag_product_list>

in Tag/Block/Product/result.php I tried

$head = $this->getLayout()->getBlock('head');
$head->setDescription($description);
$head->setRobots('index, nofollow');

Setting the description works fine. But the robot-tags are always set to NOINDEX,FOLLOW.

Can anyone tell me, where the robot-tags for tag pages or catalogsearch pages are set?

Thanks

Answer 1

Create your custom module to do so.
app/code/local/Custom/MetaChange/etc/config.xml

<?xml version="1.0"?>
<config>
    <modules>
        <Custom_MetaChange>
            <version>1.0.0</version>
        </Custom_MetaChange>
    </modules>
    <global>
        <models>
            <metaChange>
                <class>Custom_MetaChange_Model</class>
            </metaChange>
        </models>
    </global>
    <frontend>
        <events>
            <controller_action_layout_generate_xml_before>
                <observers>
                    <noindex>
                        <type>singleton</type>
                        <class>metaChange/observer</class>
                        <method>changeRobots</method>
                    </noindex>
                </observers>
            </controller_action_layout_generate_xml_before>
        </events>
    </frontend>
</config>

app/code/local/Custom/MetaChange/Model/Observer.php

<?php

class Custom_MetaChange_Model_Observer extends Mage_Core_Model_Abstract {

    public function changeRobots($observer) {

                if ($observer->getEvent()->getAction()->getFullActionName() == 'catalogsearch_result_index') {
                $layout = $observer->getEvent()->getLayout();
                $product_info = $layout->getBlock('head');
                $layout->getUpdate()->addUpdate('<reference name="head"><action method="setRobots"><value>NOINDEX,FOLLOW</value></action></reference>');
                $layout->generateXml();
            }
    }
}

Please create your module activation file.
What I think that you are trying to change robots tags after xml has been generated, you need to do it before xml is generated.
Hoping this will help you.

Answer 2

The robots tag can be set in your admin panel

System -> Configuration -> Design -> HTML Head -> Default Robots

be sure to check which store view you are on for this setting as the store view will override the default config. Make sure this setting is correct for what you want your store's pages to default to, then update the pages you want to change via XML.

Assuming you want to set these pages to INDEX/FOLLOW edit your local.xml file with the following:

<!-- Advanced Search page -->
<tag_product_list>
  <reference name="head">
    <action method="setRobots"><value>INDEX,FOLLOW</value></action>
  </reference>
</tag_product_list>

<!-- Search Results page -->
<catalogsearch_result_index translate="label">
  <reference name="head">
    <action method="setRobots"><value>INDEX,FOLLOW</value></action>
  </reference>
</catalogsearch_result_index>

<!-- Advanced Search page -->
<catalogsearch_advanced_index translate="label">
  <reference name="head">
    <action method="setRobots"><value>INDEX,FOLLOW</value></action>
  </reference>
</catalogsearch_advanced_index>

Next, in the admin go to

System -> Cache Management

and flush your Magento caches. Go to the front end of your website and make sure you clear your local cache also. View source on the page and you should see your updated robots info.

Note: I have not tested the section but the catalog search results pages have been tested and do work. A word of caution, many sites set the catalog search results and advanced search results to NOINDEX, NOFOLLOW to prevent search engine penalties for duplicate content.

Answer 3

Okay those robot tags were set by my template (argento). What I don't understand is why my local.xml didn't override it. Shouldn't that be default behaviour? The argento.xml file is located in /default, my current template in /argento with my local.xml . I guess my settings in Design/Themes aren't correct. Has anyone any idea how to make sure local.xml is always on "top"?