My magento store has the following to allow google/search engines to crawl the entire site.

<meta name="robots" content="INDEX,FOLLOW" />

I am now in need for some particular products to have:

<meta name="robots" content="NOINDEX,NOFOLLOW" />

so that google doesn't crawl/index them. Is this possible? Products would be random (i.e not all from one category) so I need to be able to do this on a product by product basis if needed.

I know i could do it on a url basis like:

<?php
$currentUrl = Mage::helper('core/url')->getCurrentUrl();
if ($currentUrl == 'xxxxxxx')
{
    <meta name="robots" content="NOINDEX,NOFOLLOW" />
}
else
{
    <meta name="robots" content="INDEX,FOLLOW" />
}
?>

but there could end up over time to be hundreds/thousands of these.

up vote 22 down vote accepted

In Magento admin, when editing a product go to the tab design and add the following to custom layout:

<reference name="head">
      <action method="setRobots"><meta>NOINDEX,NOFOLLOW</meta></action>
</reference>
  • +1 BUT he did say there could be thousands of products... – paj Aug 26 '15 at 11:41
  • 1
    @paj true but products are random so this is the only clean option. – Sander Mangel Aug 27 '15 at 9:12
  • Shouldn't it be <value>NOINDEX,NOFOLLOW</value> instead of <meta>NOINDEX,NOFOLLOW</meta>? – Arvind07 Jul 26 '17 at 5:51

you can observe the event controller_action_layout_generate_blocks_after like this:

public function setRobots($observer) 
{
     $controller = $observer->getAction();
     $fullActionName = $controller->getFullActionName();
     if ($fullActionName == 'catalog_product_view') { //if on product page
          $product = Mage::registry('product'); //access the current product if needed
          if (your condition here) { //condition to set the robots to noindex, nofollow
              $observer->getLayout()->getBlock('head')->setRobots('NOINDEX,NOFOLLOW');
          }
     }
}
  • where "your condition here" could be a custom attribute, this would allow you to mass import the custom attribute value quickly for 1000's of products. – paj Aug 26 '15 at 11:43
  • @paj. Sure. It can be anything. A product attribute, a config setting. – Marius Aug 26 '15 at 13:17

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.