Is there a way to block Google and other engines from seeing your filtered pages? We get emails talking about how Google is seeing "an extremely high number of URLs" and when I look, it's all the color options and other kinds of sidebar options we have - but it's useless content.

I tried robots.txt and nothing really is being blocked from the testing tools. I added shopby to be be disallowed and it seems to block individual pages but nothing after .com/category/shopby/ (anything else I wanted blocked)



1 Answer 1


Magento has the setRobots method to insert the values in the robots tag within the head block. So you can create a extension to change the head block via observer. In your config.xml add a event to change layout before Magento get layout xml done.


There are other events that you can use to do this, maybe even more elegant than that. But now you can handle the layout settings dynamically:

public function changeRobots($observer) {
    if ($observer->getEvent()->getAction()->getFullActionName() === 'catalog_category_view') {
        $uri = $observer->getEvent()->getAction()->getRequest()->getRequestUri();
        if (stristr($uri, "?")) {
            $layout = $observer->getEvent()->getLayout();
            $product_info = $layout->getBlock('head');
            $layout->getUpdate()->addUpdate('<reference name="head"><action method="setRobots"><value>NOINDEX,FOLLOW</value></action></reference>');

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.