<meta name="robots"> tag different for different pages

Question

Home page and categories pages have different meta name ="robots" tags.

I found out my website has

<meta name="robots" content="INDEX,FOLLOW"/> 

tag showing for the pages that has 1 column layout 'page-layout-1column' like home page and product pages.

But category pages shows meta tags as

<meta name="robots" content="NOINDEX,NOFOLLOW"/> 

and the layout for category page 2 columns left "page-layout-2columns-left". I need to add INDEX, FOLLOW for category pages also. I found out it may be due to the different page layouts. But I could not locate what I need to change.

I have checked inside content->design->configuration and found out in Search Engine Robots section INDEX, FOLLOW is selected.

I need to add INDEX, FOLLOW for category pages. Please guide me What changes I need to do to achieve this?

Answer 1

I am sure that for might be an extension, NOINDEX,NOFOLLOW is showing at Category page.

If You want to changes robots meta value then on layout_generate_blocks_before event fire an observer and change meta robot value using Magento\Framework\View\Page\Config class.

Here the observer class:

<?php


namespace StackExchange\MagentoTest\Observer;


use Magento\Framework\Event\Observer;
use Magento\Framework\Event\ObserverInterface;
use Magento\Framework\View\Page\Config as PageConfig;

class CategoryPageRobot implements  ObserverInterface
{
    private  $pageConfig;

    public function __construct(
        PageConfig $pageConfig
    ){
        $this->pageConfig = $pageConfig;
    }

    public function execute(Observer $observer)
    {
        $fullActionName = $observer->getEvent()->getData('full_action_name');

        if($fullActionName === 'catalog_category_view'){
            $this->pageConfig->setMetadata('robots','INDEX,FOLLOW');
        }
    }
}

Location: app/code/StackExchange/MagentoTest/Observer/CategoryPageRobot.php

events.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:noNamespaceSchemaLocation="urn:magento:framework:Event/etc/events.xsd">
    <event name="layout_generate_blocks_before">
        <observer name="category_page_meta_robots_value"
                  instance="StackExchange\MagentoTest\Observer\CategoryPageRobot"  />
    </event>
</config>

Location: app/code/StackExchange/MagentoTest/etc/frontend/events.xml