8

How do I add the following robots META tag to a CMS page in Magento 2 using the Layout Update XML field?

<meta name="robots" content="noindex,nofollow"/>

I have tried the suggestions in the following questions, but none of them work

For example when I add the following code to the Layout Update XML field:

<head>
    <meta name="robots" content="NOINDEX,NOFOLLOW"/>
</head>

I get this error

Please correct the XML data and try again. Element 'head': This element is not expected. Expected is one of ( referenceContainer, container, update, move ). Line: 1

Improve this question

3 Answers 3

Reset to default
8

To set the robots to NOINDEX,NOFOLLOW for a specific CMS page with a URL key of no-route-2, I usually create 2 files. Yes, this can be done in one file but I usually find that there is more than one page that I want to set the robots for and I don't like repetitive code. I'll show you both ways.

Create file Magento_Theme/layout/noindex_nofollow.xml

<?xml version="1.0"?>
<page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:View/Layout/etc/page_configuration.xsd">
    <head>
        <meta name="robots" content="NOINDEX,NOFOLLOW"/>
    </head>
</page>

Create file Magento_Theme/layout/cms_page_view_id_no-route-2.xml

<?xml version="1.0"?>
<page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:View/Layout/etc/page_configuration.xsd">
    <update handle="noindex_nofollow" />
</page>

After this, flush the cache and refresh the page and your robots for that one specific page will be updated. For any additional pages, just create duplicate the cms_page_view_id_no-route-2.xml file contents but be sure to name the new file appropriate to match the next page you are doing this for.

I should also mention, that you probably want to set this page's robots to NOINDEX,FOLLOW. That is because you dont want to index the page but you should want the search engines to continue crawling your site, assuming that the page contains mostly internal links.

In that case, you would do:

Create file Magento_Theme/layout/noindex_follow.xml

<?xml version="1.0"?>
<page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:View/Layout/etc/page_configuration.xsd">
    <head>
        <meta name="robots" content="NOINDEX,FOLLOW"/>
    </head>
</page>

Create file Magento_Theme/layout/cms_page_view_id_no-route-2.xml

<?xml version="1.0"?>
<page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:View/Layout/etc/page_configuration.xsd">
    <update handle="noindex_follow" />
</page>

The way to do this in one file for one specific page is:

Create file Magento_Theme/layout/cms_page_view_id_no-route-2.xml

<?xml version="1.0"?>
<page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:View/Layout/etc/page_configuration.xsd">
    <head>
        <meta name="robots" content="NOINDEX,NOFOLLOW"/>
    </head>
</page>
Improve this answer
17
  • Just for clarification, can I paste that entire snippet in the Layout Update XML field for the CMS page I am trying to update? Or do I have to create a dedicated Magento_Theme/layout/noindex_nofollow.xml layout file in order for it to work?
    – EpixRu
    May 3, 2018 at 15:29
  • When I try to paste the entire snippet I get an error Please correct the XML data and try again. XML declaration allowed only at the start of the document Line: 1
    – EpixRu
    May 3, 2018 at 15:30
  • Ah yes, this snippet is intented to live in a file within your theme. Unfortunately, in M2, I have not had much success setting robots by pasting the content into the "Layout Update XML" section. It worked in M1 but in M2 it still does not.
    – Shawn Abramson
    May 3, 2018 at 16:10
  • Btw, as long as you have access to your file system, you can still target a specific page, you just need to determine the layout handle for the specific page.
    – Shawn Abramson
    May 3, 2018 at 16:10
  • I tried your suggestion, but I am still unable to get it to work. I created the file app/design/frontend/<vendor>/<theme>/Magento_Theme/layout/noindex_nofollow.xml with the XML snippet you provided. On my CMS page I inserted <update handle="noindex_nofollow" /> and was able to save it successfully. Then I cleared my cache. But the page on the frontend still has the default <meta name="robots" content="INDEX,FOLLOW"/> meta tag. Is there something obvious that I am overlooking?
    – EpixRu
    May 3, 2018 at 16:49
6

It looks like this is a bug in Magento. You should be able to update HEAD tags or specifically the robots META tags using XML in the Layout Update XML field.

Here is a link to the issue on Github with the solution. https://github.com/magento/magento2/issues/4454

After vendor/magento/framework/View/Layout/etc/page_layout.xsd has been fixed you should be able to use

<head>
    <meta name="robots" content="NOINDEX,NOFOLLOW"/>
</head>

or 

<head>
    <meta name="robots" content="NOINDEX,FOLLOW"/>
</head>

in the Layout Update XML field of the page in order to update the robots META tag.

Improve this answer
2
  • I see you found a solution. Sorry to have left you hanging, I left for vacation before I could answer your new question. But glad to see you figured it out and taught me something that I should have known but didnt :-)
    – Shawn Abramson
    May 14, 2018 at 18:31
  • No worries Shawn, thanks for the assist. Your targeting solution will helpful for other needs.
    – EpixRu
    May 15, 2018 at 20:24
1

You can simply edit your existing Magento CMS pages to include a snippet to have the noindex and / or nofollow attributes set.

It will use the robots meta tag to tell the robots what to do.

To edit a page’s properties, simply click on CMS -> Pages , and open the page you would like to edit.

You will now see the page editor. Click on the Design tab to the left of the editor.

Use the field Layout Update XML to insert the code you want to use. You can find examples below.

<page xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="urn:magento:framework:View/Layout/etc/page_configuration.xsd">
    <head>
        <meta name="robots" content="NOINDEX,NOFOLLOW"/>
    </head>
</page>
Improve this answer
4
  • Magento recommends against using "action" in Magento 2. Also, I don't think reference is valid as I have never seen it used. As far as I know, you can only use referenceBlock or referenceContainer in M2.
    – Shawn Abramson
    May 3, 2018 at 6:03
  • @ShawnAbramson , sorry i just edit my code. will you please check it. now it works
    – Divyesh
    May 3, 2018 at 6:06
  • @Divyesh When I used the XML in your post I received an "Element 'page': This element is not expected." error I removed the page tags and tried again, but received another error "Element 'head': This element is not expected."
    – EpixRu
    May 3, 2018 at 14:51
  • @here - Magento devs who are running with the Magento v2.3.4 or later can find their solution here because XML input field is no longer exists in v2.3.4 or later. devdocs.magento.com/guides/v2.3/frontend-dev-guide/layouts/…
    – Ajay Gehlod
    Jun 2, 2020 at 20:54

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.