1

I know this question has been asked plenty of times before, but from what I've read, no one has ever really got to the bottom of it.

Whilst this excellent post (How best can I structure product configuration for SEO?) finalises when attributes and categories should be used, what I've yet to balance is the implications of using layered navigation from a search engine optimisation perspective.

I've been using an extension from Amasty recently to help out with layered nav, and to be fair, it functions well. However an SEO associate has told me that he's pretty horrified at the number of unique URL's that layered navigation produces. I balanced that off by saying that I can set certain aspects of the layered nav to "No Follow", but he informs me that it makes no difference.

He advised me that really, all layered nav links should include a hash tag, so that search engines will categorically not even look at them. For example -

mysite.com/shoes#shopby/red

There is an extension that I've found that does this, but I'm not convinced. I would assume that if this was the case, everyone would use it, but it's from a pretty unknown developer.

I would be really grateful for any feedback on this.

Thanks

  • Can't you just set a canonical tag? – Rick Kuipers Jun 4 '13 at 19:12
  • Hi Rick, thanks for you reply. I'm told that this is no good because Google will still waste time crawling them which will have enormous effects on rankings. – JamesAllwood Jun 4 '13 at 19:32
  • Then put up a robots.txt which has a Disallow rule for the filters with some wildcards? – Rick Kuipers Jun 4 '13 at 19:56
  • Hmm, that seems like a great idea. I'll check with my associate that this would solve the problem. Assuming that all of my layered nav links included the term "shopby", I should be able to do something like - 'code'User-agent: Googlebot [code]Disallow: /shopby/ Does that seem correct? – JamesAllwood Jun 4 '13 at 20:31
  • Yes that is correct if the URL begins with shopby. IF the URL doesn't begin with shopby you could do this: /*/shopby/ – Rick Kuipers Jun 5 '13 at 6:40
2

Old question, but in case someone fighting this issue still finds it:

One nice solution to fix the SEO issues caused by the endless number of unique URLs created by Layered Navigation is using the PRG Pattern.

Simply said, it's about replacing the GET request to a layered navigation/filter URL with a POST request (which search engine crawlers do not follow) before redirecting the user to the original layered navigation/filter URL.

This works like a charm, i. e. not changing the UX regarding Layered Navigation and 100% reliable in terms of preventing crawlers from wasting crawl budget on useless duplicate content URLs.

For further details and reading, please see

  1. Detailed explanation incl. sample request flow
  2. Why robots.txt, rel=nofollow etc. are no satisfying solutions here
  3. PRG Pattern Magento 2 Extension
  4. PRG Pattern Demo
1

Put up a robots.txt with a disallow for your filters. This should prevent crawlers from crawling the page.

User-agent: *
Disallow: /*/shopby

Depending on the way the URL is formed for your filters. I'm sure you could get a disallow rule that would match your needs.

EDIT

The only other thing I can think of is put this: rel="nofollow" in the <a href> tags that point to a filter. This should be easily doable in the templates.

  • Hi Rick. Well, apparently, whilst doing this will stop Google from crawling the content, it actually still crawls and indexes the url. The problem is not solved, it means that you can use up your crawl very quickly. Any other ideas? – JamesAllwood Jun 5 '13 at 10:33
  • See my edit, I don't think you can go any further than this. – Rick Kuipers Jun 5 '13 at 11:12
0

It all depends, if you have your technical SEO setup correctly it becomes irrelevant (more or less). The worse your hosting & architecture the more you have to worry about the small details. As no-one can know your exact configuration from technical to hosts to product urls to product count and more - it is difficult to advise. We can't really bore you with the details about technical seo as it is what the enterprise companies use. As already mentioned, you can use the robots.txt, but always remember this is just one small part of the seo problem - basically ~5%. We can tell you with have seen a CE site with only internal SEO (no category path product urls nor extensions) and a standard robots.txt listing first page next to Amazon & eBay with the same products using short tail keywords. True, the hosting was $2,000/mth although now they released second generation which is $900 or so. Skip the canonicals and just use straight product urls with one or two keyword prefixes. Basically if you start fixing this with extensions it will be the start, not the end - if your hosting is really cheap then to be honest, this will be the least of your problems.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.