2

I have magento 1.7. For seo reasons I am getting Duplicate meta descriptions Tags problem from google.

For example

  • /paralympics-paralympische-spelen/
  • /paralympics-paralympische-spelen

(with and without the slash)

AND

  • /blog/
  • /blog/?p=2
  • /blog/?p=3
  • /blog/index/list/tag/oranje/

AND

  • /troonswisseling
  • /troonswisseling?dir=asc&limit=48&order=position

How can we make only 1 page exist (instead of all these combinations)

Thanks

5

First I would login to the Webmaster Tools with Google and you can specify parameters for it to ignore like you're experiencing with some URLs. Site Configuration -> URL Parameters

I would also make sure you are submitting sitemap.xml properly to Google and other Search engines to help ensure things are crawled properly.

Robots.txt docs:

Next I would make sure your robots.txt file is being read and recognized. Here is one I've used with success before:

## robots.txt for Magento Community and Enterprise

## GENERAL SETTINGS

## Enable robots.txt rules for all crawlers
User-agent: *

## Crawl-delay parameter: number of seconds to wait between successive requests to the same server.
## Set a custom crawl rate if you're experiencing traffic problems with your server.
# Crawl-delay: 30

## Magento sitemap: uncomment and replace the URL to your Magento sitemap file
# Sitemap: http://www.example.com/sitemap/sitemap.xml

## DEVELOPMENT RELATED SETTINGS

## Do not crawl development files and folders: CVS, svn directories and dump files
Disallow: CVS
Disallow: .svn
Disallow: .idea
Disallow: .sql
Disallow: .tgz

## GENERAL MAGENTO SETTINGS

## Do not crawl Magento admin page
Disallow: /admin/

## Do not crawl common Magento technical folders
Disallow: /app/
Disallow: /downloader/
Disallow: /errors/
Disallow: /includes/
Disallow: /lib/
Disallow: /pkginfo/
Disallow: /shell/
Disallow: /var/

## Do not crawl common Magento files
Disallow: /api.php
Disallow: /cron.php
Disallow: /cron.sh
Disallow: /error_log
Disallow: /get.php
Disallow: /install.php
Disallow: /LICENSE.html
Disallow: /LICENSE.txt
Disallow: /LICENSE_AFL.txt
Disallow: /README.txt
Disallow: /RELEASE_NOTES.txt

## MAGENTO SEO IMPROVEMENTS

## Do not crawl sub category pages that are sorted or filtered.
Disallow: /*?dir*
Disallow: /*?dir=desc
Disallow: /*?dir=asc
Disallow: /*?limit=all
Disallow: /*?mode*

## Do not crawl 2-nd home page copy (example.com/index.php/). Uncomment it only if you activated Magento SEO URLs.
## Disallow: /index.php/

## Do not crawl links with session IDs
Disallow: /*?SID=

## Do not crawl checkout and user account pages
Disallow: /checkout/
Disallow: /onestepcheckout/
Disallow: /customer/
Disallow: /customer/account/
Disallow: /customer/account/login/

## Do not crawl seach pages and not-SEO optimized catalog links
Disallow: /catalogsearch/
Disallow: /catalog/product_compare/
Disallow: /catalog/category/view/
Disallow: /catalog/product/view/

## SERVER SETTINGS

## Do not crawl common server technical folders and files
Disallow: /cgi-bin/
Disallow: /cleanup.php
Disallow: /apc.php
Disallow: /memcache.php
Disallow: /phpinfo.php

## IMAGE CRAWLERS SETTINGS

## Extra: Uncomment if you do not wish Google and Bing to index your images
# User-agent: Googlebot-Image
# Disallow: /
# User-agent: msnbot-media
# Disallow: /

Source: http://turnkeye.com/blog/optimize-robots-txt-for-magento/

6

I believe you are looking for canonical links.

If you go to your admin panel and navigate to system -> Configuration -> Catalog

In the Search Engine Optimisation tab, you can find the following two entries:

  • Use Canonical Link Meta Tag For Categories
  • Use Canonical Link Meta Tag For Products

This will not delete the redundant pages, but will add the same canonical links in head of your html files, like so:

<link rel="canonical" href="http://example.com/paralympics-paralympische-spelen/"/>

That same link will be added for /paralympics-paralympische-spelen/ and /paralympics-paralympische-spelen.

Now when google indexes your site, it will probably still see both links, but since you added a canonical link to it, google will now know that both links are the same page.

  • Strange thing is that both are already set to yes: Use Canonical Link Meta Tag For Categories [STORE VIEW] Use Canonical Link Meta Tag For Products [STORE VIEW] – snh_nl Apr 4 '13 at 6:18
  • And in addtion. I mus t admit that the pages above are all either CMS or BLOG pages (not product and category). The question is then how I would add canonical to also these pages. magentocommerce.com/wiki/adding_canonical_url_to_cms_pages – snh_nl Apr 4 '13 at 6:21
0

I also add these to the robots.txt:

Disallow: /wishlist/
Disallow: /onepagecheckout/
Disallow: /review/

You don't want wishlists as that is user data. I use the onepagecheckout extension so I also exclude that. I choose to hide reviews because they create duplicate product titles out of the box and I haven't found a work-around for this yet.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.