Skip to main content
deleted 28 characters in body
Source Link
7ochem
  • 7.6k
  • 14
  • 52
  • 80

I have the following in my robots.txt for months now:

 

Disallow: /catalogsearch/
Disallow: /catalogsearch/
Disallow: /webshop/catalogsearch/

Disallow: /webshop/catalogsearch/

Disallow: /catalogsearch/
Disallow: /catalogsearch/*
Disallow: /webshop/catalogsearch/*
Disallow: /webshop/catalogsearch/

My webshop url is like www.webshopdomain.com/webshop

 

The problem is:
  Google still shows and crawling all search results (39.000 ..)
What

What am I doing wrong?

I have the following in my robots.txt for months now:

 

Disallow: /catalogsearch/
Disallow: /catalogsearch/
Disallow: /webshop/catalogsearch/

Disallow: /webshop/catalogsearch/

My webshop url is like www.webshopdomain.com/webshop

 

The problem is:
  Google still shows and crawling all search results (39.000 ..)
What am I doing wrong?

I have the following in my robots.txt for months now:

Disallow: /catalogsearch/
Disallow: /catalogsearch/*
Disallow: /webshop/catalogsearch/*
Disallow: /webshop/catalogsearch/

My webshop url is like www.webshopdomain.com/webshop

The problem is: Google still shows and crawling all search results (39.000 ..)

What am I doing wrong?

Source Link

Google Ignore's Magento's robots.txt

I have the following in my robots.txt for months now:

Disallow: /catalogsearch/
Disallow: /catalogsearch/
Disallow: /webshop/catalogsearch/

Disallow: /webshop/catalogsearch/

My webshop url is like www.webshopdomain.com/webshop

The problem is:
Google still shows and crawling all search results (39.000 ..)
What am I doing wrong?