0

We're seeing the search term "user" as one of our top 5 search terms with a popularity of 9520 that continues to increase. While it does return 3 products, this seems like an unusual search for our site. Google Analytics does not show this term in its site search metrics for our site, so it doesn't appear to be user generated.

How can I understand where this unusual search term is coming from and take measures to correct it?

  • you can sort your access log, obviously you must have some application, log management service. etc. but you can quickly do something like this in console: awk '/?q=user/ {print $1,$4,$11}' access.log | sort | uniq -c | sort -nr this is only as example because not sure what is you environment. – user2857 Jun 6 '15 at 11:50
  • this will give you just an ip address: awk '/catalogsearch/ {print $1}' access.log | sort | uniq -c | sort -nr – user2857 Jun 6 '15 at 11:56
  • @ADM, thanks for pointing out that the answer would be in the access logs even if Google wasn't recording the hit. Both of your awk scripts are useful. If you create an answer, or edit Reid's, I'll accept it. – ScottYoumans 19 mins ago – ScottYoumans Jun 8 '15 at 17:28
1

simple way:
if you need to see something realtime, sort log data, and get quick reports, you can install goaccess. http://goaccess.io/screenshots

advanced way: you can quickly do something like this in console:
awk '/?q=user/ {print $1,$4,$11}' access.log | sort | uniq -c | sort -nr
this is only as example because not sure what is you environment.

this will give you just an ip address:
awk '/catalogsearch/ {print $1}' access.log | sort | uniq -c | sort -nr

and you can even go further with geoip lookup:

for IPADDR in $(awk '/catalogsearch/ {print $1}' access.log | sort | uniq | sort -nr |head -15) 
do
echo "=============================="
curl ipinfo.io/${IPADDR}
done

perfect way: configure fail2ban or csf firewall

3

You'll have to do some digging through server access logs (without modifying catalogsearch) to see who is posting that string to the search endpoint (should be able to find the path relatively easy if you just grep through logs). Find out the IP/Client and ban them (likely some unsavory spider or some such).

Additionally you may want to disallow the entire catalogsearch path in your robots.txt (see this example at inchoo )

  • robots.txt is only for good robots, for any other evil robots you need to use firewall or .htaccess, etc... – user2857 Jun 6 '15 at 10:18
  • @ADM feel free to edit the answer; definitely overgeneralized there. – Reid Blomquist Jun 6 '15 at 10:21

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.