1

I've been having major issues with robots leaving sessions files. Over 100 sessions / second. At some point Redis just stopped accepting the sessions and Magento went to writing them back to disk. I have no idea what happened there. I flushed the DB and it still refused to use Redis. But that is a problem for another day because I then discovered the session problem.

So I've changed robots to limit crawls every 30 minutes and I blocked the Baiduspider/2.0 user agent to no avail.

I created a CRON job to search the session files every 15 minutes because I still want legitimate sessions to exist for a week.

grep -lrIZ Baiduspider/2.0; /var/session/* | xargs -0 rm -f --

So my question is there anything better / cleaner to handle this. Followup is there a command for Redis that will do the same thing I'm doing with CRON and the file system

Improve this question
3
  • Honestly since this is more around bots/spiders and Redis than Magento: blog-en.openalfa.com/… & webmasters.stackexchange.com/questions/29883/…
    – B00MER
    Mar 6, 2016 at 5:08
  • 1
    I've already made changes like these. It's not reducing the amount of bot session files. In addition I'm still looking for a solution for removing sessions of bots from Redis while leaving legitimate sessions.
    – innerspace
    Mar 6, 2016 at 7:32
  • How do you currently have Redis configured in your local.xml?
    – B00MER
    Mar 6, 2016 at 22:10

0

Reset to default

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Browse other questions tagged or ask your own question.