4

I have a problem with a Magento installation. This installation is not live and as such has no traffic etc. I watched in cpanel's file manager and it appears to be generating 15-25 files per minute in the var/sessions directory. I am in the process of searching to see what potential causes there could be but so far I haven't found anything. Does anybody know what I might check?

The reason I discovered this issue ( presumably connected) is that when I browse on the front end I am logged out of the back end.

  • 1
    It could be that they are written to one place (var/session) and read from somewhere else (/tmp/magento/var/session). Could also be a clock issue, is ntpd installed on your server? – Jonathan Selander Apr 11 '14 at 18:02
  • I will check those ... I should have mentioned in question that I have another installation on the same server but it's not exhibiting same behavior – byronyasgur Apr 11 '14 at 18:09
11

These changes will prevent bots from creating sessions in Magento:

Make a copy of app/code/core/Mage/Core/Model/Session/Abstract/Varien.php

to app/code/local/Mage/Core/Model/Session/Abstract/Varien.php

edit this new file add this new function into the file (within the class definition):

public function isBot() {
    $isbot = false;
    $bot_regex = '/^alexa|^blitz\.io|bot|^browsermob|crawl|^curl|^facebookexternalhit|feed|google web preview|^ia_archiver|^java|jakarta|^load impact|^magespeedtest|monitor|nagios|^pinterest|postrank|slurp|spider|uptime|yandex/i';  
    $userAgent = empty($_SERVER['HTTP_USER_AGENT']) ? FALSE : $_SERVER['HTTP_USER_AGENT'];
    $isBot = ! $userAgent || preg_match($bot_regex, $userAgent);
    return $isBot; 
}

then edit the start function to insert an if..return line at the beginnig like so:

public function start($sessionName=null)
    {
        if($this->isBot()){return false;} // Don't start session if visitor is a bot.
        ...
        ...
  • Thanks for that ... but is this not going to prevent my site being crawled. Also by the way I notice you're new to stack exchange ... I think you're supposed to edit your existing answer with the new info instead of generating a new answer. – byronyasgur Apr 12 '14 at 13:28
  • Just following up on this - I ended up limiting the crawl rate to 30 seconds and I'm going to set a cron to delete old sessions ... should sort it out - to be honest I don't fully understand why I'd want to do it the way you describe and I'm working on another project now anyway ... but I'll leave it here with the tick and it could be helpful to someone else - thanks – byronyasgur Apr 18 '14 at 17:21
  • 1
    I would suggest making the isBot() function protected, since you are only accessing it from within the class. – Buttle Butkus Aug 25 '15 at 0:23
  • For some reason, implementing this change caused Googlebot to not be able to access our site at all, so I'm going to have to downvote. – Buttle Butkus Sep 25 '15 at 20:10
  • this just saved our lives... we had bingbot basically dos'ing us by opening a gazillion sessions – greenone83 Sep 29 '15 at 12:13
3

Check your server logs for crawls by Googlebot or Bing or similar. It's likely that Magento is starting new sessions on each request due to the fact that these calls don't accept a session cookie. I've had to overload the session management to prevent it starting session based on the user agent of crawlers.

  • thanks but site is in maintenance mode so locked for crawling. – byronyasgur Apr 11 '14 at 17:43
  • Could you elaborate on your experience ... like I said my site is in maintenance mode ( so offline and uncrawlable I thought) but yet I found this in one of the session files bing.com/bingbot.htm so I reckon it's like what you were saying ... however I'm completely lost on this front I dond know what you mean by 'overload the session management' – byronyasgur Apr 11 '14 at 19:39
  • i'll try - explanation is quite long so see my next answer – Dave Farthing Apr 11 '14 at 21:54
3

if your sessions directory is /var/www/magento/var/session/ , put this in your crontab:

*/5 * * * * find /var/www/magento/var/session/ -exec grep -iq 'googlebot' '{}' \; -delete

This will delete all sessions from googlebot every five minutes.

2

I'm not sure if the Bots/Crawler needs a sessions, maybe some crawler may need it anyway the below solution is similar to the one implemented by Prestashop folks:

  • assigning to the the bots the old sessions
  • using the BOI IP address to match the old session

Magento doesn't store the IP along with the session ID, I guess this is because Magento support more backend for session (files, DB, Redis)
So I have implemented a solution that mix prestashop approach with Magento session management.

All the Magento session magic happens here: app/code/core/Mage/Core/Model/Session/Abstract/Varien.php
This is an abstract class extended by all the other session models,that means you cannot simply rewrite, the only way is to copy it in /local/Mage/Core/Model/Session/Abstract/Varien.php.

So the below code do 3 things:

  1. Check if the client is a bot
  2. in that case, it calculate the sessionid using the IP.
  3. disable cache for BOT access ( this prevent a bug in the EE FPC with the currencies )

In short
Add the following methods to your copy of Mage_Core_Model_Session_Abstract_Varien

public function isBot()
    {
        $bot_regex = '/BotLink|bingbot|AhrefsBot|ahoy|AlkalineBOT|anthill|appie|arale|araneo|AraybOt|ariadne|arks|ATN_Worldwide|Atomz|bbot|Bjaaland|Ukonline|borg\-bot\/0\.9|boxseabot|bspider|calif|christcrawler|CMC\/0\.01|combine|confuzzledbot|CoolBot|cosmos|Internet Cruiser Robot|cusco|cyberspyder|cydralspider|desertrealm, desert realm|digger|DIIbot|grabber|downloadexpress|DragonBot|dwcp|ecollector|ebiness|elfinbot|esculapio|esther|fastcrawler|FDSE|FELIX IDE|ESI|fido|H�m�h�kki|KIT\-Fireball|fouineur|Freecrawl|gammaSpider|gazz|gcreep|golem|googlebot|griffon|Gromit|gulliver|gulper|hambot|havIndex|hotwired|htdig|iajabot|INGRID\/0\.1|Informant|InfoSpiders|inspectorwww|irobot|Iron33|JBot|jcrawler|Teoma|Jeeves|jobo|image\.kapsi\.net|KDD\-Explorer|ko_yappo_robot|label\-grabber|larbin|legs|Linkidator|linkwalker|Lockon|logo_gif_crawler|marvin|mattie|mediafox|MerzScope|NEC\-MeshExplorer|MindCrawler|udmsearch|moget|Motor|msnbot|muncher|muninn|MuscatFerret|MwdSearch|sharp\-info\-agent|WebMechanic|NetScoop|newscan\-online|ObjectsSearch|Occam|Orbsearch\/1\.0|packrat|pageboy|ParaSite|patric|pegasus|perlcrawler|phpdig|piltdownman|Pimptrain|pjspider|PlumtreeWebAccessor|PortalBSpider|psbot|Getterrobo\-Plus|Raven|RHCS|RixBot|roadrunner|Robbie|robi|RoboCrawl|robofox|Scooter|Search\-AU|searchprocess|Senrigan|Shagseeker|sift|SimBot|Site Valet|skymob|SLCrawler\/2\.0|slurp|ESI|snooper|solbot|speedy|spider_monkey|SpiderBot\/1\.0|spiderline|nil|suke|http:\/\/www\.sygol\.com|tach_bw|TechBOT|templeton|titin|topiclink|UdmSearch|urlck|Valkyrie libwww\-perl|verticrawl|Victoria|void\-bot|Voyager|VWbot_K|crawlpaper|wapspider|WebBandit\/1\.0|webcatcher|T\-H\-U\-N\-D\-E\-R\-S\-T\-O\-N\-E|WebMoose|webquest|webreaper|webs|webspider|WebWalker|wget|winona|whowhere|wlm|WOLP|WWWC|none|XGET|Nederland\.zoek|AISearchBot|woriobot|NetSeer|Nutch|YandexBot|YandexMobileBot|SemrushBot|FatBot|MJ12bot|DotBot|AddThis|baiduspider|SeznamBot|m2e/i';
        $userAgent = empty($_SERVER['HTTP_USER_AGENT']) ? FALSE : $_SERVER['HTTP_USER_AGENT'];
        $isBot = !$userAgent || preg_match($bot_regex, $userAgent);

        return $isBot;
    }

    public function getBotSessionId()
    {
        if (isset($_SERVER['REMOTE_ADDR'])) {
            $botIp = $_SERVER['REMOTE_ADDR'];
            $isHttps = (int)Mage::app()->getFrontController()->getRequest()->isSecure();
            $storeId = Mage::app()->getStore()->getId();
            return 'BOT' . $isHttps . $storeId . sha1($botIp);
        }
        return false;
    }

Then you need to change this method as well

 public function setSessionId($id = null)
    {
        if (!is_null($id) && preg_match('#^[0-9a-zA-Z,-]+$#', $id)) {
            session_id($id);
        } elseif ($this->isBot() && $id_bot = $this->getBotSessionId()) {
            $this->setSessionId($id_bot);
        }
        return $this;
    }

public function start($sessionName = null)
    {
        if (isset($_SESSION) && !$this->getSkipEmptySessionCheck()) {
            return $this;
        }

        // getSessionSaveMethod has to return correct version of handler in any case
        $moduleName = $this->getSessionSaveMethod();
        switch ($moduleName) {
            /**
             * backward compatibility with db argument (option is @deprecated after 1.12.0.2)
             */
            case 'db':
                $moduleName = 'user';
                /* @var $sessionResource Mage_Core_Model_Resource_Session */
                $sessionResource = Mage::getResourceSingleton('core/session');
                $sessionResource->setSaveHandler();
                break;
            case 'user':
                // getSessionSavePath represents static function for custom session handler setup
                call_user_func($this->getSessionSavePath());
                break;
            case 'files':
                //don't change path if it's not writable
                if (!is_writable($this->getSessionSavePath())) {
                    break;
                }
            default:
                session_save_path($this->getSessionSavePath());
                break;
        }
        session_module_name($moduleName);

        $cookie = $this->getCookie();
        if (Mage::app()->getStore()->isAdmin()) {
            $sessionMaxLifetime = Mage_Core_Model_Resource_Session::SEESION_MAX_COOKIE_LIFETIME;
            $adminSessionLifetime = (int)Mage::getStoreConfig('admin/security/session_cookie_lifetime');
            if ($adminSessionLifetime > $sessionMaxLifetime) {
                $adminSessionLifetime = $sessionMaxLifetime;
            }
            if ($adminSessionLifetime > 60) {
                $cookie->setLifetime($adminSessionLifetime);
            }
        }

        // session cookie params
        $cookieParams = array(
            'lifetime' => $cookie->getLifetime(),
            'path' => $cookie->getPath(),
            'domain' => $cookie->getConfigDomain(),
            'secure' => $cookie->isSecure(),
            'httponly' => $cookie->getHttponly()
        );

        if (!$cookieParams['httponly']) {
            unset($cookieParams['httponly']);
            if (!$cookieParams['secure']) {
                unset($cookieParams['secure']);
                if (!$cookieParams['domain']) {
                    unset($cookieParams['domain']);
                }
            }
        }

        if (isset($cookieParams['domain'])) {
            $cookieParams['domain'] = $cookie->getDomain();
        }

        call_user_func_array('session_set_cookie_params', $cookieParams);

        if (!empty($sessionName)) {
            $this->setSessionName($sessionName);
        }

        // potential custom logic for session id (ex. switching between hosts)
        $this->setSessionId();

        Varien_Profiler::start(__METHOD__ . '/start');
        $sessionCacheLimiter = Mage::getConfig()->getNode('global/session_cache_limiter');
        if ($sessionCacheLimiter) {
            session_cache_limiter((string)$sessionCacheLimiter);
        }

        session_start();

        if ($this->isBot()) {
//            session_cache_limiter('nocache');
            /**
             * disable cache for BOT ( issue with wrong currency cached because cookie are not stored by BOT )
             * see https://magento.stackexchange.com/questions/147482/fpc-issue-wrong-currency
             */
            Mage::app()->getCookie()->renew('NO_CACHE', 1);
            /**
             * `no_cache` param should assure the cache is not saved for this request
             * @see Mage_Enterprise_PageCache_Model_Processor::canProcessRequest()
             */
            Mage::app()->getFrontController()->getRequest()->setParam('no_cache', 1);
        }

        if (Mage::app()->getFrontController()->getRequest()->isSecure() && empty($cookieParams['secure'])) {
            // secure cookie check to prevent MITM attack
            $secureCookieName = $sessionName . '_cid';
            if (isset($_SESSION[self::SECURE_COOKIE_CHECK_KEY])
                && $_SESSION[self::SECURE_COOKIE_CHECK_KEY] !== md5($cookie->get($secureCookieName))
            ) {
                if (!$this->isBot()) {
                    session_regenerate_id(false);
                }
                $sessionHosts = $this->getSessionHosts();
                $currentCookieDomain = $cookie->getDomain();
                foreach (array_keys($sessionHosts) as $host) {
                    // Delete cookies with the same name for parent domains
                    if (strpos($currentCookieDomain, $host) > 0) {
                        $cookie->delete($this->getSessionName(), null, $host);
                    }
                }
                $_SESSION = array();
            }
            if (!isset($_SESSION[self::SECURE_COOKIE_CHECK_KEY])) {
                $checkId = Mage::helper('core')->getRandomString(16);
                $cookie->set($secureCookieName, $checkId, null, null, null, true);
                $_SESSION[self::SECURE_COOKIE_CHECK_KEY] = md5($checkId);
            }
        }

Note
The bot session Id will be a little longer than the default one because
sha1 function:

returned value is a 40-character hexadecimal number

... anyway this shouldn't be a problem since PHP allows session to be 128 character long

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.