5

We run a Magento 2.1.8 Shop with Amasty Geo IP Redirect Module.

Unfortunatly it is not working correctly with varnish. Amasty wrote they know no working solution right now. I looked up for other modules - all not working with Varnish right now.

Has anyone any Idea how to achieve this?

Our current state is, that the session parameters for the redirect module are not written and the user gets redirected everytime. So switching the store is not possible.

This is a serious problem for us. Disabling Varnish is no solution since the performance increase is neccessary.

2

Okay seems like I got it working after struggling. In our case we always redirect the first visitor to the correct store. Redirect directly from Varnish. No bypassing.

It is super messy but if anyone got a better solution feel free to post it here.

Install Varnish GeoIP Module
https://rageagainstshell.com/2016/05/geoip-location-in-varnish/
Make sure to clone the correct branch (regarding your Varnish version) otherwise it won't work. In my case it was:
git clone https://github.com/varnish/libvmod-geoip --branch=4.1

Add this at the beginning of your varnish file. (Most likely under /etc/varnish/default.vcl) add
import geoip;

Inside the sub vcl_recv section add something like this:

if (req.http.Cookie !~ "PHPSESSID=") {
    set req.http.X-Country-Code = geoip.country_code(req.http.X-Forwarded-For);
    if(req.url !~ "/(pub/)?(media|static)/") {
        if (req.url !~ "/de/" && req.http.X-Country-Code == "DE") {
            set req.http.subshop = "/de/";
            return (synth(750, "Redirect"));
        } elseif (req.url !~ "/us/" && req.http.X-Country-Code ~ "(US|MX|CA)") {
            set req.http.subshop ="/us/";
            return (synth(750, "Redirect"));
         }
    }
}

If there is a session already -> Nevermind
Otherwise check if the request is not a static file
Then check if user is already in the correct store. Otherwise set variable for later use (see below)
Return synth error.

At the bottom of your default.vcl file add this to do the redirect:

sub vcl_synth {
    if (resp.status == 750) {
    if(req.url != resp.http.Location) {
    set resp.status = 301;
    set resp.http.Location = req.http.subshop + regsuball(req.url, "^(/de/|/us/|)","")
    }
    return (deliver);
    }
}

First prevent redirect loops. Set redirect response status Strip store_view_codes from url and replace it with the correct one.

1

I have implemented what I think is the best over-engineered solution, at least for our use case. We are not using Amasty's Geo-IP Redirect. Instead we chose to develop our own custom solution which probably ended up being quite similar. Plugin to frontend controller dispatch method I'm guessing?

Our use case

We run multiple websites, each with a single group/store and all under the same domain. Our websites relate to our service regions so have very dynamic countries set in the Magento config.

Our requirements

  • First time visitors should never bypass varnish cache, at least not for a page render.
  • Varnish should serve the correct version from cache based on geo-IP.
  • Magento config should still determine which website is served - so geoIP vmods and hard-coded regexes are out.

Our Solution

Let's fetch the X-Magento-Vary header from the backend using a light-weight endpoint, then use it to serve the correct page from cache. This will deliver visitors to the correct region first-go, whist minimally impacting performance. After a lot of research I decided to solve this using the cUrl VMOD. The HTTP VMOD was not an option for us because we are using Varnish v4.

Here's the basic rap:

  1. Add a light-weight endpoint to a custom Magento module for returning the X-Magento-Vary header. I added a simple controller that returns RESULT_RAW with no content, just headers. This endpoint should execute the logic in your redirect module to set the correct store, most importantly by updating the HTTP context values to ensure the correct vary string is returned in the cookies.

  2. Install the curl VMOD: https://github.com/varnish/libvmod-curl

  3. Merge the following into your .vcl, replacing the curl path with your light-weight endpoint

`

... Import the cUrl VMOD

import curl;

...

sub vcl_recv {

... # After all your return(pass) logic, get vary header from the backend if not yet checked

    # For first time visitors, determine X-Magento-Vary hash via backend API call that uses geo-ip
    if (req.http.cookie !~ "X-Varnish-GeoIP=") {

        # Copy required headers. Host and Proto prevent redirect, 
        # Forwarded-For sets correct IP for geo-IP lookup behind proxies
        curl.header_add("X-Forwarded-For: " + req.http.X-Forwarded-For);
        curl.header_add("X-Forwarded-Proto: https");
        curl.header_add("Host: " + req.http.host);
        curl.get("http://127.0.0.1:8080/geoip/redirect/varnishService");

        if (curl.header("Set-Cookie") ~ "X-Magento-Vary="){

            if (req.http.cookie){
                set req.http.cookie = req.http.cookie + 
                    "; " + regsub(curl.header("Set-Cookie"), "^.*?(X-Magento-Vary=[^;]+);*.*$", "\1");
            } else {
                set req.http.cookie = regsub(curl.header("Set-Cookie"), "^.*?(X-Magento-Vary=[^;]+);*.*$", "\1");
            }

            # Set store cookies so it's sent back to the backend for a cache MISS. This prevents subsequent geo-IP lookup
            if (curl.header("Set-Cookie") ~ "store="){
                set req.http.cookie = req.http.cookie +  
                    "; " + regsub(curl.header("Set-Cookie"), "^.*?(store=[^;]+);*.*$", "\1");
            }
        }
    }

...     

}

...


sub vcl_deliver {

... An attempt has been made to get the Vary header, this cookie stops subsequent checks.

    if (curl.status() > 0){
        # It would be better to not rely on user-sent cookie but AFAIK there's not a way to store a persistent variable in varnish.
        if (resp.http.Set-Cookie){
            set resp.http.Set-Cookie = resp.http.Set-Cookie + "; X-Varnish-GeoIP=true; path=/; domain=" + req.http.Host + ";";
        } else {
            set resp.http.Set-Cookie = "X-Varnish-GeoIP=true; path=/; domain=" + req.http.Host + ";";
        }   
    }

...

}

`

The X-Varnish-GeoIP cookie ensures that the backend is only called once in most cases. This was a better solution than relying on store or session cookies.

Troubleshooting

I encountered various issues whilst implementing my solution. I was testing across both inbuilt and varnish configured instances of Magento since I wanted to be able to turn off varnish if we ran into any issues. Here some gotchas I ran into.

  • We were initially using a controller_action_predispatch event observer for our GeoIP redirect. This was causing a cache miss for in-built cache on the first time visits to stores other than the default. The reason that this happened is because page cache uses an around plugin which always executes before the event dispatch. We moved our redirect logic to a beforeMethod plugin with a sortOrder of -1 to ensure it always executes before the page cache plugins.

  • When you do the switch to a new store view, make sure you also update the \Magento\Framwork\App\Http\Context object with new values. Not doing so can cause the returned page to be cached using the wrong X-Magento-Vary string.

Below is the code we use to set a new store view in our beforeDispatch plugin (closely mimics \Magento\Store\Model\StoreSwitcher). Note that the default store doesn't have store cookie or vary header set, which is the Magento default.

`

$this->storeManager->setCurrentStore($store->getCode());

$defaultStoreView = $this->storeManager->getDefaultStoreView();
if ($defaultStoreView !== null) {
    if ($defaultStoreView->getId() === $store->getId()) {
        $this->storeCookieManager->deleteStoreCookie($store);
        $this->httpContext->unsValue(Store::ENTITY);
        $this->httpContext->unsValue(Context::CONTEXT_CURRENCY);
    } else {
        $this->httpContext->setValue(Store::ENTITY, $store->getCode(), $defaultStoreView->getCode());
        $this->httpContext->setValue(Context::CONTEXT_CURRENCY, $store->getDefaultCurrencyCode(), $defaultStoreView->getDefaultCurrencyCode());
        $this->storeCookieManager->setStoreCookie($store);
    }
}

`

Another Solution?

This issue would more or less vanish if we were using separate domains for each store view. Magento generated vcl adds the domain to the hash, so with separate domains the backend lookup in this solution is essentially replaced by whatever domain is visited. However in that case, users which entered via the wrong domain wouldn't be redirected to the right website. That is one other advantage of our solution, albeit the overhead of a backend API call.

0

Amasty has a partial solution here By by-passing Varnish if this is the users first time visiting the site.

5
  • 1
    We have this extension and tried the partial solution but we cannot by pass it. The website speed is highly decreased without varnish. In the mean time I am trying to implement the Varnish GeoIP Module directly. – Johnny Longneck Mar 1 '18 at 12:35
  • 1
    I'm in the process of scoping my own, if I have any success I will let you know. – Chris Rogers Mar 1 '18 at 23:42
  • @ChrisRogers did you get anywhere with this? I'm wondering if it's possible to make a lightweight HTTP request to Magento backend in order to set the store cookie in the varnish request. This would only occur if there was no store cookie in the original request. This would limit the overhead for first time visitors since the HTTP request wouldn't render any page, and the requested page would still be served from varnish if cached. I think HTTP VMOD might be used for this but not certain. – micwallace May 8 '20 at 4:38
  • Hey @micwallace this approach might work depending on the implementation, potentially several solutions each with their own caveats: 1. Keep the solution from Amasty (downside being an un-cached storefront on first load). 2. Create a custom JS component to call the Geo IP Redirect controller and perform a JS redirect based on response (Downside is user sees the page before it then redirects & that it won't be 301) 3. Create a custom controller to perform the redirect request then configure Varnish to pass this specific request (still uncached). Cookies could be implemented for future requests. – Chris Rogers Jun 29 '20 at 5:36
  • @ChrisRogers I did end up implementing my idea and it's been working well in our production environment for a little over a month. I added an answer in this post with a detailed write-up of the solution. – micwallace Jul 21 '20 at 10:04
0

After going crazy to find a solution for it, finally I got a working solution for Magento 2.3.2 + Varnish + GeoIP. To accomplish it you can use the Cache Context and Page Variations which are very simple to implement.

First create a Plugin into your module for Magento\Framework\App\Http\Context.

File: app/code/Vendor/YourModule/etc/di.xml

<?xml version="1.0"?>
<config xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
    xsi:noNamespaceSchemaLocation="urn:magento:framework:ObjectManager/etc/config.xsd">
    <type name="Magento\Framework\App\Http\Context">
        <plugin name="atwix_country_cache_context_plugin"
            type="Vendor\YourModule\Plugin\CustomerCountryCacheContextPlugin" />    
    </type>
</config>

Then the Plugin implementation

File: Vendor/YourModule/Plugin/CustomerCountryCacheContextPlugin.php

/**
 * Plugin on \Magento\Framework\App\Http\Context
 */
class CustomerCountryCacheContextPlugin
{
    public function __construct(
        \Magento\Customer\Model\Session $customerSession
    ) {
        $this->customerSession = $customerSession;
    }
    /**
     * \Magento\Framework\App\Http\Context::getVaryString is used by Magento to retrieve unique identifier for selected context,
     * so this is a best place to declare custom context variables
     */
    function beforeGetVaryString(\Magento\Framework\App\Http\Context $subject)
    {
        $country = $this->yourWayToGetTheCountry();
        $defaultCountryContext = 'Brazil';
        $subject->setValue('CONTEXT_AGE', $country, $defaultCountryContext);
    }
}

And the last step is to add the X-Magento-Vary cookie to transfer context on the HTTP layer in your Varnish.

sub vcl_hash {
    if (req.http.cookie ~ "X-Magento-Vary=") {
        hash_data(regsub(req.http.cookie, "^.*?X-Magento-Vary=([^;]+);*.*$", "\1"));
    }
    ... more ...
}

For more info visit:

https://devdocs.magento.com/guides/v2.3/extension-dev-guide/cache/page-caching/public-content.html#configure-page-variations

https://www.atwix.com/magento-2/cache-context-and-page-variations-in-magento-2/

0

i found an easy way to solve my problem just updating how varnish hash my request: in my case i'm using cloudflare and he send an additional request to my server and i used this request to generate a unique hash based on it:

into VCL file

sub vcl_hash {
if (req.http.cookie ~ "X-Magento-Vary=") {
    hash_data(regsub(req.http.cookie+req.http.cf-ipcountry, "^.*?X-Magento-Vary=([^;]+);*.*$", "\1"));
}

# For multi site configurations to not cache each other's content
if (req.http.host) {
    hash_data(req.http.host+req.http.cf-ipcountry);
} else {
    hash_data(server.ip+req.http.cf-ipcountry);
}

# To make sure http users don't see ssl warning
#if (req.http.X-Forwarded-Proto) {
#    hash_data(req.http.X-Forwarded-Proto);
#}


if (req.url ~ "/graphql") {
    call process_graphql_headers;

enter image description here

now first user from new country will be by passed and the other will be from cache

for more details my first post here

https://magento2develop.blogspot.com/2020/02/geoip-magento-2-varnish-cloudflare.html

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.