Multi-Store Nginx Strange Crossover

Question

Maybe someone can shed some light on this issue. Here's what's happening, when a user visits domain2.com they are sometimes re-directed to domain1.com. Going to domain2.com for a second time (re-attempting) sends them to the correct domain. Only on the first attempt does it fail. Using a re-direct trace on domain2.com always shows it going to domain2.com even though the browser shows domain1.com. Going to domain1.com always gets you to domain1.com. To make things more confusing using a re-direct trace on domain1.com, which always shows up correctly in the browser, shows the final 302 going to domain2.com

We have a multi-store config with Nginx and PHP-FPM.

Nginx is seemingly configured properly. We've added some X headers that grab the mage code in order to verify that it's being picked up and passed along correctly. As I mentioned, the first 301 hop always has the correct X-Header with the correct Mage Code. Domain1.com always has the correct mage code through every hop. Domain1.com has the correct code until the last hop.

Here's a visual breakdown:

Domain hop breakdown:

domain2.com --> 301 --> www.domain2.com --> 302 --> domain1.com/(storecode)

domain1.com --> 301 --> www.domain1.com --> 302 --> domain1.com/(storecode)

Mage Code Breakdown:

Domain2 base --> 301 --> base --> 302 --> base

Domain1 base2 --> 301 --> base2 --> 302 --> base

Any help is appreciated. Thanks!

Here's the mapping:

    map $http_host $mage_code {
hostnames;
        www.site1.com base_us;
        www.site2.com base;
}

map $http_host $mage_type {
hostnames;
        www.site1.com website;
        www.site2.com website;
}

Here's the nginx config:

## Config 

server {
        include /etc/nginx/port.conf;
        server_name site2.com;
        return 301 $scheme://www.site2.com$request_uri;
}

server {
        include /etc/nginx/port.conf;
        server_name site1.com;
        return 301 $scheme://www.site1.com$request_uri;
}

server {
    include /etc/nginx/port.conf;
    listen 443 ssl;
    ssl_certificate     /etc/nginx/ssl/os.com.crt;
    ssl_certificate_key /etc/nginx/ssl/os.com.key;
    server_name ww.site1.com;
    root /var/www/html/os.com;
    access_log  /var/log/nginx/access_www.os.com.log  main if=$writelog;
    error_log   /var/log/nginx/error_www.os.com.log  error;

    ## Bots trap
       include /etc/nginx/conf.d/spider.conf;

    location = /errors/local.xml { deny all; }

    location = /js/index.php/x.js {
       rewrite ^(.*\.php)/ $1 last;
       }

    ## Main Magento @location
    location / {
       try_files $uri $uri/ @rewrite;
       }

    ## Export folder
       include /etc/nginx/conf.d/export.conf;

    ## These locations are protected
    location ~ /(app|var|downloader|includes|pkginfo)/ {
       deny all;
       }

    ## Extra protection and limits
       include /etc/nginx/conf.d/extra_protect.conf;

    ## Images
    location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {
       expires max;
       log_not_found off;
       access_log off;
       add_header ETag "";
       }

    location @rewrite {
       rewrite / /index.php?$args;
       }

    ## Execute PHP scripts
    location ~ \.php$ {
       include /etc/nginx/conf.d/headers.conf;
       try_files $uri =404;
       fastcgi_pass 127.0.0.1:9000;
       #fastcgi_pass 127.0.0.1:$port_switch;
       fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
       ## Store code with multi domain
       fastcgi_param MAGE_RUN_TYPE $mage_type;
       fastcgi_param MAGE_RUN_CODE $mage_code;
       include fastcgi_params;
       }
    }

server {
    include /etc/nginx/port.conf;
    listen 443 ssl;
    ssl_certificate     /etc/nginx/ssl/os.com.crt;
    ssl_certificate_key /etc/nginx/ssl/os.com.key;
    server_name www.site2.com;
    root /var/www/html/os.com;
    access_log  /var/log/nginx/access_www.os.com.log  main if=$writelog;
    error_log   /var/log/nginx/error_www.os.com.log  error;

    ## Bots trap
       include /etc/nginx/conf.d/spider.conf;

    location = /errors/local.xml { deny all; }

    location = /js/index.php/x.js {
       rewrite ^(.*\.php)/ $1 last;
       }

    ## Main Magento @location
    location / {
       try_files $uri $uri/ @rewrite;
       }

    ## Export folder
       include /etc/nginx/conf.d/export.conf;

    ## These locations are protected
    location ~ /(app|var|downloader|includes|pkginfo)/ {
       deny all;
       }

    ## Extra protection and limits
       include /etc/nginx/conf.d/extra_protect.conf;

    ## Images
    location ~* \.(js|css|png|jpg|jpeg|gif|ico)$ {
       expires max;
       log_not_found off;
       access_log off;
       add_header ETag "";
       }

    location @rewrite {
       rewrite / /index.php?$args;
       }

    ## Execute PHP scripts
    location ~ \.php$ {
       include /etc/nginx/conf.d/headers.conf;
       try_files $uri =404;
       fastcgi_pass 127.0.0.1:9000;
       #fastcgi_pass 127.0.0.1:$port_switch;
       fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
       ## Store code with multi domain
       fastcgi_param MAGE_RUN_TYPE $mage_type;
       fastcgi_param MAGE_RUN_CODE $mage_code;
       include fastcgi_params;
       }
    }

When I run a trace on site1 I get the following:

>>> http://site1.com

> --------------------------------------------
> 301 Moved Permanently
> --------------------------------------------
Status: 301 Moved Permanently
Code:   301
Server: nginx
Date:   Mon, 11 Jan 2016 05:15:41 GMT
Content-Type:   text/html
Content-Length: 178
Connection: close
Location:   http://www.site1.com/



>>> http://www.site1.com/

> --------------------------------------------
> 302 Moved Temporarily
> --------------------------------------------
Status: 302 Moved Temporarily
Code:   302
Server: nginx
Date:   Mon, 11 Jan 2016 05:15:42 GMT
Content-Type:   text/html; charset=UTF-8
Connection: close
Set-Cookie: geoip_store_code=us_en; expires=Tue, 10-Jan-2017 05:15:42 GMT; Max-Age=31536000; path=/; domain=www.site1.com; httponly
Expires:    Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control:  no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
X-Frame-Options:    SAMEORIGIN
Location:   http://www.site2.com/
X-Content-Type-Options: nosniff
X-XSS-Protection:   1; mode=block
X-UA-Compatible:    IE=Edge,chrome=1
X-Processing-Time:  1.260
X-Store-Code:   base_us
X-Store-Type:   website



>>> http://www.site2.com/

> --------------------------------------------
> 200 OK
> --------------------------------------------
Status: 200 OK
Code:   200
Server: nginx
Date:   Mon, 11 Jan 2016 05:15:44 GMT
Content-Type:   text/html; charset=UTF-8
Connection: close
Vary:   Accept-Encoding
Set-Cookie: geoip_store_code=us_en; expires=Tue, 10-Jan-2017 05:15:44 GMT; Max-Age=31536000; path=/; domain=www.site2.com; httponly
Expires:    Thu, 19 Nov 1981 08:52:00 GMT
Cache-Control:  no-store, no-cache, must-revalidate, post-check=0, pre-check=0
Pragma: no-cache
X-Frame-Options:    SAMEORIGIN
X-Content-Type-Options: nosniff
X-XSS-Protection:   1; mode=block
X-UA-Compatible:    IE=Edge,chrome=1
X-Processing-Time:  1.322
X-Store-Code:   base
X-Store-Type:   website

In the above I get the correct 301 per nginx to the www version. It then sends out a 302 to the other site. I've created custom headers to show the mage code and type for testing. As you can see this changes as well.

Answer 1

So here's the answer. Although not technical in any way whatsoever, it's now fixed and hopefully someone else who runs across this might want to look beyond the obvious.

So a little back story is that we took over this project from another development team from another country. They installed a module from MageWorx that has a section called "store changer". There's nothing that obviously stands out that the module has this ability. The ability to switch stores is under the menu item "customers".

Anyway, the server was and had been setup correctly. All the redirects happened as they should. The issue ultimately was from a module that was installed. Thanks for the help everyone!