Magento 2.3 Customer Account Create 404 errors with FPC enabled

Question

Customer account link broken after crawl. Run site check in woorank.com. Customer account create link will throw 404 error. Works fine when FPC is disabled. This issue has been pending for a long time. Does this mean this is not critical to Magento? Is there a workaround or patch for this issue?

https://github.com/magento/magento2/issues/20255

Answer 1

Short Answer:

Try updating the .htaccess file to add the HTTP HEAD block below the already existing TRACE and TRACK block. Then flush the cache and 404 should not return.

############################################
## TRACE and TRACK HTTP methods disabled to prevent XSS attacks

RewriteCond %{REQUEST_METHOD} ^TRAC[EK]
RewriteRule .* - [L,R=405]

############################################
## HEAD HTTP methods disabled due to https://github.com/magento/magento2/issues/20255

RewriteCond %{REQUEST_METHOD} HEAD
RewriteRule .* - [L,R=405]

Long Answer:

Testing this issue I found the following in my access log:

"HEAD /customer/account/create/ HTTP/1.1" 404 3708 "-" "Mozilla/5.0 (compatible; woorankreview/2.0; +https://www.woorank.com/)"

This got me thinking the HTTP HEAD request was causing the 404 to get stuck in the FPC for the requested URI. To prove this open a page on a Magento 2.3.0 or 2.3.1 site, open browser developer tools, and make a HEAD request by pasting the following into the Javascript Console:

  var xmlhttp = new XMLHttpRequest();
  xmlhttp.open("HEAD", "/customer/account/create/", true); 
  xmlhttp.onreadystatechange=function() {
      if(xmlhttp.readyState==4) { 
         console.log(xmlhttp.getAllResponseHeaders());
      }
  }
  xmlhttp.send(null); 

Which returns: HEAD https://{site.testing.domain}/customer/account/create/ 404 (Not Found)

Now the customer account creation page return 404 and the FPC cache needs to be flushed to clear.

Based on the above finds I took the quick route of blocking all HEAD request using the existing .htaccess. Obviously, NGINX would require config addition. I've read about the potential traffic drawbacks of rejecting all HEAD request so I imagine a more graceful/targeted workaround could be developed.

Credit for detailing how to make HEAD requests in a browser: https://www.laurencegellert.com/2013/03/using-your-browser-as-a-dev-tool-to-test-head-requests/

Edit: added NGINX configure details

I added the following NGINX configuration to the SSL termination server{} block (pre-Varnish) and confirmed it returns 405 for HEAD requests. I imagine it would work added in your version of nginx.conf.sample (post-Varnish) if needed.

## HEAD HTTP methods disabled due to https://github.com/magento/magento2/issues/20255
if ( $request_method = HEAD ) {
   return 405;
}

Edit: added index.php details for NGINX not behind Varnish and unable to update the server configuration

If all else fails you can try blocking 405 in index.php prior to Magento bootstrap. Find the Bootstrap::create line in index.php and add the following above it:

// HEAD HTTP methods disabled due to https://github.com/magento/magento2/issues/20255
if (false !== stripos($_SERVER['REQUEST_METHOD'], 'HEAD')) {
    header("HTTP/1.1 405 Method Not Allowed");
    http_response_code(405);
    exit();
}
$bootstrap = \Magento\Framework\App\Bootstrap::create(BP, $_SERVER);

I dream of hooking up the debugger and tracing the cause of this bug but at the moment all my schedule can offer is this rather unsatisfying workaround.