Why does the default VCL produced by Magento force search requests to bypass Varnish?

Question

The default VCL produced by Magento from the admin has the following few lines in it:

# Bypass shopping cart, checkout and search requests
if (req.url ~ "/checkout" || req.url ~ "/catalogsearch") {
    return (pass);
}

Now I understand why Magento would want requests to bypass Varnish for checkout URLs, but why search requests? Is it purely because search results can change frequently?

Answer 1

Yes, because search results change frequently and Magento doesn't know how to invalidate those results. And because the world isn't a perfect place to live :(

Anyway, you can do just:

# Bypass shopping cart, checkout
if (req.url ~ "/checkout") {
    return (pass);
}  

(note removed catalogsearch from condition)

And do some "microcaching" of popular search results by still having Varnish cache search results, simply with reasonable TTL. In vcl_backend_response:

# "Microcache" for search
if (bereq.url ~ "/catalogsearch") {
    set beresp.ttl = 30m;
} 

See always up-to-date full VCL for Magento 2 which includes this.

Answer 2

Yes, one of the biggest problems with running varnish on a Magento 1.x site was the fact that the cart page would be auto-magically cached. So if I was the first person to add something to my cart, the next person who added something (different) to their cart would see what I added. (Unless you specifically told varnish via the VCL to bypass the cart and checkout pages.)

Same thing can happen on a catalog search page result - the filterable navigation on the left hand side of a normal search results page can easily be cached.

Using a default Luma theme on Magento 2.1, here's a sample URL: http://domain.com/catalogsearch/result/?q=yoga

Here's a screenshot of the results:

On a page like this, you have the left navigation that can open or close, sort by (relevance, price), or the grid view vs. list view, items per page.