0

Our Magento 2 are connected to third party API, and every time the there is API update the varnish are being purged. The problem starts when there is 2-3 API request every second, that's mean varnish cache always empty.

Here is the deatils:

$ varnishlog -g request -q 'ReqMethod eq "PURGE"' -i 'ReqHeader'

The exact tags being cleared vary each request, but an example is:

ReqHeader X-Magento-Tags-Pattern: ((^|,)catalog_product_9401(,|$))|((^|,)catalog_product_9397(,|$))|((^|,)catalog_category_2(,|$))|((^|,)catalog_category_6(,|$))|((^|,)catalog_category_10(,|$))|((^|,)catalog_category_52(,|$))|((^|,)catalog_category_121(,|$))|((^|,

If we compare this to the cache tags used by frontpage:

BerespHeader X-Magento-Tags: store,cms_block,catalog_category,catalog_category_7,catalog_category_5,catalog_category_4,catalog_category_6,catalog_category_3,catalog_category_8,catalog_category_9,catalog_category_10,cms_block_NavigationSaleBanner,cms_page_2,cms_block_

Some categories that are used on the frontpage (eg catalog_category_10, catalog_category_6) are being continuously flushed by Magento. Reviewing Magento's debug.log it looks like this cache invalidation is being forced by API calls to the site:

[2019-02-04 07:41:44] report.DEBUG: cache_invalidate: {"method":"POST","url":"https://www.website.com.au/rest/V1/perennial/stprice","invalidateInfo":{"servers":["[object] (Zend\Uri\Uri: http://127.0.0.1:8000/)"],"tagsPattern":"((^|,)catalog_product_9521(,|$))|((^|,)catalog_product_8018(,|$))|((^|,)catalog_category_2(,|$))|((^|,)catalog_category_6(,|$))|((^|,)catalog_category_10(,|$))|((^|,)catalog_category_52(,|$))|((^|,)catalog_category_121(,|$))|((^|,)catalog_category_243(,|$))|((^|,)catalog_category_508(,|$))|((^|,)catalog_category_1(,|$))"},"is_exception":false} []

These API calls are coming from the IP address 123.51.107.62

127.0.0.1 123.51.107.62, 162.158.2.110, 127.0.0.1 - [04/Feb/2019:02:43:48 -0500] "POST /rest/V1/perennial/stprice HTTP/1.1" 200 191 "-" "OpenEdge-HttpClient/0.4.0 (WIN32/64) OpenEdge/11.7.2.0.1497 Lib-ABLSockets/0.4.0" 127.0.0.1 123.51.107.62, 162.158.2.158, 127.0.0.1 - [04/Feb/2019:02:43:50 -0500] "POST /rest/V1/perennial/stprice HTTP/1.1" 200 191 "-" "OpenEdge-HttpClient/0.4.0 (WIN32/64) OpenEdge/11.7.2.0.1497 Lib-ABLSockets/0.4.0" 127.0.0.1 123.51.107.62, 162.158.2.158, 127.0.0.1 - [04/Feb/2019:02:43:53 -0500] "POST /rest/V1/perennial/stprice HTTP/1.1" 200 191 "-" "OpenEdge-HttpClient/0.4.0 (WIN32/64) OpenEdge/11.7.2.0.1497 Lib-ABLSockets/0.4.0" 127.0.0.1 123.51.107.62, 162.158.2.170, 127.0.0.1 - [04/Feb/2019:02:43:59 -0500] "POST /rest/V1/perennial/stprice HTTP/1.1" 200 192 "-" "OpenEdge-HttpClient/0.4.0 (WIN32/64) OpenEdge/11.7.2.0.1497 Lib-ABLSockets/0.4.0"

These API calls are likely updating some aspect of the product/categories that causes Magento to force a cache invalidation. Additionally as the requests are coming in slowly (one every few seconds), the cache invalidation is re-triggered over and over -- ensuring Varnish can't cache the frontpage for longer than a few seconds.

Is there any way to modify Varnish VCL file (to work-around the constant purging)??

Thanks

  • have you checked my answer? if it's help please mark my answer correct and upvote my answer. keep coding :) thanks :) – Muhammad Hasham Feb 8 at 5:36
  • Instead of asking here, I suggest checking with the API's developers/users. If they send PURGE request, then it means that some content was updated through their API and needs to be refreshed. If you chose to configure Varnish to ignore these changes, you'd likely be getting some stale data from the API. Solving it on your own end (Varnish side) seems wrong. The API should be fixed not to send PURGE requests and/or your homepage should not include the products which (or their categories) are constantly being updated via the API. – Danila Vershinin Feb 8 at 19:16

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.