products disappear from category page or search result, index error cluster_block_exception

Question

Products sometimes disappear from the category pages in the frontend. It might be after reindex, or clear cache. If i go to magento admin panel->catalog-> and highlight the category, make some changes, then"save". It sometimes bring back the products into the category.

what might be the cause to the issue? i'm thinking some permission not set correctly?

UPDATED: seems like making some changes in the category (for example, edit product positions), then clear clean sometimes bring back the product, but it is gone again after awhile (when reindex runs?)

magento version: 2.2.6

products are migrated using data-migration-tool from magento 1.9.2

sometimes when i reindex, this error occurs, but 90% it return success message.

Catalog Search indexer process unknown error:
{"error":{"root_cause":[{"type":"cluster_block_exception","reason":"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"}],"type":"cluster_block_exception","reason":"blocked by: [FORBIDDEN/12/index read-only / allow delete (api)];"},"status":403}

Answer 1

some possible solutions:

1. ElasticSearch default configuration will not allocate any more disk space when more than 90% of the disk are used overall, it might locked it to read-only mode.

To unlock writes to your cluster (all indexes) using below command (change to your elastic ip address and ports)

curl -XPUT -H "Content-Type: application/json" http://localhost:9200/_all/_settings -d '{"index.blocks.read_only_allow_delete": null}'

2. if you use other 3rd party search extension such as amasty elastic search etc. use below command to reindex

php bin/magento indexer:reset
php bin/magneto indexer:reindex
php bin/magento indexer:reindex amasty_elastic_relevance_rule_rule amasty_elastic_relevance_rule_product amasty_elastic_popup_data amasty_xsearch_category_fulltext
php bin/magento c:c
php bin/magento c:f

you can find the list of indexes added by the 3rd party extension using below:

    php bin/magento indexer:info

This will give the list of indexes status:

php bin/magento indexer:status