Issue with Elastic search 6.x in Magento 2.3.3

Question

report.CRITICAL: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [category_ids] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"magento233_ela6_product_1_v6","node":"C3kR1cBBR0y_TcZ3EZQgmQ","reason":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [category_ids] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}}],"caused_by":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [category_ids] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.","caused_by":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [category_ids] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}}},"status":400} {"exception":"[object] (Elasticsearch\Common\Exceptions\BadRequest400Exception(code: 400): {\"error\":{\"root_cause\":[{\"type\":\"illegal_argument_exception\",\"reason\":\"Fielddata is disabled on text fields by default. Set fielddata=true on [category_ids] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.\"}],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[{\"shard\":0,\"index\":\"magento233_ela6_product_1_v6\",\"node\":\"C3kR1cBBR0y_TcZ3EZQgmQ\",\"reason\":{\"type\":\"illegal_argument_exception\",\"reason\":\"Fielddata is disabled on text fields by default. Set fielddata=true on [category_ids] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.\"}}],\"caused_by\":{\"type\":\"illegal_argument_exception\",\"reason\":\"Fielddata is disabled on text fields by default. Set fielddata=true on [category_ids] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.\",\"caused_by\":{\"type\":\"illegal_argument_exception\",\"reason\":\"Fielddata is disabled on text fields by default. Set fielddata=true on [category_ids] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.\"}}},\"status\":400}

Facing this issue when we have Magento 2.3.3 and ES 6.x and PHP7.2

Magento given doc https://magento1544124303.zendesk.com/hc/en-us/articles/360031328652-illegal-argument-exception-on-category-page-with-Elasticsearch-5-0-Fielddata-is-disabled-on-text-fields-by-default- not even help here.

Anybody having idea how to resolve this issue?

Answer 1

Got across this several times and we came across this https://support.magento.com/hc/en-us/articles/360027356612-Elasticsearch-5-is-configured-but-search-page-does-not-load-with-Fielddata-is-disabled-error which not only identifies the problem but also gives a fix which works with several Magento versions. I tried 2.3.5 and 2.4.2.

By default, only certain types of product attributes can be used in Layered Navigation. They are Yes/No, Dropdown, Multipleselect, and Price. That is why in Magento Admin, you cannot set an attribute of any other type as Use in Layered Navigation = Filterable or Use in Search Results Layered Navigation = Yes. But there is a technical possibility to get around this limitation by directly changing the is_filterable and is_filterable_in_search values in the database. If this happens, and any other attribute type, like Date, Text, etc., is set to be used in Layered Navigation, Elasticsearch 5 (my edit: and others like ElasticSearch 6 and 7) throws an exception.

Fielddata is disabled on text fields by default. Set fielddata=true on [%attribute_code%]] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory.

Step 1. Identify the product attributes which generate the issue using an SQL tool like phpmyadmin

SELECT ea.attribute_code, ea.frontend_input, cea.is_filterable, cea.is_filterable_in_search FROM eav_attribute AS ea
    INNER JOIN catalog_eav_attribute AS cea ON ea.attribute_id = cea.`attribute_id`
    WHERE (is_filterable = 1 OR is_filterable_in_search = 1) AND frontend_input NOT IN ('boolean', 'multiselect', 'select', 'price');

To fix the issue, you need to set is_filterable (that is, used in Layered Navigation) and filterable_in_search (that is, used in search results Layered Navigation) to "0" (not used).

UPDATE catalog_eav_attribute AS cea
    INNER JOIN eav_attribute AS ea
        ON ea.attribute_id = cea.attribute_id
SET cea.is_filterable = 0, cea.is_filterable_in_search = 0
WHERE (cea.is_filterable = 1 OR cea.is_filterable_in_search = 1) 
    AND frontend_input NOT IN ('boolean', 'multiselect', 'select', 'price');

then reindex all

bin/magento indexer:reindex

The fix says to reindex only catalogsearch_fulltext but this won't work if you have custom search modules since they use different indexes.