4

I tried to upgrade my Magento 2.3.5 install to Magento 2.4.0, but am unable to get category pages to display products. Everything is in Docker, so I am using Elasticsearch 7.6 and have specified elasticsearch7 for catalog/search/engine. The test from the Magento admin is successful. The category pages were working with 2.3.5 so I am assuming this is something related to the switch to Elasticsearch. I've looked at various upgrade guides without much luck as reindexing isn't fixing it for me.

The search works and returns products, as does loading a product detail page directly, and viewing the products in the Admin, however viewing any category page just shows:

We can't find products matching the selection.

Things I've tried:

  1. Updated to Magento 2.4 using composer (via Docker) and then ran magento setup:upgrade and cleared cache.

  2. Verified that products are enabled, visible, in stock, have quantity, and are assigned to category

  3. Disabled all non-Magento modules, deleted ./generated/code, ./var/view_preprocessed, changed theme to Magento Blank.

  4. Cleared Cache / Reindexed

bash-5.0$ magento indexer:reindex
Design Config Grid index has been rebuilt successfully in 00:00:00
Customer Grid index has been rebuilt successfully in 00:00:00
Category Products index has been rebuilt successfully in 00:00:00
Product Categories index has been rebuilt successfully in 00:00:00
Catalog Rule Product index has been rebuilt successfully in 00:00:01
Product EAV index has been rebuilt successfully in 00:00:00
Stock index has been rebuilt successfully in 00:00:00
Product Price index has been rebuilt successfully in 00:00:00
Catalog Product Rule index has been rebuilt successfully in 00:00:00
Catalog Search index has been rebuilt successfully in 00:00:05

bash-5.0$ magento cache:clean && magento cache:flush
Cleaned cache types:
config
layout
block_html
collections
reflection
db_ddl
compiled_config
eav
customer_notification
config_integration
config_integration_api
full_page
config_webservice
translate
Flushed cache types:
config
layout
block_html
collections
reflection
db_ddl
compiled_config
eav
customer_notification
config_integration
config_integration_api
full_page
config_webservice
translate
  1. Deleted Elasticsearch indices, reindexed, and checked using curl
bash-5.0$ curl elasticsearch:9200/_cat/indices

bash-5.0$ curl elasticsearch:9200/
{
  "name" : "18af19145c92",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "-95jfv_yQNqZWqXk1Z3nBg",
  "version" : {
    "number" : "7.6.2",
    "build_flavor" : "oss",
    "build_type" : "tar",
    "build_hash" : "ef48eb35cf30adf4db14086e8aabd07ef6fb113f",
    "build_date" : "2020-03-26T06:34:37.794943Z",
    "build_snapshot" : false,
    "lucene_version" : "8.4.0",
    "minimum_wire_compatibility_version" : "6.8.0",
    "minimum_index_compatibility_version" : "6.0.0-beta1"
  },
  "tagline" : "You Know, for Search"
}

bash-5.0$ magento indexer:reindex
Design Config Grid index has been rebuilt successfully in 00:00:00
Customer Grid index has been rebuilt successfully in 00:00:00
Category Products index has been rebuilt successfully in 00:00:00
Product Categories index has been rebuilt successfully in 00:00:00
Catalog Rule Product index has been rebuilt successfully in 00:00:00
Product EAV index has been rebuilt successfully in 00:00:00
Stock index has been rebuilt successfully in 00:00:00
Product Price index has been rebuilt successfully in 00:00:00
Catalog Product Rule index has been rebuilt successfully in 00:00:00
Catalog Search index has been rebuilt successfully in 00:00:04

bash-5.0$ curl elasticsearch:9200/_cat/indices
yellow open magento2_product_1_v2 J4HvEAGeTFmxlovr5hl7zw 1 1 694 0 1.6mb 1.6mb

I noticed that there is no "category" index being created but not sure if there should be...

  1. Enabled and disabled Flat Categories

  2. Deleted and re-imported all Products

After deleting and re-importing the products I reindexed, cleared cache, regenerated thumbnails, etc. but still seeing the same behavior.

Edit - I deleted the database and reinstalled from scratch, imported products/categories, cleared cache, etc and the category pages are still not working. Can someone with a working install let me know how many indices are created in Elasticsearch? Can someone provide a same product import CSV that is working correctly for me to test?

  • Check \Magento\CatalogSearch\Model\ResourceModel\Fulltext\Collection::_renderFiltersBefore - maybe Elasticsearch throws an error and Magento replaces search result with empty data. – Roman Kabanov Sep 17 '20 at 19:26
  • @RomanKabanov I set a breakpoint on _renderFiltersBefore() and stepped through it, no exceptions there.. $this->_filters is empty as well. Nothing obvious going all the way through to the block. – doublesharp Sep 17 '20 at 19:39
  • Please first find the Elasticsearch status Elasticsearch is running or not. – Chirag Gondaliya Sep 18 '20 at 4:03
  • @ChiragGondaliya the service is running, I included responses from it in the question. – doublesharp Sep 18 '20 at 17:44
  • @doublesharp, there's only one index magento2_product_1_v50. Do you use multi source inventory? I had all empty categories after playing with sources and assigning default website to none-default stock. Stores > Inventory > Stocks. Additionally, how about switching "Display Out of Stock Products: yes" temporarily? Stores > Configuration > Catalog > Inventory > Stock Options – Roman Kabanov Sep 18 '20 at 18:21
10

I located an exception in ./magento/var/log/exception.log that showed an issue with the configuration of a custom attribute. The attribute was set to a type of text for the attribute custom_attribute in the eav_attribute table, but is_filterable set to 1 in the catalog_eav_attribute table. Setting this value to to 0 and reindexing/clearing cache allowed the category pages to load as expected.

[2020-09-18 19:46:47] main.CRITICAL: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [custom_attribute] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"magento2_product_1_v10","node":"fqGZNT31RSOuoKWU0r5GXQ","reason":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [custom_attribute] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}}],"caused_by":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [custom_attribute] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.","caused_by":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [custom_attribute] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}}},"status":400} {"exception":"[object] (Elasticsearch\\Common\\Exceptions\\BadRequest400Exception(code: 400): {\"error\":{\"root_cause\":[{\"type\":\"illegal_argument_exception\",\"reason\":\"Fielddata is disabled on text fields by default. Set fielddata=true on [custom_attribute] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.\"}],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[{\"shard\":0,\"index\":\"magento2_product_1_v10\",\"node\":\"fqGZNT31RSOuoKWU0r5GXQ\",\"reason\":{\"type\":\"illegal_argument_exception\",\"reason\":\"Fielddata is disabled on text fields by default. Set fielddata=true on [custom_attribute] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.\"}}],\"caused_by\":{\"type\":\"illegal_argument_exception\",\"reason\":\"Fielddata is disabled on text fields by default. Set fielddata=true on [custom_attribute] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.\",\"caused_by\":{\"type\":\"illegal_argument_exception\",\"reason\":\"Fielddata is disabled on text fields by default. Set fielddata=true on [custom_attribute] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.\"}}},\"status\":400} at /magento/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php:632)"} []
[2020-09-19 17:58:37] main.CRITICAL: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [custom_attribute] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"magento2_product_1_v2","node":"6KxafAupTSqGYWZUYyULWg","reason":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [custom_attribute] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}}],"caused_by":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [custom_attribute] in order to load field data by uninverting the inverted index. Note that this can use significant memory.","caused_by":{"type":"illegal_argument_exception","reason":"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [custom_attribute] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}}},"status":400} {"exception":"[object] (Elasticsearch\\Common\\Exceptions\\BadRequest400Exception(code: 400): {\"error\":{\"root_cause\":[{\"type\":\"illegal_argument_exception\",\"reason\":\"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [custom_attribute] in order to load field data by uninverting the inverted index. Note that this can use significant memory.\"}],\"type\":\"search_phase_execution_exception\",\"reason\":\"all shards failed\",\"phase\":\"query\",\"grouped\":true,\"failed_shards\":[{\"shard\":0,\"index\":\"magento2_product_1_v2\",\"node\":\"6KxafAupTSqGYWZUYyULWg\",\"reason\":{\"type\":\"illegal_argument_exception\",\"reason\":\"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [custom_attribute] in order to load field data by uninverting the inverted index. Note that this can use significant memory.\"}}],\"caused_by\":{\"type\":\"illegal_argument_exception\",\"reason\":\"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [custom_attribute] in order to load field data by uninverting the inverted index. Note that this can use significant memory.\",\"caused_by\":{\"type\":\"illegal_argument_exception\",\"reason\":\"Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [custom_attribute] in order to load field data by uninverting the inverted index. Note that this can use significant memory.\"}}},\"status\":400} at /magento/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php:632)"} []

Show misconfigured attributes:

SELECT *
FROM catalog_eav_attribute cea 
JOIN eav_attribute ea
  ON ea.attribute_id = cea.attribute_id 
  AND backend_type IN ('text')
WHERE 
  cea.is_filterable = 1 OR cea.is_filterable_in_search = 1

Set is_filterable to 0 to allow category pages to load.

UPDATE catalog_eav_attribute cea 
JOIN eav_attribute ea
  ON ea.attribute_id = cea.attribute_id 
  AND backend_type in ('text')
SET 
  cea.is_filterable = 0, cea.is_filterable_in_search = 0
WHERE 
  cea.is_filterable = 1 OR cea.is_filterable_in_search = 1

Reindex/Refresh Cache

magento indexer:reindex
magento cache:clean
magento cache:flush
  • 1
    Thanks a lot for this explanation and the solution here. We had the same issue, but in our case the backend_type was 'varchar'. So the issue could also happen in other cases. Important is the attribute code which is written down in the exception message – norgeindian Oct 5 '20 at 8:36
  • thanks @norgeindian - I updated the answer for handle varchar and text. – doublesharp Nov 12 '20 at 15:55
  • 1
    Just to add to this. is_filterable_in_search set to 1 on a text field also causes this issue – elfling Nov 16 '20 at 16:46
  • @elfling thanks, I added that column to the answer. – doublesharp Nov 16 '20 at 18:06
  • @doublesharp Thank, sql query mentioned by you helped me to identify the issue. – Mukesh Jan 8 at 13:20
0
Please run following command and then try:

service elasticsearch start
  • Yes, it is running, if you read the question I tested querying the easticsearch service using curl, and I am able to delete and recreate the indices. The search also works, just not the categories, and I have run a reindex many times. – doublesharp Sep 18 '20 at 17:42
  • It's also running in Docker so it is running in a different container than Magento, and it is running under Alpine Linux so there is now service command. Connecting to the container with docker exec and running ps aux shows the java process though. – doublesharp Sep 18 '20 at 17:43
-1

I got similar issue (no products listing) and solved by following command:

php bin/magento index:reindex
  • If you read the question I tried that many times. The issue was a misconfigured custom product attribute - it was set to a backend_type of text but is_filterable was set to 1. – doublesharp Sep 22 '20 at 16:20
  • My case is different, and above solution rocks. – Kashif Sep 22 '20 at 16:49
  • Since your case is different it's not an appropriate answer for this question... – doublesharp Sep 22 '20 at 17:36
  • Actually, your question is not appropriate – Kashif Sep 22 '20 at 17:41

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.