Just upgraded my DEV server to Magento 2.4.2-p1.
I am very confused about why its using HUGE amounts of RAM when reindexing Catalog Search.
You can see on this screenshot it's filled 115GB of RAM.
The catalog is big (300k +). This issue is not happening on current PRODUCTION site which is running on 2.3.3.
I did
redis-cli flushall
when I was upgrading Magento. So it cant really be an issue with duplicate keys.
My env.php settings for cache look like this (both sites use the same settings):
'cache' => [
'frontend' => [
'default' => [
'backend' => 'Cm_Cache_Backend_Redis',
'backend_options' => [
'server' => '127.0.0.1',
'database' => '0',
'port' => '6379'
],
'id_prefix' => '123_'
],
'page_cache' => [
'backend' => 'Cm_Cache_Backend_Redis',
'backend_options' => [
'server' => '127.0.0.1',
'port' => '6379',
'database' => '1',
'compress_data' => '0'
],
'id_prefix' => '123_'
]
]
],
We're running Varnish as Full Page Cache, but its not connected yet for testing purposes.
I mentioned this cache overload in my other thread: https://stackoverflow.com/questions/68278489/docker-compose-mnt-sdb-is-always-full/68287144#68287144
^^^ here because cache was set to file storage it would fill up the hard-drive within 15 minutes. It would create 100GB just like that.
I was thinking maybe it's using DDL cache is doing this?
https://devdocs.magento.com/guides/v2.4/config-guide/cli/config-cli-subcommands-cache.html
UPDATE:
After about 2 hours, Redis finally blew up and crashed:
It looks like maybe the plugin that is managing the ElasticSearch + Catalog Search indexes is not actually saving the data into the DB? I contacted devs to see what they say.
UPDATE: Looks like I'm not alone in this. There is a GitHub issue with other people reporting the same issue: https://github.com/magento/magento2/issues/32118