Hide out of stock products but show out of stock swatches on a product

Question

Magento version 2.3.4

What I want to achieve:

Don't show products that are fully out of stock, but show out of stock swatches of products.

What I tried:

1. Stores > Configuration > Catalog > Inventory > Stock Options > Display Out of Stock Products: Yes. This option hides out of stock products but also hides out of stock swatches.
2. I found someone that tried this

Magento 2 - Show swatches of simple products (out of stock) in configurable product

So I removed $_product->isSaleable() from

vendor/magento/module-catalog/view/frontend/templates/product/view/form.phtml

This did nothing. It still hides the swatches.

Is there any other option to show the out of stock swatches?