Is it possible to show image swatches only on product page, and on category pages, related, upsell sliders to stay default color swatches. By default magento configuration its colors or image swatches everywhere.
If you want to have a red swatch (for example), you have to create an attribute called color, set the input type to swatch and give it an option of red.
You can't give it 2 values where one is a #hex color and another one an image.
You might have to create 2 attributes where one is a color swatch and one is an image swatch, and choose where they show up. One on the product page and the other one everywhere else.
It would be a maintenance nightmare and I wouldn't recommend doing this, but it would work