The image quality looks fine to me.
The thumbnail is being rasterized (resampled) down to 170px by 170px, and the browser is rendering the image at that same size on my screen (width=170px as well as computed width all match).
The preview is actually displaying the full size 3024 x 3024, but is scaled down in the browser to 302px by 302px. As you play with the zoom and focus in on the text you can see that it remains sharp all the way.
You can adjust the theme to display different size thumbnails. It is the theme that defines this although some themes have their settings available in system config so as to not require code changes. By modifying the theme, you could display the thumbnail at 340px by 340px and scale it down 50% so that it looks sharp on retina monitors and devices such as the iphone... but this would still not look much different on regular (not Hi-DPI) devices.
What you might be looking for is a sharpening effect in the downsampler that generates the thumbnail image. That would appear to make the text more crisp because it enhances the contrast between color edges. As far as I know this is not a possible setting in the Magento image class, although it IS an option in the GD library that Magento is using.
(If you are a developer you can look at the code for image resampling in the "lib" folder under the class Varien_Image, as well as the Mage_Catalog_Helper_Image in "core" that generates the preview and thumbnail images from the theme. Sharpening an image requires https://www.php.net/manual/en/function.imageconvolution.php).