I haven't done something similar but as I checked the code right now, you can create an after Plugin for this method ```\Magento\Shipping\Model\CarrierFactory::createIfActive```, there you can add your validation and return true or false for the shipping method you allow or not.

Although I haven't tried it yet, maybe it will guide you to the right direction.