Is it okay to modify
lib/web/mage/validation.js directly? If not, how do I override it? There's a bug there I want to fix. Thank you.
No don't edit that file directly as it is a core file, you can overwrite it by adding it to your theme here:
Or alternatively you can use Require JS to change the file called as this answer explains