Since, my theme does not handle form validation such as preventing symbols in text fields. I decided to use html5 input validations such as min/max for numbers and pattern for text as simple workaround.
I edited register.phtml template at /persistent/customer/form and added html5 validation attributes to email, telephone, pincode etc.
However, I was unable to find input fields of First Name and Last name. They were being generated probably by this snippet :
How do I add a pattern such as
pattern="[a-zA-Z0-9\s]+" to First Name and Last Name input fields