Does Magento newsletter system process mail bounces?
Or do I need a plugin for this?
It does not process mail bounces natively. Hard/soft bounces (connections receiving an error status) are a function of your Mail Transport agent. Other "bounces" such as vacation replies or unsubscribe replies would have to be analyzed by intercepting mail in an actual inbox.
I would recommend you to look at hosted SMTP services or an ESP such as Mailchimp which have tight integration with Magento's email services to handle this type of requirement.