In Magento 1, you could use the same Observer class with multiple methods and invoke those. As per the new structure, do we need to create a new Observer for every event handler or observer? Or is there a way to combine multiple event handlers in the same observer?
TL;DR: Yes, you need a class for every event you are observing.
If you add all your observer methods in one single class then you end up with a class that knows too much and does too much. This violates the Single Responsibility Principle
Theoretically you could use the same observer for 2 events if the events are similar (they send the same data), but you should not do that.
You should keep your events separated.
This also makes it easier to rewrite and observer. You can just define a preference in
di.xml for the class that observes a certain event.