First of all, the 'helper class' architectural pattern is more of an anti-pattern and its use is discouraged in Magento 2.
If you have dependencies that may be used in some contexts, but not in others, that sounds like the class is trying to do too much. Try decomposing (i.e. Refactoring) the class into two or more new classes which have more specific purposes.
For logic that you want to re-use across multiple classes, investigate using Traits rather than helpers:
but wouldn't it be more efficient in terms of performance to load the
dependencies on demand using object manager?
I can't argue based on performance. I can tell you that using the Object Manager in your code is discouraged in Magento 2 development and it incurs technical debt. If you do use the Object Manager, then you are bypassing Dependency Injection and all the benefits that go with it. Magento relies on DI in order to make other features of the framework function as well (as you note in your answer).
You may not be using those features in your module now, but if you need to use them for some future change to your module and you're not using DI, you'll have the burden of having to refactor away all those uses of the object manager, i.e. That is when you pay that technical debt you incurred earlier. So you're not really saving time - you're just kicking that technical debt down the road.
If you are worried that your dependencies will have a performance impact, then there are ways to mitigate that:
You can inject Proxy Classes in order to reduce the burden of loading all the dependencies. Proxy Classes are classes which extend from the 'real' class, but don't contain any actual logic. They work more like placeholders. If a method defined in a Proxy Class is called, then the Proxy Class calls the method on the 'real' class, which are lazy-loaded.
The DevDocs has a more complete explanation and examples:
Context classes were introduced to allow for backwards compatible constructor modifications of classes that were supposed to be extended by extension developers. If you need to provide backwards-compatibility, then this is the way to go.
However, as Magento 2 moves from inheritance-based APIs (i.e. Adding new functionality by adding a new class which extends an existing class) all such classes will be deprecated together with the classes they were introduced for.