Aside from jQuery being lightly adopted during the development of Magento we must consider another aspect:
Magento was developed around 2007 at a time when TC39 was proposing EcmaScript 5 and it was not fully implemented or developed. Prototype.JS aimed to solve this issue by bringing ECMA5 features to legacy browsers which lacked support at the time. Prototype attempted to fill this need first and foremost and so it was adopted by the developers of Magento to bring ECMA5 features to the frontend framework being developed.
Another consideration is that one of the pillars of the Magento platform was extreme flexibility and extendability. Prototype provides the ability to have OOP-like features where you can create new objects that extend or inherit from other given objects and call the super class directly from child classes. It's quite powerful in that regard. Even today in jQuery this is possible to some extent with widgets but now we use native ECMA5 OOP features found in modern browsers.
Let's also consider AJAX. Google Maps and Gmail are largely considered to be the landmark uses of AJAX from that era, with Maps being launched just 2 years prior to Magento in 2005. AJAX was not terribly uncommon or painful to work with at the time but using a library gave you a standard API in 2007 to achieving cross-browser compatibility without extra complexity. Magento was very ambitious in its use of AJAX (for a free product!) for the checkout, admin panel, and therefore it adopted the best library for managing AJAX at the time. That library was Prototype.