UI Modal is a global UI component and it is using on-demand (it means you cannot remove it with XML structs like <remove src="modal.js"/> or something similar).

It's not only using on the frontend, but it is also using widely in adminhtml as well. For e.g:

src/vendor/magento/module-catalog/view/adminhtml/templates/catalog/category/tree.phtml

It is a part of Magento Minicart (Cart Sidebar)

src/vendor/magento/module-checkout/view/frontend/web/template/sidebar.html

However, if you don't want to use the authentication popup functionality (login popup in Checkout page). You can try to remove that block in XML by using this

<referenceBlock name="authentication-popup" remove="true"/>

Completely removing Magento_Ui/js/modal/modal is impossible in Magento 2! It will break a lot of functionalities for sure. Of course, in theory, you still can disable the whole functionality of modal UI with a JS mixin like this:

/**
 * StackOverflow No Modal :(
 *
 * @category  Mage
 *
 * @author    Toan Nguyen <https://github.com/nntoan>
 */
define([
    'jquery',
    'uiRegistry',
    'mage/utils/wrapper',
    'mage/validation'
], function ($, registry, wrapper) {
    'use strict';

    return function (modal) {

        modal.prototype._create = wrapper.wrap(modal.prototype._create, function (original) {
            return null;
        });

        return modal;
    };
});

If I was misunderstanding you, feel free to leave a comment :)

Cheers & Happy coding!

UI Modal is a global UI component and it is using on-demand (it means you cannot remove it with XML structs like <remove src="modal.js"/> or something similar).

It's not only using on the frontend, but it is also using widely in adminhtml as well. For e.g:

src/vendor/magento/module-catalog/view/adminhtml/templates/catalog/category/tree.phtml

It is a part of Magento Minicart (Cart Sidebar)

src/vendor/magento/module-checkout/view/frontend/web/template/sidebar.html

However, if you don't want to use the authentication popup functionality (login popup in Checkout page). You can try to remove that block in XML by using this

<referenceBlock name="authentication-popup" remove="true"/>

Completely removing Magento_Ui/js/modal/modal is impossible in Magento 2! It will break a lot of functionalities for sure. Of course, in theory, you still can disable the whole functionality of modal UI with a JS mixin like this:

/**
 * StackOverflow No Modal :(
 *
 * @category  Mage
 *
 * @author    Toan Nguyen <https://github.com/nntoan>
 */
define([
    'jquery',
    'uiRegistry',
    'mage/utils/wrapper',
    'mage/validation'
], function ($, registry, wrapper) {
    'use strict';

    return function (modal) {

        modal.prototype._create = wrapper.wrap(modal.prototype._create, function (original) {
            return null;
        });

        return modal;
    };
});

If I was misunderstanding you, feel free to leave a comment :)