To anyone finding this in the future. My issue was around having a ShopperApproved integration.
I was able to narrow this down by disabling all extensions and then enabling them one at a time until the issue returned. After this I was able to say for certain that it was the ShopperApproved integration.
I then went into that extension and commented out all JS code that was added to the page. I then uncommented it file by file to see which was causing the issue.
In my case there were three different places that were an issue, but they all involved using script to add an external JS file to the HTML head. That JS file loaded jQuery if it wasn't found.
My solution was to wrap that code into the windows onload event so it would fire after all other JS. There is probably a better solution, but this worked. Since the issue was specific to my code base I will refrain from adding specific code here.