This took a minute to figure out. I had to use a tool called beyond compare to fix it.
I did the upgrade using the "magento connect" via the admin. However this caused several issues. I upgraded 18.104.22.168 to 22.214.171.124
To fix this, you need to download the 126.96.36.199 CE from the magento site, then compare your directory structure to the files you just downloaded. You'll notice that many rogue directories have been created as file names (example: filename.php is actually a directory) You need to delete the directories and then copy the correct "file" over. You'll also notice that several hundred other files did not copy over. You'll need to copy those over too.
Long story short, don't use the magento connect upgrade, do it manually from the download.