NORMALLY:
The <remove src=...>
needs to match the string to include the CSS file. So don't insert a full URL, use the same string that caused the CSS file to be loaded.
For example, I edited the core (gasp!) and snuck the following two lines into a Magento_Customer module layout file for test purposes.
<css src="Magento_Customer::css/foo.css"/>
<remove src="Magento_Customer::css/foo.css"/>
If you just have the first line, the CSS file is included. If you include the second line, it removes it again. But it is doing the string compare using the string before being expanded to a URL (which makes sense as the domain name can change).
HOWEVER IN YOUR SPECIFIC EXAMPLE:
As the other reply correctly said, the URL to this particular CSS file is not injected by a layout file. Loading the Luma sample data puts a few bits of magic into the database directly. In this case it is the "HTML Head" with the magic {{MEDIA_URL}} string that injects it. So the approach I just described won't work as it manipulates the layout tree, not the final HTML.
So to remove, edit the "HTML Head" in the Admin UI to remove the injected link.