It's a good idea to look at the varnishlog output. You pasted client-side varnishlog output, but the reason why fetches fail is reported in the backend-side varnishlog output.
The part where the backend request happens will have a FetchError tag, that will tell you what was going on.
Looking for fetch errors
Here's an example where I reached the backend's ...