Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HEAD fails on compressed response #253

Closed
blyxxyz opened this issue May 12, 2022 · 4 comments · Fixed by #257
Closed

HEAD fails on compressed response #253

blyxxyz opened this issue May 12, 2022 · 4 comments · Fixed by #257
Assignees

Comments

@blyxxyz
Copy link
Collaborator

blyxxyz commented May 12, 2022

$ xh head httpbin.org/gzip
[...]
xh: error: error decoding gzip response body: failed to fill whole buffer

This didn't happen before #241, so reqwest deals with it correctly. I don't know how.

@ducaale
Copy link
Owner

ducaale commented May 12, 2022

This comment from seanmonstar/reqwest#83 might be relevant

// libflate does a read_exact([0; 2]), so its impossible to tell
// if the stream was empty, or truly had an UnexpectedEof.
// Therefore, we need to peek a byte to make check for EOF first.

@ducaale ducaale self-assigned this May 14, 2022
@blyxxyz
Copy link
Collaborator Author

blyxxyz commented May 14, 2022

It took me a while to understand, but I think the relevant piece in the current reqwest code is here: https://github.com/seanmonstar/reqwest/blob/2a6e012009fb79065767cb49a8a000d354c47ba6/src/async_impl/decoder.rs#L285

reqwest doesn't wrap a decoder around the stream until it did a poll_peek to make sure that there's actual data to receive. If there's no data then it creates a "fake" empty stream.

So we have the following situations:

  • When xh 0.15 receives an empty gzip response of any kind it turns it into an empty stream, never even constructing a decoder.
  • When xh 0.16 receives an empty gzip response to a GET request, marked with Content-Length: 0, we handle that explicitly by not trying to decode.
  • When xh 0.16 receives an empty gzip HEAD response it doesn't trigger the explicit handling so we try to decode it and fail.

So we need to skip the gzip decoding or ignore the error if the body is totally empty.

Maybe we can fold this into the existing InnerReader trick? Set a flag if it succeeds in reading any data, and then if the decoder errors we can check that and return Ok(0) if it's still false.

@ducaale
Copy link
Owner

ducaale commented May 14, 2022

I will cut a patch release that contains this fix

Edit: Actually, let's wait for #256 to get merged first.

@ducaale ducaale reopened this May 14, 2022
@ducaale
Copy link
Owner

ducaale commented May 23, 2022

Fixed in xh v0.16.1

@ducaale ducaale closed this as completed May 23, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants