Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

How do you actually download a tarball? #559

Open
winstonpurnomo opened this issue Jan 31, 2024 · 4 comments
Open

How do you actually download a tarball? #559

winstonpurnomo opened this issue Jan 31, 2024 · 4 comments

Comments

@winstonpurnomo
Copy link

The download_tarball method returns an object, once awaited and unwrapped, is a Response<Body>. I'm not sure how to turn this into an actual working tar.gz file on my computer, since the hyper::body::to_bytes method has been deprecated recently.
@winstonpurnomo winstonpurnomo changed the title How do you actually download a tarball How do you actually download a tarball? Jan 31, 2024
@manchicken
Copy link
Contributor

manchicken commented Feb 4, 2024 •
edited
Loading

https://hyper.rs/guides/1/client/basic/ has some detail on how to stream payloads. They have an example here: https://github.com/hyperium/hyper/blob/master/examples/client.rs#L64-L76

Streaming like this will be a much better approach than trying to hold the entire payload in memory at once.

@XAMPPRocky, it does mean we'll need to adapt to this breaking change in hyper. We're using ::to_bytes() in a few places:

src/api/actions.rs:        body::to_bytes(body).await.context(HyperSnafu)
src/from_response.rs:        let body = hyper::body::to_bytes(body)
src/lib.rs:            body::to_bytes(response.into_body())
src/lib.rs:        let body_bytes = body::to_bytes(res.into_body()).await.context(HyperSnafu)?;
src/models/repos.rs:            body::to_bytes(response.into_body())
src/page.rs:            body::to_bytes(response.into_body())
tests/follow_redirect.rs:                    format!("/repos/{new_owner}/{repo}/stargazers").into_bytes(),

@manchicken manchicken mentioned this issue Feb 4, 2024
Open
@manchicken
Copy link
Contributor

(This appears to be a duplicate of #493)

@winstonpurnomo
Copy link
Author

https://hyper.rs/guides/1/client/basic/ has some detail on how to stream payloads. They have an example here: https://github.com/hyperium/hyper/blob/master/examples/client.rs#L64-L76

Streaming like this will be a much better approach than trying to hold the entire payload in memory at once.

Sorry to ask a follow up, but the example provided:

 let mut res = sender.send_request(req).await?;

res is a Response<Incoming> type, while download_tarball, after awaiting and unwrapping, is a Response<Body>, which doesn't have a frame() method I can call.

@SSS475
Copy link

SSS475 commented Feb 22, 2024

I was able to stream a repo into a tarball on disk utilizing the following.

use http_body_util::BodyExt;

let mut resp = octocrab
    .repos(REPO_OWNER, REPO_NAME)
    .download_tarball(Reference::Branch("BRANCH_NAME".to_owned()))
    .await?;

let body = resp.body_mut();

let mut file = File::create("test.tar.gz")?;

while let Some(next) = body.frame().await {
    let frame = next?;
    if let Some(chunk) = frame.data_ref() {
        file.write_all(&chunk)?;
    }
}

In order to get frame in scope requires the BodyExt import.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@manchicken @SSS475 @winstonpurnomo