Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use randomized content ID for Azure multipart uploads #6869

Merged
merged 10 commits into from
Dec 12, 2024
Merged

Use randomized content ID for Azure multipart uploads #6869

Changes from 1 commit
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
3e22cbc
Use randomized content ID for Azure multipart uploads
avarnon Dec 9, 2024
4181cde
Update object_store/src/azure/client.rs
avarnon Dec 11, 2024
c2839a5
fixup! Use randomized content ID for Azure multipart uploads
avarnon Dec 11, 2024
a4e8628
fixup! Use randomized content ID for Azure multipart uploads
avarnon Dec 11, 2024
d11eb21
fixup! Use randomized content ID for Azure multipart uploads
avarnon Dec 12, 2024
5edc040
fixup! Use randomized content ID for Azure multipart uploads
avarnon Dec 12, 2024
7ca0643
fixup! Use randomized content ID for Azure multipart uploads
avarnon Dec 12, 2024
b4fd825
fixup! Use randomized content ID for Azure multipart uploads
avarnon Dec 12, 2024
fae706b
fixup! Use randomized content ID for Azure multipart uploads
avarnon Dec 12, 2024
e6a4c74
fixup! Use randomized content ID for Azure multipart uploads
avarnon Dec 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions object_store/src/azure/client.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ use base64::Engine;
use bytes::{Buf, Bytes};
use chrono::{DateTime, Utc};
use hyper::http::HeaderName;
use rand::Rng as _;
use reqwest::{
header::{HeaderMap, HeaderValue, CONTENT_LENGTH, CONTENT_TYPE, IF_MATCH, IF_NONE_MATCH},
Client as ReqwestClient, Method, RequestBuilder, Response,
Expand Down Expand Up @@ -556,10 +557,11 @@ impl AzureClient {
pub(crate) async fn put_block(
&self,
path: &Path,
part_idx: usize,
_part_idx: usize,
payload: PutPayload,
) -> Result<PartId> {
let content_id = format!("{part_idx:20}");
let part_idx = u128::from_be_bytes(rand::thread_rng().gen());
let content_id = format!("{part_idx:40}");
avarnon marked this conversation as resolved.
Show resolved Hide resolved
let block_id = BASE64_STANDARD.encode(&content_id);

self.put_request(path, payload)
Expand Down
Loading