Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

5GB を超えるファイルのアップロード帯域制限に対応する #50

Merged
merged 5 commits into from
Oct 31, 2024

Conversation

tnamao
Copy link
Collaborator

@tnamao tnamao commented Oct 30, 2024

変更履歴

  • [FIX] 5GB を超えるファイルのアップロード時に帯域制限がかかるように修正する
    • 帯域制限設定を行なってもマルチパートアップロードを有効にし、マルチパートアップロードの並列アップロード数を 1 つずつにすることで帯域制限を行う
    • この修正以前は、帯域制限設定を行うとマルチパートアップロードが無効となり 5GB を超えるファイルのアップロードができなかった
    • @tnamao

This pull request includes changes to implement bandwidth throttling for large file uploads and updates to dependencies. The most important changes include adding bandwidth throttling for files over 5GB and updating the CI and Go versions.

Bandwidth Throttling for Large File Uploads:

  • CHANGES.md: Added a change log entry describing the implementation of bandwidth throttling for files over 5GB by enabling multipart uploads with a single thread.
  • s3.go: Modified the uploadWebMFileWithRateLimit function to set the NumThreads option to 1 for multipart uploads when bandwidth is limited.

Dependency Updates:

  • CHANGES.md: Updated the CI staticcheck to version 2024.1.1 and Go to version 1.23.2.

@tnamao tnamao requested review from Hexa and voluntas October 30, 2024 02:41
n, err := s3Client.PutObject(ctx, osConfig.BucketName, dst, fileReader, fileSize,
minio.PutObjectOptions{ContentType: "application/octet-stream", DisableMultipart: true})
minio.PutObjectOptions{ContentType: "application/octet-stream", NumThreads: 1})
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯

CHANGES.md Outdated Show resolved Hide resolved
Copy link
Member

@voluntas voluntas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

変更履歴だけ。あとは ok です。

Copy link
Contributor

@Hexa Hexa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CHANGES の文言だけで、他は LGTM です

CHANGES.md Outdated Show resolved Hide resolved
@tnamao
Copy link
Collaborator Author

tnamao commented Oct 31, 2024

マージします

@tnamao tnamao merged commit c0d7a2b into develop Oct 31, 2024
1 check passed
@tnamao tnamao deleted the feature/large-file-bandwidth-limitation branch October 31, 2024 03:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants