Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Could not upload files bigger than 60MB #64

Open
youcmd opened this issue Aug 19, 2023 · 5 comments · May be fixed by #72
Open

Could not upload files bigger than 60MB #64

youcmd opened this issue Aug 19, 2023 · 5 comments · May be fixed by #72
Assignees
Labels
bug Something isn't working

Comments

@youcmd
Copy link

youcmd commented Aug 19, 2023

the file only 68MB

python transferwee.py upload -v "audio.m4a"

Checking that all files exists
Checking for no duplicate filenames
Preparing to upload
Could not find any csrf-token
From storm_upload_token WETRANSFER_STORM_PREFLIGHT URL is: https://storm-us-east-1.wetransfer.net/api/v2/batch/preflight
From storm_upload_token WETRANSFER_STORM_BLOCK URL is: https://storm-us-east-1.wetransfer.net/api/v2/blocks
From storm_upload_token WETRANSFER_STORM_BLOCK URL is: https://storm-us-east-1.wetransfer.net/api/v2/batch
Get transfer id 84d0a30e9763ab565da61da02ae126c920230819085919
Doing preflight storm
Preparing storm block upload
Traceback (most recent call last):
  File "/content/transferwee.py", line 684, in <module>
    print(upload(args.files, args.n, args.m, args.f, args.t))
  File "/content/transferwee.py", line 588, in upload
    for f, b in zip(files, blocks["data"]["blocks"]):
KeyError: 'data'

i try add this code in 587 logger.debug(blocks)

{'ok': False, 'error': {'message': 'Block at position 0 ({:content_length=>71567915, :content_md5_hex=>"3dbb5db155ffebc8d7ce20c804ada891"}) is too large (max is 62914560)'}}
@iamleot iamleot self-assigned this Aug 19, 2023
@iamleot iamleot changed the title i can't upload Could not upload files bigger than 60MB Aug 19, 2023
@iamleot
Copy link
Owner

iamleot commented Aug 19, 2023

I have adjusted the issue title according the very likely root case of the problem. I will try to reproduce it and hopefully fix it. I guess that we need to upload each file in block of 60MB or similar.

Thanks for the report and debug information!

@iamleot iamleot added the bug Something isn't working label Aug 19, 2023
@SaadDevCs
Copy link

Any solution for this issue ?

@iamleot
Copy link
Owner

iamleot commented Aug 30, 2023

Hello!
It needs fixing in several places. Basically around the code there are some XXX: comments that assumed that one single block upload was ok (but there are also some other places not marked with such comments). This assumption was wrong. I need to sit down and review and adjust all API calls related to file and block uploads.

As a possible workaround, at least on Unix split(1) can be used as a workaround.

@iamleot
Copy link
Owner

iamleot commented Oct 22, 2023

Please also see #70, probably the limit was changed to even lower block size.

sharevb added a commit to sharevb/transferwee that referenced this issue Oct 25, 2023
sharevb added a commit to sharevb/transferwee that referenced this issue Nov 29, 2023
As of issue iamleot#76, PR iamleot#77 and to accomodate issue iamleot#64, thus case of "many small" or "really big" files, chunk big files in many blocks and blocks request also in chunks if more than 100 blocks need for the transfer of all files
@sharevb
Copy link
Contributor

sharevb commented Nov 29, 2023

Updated with #76

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants