Upload progress

[oleg-moseyko]

Describe the feature

I would like to have ability to get multipart upload progress. How can to do this?

Use Case

func multipartUpload(bucket: String, key: String, fileUrl: URL, uuid: String, meta: Meta, finishBlock: @escaping FinishBlock) async {
logger.info("AWSUploadRun start...", subsystem: .upload)
do {
let identityResolver = try await self.getTempCredentials()
let region = "us-east-1"
let configuration = try await S3Client.S3ClientConfiguration(awsCredentialIdentityResolver: identityResolver, region: region)
configuration.region = "us-east-1"
let client = S3Client(config: configuration)
let fileData = try Data(contentsOf: fileUrl)
let fileSize = fileData.count
let partSize = 5 * 1024 * 1024
let expectedParts = Array(1...(fileSize / partSize + (fileSize % partSize == 0 ? 0 : 1)))
let objectKey = key
var metadata : [String: String] = [:]
for (metaKey, metaValue) in meta.data {
metadata[metaKey] = "(metaValue)".mimeEncode
}

        print("metadata", metadata)
        
        let output = try await client.createMultipartUpload(
            input: .init(bucket: bucket, key: objectKey, metadata: metadata)
        )
        var uploadedParts: [S3ClientTypes.CompletedPart] = []
        for idx in expectedParts  {
           // let body = ClientRuntime.ByteStream.from(data: chunk)
            
            
            let start = (idx - 1) * partSize
                let end = min(start + partSize, fileSize)
                let chunk = fileData.subdata(in: start..<end)
            
            let part = try await client.uploadPart(input: .init(
                body: ByteStream.data(chunk),
                bucket: bucket,
                key: objectKey,
                partNumber: idx,
                uploadId: output.uploadId
                ))
            
            let completedPart = S3ClientTypes.CompletedPart(eTag: part.eTag, partNumber: idx)
            
            uploadedParts.append(completedPart)
        }
        
        print("LISTING PARTS")
        let listedParts = client.listPartsPaginated(input: .init(
            bucket: bucket,
            key: objectKey,
            maxParts: 2,
            uploadId: output.uploadId
            ))
        
        await print(try listedParts.parts()) // this used to infinite loop, now prints all parts across all pages
        
        print("LISTED")
        // Complete the multipart upload
        
        let completeMultipartUploadInput = CompleteMultipartUploadInput(
            bucket: bucket,
            key: objectKey,
            multipartUpload: .init(parts: uploadedParts),
            uploadId: output.uploadId
        )
        _ = try await client.completeMultipartUpload(input: completeMultipartUploadInput)
        finishBlock ()
    } catch {
        print("COULDN'T upload FILE", error.localizedDescription)
    }

}

Proposed Solution

No response

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

[sichanyoo]

Hello, thank you for submitting the ticket.

AWS SDK for Swift currently does not provide a out-of-the-box way to track multipart upload transfer progress at the moment. We've looked into making an utility wrapper for S3Client called something like S3TransferManager in the past, which would provide this feature.

I'll bring this up to the team and let you know if the work gets prioritized / picked up, thanks.

[harlanhaskins]

Seconding this feature request -- I think unfortunately without the ability to monitor upload progress, we can't use the AWS SDK and will have to do the upload manually.