Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(Storage): Implementing support for multiple buckets #3817

Merged
merged 3 commits into from
Aug 19, 2024

Conversation

ruisebas
Copy link
Member

@ruisebas ruisebas commented Aug 15, 2024

Description

This PR makes the main changes in order to add support for multiple buckets to our existing Storage implementation.

The main changes are the following:

AmplifyOutputsData

Added the new optional buckets array to the Storage configuration, plus a convenience initializer for testing purposes.

AWSS3StoragePlugin

We have pretty much all of the service-related logic encapsulated inside the AWSS3StorageServiceBehavior protocol, which is implemented by AWSS3StorageService. This implementation, along its many related objects, heavily relies on the assumption of only one bucket and region.

To avoid a complete refactor, we will now have one storage service per bucket. S3 bucket names are globally unique, so we can store them in a dictionary using the bucket name as key for easy retrieval.

On plugin configuration we will only instantiate the "default" storage service, keeping the existing behaviour. We will only instantiate new services (and store them in the dictionary) when required, to prevent unnecessary memory usage if a customer has many buckets in their configuration but never attempts any Storage operation on most of them.

I've also added an optional additionalBucketsByName dictionary that is set when Amplify is configured with AmplifyOutputs. This is needed because we need to be able to perform the bucket lookup when the customer provides a StorageBucket.fromOutputs(:) bucket.

Finally, i've changed the type of authService to be AWSAuthCredentialsProviderBehavior. This was an oversight during the AmplifyCredentials refactor, but it's not a breaking change because AWSAuthService conforms to both.

AWSS3Storage<*>Operation

The APIs for uploading and downloading do not directly throw errors, instead they return an AmplifyTask that can be used to monitor progress and completion.

Since retrieving the storage service can throw (e.g. if a wrong bucket name is passed), and the operations need a service in order to execute, I've created the AWSS3StorageServiceProvider callback that can throw. We can invoke this provider when the operation is actually running, since we can report errors by then.

StorageConfiguration

I've removed the .default static property and instead crated a new .init(forBucket:) that just appends it to the default sessionIdentifier in order for it to be unique. This was needed because otherwise the URLSession delegate methods were not being reported to the proper storage service, which made it impossible to properly propagate the results.

General Checklist

  • Added new tests to cover change, if needed. TBA in a follow up PR
  • Build succeeds with all target using Swift Package Manager
  • All unit tests pass
  • All integration tests pass
  • Security oriented best practices and standards are followed (e.g. using input sanitization, principle of least privilege, etc)
  • Documentation update for the change if required TBA in a follow up PR
  • PR title conforms to conventional commit style
  • New or updated tests include Given When Then inline code documentation and are named accordingly testThing_condition_expectation()
  • If breaking change, documentation/changelog update with migration instructions

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@ruisebas ruisebas requested a review from a team as a code owner August 15, 2024 22:00
@ruisebas ruisebas changed the title Ruisebas/storagebucket apis feat(Storage): Implementing support for multiple buckets Aug 15, 2024
Copy link
Contributor

API Breakage Report

✅ No Public API Breaking Change detected

@github-actions github-actions bot requested a review from a team as a code owner August 15, 2024 22:06
}

/// The default bucket
var defaultBucket: ResolvedStorageBucket!
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it possible to avoid unsafe unwrapping?

@ruisebas ruisebas merged commit 0425e51 into feature/multibucket-storage Aug 19, 2024
@ruisebas ruisebas deleted the ruisebas/storagebucket_apis branch August 19, 2024 18:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants