Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

S3 endpoint iterates through untargeted folders #63

Open
vacri opened this issue Aug 14, 2018 · 1 comment
Open

S3 endpoint iterates through untargeted folders #63

vacri opened this issue Aug 14, 2018 · 1 comment

Comments

@vacri
Copy link

vacri commented Aug 14, 2018

Hi,

If I have an s3 target s3://mybackup/buttersink/, buttersink will iterate through all the files in s3://mybackup, ignoring the /buttersink/ restriction - so if I have a generic 'backup' bucket for various services, buttersink will needlessly go through all my backup files that live in other directories, which can take a while

Currently I've solved this by giving buttersink its own bucket, but this means I need to adjust user permissions to allow people to see that.

This is with pip-installed v0.6.9

@AmesCornish
Copy link
Owner

At the moment, I consider this a "feature". Buttersink will find needed diffs in other parts of your destination bucket and copy them over to the desired key. This saves all the expense of re-transferring a diff that's already in the destination bucket. Obviously there is a trade-off in that buttersink examines all the keys in the bucket. It would not be too hard to create a flag for this behavior. Thoughts?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants