Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

WIP: Azure Blob Storage implementation #2223

Closed
wants to merge 14 commits into from
Closed

WIP: Azure Blob Storage implementation #2223

wants to merge 14 commits into from

Conversation

jkroepke
Copy link
Contributor

@jkroepke jkroepke commented Feb 1, 2024

Blocked by


What type of PR is this?

feature

Which issue does this PR fix:

#2221

What does this PR do / Why do we need it:

Native support for Azure blob storage

If an issue # is not available please add repro steps and logs showing the issue:

Testing done on this change:

Automation added to e2e:

Will this break upgrades or downgrades?

Does this PR introduce any user-facing change?:

Native support for Azure Blob storage

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@jkroepke
Copy link
Contributor Author

jkroepke commented Feb 1, 2024

This PR is marked as WIP, since it depends on the distribution v3 packages which is not released yet.

@eusebiu-constantin-petu-dbk
Copy link
Collaborator

Hello @jkroepke Isn't it possbile to reuse the s3 driver? and make it more generic. in theory it should work with all major cloud providers.

@jkroepke
Copy link
Contributor Author

jkroepke commented Feb 2, 2024

in theory it should work with all major cloud providers.

In practice, there is no S3 Compatibility API at Azure. The distribution project maintains an azure driver which I re-use here:

https://github.com/distribution/distribution/blob/main/docs/content/storage-drivers/azure.md

If upstream distribution project nativly support Azure, it should not be a major issue for zot? I thought, zot fully depends on it and the usage for storage driver is transparent.

@eusebiu-constantin-petu-dbk
Copy link
Collaborator

in theory it should work with all major cloud providers.

In practice, there is no S3 Compatibility API at Azure. The distribution project maintains an azure driver which I re-use here:

https://github.com/distribution/distribution/blob/main/docs/content/storage-drivers/azure.md

If upstream distribution project nativly support Azure, it should not be a major issue for zot? I thought, zot fully depends on it and the usage for storage driver is transparent.

I see, understood, ok.

@rchincha rchincha added this to the v2.1.0 milestone Mar 1, 2024
@rchincha
Copy link
Contributor

rchincha commented Mar 1, 2024

Let's target this for v2.1.0

@jkroepke
Copy link
Contributor Author

jkroepke commented Mar 1, 2024

Let's target this for v2.1.0

A requirement for this PR would be github.com/docker/distribution v3.0.0-alpha.1 or higher. Thats something that should not be part of the PR here.

@jkroepke jkroepke mentioned this pull request Jul 4, 2024
@rchincha
Copy link
Contributor

rchincha commented Aug 2, 2024

@jkroepke I think you can make progress on this one now?

jkroepke and others added 14 commits August 3, 2024 09:54
Signed-off-by: Jan-Otto Kröpke <[email protected]>
Signed-off-by: Jan-Otto Kröpke <[email protected]>
Signed-off-by: Jan-Otto Kröpke <[email protected]>
Signed-off-by: Jan-Otto Kröpke <[email protected]>
Signed-off-by: Jan-Otto Kröpke <[email protected]>
Signed-off-by: Jan-Otto Kröpke <[email protected]>
Signed-off-by: Jan-Otto Kröpke <[email protected]>
Signed-off-by: Jan-Otto Kröpke <[email protected]>
@jkroepke
Copy link
Contributor Author

jkroepke commented Aug 6, 2024

@jkroepke I think you can make progress on this one now?

I will do, but sadly it does not work out of the box and I have to investigate here whats going wrong. it will take some time.

Using Azure Tables as remote cache will be done in a separate PR.

@rchincha
Copy link
Contributor

@jkroepke do you have an update on this? or need any assistance?

@jkroepke
Copy link
Contributor Author

Hey @rchincha

Sorry, but I have other priorities that I need to address, and it might take a while before I can return to this.

@rchincha
Copy link
Contributor

@jkroepke any updates on this? This would be useful for you and for the community at large.

@rchincha rchincha modified the milestones: v2.1.0, v2.2.0 Sep 12, 2024
@jkroepke
Copy link
Contributor Author

Hi @rchincha

After reviewing #2584 and #724, I need to pause any further community work for now. Our initial tests did not cover signatures, but the lack of signature support could potentially cause issues in future scenarios.

@jkroepke jkroepke closed this Sep 19, 2024
@jkroepke jkroepke deleted the azure branch September 19, 2024 21:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants