Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added a oci function to support generic image #48

Merged
merged 6 commits into from
Oct 18, 2024
Merged

added a oci function to support generic image #48

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
7b48327
added a Oci function to support generic image
7h3-3mp7y-m4n Oct 13, 2024
95151cd
added the unit test and updated the function for remote url
7h3-3mp7y-m4n Oct 15, 2024
f02507f
Added the UnitTest with right image pull
7h3-3mp7y-m4n Oct 15, 2024
666ca89
fix test function name
7h3-3mp7y-m4n Oct 15, 2024
0c8b008
fix example_test name convention
7h3-3mp7y-m4n Oct 15, 2024
b417b1e
Added the changes and rest of the testcases
7h3-3mp7y-m4n Oct 17, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,5 +13,9 @@ require (

require (
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/opencontainers/go-digest v1.0.0 // indirect
github.com/opencontainers/image-spec v1.1.0 // indirect
github.com/pmezard/go-difflib v1.0.0 // indirect
golang.org/x/sync v0.6.0 // indirect
oras.land/oras-go/v2 v2.5.0 // indirect
)
8 changes: 8 additions & 0 deletions go.sum
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,21 @@ github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c
github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
github.com/mitchellh/mapstructure v1.5.0 h1:jeMsZIYE/09sWLaz43PL7Gy6RuMjD2eJVyuac5Z2hdY=
github.com/mitchellh/mapstructure v1.5.0/go.mod h1:bFUtVrKA4DC2yAKiSyO/QUcy7e+RRV2QTWOzhPopBRo=
github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
github.com/opencontainers/go-digest v1.0.0/go.mod h1:0JzlMkj0TRzQZfJkVvzbP0HBR3IKzErnv2BNG4W4MAM=
github.com/opencontainers/image-spec v1.1.0 h1:8SG7/vwALn54lVB/0yZ/MMwhFrPYtpEHQb2IpWsCzug=
github.com/opencontainers/image-spec v1.1.0/go.mod h1:W4s4sFTMaBeK1BQLXbG4AdM2szdn85PY75RI83NrTrM=
github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/santhosh-tekuri/jsonschema/v5 v5.3.1 h1:lZUw3E0/J3roVtGQ+SCrUrg3ON6NgVqpn3+iol9aGu4=
github.com/santhosh-tekuri/jsonschema/v5 v5.3.1/go.mod h1:uToXkOrWAZ6/Oc07xWQrPOhJotwFIyu2bBVN41fcDUY=
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
golang.org/x/sync v0.6.0 h1:5BMeUDZ7vkXGfEr1x9B4bRcTH4lpkTkpdh0T/J+qjbQ=
golang.org/x/sync v0.6.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM=
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
oras.land/oras-go/v2 v2.5.0 h1:o8Me9kLY74Vp5uw07QXPiitjsw7qNXi8Twd+19Zf02c=
oras.land/oras-go/v2 v2.5.0/go.mod h1:z4eisnLP530vwIOUOJeBIj0aGI0L1C3d53atvCBqZHg=
11 changes: 11 additions & 0 deletions uriget/example_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -51,3 +51,14 @@ func ExampleWithHttpClient() {
fmt.Println(err)
// Output: failed to make get request: Get "https://example.com": no proxy
}
func ExampleGetFile_oci() {
testUrl := "oci://ghcr.io/score-spec/score-compose:0.18.0"
7h3-3mp7y-m4n marked this conversation as resolved.
Show resolved Hide resolved
buff, err := GetFile(context.Background(), testUrl)
if err != nil {
fmt.Println("failed to pull OCI image:", err)
return
}
fmt.Println(len(buff) > 0, err)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we print err here? It needs to be nil otherwise we would be in the other branch.

// Output:
// true <nil>
}
41 changes: 41 additions & 0 deletions uriget/uriget.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,10 @@ import (
"path/filepath"
"strings"
"time"

"oras.land/oras-go/v2"
"oras.land/oras-go/v2/content/oci"
"oras.land/oras-go/v2/registry/remote"
)

// options is a struct holding fields that may need to have overrides in certain environments or during unit testing.
Expand Down Expand Up @@ -110,6 +114,8 @@ func GetFile(ctx context.Context, rawUri string, optionFuncs ...Option) ([]byte,
fallthrough
case "git-https":
return opts.getGit(ctx, u)
case "oci":
return opts.getOci(ctx, u)
default:
return nil, fmt.Errorf("unsupported scheme '%s'", u.Scheme)
}
Expand Down Expand Up @@ -233,3 +239,38 @@ func (o *options) getGit(ctx context.Context, u *url.URL) ([]byte, error) {
o.logger.Printf("Read %d bytes from %s", len(buff), filepath.Join(td, subPath))
return buff, nil
}

func (o *options) getOci(ctx context.Context, u *url.URL) ([]byte, error) {
delca85 marked this conversation as resolved.
Show resolved Hide resolved
parts := strings.Split(u.Host+u.Path, "/")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can use ParseReference to parse the artifact url into a Reference.

ref, err := registry.ParseReference(u.Host + u.Path)
if err != nil {
  return nil, fmt.Errorf("can't parse artifact url in a valid reference: %w", err)
}
registry := ref.Registry
repo := ref.Repository
var tag = "latest"
if ref.Reference != "" {
  tag = ref.Reference
}

We might get rid of all the parsing stuff from line 244 to line 256.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

alright

if len(parts) < 2 {
return nil, fmt.Errorf("invalid OCI URL format")
}
registry := parts[0]
repo := strings.Join(parts[1:len(parts)-1], "/")
tag := "latest"
lastPart := parts[len(parts)-1]
if strings.Contains(lastPart, ":") {
split := strings.Split(lastPart, ":")
repo = strings.Join(parts[1:len(parts)-1], "/") + "/" + split[0]
tag = split[1]
}
store, err := oci.New(o.tempDir)
if err != nil {
return nil, fmt.Errorf("failed to create OCI layout store: %w", err)
}
repoUrl := fmt.Sprintf("%s/%s", registry, repo)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we convert the url into a Reference, we don't need to compose the repo url anymore, but we can leverage String method from it (registry and repo vars will not be needed anymore).

remoteRepo, err := remote.NewRepository(repoUrl)
if err != nil {
return nil, fmt.Errorf("failed to connect to remote repository: %w", err)
}
if strings.HasPrefix(repoUrl, "localhost:") || strings.HasPrefix(repoUrl, "127.0.0.1:") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need this? Genuine question, I am not sure about it, it sounds weird to me that this is not handled by oras.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I added it cause I was pulling a local image from my docker remote repo, Basically, the terminal kept complaining about the HTTP request so I had to filter it, to avoid the errors and to pull the image, Maybe it was no use , so I'll remove it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't able to find anything like that in all the examples from oras library (here some examples) but I'm not an expert here, as said, it's only a gut feeling that this might be not required :).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And yes you are right :)

remoteRepo.PlainHTTP = true
}
manifestDescriptor, err := oras.Copy(ctx, remoteRepo, tag, store, tag, oras.DefaultCopyOptions)
if err != nil {
return nil, fmt.Errorf("failed to pull OCI image: %w", err)
}

o.logger.Printf("Pulled OCI image: %s with manifest descriptor: %v", u.String(), manifestDescriptor.Digest)
return []byte(manifestDescriptor.Digest), nil
}