Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Problem configuring proxy for docker.io #14

Open
piontec opened this issue Sep 28, 2023 · 3 comments
Open

Problem configuring proxy for docker.io #14

piontec opened this issue Sep 28, 2023 · 3 comments

Comments

@piontec
Copy link

piontec commented Sep 28, 2023 •
edited
Loading

Hey!
I've configured containerd as below:

root@wc1-oci-reg-control-plane:/etc/containerd/certs.d# cat docker.io/hosts.toml 
server = "https://registry-1.docker.io"

[host."http://172.17.0.1:30180"]
	capabilities = ["pull", "resolve"]

Proxy seems to be deployed fine:

root@wc1-oci-reg-control-plane:/etc/containerd/certs.d# curl -s 172.17.0.1:30180/metrics | head
# HELP http_incoming_requests Incoming Requests
# TYPE http_incoming_requests counter
http_incoming_requests{endpoint="/",method="GET",status="200"} 206
http_incoming_requests{endpoint="/metrics",method="GET",status="200"} 4
http_incoming_requests{endpoint="/v2/{image:[^{}]+}/manifests/{reference}",method="HEAD",status="500"} 1
# HELP http_response_code Response Codes

Still, when I try to pull a simple image from docker hub:

root@wc1-oci-reg-control-plane:/etc/containerd/certs.d# crictl pull docker.io/alpine:latest
Image is up to date for sha256:7e01a0d0a1dcd9e539f8e9bbd80106d59efbdf97293b3d38f5d7a34501526cdb

I get some 403 in the oci-registry log stream and nothing is saved in the data directory:

Logs

2023-09-28T13:44:05.762002Z DEBUG reqwest::connect: starting new connection: https://docker.io/    
2023-09-28T13:44:05.762247Z DEBUG hyper::client::connect::dns: resolving host="docker.io"
2023-09-28T13:44:11.774785Z DEBUG hyper::client::connect::http: connecting to 54.165.156.197:443
2023-09-28T13:44:11.887714Z DEBUG hyper::client::connect::http: connected to 54.165.156.197:443
2023-09-28T13:44:11.887741Z DEBUG rustls::client::hs: No cached session for DnsName("docker.io")    
2023-09-28T13:44:11.887790Z DEBUG rustls::client::hs: Not resuming any session    
2023-09-28T13:44:12.000008Z DEBUG rustls::client::hs: Using ciphersuite TLS13_AES_128_GCM_SHA256    
2023-09-28T13:44:12.000046Z DEBUG rustls::client::tls13: Not resuming    
2023-09-28T13:44:12.000356Z DEBUG rustls::client::tls13: TLS1.3 encrypted extensions: [ServerNameAck]    
2023-09-28T13:44:12.000366Z DEBUG rustls::client::hs: ALPN protocol is None    
2023-09-28T13:44:12.000914Z DEBUG hyper::proto::h1::io: flushed 378 bytes
2023-09-28T13:44:12.338080Z DEBUG hyper::proto::h1::io: parsed 2 headers
2023-09-28T13:44:12.338106Z DEBUG hyper::proto::h1::conn: incoming body is empty
2023-09-28T13:44:12.338161Z DEBUG hyper::client::pool: pooling idle connection for ("https", docker.io)
2023-09-28T13:44:12.338206Z DEBUG reqwest::async_impl::client: redirecting 'https://docker.io/v2/library/alpine/manifests/latest?ns=docker.io' to 'https://www.docker.com/v2/library/alpine/manifests/latest?ns=docker.io'    
2023-09-28T13:44:12.338238Z DEBUG hyper::client::pool: reuse idle connection for ("https", www.docker.com)
2023-09-28T13:44:12.338309Z DEBUG Connection: h2::codec::framed_write: send frame=Headers { stream_id: StreamId(3), flags: (0x5: END_HEADERS | END_STREAM) } peer=Client
2023-09-28T13:44:12.353054Z DEBUG Connection: h2::codec::framed_read: received frame=Headers { stream_id: StreamId(3), flags: (0x4: END_HEADERS) } peer=Client
2023-09-28T13:44:12.353139Z DEBUG Connection: h2::codec::framed_read: received frame=Data { stream_id: StreamId(3) } peer=Client
2023-09-28T13:44:12.353152Z DEBUG Connection: h2::codec::framed_read: received frame=Data { stream_id: StreamId(3), flags: (0x1: END_STREAM) } peer=Client
2023-09-28T13:44:12.353282Z DEBUG hyper::client::pool: reuse idle connection for ("https", docker.io)
2023-09-28T13:44:12.353414Z DEBUG hyper::proto::h1::io: flushed 365 bytes
2023-09-28T13:44:12.465971Z DEBUG hyper::proto::h1::io: parsed 2 headers
2023-09-28T13:44:12.465987Z DEBUG hyper::proto::h1::conn: incoming body is empty
2023-09-28T13:44:12.466013Z DEBUG hyper::client::pool: pooling idle connection for ("https", docker.io)
2023-09-28T13:44:12.466036Z DEBUG reqwest::async_impl::client: redirecting 'https://docker.io/v2/library/alpine/manifests/latest' to 'https://www.docker.com/v2/library/alpine/manifests/latest'    
2023-09-28T13:44:12.466054Z DEBUG hyper::client::pool: reuse idle connection for ("https", www.docker.com)
2023-09-28T13:44:12.466092Z DEBUG Connection: h2::codec::framed_write: send frame=Headers { stream_id: StreamId(5), flags: (0x5: END_HEADERS | END_STREAM) } peer=Client
2023-09-28T13:44:12.489660Z DEBUG Connection: h2::codec::framed_read: received frame=Headers { stream_id: StreamId(5), flags: (0x4: END_HEADERS) } peer=Client
2023-09-28T13:44:12.489703Z DEBUG Connection: h2::codec::framed_read: received frame=Data { stream_id: StreamId(5) } peer=Client
2023-09-28T13:44:12.489711Z DEBUG Connection: h2::codec::framed_read: received frame=Data { stream_id: StreamId(5), flags: (0x1: END_STREAM) } peer=Client
2023-09-28T13:44:12.489757Z ERROR oci_registry::api::error: 500: Error with upstream registry: unexpected HTTP status 403 Forbidden
2023-09-28T13:44:12.489802Z DEBUG actix_web::middleware::logger: Error in response: Upstream(UnexpectedHttpStatus(403))    
2023-09-28T13:44:12.489829Z  INFO actix_web::middleware::logger: 10.244.0.1 "HEAD /v2/library/alpine/manifests/latest?ns=docker.io HTTP/1.1" 500 66 "-" "containerd/v1.7.1" 19.567011    
2023-09-28T13:44:17.571069Z DEBUG rustls::common_state: Sending warning alert CloseNotify

Tested on helm chart v0.3.20 with the following config file

replicas: 1
registry:
  storage:
    mode: filesystem
service:
  type: NodePort
@mcronce
Copy link
Owner

mcronce commented Oct 22, 2023

It looks like we failed to authenticate with docker.io before trying to pull the manifest...that's odd. I'll take a look.

@piontec
Copy link
Author

piontec commented Jun 1, 2024 via email

@covert8
Copy link
Contributor

covert8 commented Jun 3, 2024 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@mcronce @piontec @covert8