FAQ

Frequently Asked Questions

At any time, let's continue to update.

Q: Where can I find documentation?

A: You could find them in: https://github.com/dragonflyoss/nydus/tree/master/docs

Q: How to sync nydus images between different registries?

A: You could using nerdctl to sync nydus images with --unpack false

nerdctl pull --unpack false tomcat:latestnydusv6
nerdctl tag tomcat:latestnydusv6 ghcr.io/username/tomcat:latestnydusv6
nerdctl push --allow-nondistributable-artifacts   ghcr.io/username/tomcat:latestnydusv6

Q: Can't run the image built by nydus >= v2.2.x in nydus <=v2.1.3

A: Try nydus >= v2.1.4, the version fixed the nydusd error when running the images created by nydusify/buildkit/acceld using nydus-image >= 2.2, strongly recommend all users to upgrade.

Q: How to migrate the root dir of nydus shanpshotter?

A: Please refer to: https://github.com/containerd/nydus-snapshotter/issues/488

Troubleshooting

Q: Can't start nydus image in k8s. Error: failed to create snapshot: missing parent "k8s.io/2/sha256:******" bucket: not found

See the below Question.

Q: Can't start nydus pod in K8s. Error: failed to get reader from content store: content digest sha256:*** not found

A: In some environments, the containerd will discard unpacked layers in the content store. So when the pause container is started, it is found that the snapshotter is nydus and needs to be unpacked again, but the data cannot be found in the content store.

1. You could try to set discard_unpapcked_layers = false in /etc/containerd/config.toml

[plugins."io.containerd.grpc.v1.cri".containerd]
  discard_unpacked_layers = false

2. Try ctr -n k8s.io content fetch $pause-image-name on the node, the $pause-image-name is the k8s pause image you are using.

This case occurs after using the default snapshotter (usually overlay snapshotter) of containerd to pull OCI v1 images and then switching to nydus snapshotter, you can put the command in the snapshotter startup script.

Q: Can't start nydus pod in K8s. Error: failed to find image ref of snapshot

A: Containerd CRI config disable_snapshot_annotations is not set to false. disable_snapshot_annotations disables to pass additional annotations (image related information) to snapshotters. These annotations are required by nydus snapshotter.

1. You could configure disable_snapshot_annotations = false in /etc/containerd/config.toml

[plugins."io.containerd.grpc.v1.cri".containerd]
   snapshotter = "nydus"
   disable_snapshot_annotations = false

Q: Can't start nydus image. Error: failed to get stream processor for application/vnd.oci.image.layer.nydus.blob.v1: no processor for media-type: unknown

A: Nydus-snapshotter is not selected by containerd, causing the nydus image to fail to be parsed.

1. You could try to configure [proxy_plugins] in /etc/containerd/config.toml

[proxy_plugins]
  [proxy_plugins.nydus]
    type = "snapshot"
    address = "/run/containerd-nydus/containerd-nydus-grpc.sock"

2. If you wish to switch containerd's default snapshotter to nydus, you can also configure [plugins."io.containerd.grpc.v1.cri".containerd] in /etc/containerd/config.toml

[plugins."io.containerd.grpc.v1.cri".containerd]
   snapshotter = "nydus"
   disable_snapshot_annotations = false

Q: Can't pull nydus image from the registry. Error: temporarily failed to get blob.meta, Input/output error (os error 5)

A: There may be a problem with the network accessing the registry.

1. Please check whether the firewall has opened access to the registry. In addition to the registry, the object storage service (AWS S3 / Alibaba OSS) associated with it may also need to allow access.
2. Please check that the authentication to the registry is configured correctly. Try docker login your.registry.com or nerdctl login your.registry.com

Q: Can't start nydus image. Error: "invalid super flags 1b4" or Mount(MountFilesystem(Other("Rafs(FillSuperblock(Os { code: 22, kind: InvalidInput, message: \"Invalid argument\" }))")))

A: Try nydus >= v2.1.4, the version fixed the nydusd error when running the images created by nydusify/buildkit/acceld using nydus-image >= 2.2, strongly recommend all users to upgrade.

Q: Can't start nydus image. Error: transport: Error while dialing dial unix /run/containerd-nydus/containerd-nydus-grpc.sock: connect: connection refused

A: Nydus snapshotter (containerd-nydus-grpc) is not running, consider starting it.

Or the socket running by nydus-snapshotter is different from the containerd configuration, and needs to be modified to the same.

Q: Can't start nydus snapshotter container by helm/docker. Error: transport: Error while dialing dial unix /run/containerd-nydus/containerd-nydus-grpc.sock: connect: connection refused

A: Please remove nydus snapshotter configuration from [plugins."io.containerd.grpc.v1.cri".containerd] in /etc/containerd/config.toml.

You have already configured nydus in containerd, but the snapshotter itself needs to be started in the overlay, because it cannot depend on itself circularly.

Q: Kubelet throws errors: Failed to get the info of the filesystem with mountpoint" err="failed to get device for dir "/var/lib/containerd/io.containerd.snapshotter.v1.nydus": stat failed on /var/lib/containerd/io.containerd.snapshotter.v1.nydus with error: no such file or directory" mountpoint="/var/lib/containerd/io.containerd.snapshotter.v1.nydus"

A: See https://github.com/containerd/containerd/issues/10095, the recommend workaround: ln -s /var/lib/containerd-nydus /var/lib/containerd/io.containerd.snapshotter.v1.nydus.