# Find snapshot path SNAPSHOT_PATH=$(crictl inspect <container> | jq -r '.info.rootDir') cp -al $SNAPSHOT_PATH /tmp/clone-rootfs Now modify /tmp/clone-rootfs without affecting the original (COW at file level)
This article explores the relationship between CRI-compliant runtimes (containerd and CRI-O), the filesystem tools that manipulate container storage, and how the humble link (both symbolic and hard) functions as the architectural glue holding container layers together. Before diving into tools and links, we must establish a baseline. The CRI is a Kubernetes API that kubelet uses to communicate with container runtimes. It abstracts the runtime implementation, allowing Kubernetes to work with Docker (via dockershim, now deprecated), containerd, CRI-O, and others. cri file system tools link
/var/lib/containers/storage/overlay/<layer-id>/merged -> /var/lib/containers/storage/overlay/<layer-id>/../<parent-id>/merged Scenario 1: "No such file or directory" inside a container Even though the file exists in the image, the container cannot see it. This is often due to a broken symbolic link in a lower layer . If your cluster uses containerd, ctr provides direct
If your cluster uses containerd, ctr provides direct access to namespaces and snapshots. and disaster recovery. Master these tools.
Introduction: The Hidden Complexity of Container Filesystems In the world of containerized applications, the storage layer is often treated as a black box. Developers run docker run or kubectl apply , and somehow, the files appear. But beneath the surface lies a sophisticated ecosystem of snapshots, layers, and mount points. For those managing Kubernetes clusters using the Container Runtime Interface (CRI), understanding CRI file system tools and the critical role of the link (symbolic or hard link) is not just an advanced skill—it is a necessity for debugging, performance tuning, and disaster recovery.
Master these tools. Respect the link. Debug with confidence. Have a specific CRI filesystem issue related to links? Use the commands above to inspect your environment, and always test link operations in a non-production cluster first.
ctr -n k8s.io snapshot rm <snapshot-key> ctr -n k8s.io snapshot gc # Garbage collects unlinked snapshots Check /var/lib/containerd/io.containerd.snapshotter.v1.overlayfs/metadata.db (a BoltDB file) for orphaned links. Tools like boltdb-viewer can inspect it. Scenario 3: Migrating container rootfs to another disk using symlinks Suppose your /var/lib/containerd partition is full. You can move the storage directory and create a symbolic link.