I don’t normally write blog posts that regurgitate information from normal documentation, but this particular subject irks me.

If you are running an internal Kubernetes (k8s) platform, you owe it to yourself to make sure there is nothing external to your platform determining your reliability.

You could ask yourself: How many internet dependencies do you have to start a pod? Should be zero, right???

If you use stock k8s, you might be surprised to know that each of your k8s nodes is actually reaching out to registry.k8s.io on first pod creation to get the pause image:

$ sudo crictl images
IMAGE                                     TAG                 IMAGE ID            SIZE
registry.k8s.io/pause                     3.9                 e6f1816883972

If you want to change that, you can update your containerd (1.x) toml:

[plugins."io.containerd.grpc.v1.cri"]
  sandbox_image = "YOUR_REGISTRY/pause:3.10"

And depend on one less thing. The rest of the blog post will go deeper into why this is the case.

The pause image is the container image that backs the k8s “sandbox” of a pod. This pause container is designed to hold the linux namespaces. The pause container used to also reap zombie processes from the other containers in a pod, its duty as PID1, but that isn’t the case by default anymore in k8s 1.8+.

The sandbox of a pod is part of the CRI spec. The CRI spec is a generic way for k8s to talk pods (and sandboxes) that is not specific to any particular container runtime (like containerd). Any container runtime that implements the CRI spec can, in theory, run k8s pods.

This means that the pause image has more to do with CRI than it does with k8s.

When a CRI-enabled container runtime needs to create a sandbox, at least with the case of containerd, it does this by creating a real container.

The image containerd is configured to use (by default) to create that sandbox, is the pause image. You can see this in code here.

Per the current docs, you can overwrite the containerd sandbox image with a containerd configuration like this (assuming you have mirrored to a local registry):

(containerd 1.x)

[plugins."io.containerd.grpc.v1.cri"]
  sandbox_image = "YOUR_REGISTRY/pause:3.10"

(containerd 2.x)

version = 3

[plugins]
  [plugins.'io.containerd.cri.v1.images']
    ...
    [plugins.'io.containerd.cri.v1.images'.pinned_images]
      sandbox = 'YOUR_REGISTRY/pause:3.10'

Don’t take my word for it here, this particular setting has changed over time, check the official docs.

If you go to registry.k8s.io you will see:

Please note that there is NO uptime SLA as this is a free, volunteer managed service. We will however do our best to respond to issues and the system is designed to be reliable and low-maintenance. If you need higher uptime guarantees please consider mirroring images to a location you control.

So yea, this is your PSA. Please mirror like they recommend and reconfigure as needed to not depend on the internet.