Container sandboxing in 2026: revisit gVisor, Kata Containers and Firecracker before you run another untrusted workload

A normal container is a set of Linux processes with namespaces, cgroups, capabilities and a seccomp profile around them. That is a good packaging boundary and a reasonable resource boundary. It is a thinner security boundary than many teams assume, because every container on a node still talks to the same host kernel. If the code inside the container is your own, reviewed and built by your pipeline, that is usually an acceptable trade. If the code was written by a stranger an hour ago, or generated by a model a minute ago, it deserves a second wall.

This post looks at three open-source ways to build that second wall: gVisor, Kata Containers and Firecracker. All three are actively maintained on 26 October 2026, and two of them have had big changes this year. gVisor is moving from Google to the CNCF. Kata Containers 4.0 replaced its Go runtime with a Rust one as the default. Firecracker keeps shipping steadily from AWS. They solve the same problem in different ways. gVisor puts a second kernel, written in Go, between the application and the host kernel. Kata Containers runs each pod inside a small virtual machine with its own guest kernel. Firecracker is a minimal virtual machine monitor that other tools, Kata among them, can use to run those small virtual machines.

We have covered lightweight Kubernetes distributions, admission policy and pod networking before. This one sits under all of them: what actually stands between a pod and the host kernel, and how to choose a stronger boundary for the pods that need it.

A scenario to keep us honest

Take an invented Visakhapatnam company that runs an online coding assessment platform for engineering colleges in Andhra Pradesh and Odisha. Colleges use it for lab exams, placement mock tests and weekly practice. Students submit code in C, C++, Java and Python. The platform compiles and runs each submission against hidden test cases and returns a score. About 35 engineers.

The setup looks like this:

• A managed Kubernetes cluster on cloud virtual machines. Worker nodes are ordinary VMs, not bare metal.
• A queue of submissions. A pool of “runner” pods picks up each submission, compiles it, runs it with time and memory limits, compares the output and writes the result back.
• Runner pods use the default runtime, runc, with a non-root user and the default seccomp profile. The service account token is mounted because an early version of the runner used it to read a ConfigMap.
• Hidden test cases are fetched by the runner from an internal object storage bucket at the start of each job.
• A separate GPU node runs a small model that writes feedback on code style. Only the company’s own code runs there.

During a placement mock test week, a student’s Python submission did not try to escape anything. It simply listed environment variables and files it could see, found the mounted service account token and the internal object storage endpoint, and printed them as program output. Over the next few attempts it used the runner’s network access to list the bucket and print the hidden test cases for the next day’s paper. The output landed in the student’s own result page. Copies were in a messaging group by evening. The same week, another submission started a fork bomb that hit the pod’s process limit only after it had slowed every other runner pod on that node, because no PID limit had been set for those pods.

No kernel exploit was involved. The damage came from what the pod could reach, not from breaking out of it. That is the first lesson, and it matters for the rest of this post: a sandbox runtime protects the host kernel from the code. It does not hide a token you mounted into the sandbox, or block network paths you left open. The company needed both a stronger runtime for code it did not write and a much smaller set of things that code could see.

The short version

• A container is not a sandbox by default. runc containers share the host kernel. A kernel bug reachable through an allowed system call is a path from the container to the node, and from the node to every other pod on it.
• gVisor (Apache 2.0, created at Google, now being donated to the CNCF) runs an application kernel called the Sentry in user space. The application’s system calls go to the Sentry, not to the host kernel. It does not need hardware virtualization, so it runs on ordinary cloud VMs. The cost is overhead on system-call-heavy and file-heavy work, and some Linux behaviour that is not implemented.
• Kata Containers (Apache 2.0, an OpenInfra project, now under the Linux Foundation) runs each pod in a lightweight virtual machine with its own guest kernel. You get a hardware virtualization boundary, broad Linux compatibility, GPU passthrough with QEMU, and confidential computing options. It needs KVM on the node, which on cloud VMs means nested virtualization or bare metal.
• Firecracker (Apache 2.0, built at AWS for Lambda and Fargate) is a minimal virtual machine monitor, not a Kubernetes runtime on its own. In Kubernetes you meet it through Kata’s kata-fc class, which still uses the deprecated Go runtime and needs a devmapper snapshotter. Used directly, it is a building block for your own sandbox service.
• RuntimeClass is the switch. You configure a handler in containerd or CRI-O, create a RuntimeClass that points to it, and set runtimeClassName on the pods that need it. Everything else keeps using runc.
• The runtime is one layer. No token, no unneeded network, resource limits including PIDs, and an admission rule that forces the right RuntimeClass matter as much as the runtime itself.

For the Visakhapatnam team, a sensible answer is gVisor for the runner pods on a dedicated node pool, because their nodes are ordinary cloud VMs and the workload is short CPU-bound programs. Kata becomes worth it later if they move runners to bare metal or need full Linux behaviour. Firecracker is the right tool only if they decide to build their own execution service outside Kubernetes.

Where things stand on 26 October 2026

Versions and dates below are from GitHub release pages, the GitHub API and each project’s own documentation, checked on 26 October 2026. Times are in IST.

• gVisor release-20261019.0, published 23 October 2026 at 12:06 AM. The one before it, release-20261012.0, came on 17 October 2026 at 2:23 AM. gVisor does not use semantic versions; releases are named by date and come out roughly every week. Recent changelogs include nvproxy work such as nvidia-smi support and more checkpoint and restore improvements. The install guide says gVisor supports x86_64 and ARM64 and needs Linux 5.6 or later on the host. Licence: Apache 2.0.
• Kata Containers 4.3.1, released 22 October 2026 at 3:17 AM, one day after 4.3.0 (21 October 2026 at 2:31 AM). 4.3.0 carried two security fixes: a permissive annotation allowlist in the default kata-deploy setup that could let pod authors change host and guest behaviour (rated CVSS 7.9), and a guest-to-host escape through out-of-bounds memory access in several virtio drivers in the Dragonball VMM (rated CVSS 6.4). The 4.x line began with 4.0.0 on 20 July 2026 at 9:51 PM, followed by 4.1.0 on 21 August and 4.2.0 on 15 September. Licence: Apache 2.0.
• Firecracker v1.17.0, released 10 September 2026 at 7:49 PM, together with the patch v1.16.2 two minutes earlier. 1.17 added transparent huge pages for guest memory, booting bzImage kernels on x86_64, official support for 6.18 guest kernels and virtio-blk discard. v1.16.2 changed the snapshot format version, so snapshots from 1.16.0 or 1.16.1 cannot be loaded by 1.16.2, and the other way round. The README says releases come typically every two or three months. Licence: Apache 2.0.
• firecracker-containerd has no tagged releases. Its last commits were in July 2026, including an upgrade to containerd v1.7.33 on 11 July 2026. Its README still lists CRI conformance and Kubernetes compatibility as longer-term roadmap items, and it needs a specially built containerd binary.
• Context: Kubernetes v1.37.1 (24 September 2026 at 12:43 AM) and containerd v2.4.2 (14 October 2026 at 4:05 AM) are the latest releases of those projects. RuntimeClass has been stable since Kubernetes 1.20 and Pod Overhead since 1.24.

On governance, each project has a different owner, and two of them changed this year.

• gVisor was open-sourced by Google in 2018. On 2 October 2026 the gVisor blog announced that Google is donating the project, including its name and trademarks, to the CNCF. The post gives the timeline: application submitted on 7 September, reviewed on 22 September, accepted on 28 September. The CNCF sandbox issue shows the vote as passed, and on 26 October the onboarding was still in progress, with the repository still under the google GitHub organisation. The post says the repository will move out of that organisation over the coming months, that governance will move to a maintainer-based model and later to organisation-based voting, and names Ant Group, Modal and Tines as committed new maintainers. Plan for repository URL changes, the same way Velero users did this year.
• Kata Containers was launched in December 2017 under what was then the OpenStack Foundation, by merging Intel Clear Containers and Hyper’s runV. Version 1.0 came in May 2018 and 2.0 in October 2020. The OpenStack Foundation later became the OpenInfra Foundation, which completed joining the Linux Foundation on 1 June 2025. The OpenInfra announcement says its projects keep their own governance and community processes.
• Firecracker was announced by AWS on 26 November 2018 and is maintained in the firecracker-microvm GitHub organisation. AWS says it powers Lambda and Fargate. It is not in a foundation.

All three are alive and maintained. The thing to plan for is not abandonment but change: gVisor’s move, and Kata’s move from the Go runtime to the Rust one.

The first question: what are you defending against

Before comparing runtimes, write down the threat. “Untrusted” covers very different things.

• Code you did not write, running with your privileges. Student submissions, customer plugins, CI jobs from forks, notebooks, model-generated code run by an agent. The main risk is that the code reaches the host kernel through a bug, or reaches things the pod can see.
• Tenants who must not see each other. A multi-tenant SaaS where each customer’s job runs on shared nodes. The risk is one tenant reading another tenant’s memory, files or traffic through the shared kernel.
• Your own code with a dangerous parser. Image conversion, PDF rendering, archive extraction, document preview. The code is trusted, the input is not.
• Data you must protect from the infrastructure owner. Regulated data where even the cloud or the node administrator should not be able to read memory. That is confidential computing, a different and harder problem.

The first three are where gVisor, Kata and Firecracker help. The fourth needs hardware trusted execution environments, which Kata supports through its confidential computing classes, and which this post only touches on.

Also write down what the sandbox is not for. The gVisor performance guide makes a fair point: a sandbox gives little benefit to a trusted database, because the user data is already inside the sandbox and an attacker has no need to break out. Put sandboxes around the code you do not trust, not around everything.

What each tool is

gVisor

gVisor’s main binary is runsc, an OCI runtime that sits where runc would. Inside each sandbox, an application kernel called the Sentry implements Linux system calls in Go. When the application makes a system call, it is handled by the Sentry, which makes a smaller, filtered set of host system calls of its own. File access goes through a separate component called the Gofer, or through host file descriptors donated to the sandbox in the newer directfs mode. The Sentry itself runs under a tight seccomp filter.

How the Sentry intercepts system calls depends on the “platform”:

• systrap, the default since mid-2023, uses seccomp’s SECCOMP_RET_TRAP to send the application’s system calls to the Sentry. It works inside ordinary VMs and on machines without virtualization support.
• KVM uses hardware virtualization extensions for faster address space switches. The docs say it runs best on bare metal and that systrap often performs better inside a nested VM.
• ptrace still exists but is no longer supported and is expected to be removed.

For Kubernetes, gVisor ships a containerd shim, containerd-shim-runsc-v1, registered as runtime type io.containerd.runsc.v1. GKE Sandbox is Google’s managed version of this setup. For GPUs, gVisor has nvproxy, which passes a vetted set of NVIDIA driver ioctl calls from the sandbox to the host driver. More on that below.

Kata Containers

Kata Containers runs each Kubernetes pod inside a lightweight virtual machine. On the host there is a containerd shim (io.containerd.kata.v2) and a virtual machine monitor. Inside the VM there is a guest kernel and the Kata agent, which starts the pod’s containers. Container files reach the guest through virtio-fs, or through block devices in newer configurations. The pod sees its own kernel; the host kernel only sees a VMM process and KVM.

Since 4.0, the default runtime is runtime-rs, written in Rust. The old Go runtime is deprecated: it gets critical bug fixes and CVE fixes, but no new features. The 4.0 release notes list QEMU, Cloud Hypervisor and the built-in Dragonball VMM as hypervisors for the Rust runtime.

The kata-deploy Helm chart installs everything on the nodes and creates one RuntimeClass per configuration. The docs suggest kata-qemu-runtime-rs as the starting point for general workloads. Other classes cover Cloud Hypervisor (kata-clh-runtime-rs), Dragonball (kata-dragonball), NVIDIA GPU passthrough (kata-qemu-nvidia-gpu*), confidential computing with AMD SEV-SNP, Intel TDX or IBM Secure Execution, and “peer pods” that run outside the worker node.

Firecracker

Firecracker is a virtual machine monitor written in Rust that uses KVM. It is deliberately small. It offers virtio network and block devices, vsock, an entropy device, pmem devices, a serial console, memory hotplug and snapshots, all configured through a REST API on a Unix socket. It does not emulate a BIOS or a full PC. In production it is meant to be started by its jailer binary, which sets up a chroot, namespaces and cgroups and then drops privileges, plus Firecracker’s own per-thread seccomp filters.

The important thing for Kubernetes users: Firecracker is not a container runtime. Something has to turn a pod into a microVM. Today there are two options:

• Kata Containers with kata-fc. The kata-deploy docs list it as a Go runtime class, so it rides on the deprecated runtime, and it needs the containerd devmapper snapshotter with a thin pool that you set up yourself. Firecracker’s device model does not include virtio-fs for sharing a host directory, which is why image layers reach the guest as block devices.
• firecracker-containerd. An AWS project that lets containerd run containers inside Firecracker microVMs. It needs a custom containerd build, has no tagged releases, and lists CRI and Kubernetes compatibility as future work.

Where Firecracker shines is when you build the sandbox service yourself: a job queue, a pool of pre-booted or snapshotted microVMs, and your own API in front. That is how serverless platforms use it.

Side by side, without a table

The isolation boundary

• gVisor: the boundary is the Sentry, a separate kernel implementation in user space, plus seccomp on the Sentry. An attacker first has to break the Sentry, and then has only the limited host system calls the Sentry is allowed to make.
• Kata: the boundary is hardware virtualization. An attacker has to break the guest kernel and then the VMM or KVM. The VMM matters: QEMU has the largest device model, Cloud Hypervisor and Dragonball are smaller, and the 4.3.0 Dragonball advisory is a reminder that VMM bugs are real.
• Firecracker: the same kind of boundary as Kata, with the smallest device model of the three VMMs here, wrapped by the jailer.

What the host needs

• gVisor: Linux 5.6 or later, x86_64 or ARM64. No virtualization extensions needed with systrap, so ordinary cloud VMs work.
• Kata: CPU virtualization extensions and /dev/kvm on every node that runs Kata pods, plus the vhost_vsock and vhost_net modules. On cloud VMs this means instance types with nested virtualization, or bare metal. The docs also cover Microsoft Hypervisor hosts with /dev/mshv through Cloud Hypervisor’s mshv backend.
• Firecracker: /dev/kvm with read and write access, x86_64 or aarch64 hosts, and a host kernel from its supported list. Its own getting started guide uses EC2 .metal instances because, as it says, EC2 only offers KVM on those.

Compatibility with normal Linux software

• gVisor: implements most of the Linux system call surface, but not all of it, and some behaviour differs. This year the project added enough for systemd to run inside the sandbox. In April 2026 Tencent published a guest post on the gVisor blog saying that in more than 74,000 side-by-side comparisons with runc, about 1.7% of test cases showed compatibility issues that they then investigated and fixed. That is Tencent’s own measurement on their workloads, not a general number, but it gives a sense of the shape: most programs work, some need fixes.
• Kata: a real Linux guest kernel, so system call compatibility is close to normal. The limits are at the edges: no host networking (hostNetwork is not supported and the docs warn it may break host networking), privileged containers behave differently, and hostPath and some volume types work through file sharing or copying rather than direct mounts.
• Firecracker: a real guest kernel too, but with a minimal device set. Whatever is not virtio net, block, vsock or the few other devices simply is not there.

Performance trade-offs

• gVisor: its performance guide says raw CPU work and memory access have no added cost once mappings are in place, because the code runs natively. The costs are in system calls, which go through the Sentry, and in file system and network operations, where the guide says gVisor’s own implementations are less optimised than Linux’s. Its examples: small Redis operations and serving small static files show large overhead, while CPU-bound jobs, TensorFlow training and an ffmpeg transcode show little. The October donation post adds that some companies run kernel patches that make gVisor much faster, and that out-of-the-box performance on I/O-heavy workloads has given poor first impressions.
• Kata: the guest gets near-native CPU, but each pod pays for a guest kernel and VMM in memory, a VM boot at pod start, and virtio for disk and network. File sharing through virtio-fs adds overhead on file-heavy work; newer block-based options in 4.x avoid the shared file system. No universal numbers; measure your workload.
• Firecracker: its specification, enforced by its own CI on AWS .metal instances, says the VMM process starts in 8 CPU ms, VMM memory overhead is at most 5 MiB for a 1 vCPU, 128 MiB guest, and it takes at most 125 ms from the InstanceStart call to the guest’s /sbin/init, with a minimal kernel and root file system. These are the project’s own figures on its own test setup. They describe the VMM, not a full Kubernetes pod start with image pull and networking.

Start-up and density

• gVisor: the sandbox is a set of processes, so it starts like a container with some extra setup, and its memory grows and shrinks with use. The donation post argues this allows tighter bin packing than VM-based runtimes.
• Kata: a VM per pod. Memory for the guest is set at start, with hotplug and virtio-mem options in 4.x. Use Pod Overhead in the RuntimeClass so the scheduler knows the real cost.
• Firecracker: designed for density and fast start. Snapshots let you restore a pre-booted microVM, which is how serverless platforms hide boot time. Note the snapshot format change in 1.16.2 if you keep snapshots across upgrades.

GPUs and devices

• gVisor: nvproxy supports NVIDIA GPUs on the T4, A100, A10G, L4 and H100 list, with other GPUs of the same architectures likely to work. It allows compute and utility driver capabilities by default, supports only selected driver versions with a strict match, and exposes only /dev/nvidiactl, /dev/nvidia-uvm and /dev/nvidia#, so MIG is not supported. The docs are honest about the limit: gVisor protects against many Linux kernel bugs, but much less against bugs in the NVIDIA driver itself, because supported ioctl calls go straight to it. Keep drivers patched.
• Kata: GPU passthrough through VFIO, best supported with QEMU. The 4.0 notes list Cloud Hypervisor and Dragonball as GPU-capable but not tested in upstream CI. The NVIDIA classes need host IOMMU and VFIO setup, and there are confidential GPU variants for SEV-SNP and TDX.
• Firecracker: no GPU support. The original announcement said there is no support for graphics or accelerators of any kind and no hardware passthrough, and Kata’s hypervisor list still shows no GPU support for Firecracker.

Kubernetes integration

• gVisor: install runsc and the shim on nodes, add a containerd runtime entry with type io.containerd.runsc.v1, create a RuntimeClass with handler runsc. Simple and well documented.
• Kata: kata-deploy Helm chart does the node work and creates RuntimeClasses. You still choose which classes to enable, and the Firecracker, GPU and confidential classes need extra profiles and host setup.
• Firecracker: through Kata’s kata-fc class only, for a standard Kubernetes cluster.

Operations and upgrades

• gVisor: weekly dated releases from an apt repository. Pin the version on nodes and roll it like any node package. Watch the repository move after the CNCF donation.
• Kata: a monthly-ish release cadence, Helm upgrades, and a migration from the Go runtime to runtime-rs if you used Kata before 4.0. The guest kernel and image are part of what you upgrade.
• Firecracker: releases every two or three months, a documented support window (patches for the last two minor releases for up to a year, and any minor release for at least six months), plus guest kernel and root file system images you maintain.

How a sandboxed pod actually starts

It helps to see one full cycle with RuntimeClass, because most surprises come from a step people did not know was there.

1. Admission. The pod spec names a runtimeClassName. Admission merges the RuntimeClass’s node selector and tolerations into the pod, and adds the Pod Overhead to its resource accounting.
2. Scheduling. The scheduler places the pod only on nodes that match the selector, counting the overhead against node capacity.
3. Kubelet to CRI. The kubelet asks containerd or CRI-O to create a pod sandbox with the handler named in the RuntimeClass.
4. Shim start. containerd looks up that handler in its config and starts the matching shim: containerd-shim-runsc-v1 for gVisor, containerd-shim-kata-v2 for Kata.
5. Sandbox creation. For gVisor, runsc starts the Sentry and Gofer processes and applies seccomp filters. For Kata, the shim starts the VMM, boots the guest kernel and connects to the agent over vsock.
6. Networking. The CNI plugin sets up the pod network as usual. gVisor then runs its own network stack inside the sandbox by default. Kata connects the pod’s network interface to the VM through a tap device.
7. Containers. Image layers are made available to the sandbox, through the Gofer or directfs for gVisor, through virtio-fs or block devices for Kata, and the containers start inside.

If the handler is missing on the chosen node, step 4 fails and the pod goes to Failed. That is the most common first-day error, and it is why the node selector in the RuntimeClass is not optional on a mixed cluster.

A worked plan for the Visakhapatnam team

Step 1: shrink what the runner can see

Do this first, before touching the runtime, because it would have stopped the actual incident.

• automountServiceAccountToken: false on the runner pods, and no RBAC for the runner’s service account.
• Hidden test cases delivered to the runner by the orchestrator, for example written into an emptyDir by an init container that has the credentials, rather than fetched by the runner with credentials of its own. The student code never sees the bucket endpoint or a key.
• A default-deny NetworkPolicy in the runners namespace with no egress at all for the submission container. Our pod networking post covers which CNI plugins enforce policy.
• Resource limits on CPU and memory, a PID limit on the runner nodes through the kubelet’s podPidsLimit, and readOnlyRootFilesystem, non-root user and dropped capabilities in the security context.

Step 2: a dedicated node pool with gVisor

Create a separate node pool for runners, labelled and tainted, so that only sandboxed pods land there and nothing else lands there. Install gVisor from its apt repository on those nodes, as the install guide describes:

curl -fsSL https://gvisor.dev/archive.key | sudo gpg --dearmor -o /usr/share/keyrings/gvisor-archive-keyring.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/gvisor-archive-keyring.gpg] https://storage.googleapis.com/gvisor/releases release main" | sudo tee /etc/apt/sources.list.d/gvisor.list > /dev/null
sudo apt-get update && sudo apt-get install -y runsc

In production, pin a specific dated release rather than always taking the latest, and roll new releases through a staging pool first. On a managed service, use the provider’s node image customisation or a privileged installer DaemonSet that runs only on this pool. If your provider offers gVisor as a managed option, as GKE Sandbox does, use that instead.

Then register the runtime with containerd. For containerd 2.x the config uses version 3 and the io.containerd.cri.v1.runtime plugin:

version = 3

[plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.runsc]
  runtime_type = 'io.containerd.runsc.v1'

[plugins.'io.containerd.cri.v1.runtime'.containerd.runtimes.runsc.options]
  TypeUrl = 'io.containerd.runsc.v1.options'
  ConfigPath = '/etc/containerd/runsc.toml'

And keep the runsc options in their own file, so they are visible and reviewable:

# /etc/containerd/runsc.toml
[runsc_config]
  platform = "systrap"

systrap is already the default; writing it down means nobody wonders later. These nodes are cloud VMs, which is exactly where the gVisor docs recommend systrap over KVM. Restart containerd on the pool and check that the shim is found.

Step 3: the RuntimeClass

apiVersion: node.k8s.io/v1
kind: RuntimeClass
metadata:
  name: gvisor
handler: runsc
scheduling:
  nodeSelector:
    sandbox.example.in/runtime: gvisor
  tolerations:
    - key: sandbox.example.in/runtime
      operator: Equal
      value: gvisor
      effect: NoSchedule
overhead:
  podFixed:
    cpu: 100m
    memory: 64Mi

The overhead values here are placeholders. Measure the real per-pod cost on your own nodes with your own runner image, then set them. Without overhead the scheduler packs more pods per node than the memory can hold. The Kubernetes docs also recommend that only cluster administrators can create or change RuntimeClasses; keep it that way.

Then set runtimeClassName: gvisor in the runner pod template.

Step 4: make the RuntimeClass mandatory

A RuntimeClass that a pod can simply leave out is a suggestion. In the runners namespace, make it a rule. With a ValidatingAdmissionPolicy, which is built into Kubernetes, that could look like this:

apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicy
metadata:
  name: sandbox-runtime-required
spec:
  failurePolicy: Fail
  matchConstraints:
    resourceRules:
      - apiGroups: [""]
        apiVersions: ["v1"]
        operations: ["CREATE"]
        resources: ["pods"]
  validations:
    - expression: "has(object.spec.runtimeClassName) && object.spec.runtimeClassName == 'gvisor'"
      message: "Pods in sandboxed namespaces must use the gvisor RuntimeClass."
    - expression: "!has(object.metadata.annotations) || object.metadata.annotations.all(k, !k.startsWith('dev.gvisor.') && !k.startsWith('io.katacontainers.'))"
      message: "Sandbox runtime annotations are not allowed in sandboxed namespaces."
---
apiVersion: admissionregistration.k8s.io/v1
kind: ValidatingAdmissionPolicyBinding
metadata:
  name: sandbox-runtime-required
spec:
  policyName: sandbox-runtime-required
  validationActions: ["Deny"]
  matchResources:
    namespaceSelector:
      matchLabels:
        sandbox.example.in/required: "true"

The second rule is there for a reason. Both runtimes accept per-pod annotations that change sandbox behaviour, and containerd forwards only the annotations its runtime config allows. Kata’s 4.3.0 advisory was about exactly this: kata-deploy used to forward all io.katacontainers.* annotations, which let a pod author override most of the Kata configuration. Keep the containerd allowlist narrow and block these annotations from tenants at admission as well. Our admission policy post covers how to roll a policy like this out in audit mode first.

Step 5: test compatibility with real submissions

Before switching, replay a few thousand past submissions in all four languages through both runtimes and compare the outputs, exit codes and run times. Look at:

• Compile times for C++ and Java, which are file-heavy and may slow down noticeably.
• Programs that read /proc, use threads heavily, or measure time.
• Memory limits: the JVM and Python read cgroup limits, so check they behave the same.
• Whether a fork bomb now stays inside the sandbox and the pod’s limits.

Raise time limits for compile steps if needed, and tell colleges in advance if scores could change because of timing.

Step 6: what Kata or Firecracker would look like instead

If the team later moves runners to bare metal, or to an instance type with nested virtualization, Kata becomes a reasonable choice, especially if some colleges ask for full Linux behaviour, such as running a database inside a lab exam. Install the chart and enable only the class you need:

export VERSION=4.3.1
export CHART="oci://ghcr.io/kata-containers/kata-deploy-charts/kata-deploy"
cat > kata-values.yaml <<'VALUES'
shims:
  disableAll: true
  qemu-runtime-rs:
    enabled: true
VALUES
helm install kata-deploy "${CHART}" --version "${VERSION}" -n kube-system -f kata-values.yaml
kubectl get runtimeclasses

Then use runtimeClassName: kata-qemu-runtime-rs and update the admission policy. Check what node selector and overhead the chart put on the RuntimeClass with kubectl get runtimeclass kata-qemu-runtime-rs -o yaml, and run the Kata docs’ test pod: uname -r inside should print the guest kernel, not the host’s.

If the team instead decides to build its own execution service, separate from Kubernetes, with a pool of pre-booted microVMs restored from snapshots, Firecracker is the right building block. That is a bigger engineering project: guest kernel and root file system images, the jailer, networking with tap devices, snapshot management and your own scheduler. It makes sense at large scale or when start-up time is the product. It does not make sense to save a few hundred milliseconds for a lab exam.

Why seccomp, AppArmor and rootless are not the same thing

Teams often ask whether a stricter seccomp profile, AppArmor or SELinux, user namespaces or a non-root user would do the job. They help, and you should use them. But they all narrow access to the same host kernel. A seccomp profile reduces the number of system calls; the ones that remain still go to the host kernel. User namespaces change what root means inside the container, and Kubernetes support for them has been maturing for a while, but kernel bugs reachable from inside a user namespace are a known class of problems.

gVisor and Kata change who answers the system call. With gVisor, the Sentry answers. With Kata, a guest kernel answers. That is a different kind of control, and it is why it is worth the cost for code you do not trust. Use both: hardening on every pod, and a sandbox runtime for the pods that run strangers’ code.

Operational costs people forget

• Node pools. A sandbox runtime usually means a separate pool with labels, taints and its own image. For Kata, it also means instance types with nested virtualization or bare metal, which may cost more or have less choice.
• Pod Overhead. Leaving it out makes the scheduler optimistic. Measure, set it and review it after runtime upgrades.
• Compatibility testing. Every new language version, base image or runtime release can change behaviour. Keep a replay set and run it before each upgrade.
• Debugging. kubectl exec and logs still work, but tools that look at host processes, eBPF agents and some monitoring see the sandbox differently. With Kata, host-level tools see the VMM, not the processes inside. Agree with the observability team on what they will see.
• Storage behaviour. Shared volumes, inotify across containers, hostPath and subPath behave differently in sandboxes. gVisor has mount hint annotations for shared volumes; Kata has its own list of storage limitations.
• Upgrade cadence. gVisor ships weekly, Kata roughly monthly, Firecracker every two or three months, and each with its own security advisories. Somebody must read them.
• The governance moves. gVisor’s repository and maintainer changes, and Kata’s Go runtime deprecation, will mean config and URL updates during the next year.

When something else is right

• The code is yours and reviewed. For your own services, hardened runc with seccomp, a non-root user, dropped capabilities and a good network policy is usually enough. A sandbox runtime adds cost without much gain.
• You want each tenant on separate machines. Dedicated nodes or node pools per tenant, or even separate clusters, give a simpler boundary when the tenant count is small. It costs more hardware but less runtime complexity.
• The provider already does it. Managed sandboxes such as GKE Sandbox for gVisor, or serverless container services built on microVMs, give you the boundary without running it. Compare that against running your own.
• You need protection from the infrastructure owner. That is confidential computing with hardware TEEs, through Kata’s SNP, TDX or SE classes or a provider’s confidential VMs. A normal sandbox does not give it.
• The untrusted code is small and can be compiled to WebAssembly. For plugins and small functions, a WebAssembly runtime with no system access by design can be lighter than any of these. It needs the code to target WebAssembly, which student C programs usually will not.

Decision guide

• Nodes are ordinary cloud VMs without nested virtualization, workload is short CPU-bound programs, density matters: gVisor with systrap.
• Bare metal or nested virtualization available, workload needs full Linux behaviour or a kernel of its own: Kata with kata-qemu-runtime-rs.
• GPU workloads from untrusted users: gVisor with nvproxy if your GPU, driver and workload fit its supported list; Kata with the NVIDIA GPU classes for VFIO passthrough; plan driver patching either way.
• Confidential computing requirement: Kata’s TEE classes on matching hardware.
• You are building your own serverless or code execution service with snapshots and fast start as the core feature: Firecracker directly, with the jailer.
• You want Firecracker under Kubernetes pods: kata-fc, knowing it uses the deprecated Go runtime and needs devmapper; test it hard before relying on it.
• Code is trusted: stay on runc, harden it, and spend the effort on network policy and secrets instead.

A short checklist

• A written list of which workloads run code you did not write, and what each could reach today.
• No service account token, no cloud credentials and no unneeded network routes in pods that run untrusted code.
• CPU, memory and PID limits on those pods.
• A dedicated, labelled and tainted node pool with the sandbox runtime installed and pinned.
• A RuntimeClass with node selector, tolerations and measured Pod Overhead.
• An admission rule that forces the RuntimeClass and blocks runtime-specific annotations in sandboxed namespaces.
• A narrow containerd pod_annotations allowlist for each sandbox runtime.
• A compatibility replay set, run before every runtime, kernel or image upgrade.
• Someone subscribed to gVisor, Kata and Firecracker security advisories and releases.

Common mistakes

• Sandboxing the pod but mounting the keys inside it. A sandbox does not hide what you put in it. Tokens, environment secrets and open network paths are still there for the code to use.
• Creating a RuntimeClass and not enforcing it. If a pod can leave out runtimeClassName, one day it will.
• No node selector on a mixed cluster. Pods land on nodes without the handler and fail at start.
• Skipping Pod Overhead. The scheduler over-packs nodes, and the out-of-memory killer finds out first.
• Trying Kata on cloud VMs without nested virtualization. There is no /dev/kvm, and nothing starts. Check with ls -l /dev/kvm before planning.
• Expecting Firecracker to plug into Kubernetes on its own. It is a VMM. Something has to turn a pod into a microVM.
• Forwarding all runtime annotations from pods. It lets pod authors change sandbox settings. Kata’s 4.3.0 advisory is the example.
• Assuming the GPU path is as strong as the CPU path. With gVisor, supported NVIDIA ioctl calls go to the host driver. With passthrough, the device is in the guest. Driver and firmware patching is still your job.
• Measuring with a toy benchmark. Sandbox overhead depends on system calls and file operations. Test with your real workload.

About a hands-on drill

No hands-on drill was run for this post. The commands and manifests above are written from each project’s documentation and published examples, and are meant as starting points. Try them on a test cluster with test workloads, and measure overhead on your own nodes, before relying on them in production.

Unlearn, re-learn

Unlearn the idea that a container is a sandbox. It is a set of processes on a shared kernel with some fences around them, and the fences were designed for packaging and resource control before security. Unlearn, too, the idea that a sandbox runtime alone makes untrusted code safe; the Visakhapatnam incident needed no escape at all. Re-learn isolation as layers: what the code can see, what it can reach on the network, how much it can consume, and finally which kernel answers its system calls. For code you wrote, the first three layers on runc are usually enough. For code a stranger wrote, add the fourth, pick gVisor or Kata based on your nodes and workload, make it mandatory with RuntimeClass and admission policy, and test it with real workloads before every upgrade.

Sources

• gVisor repository and releases
• gVisor docs: installation, platform guide, performance guide, containerd quick start, containerd advanced configuration, GPU support
• gVisor blog: gVisor is being donated to CNCF, systemd in gVisor, Tencent’s agentic RL sandboxes
• CNCF: gVisor sandbox application
• Kata Containers repository, releases, 4.0.0 release notes and 4.3.0 release notes
• Kata docs: installation, kata-deploy RuntimeClasses, hypervisors, limitations
• Kata advisory: permissive annotation allowlist
• katacontainers.io and OpenInfra: OpenInfra joins the Linux Foundation
• Firecracker repository, releases, specification, getting started, release policy
• AWS: Firecracker announcement, 26 November 2018 and open source announcement
• firecracker-containerd repository
• Kubernetes docs: RuntimeClass and Pod Overhead
• containerd: CRI plugin configuration