gVisor Isolation Tier

Run agents in an environment under gVisor (runsc) — a userspace kernel that intercepts syscalls before they reach the host kernel — for stronger isolation than the default runc sandbox, without any special hardware.

By default, every agent already runs as a sandboxed pod (hardened security context, network policy, warm pool) under runc, the standard container runtime that shares the host kernel. The gVisor tier adds a second layer: the agent's syscalls are served by gVisor's own userspace kernel, so a compromised or misbehaving agent never talks to the host kernel directly.

Isolation tier is an environment-level setting

You choose the tier once, when creating an environment. Every agent deployed or promoted to that environment runs under that tier. Existing runc environments are never affected by adding a gVisor node — the setup below only touches the new node.

How It Works

1. An environment is created with isolation tier gvisor.
2. On deploy or promote, Agent Manager sets runtimeClassName: gvisor on the agent's pod spec.
3. The gvisor RuntimeClass's scheduling stanza automatically injects a nodeSelector and toleration, so the pod lands on your dedicated gVisor node and runs under runsc.

Environment (isolation tier: gvisor)
  └─ deploy / promote → runtimeClassName: gvisor
       └─ RuntimeClass "gvisor" scheduling → pod on the gVisor node, runsc kernel

The tier uses a dedicated node: installing runsc reconfigures and restarts containerd, which must never be done on a node serving live workloads. Adding a new, empty node means zero downtime for existing agents.

Requirements

Requirement	Details
A new (empty) Linux node	Joined to the same Kubernetes cluster; dedicated to gVisor agents
Architecture	x86_64 (see warning below)
OS	Ubuntu 20.04+, Debian 11+, RHEL 8+, Amazon Linux 2023, or any Linux with systemd
Container runtime	containerd, running under systemd
Access	Root on the node; outbound access to storage.googleapis.com

x86_64 nodes only

Agent Manager builds agent images for amd64, and gVisor cannot emulate a foreign architecture (there is no QEMU-style binary emulation under runsc). ARM nodes — including Apple-silicon Macs, where amd64 images normally run via emulation — cannot run Agent Manager agents under gVisor, even though gVisor itself ships arm64 binaries. Use an x86_64 node for the gVisor tier.

Set Up a gVisor Node

Step 1: Add a new node to the cluster

Provision a new Linux node matching the requirements above and join it to your cluster using your platform's standard mechanism (node pool / node group / kubeadm join / k3s agent). Leave it empty — do not schedule workloads on it yet.

Taint the node as early as possible

If your platform lets you set taints at node-pool creation, apply gvisor=true:NoSchedule there. An untainted node starts accepting regular platform pods immediately, and you will have to drain them all off before installing the runtime (a runtime install restarts containerd, which restarts every pod on the node). If the node has already picked up workloads, kubectl cordon + kubectl drain --ignore-daemonsets --delete-emptydir-data it before Step 2.

Managed gVisor on GKE

Google Kubernetes Engine offers gVisor as a built-in option (GKE Sandbox): create the node pool with --sandbox type=gvisor and skip Step 2 entirely — GKE registers the gvisor RuntimeClass for you. Continue from Step 3 (label, taint, Fluent Bit toleration).

Step 2: Install runsc on the node

Run the installer on the node as root. It downloads the runsc binary and containerd shim, registers the runsc runtime in containerd, and restarts containerd (only containerd restarts — the kubelet is unaffected):

curl -fsSL https://raw.githubusercontent.com/wso2/agent-manager/amp/v0.0.0-dev/deployments/setup/install-gvisor.sh \
  | sudo bash

The script is idempotent — safe to re-run.

RKE2 nodes (bundled containerd)

RKE2 bundles its own containerd — there is no standalone containerd systemd service for the installer to configure or restart, so the standard installer's containerd wiring does not apply. After the installer has placed the binaries (or after downloading runsc and containerd-shim-runsc-v1 into /usr/local/bin yourself), register the runtime through RKE2's containerd template instead:

sudo tee /var/lib/rancher/rke2/agent/etc/containerd/config.toml.tmpl > /dev/null <<'EOF'
{{ template "base" . }}

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runsc]
  runtime_type = "io.containerd.runsc.v1"
EOF
sudo systemctl restart rke2-agent

Rewrite, never append

The template must contain {{ template "base" . }} exactly once. If the file already exists, merge your runtime block into it and rewrite the whole file (tee/cat >); appending a second copy of the template line produces an invalid containerd config, and the node's kubelet will not come back after the restart. Recovery then requires SSH/console access to the node — kubectl cannot reach it.

The same pattern applies to k3s nodes at /var/lib/rancher/k3s/agent/etc/containerd/config.toml.tmpl (restart k3s-agent), which the local-development script below handles automatically.

Step 3: Register the node for the gVisor tier

From any machine with kubectl access to the cluster:

# 1. Apply the RuntimeClass (once per cluster)
kubectl apply -f https://raw.githubusercontent.com/wso2/agent-manager/amp/v0.0.0-dev/deployments/k8s/gvisor-runtimeclass.yaml

# 2. Label and taint the node (find the name with: kubectl get nodes)
kubectl label node <node-name> gvisor=true --overwrite
kubectl taint node <node-name> gvisor=true:NoSchedule --overwrite

# 3. Let the Fluent Bit log DaemonSet tolerate the taint (so agent logs are collected)
kubectl patch daemonset fluent-bit -n openchoreo-observability-plane --type=json \
  -p='[{"op":"add","path":"/spec/template/spec/tolerations","value":[{"operator":"Exists"}]}]'

The label is what the RuntimeClass schedules onto; the taint keeps regular runc pods off the node.

Verify

kubectl get runtimeclass gvisor
kubectl get node <node-name> --show-labels   # gvisor=true

Local development (k3d)

On a local k3d cluster created from this repository's dev setup, one command adds a gVisor agent node, installs runsc into it, and registers everything:

make setup-gvisor

If your cluster was created by the installation guides (cluster name amp-local) rather than the dev Makefile, point the script at it:

CLUSTER_NAME=amp-local ./deployments/setup/setup-gvisor-node.sh

Create a gVisor Environment

Isolation tier is set at environment creation time. Open the Create Environment drawer in the console (Environments → Create Environment), select Sandboxing T2 (gVisor) from the Isolation Tier dropdown, and run the generated command — it includes ISOLATION_TIER=gvisor automatically.

If you script environment creation directly, set the variable yourself:

curl -fsSL https://raw.githubusercontent.com/wso2/agent-manager/amp/v0.0.0-dev/deployments/scripts/add-environment.sh \
  -o add-environment.sh

ISOLATION_TIER=gvisor \
  ENV_NAME=secure-dev \
  DISPLAY_NAME="Secure Dev" \
  AGENT_MANAGER_TOKEN=<token> \
  bash add-environment.sh

The tier appears as a Sandboxing T2 shield chip in the environment list, and as a shield icon next to the environment name on the agent's Deploy and Overview pages — hover over it to confirm Sandboxing Tier 2 — gVisor. Deploy or promote an agent to the environment as usual — no agent-side changes are needed.

Verify

# The pod runs on the gVisor node under the gvisor RuntimeClass
kubectl get pod <agent-pod> -n <namespace> -o wide
kubectl get pod <agent-pod> -n <namespace> -o jsonpath='{.spec.runtimeClassName}'   # gvisor

# gVisor's kernel is visible from inside the pod
kubectl exec <agent-pod> -n <namespace> -- dmesg | head -3   # gVisor boot lines

Troubleshooting

Agent pod stuck in Pending

The pod can't be scheduled onto the gVisor node. Check, in order:

kubectl get runtimeclass gvisor                       # must exist
kubectl get node <node-name> --show-labels            # must include gvisor=true
kubectl describe pod <pod> -n <ns>                    # scheduling events

If the RuntimeClass, label, and taint are all present, confirm the node is Ready.

Pod runs, but no traces / metrics / try-it (DNS or cross-node failures)

Telemetry and gateway traffic ride the cross-node path between the gVisor node and the rest of the cluster. gVisor's default userspace network stack (netstack) does not work on every CNI/overlay. Test it with a throwaway runsc pod on the node:

kubectl run nc-runsc --image=busybox:1.36 --restart=Never \
  --overrides='{"spec":{"runtimeClassName":"gvisor"}}' \
  -- sh -c 'nslookup kubernetes.default && echo OK; sleep 1'
kubectl logs nc-runsc; kubectl delete pod nc-runsc

If DNS or connectivity fails, re-run the installer in host-network passthrough mode — runsc keeps full syscall isolation but uses the host network stack inside the pod's network namespace:

# on the node
GVISOR_NETWORK_HOST=true sudo -E bash install-gvisor.sh
# local k3d
GVISOR_NETWORK_HOST=true make setup-gvisor

Then redeploy the agent.

No runtime logs from agents on the gVisor node

The Fluent Bit DaemonSet must run on the tainted node. Check it covers the gVisor node:

kubectl get pods -n openchoreo-observability-plane -o wide | grep fluent-bit

If no Fluent Bit pod is on the gVisor node, apply the toleration patch from Step 3.

Known gVisor limitations

• kubectl port-forward to a gVisor pod is not supported — reach agents through their Service/gateway endpoint (which is how agent traffic flows anyway).
• /proc inside the pod is synthetic; eBPF tooling is unavailable.
• Per-container usage metrics are attributed to the sandbox as a whole (reported under a sandbox container label), since the workload runs inside gVisor's kernel rather than a host cgroup per container.

Next Steps

• Need even stronger, VM-level isolation? See the Kata Containers isolation tier.
• Environment management — pipelines, promotion, and environment settings.