# gVisor Isolation Tier

Run agents in an environment under [gVisor](https://gvisor.dev/) (`runsc`) — a userspace kernel that intercepts syscalls before they reach the host kernel — for stronger isolation than the default runc sandbox, without any special hardware.

By default, every agent already runs as a sandboxed pod (hardened security context, network policy, warm pool) under **runc**, the standard container runtime that shares the host kernel. The gVisor tier adds a second layer: the agent's syscalls are served by gVisor's own userspace kernel, so a compromised or misbehaving agent never talks to the host kernel directly.

Isolation tier is an environment-level setting

You choose the tier once, when [creating an environment](/agent-manager/docs/next/administration/environment-management/.md). Every agent deployed or promoted to that environment runs under that tier. Existing runc environments are never affected by adding a gVisor node — the setup below only touches the new node.

## How It Works[​](#how-it-works "Direct link to How It Works")

1. An environment is created with isolation tier `gvisor`.
2. On deploy or promote, Agent Manager sets `runtimeClassName: gvisor` on the agent's pod spec.
3. The `gvisor` RuntimeClass's scheduling stanza automatically injects a nodeSelector and toleration, so the pod lands on your dedicated gVisor node and runs under `runsc`.

```
Environment (isolation tier: gvisor)
  └─ deploy / promote → runtimeClassName: gvisor
       └─ RuntimeClass "gvisor" scheduling → pod on the gVisor node, runsc kernel
```

The tier uses a **dedicated node**: installing `runsc` reconfigures and restarts containerd, which must never be done on a node serving live workloads. Adding a new, empty node means zero downtime for existing agents.

## Requirements[​](#requirements "Direct link to Requirements")

| Requirement              | Details                                                                          |
| ------------------------ | -------------------------------------------------------------------------------- |
| A new (empty) Linux node | Joined to the same Kubernetes cluster; dedicated to gVisor agents                |
| Architecture             | **x86\_64** (see warning below)                                                  |
| OS                       | Ubuntu 20.04+, Debian 11+, RHEL 8+, Amazon Linux 2023, or any Linux with systemd |
| Container runtime        | containerd, running under systemd                                                |
| Access                   | Root on the node; outbound access to `storage.googleapis.com`                    |

x86\_64 nodes only

Agent Manager builds agent images for **amd64**, and gVisor cannot emulate a foreign architecture (there is no QEMU-style binary emulation under `runsc`). ARM nodes — including Apple-silicon Macs, where amd64 images normally run via emulation — cannot run Agent Manager agents under gVisor, even though gVisor itself ships arm64 binaries. Use an x86\_64 node for the gVisor tier.

## Set Up a gVisor Node[​](#set-up-a-gvisor-node "Direct link to Set Up a gVisor Node")

### Step 1: Add a new node to the cluster[​](#step-1-add-a-new-node-to-the-cluster "Direct link to Step 1: Add a new node to the cluster")

Provision a new Linux node matching the requirements above and join it to your cluster using your platform's standard mechanism (node pool / node group / `kubeadm join` / k3s agent). Leave it empty — do not schedule workloads on it yet.

Taint the node as early as possible

If your platform lets you set taints at node-pool creation, apply `gvisor=true:NoSchedule` there. An untainted node starts accepting regular platform pods immediately, and you will have to drain them all off before installing the runtime (a runtime install restarts containerd, which restarts every pod on the node). If the node has already picked up workloads, `kubectl cordon` + `kubectl drain --ignore-daemonsets --delete-emptydir-data` it before Step 2.

Managed gVisor on GKE

Google Kubernetes Engine offers gVisor as a built-in option (**GKE Sandbox**): create the node pool with `--sandbox type=gvisor` and skip Step 2 entirely — GKE registers the `gvisor` RuntimeClass for you. Continue from Step 3 (label, taint, Fluent Bit toleration).

### Step 2: Install runsc on the node[​](#step-2-install-runsc-on-the-node "Direct link to Step 2: Install runsc on the node")

Run the installer **on the node** as root. It downloads the `runsc` binary and containerd shim, registers the `runsc` runtime in containerd, and restarts containerd (only containerd restarts — the kubelet is unaffected):

```
curl -fsSL https://raw.githubusercontent.com/wso2/agent-manager/amp/v0.0.0-dev/deployments/setup/install-gvisor.sh \
  | sudo bash
```

The script is idempotent — safe to re-run.

RKE2 nodes (bundled containerd)

RKE2 bundles its own containerd — there is no standalone `containerd` systemd service for the installer to configure or restart, so the standard installer's containerd wiring does not apply. After the installer has placed the binaries (or after downloading `runsc` and `containerd-shim-runsc-v1` into `/usr/local/bin` yourself), register the runtime through RKE2's containerd template instead:

```
sudo tee /var/lib/rancher/rke2/agent/etc/containerd/config.toml.tmpl > /dev/null <<'EOF'
{{ template "base" . }}

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runsc]
  runtime_type = "io.containerd.runsc.v1"
EOF
sudo systemctl restart rke2-agent
```

Rewrite, never append

The template must contain `{{ template "base" . }}` exactly once. If the file already exists, merge your runtime block into it and rewrite the whole file (`tee`/`cat >`); appending a second copy of the template line produces an invalid containerd config, and the node's kubelet will not come back after the restart. Recovery then requires SSH/console access to the node — `kubectl` cannot reach it.

The same pattern applies to k3s nodes at `/var/lib/rancher/k3s/agent/etc/containerd/config.toml.tmpl` (restart `k3s-agent`), which the local-development script below handles automatically.

### Step 3: Register the node for the gVisor tier[​](#step-3-register-the-node-for-the-gvisor-tier "Direct link to Step 3: Register the node for the gVisor tier")

From any machine with kubectl access to the cluster:

```
# 1. Apply the RuntimeClass (once per cluster)
kubectl apply -f https://raw.githubusercontent.com/wso2/agent-manager/amp/v0.0.0-dev/deployments/k8s/gvisor-runtimeclass.yaml

# 2. Label and taint the node (find the name with: kubectl get nodes)
kubectl label node <node-name> gvisor=true --overwrite
kubectl taint node <node-name> gvisor=true:NoSchedule --overwrite

# 3. Let the Fluent Bit log DaemonSet tolerate the taint (so agent logs are collected)
kubectl patch daemonset fluent-bit -n openchoreo-observability-plane --type=json \
  -p='[{"op":"add","path":"/spec/template/spec/tolerations","value":[{"operator":"Exists"}]}]'
```

The label is what the RuntimeClass schedules onto; the taint keeps regular runc pods off the node.

Verify

```
kubectl get runtimeclass gvisor
kubectl get node <node-name> --show-labels   # gvisor=true
```

### Local development (k3d)[​](#local-development-k3d "Direct link to Local development (k3d)")

On a local k3d cluster created from this repository's dev setup, one command adds a gVisor agent node, installs runsc into it, and registers everything:

```
make setup-gvisor
```

If your cluster was created by the [installation guides](/agent-manager/docs/next/getting-started/on-k3d/.md) (cluster name `amp-local`) rather than the dev Makefile, point the script at it:

```
CLUSTER_NAME=amp-local ./deployments/setup/setup-gvisor-node.sh
```

## Create a gVisor Environment[​](#create-a-gvisor-environment "Direct link to Create a gVisor Environment")

Isolation tier is set at environment creation time. Open the **Create Environment** drawer in the console (Environments → Create Environment), select **Sandboxing T2 (gVisor)** from the **Isolation Tier** dropdown, and run the generated command — it includes `ISOLATION_TIER=gvisor` automatically.

If you script environment creation directly, set the variable yourself:

```
curl -fsSL https://raw.githubusercontent.com/wso2/agent-manager/amp/v0.0.0-dev/deployments/scripts/add-environment.sh \
  -o add-environment.sh

ISOLATION_TIER=gvisor \
  ENV_NAME=secure-dev \
  DISPLAY_NAME="Secure Dev" \
  AGENT_MANAGER_TOKEN=<token> \
  bash add-environment.sh
```

The tier appears as a **Sandboxing T2** shield chip in the environment list, and as a shield icon next to the environment name on the agent's Deploy and Overview pages — hover over it to confirm *Sandboxing Tier 2 — gVisor*. Deploy or promote an agent to the environment as usual — no agent-side changes are needed.

## Verify[​](#verify "Direct link to Verify")

```
# The pod runs on the gVisor node under the gvisor RuntimeClass
kubectl get pod <agent-pod> -n <namespace> -o wide
kubectl get pod <agent-pod> -n <namespace> -o jsonpath='{.spec.runtimeClassName}'   # gvisor

# gVisor's kernel is visible from inside the pod
kubectl exec <agent-pod> -n <namespace> -- dmesg | head -3   # gVisor boot lines
```

## Troubleshooting[​](#troubleshooting "Direct link to Troubleshooting")

Agent pod stuck in Pending

The pod can't be scheduled onto the gVisor node. Check, in order:

```
kubectl get runtimeclass gvisor                       # must exist
kubectl get node <node-name> --show-labels            # must include gvisor=true
kubectl describe pod <pod> -n <ns>                    # scheduling events
```

If the RuntimeClass, label, and taint are all present, confirm the node is Ready.

Pod runs, but no traces / metrics / try-it (DNS or cross-node failures)

Telemetry and gateway traffic ride the cross-node path between the gVisor node and the rest of the cluster. gVisor's default userspace network stack (netstack) does not work on every CNI/overlay. Test it with a throwaway runsc pod on the node:

```
kubectl run nc-runsc --image=busybox:1.36 --restart=Never \
  --overrides='{"spec":{"runtimeClassName":"gvisor"}}' \
  -- sh -c 'nslookup kubernetes.default && echo OK; sleep 1'
kubectl logs nc-runsc; kubectl delete pod nc-runsc
```

If DNS or connectivity fails, re-run the installer in host-network passthrough mode — runsc keeps full syscall isolation but uses the host network stack inside the pod's network namespace:

```
# on the node
GVISOR_NETWORK_HOST=true sudo -E bash install-gvisor.sh
# local k3d
GVISOR_NETWORK_HOST=true make setup-gvisor
```

Then redeploy the agent.

No runtime logs from agents on the gVisor node

The Fluent Bit DaemonSet must run on the tainted node. Check it covers the gVisor node:

```
kubectl get pods -n openchoreo-observability-plane -o wide | grep fluent-bit
```

If no Fluent Bit pod is on the gVisor node, apply the toleration patch from Step 3.

Known gVisor limitations

* `kubectl port-forward` to a gVisor pod is not supported — reach agents through their Service/gateway endpoint (which is how agent traffic flows anyway).
* `/proc` inside the pod is synthetic; eBPF tooling is unavailable.
* Per-container usage metrics are attributed to the sandbox as a whole (reported under a `sandbox` container label), since the workload runs inside gVisor's kernel rather than a host cgroup per container.

## Next Steps[​](#next-steps "Direct link to Next Steps")

* Need even stronger, VM-level isolation? See the [Kata Containers isolation tier](/agent-manager/docs/next/administration/isolation-tiers/kata/.md).
* [Environment management](/agent-manager/docs/next/administration/environment-management/.md) — pipelines, promotion, and environment settings.