Skip to main content
Copy page
Version: v0.9.x

Register an AI Gateway

AI Gateways are organization-level infrastructure components that route LLM traffic through a controlled proxy. You can register multiple gateways (e.g., for different environments or teams), and each LLM Service Provider is exposed through a gateway's invoke URL.

Agent manager currently supports WSO2 AI Gateway (https://github.com/wso2/api-platform/tree/gateway/v0.9.0/docs/ai-gateway)

Prerequisites​

Before registering a gateway, ensure you have:

  • Admin access to the WSO2 Agent Manager Console
  • One of the following available depending on your chosen deployment method:
    • Quick Start / Docker: cURL, unzip, Docker installed and running
    • Virtual Machine: cURL, unzip, and a Docker-compatible container runtime (Docker Desktop, Rancher Desktop, Colima, or Docker Engine + Compose plugin)
    • Kubernetes: cURL, unzip, Kubernetes 1.32+, Helm 3.18+

Step 1: Navigate to AI Gateways​

  1. Log in to the WSO2 Agent Manager Console (http://localhost:3000).

  2. Go to Organization level by closing the projects section from the top navigation

  3. In the left sidebar, click AI Gateways under the INFRASTRUCTURE section.

    The AI Gateways page lists all registered gateways with their Name, Status, and Last Updated time.

    Agent manager comes with a pre-configured AI gateway which is ready to be used out-of-the-box.

Step 2: Add a New AI Gateway​

  1. Click the + Add AI Gateway button (top right).

  2. Fill in the Gateway Details form:

    FieldDescriptionExample
    NameA descriptive name for the gatewayProduction AI Gateway
    Virtual HostThe FQDN or IP address where the gateway will be reachableapi.production.example.com
    Critical production gatewayToggle to mark this gateway as critical for production deploymentsEnabled / Disabled

  3. Click Create AI Gateway.

Step 3: Configure and Start the Gateway​

After creating the gateway, you are taken to the gateway detail page. It shows:

  • Virtual Host: The internal cluster URL for the gateway runtime.
  • Environments: The environments (e.g., Default) this gateway serves.

The Get Started section provides instructions to deploy the gateway process using one of the methods below.

Quick Start (Docker)​

Prerequisites: cURL, unzip, Docker installed and running.

Step 1 – Download the Gateway

curl -sLO https://github.com/wso2/api-platform/releases/download/ai-gateway/v0.9.0/ai-gateway-v0.9.0.zip && \
unzip ai-gateway-v0.9.0.zip

Step 2 – Configure the Gateway

Generate a registration token by clicking Reconfigure on the gateway detail page. This produces a configs/keys.env file with the token and connection details.

Step 3 – Start the Gateway

cd ai-gateway-v0.9.0
docker compose --env-file configs/keys.env up

Virtual Machine​

Prerequisites: cURL, unzip, and a Docker-compatible container runtime:

  • Docker Desktop (Windows / macOS)
  • Rancher Desktop (Windows / macOS)
  • Colima (macOS)
  • Docker Engine + Compose plugin (Linux)

Verify the runtime is available:

docker --version
docker compose version

Then follow the same Download → Configure → Start steps as Quick Start above.

Kubernetes​

Prerequisites: cURL, unzip, Kubernetes 1.32+, Helm 3.18+.

Configure: Click Reconfigure to generate a gateway registration token.

Install the Helm chart:

helm install gateway oci://ghcr.io/wso2/api-platform/helm-charts/gateway --version 0.9.0 \
  --set gateway.controller.controlPlane.host="" \
  --set gateway.controller.controlPlane.port=443 \
  --set gateway.controller.controlPlane.token.value="your-gateway-token" \
  --set gateway.config.analytics.enabled=true

Replace your-gateway-token with the token generated in the Reconfigure step.

Verifying the Gateway​

Once running, the gateway appears in the AI Gateways list with status Active. The gateway detail page shows the virtual host URL, which is the base URL for all LLM provider invoke URLs routed through this gateway.

Notes​

  • A Default AI Gateway is pre-provisioned in new organizations.
  • Each gateway can serve multiple environments (e.g., Default, Production).
  • The registration token (generated via Reconfigure) is environment-specific and must be kept secret.
  • Marking a gateway as Critical production gateway helps signal its importance for operational monitoring.