# Configure LLM Providers for an Agent

Agents can be configured to use one or more LLM Service Providers registered at the organization level. The configuration process differs slightly between **Platform-hosted** and **External** agents, but both follow the same pattern: attach an org-level provider to the agent with optional guardrails.

## Prerequisites[​](#prerequisites "Direct link to Prerequisites")

* At least one LLM Service Provider registered at the org level (see [Register an LLM Service Provider](/agent-manager/docs/v0.16.x/tutorials/register-llm-service-provider/.md))
* An agent created in a project (Platform-hosted or External)

***

## Overview: Agent Types[​](#overview-agent-types "Direct link to Overview: Agent Types")

| Type         | Description                                                                                                                               |
| ------------ | ----------------------------------------------------------------------------------------------------------------------------------------- |
| **Platform** | Agent code is built and deployed by the platform from a GitHub repository. The platform injects LLM credentials as environment variables. |
| **External** | Agent is deployed and managed externally. The platform registers it and provides the gateway endpoint URL + API key for the LLM provider. |

***

## Configuring LLM for a Platform-Hosted Agent[​](#configuring-llm-for-a-platform-hosted-agent "Direct link to Configuring LLM for a Platform-Hosted Agent")

### Step 1: Open the Agent[​](#step-1-open-the-agent "Direct link to Step 1: Open the Agent")

1. Navigate to your project (**Projects** → select project → **Agents**).
2. Click on a **Platform**-tagged agent.
3. In the left sidebar, click **Configure**.

### Step 2: Add an LLM Configuration[​](#step-2-add-an-llm-configuration "Direct link to Step 2: Add an LLM Configuration")

The **Configure** page displays the **LLM Configurations** section listing all LLM configurations currently attached to this agent.

1. Click **+ Add LLM Configuration**.

2. Under **Service Provider**, click **Select a Service Provider**.

   <!-- -->

   * A side panel opens listing all org-level LLM Service Providers with their template, rate limiting status, and guardrails.
   * Select the desired provider. The panel closes automatically.

3. Optionally, under **Guardrails**, click **+ Add Guardrail** to attach guardrails specific to this agent's use of the provider.

4. Click **Save**.

> **Note**: The configuration name is auto-generated from the selected provider's template (e.g., `openai`, `anthropic`). If a name already exists, a numeric suffix is appended (e.g., `openai-2`).

> **Multiple environments**: If your organization has more than one environment, tabs appear at the top of the Service Provider section — one per environment. Select a provider for each environment tab before saving. Tabs with no provider selected are marked with a warning indicator.

### Step 3: Set Environment Variable Names[​](#step-3-set-environment-variable-names "Direct link to Step 3: Set Environment Variable Names")

After saving, the **Environment Variables & Integration Guide** panel opens automatically. It shows the two environment variables the platform will inject into the agent at runtime:

| Variable Key | Default Name (example) | Description                                      |
| ------------ | ---------------------- | ------------------------------------------------ |
| `url`        | `OPENAI_URL`           | Base URL of the LLM provider endpoint            |
| `apikey`     | `OPENAI_API_KEY`       | API key for authenticating with the LLM provider |

The names are auto-generated from the provider template (e.g., `OPENAI_URL`, `OPENAI_API_KEY` for an OpenAI provider). If your agent code already reads different variable names, edit them directly in the table and click **Save**.

> **Note**: Variable names are shared across all environments. The platform injects environment-specific values at runtime. If the agent is in a deployed state, the updated LLM configuration will be injected into the running pod automatically — no rebuild or redeploy is required.

### Step 4: Use the Provider in Agent Code[​](#step-4-use-the-provider-in-agent-code "Direct link to Step 4: Use the Provider in Agent Code")

The panel provides two ready-to-use integration options (toggle between them):

* **Python** — a code snippet using the correct SDK client for the provider template, pre-populated with the environment variable names.
* **Copilot Prompt** — a prompt you can paste into an AI coding assistant (e.g., GitHub Copilot, Claude) to automatically update your code to use the injected environment variables.

Example Python snippet (OpenAI template):

```
import os
from openai import OpenAI

url = os.environ.get('OPENAI_URL')
apikey = os.environ.get('OPENAI_API_KEY')

client = OpenAI(
    base_url=url,
    api_key="",
    default_headers={"API-Key": apikey, "Authorization": ""}
)
```

You can reopen this panel at any time from the LLM configuration detail page by clicking **Environment Variables & Integration Guide** in the top-right corner.

***

## Configuring LLM for an External Agent[​](#configuring-llm-for-an-external-agent "Direct link to Configuring LLM for an External Agent")

### Step 1: Create and Register the Agent[​](#step-1-create-and-register-the-agent "Direct link to Step 1: Create and Register the Agent")

1. Navigate to your project (**Projects** → select project → **Agents**).

2. Click **+ Add Agent**.

3. On the **Add a New Agent** screen, select **Externally-Hosted Agent**.

   > This option is for connecting an existing agent running outside the platform to enable observability and governance.

4. Fill in the **Agent Details**:

   | Field                        | Description                               | Example                |
   | ---------------------------- | ----------------------------------------- | ---------------------- |
   | **Name**                     | A unique identifier for the agent         | `my-external-agent`    |
   | **Description** *(optional)* | Short description of what this agent does | `Customer support bot` |

5. Click **Register**.

After registration, the agent is created with status **Registered** and the **Setup Agent** panel opens automatically.

***

### Step 2: Instrument the Agent (Setup Agent)[​](#step-2-instrument-the-agent-setup-agent "Direct link to Step 2: Instrument the Agent (Setup Agent)")

The **Setup Agent** panel provides a **Zero-code Instrumentation Guide** to connect your agent to the platform for observability (traces). Select your language from the **Language** dropdown (Python or Ballerina).

#### Python[​](#python "Direct link to Python")

1. **Install the AMP instrumentation package**:

   ```
   pip install amp-instrumentation
   ```

   Provides the ability to instrument your agent and export traces.

2. **Generate API Key** — choose a **Token Duration** (default: 1 year) and click **Generate**. Copy the token immediately — it will not be shown again.

3. **Set environment variables**:

   ```
   export AMP_OTEL_ENDPOINT="http://localhost:22893/otel"
   export AMP_AGENT_API_KEY="<your-generated-token>"
   ```

   Sets the agent endpoint and agent-specific API key so traces can be exported securely.

#### Ballerina[​](#ballerina "Direct link to Ballerina")

1. **Import the Amp module** in your Ballerina program:

   ```
   import ballerinax/amp as _;
   ```

2. **Add the following to `Ballerina.toml`**:

   ```
   [build-options]
   observabilityIncluded = true
   ```

3. **Update `Config.toml`**:

   ```
   [ballerina.observe]
   tracingEnabled = true
   tracingProvider = "amp"
   ```

4. **Generate API Key** — choose a **Token Duration** and click **Generate**. Copy the token immediately.

5. **Set environment variables**:

   ```
   export BAL_CONFIG_VAR_BALLERINAX_AMP_OTELENDPOINT="http://localhost:22893/otel"
   export BAL_CONFIG_VAR_BALLERINAX_AMP_APIKEY="<your-generated-token>"
   ```

You can reopen the Setup Agent panel at any time from the agent **Overview** page by clicking **Setup Agent**.

***

### Step 3: Add an LLM Configuration[​](#step-3-add-an-llm-configuration "Direct link to Step 3: Add an LLM Configuration")

1. In the left sidebar, click **Configure**.

2. The **Configure Agent** page shows the **LLM Configurations** section (empty for a new agent).

3. Click **+ Add LLM Configuration**.

4. Under **Service Provider**, click **Select a Service Provider**.

   <!-- -->

   * A side panel opens listing all org-level LLM Service Providers, showing the template (e.g., OpenAI), deployment time, rate limiting status, and guardrails.
   * Select the desired provider.

5. Optionally, under **Guardrails**, click **+ Add Guardrail** to attach content safety policies.

6. Click **Save**.

***

### Step 4: Connect Your Agent Code to the LLM[​](#step-4-connect-your-agent-code-to-the-llm "Direct link to Step 4: Connect Your Agent Code to the LLM")

Immediately after saving, the **Connect to LLM Provider** panel opens automatically with everything needed to call the LLM from your agent code:

| Field            | Description                                                                             |
| ---------------- | --------------------------------------------------------------------------------------- |
| **Endpoint URL** | The gateway URL for this provider — use this as the base URL in your LLM client         |
| **Header Name**  | The HTTP header to pass the API key (`API-Key`)                                         |
| **API Key**      | The generated client key — **copy it now**, it will not be shown again                  |
| **Example cURL** | A ready-to-use cURL command showing the Endpoint URL, Header Name, and API Key together |

Example cURL:

```
curl -X POST <endpoint-url> \
  --header "API-Key: <your-api-key>" \
  -d '{"model": "", "messages": [{"role": "user", "content": "Hi..."}]}'
```

Configure your agent's LLM client using the Endpoint URL as the base URL and pass the API Key in the `API-Key` header on every request.

You can reopen the connection details panel at any time from the LLM configuration detail page by clicking **Connect to LLM Provider** in the top-right corner. However, the API Key is only displayed once at creation time — if lost, delete the configuration and re-add it to generate a new key.

### Step 5: Run the Agent[​](#step-5-run-the-agent "Direct link to Step 5: Run the Agent")

Run your agent.

Example: Python agent with instrumentation

```
amp-instrument python main.py
```

***

## Managing LLM Configurations[​](#managing-llm-configurations "Direct link to Managing LLM Configurations")

From the **Configure Agent** page, the LLM Configurations table shows all attached configurations with:

* **Name**: The auto-generated name for this LLM binding (derived from the provider template, e.g., `openai`). Click a row to open the configuration detail page.
* **Created**: When the binding was created.
* **Actions**: Delete icon to remove the configuration from the agent.

Multiple configurations can be attached to a single agent, allowing the agent code to use different LLMs for different tasks by referencing their respective environment variable names (platform agents) or endpoint URLs and API keys (external agents).

### Changing the Provider[​](#changing-the-provider "Direct link to Changing the Provider")

From the LLM configuration detail page, click the **pencil icon** on the provider card to swap the linked org-level provider. This opens the provider selection drawer. Select a new provider and click **Save** to apply the change.

***

## Notes[​](#notes "Direct link to Notes")

* LLM provider credentials are **never exposed** to agent code directly — only the injected environment variables (platform agents) or the gateway endpoint + API key (external agents) are available at runtime.
* For platform agents, if the agent is in a deployed state, the updated LLM configuration is injected into the running pod automatically — no rebuild or redeploy is required.
* For external agents, the Endpoint URL routes traffic through the AI Gateway, enabling centralized rate limiting, access control, and guardrails configured at the org level.
* The external agent API Key shown after saving is a **one-time display** — it cannot be retrieved again. If lost, delete the LLM configuration and re-add it to generate a new key.
* The **Setup Agent** instrumentation step is for observability (traces) only and is independent of LLM configuration.
* Guardrails added at the agent-LLM binding level are applied **in addition to** any guardrails configured on the provider itself.