# Configure LLM Providers for an Agent

Agents can be configured to use one or more LLM Service Providers registered at the organization level. The configuration process differs slightly between **Platform-hosted** and **External** agents, but both follow the same pattern: attach an org-level provider to the agent with an optional name, description, and guardrails.

## Prerequisites[​](#prerequisites "Direct link to Prerequisites")

* At least one LLM Service Provider registered at the org level (see [Register an LLM Service Provider](/agent-manager/docs/v0.11.x/tutorials/register-llm-service-provider/.md))
* An agent created in a project (Platform-hosted or External)

***

## Overview: Agent Types[​](#overview-agent-types "Direct link to Overview: Agent Types")

| Type         | Description                                                                                                                               |
| ------------ | ----------------------------------------------------------------------------------------------------------------------------------------- |
| **Platform** | Agent code is built and deployed by the platform from a GitHub repository. The platform injects LLM credentials as environment variables. |
| **External** | Agent is deployed and managed externally. The platform registers it and provides the invoke URL + API key for the LLM provider.           |

***

## Configuring LLM for a Platform-Hosted Agent[​](#configuring-llm-for-a-platform-hosted-agent "Direct link to Configuring LLM for a Platform-Hosted Agent")

### Step 1: Open the Agent[​](#step-1-open-the-agent "Direct link to Step 1: Open the Agent")

1. Navigate to your project (**Projects** → select project → **Agents**).
2. Click on a **Platform**-tagged agent.
3. In the left sidebar, click **Configure**.

### Step 2: Add an LLM Provider[​](#step-2-add-an-llm-provider "Direct link to Step 2: Add an LLM Provider")

The **Configure** page displays the **LLM Providers** section listing all LLM providers currently attached to this agent.

1. Click **+ Add Provider**.

2. Fill in the **Basic Details**:

   | Field           | Description                                          | Example                   |
   | --------------- | ---------------------------------------------------- | ------------------------- |
   | **Name**        | A logical name for this LLM binding within the agent | `OpenAI GPT5`             |
   | **Description** | Optional description                                 | `Primary reasoning model` |

3. Under **LLM Service Provider**, click **Select a Provider**.

   * A side panel opens listing all org-level LLM Service Providers with their template, rate limiting status, and guardrails.
   * Select the desired provider and close the panel.

4. Optionally, under **Guardrails**, click **+ Add Guardrail** to attach guardrails specific to this agent's use of the provider.

5. Click **Save**.

### Step 3: Use the Provider in Agent Code[​](#step-3-use-the-provider-in-agent-code "Direct link to Step 3: Use the Provider in Agent Code")

After saving, the platform generates **environment variables** that are automatically injected into the agent's deployment runtime. You can view these on the LLM provider detail page under **Environment Variables References**:

| Variable Name     | Description                                      |
| ----------------- | ------------------------------------------------ |
| `<NAME>_API_KEY`  | API Key for authenticating with the LLM provider |
| `<NAME>_BASE_URL` | Base URL of the LLM Provider API endpoint        |

Where `<NAME>` is derived from the provider name (uppercased, e.g., `OPENAI_GPT5` for a provider named `OpenAI GPT5`).

If your agent is already configured to read a different environment variable name, update the system provided variable name and click **Save**.

**Python code snippet** (shown in the UI):

```
import os
from openai import OpenAI

apikey = os.environ.get('OPENAI_GPT5_API_KEY')
url = os.environ.get('OPENAI_GPT5_URL')

client = OpenAI(
    base_url=url,
    api_key="",
    default_headers={"API-Key": apikey, "Authorization": ""}
)
```

> **Note**: The platform also provides an **AI Prompt** snippet — a ready-made prompt you can paste into an AI coding assistant to automatically update your code to use the injected environment variables.

### Step 4: Build and Deploy[​](#step-4-build-and-deploy "Direct link to Step 4: Build and Deploy")

1. After configuring the LLM provider, click **Build** in the sidebar.
2. Click **Trigger a Build** to build the agent from its GitHub source.
3. Once the build completes, click **Deploy** to deploy to the target environment.
4. The deployed agent URL appears on the **Overview** page (e.g., `http://default-default.localhost:19080/agent-name`).

***

## Configuring LLM for an External Agent[​](#configuring-llm-for-an-external-agent "Direct link to Configuring LLM for an External Agent")

### Step 1: Create and Register the Agent[​](#step-1-create-and-register-the-agent "Direct link to Step 1: Create and Register the Agent")

1. Navigate to your project (**Projects** → select project → **Agents**).

2. Click **+ Add Agent**.

3. On the **Add a New Agent** screen, select **Externally-Hosted Agent**.

   > This option is for connecting an existing agent running outside the platform to enable observability and governance.

4. Fill in the **Agent Details**:

   | Field                        | Description                               | Example                |
   | ---------------------------- | ----------------------------------------- | ---------------------- |
   | **Name**                     | A unique identifier for the agent         | `my-external-agent`    |
   | **Description** *(optional)* | Short description of what this agent does | `Customer support bot` |

5. Click **Register**.

After registration, the agent is created with status **Registered** and the **Setup Agent** panel opens automatically.

***

### Step 2: Instrument the Agent (Setup Agent)[​](#step-2-instrument-the-agent-setup-agent "Direct link to Step 2: Instrument the Agent (Setup Agent)")

The **Setup Agent** panel provides a **Zero-code Instrumentation Guide** to connect your agent to the platform for observability (traces). Select your language from the **Language** dropdown (Python or Ballerina).

#### Python[​](#python "Direct link to Python")

1. **Install the AMP instrumentation package**:

   ```
   pip install amp-instrumentation
   ```

   Provides the ability to instrument your agent and export traces.

2. **Generate API Key** — choose a **Token Duration** (default: 1 year) and click **Generate**. Copy the token immediately — it will not be shown again.

3. **Set environment variables**:

   ```
   export AMP_OTEL_ENDPOINT="http://localhost:22893/otel"
   export AMP_AGENT_API_KEY="<your-generated-token>"
   ```

   Sets the agent endpoint and agent-specific API key so traces can be exported securely.

#### Ballerina[​](#ballerina "Direct link to Ballerina")

1. **Import the Amp module** in your Ballerina program:

   ```
   import ballerinax/amp as _;
   ```

2. **Add the following to `Ballerina.toml`**:

   ```
   [build-options]
   observabilityIncluded = true
   ```

3. **Update `Config.toml`**:

   ```
   [ballerina.observe]
   tracingEnabled = true
   tracingProvider = "amp"
   ```

4. **Generate API Key** — choose a **Token Duration** and click **Generate**. Copy the token immediately.

5. **Set environment variables**:

   ```
   export BAL_CONFIG_VAR_BALLERINAX_AMP_OTELENDPOINT="http://localhost:22893/otel"
   export BAL_CONFIG_VAR_BALLERINAX_AMP_APIKEY="<your-generated-token>"
   ```

You can reopen the Setup Agent panel at any time from the agent **Overview** page by clicking **Setup Agent**.

***

### Step 3: Add an LLM Provider[​](#step-3-add-an-llm-provider "Direct link to Step 3: Add an LLM Provider")

1. In the left sidebar, click **Configure**.

2. The **Configure Agent** page shows the **LLM Providers** section (empty for a new agent).

3. Click **+ Add Provider**.

4. Fill in the **Basic Details**:

   | Field           | Description                         | Example                           |
   | --------------- | ----------------------------------- | --------------------------------- |
   | **Name**        | A logical name for this LLM binding | `openai-provider`                 |
   | **Description** | Optional description                | `Main model for customer queries` |

5. Under **LLM Service Provider**, click **Select a Provider**.

   * A side panel opens listing all org-level LLM Service Providers, showing the template (e.g., OpenAI), deployment time, rate limiting status, and guardrails.
   * Select the desired provider.

6. Optionally, under **Guardrails**, click **+ Add Guardrail** to attach content safety policies.

7. Click **Save**.

***

### Step 4: Connect Your Agent Code to the LLM[​](#step-4-connect-your-agent-code-to-the-llm "Direct link to Step 4: Connect Your Agent Code to the LLM")

Immediately after saving, the provider detail page is shown with a **Connect to your LLM Provider** section containing everything needed to call the LLM from your agent code:

| Field            | Description                                                                             |
| ---------------- | --------------------------------------------------------------------------------------- |
| **Endpoint URL** | The gateway URL for this provider — use this as the base URL in your LLM client         |
| **Header Name**  | The HTTP header to pass the API key (`API-Key`)                                         |
| **API Key**      | The generated client key — **copy it now**, it will not be shown again                  |
| **Example cURL** | A ready-to-use cURL command showing the Endpoint URL, Header Name, and API Key together |

Example cURL:

```
curl -X POST <endpoint-url> \
  --header "API-Key: <your-api-key>" \
  -d '{"your": "data"}'
```

Configure your agent's LLM client using the Endpoint URL as the base URL and pass the API Key in the `API-Key` header on every request.

Below the connection details, the page also shows:

* **LLM Service Provider**: the linked org-level provider (name, template, rate limiting and guardrails status)
* **Guardrails**: agent-level guardrails attached to this LLM binding

### Step 5: Run the Agent[​](#step-5-run-the-agent "Direct link to Step 5: Run the Agent")

Run your agent.

Example: Python agent with instrumentation

```
amp-instrument python main.py
```

***

## Managing Attached LLM Providers[​](#managing-attached-llm-providers "Direct link to Managing Attached LLM Providers")

From the **Configure Agent** page, the LLM Providers table shows all attached providers with:

* **Name**: The logical name given to this LLM binding.
* **Description**: Optional description.
* **Created**: When the binding was created.
* **Actions**: Delete icon to remove the provider from the agent.

Multiple providers can be attached to a single agent, allowing the agent code to use different LLMs for different tasks by referencing their respective environment variable names (platform agents) or endpoint URLs and API keys (external agents).

***

## Notes[​](#notes "Direct link to Notes")

* LLM provider credentials are **never exposed** to agent code directly — only the injected environment variables are available at runtime.
* For platform agents, environment variables are re-injected on each deployment; no manual secret management is required.
* For external agents, the Endpoint URL routes traffic through the AI Gateway, enabling centralized rate limiting, access control, and guardrails configured at the org level.
* The external agent API Key shown after saving is a **one-time display** — it cannot be retrieved again. If lost, delete the LLM provider binding and re-add it to generate a new key.
* The **Setup Agent** instrumentation step is for observability (traces) only and is independent of LLM configuration.
* Guardrails added at the agent-LLM binding level are applied **in addition to** any guardrails configured on the provider itself.