> ## Documentation Index
> Fetch the complete documentation index at: https://flokoa.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Connect a language model provider to your Flokoa agents

> Create ModelProvider and Model resources to connect OpenAI, Anthropic, Google, or AWS Bedrock and attach a model to your running agents.

Flokoa uses two separate Custom Resources to wire a language model into an agent: a **ModelProvider**, which holds the connection configuration and credential reference for a given provider, and a **Model**, which names a specific model and tunes its inference parameters. This separation means you can define your OpenAI credentials once and reuse them across as many models and agents as you need. Follow the steps below to go from raw API key to an agent that can call an LLM.

<Warning>
  Never hardcode API keys directly in your manifest files. Anyone with read access to your Git repository or cluster would be able to extract them. Always store credentials in Kubernetes Secrets and reference them by name as shown in this guide.
</Warning>

<Steps>
  <Step title="Create the API key secret">
    Create a Kubernetes Secret in the same namespace where your ModelProvider will live. Choose the command that matches your provider:

    ```bash theme={null}
    # OpenAI
    kubectl create secret generic openai-credentials \
      --from-literal=api-key=sk-proj-xxx

    # Anthropic
    kubectl create secret generic anthropic-credentials \
      --from-literal=api-key=sk-ant-xxx

    # Google AI
    kubectl create secret generic google-credentials \
      --from-literal=api-key=AIzaSy-xxx
    ```

    For AWS Bedrock, credentials are typically supplied through IAM Roles for Service Accounts (IRSA) rather than a static secret — see Step 2 for details.
  </Step>

  <Step title="Create a ModelProvider">
    Create a ModelProvider resource that references the secret you just created. Exactly one provider block must be present in each ModelProvider spec.

    <Tabs>
      <Tab title="OpenAI">
        ```yaml theme={null}
        apiVersion: agent.flokoa.ai/v1alpha1
        kind: ModelProvider
        metadata:
          name: openai-provider
        spec:
          apiKeySecretRef:
            name: openai-credentials
            key: api-key
          openai: {}
        ```

        Common OpenAI models: `gpt-4o`, `gpt-4o-mini`, `gpt-4-turbo`, `o1`, `o3-mini`
      </Tab>

      <Tab title="Anthropic">
        ```yaml theme={null}
        apiVersion: agent.flokoa.ai/v1alpha1
        kind: ModelProvider
        metadata:
          name: anthropic-provider
        spec:
          apiKeySecretRef:
            name: anthropic-credentials
            key: api-key
          anthropic: {}
        ```

        Common Anthropic models: `claude-sonnet-4-20250514`, `claude-opus-4-20250514`, `claude-3-5-sonnet-20241022`, `claude-3-5-haiku-20241022`
      </Tab>

      <Tab title="Google">
        ```yaml theme={null}
        apiVersion: agent.flokoa.ai/v1alpha1
        kind: ModelProvider
        metadata:
          name: google-provider
        spec:
          apiKeySecretRef:
            name: google-credentials
            key: api-key
          google: {}
        ```

        Common Google models: `gemini-2.0-flash-exp`, `gemini-1.5-pro`, `gemini-1.5-flash`

        For Vertex AI (service-account based auth), use the `google.project`, `google.location`, and `google.serviceAccountKeySecretRef` fields instead of `apiKeySecretRef`.
      </Tab>

      <Tab title="AWS Bedrock">
        ```yaml theme={null}
        apiVersion: agent.flokoa.ai/v1alpha1
        kind: ModelProvider
        metadata:
          name: bedrock-provider
        spec:
          bedrock:
            region: "us-east-1"
        ```

        For Bedrock, AWS credentials are resolved from the environment at runtime. The recommended approach on EKS is to attach an IAM role to the operator's service account via IRSA — no `apiKeySecretRef` is required in that case. You may also supply credentials through environment variables in the operator pod or via an EC2 instance profile.

        Common Bedrock models: `anthropic.claude-3-5-sonnet-20241022-v2:0`, `anthropic.claude-3-5-haiku-20241022-v1:0`, `amazon.nova-pro-v1:0`
      </Tab>
    </Tabs>

    Apply your chosen manifest:

    ```bash theme={null}
    kubectl apply -f modelprovider.yaml
    ```
  </Step>

  <Step title="Verify the ModelProvider">
    Check that the operator has validated the provider configuration and marked it as ready:

    ```bash theme={null}
    kubectl get modelproviders
    ```

    ```
    NAME                PROVIDER    READY   AGE
    openai-provider     openai      true    30s
    ```

    If `READY` shows `false`, describe the resource for detailed condition messages:

    ```bash theme={null}
    kubectl describe modelprovider openai-provider
    ```

    Common issues at this stage are a missing secret, a wrong key name in the secret, or a typo in the provider block.
  </Step>

  <Step title="Create a Model">
    A Model resource names the specific model string the LLM provider expects, sets inference parameters, and links back to the ModelProvider you just created. Save the following as `model.yaml` and adjust the values to match your provider:

    ```yaml theme={null}
    apiVersion: agent.flokoa.ai/v1alpha1
    kind: Model
    metadata:
      name: gpt-4o-default
    spec:
      model: "gpt-4o"
      providerRef:
        name: openai-provider
      parameters:
        temperature: "0.7"
        maxTokens: 4096
    ```

    The `model` field must match the exact string your provider accepts. Refer to the provider tabs in Step 2 for common model names.

    Apply the manifest:

    ```bash theme={null}
    kubectl apply -f model.yaml
    ```
  </Step>

  <Step title="Attach the model to an Agent">
    Reference the Model resource from your Agent spec using the `spec.model` field:

    ```yaml theme={null}
    apiVersion: agent.flokoa.ai/v1alpha1
    kind: Agent
    metadata:
      name: my-first-agent
    spec:
      model:
        name: gpt-4o-default
      runtime:
        type: standard
        spec:
          container:
            name: agent
            image: ghcr.io/example/simple-agent:v1.0.0
            ports:
            - containerPort: 8080
              name: http
    ```

    The operator resolves the reference chain at reconciliation time: Agent → Model → ModelProvider → Secret. Your agent container will receive the resolved model name, inference parameters, provider endpoint, and API key.

    To reference a model that lives in a different namespace, add the `namespace` field:

    ```yaml theme={null}
    spec:
      model:
        name: gpt-4o-default
        namespace: shared-models
    ```
  </Step>

  <Step title="Verify the Model is ready">
    Confirm the Model has resolved its provider reference successfully:

    ```bash theme={null}
    kubectl get models
    ```

    ```
    NAME              READY   AGE
    gpt-4o-default    true    45s
    ```

    Check the full status if needed:

    ```bash theme={null}
    kubectl describe model gpt-4o-default
    ```

    A `Ready: True` condition with reason `ProviderFound` confirms the model is correctly wired to its provider and your agents can begin using it.
  </Step>
</Steps>

<Tip>
  You can deploy ModelProvider and Model resources in a dedicated shared namespace — for example `shared-resources` and `shared-models` — and reference them from agents in any application namespace using the `namespace:` field. This prevents teams from duplicating provider configuration and ensures a single secret rotation point.
</Tip>