Connect a language model provider to your Flokoa agents

Flokoa uses two separate Custom Resources to wire a language model into an agent: a ModelProvider, which holds the connection configuration and credential reference for a given provider, and a Model, which names a specific model and tunes its inference parameters. This separation means you can define your OpenAI credentials once and reuse them across as many models and agents as you need. Follow the steps below to go from raw API key to an agent that can call an LLM.

Never hardcode API keys directly in your manifest files. Anyone with read access to your Git repository or cluster would be able to extract them. Always store credentials in Kubernetes Secrets and reference them by name as shown in this guide.

Create the API key secret

Create a Kubernetes Secret in the same namespace where your ModelProvider will live. Choose the command that matches your provider:

# OpenAI
kubectl create secret generic openai-credentials \
  --from-literal=api-key=sk-proj-xxx

# Anthropic
kubectl create secret generic anthropic-credentials \
  --from-literal=api-key=sk-ant-xxx

# Google AI
kubectl create secret generic google-credentials \
  --from-literal=api-key=AIzaSy-xxx

For AWS Bedrock, credentials are typically supplied through IAM Roles for Service Accounts (IRSA) rather than a static secret — see Step 2 for details.

Create a ModelProvider

Create a ModelProvider resource that references the secret you just created. Exactly one provider block must be present in each ModelProvider spec.

OpenAI
Anthropic
Google
AWS Bedrock

apiVersion: agent.flokoa.ai/v1alpha1
kind: ModelProvider
metadata:
  name: openai-provider
spec:
  apiKeySecretRef:
    name: openai-credentials
    key: api-key
  openai: {}

Common OpenAI models: gpt-4o, gpt-4o-mini, gpt-4-turbo, o1, o3-mini

apiVersion: agent.flokoa.ai/v1alpha1
kind: ModelProvider
metadata:
  name: anthropic-provider
spec:
  apiKeySecretRef:
    name: anthropic-credentials
    key: api-key
  anthropic: {}

Common Anthropic models: claude-sonnet-4-20250514, claude-opus-4-20250514, claude-3-5-sonnet-20241022, claude-3-5-haiku-20241022

apiVersion: agent.flokoa.ai/v1alpha1
kind: ModelProvider
metadata:
  name: google-provider
spec:
  apiKeySecretRef:
    name: google-credentials
    key: api-key
  google: {}

Common Google models: gemini-2.0-flash-exp, gemini-1.5-pro, gemini-1.5-flashFor Vertex AI (service-account based auth), use the google.project, google.location, and google.serviceAccountKeySecretRef fields instead of apiKeySecretRef.

apiVersion: agent.flokoa.ai/v1alpha1
kind: ModelProvider
metadata:
  name: bedrock-provider
spec:
  bedrock:
    region: "us-east-1"

For Bedrock, AWS credentials are resolved from the environment at runtime. The recommended approach on EKS is to attach an IAM role to the operator’s service account via IRSA — no apiKeySecretRef is required in that case. You may also supply credentials through environment variables in the operator pod or via an EC2 instance profile.Common Bedrock models: anthropic.claude-3-5-sonnet-20241022-v2:0, anthropic.claude-3-5-haiku-20241022-v1:0, amazon.nova-pro-v1:0

Apply your chosen manifest:

kubectl apply -f modelprovider.yaml

Verify the ModelProvider

Check that the operator has validated the provider configuration and marked it as ready:

kubectl get modelproviders

NAME                PROVIDER    READY   AGE
openai-provider     openai      true    30s

If READY shows false, describe the resource for detailed condition messages:

kubectl describe modelprovider openai-provider

Common issues at this stage are a missing secret, a wrong key name in the secret, or a typo in the provider block.

Create a Model

A Model resource names the specific model string the LLM provider expects, sets inference parameters, and links back to the ModelProvider you just created. Save the following as model.yaml and adjust the values to match your provider:

apiVersion: agent.flokoa.ai/v1alpha1
kind: Model
metadata:
  name: gpt-4o-default
spec:
  model: "gpt-4o"
  providerRef:
    name: openai-provider
  parameters:
    temperature: "0.7"
    maxTokens: 4096

The model field must match the exact string your provider accepts. Refer to the provider tabs in Step 2 for common model names.Apply the manifest:

kubectl apply -f model.yaml

Attach the model to an Agent

Reference the Model resource from your Agent spec using the spec.model field:

apiVersion: agent.flokoa.ai/v1alpha1
kind: Agent
metadata:
  name: my-first-agent
spec:
  model:
    name: gpt-4o-default
  runtime:
    type: standard
    spec:
      container:
        name: agent
        image: ghcr.io/example/simple-agent:v1.0.0
        ports:
        - containerPort: 8080
          name: http

The operator resolves the reference chain at reconciliation time: Agent → Model → ModelProvider → Secret. Your agent container will receive the resolved model name, inference parameters, provider endpoint, and API key.To reference a model that lives in a different namespace, add the namespace field:

spec:
  model:
    name: gpt-4o-default
    namespace: shared-models

Verify the Model is ready

Confirm the Model has resolved its provider reference successfully:

kubectl get models

NAME              READY   AGE
gpt-4o-default    true    45s

Check the full status if needed:

kubectl describe model gpt-4o-default

A Ready: True condition with reason ProviderFound confirms the model is correctly wired to its provider and your agents can begin using it.

You can deploy ModelProvider and Model resources in a dedicated shared namespace — for example shared-resources and shared-models — and reference them from agents in any application namespace using the namespace: field. This prevents teams from duplicating provider configuration and ensures a single secret rotation point.