ModelProvider: connect Flokoa to LLM providers

The ModelProvider CRD stores the connection configuration that Flokoa uses to reach a Large Language Model provider. It separates authentication and endpoint settings from model parameters, so you can share a single provider across many Model resources without duplicating credentials. When you create a ModelProvider, the operator reads the referenced Kubernetes Secret, validates the configuration, and marks the resource ready for use by any Model in the cluster.

API reference

apiVersion: agent.flokoa.ai/v1alpha1
kind: ModelProvider

Supported providers

OpenAI

Connect to the OpenAI API for GPT-4o, o1, o3-mini, and other OpenAI models. Also compatible with any OpenAI-spec endpoint.

Anthropic

Connect to Anthropic’s API to use Claude Sonnet, Claude Opus, and other Claude models.

Google

Connect via API key for Google AI (Gemini) or use a service account for Vertex AI deployments.

AWS Bedrock

Connect to AWS Bedrock for managed model inference. Supports IRSA for keyless authentication on EKS.

Provider configuration

OpenAI
Anthropic
Google
AWS Bedrock

Create a Kubernetes Secret with your OpenAI API key, then create the ModelProvider:

kubectl create secret generic openai-credentials \
  --from-literal=api-key=sk-proj-xxx

apiVersion: agent.flokoa.ai/v1alpha1
kind: ModelProvider
metadata:
  name: openai-provider
spec:
  apiKeySecretRef:
    name: openai-credentials
    key: api-key

  openai:
    # Optional: override the default API endpoint
    baseURL: "https://api.openai.com/v1"

    # Optional: your OpenAI organization ID
    organizationID: "org-xxx"

    # Optional: request timeout in seconds
    timeoutSeconds: 60

Omit baseURL to use the default OpenAI endpoint. Set it only when pointing at a compatible third-party or self-hosted endpoint.

Create a Kubernetes Secret with your Anthropic API key, then create the ModelProvider:

kubectl create secret generic anthropic-credentials \
  --from-literal=api-key=sk-ant-xxx

apiVersion: agent.flokoa.ai/v1alpha1
kind: ModelProvider
metadata:
  name: anthropic-provider
spec:
  apiKeySecretRef:
    name: anthropic-credentials
    key: api-key

  anthropic:
    # Optional: override the default Anthropic endpoint
    baseURL: "https://api.anthropic.com"

    # Optional: request timeout in seconds
    timeoutSeconds: 60

Google supports two authentication modes depending on whether you are using the Google AI API (API key) or Vertex AI (service account).Google AI — API key:

kubectl create secret generic google-credentials \
  --from-literal=api-key=AIzaSy-xxx

apiVersion: agent.flokoa.ai/v1alpha1
kind: ModelProvider
metadata:
  name: google-provider
spec:
  apiKeySecretRef:
    name: google-credentials
    key: api-key

  google:
    timeoutSeconds: 60

Vertex AI — service account:

kubectl create secret generic gcp-sa-credentials \
  --from-file=service-account.json=/path/to/key.json

apiVersion: agent.flokoa.ai/v1alpha1
kind: ModelProvider
metadata:
  name: vertex-ai-provider
spec:
  google:
    project: "my-gcp-project"
    location: "us-central1"

    serviceAccountKeySecretRef:
      name: gcp-sa-credentials
      key: service-account.json

    timeoutSeconds: 60

Vertex AI does not use apiKeySecretRef. Authentication is handled entirely through the service account key referenced in google.serviceAccountKeySecretRef.

AWS Bedrock does not require an API key field. AWS credentials are resolved from the environment — IAM Roles for Service Accounts (IRSA) is the recommended approach on EKS.

apiVersion: agent.flokoa.ai/v1alpha1
kind: ModelProvider
metadata:
  name: bedrock-provider
spec:
  bedrock:
    region: "us-east-1"

    # Optional: cross-region inference profile ARN
    inferenceProfileARN: "arn:aws:bedrock:us-east-1:123456789012:inference-profile/xxx"

Configure AWS credentials via IRSA (recommended), EC2 instance profiles, or environment variables injected into the operator pod. Do not store AWS access keys in the ModelProvider spec.

Advanced configuration

Custom endpoints and Azure OpenAI

You can point the openai provider at any OpenAI-compatible endpoint, including Azure OpenAI deployments:

apiVersion: agent.flokoa.ai/v1alpha1
kind: ModelProvider
metadata:
  name: azure-openai
spec:
  apiKeySecretRef:
    name: azure-credentials
    key: api-key

  openai:
    baseURL: "https://your-resource.openai.azure.com/openai/deployments/your-deployment"
    timeoutSeconds: 60

  defaultHeaders:
    api-version: "2024-02-01"

Default request headers

Use defaultHeaders to attach arbitrary HTTP headers to every request made through this provider. This is useful for routing, tenant identification, or custom authentication schemes:

spec:
  defaultHeaders:
    X-Tenant-ID: "tenant-123"
    X-Request-Source: "flokoa"

TLS configuration

For custom or internal endpoints that use self-signed certificates, configure the tls block:

spec:
  apiKeySecretRef:
    name: api-credentials
    key: api-key

  openai:
    baseURL: "https://internal-llm.company.local/v1"

  tls:
    # Provide a custom CA certificate
    caSecretRef:
      name: custom-ca-cert
      key: ca.crt

    # Also trust the system CA bundle alongside your custom CA
    useSystemCAs: true

    # Only set insecureSkipVerify: true in non-production environments
    insecureSkipVerify: false

# Create the CA certificate secret
kubectl create secret generic custom-ca-cert \
  --from-file=ca.crt=/path/to/ca.crt

Setting insecureSkipVerify: true disables certificate validation entirely. Never use this in production — it exposes your API traffic to man-in-the-middle attacks.

Status fields

After reconciliation the operator sets the following status fields:

status:
  provider: openai       # Resolved provider type
  ready: true

  conditions:
    - type: Ready
      status: "True"
      lastTransitionTime: "2026-01-15T10:30:00Z"
      reason: SecretFound
      message: "Provider is configured and ready"

  observedGeneration: 1
  secretHash: "abc123..."  # Changes when the referenced Secret is updated

Field	Description
`provider`	The detected provider type (`openai`, `anthropic`, `google`, `bedrock`)
`ready`	`true` when the provider is fully configured and reachable
`conditions`	Standard condition array; check the `Ready` condition for errors
`secretHash`	Hash of the referenced Secret — the operator re-reconciles on change
`observedGeneration`	The last spec generation reconciled by the operator

Security best practices

Never commit API keys to version control — always store them in Kubernetes Secrets.
Restrict Secret access with RBAC — grant get on specific Secret names only to the service accounts that need them.
Rotate keys regularly and update the Secret; the operator detects the secretHash change and reconciles automatically.
Isolate environments with namespaces — keep development, staging, and production providers in separate namespaces.
Always verify TLS for custom endpoints — do not use insecureSkipVerify: true in production.
Prefer IRSA or Workload Identity over long-lived API keys wherever your provider supports it (e.g., AWS Bedrock with IRSA, Vertex AI with Workload Identity).
Enable Kubernetes audit logging to track all Secret access events.
Consider an external secrets operator (e.g., External Secrets Operator with AWS Secrets Manager or HashiCorp Vault) for centralised secret rotation.

Troubleshooting

ModelProvider is not becoming Ready

# Inspect status conditions
kubectl describe modelprovider <name>

# Verify the Secret exists with the correct key
kubectl get secret openai-credentials -o jsonpath='{.data.api-key}' | base64 -d

Common causes:

The Secret named in apiKeySecretRef.name does not exist.
The Secret key does not match apiKeySecretRef.key.
The API key stored in the Secret is invalid or has been revoked.

Authentication errors when calling the LLM

# Check the current provider configuration
kubectl get modelprovider openai-provider -o yaml

# Update the Secret with a fresh key
kubectl create secret generic openai-credentials \
  --from-literal=api-key=sk-proj-new-key \
  --dry-run=client -o yaml | kubectl apply -f -

After updating the Secret the operator automatically re-reads it and re-reconciles the provider. No changes to the ModelProvider manifest are needed.

Connection timeouts to the provider API

Increase timeoutSeconds in the provider block (e.g., timeoutSeconds: 120).
Check that egress NetworkPolicies allow traffic from the operator pod to the provider’s HTTPS endpoint.
Verify DNS resolves correctly from within the cluster for custom baseURL values.
If you are behind a corporate HTTP proxy, ensure HTTP_PROXY and NO_PROXY are set on the operator Deployment.

Verifying connectivity end-to-end

Create a minimal Model resource that references the provider and watch whether it transitions to ready:

apiVersion: agent.flokoa.ai/v1alpha1
kind: Model
metadata:
  name: connectivity-test
spec:
  model: "gpt-4o-mini"
  providerRef:
    name: openai-provider

kubectl get model connectivity-test -w

If the Model becomes ready, the provider credentials and network path are both working correctly.

Get Started

Core Resources

Guides

Python SDK

Operations

ModelProvider: connect Flokoa to LLM providers

API reference

Supported providers

OpenAI

Anthropic

Google

AWS Bedrock

Provider configuration

Advanced configuration

Custom endpoints and Azure OpenAI

Default request headers

TLS configuration

Status fields

Security best practices

Troubleshooting

​API reference

​Supported providers

OpenAI

Anthropic

Google

AWS Bedrock

​Provider configuration

​Advanced configuration

​Custom endpoints and Azure OpenAI

​Default request headers

​TLS configuration

​Status fields

​Security best practices

​Troubleshooting

API reference

Supported providers

Provider configuration

Advanced configuration

Custom endpoints and Azure OpenAI

Default request headers

TLS configuration

Status fields

Security best practices

Troubleshooting