> ## Documentation Index
> Fetch the complete documentation index at: https://flokoa.dev/llms.txt
> Use this file to discover all available pages before exploring further.
# ModelProvider: connect Flokoa to LLM providers
> Configure API credentials and connection settings for OpenAI, Anthropic, Google, and AWS Bedrock so agents across your cluster can access LLMs.
The `ModelProvider` CRD stores the connection configuration that Flokoa uses to reach a Large Language Model provider. It separates authentication and endpoint settings from model parameters, so you can share a single provider across many `Model` resources without duplicating credentials. When you create a `ModelProvider`, the operator reads the referenced Kubernetes Secret, validates the configuration, and marks the resource ready for use by any `Model` in the cluster.
## API reference
```
apiVersion: agent.flokoa.ai/v1alpha1
kind: ModelProvider
```
***
## Supported providers
Connect to the OpenAI API for GPT-4o, o1, o3-mini, and other OpenAI models. Also compatible with any OpenAI-spec endpoint.
Connect to Anthropic's API to use Claude Sonnet, Claude Opus, and other Claude models.
Connect via API key for Google AI (Gemini) or use a service account for Vertex AI deployments.
Connect to AWS Bedrock for managed model inference. Supports IRSA for keyless authentication on EKS.
***
## Provider configuration
Create a Kubernetes Secret with your OpenAI API key, then create the `ModelProvider`:
```bash theme={null}
kubectl create secret generic openai-credentials \
--from-literal=api-key=sk-proj-xxx
```
```yaml theme={null}
apiVersion: agent.flokoa.ai/v1alpha1
kind: ModelProvider
metadata:
name: openai-provider
spec:
apiKeySecretRef:
name: openai-credentials
key: api-key
openai:
# Optional: override the default API endpoint
baseURL: "https://api.openai.com/v1"
# Optional: your OpenAI organization ID
organizationID: "org-xxx"
# Optional: request timeout in seconds
timeoutSeconds: 60
```
Omit `baseURL` to use the default OpenAI endpoint. Set it only when pointing at a compatible third-party or self-hosted endpoint.
Create a Kubernetes Secret with your Anthropic API key, then create the `ModelProvider`:
```bash theme={null}
kubectl create secret generic anthropic-credentials \
--from-literal=api-key=sk-ant-xxx
```
```yaml theme={null}
apiVersion: agent.flokoa.ai/v1alpha1
kind: ModelProvider
metadata:
name: anthropic-provider
spec:
apiKeySecretRef:
name: anthropic-credentials
key: api-key
anthropic:
# Optional: override the default Anthropic endpoint
baseURL: "https://api.anthropic.com"
# Optional: request timeout in seconds
timeoutSeconds: 60
```
Google supports two authentication modes depending on whether you are using the Google AI API (API key) or Vertex AI (service account).
**Google AI — API key:**
```bash theme={null}
kubectl create secret generic google-credentials \
--from-literal=api-key=AIzaSy-xxx
```
```yaml theme={null}
apiVersion: agent.flokoa.ai/v1alpha1
kind: ModelProvider
metadata:
name: google-provider
spec:
apiKeySecretRef:
name: google-credentials
key: api-key
google:
timeoutSeconds: 60
```
**Vertex AI — service account:**
```bash theme={null}
kubectl create secret generic gcp-sa-credentials \
--from-file=service-account.json=/path/to/key.json
```
```yaml theme={null}
apiVersion: agent.flokoa.ai/v1alpha1
kind: ModelProvider
metadata:
name: vertex-ai-provider
spec:
google:
project: "my-gcp-project"
location: "us-central1"
serviceAccountKeySecretRef:
name: gcp-sa-credentials
key: service-account.json
timeoutSeconds: 60
```
Vertex AI does not use `apiKeySecretRef`. Authentication is handled entirely through the service account key referenced in `google.serviceAccountKeySecretRef`.
AWS Bedrock does not require an API key field. AWS credentials are resolved from the environment — IAM Roles for Service Accounts (IRSA) is the recommended approach on EKS.
```yaml theme={null}
apiVersion: agent.flokoa.ai/v1alpha1
kind: ModelProvider
metadata:
name: bedrock-provider
spec:
bedrock:
region: "us-east-1"
# Optional: cross-region inference profile ARN
inferenceProfileARN: "arn:aws:bedrock:us-east-1:123456789012:inference-profile/xxx"
```
Configure AWS credentials via IRSA (recommended), EC2 instance profiles, or environment variables injected into the operator pod. Do not store AWS access keys in the `ModelProvider` spec.
***
## Advanced configuration
### Custom endpoints and Azure OpenAI
You can point the `openai` provider at any OpenAI-compatible endpoint, including Azure OpenAI deployments:
```yaml theme={null}
apiVersion: agent.flokoa.ai/v1alpha1
kind: ModelProvider
metadata:
name: azure-openai
spec:
apiKeySecretRef:
name: azure-credentials
key: api-key
openai:
baseURL: "https://your-resource.openai.azure.com/openai/deployments/your-deployment"
timeoutSeconds: 60
defaultHeaders:
api-version: "2024-02-01"
```
### Default request headers
Use `defaultHeaders` to attach arbitrary HTTP headers to every request made through this provider. This is useful for routing, tenant identification, or custom authentication schemes:
```yaml theme={null}
spec:
defaultHeaders:
X-Tenant-ID: "tenant-123"
X-Request-Source: "flokoa"
```
### TLS configuration
For custom or internal endpoints that use self-signed certificates, configure the `tls` block:
```yaml theme={null}
spec:
apiKeySecretRef:
name: api-credentials
key: api-key
openai:
baseURL: "https://internal-llm.company.local/v1"
tls:
# Provide a custom CA certificate
caSecretRef:
name: custom-ca-cert
key: ca.crt
# Also trust the system CA bundle alongside your custom CA
useSystemCAs: true
# Only set insecureSkipVerify: true in non-production environments
insecureSkipVerify: false
```
```bash theme={null}
# Create the CA certificate secret
kubectl create secret generic custom-ca-cert \
--from-file=ca.crt=/path/to/ca.crt
```
Setting `insecureSkipVerify: true` disables certificate validation entirely. Never use this in production — it exposes your API traffic to man-in-the-middle attacks.
***
## Status fields
After reconciliation the operator sets the following status fields:
```yaml theme={null}
status:
provider: openai # Resolved provider type
ready: true
conditions:
- type: Ready
status: "True"
lastTransitionTime: "2026-01-15T10:30:00Z"
reason: SecretFound
message: "Provider is configured and ready"
observedGeneration: 1
secretHash: "abc123..." # Changes when the referenced Secret is updated
```
| Field | Description |
| -------------------- | ----------------------------------------------------------------------- |
| `provider` | The detected provider type (`openai`, `anthropic`, `google`, `bedrock`) |
| `ready` | `true` when the provider is fully configured and reachable |
| `conditions` | Standard condition array; check the `Ready` condition for errors |
| `secretHash` | Hash of the referenced Secret — the operator re-reconciles on change |
| `observedGeneration` | The last spec generation reconciled by the operator |
***
## Security best practices
1. **Never commit API keys to version control** — always store them in Kubernetes Secrets.
2. **Restrict Secret access with RBAC** — grant `get` on specific Secret names only to the service accounts that need them.
3. **Rotate keys regularly** and update the Secret; the operator detects the `secretHash` change and reconciles automatically.
4. **Isolate environments with namespaces** — keep development, staging, and production providers in separate namespaces.
5. **Always verify TLS** for custom endpoints — do not use `insecureSkipVerify: true` in production.
6. **Prefer IRSA or Workload Identity** over long-lived API keys wherever your provider supports it (e.g., AWS Bedrock with IRSA, Vertex AI with Workload Identity).
7. **Enable Kubernetes audit logging** to track all Secret access events.
8. **Consider an external secrets operator** (e.g., External Secrets Operator with AWS Secrets Manager or HashiCorp Vault) for centralised secret rotation.
***
## Troubleshooting
```bash theme={null}
# Inspect status conditions
kubectl describe modelprovider
# Verify the Secret exists with the correct key
kubectl get secret openai-credentials -o jsonpath='{.data.api-key}' | base64 -d
```
Common causes:
* The Secret named in `apiKeySecretRef.name` does not exist.
* The Secret key does not match `apiKeySecretRef.key`.
* The API key stored in the Secret is invalid or has been revoked.
```bash theme={null}
# Check the current provider configuration
kubectl get modelprovider openai-provider -o yaml
# Update the Secret with a fresh key
kubectl create secret generic openai-credentials \
--from-literal=api-key=sk-proj-new-key \
--dry-run=client -o yaml | kubectl apply -f -
```
After updating the Secret the operator automatically re-reads it and re-reconciles the provider. No changes to the `ModelProvider` manifest are needed.
* Increase `timeoutSeconds` in the provider block (e.g., `timeoutSeconds: 120`).
* Check that egress NetworkPolicies allow traffic from the operator pod to the provider's HTTPS endpoint.
* Verify DNS resolves correctly from within the cluster for custom `baseURL` values.
* If you are behind a corporate HTTP proxy, ensure `HTTP_PROXY` and `NO_PROXY` are set on the operator Deployment.
Create a minimal `Model` resource that references the provider and watch whether it transitions to ready:
```yaml theme={null}
apiVersion: agent.flokoa.ai/v1alpha1
kind: Model
metadata:
name: connectivity-test
spec:
model: "gpt-4o-mini"
providerRef:
name: openai-provider
```
```bash theme={null}
kubectl get model connectivity-test -w
```
If the Model becomes ready, the provider credentials and network path are both working correctly.