Securing AI Agent Infrastructure: Authentication, Authorization, and Secret Management for Production Systems

Every AI agent system in production is a distributed system. An agent runtime sends queries to an embedding service, searches a vector database, invokes a reranker, and delivers context to an LLM — each interaction crosses a network boundary between distinct services. When the architecture spans these multiple services, the security posture of the system is defined not by the strongest link, but by the weakest authentication mechanism among them.

The security landscape for AI agent infrastructure has matured rapidly. Two years ago, the dominant pattern was a single API key shared across all services, stored in a .env file. Today, production systems face a more complex threat model: stolen embedding provider keys causing abuse and attacker-supplied data submission, compromised embedding workers reading source storage, unauthorized agent contexts querying sensitive document collections, leaked vector database credentials exposing corpus data, and supply-chain attacks through unauthenticated model registries. The network architecture for AI agent deployments, covered in our previous article, provides the segmentation foundation — but segmentation without authentication is just organized chaos.

This guide covers the security controls that turn a connected AI infrastructure into a secure one: service-to-service authentication patterns (mTLS, SPIFFE, and API keys), authorization models for vector databases and embedding endpoints, secret management for agent API keys and LLM tokens, network policy enforcement for AI workloads, audit logging for agent actions, and operational runbooks for the failure modes specific to AI infrastructure security.

The Threat Model for AI Agent Infrastructure

Before selecting security controls, it is essential to understand what you are protecting against. The AI agent threat model differs from traditional web application security in several important ways:

Credential exposure across embedding and vector paths. A stolen provider embedding API key lets an attacker spend against your account and submit attacker-controlled text to the provider, but it does not read existing documents or return vector store contents by itself. Document exfiltration becomes possible when the compromised component is an embedding worker or internal embedding proxy that can read source storage, or when the attacker obtains a vector database credential that can query or export the corpus directly.

Vector database data poisoning. An unauthorized write to a vector database can inject malicious documents into the retrieval corpus. When those documents are retrieved by an agent acting on a user query, the poisoned context can manipulate the LLM's response — a vector-level prompt injection that bypasses traditional input sanitization.

LLM token theft. LLM API keys are high-value targets because they provide direct access to model inference at your expense. Major providers enforce rate limits, and some also support organization or workspace spend limits, but a stolen key can still generate costs quickly within those limits. Treat provider-side budgets, per-project caps, usage alerts, and rapid key rotation as part of the credential control plane.

Side-channel data leakage through timing. In multi-tenant vector database deployments, the presence or absence of documents in specific collections can be inferred through query timing — an attacker can probe whether a sensitive document exists by measuring vector search latency with and without the document in the collection.

The AI agent threat model is defined by the same principle as any distributed system: trust is transitive across every authenticated hop. A stolen provider key, a compromised worker with storage access, and a leaked vector database credential have different blast radii and need different controls.

Service-to-Service Authentication

Mutual TLS (mTLS)

Mutual TLS is the strongest authentication mechanism for service-to-service communication in AI agent infrastructure. Unlike API keys (which authenticate at the application layer after a TCP connection is established) or JWT tokens (which can be leaked in logs or intercepted), mTLS authenticates during the TLS handshake before any application data is exchanged. An unauthenticated client can still open TCP to a reachable endpoint, but it should not complete the TLS/mTLS handshake or obtain an authenticated application channel.

The three mTLS deployment models for AI agent systems:

• Service mesh or dataplane identity (Istio, Linkerd, Cilium): The recommended approach for Kubernetes-based AI deployments. Istio and Linkerd commonly enforce mTLS through sidecars or mesh dataplanes that handle certificate issuance, rotation, and peer authentication transparently to the application. Cilium is sidecar-free and eBPF-based, with Envoy used for L7 functions where configured; its mutual authentication feature has beta limitations, so validate the exact semantics before relying on it as the only enforcement layer. No application code changes required when the mesh or dataplane covers the selected traffic path.
• Application-level TLS or mTLS: For non-Kubernetes deployments or when a service mesh is not feasible, configure TLS directly in the client and server. Qdrant supports TLS for REST and gRPC; for client-certificate validation, use Qdrant's HTTPS client certificate validation where it fits your SDK path, or terminate and enforce mTLS in a trusted proxy or service mesh in front of Qdrant.
• SPIFFE/SPIRE identity federation: For heterogeneous deployments spanning Kubernetes, VMs, and bare metal, SPIFFE provides a standardized identity document (X.509 SVID) that works across all environments. The agent runtime gets a SPIFFE identity like spiffe://nsi.io/agent/runtime/production, and the vector database enforces that only identities matching the pattern spiffe://nsi.io/agent/runtime/* can query the document collection.

Qdrant server configuration (YAML):

# Qdrant TLS and client-certificate validation
# Qdrant accepts TLS for REST and gRPC. Client-certificate validation
# is configured on the server for HTTPS clients; many Kubernetes
# deployments instead terminate and enforce mTLS in a service mesh.
service:
  enable_tls: true
  # Verify HTTPS client certificates against tls.ca_cert.
  # Keep Qdrant on a private interface and test client behavior before
  # relying on this for every SDK and protocol path.
  verify_https_client_certificate: true

tls:
  cert: /etc/qdrant/tls/server.crt
  key: /etc/qdrant/tls/server.key
  ca_cert: /etc/qdrant/tls/ca.crt

Qdrant application client behind the mesh or trusted proxy (Python):

# Istio/Linkerd sidecars or the selected mesh dataplane enforce workload
# identity, certificate rotation, and peer authorization. Cilium uses an
# eBPF datapath and per-node/integrated Envoy where configured, not a
# per-pod sidecar model.
import os
from qdrant_client import QdrantClient

client = QdrantClient(
    url="https://qdrant.ai-agent.svc.cluster.local:6333",
    api_key=os.environ["QDRANT_API_KEY"],
)

API Key Authentication for Embedding and LLM Services

Embedding and LLM API calls are authenticated through API keys by design — the provider validates your key against their service. The security challenge is not the authentication mechanism itself, but key management at scale across a distributed agent system.

API key hierarchy for AI workloads:

• Indexing key — Used by document ingestion pipelines. Has permission to call embedding APIs at high throughput for batch processing. This key sees the highest volume of API calls and should have the strictest rate limits and monitoring.
• Query key — Used by agent runtimes for interactive retrieval. Lower throughput but latency-sensitive. Should have tight provider-side limits and no vector database write or collection-management permissions.
• Admin key — Used only for configuration changes, provider portal access, and incident recovery. Should never be embedded in application configuration or environment variables.

# API key routing with per-key rate limits and audit
from openai import OpenAI, RateLimitError

class EmbeddingKeyManager:
    """Manages separate API keys for indexing vs. query workloads."""

    def __init__(self):
        # Keys retrieved from Vault at startup — never hardcoded
        self.index_key = VaultClient.get_secret("openai/embedding-index-key")
        self.query_key = VaultClient.get_secret("openai/embedding-query-key")

    def get_client(self, workload: str) -> OpenAI:
        if workload == "index":
            client = OpenAI(api_key=self.index_key)
            # Indexing: high concurrency, long timeout, relaxed rate limits
            return client
        elif workload == "query":
            client = OpenAI(api_key=self.query_key)
            # Query: low latency, strict timeout, user-attributed
            return client
        # Admin operations never reach application code

Authorization Models for Vector Databases

Collection-Level RBAC

Vector databases store potentially sensitive document collections — internal wikis, customer support histories, proprietary research. Authorization must control not just whether a service can connect to the database, but which collections it can read and whether it can write.

Qdrant API key RBAC model:

Qdrant's general API key authentication predates its granular controls; read-only API keys are available from v1.7, granular access API keys with per-collection read/write scoping from v1.9, and audit logging from v1.17. For open source Qdrant, granular API keys require an admin api_key and jwt_rbac: true; Qdrant Cloud enables granular access key authentication by default. This is the minimum viable authorization model for production deployments:

# Create scoped API keys for Qdrant
# Admin key (full access)
# POST /collections/{name}/cluster — cluster management

# Indexer key — write + read to ingestion collections only
# POST /collections/docs/points — upsert documents
# POST /collections/docs/points/search — verify ingestion
# No access to user-facing query collections

# Querier key — read-only on query collections
# POST /collections/user-docs/points/search
# POST /collections/internal-wiki/points/search
# POST is denied for /collections/*/points (upsert)

# In Python — the Qdrant client authenticates with the scoped key
from qdrant_client import QdrantClient

# Querier agent — read-only
query_client = QdrantClient(
    url="https://qdrant-cluster.internal",
    api_key=VAULT.get("qdrant/query-key"),
)

# Indexer pipeline — read-write on ingestion collections
index_client = QdrantClient(
    url="https://qdrant-cluster.internal",
    api_key=VAULT.get("qdrant/indexer-key"),
)

Authorization for Hybrid Search

When the retrieval pipeline includes a BM25 backend (Elasticsearch, OpenSearch, or PostgreSQL full-text search) alongside the vector database, authorization must be consistent across both. A querier should not be able to bypass vector-level restrictions by querying the BM25 index directly:

• Unified authentication token — The agent presents a single credential that both the vector database and BM25 index validate independently. SPIFFE identities are ideal here because the same identity document authenticates to both services.
• Consistent collection/document-level access — If the vector database restricts access to the "internal-wiki" collection, the BM25 index must enforce the same restriction on the corresponding search index. Use the same authorization policy (OPA, Kyverno, or custom middleware) for both backends.
• Reranker access control — The reranker service should authenticate callers and reject requests from unauthorized sources. The reranker's cross-encoder model processes document content — unauthenticated access to the reranker is an indirect data exfiltration channel.

Multi-Tenant Isolation

For RAG systems serving multiple tenants from a shared vector database, collection-level isolation is the minimum isolation boundary. Each tenant gets its own Qdrant collection (or Elasticsearch index) with dedicated credentials:

# Tenant-scoped Qdrant API keys in a multi-tenant deployment
# Tenant A: collection "tenant-a-docs", read-only
# Tenant B: collection "tenant-b-docs", read-only
# Ingress pipeline: collection "ingestion", read-write

# Enforcement is at the collection level — not the application level.
# A compromised querier credential for Tenant A cannot access
# Tenant B's documents, even if both agents run in the same Kubernetes pod.

Critical consideration for multi-tenant systems: Vector search results include the payload fields of matched points. If the payload contains tenant identifiers or metadata that reveals cross-tenant information, the vector database must strip these fields from search results based on the caller's authorization scope. Qdrant supports payload filtering per request, but the enforcement depends on the calling application — not the database itself. For true multi-tenant isolation, use separate Qdrant clusters per tenant or implement a proxy layer that enforces payload-scoping rules.

Secret Management for AI Agent Systems

The Secret Surface Area of an AI Agent Deployment

An AI agent deployment has a larger secret surface area than a typical web application because it authenticates to multiple external and internal services:

HashiCorp Vault Integration for AI Workloads

HashiCorp Vault is the most widely deployed secrets backend for AI infrastructure, and for good reason: it supports dynamic secrets (short-lived credentials generated on demand), automatic rotation, and comprehensive audit logging. Integration with agent runtimes follows the sidecar pattern:

# Agent startup — request secrets from Vault after Kubernetes auth
from pathlib import Path

from hvac import Client
from hvac.api.auth_methods import Kubernetes

vault = Client(
    url="https://vault.nsi.io:8200",
    verify="/var/run/secrets/kubernetes.io/serviceaccount/ca.crt",
)

jwt = Path(
    "/var/run/secrets/kubernetes.io/serviceaccount/token"
).read_text(encoding="utf-8")

login = Kubernetes(vault.adapter).login(
    role="agent-runtime",
    jwt=jwt,
)
vault.token = login["auth"]["client_token"]

secrets = vault.secrets.kv.v2.read_secret_version(
    path="ai-agent/production/embedding",
    mount_point="secret",
)
openai_key = secrets["data"]["data"]["openai-api-key"]

# Store in memory — never write to disk.
# Prefer Vault Agent sidecar or injector where possible so application
# code does not handle Kubernetes JWTs or Vault token renewal directly.

Kubernetes Secret Patterns for AI Workloads

When Vault is not available, Kubernetes Secrets with External Secrets Operator provide a simpler alternative. The critical pattern is to avoid embedding secrets in the application image or ConfigMap:

# ExternalSecret for OpenAI API key
# The secret data is stored in AWS Secrets Manager, GCP Secret Manager,
# or Azure Key Vault — never in plaintext in the repository.
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata:
  name: openai-embedding-key
  namespace: ai-agent
spec:
  refreshInterval: 1h
  secretStoreRef:
    name: aws-secret-store
    kind: SecretStore
  target:
    name: openai-embedding-key
  data:
  - secretKey: api-key
    remoteRef:
      key: ai-agent/prod/openai-embedding-key

---
# Pod mounts the ExternalSecret as a volume
apiVersion: v1
kind: Pod
metadata:
  name: agent-runtime
  namespace: ai-agent
spec:
  containers:
  - name: agent
    volumeMounts:
    - name: secrets
      mountPath: /etc/secrets
      readOnly: true
  volumes:
  - name: secrets
    secret:
      secretName: openai-embedding-key

The No-Disk Secret Pattern

For the highest-security deployments, secrets should never be written to disk — even as mounted files. The pattern:

• Secrets are fetched at startup from Vault or a secrets store, stored in application memory as a Python dictionary or Go map.
• Secrets are never logged — implement a custom log sanitizer that masks known secret patterns (sk-proj-*, sk-*, api_key values) before writing log entries.
• Secrets are never serialized — if the application crashes and writes a core dump, API keys in memory are exposed. Configure core dump filtering and limit core dump retention.
• Lease-based access — Each secret has a TTL. The application must handle secret rotation at runtime by watching the lease and refreshing from Vault before expiry.

In our audits of production AI deployments, the most common critical finding is not a weak algorithm or misconfigured firewall — it is an API key embedded in a Docker image that has been deployed to 47 production pods, each of which logs the key on every embedding request.

Network Policy Enforcement for AI Workloads

The Three-Zone Security Model Revisited

The three-zone network segmentation model (execution, inference, data) introduced in our network architecture article provides the structural foundation for security. Enforcement is implemented through Kubernetes NetworkPolicies:

# Default deny all ingress/egress for the AI agent namespace
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: default-deny-ai-agent
  namespace: ai-agent
spec:
  podSelector: {}
  policyTypes:
  - Ingress
  - Egress

---
# Allow execution zone to call inference zone
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-agent-to-llm
  namespace: ai-agent
spec:
  podSelector:
    matchLabels:
      app: agent-runtime
  egress:
  - to:
    - namespaceSelector:
        matchLabels:
          zone: inference
    ports:
    - port: 443  # HTTPS to LLM API endpoint
  policyTypes:
  - Egress

---
# Allow execution zone to call data zone (vector DB)
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: allow-agent-to-vector-db
  namespace: ai-agent
spec:
  podSelector:
    matchLabels:
      app: agent-runtime
  egress:
  - to:
    - namespaceSelector:
        matchLabels:
          zone: data
      podSelector:
        matchLabels:
          app: qdrant
    ports:
    - port: 6334  # gRPC
  policyTypes:
  - Egress

Internet Egress Controls

AI agent workloads have a legitimate need for internet egress — embedding API calls, LLM API calls, and model registry access all require external connectivity. The security challenge is ensuring that only authorized egress occurs:

• Dedicated egress gateway for AI API traffic — Route all embedding and LLM API calls through a dedicated forward proxy (Squid, Envoy, or a cloud NAT gateway). This proxy enforces allow-list domain policies: only api.openai.com, api.cohere.com, and similar known endpoints are permitted.
• Egress audit logging — Every outbound connection is logged with source pod identity, destination, request size, and response status. In a production deployment processing 200 queries/second, this generates approximately 17 million logs per day — stream them to a SIEM with compression and sampling.
• No direct internet access for data zone services — Vector databases and metadata stores must never have direct internet egress. All external data exchange for these services goes through a controlled gateway with explicit allowlisting.

# Egress allowlist for AI API traffic
# Cilium Clusterwide NetworkPolicy
apiVersion: cilium.io/v2
kind: CiliumClusterwideNetworkPolicy
metadata:
  name: ai-egress-allowlist
spec:
  endpointSelector:
    matchLabels:
      egress-role: ai-api
  egress:
  - toFQDNs:
    - matchName: "api.openai.com"
    - matchName: "api.cohere.com"
    - matchName: "api.voyageai.com"
    - matchName: "api.anthropic.com"
    toPorts:
    - ports:
      - port: "443"
        protocol: TCP
  - toEndpoints:
    - matchLabels:
        "k8s:io.kubernetes.pod.namespace": kube-system
        "k8s:k8s-app": kube-dns
    toPorts:
    - ports:
      - port: "53"
        protocol: ANY
      rules:
        dns:
        - matchPattern: "*"
  - toCIDR:
    # Internal Vault service VIP or load balancer only.
    # Replace with your exact Vault address; do not allow all RFC1918 space.
    - 10.32.14.25/32
    toPorts:
    - ports:
      - port: "8200"
        protocol: TCP
  - toCIDR:
    # Kubernetes API endpoint only, if this workload truly needs it.
    - 10.32.0.1/32
    toPorts:
    - ports:
      - port: "6443"
        protocol: TCP

mTLS Enforcement at the Network Level

Cilium and Istio can enforce authenticated service-to-service traffic below the application layer, which reduces the chance that a compromised pod can bypass peer authentication by connecting directly to a target service IP. Validate the exact semantics in your mesh: Cilium mutual authentication is still documented as beta, relies on SPIFFE/SPIRE, works only within a Cilium-managed cluster, is not compatible with Cluster Mesh trust-domain sharing, and does not replace encryption settings or application authorization.

# Cilium mTLS enforcement for the AI agent namespace
# Only pods with valid SPIFFE identities can communicate
apiVersion: cilium.io/v2
kind: CiliumNetworkPolicy
metadata:
  name: enforce-mtls-agent
  namespace: ai-agent
spec:
  endpointSelector:
    matchLabels:
      app: agent-runtime
  ingress:
  - fromEndpoints:
    - matchLabels:
        app: api-gateway
    authentication:
      mode: "required"  # Require Cilium mutual authentication

Audit Logging for Agent Actions

What to Log

Audit logging for AI agent systems must capture the full chain of authentication and authorization events:

• Authentication events: Every service-to-service authentication attempt — success, failure, and the identity presented.
• Authorization decisions: Every access control decision — which identity accessed which collection, with which action (read/write/search), and whether it was permitted or denied.
• Secret access events: Every retrieval of a secret from the secrets backend, including which workload requested it and with what lease duration.
• API key creation and rotation: When an API key is created, rotated, or revoked — including the admin identity that performed the action.
• Network policy changes: Changes to NetworkPolicy objects that could affect the security posture of AI workloads.

# Structured audit log entry for a vector database query
{
  "timestamp": "2026-07-20T08:00:00.123Z",
  "event_type": "vector_search",
  "identity": {
    "spiffe_id": "spiffe://nsi.io/agent/runtime/prod-querier",
    "kubernetes_pod": "agent-runtime-7d8f9c",
    "kubernetes_namespace": "ai-agent"
  },
  "target": {
    "service": "qdrant",
    "collection": "internal-wiki",
    "action": "search"
  },
  "authorization": {
    "result": "permit",
    "policy": "qdrant-collection-rbac-v2",
    "reason": "collection scope matched"
  },
  "network": {
    "source_ip": "10.0.1.42",
    "destination_ip": "10.0.2.15",
    "protocol": "gRPC",
    "mtls_enforced": true
  }
}

Operational Runbooks for AI Security Incidents

Compromised Embedding API Key

Symptom: Sudden spike in embedding API costs, unusual embedding patterns (embedding of non-document text), or alerts from the provider about suspicious API usage. Immediate action: Rotate the compromised key in the provider portal and Vault simultaneously. For Vault KV v2, write a new data version with vault kv patch -mount=secret ai-agent/production/embedding openai-api-key=<new-key> for a partial update, or vault kv put -mount=secret ai-agent/production/embedding openai-api-key=<new-key> ... when replacing the complete secret payload. Then force or wait for External Secrets Operator reconciliation according to its refreshInterval. Root cause investigation: Check Vault audit logs for which pod retrieved the key, then check the pod's network logs for exfiltration patterns. Remediation: Implement key rotation with a 24-hour maximum lease TTL and enable provider-side usage alerts at 2x normal daily spend.

Unauthorized Vector Database Access

Symptom: Authorization denied logs from Qdrant or Pinecone, or discovery of unknown collections in the vector database. Immediate action: Revoke all API keys and reissue with tighter scopes. Enable Qdrant's audit_log configuration to capture every query with identity metadata. Root cause: Typically a misconfigured NetworkPolicy or a leaked credential. Check whether the unauthorized access came from within the cluster (NetworkPolicy violation) or from outside (exposed endpoint). Remediation: Verify NetworkPolicy enforcement with cilium connectivity test and implement strict ingress rules on all Qdrant ports.

Secret Leakage via Application Logs

Symptom: API keys appearing in log aggregation systems (ELK, Datadog, Splunk) or alerts from a secrets scanner (GitGuardian, truffleHog, or custom regex-based scanner) on log export files. Immediate action: Rotate the leaked key and scrub the logs. Implement a log sanitizer that redacts patterns matching (sk-[A-Za-z0-9_-]{20,}) for OpenAI-style keys, provider-specific patterns maintained in a central scanner, and (eyJ[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}\.[A-Za-z0-9_-]{10,}) for JWT tokens. Root cause: The application code logs the full request object, including the Authorization header or API key parameter. Remediation: Add a log sanitization middleware that intercepts all log messages and masks known secret patterns before writing.

Conclusion

Securing AI agent infrastructure requires the same rigor as securing any distributed system, with additional attention to the unique threat surface of LLM and embedding pipelines. The security controls that protect traditional microservices — mTLS, RBAC, secret management, network policies, and audit logging — apply directly, but they must be adapted to the AI-specific workload patterns: embedding API calls to external services, vector database query patterns that reveal collection structure, and the different blast radii of stolen provider keys, compromised embedding workers, and leaked vector database credentials.

The four principles that guide AI infrastructure security are: authenticate every service-to-service interaction at the transport layer (mTLS eliminates an entire class of credential theft attacks), authorize at the collection level (RBAC for vector databases prevents cross-collection data access even within the same cluster), manage secrets with lease-based access (short-lived credentials with automatic rotation eliminate the credential exposure window), and enforce network segmentation at the policy level (default-deny NetworkPolicies with dedicated egress gateways for AI API traffic).

AI agent systems are becoming the default architecture for production intelligence workflows. Security must be integrated into the infrastructure from day one — not bolted on after a credential leak or data exposure. The controls described in this guide provide the foundation for a secure AI deployment at any scale.

In AI agent security, the question is never whether your system will be probed — it is whether your authentication and authorization layers will hold when it is.