Troubleshooting

First confirm service status and recent configuration changes, then start troubleshooting from the earliest failing component.

API Cannot Connect to Database

Check POSTGRES_SERVER, POSTGRES_PORT, the database password, and PostgreSQL readiness. The service name depends on the deployment: db in Compose, postgres in Kubernetes, localhost for source development. Confirm pg_search and pg_stat_statements are preloaded (the supplied image starts with shared_preload_libraries=pg_search,pg_stat_statements -c pg_stat_statements.track=all), then restart the database before checking extensions. In Kubernetes, API/Worker/Beat all carry a wait-for-postgres initContainer; a pod stuck in Init means the database is not ready yet.

Frontend Cannot Access API

Confirm the external proxy or Ingress routes /api to api:8000, and that the frontend build-time NEXT_PUBLIC_API_URL uses the same prefix as the backend API_V1_STR (default /api/v1). Do not set the API address to an internal host the browser cannot resolve inside the container, and do not use localhost as the API address for Sandbox Worker.

When browser calls work but server-side page fetching fails, check whether frontend sets BACKEND_INTERNAL_URL: the code default is http://localhost:8000 (Compose injects http://api:8000), and the single-file K8s manifest and Helm defaults do not inject it, so SSR targets localhost:8000 inside the pod.

Worker Does Not Process Tasks

Check the Redis connection, password, queue names, and Worker logs. Knowledge Base uses Celery background processing; beat must run only one replica. Before cleaning or restarting Workers, confirm no production documents are being processed.

Locate the backlog by queue. The five queues are default, agent, knowledge, workflow (all consumed by worker) and sandbox (consumed by sandbox-worker):

# add -a <password> when REDIS_PASSWORD is set
docker compose exec redis redis-cli llen agent
docker compose exec redis redis-cli llen knowledge
docker compose logs -f worker

Tasks sitting in a queue with nothing consuming them mean the worker command or deployment for that queue is wrong. The classic case is a worker whose -Q omits agent: every durable AgentRun piles up in the agent queue and conversations stay in progress with no error. Full queue-to-task mapping: Deployment architecture → Celery queue layout.

Sandbox Artifact Upload Fails

Confirm SANDBOX_ARTIFACT_UPLOAD_BASE_URL=http://api:8000, the API and Sandbox Worker use the same signing configuration, or configure a dedicated SANDBOX_ARTIFACT_UPLOAD_API_KEY. Artifacts must stay inside /workspace (the policy rejects anything else), uploads go to the internal endpoint /api/v1/upload/sandbox-artifact, and both the per-file and total limits default to 10MB. Worker and sandbox-worker mount no uploads volume, so they must be able to reach the API through API_INTERNAL_BASE_URL.

Sandbox Jobs Fail with bwrap User Namespace Error

bwrap: No permissions to create new namespace, likely because the kernel does not allow non-privileged user namespaces.

The supplied deployments (Docker Compose, Helm, Kubernetes) run the sandbox worker as root with CAP_SYS_ADMIN, so user namespace creation is privileged and this error should not occur. It typically means the worker is not running with the supplied security configuration (for example a custom deployment kept the worker non-root, cap_add: SYS_ADMIN is missing, or seccomp=unconfined was overridden). Custom non-root deployments must enable unprivileged user namespaces at the node level (seccomp=unconfined does not help): on Ubuntu 23.10+ run sysctl -w kernel.apparmor_restrict_unprivileged_userns=0; on Debian run sysctl -w kernel.unprivileged_userns_clone=1, persist via /etc/sysctl.d/, and verify with unshare -U true. In Kubernetes apply the sysctl to every node; it cannot be set per pod. Full details: Code Sandbox → Host Kernel Requirements.

Redis Connection Fails

Confirm Redis is running: docker compose ps redis and docker compose logs redis; test with docker compose exec redis redis-cli ping (expect PONG, add -a <password> when a password is set); check REDIS_HOST/REDIS_PORT/REDIS_PASSWORD. The Celery broker and result backend are derived from REDIS_*; no separate configuration exists.

Qdrant Connection Fails

Confirm Qdrant is running: docker compose ps qdrant; test with curl http://localhost:6333/healthz; check QDRANT_URL (Compose uses http://qdrant:6333) and QDRANT_API_KEY.

SSO Login Fails

SSO providers are configured in Site Settings > SSO (no CLI check command). Confirm the callback URL matches FRONTEND_URL; review the provider logs; clear browser cookies and the SSO session, then retry.

Agent Does Not Respond

Check the API logs for agent-related errors; LLM keys/base URLs are stored per provider in the database and managed on the Models page — they are not environment variables (env | grep OPENAI_API_KEY finding nothing is normal). Use the connection test on the Models page; check provider quota/rate limits; review Worker logs.

Workflow Execution Fails

Check docker compose logs api | grep workflow; confirm Worker and Beat are running (docker compose ps worker); for runs stuck in Running, restart with docker compose restart worker beat; verify node configuration, circular dependencies, and test individual nodes.

Document Upload Fails

The per-knowledge-base upload limit is the kb_document_max_upload_size_mb site setting (default 50MB), not an environment variable. The API exclusively owns the upload directory (Workers use the internal upload gateway): check disk with docker compose exec api df -h /app/uploads. If processing fails after a successful upload, re-trigger processing from the Knowledge Base document UI.

Retrieval Fails

1. Document status must be completed.
2. The embedding model must exist, be authorized, and vector dimensions must match.
3. Qdrant must be reachable.
4. Hybrid/full-text retrieval requires PostgreSQL and pg_search.
5. On rerank failure, check rerank model credentials and provider quota.
6. Read the retrieval lab's diagnostics and timings.

HTTP Tool or Intranet Requests Blocked

Custom HTTP tools, workflow HTTP request nodes, Knowledge Base URL import/preview extraction, and database connection testing are all governed by Site Settings > Security > Outbound Network Allowlist (ssrf_allowed_targets). This is a database SiteSetting, not an environment variable; it allows public addresses by default and blocks private (RFC 1918) and loopback addresses. Internal targets must be explicitly added, up to 200 entries, one per line:

• Single IP: 192.168.1.50, 10.20.1.10
• CIDR subnet: 10.10.0.0/16, 172.20.0.0/24
• Exact domain: oa.company.local, api.internal.service
• Wildcard domain: *.corp.internal

Loopback is permanently blocked to prevent agent abuse of host local services and Redis/Docker ports. To reach a companion service on the host, use the host LAN IP or host.docker.internal. Deployments using TUN-mode transparent proxies (Clash, Surge, Mihomo) require no extra configuration: the 198.18.0.0/15 benchmark range is built-in and permitted so ipaddress does not treat resolved public domains as private.

The Model Endpoint Allowlist (Site Settings > Security > Model Endpoint Allowlist, model_endpoint_allowlist) is an independent policy scoped strictly to LLM, embedding, rerank, and media providers. It is a strict allowlist: an empty list blocks every model endpoint, but administrators may approve local inference services like http://localhost:11434. Do not substitute the outbound network allowlist for it.

Model Calls Fail

Check the model's is_enabled, team authorization, endpoint allowlist, API Key, context length, provider rate limits, and type capabilities. Error code 6103 means quota exceeded; 6104 means unauthorized.