Skip to content

feat: Cloud-native deployment with unified configuration, security hardening, and Ansible automation

Jérôme Revillard requested to merge deployment-stabilization into main

Summary

Consolidates the GENIE.AI deployment infrastructure for cloud-native Docker Swarm, rewrites the API gateway, and adds full Ansible automation for idempotent deployments.

Changes by area

1. Docker Compose consolidation

  • Single compose file: Merged components/docker-compose.yaml and GPU-specific compose files (docker-compose-t4.yaml, docker-compose-RTX6000-ADA.yaml) into root docker-compose.yaml (Swarm-compatible).
  • DEPLOY_OPEA variable: Controls OPEA service replicas via ${DEPLOY_OPEA:-1} (set 0 to skip GPU services).
  • Removed: depends_on directives (Swarm uses healthchecks + restart policy), stale HUGGINGFACEHUB_API_TOKEN.
  • Bind mounts: Centralized under ./data/ with DATA_DIR variable.
  • GPU env files: env.t4 and env.rtx6000 now contain only GPU-specific overrides (memory, model length, TEI image versions).
  • Removed stale files: env-T4, components/docker-compose.yaml, configs/opea-config/docker-compose*.yaml, per-component env files, per-component Dockerfile-single-node files.

2. API Gateway overhaul

  • Kong auto-config: Routes/plugins applied by kong-config one-shot Swarm service — no manual post-deploy script needed.
  • Internalized Kong ports: Kong no longer exposes ports to the host; only NGINX exposes 80/443.
  • NGINX template-based config: Auto-rendered at startup via default.conf.template. Uses Docker DNS resolver for lazy upstream hostname resolution (fixes startup race conditions).
  • SPA routing fix: Replaced http-server with nginx for proper Vue Router history mode (fixes 404 on page refresh).
  • Removed: Entire api-gateway-solution/old-config/, kong-backups/, journal-check.sh, stale Kong shell scripts (~4,000 lines of legacy automation).

3. Ansible deployment automation (new)

  • deploy/ansible/: Full playbook with install, prepare, build, deploy tags for granular re-runs.
  • Per-environment secrets: Ansible Vault with directory-based group_vars/<env>/vars.yml + vault.yml.
  • Docker/NVIDIA version pinning: Packages held via dpkg --set-selections to prevent accidental upgrades.
  • Image build caching: Git SHA labels to skip rebuilds when code hasn't changed.
  • Local Docker registry: Pushes built images to a local registry for Swarm distribution.
  • .env generation: Jinja2 template handles special characters in passwords (avoids source issues).
  • GPU support: gpu_env_file option (env.t4, env.rtx6000) for GPU-specific model parameters.
  • Post-deploy verification: Service replica health checks and HTTP smoke tests.
  • Teardown playbook: Remove stack with optional volume and registry cleanup.
  • SSL certificates: Self-signed generation or provided certs with proper file permissions.

4. Backend cloud-native adaptation

  • Proxy-aware frontend URL: Auth controller derives frontend URL from X-Forwarded-Proto/X-Forwarded-Host headers (proxy deployments).
  • Email verification flow: Redirects updated to /verify-email-success with status/reason query params.
  • Service config: db-connection-service.js and appConfig.js updated for centralized env-based configuration.

5. Configuration system (config/)

  • LLM prompts: Three-tier priority system — ENV VAR > FILE (config/prompts/) > CODE DEFAULT.
  • Prompt files: chatqna-system.txt, chatqna-abstention.txt, label-selector.txt.
  • secrets/: Directory structure for SSL certificates with .gitignore and README.md.

6. Documentation

  • CLAUDE.md: AI agent project context with architecture, commands, code standards.
  • docs/docker-swarm-setup.md: Comprehensive Docker Swarm deployment guide.
  • README_GPU.md: GPU deployment reference (T4, RTX 6000 Ada).
  • GENIE.AI-Installation-Configuration-Guide.md: Major rewrite with cloud-native instructions.
  • Tech specs: 6 tech specs in _bmad-output/ (API gateway, SPA routing, Kong init, Swarm deployment, Ansible, regression plan).

7. BMAD framework (_bmad/, .claude/commands/)

  • Full BMAD framework integration (core 6.0.4, cis 0.1.9, tea 1.7.2) for AI-assisted project management.
  • Agent definitions, workflow templates, and skill commands for Claude Code.

Files changed

712 files changed, +101,689 / -24,260 lines

Note: ~60% of the additions are the BMAD framework (_bmad/) which is a vendored dependency tracked for worktree access.

Test plan

  • Clean stack deploy from scratch (docker stack rm then docker stack deploy)
  • 11 core services at 1/1 replicas
  • OPEA services at 0/0 replicas (DEPLOY_OPEA=0)
  • Frontend returns 200 on https://localhost/
  • API health returns 200 through Kong (https://localhost/api/health)
  • docker compose config --env-file .env validates without errors
  • Ansible playbook syntax check passes (ansible-playbook --syntax-check)
  • Ansible full deploy to test VM (10.0.0.110, RTX 6000 Ada, ports 1080/1443)
  • Image build with git SHA caching (skip unchanged)
  • Image push to local Docker registry
  • SPA routing: page refresh returns 200 (no 404)
  • Deploy with DEPLOY_OPEA=1 on GPU node and verify full RAG pipeline
Edited by Jérôme Revillard

Merge request reports