Skip to content

feat(#758): Remote GPU Node — Standalone Compose + Ansible + nginx Proxy

Jérôme Revillard requested to merge feat/testing-framework/758-gpu-node-architecture into feat/testing-framework/prd

Summary

Remote GPU node architecture for GENIE.AI (#758 (closed)). Deploys AI model services (vLLM, TEI, docling-serve) on a dedicated GPU node behind nginx with TLS termination, API key authentication, and path-based routing on port 443.

Changes

GPU Node (deploy-gpu.yml)

  • Path-based routing on port 443: /llm/, /translation/, /embed/, /rerank/, /docling/
  • nginx proxy with TLS termination, API key authentication (X-API-Key header)
  • Self-signed cert generation (configurable, disabled with Let's Encrypt)
  • Smoke tests validate all 5 endpoints post-deploy
  • Graceful config reload (nginx -s reload on config changes)

App Node (docker-compose.yaml)

  • Dual profiles: profiles: [opea] (orchestrators) vs profiles: [gpu-models] (GPU-heavy)
  • GPU_MODEL_REPLICAS: set to 0 in Swarm to skip GPU containers on app node
  • Endpoint overrides: ${VAR:-default} pattern for 7 orchestrator services
  • Removed depends_on GPU service refs from 5 orchestrators (Compose-mode compat)
  • ChatQnA: VLLM_LLM_ENDPOINT support for remote GPU with HTTPS/protocol/path-prefix
  • API key propagation: OPENAI_API_KEY from VLLM_API_KEY for chatqna

Ansible (env.j2, all.yml)

  • Section 18: auto-generates GPU endpoint URLs + GPU_MODEL_REPLICAS=0 when gpu_node_host set
  • New vars: gpu_node_host, vllm_api_key in group_vars/all.yml

Python (genieai_chatqna.py)

  • VLLM_LLM_ENDPOINT env var support (mirrors existing VLLM_TRANSLATION_ENDPOINT)
  • URL parsing for host/port/protocol/endpoint prefix
  • protocol=https passed to OPEA MicroService for remote endpoints

Tests

  • Config validator updated: all ports → 443, legacy port detection, docling image fix
  • 24/24 tests passing

Documentation

  • CLAUDE.md, ansible README, docker-compose-setup.md, docker-swarm-setup.md, env template
  • All --profile commands updated for dual-profile architecture
  • Clear warnings: DEPLOY_OPEA=0 must NOT be used to skip GPU services

Architecture

App node (--profile opea)                GPU node (standalone)
┌─────────────────────────┐              ┌─────────────────────────────┐
│ Frontend, Backend,        │              │ nginx (port 443, TLS)     │
│ ArangoDB, Redis, ...      │  HTTPS 443   │   /llm/         → vLLM     │
│                         │ ──────────── │   /translation/ → vLLM T   │
│ ChatQnA ────────────────────────────→ │   /embed/       → TEI Emb  │
│ Retriever ──────────────────────────→ │   /rerank/      → TEI Rer  │
│ Dataprep ──────────────────────────→ │   /docling/     → docling   │
└─────────────────────────┘              └─────────────────────────────┘

Usage

Compose (remote GPU)

GPU_NODE_HOST=<gpu-node-host>
VLLM_API_KEY=<api-key>
docker compose --profile opea up -d

Compose (local GPU)

docker compose --profile opea --profile gpu-models up -d

Swarm (remote GPU)

# In group_vars/<env>/vars.yml:
gpu_node_host: "<gpu-node-host>"
# Ansible auto-sets GPU_MODEL_REPLICAS=0 and endpoint URLs

Default Behavior

Zero impact when GPU_NODE_HOST is not set — all defaults unchanged.

Closes #758 (closed)

Edited by Jérôme Revillard

Merge request reports