Docker Compose: Expose GPU service ports for remote access
Parent: #758 (closed)
Summary
Add host port bindings to the 5 GPU model services (including the new docling-service) so they can be reached from other nodes. All services must be stateless and idempotent for concurrent multi-instance access.
Ports to Expose
| Service | Container Port | Host Port (default) | Env Var Override |
|---|---|---|---|
| vllm | 8000 | 8080 | VLLM_HOST_PORT |
| vllm-translation-guardrail | 9031 | 9031 | VLLM_TRANSLATION_HOST_PORT |
| tei (embedding) | 80 | 8081 | TEI_EMBEDDING_HOST_PORT |
| tei_reranker | 80 | 8082 | TEI_RERANKING_HOST_PORT |
| docling-service (NEW) | 8000 | 8083 | DOCLING_HOST_PORT |
Implementation
Add ports sections to each GPU service in docker-compose.yaml:
vllm:
ports:
- "${VLLM_HOST_PORT:-8080}:8000"
vllm-translation-guardrail:
ports:
- "${VLLM_TRANSLATION_HOST_PORT:-9031}:${VLLM_TRANSLATION_SERVICE_PORT:-9031}"
tei:
ports:
- "${TEI_EMBEDDING_HOST_PORT:-8081}:80"
tei_reranker:
ports:
- "${TEI_RERANKING_HOST_PORT:-8082}:80"
docling-service:
ports:
- "${DOCLING_HOST_PORT:-8083}:8000"
Idempotency & Statelessness
All 5 services must be safe for concurrent access from multiple app instances:
- vllm: Already stateless — OpenAI-compatible API, no per-request state
- vllm-translation: Already stateless — same
- tei: Already stateless — embedding/reranking are pure functions
- tei_reranker: Already stateless — same
- docling-service: Must be designed stateless — process document, return result, no persisted state
Security Note
For production, consider:
- Binding to internal network interface only (e.g.,
10.0.0.50:8080:8000) - Firewall rules restricting access to app node IPs
- TLS termination via nginx reverse proxy on the GPU node
Acceptance Criteria
-
5 GPU services have host port bindings with configurable host ports -
Default ports avoid common conflicts (8080, 9031, 8081, 8082, 8083) -
Port overrides work via env vars -
docker compose configvalidates without errors
Edited by david-full