Skip to content

Environment templates: Add remote model endpoints configuration section

Parent: #758 (closed)

Summary

Add environment variable entries for ALL configurable inter-service URLs to the env template, env.j2 Ansible template, and group_vars/all.yml defaults. This covers both OPEA/AI remote endpoints AND core service URLs.

Files to Change

1. env (project root template)

Add new Section 18 with all configurable service URLs:

# =============================================================================
# SECTION 18: REMOTE SERVICE ENDPOINTS (optional)
# =============================================================================
# Set these to point services at remote nodes.
# Leave empty (default) for local Docker networking.
#
# GPU model endpoints (remote GPU node):
# VLLM_ENDPOINT=http://10.0.0.50:8080
# TEI_EMBEDDING_ENDPOINT=http://10.0.0.50:8081
# TEI_RERANKING_ENDPOINT=http://10.0.0.50:8082
# VLLM_TRANSLATION_ENDPOINT=http://10.0.0.50:9031
# DOCLING_ENDPOINT=http://10.0.0.50:8083
#
# ChatQnA full-URL overrides (take precedence over HOST_IP+PORT):
# LLM_SERVER_URL=http://10.0.0.50:8080
# EMBEDDING_SERVER_URL=http://10.0.0.50:6000
# RETRIEVER_SERVICE_URL=http://10.0.0.60:7000
# RERANK_SERVER_URL=http://10.0.0.60:8000
# TRANSLATION_SERVICE_URL=http://10.0.0.50:9031
# GUARDRAIL_SERVICE_URL=http://10.0.0.50:9090
#
# Core service URLs (override for fully distributed deployment):
# ARANGO_URL=http://10.0.0.60:8529
# BACKEND_SERVICE_URL=http://10.0.0.60:3000
# DOCUMENT_REPOSITORY_URL=http://10.0.0.60:3001
# DOC_REPO_URL=http://10.0.0.60:3001
# KEYCLOAK_INTERNAL_URL=http://10.0.0.70:8080
# TRANSLATION_CACHE_HOST=10.0.0.60
#
# Kong upstream hosts (override when Kong is on a different node):
# KONG_BACKEND_HOST=10.0.0.60
# KONG_DOCREPO_HOST=10.0.0.60
# KONG_KEYCLOAK_HOST=10.0.0.70

2. deploy/ansible/templates/env.j2

Add Jinja2 conditionals for all variables:

# =============================================================================
# SECTION 18: REMOTE SERVICE ENDPOINTS (optional)
# =============================================================================
{% if vllm_endpoint is defined and vllm_endpoint %}VLLM_ENDPOINT={{ vllm_endpoint }}
{% endif %}
{% if tei_embedding_endpoint is defined and tei_embedding_endpoint %}TEI_EMBEDDING_ENDPOINT={{ tei_embedding_endpoint }}
{% endif %}
{% if tei_reranking_endpoint is defined and tei_reranking_endpoint %}TEI_RERANKING_ENDPOINT={{ tei_reranking_endpoint }}
{% endif %}
{% if vllm_translation_endpoint is defined and vllm_translation_endpoint %}VLLM_TRANSLATION_ENDPOINT={{ vllm_translation_endpoint }}
{% endif %}
{% if docling_endpoint is defined and docling_endpoint %}DOCLING_ENDPOINT={{ docling_endpoint }}
{% endif %}
{% if llm_server_url is defined and llm_server_url %}LLM_SERVER_URL={{ llm_server_url }}
{% endif %}
{% if embedding_server_url is defined and embedding_server_url %}EMBEDDING_SERVER_URL={{ embedding_server_url }}
{% endif %}
{% if retriever_service_url is defined and retriever_service_url %}RETRIEVER_SERVICE_URL={{ retriever_service_url }}
{% endif %}
{% if rerank_server_url is defined and rerank_server_url %}RERANK_SERVER_URL={{ rerank_server_url }}
{% endif %}
{% if translation_service_url is defined and translation_service_url %}TRANSLATION_SERVICE_URL={{ translation_service_url }}
{% endif %}
{% if guardrail_service_url is defined and guardrail_service_url %}GUARDRAIL_SERVICE_URL={{ guardrail_service_url }}
{% endif %}
{% if arango_url is defined and arango_url %}ARANGO_URL={{ arango_url }}
{% endif %}
{% if backend_service_url is defined and backend_service_url %}BACKEND_SERVICE_URL={{ backend_service_url }}
{% endif %}
{% if document_repository_url is defined and document_repository_url %}DOCUMENT_REPOSITORY_URL={{ document_repository_url }}
{% endif %}
{% if doc_repo_url is defined and doc_repo_url %}DOC_REPO_URL={{ doc_repo_url }}
{% endif %}
{% if keycloak_internal_url is defined and keycloak_internal_url %}KEYCLOAK_INTERNAL_URL={{ keycloak_internal_url }}
{% endif %}
{% if translation_cache_host is defined and translation_cache_host %}TRANSLATION_CACHE_HOST={{ translation_cache_host }}
{% endif %}
{% if kong_backend_host is defined and kong_backend_host %}KONG_BACKEND_HOST={{ kong_backend_host }}
{% endif %}
{% if kong_docrepo_host is defined and kong_docrepo_host %}KONG_DOCREPO_HOST={{ kong_docrepo_host }}
{% endif %}
{% if kong_keycloak_host is defined and kong_keycloak_host %}KONG_KEYCLOAK_HOST={{ kong_keycloak_host }}
{% endif %}

3. deploy/ansible/group_vars/all.yml

Add defaults:

# =============================================================================
# REMOTE SERVICE ENDPOINTS (empty = local Docker networking)
# =============================================================================
# GPU model endpoints
vllm_endpoint: ""
tei_embedding_endpoint: ""
tei_reranking_endpoint: ""
vllm_translation_endpoint: ""
docling_endpoint: ""
# ChatQnA full-URL overrides
llm_server_url: ""
embedding_server_url: ""
retriever_service_url: ""
rerank_server_url: ""
translation_service_url: ""
guardrail_service_url: ""
# Core service URLs
arango_url: ""
backend_service_url: ""
document_repository_url: ""
doc_repo_url: ""
keycloak_internal_url: ""
translation_cache_host: ""
# Kong upstream hosts
kong_backend_host: ""
kong_docrepo_host: ""
kong_keycloak_host: ""

Acceptance Criteria

  • env template has Section 18 with all variables documented and examples
  • env.j2 renders correctly with and without remote endpoint vars
  • group_vars/all.yml has empty-string defaults for all variables
  • ansible-playbook deploy.yml --tags deploy --check passes
  • Generated .env without overrides matches current behavior
Edited by david-full