docs(ansible): achieve read-once-and-deploy documentation standard
Goal
The Ansible deployment documentation should be comprehensive enough that a new team member can read it once and successfully deploy to a fresh server without trial-and-error. Current gaps prevent this.
Documentation Gaps
1. Ansible Troubleshooting Guide
Priority: HIGH — Directly caused hours of wasted time during cloud deployment.
Add a troubleshooting section to deploy/ansible/README.md covering:
-
'swarm_registry_url' is undefined— occurs when running--tags build,deploywithout--tags install -
'image_tag' is undefined— occurs when task ordering is wrong in build play -
Permission denied (publickey)— SSH key permissions on WSL (Windows mount has 777, must copy to /tmp/) -
Problem running vault password script (Exec format error)— same WSL permissions issue for vault password file -
Missing required vault variable— list ALL required vault variables with descriptions -
nginx_permissions_policy is undefined— required variable not in all env templates - Images pulling
latestinstead of git SHA — IMAGE_TAG not set when deploy play runs alone
2. Required Vault Variables Reference
Priority: HIGH
Create a complete checklist of ALL vault variables required for deployment. Currently only listed inline in the playbook error message. Should include:
- Required variables with descriptions and example values
- Optional variables with defaults
- Environment-specific notes (e.g., mobile client ID not needed for web-only deployments)
3. WSL-Specific Setup Guide
Priority: MEDIUM
Windows users running Ansible via WSL hit permission issues that Linux/Mac users don't:
- SSH keys and vault password files on
/mnt/mounts have 777 permissions - Must copy to
/tmp/and chmod before use - Document the ssh-agent workaround for passphrase-protected keys
- Ansible warnings about world-writable directory
4. RAG Pipeline End-to-End Overview
Priority: LOW
Individual OPEA services are documented but no single doc ties the pipeline together:
- dataprep (ingestion/chunking) → embedding → retriever (ArangoDB) → reranker → chatqna (LLM)
- Data flow, API contracts between services, configuration dependencies
5. Observability User Guide
Priority: LOW
Technical config is documented but no guide for:
- Accessing Grafana via Kong route
- Interpreting the provisioned dashboards
- Querying VictoriaLogs/VictoriaTraces for debugging
6. Environment Variable Reference (env.j2)
Priority: MEDIUM
The env.j2 template is the source of truth for all configuration but is not documented. Create a reference table mapping each variable to:
- Which group_vars file sets it (all.yml vs env vars.yml vs vault.yml)
- Default value
- Whether it's required or optional
- What it controls