Skip to content

docs: RAG + Observability + KB + Operations sections, relocate strategy docs, trim overview, persona-oriented

Jérôme Revillard requested to merge docs/rag-observability-sections into main

What

Comprehensive user-facing documentation review and restructure: dedicated sections for the topics that had none, content relocated out of the repo root into the published site, scan-dump pages trimmed, and a quality baseline applied across every page. Site is now organised around the people who read it (evaluators, operators, content managers, developers).

Sections added (4 new, 20 new pages)

  • RAG Pipeline (rag/, 7 pages) — pipeline architecture, hybrid retrieval, reranking, generation, data labelling, contextual retrieval, choosing models. (The choosing-models and data-labeling pages are the relocated GENIE-AI-ChoosingLLMs.md and GENIE.AI-Data-Labelling-Strategy.md.)
  • Observability (observability/, 5 pages) — stack overview, tracing, dashboards, alerting, configuration.
  • Knowledge Base (knowledge-base/, 4 pages) — ingestion, document lifecycle, labelling & taxonomy, content guidance. (Fills the content-manager persona gap.)
  • Operations (operations/, 4 pages) — backup & restore, updates (incl. the re-ingestion rule), scaling, troubleshooting. (Fills the day-2 operator gap.)

Restructured / relocated

  • core/project-overview.md: 917 → 125 lines. Was a scan-dump kitchen-sink; now an orientation layer that links out to the dedicated pages.
  • architecture/logging-architecture.md → docs/ (internal). It was a vendor evaluation (SigNoz/OpenSearch candidates); the public observability/ section documents the actual Victoria stack.
  • mobile-deployment-guide → mobile/ (was in deployment/).
  • 3 repo-root orphans → deployment/: GENIE.AI-Installation-Configuration-Guide.md (92K, now install-guide.md), README_GPU.md → gpu.md, E2E-nVidia-A40-Install-Guide.md → a40-install.md. All cross-references updated.

Quality baseline (every page)

  • Added a user-oriented description to the 21 pages whose front matter had none.
  • Stripped 12 duplicate # Title H1 lines (Docsy rendered them as a second title).
  • Cross-linked the new sections from architecture.md, project-overview, rag/_index, about.
  • about.md expanded into a decision-maker hub (sovereignty, DPG, security & data protection, persona→entry-point table).

Accuracy fix (code is source of truth)

Contextual-retrieval defaults in docs contradicted the code. Aligned all docs to the code: CONTEXTUAL_RETRIEVAL_ENABLED=true (on by default), CONTEXTUAL_STRATEGY=doc_level, CONTEXTUAL_LABEL_RAW=true. Fixed across env, CLAUDE.md, the RAG pages, architecture.md, the installation guide, and the dataprep README.

Verified

  • hugo --gc --minify builds clean locally (0 errors); all 46 relref targets resolve.
  • 9 → 11 sections, ~49 pages, build green.

Follow-ups (not in this MR)

  • Full CLAUDE.md↔code drift audit beyond contextual-retrieval.
  • Re-orient remaining dev-heavy pages (source-tree, UI inventories) for a broader audience.

Merge request reports