feat(confidence): surface LLM self-grade transparently via confidence_score
Context
Follow-up to !172 (merged). When LLM_SELF_CONFIDENCE_ENABLED=1, the citizen-facing confidence_score now reflects the LLM's self-assessed groundedness ([[CONF:0-100]] sentinel), falling back to the retrieval confidence when the sentinel is absent — so the existing client badge never disappears. Transparent to web + mobile clients (they already render confidence_score): enabling the feature surfaces the LLM value with no client change.
Changes
chatqna (genieai_chatqna.py):
-
_assemble_source_documentsreturnsretrieval_confidence_score(rank-weighted). - New
_display_confidence(self ?? retrieval)helper. - Metadata (stream + non-stream): always emit
retrieval_confidence_score; when flag on →confidence_score = _display_confidence(...)+ rawself_confidence; when off →confidence_score = retrieval(unchanged).
Backend (query-service.js): pass retrieval_confidence_score + self_confidence through the SSE metadata reconstruction (whitelist) so they persist to the DB / QueryInspector / eval.
Admin (QueryInspectorDetail.vue): show retrieval_confidence_score + self_confidence alongside confidence_score (admin-only eval view; List unchanged).
Verification
- chatqna pytest: 131 passed (+3
_display_confidence; streaming tests updated for new semantics). - backend jest query-service: 122 passed (+2 reconstruction).
- backend jest query-routes: passed.
- frontend QueryInspector jest: 26 passed.
- ruff + eslint + prettier: clean.
⚠ ️ Product note
The LLM self-grade is an uncalibrated self-assessment — 2B models are overconfident and may omit the sentinel (→ fallback retrieval). Surfacing it to citizens is a deliberate decision (el-salvador will enable it). The raw retrieval + self values remain available to admins (QueryInspector) for calibration evaluation. Null-case (no sentinel) → confidence_score falls back to retrieval → badge never disappears.