Skip to content

fix(chatqna): withhold full [[CONF: sentinel prefix in stream (prod leak)

Jérôme Revillard requested to merge worktree-fix-sentinel-stream-leak into main

Prod bug (urgent)

On el-salvador with LLM_SELF_CONFIDENCE_ENABLED=1, the [[CONF:<n>]] self-grade sentinel leaked into the user-visible answer and self_confidence was never captured (so confidence_score fell back to the retrieval value). Confirmed on deployed image bc585a79 (!175 (merged) code present, regex correct, flag ON).

Root cause

Streaming path _streaming_marker_tail_len withheld only proper prefixes of marker literals (limit = len(marker) - 1). When the tokenizer emits the full [[CONF: prefix as a chunk boundary (number + closing arrive later), that bare prefix has no regex substitute (only the COMPLETE [[CONF:<n>]] is extracted by _extract_self_confidence) — so it was neither substituted nor withheld → leaked, and the value was lost.

Conversation markers are unaffected: their full literals are regex-substituted (_CONV_SEP_RE/_CONV_ROLE_RE) before the tail check, so they never sit at the tail. The sentinel prefix is the only one needing the full-literal withhold.

Fix

limit = min(n, len(marker) - 1)limit = min(n, len(marker)) (allow the full marker length).

Proof

New regression test test_streaming_sentinel_split_at_full_prefix_is_withheld reproduces the prod failure: fails with len-1 (sentinel leaks), passes with the fix.

chatqna suite green (132), ruff clean.

Merge request reports

Loading