Skip to content

fix(chatqna): rag.chunk_count metric was always 0 (hasattr on dict)

Jérôme Revillard requested to merge fix/chatqna-chunk-count-metric into main

fix(chatqna): rag.chunk_count metric was always 0

Bug

The rag.chunk_count attribute on the chatqna.orchestrate span was stuck at 0. The code used hasattr(node_output, "retrieved_docs"), but node outputs are dicts — the docs are a key, not an attribute. hasattr(dict, "retrieved_docs") is always False, so the loop never set the count.

Dashboards/alerts reading rag.chunk_count showed 0 even when chunks were retrieved.

Evidence (live trace 366658ab, el-salvador)

  • retriever.hybrid_search → 4 chunks
  • reranker.rerank → input 4, output 1 (slice)
  • chatqna.orchestraterag.chunk_count = 0 (should be 1)

Fix

Extracted _count_final_chunks(result_dict) helper:

  • dict access val.get("retrieved_docs") (getattr fallback for objects)
  • iterates all nodes, keeps the deepest stage's count (reranker, post-slice) → reflects chunks fed to the LLM

Tests

+5 (TestCountFinalChunks: empty, single-node, reranker-wins-post-slice, node-skipped, None-safe). 143 chatqna tests pass; ruff clean.

Edited by Jérôme Revillard

Merge request reports

Loading