Skip to content

fix(dataprep): scope chunk labels to document file_labels

Jérôme Revillard requested to merge fix/label-scope-to-file-labels into main

Problem

Chunks were labeled with taxonomy labels that fall outside the document's own scope (file_labels).

Reproduced on el-salvador: tomato guide 1782731590193_d684ff9f (Guia-Centa-Tomate_no_table.md, 48 chunks) received Cucumber (chunks 37/39/40), Onion (39) and Cabbage (32/35/47) — sibling crops absent from its file_labels — on pest-management chunks whose text never mentions those crops.

Root cause

_label_with_llm prompts the LLM with the full taxonomy (all_labels) and _finalize_chunk_labels validates suggestions against that same full taxonomy, never against file_labels. As long as a label is a valid taxonomy entry (Cucumber/Cabbage/Onion are), it can land on any chunk. The flat el-salvador taxonomy (crops + topics as sibling services) amplifies this — the LLM sprinkles sibling-crop labels on generic chunks.

file_labels was used only as a fallback (empty taxonomy / 3× LLM failure).

Fix

_finalize_chunk_labels now scopes resolved labels to the document's file_labels:

  • taxonomy canonicalisation (exact + synonym match) runs first, unchanged;
  • any resolved label not in file_labels is dropped before storage;
  • empty file_labels ⇒ no filtering (guard: documents without a scope keep all taxonomy-validated labels — no behaviour change for them);
  • dropped labels are appended to the ingestion-log entry for observability.

Surgical, ~10 lines, no change to the LLM call or prompt.

Verification

  • TDD: failing test first (Cucumber survives → AssertionError), then green after fix.
  • Full tests/test_dataprep.py: 72 passed, 0 failures.
  • ruff check + ruff format --check: clean.
  • Pending: re-ingest the tomato guide on a stack and confirm 0 out-of-scope labels; re-check the cucumber doc (Cucumber ∈ its file_labels ⇒ no under-labeling regression).

Trade-off

A chunk that genuinely discusses an out-of-scope topic (e.g. a tomato chunk mentioning companion-planted cucumber) will no longer receive that label, so it won't surface in a label-filtered search for it. For monocrop-guide taxonomies this is the desired scoping; flagging for reviewer awareness.

Merge request reports

Loading