Skip to content

fix(retriever): label filter via search_start data contract

Problem

The OPEA MicroService framework silently drops custom fields (like context) between mega-service nodes — it creates a dynamic __main__ input type that only preserves standard EmbedDoc fields. As a result, the retriever's DB-level label filter never activated, and wrong-category chunks leaked through (e.g. tomato pest chunks returned for an onion query).

Root cause (confirmed via live probes)

  • chatqna align_inputs sets inputs["context"]
  • Orchestrator POST body has context (probe DBG2_PRE_POST: True)
  • Retriever's parsed input does not have context (hasattr=False, input_module=__main__)
  • The OPEA MicroService's dynamic type creation drops non-standard fields

Fix — encode filter labels in search_start

search_start is a standard EmbedDoc string field that survives the framework's parsing. Encode filter labels in it:

Format: {base_mode}::labels:{label1},{label2},...
Example: chunk::labels:Onion,Vegetables

The retriever decodes this before any search-mode reads → builds labels_to_filter → the existing DB-level label filter (BM25 aql_filter_clause + dense post-filter) activates → pre-top-K filtering → only label-matching chunks retrieved.

Files changed

File Change
core/label_contract.py Newencode_filter_labels() + decode_filter_labels() (the contract, documented)
chatqna/genieai_chatqna.py Encode categoryLabel + serviceLabels into search_start (align_inputs, RETRIEVER node)
retriever/genieai_retriever_arangodb.py Decode search_start before mode selection + dense post-filter (defense-in-depth)
retriever/genieai_retriever_microservice.py GenieEmbedDoc as sole input type (was a Union of 5 — simpler + correct for GENIE)
tests/test_label_contract.py New — 18 tests (encode, decode, roundtrip, edge cases)
tests/test_retriever.py Fix mock doc chunk_labels for dense post-filter

Verified

  • 18/18 label-contract tests pass
  • ruff check + format clean
  • Deployed + tested on el-salvador: onion query → all 4 candidates are onion (zero tomato)
Edited by Jérôme Revillard

Merge request reports

Loading