fix(retriever): label filter via search_start data contract
Problem
The OPEA MicroService framework silently drops custom fields (like context) between mega-service nodes — it creates a dynamic __main__ input type that only preserves standard EmbedDoc fields. As a result, the retriever's DB-level label filter never activated, and wrong-category chunks leaked through (e.g. tomato pest chunks returned for an onion query).
Root cause (confirmed via live probes)
- chatqna
align_inputssetsinputs["context"]✅ - Orchestrator POST body has context (probe
DBG2_PRE_POST: True)✅ - Retriever's parsed input does not have context (
hasattr=False,input_module=__main__)❌ - The OPEA MicroService's dynamic type creation drops non-standard fields
Fix — encode filter labels in search_start
search_start is a standard EmbedDoc string field that survives the framework's parsing. Encode filter labels in it:
Format: {base_mode}::labels:{label1},{label2},...
Example: chunk::labels:Onion,Vegetables
The retriever decodes this before any search-mode reads → builds labels_to_filter → the existing DB-level label filter (BM25 aql_filter_clause + dense post-filter) activates → pre-top-K filtering → only label-matching chunks retrieved.
Files changed
| File | Change |
|---|---|
core/label_contract.py |
New — encode_filter_labels() + decode_filter_labels() (the contract, documented) |
chatqna/genieai_chatqna.py |
Encode categoryLabel + serviceLabels into search_start (align_inputs, RETRIEVER node) |
retriever/genieai_retriever_arangodb.py |
Decode search_start before mode selection + dense post-filter (defense-in-depth) |
retriever/genieai_retriever_microservice.py |
GenieEmbedDoc as sole input type (was a Union of 5 — simpler + correct for GENIE) |
tests/test_label_contract.py |
New — 18 tests (encode, decode, roundtrip, edge cases) |
tests/test_retriever.py |
Fix mock doc chunk_labels for dense post-filter |
Verified
- 18/18 label-contract tests pass
✅ - ruff check + format clean
✅ - Deployed + tested on el-salvador: onion query → all 4 candidates are onion (zero tomato)
✅
Edited by Jérôme Revillard