Story 4.2: Test Retriever Hybrid Search Logic

Sprint Key: 4-2-test-retriever-hybrid-search-logic Epic: 4 PRD: testing-framework

Story 4.2: Test Retriever Hybrid Search Logic

Status: ready-for-dev

Story

As a developer, I want pytest tests for the retriever's hybrid search (vector + graph + labels), so that retrieval logic is validated without real ArangoDB or embedding services.

Acceptance Criteria

AC1: Test file created — genie-ai-overlay/tests/test_retriever.py is created with ITU copyright header, covering GenieaiArangoRetriever from genie-ai-overlay/retriever/genieai_retriever_arangodb.py.
AC2: Hybrid search combination — Tests verify that invoke() combines vector similarity search, graph traversal (fetch_neighborhoods), and label filtering in a single call when enable_traversal=True and labels are provided.
AC3: Query construction for search modes — Tests verify _build_subquery() produces correct AQL for all three search_start modes: "node", "edge", and "chunk". Each mode produces structurally different AQL queries. Custom traversal_query overrides the built-in query.
AC4: Score threshold filtering — Tests verify that fetch_neighborhoods() filters results below traversal_score_threshold. Invalid distance_strategy values (not "COSINE" or "EUCLIDEAN_DISTANCE") raise HTTPException(400).
AC5: Pagination and top-K — Tests verify invoke() passes k, fetch_k, lambda_mult parameters to vector search methods (asimilarity_search_with_relevance_scores, amax_marginal_relevance_search, asimilarity_search). Tests verify LIMIT in AQL subqueries respects traversal_max_returned.
AC6: Empty ArangoDB results — Tests verify invoke() returns [] when ArangoDB returns empty collections (collection_count == 0), missing graph (has_graph() == False), missing collection, or missing embedding field on documents.
AC7: ArangoDB ConnectionError — Tests verify check_health() returns False when ArangoDB throws an exception. Tests verify invoke() returns [] (not raises) when embedding generation or vector search fails.
AC8: Label filter AQL construction — Tests verify the AQL filter clause is constructed correctly for both "OR" and "AND" filter strategies. Invalid filter_strategy raises HTTPException(400). Empty labels produce no filter clause.
AC9: Summarization prompt — Tests verify generate_summarization_prompt() produces a prompt containing the query and document text.
AC10: All services mocked — ArangoDB (arango), LangChain (langchain_arangodb, langchain_community, langchain_huggingface, langchain_openai), OpenAI (openai), and comps are fully mocked. No real external connections. All tests pass Ruff linting and formatting.

Tasks / Subtasks

Dev Notes

Critical: Cannot Instantiate Retriever Normally

GenieaiArangoRetriever.__init__ calls _initialize_client() which creates a real ArangoDB connection (ArangoClient(hosts=ARANGO_URL)) and optionally _initialize_llm() which creates a real LLM client. These MUST be bypassed for testing.

Approach: Patch _initialize_client and _initialize_llm as no-ops, then manually set self.db to a mock:

def create_retriever(db_mock=None, with_llm=False):
    """Create a GenieaiArangoRetriever with mocked dependencies."""
    with patch.object(GenieaiArangoRetriever, "_initialize_client"):
        with patch.object(GenieaiArangoRetriever, "_initialize_llm"):
            retriever = GenieaiArangoRetriever(
                name="test-retriever",
                description="Test retriever",
            )
    retriever.db = db_mock or MagicMock()
    if with_llm:
        retriever.llm = MagicMock()
    return retriever

Critical: Import-Time Dependencies Must Be Pre-Populated

The retriever module imports these at the top level (module-level, not lazy). If pytest collects the test file, Python evaluates the retriever's imports, which will fail without mocks:

# From genieai_retriever_arangodb.py — import-time deps:
from arango import ArangoClient                           # needs mock
from arango.database import StandardDatabase              # needs mock
from comps import CustomLogger, EmbedDoc, ...             # already mocked in conftest.py
from comps.cores.proto.genieai_api_protocol import ...    # already mocked in conftest.py
from langchain_arangodb import ArangoVector               # needs mock
from langchain_community.embeddings import HuggingFaceBgeEmbeddings  # needs mock
from langchain_huggingface import HuggingFaceEndpointEmbeddings      # needs mock
from langchain_openai import ChatOpenAI, OpenAIEmbeddings            # needs mock
import openai                                              # needs mock

Must add to conftest.py or a test-local setup:

# Add AFTER existing sys.modules entries in conftest.py:
sys.modules.setdefault("langchain_arangodb", MagicMock())
sys.modules.setdefault("langchain_community", MagicMock())
sys.modules.setdefault("langchain_community.embeddings", MagicMock())
sys.modules.setdefault("langchain_huggingface", MagicMock())
sys.modules.setdefault("langchain_openai", MagicMock())
sys.modules.setdefault("openai", MagicMock())

CRITICAL: These must be added to the existing conftest.py sys.modules block (lines 14-26), NOT in the test file. Pytest collects conftest.py before test files, so the mocks must be in place at collection time. Adding them in the test file's module-level code is too late — pytest will have already tried to import the retriever module.

Retriever Method Architecture

GenieaiArangoRetriever
├── __init__(name, description, config)
│   ├── _initialize_client()     # ArangoDB connection — MUST PATCH
│   └── _initialize_llm()        # LLM connection — MUST PATCH (only if SUMMARIZER_ENABLED)
├── check_health() → bool         # db.version() check
├── generate_summarization_prompt(query, text) → str  # Pure string formatting
├── _build_subquery(...) → str    # Pure AQL string builder
├── fetch_neighborhoods(...) → dict  # Graph traversal (sync, uses ThreadPoolExecutor)
└── invoke(input) → list          # Main async entry point (200+ lines)

invoke() Flow (the 200-line method)

Understanding the invoke() flow is essential for writing correct tests:

1. Parse input (model_dump) → extract query, search params, labels
2. Build AQL filter clause from labels (OR/AND strategy)
3. Validate graph and collection existence (db.has_graph, has_vertex_collection)
4. Check collection count > 0 and >= num_centroids
5. Get embedding dimension from random document
6. Initialize embeddings (OpenAI > TEI > HuggingFace fallback chain)
7. Create ArangoVector instance
8. Compute embedding for query (if not pre-embedded)
9. Execute vector search (similarity_score_threshold / mmr / similarity)
10. Enrich results with file_id metadata (AQL lookup, chunk mode only)
11. Optionally traverse graph neighborhoods (enable_traversal)
12. Optionally summarize results (enable_summarizer)
13. Return search_res list

Key early-return paths in invoke():

Empty query → []
Missing graph → [] (logged error)
Missing collection in graph → [] (logged error)
Empty collection (count=0) → [] (logged error)
Collection < num_centroids → [] (logged error)
Missing embedding field → [] (logged error)
Zero dimension → [] (logged error)
Embedding generation failure → [] (logged error)
ArangoVector init failure → [] (logged error)
Vector search failure → [] (logged error)
Empty search results → [] (logged info)

ALL error paths return [], never raise. This is a critical behavior to test.

The `input` Object Shape

invoke() accepts ChatCompletionRequest | RetrievalRequest | RetrievalRequestArangoDB | GenieEmbedDoc. For tests, use MagicMock with .model_dump() returning a dict with these keys:

def create_mock_input(query="test query", **overrides):
    """Create a mock input object matching invoke() expectations."""
    mock = MagicMock()
    defaults = {
        "input": query,
        "text": query,
        "embedding": None,
        "graph_name": "GRAPH",
        "search_start": "node",
        "search_mode": "vector",
        "enable_traversal": False,
        "enable_summarizer": False,
        "distance_strategy": "COSINE",
        "use_approx_search": False,
        "num_centroids": 1,
        "traversal_max_depth": 1,
        "traversal_max_returned": 3,
        "traversal_score_threshold": 0.5,
        "traversal_query": None,
        "context": {},
        "filter_strategy": "OR",
        "search_type": "similarity_score_threshold",
        "k": 4,
        "fetch_k": 20,
        "lambda_mult": 0.5,
        "score_threshold": 0.5,
    }
    defaults.update(overrides)
    mock.model_dump.return_value = defaults
    mock.embedding = overrides.get("embedding", None)
    mock.search_type = defaults["search_type"]
    mock.k = defaults["k"]
    mock.fetch_k = defaults["fetch_k"]
    mock.lambda_mult = defaults["lambda_mult"]
    mock.score_threshold = defaults["score_threshold"]
    return mock

Mocking ArangoVector for invoke() Tests

invoke() creates ArangoVector(...) inline. The test must patch the class to return a mock with async methods:

mock_vector_db = AsyncMock()
mock_vector_db.db = db_mock
mock_vector_db.asimilarity_search_with_relevance_scores = AsyncMock(return_value=[])
mock_vector_db.amax_marginal_relevance_search = AsyncMock(return_value=[])
mock_vector_db.asimilarity_search = AsyncMock(return_value=[])

with patch("genieai_retriever_arangodb.ArangoVector", return_value=mock_vector_db):
    ...

IMPORTANT: The import in the retriever is from langchain_arangodb import ArangoVector. Since langchain_arangodb is mocked at sys.modules level, ArangoVector from the mock will be a MagicMock(). You need to patch the specific reference in the retriever module's namespace: genieai_retriever_arangodb.ArangoVector.

Embedding Provider Fallback Chain

invoke() selects embeddings provider in this order:

OPENAI_API_KEY + OPENAI_EMBED_ENABLED → OpenAIEmbeddings
TEI_EMBEDDING_ENDPOINT + HF_TOKEN → HuggingFaceEndpointEmbeddings
Default → HuggingFaceBgeEmbeddings

Since all langchain modules are mocked, the constructor calls won't fail, but you should be aware of which path tests take based on env vars set by set_env_vars autouse fixture.

Threading in fetch_neighborhoods

fetch_neighborhoods() has two code paths:

max_workers == 1 (default): Single AQL query with all keys
max_workers > 1: ThreadPoolExecutor, one query per key

The ARANGO_TRAVERSAL_CONCURRENT_BATCHES env var (default: 1) controls this. Tests should cover both paths by patching the config import or setting the env var.

Worker count validation:

> 4 → capped to 4, logs error
< 1 → defaulted to 1, logs error
== 1 → single-query mode, logs info

Story 4.1 Review Findings (Deferred — May Need Attention)

From the 4-1 code review:

Missing comps submodule mocks for comps.retrievers.src.* and comps.rerankings.src.* — may need to add if retriever microservice imports them. The main retriever module (genieai_retriever_arangodb.py) does NOT import from comps.retrievers, so this should be fine.
Mock response shapes may need dict-access support — retriever code uses r["doc"].page_content and r["doc"].id (attribute access). Current mocks use MagicMock which supports both. Verify this works.

Files to Create/Modify

File	Action
`genie-ai-overlay/tests/conftest.py`	MODIFY — add `langchain_*` and `openai` to sys.modules pre-population
`genie-ai-overlay/tests/test_retriever.py`	NEW — all retriever tests

Files NOT Modified

All existing Python service files in genie-ai-overlay/retriever/ remain unchanged.

Project Structure Notes

Retriever source: genie-ai-overlay/retriever/genieai_retriever_arangodb.py (854 lines)
Retriever config: genie-ai-overlay/retriever/config.py (env var definitions)
Retriever microservice wrapper: genie-ai-overlay/retriever/genieai_retriever_microservice.py (not tested here — HTTP wrapper, tested in story 4.6)
Protocol models: genie-ai-overlay/core/genieai_api_protocol.py (Pydantic models)
Test location: genie-ai-overlay/tests/test_retriever.py
Ruff config: genie-ai-overlay/pyproject.toml (target-version py310, line-length 120, quote-style double)

References

[Source: _bmad-output/planning-artifacts/epics.md#Story 4.2] — Original story definition and acceptance criteria
[Source: _bmad-output/planning-artifacts/architecture.md#OPEA Testing] — pytest patterns, mock strategy, fixture design, test naming conventions
[Source: _bmad-output/implementation-artifacts/4-1-configure-pytest-and-create-shared-fixtures-for-opea.md] — Previous story: conftest.py design, sys.modules mock strategy, deferred findings
[Source: genie-ai-overlay/retriever/genieai_retriever_arangodb.py] — Source under test (854 lines)
[Source: genie-ai-overlay/retriever/config.py] — Env var configuration (ARANGO_TRAVERSAL_, ARANGO_SEARCH_, embedding config)
[Source: genie-ai-overlay/core/genieai_api_protocol.py] — Pydantic request models (RetrievalRequestArangoDB, ChatCompletionRequest)
[Source: genie-ai-overlay/tests/conftest.py] — Existing shared fixtures (mock_arangodb, mock_comps, mock_vllm, mock_tei)
[Source: _bmad-output/project-context.md#Python OPEA Services] — PEP 8, ruff, CustomLogger, copyright headers

Story 4.2: Test Retriever Hybrid Search Logic

Story 4.2: Test Retriever Hybrid Search Logic

Story

Acceptance Criteria

Tasks / Subtasks

Dev Notes

Critical: Cannot Instantiate Retriever Normally

Critical: Import-Time Dependencies Must Be Pre-Populated

Retriever Method Architecture

invoke() Flow (the 200-line method)

The `input` Object Shape

Mocking ArangoVector for invoke() Tests

Embedding Provider Fallback Chain

Threading in fetch_neighborhoods

Story 4.1 Review Findings (Deferred — May Need Attention)

Files to Create/Modify

Files NOT Modified

Project Structure Notes

References

Dev Agent Record

Agent Model Used

Debug Log References

Completion Notes List

File List

Story 4.2: Test Retriever Hybrid Search Logic

Story 4.2: Test Retriever Hybrid Search Logic

Story

Acceptance Criteria

Tasks / Subtasks

Dev Notes

Critical: Cannot Instantiate Retriever Normally

Critical: Import-Time Dependencies Must Be Pre-Populated

Retriever Method Architecture

invoke() Flow (the 200-line method)

The input Object Shape

Mocking ArangoVector for invoke() Tests

Embedding Provider Fallback Chain

Threading in fetch_neighborhoods

Story 4.1 Review Findings (Deferred — May Need Attention)

Files to Create/Modify

Files NOT Modified

Project Structure Notes

References

Dev Agent Record

Agent Model Used

Debug Log References

Completion Notes List

File List

The `input` Object Shape