Skip to content

RAG/Refactor: Implement Context-Aware Vector Retrieval for Multi-Turn Dialogues

📋 Problem Statement

In multi-turn Retrieval-Augmented Generation (RAG) pipelines, the vector search component is currently entirely stateless regarding conversation history. When a user asks a follow-up question containing deictic or pronoun-heavy terms (e.g., "it," "that," "they," "here," "can you elaborate more on this?"), the isolated query lacks explicit semantic meaning.

Feeding only this standalone, extracted phrase into our embedding model yields poor document retrieval because the vector database is blind to the established conversational context. Conversely, concatenating the entire raw conversation history introduces heavy semantic noise, causing "history drift" and prompting the system to retrieve irrelevant document chunks.

🔍 Current Technical Analysis

The GENIE.AI ChatQnA pipeline currently isolates the latest query from the historical context right before the retrieval process starts:

  1. Isolation of User Query: Inside ChatQnAService.handle_request, the application flattens the chat history and uses a regex pattern (USER_MSG_PATTERN) to extract only the last translated message content:
user_messages = USER_MSG_PATTERN.findall(translated_history_string)
if user_messages:
    last_translated_message_content = user_messages[-1].strip()
  1. Scheduling Entry Point: This isolated string is mapped as the only text search input into the MegaService scheduler:
initial_inputs={"text": last_translated_message_content}
  1. Naive Alignment for Embedding: Within the align_inputs function, when mapping payloads for the ServiceType.EMBEDDING microservice, the code directly assigns this isolated string to the embedder without context enrichment:
elif self.services[cur_node].service_type == ServiceType.EMBEDDING:
    inputs["input"] = inputs["text"]
    del inputs["text"]

Because the EMBEDDING and RETRIEVER steps only see last_translated_message_content, the system fails to capture multi-turn semantic relationships.

🛠 Target Architecture & Affected Components

To fix this limitation, some form of LLM-based enrichment approaches or more lightweight algorithmic context refinement strategies might be implemented prior to running the vector search.

Components Requiring Modifications:

  • genieai_chatqna.py -> align_inputs(): Could be modified to inject context into the embedding input dynamically if we use vector arithmetic or structural manipulation.

  • genieai_chatqna.py -> ChatQnAService.handle_request(): Could be intercepted right before scheduling megaservice.schedule to execute text-level query enrichment or coreference resolution.

  • Pipeline Configuration (add_remote_service_genieai): If we decide to use an architectural solution, we may need to introduce a new query pre-processing step or microservice to the graph structure.

🎯 Definition of Done (DoD)

  • Research and benchmark LLM-based and lightweight algo-based context preservation methods.
  • Implement an approved context-aware query refinement mechanism into genieai_chatqna.py.
  • Verify that ambiguous follow-up questions (e.g., "Can you elaborate on this?") retrieve documents relevant to the previous turn's subject matter.
  • Ensure there is no significant regression in response times or token bloat within resource-constrained test beds.