Skip to content

feat: Implement RAG evaluation logging pipeline

Edouard Legoupil requested to merge feature-semantic-chunking into main_dev

Created by: Edouard-Legoupil

  • Adds a new rag_evaluation_logs table to the database to store RAG query data.

  • Modifies the VectorSearchTool to log the query and retrieved context to this new table.

  • Implements a log_rag_output callback function to capture the final generated answer from the agent.

  • Attaches this callback to the write_task in the ContentGenerationCrew.

  • Updates the accuracy_improvement_plan.md with a detailed, actionable guide for this implementation.

  • This provides a complete, end-to-end pipeline for logging RAG operations, enabling "LLM as a Judge" evaluation.

  • Does my code meet the quality standards for releasing packages?

  • Does the reviewer have all the information to validate the features/issues without too much research?

  • Does the customer who will validate the associated tickets have the information to do so without wasting time?

Issues to validate to close :

  • issue #

Processed issues to keep open or in progress:

  • issue #

Checklist:

  • Does the package check go local?
  • Does the CI pass?
  • Are the added / fixed features documented, tested?
  • Are the added features / solved problems briefly presented in the PR message?
  • Are the changes related to tickets / issues that I have listed in the commits and in the PR itself?
  • Are the tickets in "review" mode in the Project Tracking Board?
  • Does each ticket, if it is to be closed after acceptance of the PR, contain a comment that tells how to validate it?

Merge request reports