Index Data and Create the Chatbot Database
This issue focuses on developing and implementing a robust data indexing pipeline for RAG to organize the knowledge base and facilitate retrieval for the chatbot. The pipeline should be aligned with the high-level design proposed by the working group (combining vector database and knowledge graph database).
The intention is to use ArangoDB as a single database with vector search and knowledge graph capability. It will be required to select an embedding model for vector embeddings and an LLM for knowledge graph (entities and relationships extraction).
Links that might be useful:
Edited by Chiara MARGARI