feat(el-salvador): deployment

Summary

This MR introduces a custom chunking strategy specifically designed for Markdown (.md) files in the dataprep service. Instead of relying on generic line-based or character-based splitting, it groups headings with their following text blocks and packs them dynamically into chunks based on estimated token limits.

Changes

• genie-ai-overlay/dataprep/genieai_dataprep_arangodb.py:
  • Implemented _chunk_markdown_by_paragraphs which uses greedy paragraph packing.
  • Ensures headings (e.g., #, ##) remain glued to their following text blocks rather than getting separated during chunking.
  • Splits oversized paragraphs exceeding a 1000-token threshold into smaller, grammatically clean sub-chunks.
  • Configured .md file path route to utilize this dynamic chunking approach.
• genie-ai-overlay/tests/test_dataprep.py:
  • Added test_custom_markdown_paragraph_chunking to verify normal paragraph packing and heading preservation.
  • Added test_custom_markdown_chunking_token_limits to assert that token threshold limits are respected and oversized paragraphs are safely split.

Verification Plan

Run the automated pytest suite to ensure the new chunking algorithm operates as expected and does not break existing test cases: