Skip to content
  • Jérôme Revillard's avatar
    feat(defaults): per_chunk contextual + slice reranker top-3 · e4adf4d0
    Jérôme Revillard authored
    Change proven defaults from el-salvador R&D testing:
    
    CONTEXTUAL_STRATEGY: doc_level -> per_chunk
      Anthropic recipe — section-tailored context per chunk (1 LLM call/chunk
      but better embedding diversity than doc_level).
    
    DATAPREP_CONTEXTUAL_DOC_BUDGET: 6000 -> 100000
      per_chunk needs the full doc visible per call (was tuned for doc_level).
    
    RERANKING_STRATEGY: adaptive -> slice
      Adaptive's multiplicative novelty formula was structurally flawed
      (novelty cliff). Slice top-N by TEI score dominates on both recall
      AND precision. Offline sweep: slice recall 0.91 vs adaptive 0.69.
    
    RERANKER_TOP_N: 1 -> 3
      Balanced compromise: top-1 = precision king, top-5 = recall king,
      top-3 = best of both. Also aligned chatqna default (was 2).
    
    Updated: 4 code files, 2 test files, .gitignore for .tokensave/
    e4adf4d0
Loading