Skip to content

fix(el-salvador): updates to formulas in adaptive reranker strategy

Roman CHESTNOV requested to merge fix/reranker-adaptive-relevance-cost-fnc-adjustment into el-salvador-contextual

Reranker: Formulas used to calculate relevance and confusion cost as part of the adaptive chunk selection strategy have been updated to stabilise Value function behaviour.

Relevance calculation formula has been updated by wrapping the skew in a normalisation function tanh(x): relevance = score + (score - avg_score + skew) changed to relevance = score + (score - avg_score + avg_score*(np.tanh(skew)). In combination with a multiplier (avg_score), this caps the value of the skew adjuster to [-avg_score; +avg_score], which prevents relevance value fluctuations at edge cases.

Denominator in the confusion cost formula has been changed from: denominator = max_score - avg_score + skew to: denominator = max_score - avg_score. This prevents confusion_cost from turning negative with extreme skew value

Merge request reports