Skip to content

fix(chatqna): handle max_tokens=None after pydantic upgrade

Jérôme Revillard requested to merge worktree-fix-max-tokens into main

Problem

After CVE pipeline rebuild, ChatQnA returns 500 on chat requests: pydantic_core.ValidationError: max_tokens — Input should be a valid integer [type=int_type, input_value=None].

MR !248 (merged) intentionally removed the hardcoded 1024 default for max_tokens, passing None to let the LLM use its own default. But the rebuild with newer pydantic (>=2.13) now rejects None for int fields on LLMParams.

Fix

Conditionally omit max_tokens from LLMParams kwargs when chat_request.max_tokens is None — same pattern as chat_template and model.

Tests

2 new tests:

  • test_max_tokens_none_omitted_from_kwargs: verifies max_tokens is NOT in LLMParams kwargs when None
  • test_max_tokens_set_included_in_kwargs: verifies max_tokens IS passed when set

Full test suite: 648 passed.

Merge request reports