Skip to content

fix(dataprep): remove dead uploadDate from ingest contract (422 → 500 on re-ingest)

Jérôme Revillard requested to merge fix/ingest-upload-date-422 into main

Problem

Re-ingesting a document returned HTTP 500 in the frontend:

api/files/<id>/ingest  → 500
document-repository: "Ingest file error: Request failed with status code 422"
dataprep: POST /v1/dataprep/ingest_file → 422 Unprocessable Entity

Root cause

The document-repository forwarded uploadDate: file.uploaded_date to the dataprep ingest endpoint, but DocRepoIngestPayload required uploadDate: str. On the El Salvador deployment 106/107 files have uploaded_date == null (seeded outside the app), so pydantic rejected the payload → 422, which the document-repository wrapped as 500.

upload_date is dead in dataprep: accepted by the request model and passed to ArangoDBDataprepRequestFromDocRepo, but never read by ingest_file_with_guardrail nor stored on chunk/document nodes. It is document-repository metadata that does not belong in the ingest contract.

Fix

  • Remove uploadDate from DocRepoIngestPayload
  • Stop forwarding it from the document-repository _ingestFileById
  • Add regression test (TestDocRepoIngestPayload) asserting the payload validates without uploadDate and tolerates a null value

Generic fix (from main). El Salvador will be rebased on top.

Verification

  • ruff check + format: clean
  • pytest overlay: 459/459
  • eslint doc-repo: clean
  • jest doc-repo: 400/400

Merge request reports