Skip to content

Story 1.5: Write contract tests green on v1.3, prove red on a bare v1.5 bump

Sprint Key: 1-5-write-contract-tests-green-on-v1-3-prove-red-on-a-bare-v1-5-bump Epic: 1 PRD: opea-1.5-upgrade

Story 1.5: Write contract tests green on v1.3, prove red on a bare v1.5 bump

Status: ready-for-dev

Story

As a platform engineer, I want the contract-test suite written and red-green validated against real comps, So that the safety net is proven to catch a real 1.5 break before re-grafting.

Acceptance Criteria

  1. Suite targets real comps, not the mocked conftest. The suite contains: orchestrator wire test, one-doc ingest with production config, focused label-filter test, telemetry-from-dashboards assertion, and one E2E cross-service pipeline test (PRD FR-10, architecture pattern 3). It runs inside the built image (architecture D3 — isolated from the sys.modules-mocked conftest.py), via docker run <image> pytest /contracts/... -p no:cacheprovider.
  2. Green on v1.3. The suite passes against the current OPEA_VERSION="v1.3" images (post-1.4 cleanup) with model/DB endpoints HTTP-mocked, no GPU.
  3. Red on a bare v1.5 bump — proven, evidence committed. Before any re-graft, the suite is re-run against the module images built with OPEA_VERSION="v1.5" (bare bump — no overlay re-graft, no shim). It goes red, and the failure reason (which contract broke, which assertion, which exception) is committed as a CI artifact.
  4. Sensitivity check — no green-on-green. Every test asserts a v1.5-specific shape (a surface the bump actually changes): the docarray rename (docarray.py→opea_docarray.py), the compiled-lock/REQ_PATH change, Python 3.11 sitecustomize path, schedule() kwargs forwarding, langchain-arangodb filter semantics, telemetry span/attribute names. A test that would pass identically on both versions does not count.

Tasks / Subtasks

  • T1: Establish the genie-ai-overlay/contracts/ suite skeleton + in-image harness (AC: 1, 2)
  • T2: Orchestrator wire test (highest ROI, FR-10 first bullet) (AC: 1, 3, 4)
  • T3: One-doc ingest smoke with production config (FR-10 second bullet) (AC: 1, 3, 4)
  • T4: Focused label-filter test (FR-10 third bullet) (AC: 1, 3, 4)
  • T5: Telemetry assertion derived from Grafana dashboard provisioning (FR-10 fourth bullet, NFR-T1) (AC: 1, 3, 4)
  • T6: E2E cross-service pipeline contract test (FR-10 fifth bullet, architecture pattern 3) (AC: 1, 3, 4)
  • T7: NFR-P coarse budgets (FR-10, architecture D6) (AC: 1, 3, 4)
  • T8: Red-green validation + evidence (AC: 2, 3, 4)
Edited by Jérôme Revillard