Skip to content

Story 1.7: Configure CI Caching and Path-Based Triggers

Sprint Key: 1-7-configure-ci-caching-and-path-based-triggers Epic: 1 PRD: testing-framework

Story 1.7: Configure CI Caching and Path-Based Triggers

Status: ready-for-dev

Story

As a developer, I want CI jobs to use caching and only run when relevant files change, So that pipeline execution is fast and efficient.

Acceptance Criteria

  1. AC1 — Path-based isolation: When a merge request modifies only components/gov-chat-backend/, only lint:backend, test:backend, and contract:backend run (not frontend, Python, mobile, etc.)
  2. AC2 — Node.js caching: npm ci uses cached node_modules keyed on package-lock.json hash with fallback keys for cache misses
  3. AC3 — Python caching: Python jobs use cached .venv keyed on pyproject.toml hash; Python lint uses cached pip packages
  4. AC4 — Flutter caching: Flutter jobs use cached SDK, .dart_tool/, and .pub-cache/
  5. AC5 — Full suite on main: Full suite runs on main branch pushes regardless of path changes
  6. AC6 — Pipeline time budget: Total mandatory pipeline time is under 10 minutes (NFR1)
  7. AC7 — No duplicate pipelines: MR pipelines do not trigger duplicate branch pipelines

Tasks / Subtasks

  • Task 1: Audit existing caching and path-based triggers (AC: #1 (closed)–#6 (closed))

    • 1.1 Verify all jobs have rules:changes with correct path globs
    • 1.2 Verify all jobs have cache with correct key files and paths
    • 1.3 Document findings and gaps

  • Task 2: Add workflow: rules to prevent duplicate pipelines (AC: #7 (closed))

    • 2.1 Add top-level workflow: rules to suppress branch pipelines when MR pipeline exists
    • 2.2 Verify scheduled/manual pipelines still work

  • Task 3: Add caching to lint:python (AC: #3 (closed))

    • 3.1 Add pip cache to lint:python job for ruff package
    • 3.2 Key on ruff version or pip requirements

  • Task 4: Add fallback cache keys to all jobs (AC: #2 (closed), #3 (closed), #4 (closed))

    • 4.1 Add cache.key.prefix fallback to node jobs (use $CI_DEFAULT_BRANCH fallback)
    • 4.2 Add fallback to Python .venv cache
    • 4.3 Verify Flutter fallback already works via .flutter_base

  • Task 5: Add interruptible: true to standalone jobs (AC: #6 (closed))

    • 5.1 Add interruptible: true to lint:python (standalone, no template)
    • 5.2 Add interruptible: true to test:python (standalone, no template)

  • Task 6: Verify pipeline performance (AC: #6 (closed))

    • 6.1 Document expected parallel execution times per stage
    • 6.2 Confirm total mandatory pipeline (lint + test + contract + config) fits < 10 min budget

  • Task 7: Update .gitlab-ci.yml and validate (AC: all)

    • 7.1 Apply all changes to .gitlab-ci.yml
    • 7.2 Validate YAML syntax (gitlab-ci-lint or manual check)
    • 7.3 Run npm run lint in all modified components to verify no regressions

Dev Notes

Current State Analysis

Most ACs are already satisfied by stories 1-2 through 1-5. Each story added its jobs with caching and path-based triggers already configured. The current .gitlab-ci.yml (450 lines) has:

  • Stages: lint → test → contract → config → e2e
  • 5 lint jobs: backend, frontend, doc-repo, python, dart — all have path-based triggers ✅
  • 6 test jobs: backend, frontend, doc-repo, python, flutter — all have caching ✅
  • 2 contract jobs: backend, doc-repo — all have path-based triggers ✅
  • 1 config job: config:validate — has path-based triggers ✅
  • 1 e2e job: patrol:e2e — has path-based triggers ✅

Gaps Found (This Story's Actual Work)

Gap 1: lint:python has NO cache (AC3)

lint:python:
  image: python:3.10-slim
  stage: lint
  interruptible: true  # present ✅
  before_script:
    - pip install ruff  # installs every run — no cache

Fix: Add pip cache keyed on ruff version or a stable key.

Gap 2: No workflow: rules — duplicate pipelines (AC7) Without workflow: rules, a push to a branch that has an open MR triggers TWO pipelines: one from the push, one from the MR. This wastes runner resources and slows feedback. Fix: Add top-level workflow: rules to run pipelines only on MRs and main branch.

Gap 3: No fallback cache keys (AC2, AC3, AC4) Current cache keys are exact-match only. If the key doesn't exist (first run, cache evicted), jobs start from zero. GitLab CI supports fallback keys via cache:key:prefix combined with $CI_DEFAULT_BRANCH. Fix: Add fallback keys where applicable.

Gap 4: test:python missing interruptible: true The test:python job doesn't extend any template with interruptible: true. If a newer commit is pushed, this job can't be cancelled mid-run. Fix: Add interruptible: true.

Cache Strategy Summary (Established by Stories 1-2 to 1-5)

Job Cache Key Paths Prefix
lint:backend package-lock.json node_modules/ lint-backend
lint:frontend package-lock.json node_modules/ lint-frontend
lint:doc-repo package-lock.json node_modules/ lint-doc-repo
lint:python NONE — —
lint:dart pubspec.lock (via .flutter_base) .dart_tool/, .pub-cache/ —
test:backend package-lock.json node_modules/ test-backend
test:frontend package-lock.json node_modules/ test-frontend
test:doc-repo package-lock.json node_modules/ test-doc-repo
test:python pyproject.toml .venv/ test-python
flutter:test pubspec.lock (via .flutter_base) .dart_tool/, .pub-cache/ —
contract:backend package-lock.json node_modules/ contract-backend
contract:doc-repo package-lock.json node_modules/ contract-doc-repo
config:validate package-lock.json node_modules/ config-validate

Cache prefix convention: <stage>-<component> prevents cross-stage cache pollution. Different stages may install different subsets of dependencies.

Path-Based Trigger Summary (Established by Stories 1-2 to 1-5)

All jobs use this rules pattern:

rules:
  - if: '$CI_PIPELINE_SOURCE == "merge_request_event"'
    changes:
      - "<component-path>/**/*"
      - ".gitlab-ci.yml"
    when: on_success
  - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'
    when: on_success
Job changes Paths
lint:backend components/gov-chat-backend/**/*
lint:frontend components/gov-chat-frontend/**/*
lint:doc-repo components/document-repository/**/*
lint:python genie-ai-overlay/**/*.py, pyproject.toml
lint:dart mobile/genie_ai_mobile/**/*
test:backend components/gov-chat-backend/**/*
test:frontend components/gov-chat-frontend/**/*
test:doc-repo components/document-repository/**/*
test:python genie-ai-overlay/**/*.py, pyproject.toml
flutter:test mobile/genie_ai_mobile/**/*
contract:backend components/gov-chat-backend/**/*
contract:doc-repo components/document-repository/**/*
config:validate env, env., docker-compose.yaml, tests/config-validator/**/, tests/fixtures/config/**/*

All jobs also include .gitlab-ci.yml in their changes list and run on main branch pushes.

Hidden Templates (Do Not Modify)

.node_base    → image: node:20-alpine, interruptible: true
.lint_node    → extends .node_base, stage: lint
.test_node    → extends .node_base, stage: test, NODE_ENV: test, artifacts defaults
.contract_node → extends .node_base, stage: contract, NODE_ENV: test, artifacts defaults
.flutter_base → image: debian:bookworm-slim, Flutter SDK install with cache

Pipeline Performance Budget (NFR1: < 10 minutes)

Mandatory stages are lint → test → contract → config. Within each stage, jobs run in parallel.

Stage Longest Job (estimated) Notes
lint ~1 min (Flutter SDK install on cache miss) All parallel
test ~2-3 min (Flutter or Python) All parallel, cache helps
contract ~1 min 2 parallel jobs
config ~30 sec File I/O only
Total ~4-5 min Well within 10 min budget

With caching, npm ci becomes a no-op on cache hit, keeping most jobs under 1 minute.

Project Structure Notes

  • Single file to modify: .gitlab-ci.yml at repository root
  • No new files created — purely CI configuration optimization
  • All existing jobs, templates, and stages remain unchanged
  • Only additive changes: workflow: section, cache additions, interruptible flag

References

  • [Source: _bmad-output/planning-artifacts/epics.md#Epic 1 Story 1.7]
  • [Source: _bmad-output/planning-artifacts/architecture.md#CI/CD Pipeline Architecture]
  • [Source: _bmad-output/implementation-artifacts/1-5-create-ci-pipeline-configuration-validation-stage.md#Dev Notes]
  • [Source: _bmad-output/implementation-artifacts/1-4-create-ci-pipeline-contract-test-stage.md#Dev Notes]
  • [Source: _bmad-output/implementation-artifacts/1-3-create-ci-pipeline-test-stage.md#Dev Notes]
  • [Source: .gitlab-ci.yml — current 450-line pipeline]

Dev Agent Record

Agent Model Used

(To be filled during implementation)

Debug Log References

Completion Notes List

File List

Edited by Jérôme Revillard