Skip to content

feat(observability): add Grafana alerting rules, dashboard, and SLO provisioning (story 7-11)

Jérôme Revillard requested to merge feat/testing-framework/7-11-observability-slos into feat/testing-framework/prd

Story 7.11: Observability SLOs

Meta-monitoring for the observability stack — "who watches the watchers".

Changes

  • 4 alert rules: OTel Collector Down (critical), VM Storage High, VLogs Ingestion Drop, VTraces Export Failures (warning)
  • Dashboard: Observability Stack Health with 8 panels (ingestion, storage, latency, health checks)
  • Notification routing: contact point + notification policy
  • Infrastructure: uid added to vm-datasource.yml, Ansible vars
  • Tests: 63 unit tests — all pass
  • Docs: retention policy + alerting documentation

Closes #820 (closed)

Merge request reports