food-market/docs/flaky-tests.md at 5b87b5d599734070dc37792dbd164e3a22bef73f - nns/food-market - food-market git

nns/food-market

nns cf760fab10

Auto-tag / Create date-tag (push) Waiting to run

Details

CI / Backend (.NET 8) (push) Waiting to run

Details

CI / Web (React + Vite) (push) Waiting to run

Details

CI / POS (WPF, Windows) (push) Waiting to run

Details

feat(s26): flaky-test detection + observability dashboards (8/8 ✓ 10/10 cert)

После 24 спринтов regress-suite разросся; нестабильность блокирует
доверие. Этот спринт: ловит flaky тесты, добавляет observability
(Grafana + Prometheus alerts + RUNBOOK), сертифицирует 10× cert-прогон.

1. tests/regression/find-flaky.sh — 10× прогон + JSON-агрегатор →
   docs/flaky-tests.md (per-test pass/fail sequence + reproduce).
2. OrgFactory.signupWithRetry теперь honors Retry-After header
   (api-client.ts:ApiError.retryAfterSec). Stage rate-limit поднят:
   RATE_SIGNUP_HOUR=5000, RATE_PER_IP_MIN=5000 (~/food-market-stage/deploy/.env).
3. fullyParallel=true + workers=4 = тесты идут в недетерминированном
   порядке; isolation работает (OrgFactory per-test).
4. workers=4 даёт **2.4× ускорение** (66.6s → 27.7s). Worker-scoped
   фикстура lib/worker-org.ts добавлена как opt-in.
5. deploy/grafana/dashboards/quality-watchdog.json (10 панелей:
   smoke success ratio 7d, incidents, multi-tenant violations,
   current emoji, p95 by endpoint, step failures, RPS, DB p95,
   docs posted, disk free) + dashboards/README.md.
   quality-watchdog.sh пишет Prometheus textfile экспорт в
   ~/.fm-watchdog/textfile/quality_watchdog.prom для node_exporter.
6. deploy/prometheus/alerts.yml — 10 правил, 4 группы (uptime,
   errors, database, quality-watchdog). MultiTenantViolation = P0.
   deploy/prometheus/prometheus.yml — reference config.
7. docs/RUNBOOK.md +178 строк: action per alert (api-down,
   rps-drop, http-errors-spike/growing, doc-posting-errors,
   db-p95-high, disk-free-low, watchdog-red, multi-tenant-violation,
   watchdog-incident). Junior-friendly с конкретными командами.

**Cert-прогон (10× workers=4):** 420/420 passed, 0 flaky, avg 30.1s/run,
total 300.6s (< 5min budget).

Изменения вне репо:
- ~/food-market-stage/deploy/.env — RATE_* limits bumped.
- ~/quality-watchdog.sh — добавлен .prom textfile экспорт.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

2026-06-08 14:44:19 +05:00

12 lines

325 B

Markdown

Raw Blame History

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

 # Flaky tests report
 _Сгенерировано `tests/regression/find-flaky.sh` — 10 прогонов suite._
 **Всего уникальных тестов:** 42
 **Flaky:** 0 (0%)
 **Всегда зелёные:** 42
 **Всегда красные:** 0
 ## 🟢 Нет flaky тестов
 Suite стабилен.