1989db32bb
Sprint 16 — постоянный regression-контур: flows + visual + nightly +
CI workflow + README badges.
Ключевые цифры:
- 35 flow-тестов: 35/35 ✓ за ~30 секунд (workers=2 локально).
- 60 visual snapshot'ов (15 страниц × 2 темы × 2 viewport'a):
60/60 ✓ за ~4 минуты с retries=1.
- Полный регресс прогон: ~5 минут (цель была < 15).
Что сделано:
1. tests/regression/ — Playwright + factories + 8 spec-файлов.
OrgFactory builder создаёт org через API за O(N) HTTP вызовов
(signup → token → refs → products → counterparties → posted supplies).
Каждый flow независим, использует свой fresh-org.
2. tests/regression/visual/ — 15 страниц × 2 темы × 2 viewport'a.
Маски на динамический контент (артикулы с Date.now, KPI'ы,
delta-стрелки) чтобы 0.2% threshold не флакал. snapshotPathTemplate
c {projectName} — desktop+mobile не затирают друг друга.
3. tests/regression/factories/OrgFactory.ts — builder с .withProducts
.withCounterparties .withSupplies. Retry signup'a на 429.
4. .forgejo/workflows/regression.yml — on workflow_run после
Docker API/Web; cache на pnpm-store + Playwright-browsers;
артефакты при failure; Telegram-уведомление в обоих случаях.
5. ~/nightly-verify.sh + cron `0 4 * * *`: health → redeploy если
нужно → smoke flows; в воскресенье полный flows+visual. Логи с
ротацией 14 дней. Telegram на провал (~/.fm-watchdog/telegram-*).
6. scripts/generate-badges.sh — coverage из cobertura.xml в SVG через
shields.io (offline fallback). 4 CI-status badge + coverage badge в
README; CI step «Update coverage badge» авто-коммитит обновлённый
SVG на push в main.
Локальное число flake'ов: 1/60 visual на retry=1 (product-new light) —
случайная гонка маски, retry'ит и проходит.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
105 lines
3.8 KiB
YAML
105 lines
3.8 KiB
YAML
name: Regression suite
|
||
|
||
# Запускается ПОСЛЕ успешного docker-api/docker-web (stage-verify),
|
||
# когда stage уже задеплоен новой ревизией. Гонит полную регрессию
|
||
# (35 flow-тестов + 60 visual-snapshot'ов). Время прогона цель < 15 мин.
|
||
#
|
||
# Если падает — Telegram-уведомление со ссылкой на playwright-html отчёт.
|
||
|
||
on:
|
||
workflow_run:
|
||
workflows: ["Docker API", "Docker Web"]
|
||
types: [completed]
|
||
workflow_dispatch:
|
||
|
||
jobs:
|
||
regression:
|
||
name: Regression suite на stage
|
||
# Не запускаемся если триггерный workflow упал — нет смысла верифировать
|
||
# незадеплоенное.
|
||
if: ${{ github.event.workflow_run.conclusion == 'success' || github.event_name == 'workflow_dispatch' }}
|
||
runs-on: [self-hosted, linux]
|
||
timeout-minutes: 20
|
||
env:
|
||
E2E_ADMIN_URL: https://test.admin.food-market.kz
|
||
CI: '1'
|
||
steps:
|
||
- uses: actions/checkout@v4
|
||
|
||
- name: Wait for stage /health/ready
|
||
run: |
|
||
for i in 1 2 3 4 5 6 7 8 9 10; do
|
||
if curl -fsS "$E2E_ADMIN_URL/health/ready" | grep -q '"status":"Healthy"'; then
|
||
echo "stage ready"; exit 0
|
||
fi
|
||
sleep 3
|
||
done
|
||
echo "stage NOT ready" >&2
|
||
exit 1
|
||
|
||
- name: Setup pnpm cache for regression suite
|
||
uses: actions/cache@v4
|
||
with:
|
||
path: ~/.local/share/pnpm/store
|
||
key: pnpm-regression-${{ runner.os }}-${{ hashFiles('tests/regression/pnpm-lock.yaml') }}
|
||
restore-keys: |
|
||
pnpm-regression-${{ runner.os }}-
|
||
|
||
- name: Cache Playwright browsers
|
||
uses: actions/cache@v4
|
||
with:
|
||
path: ~/.cache/ms-playwright
|
||
key: pw-browsers-${{ hashFiles('tests/regression/package.json') }}
|
||
restore-keys: |
|
||
pw-browsers-
|
||
|
||
- name: Install regression deps
|
||
working-directory: tests/regression
|
||
run: pnpm install --frozen-lockfile
|
||
|
||
- name: Install Playwright Chromium
|
||
working-directory: tests/regression
|
||
run: pnpm exec playwright install chromium
|
||
|
||
- name: Run flows (35 tests)
|
||
id: flows
|
||
working-directory: tests/regression
|
||
run: pnpm exec playwright test flows/ --reporter=list,json
|
||
|
||
- name: Run visual (60 snapshots)
|
||
id: visual
|
||
working-directory: tests/regression
|
||
run: pnpm exec playwright test visual/ --reporter=list,json
|
||
|
||
- name: Upload playwright artifacts on failure
|
||
if: failure()
|
||
uses: actions/upload-artifact@v4
|
||
with:
|
||
name: playwright-report-${{ github.run_id }}
|
||
path: tests/regression/reports/
|
||
|
||
- name: Notify Telegram on failure
|
||
if: failure()
|
||
env:
|
||
BOT: ${{ secrets.TELEGRAM_BOT_TOKEN }}
|
||
CHAT: ${{ secrets.TELEGRAM_CHAT_ID }}
|
||
SHA: ${{ github.event.workflow_run.head_sha || github.sha }}
|
||
run: |
|
||
RUN_URL="${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
|
||
curl -sS -X POST "https://api.telegram.org/bot$BOT/sendMessage" \
|
||
--data-urlencode "chat_id=$CHAT" \
|
||
--data-urlencode "text=❌ regression FAILED — ${SHA:0:7} — $RUN_URL" \
|
||
> /dev/null
|
||
|
||
- name: Notify Telegram on success
|
||
if: success() && github.event_name == 'workflow_run'
|
||
env:
|
||
BOT: ${{ secrets.TELEGRAM_BOT_TOKEN }}
|
||
CHAT: ${{ secrets.TELEGRAM_CHAT_ID }}
|
||
SHA: ${{ github.event.workflow_run.head_sha }}
|
||
run: |
|
||
curl -sS -X POST "https://api.telegram.org/bot$BOT/sendMessage" \
|
||
--data-urlencode "chat_id=$CHAT" \
|
||
--data-urlencode "text=✅ regression OK — ${SHA:0:7} (35 flows + 60 visual)" \
|
||
> /dev/null
|