ci/deploy: stage deploy workflow + notifications + server plan
.github/workflows/deploy-stage.yml: - Triggers on successful "Docker Images" workflow or manual dispatch. - SSHes to stage server via STAGE_SSH_KEY, copies deploy/docker-compose.yml and nginx.conf, writes .env with current SHA + POSTGRES_PASSWORD. - `docker compose pull && up -d --remove-orphans`. - Smoke-tests /health with 5 retries (5s each). - Pings Telegram on success/failure with commit SHA + stage URL. .github/workflows/notify.yml: - Separate workflow_run listener for CI/Docker failures, sends Telegram message with link to the failed run. deploy/docker-compose.yml port remap (stage server already uses 80/443/5000/5432): - API: 8080 (was 8080, confirmed free) - Web: 8081 (was 80 — taken by legacy nginx) - Postgres: 127.0.0.1:5434 (was 5433 — and now localhost-only, safer) docs/stage-setup.md — one-time server setup runbook: - Verified specs: Ubuntu 24.04, 4 CPU, 15 GB RAM, 4 GB free disk (tight). - Step 1: `sudo usermod -aG docker nns` so deploy doesn't need sudo. - Step 2: generate STAGE_POSTGRES_PASSWORD secret via `openssl rand`. - Step 3: port-conflict check. - Step 4: first manual deploy via gh workflow run. - Disk-usage monitoring via cron → Telegram when >85%. Secrets now in repo: TELEGRAM_BOT_TOKEN, TELEGRAM_CHAT_ID, STAGE_SSH_HOST, STAGE_SSH_PORT, STAGE_SSH_USER, STAGE_SSH_KEY Still needed from user: STAGE_POSTGRES_PASSWORD (one openssl command). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
nurdotnet
parent
fa2fae9503
commit
75d73b9dcd
86
.github/workflows/deploy-stage.yml
vendored
Normal file
86
.github/workflows/deploy-stage.yml
vendored
Normal file
|
|
@ -0,0 +1,86 @@
|
|||
name: Deploy stage
|
||||
|
||||
on:
|
||||
workflow_run:
|
||||
workflows: ["Docker Images"]
|
||||
types: [completed]
|
||||
branches: [main]
|
||||
workflow_dispatch:
|
||||
|
||||
concurrency:
|
||||
group: deploy-stage
|
||||
cancel-in-progress: false
|
||||
|
||||
jobs:
|
||||
deploy:
|
||||
name: docker compose pull + up
|
||||
runs-on: ubuntu-latest
|
||||
if: ${{ github.event.workflow_run.conclusion == 'success' || github.event_name == 'workflow_dispatch' }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Add SSH key
|
||||
run: |
|
||||
mkdir -p ~/.ssh
|
||||
echo "${{ secrets.STAGE_SSH_KEY }}" > ~/.ssh/id_ed25519
|
||||
chmod 600 ~/.ssh/id_ed25519
|
||||
ssh-keyscan -p ${{ secrets.STAGE_SSH_PORT }} -H ${{ secrets.STAGE_SSH_HOST }} >> ~/.ssh/known_hosts 2>/dev/null
|
||||
|
||||
- name: Copy compose files
|
||||
run: |
|
||||
SSH="ssh -p ${{ secrets.STAGE_SSH_PORT }} ${{ secrets.STAGE_SSH_USER }}@${{ secrets.STAGE_SSH_HOST }}"
|
||||
SCP="scp -P ${{ secrets.STAGE_SSH_PORT }}"
|
||||
$SSH 'mkdir -p ~/food-market-stage/deploy'
|
||||
$SCP deploy/docker-compose.yml ${{ secrets.STAGE_SSH_USER }}@${{ secrets.STAGE_SSH_HOST }}:~/food-market-stage/deploy/
|
||||
$SCP deploy/nginx.conf ${{ secrets.STAGE_SSH_USER }}@${{ secrets.STAGE_SSH_HOST }}:~/food-market-stage/deploy/
|
||||
|
||||
- name: Write .env (tags + port overrides)
|
||||
run: |
|
||||
SSH="ssh -p ${{ secrets.STAGE_SSH_PORT }} ${{ secrets.STAGE_SSH_USER }}@${{ secrets.STAGE_SSH_HOST }}"
|
||||
SHA="${{ github.event.workflow_run.head_sha || github.sha }}"
|
||||
$SSH "cat > ~/food-market-stage/deploy/.env" <<ENV
|
||||
API_TAG=$SHA
|
||||
WEB_TAG=$SHA
|
||||
POSTGRES_PASSWORD=${{ secrets.STAGE_POSTGRES_PASSWORD }}
|
||||
ENV
|
||||
|
||||
- name: Login to ghcr on stage
|
||||
run: |
|
||||
SSH="ssh -p ${{ secrets.STAGE_SSH_PORT }} ${{ secrets.STAGE_SSH_USER }}@${{ secrets.STAGE_SSH_HOST }}"
|
||||
$SSH "echo '${{ secrets.GITHUB_TOKEN }}' | docker login ghcr.io -u ${{ github.actor }} --password-stdin"
|
||||
|
||||
- name: Pull + up (stage compose)
|
||||
id: deploy
|
||||
run: |
|
||||
SSH="ssh -p ${{ secrets.STAGE_SSH_PORT }} ${{ secrets.STAGE_SSH_USER }}@${{ secrets.STAGE_SSH_HOST }}"
|
||||
$SSH 'cd ~/food-market-stage/deploy && docker compose pull && docker compose up -d --remove-orphans'
|
||||
|
||||
- name: Smoke test /health
|
||||
run: |
|
||||
SSH="ssh -p ${{ secrets.STAGE_SSH_PORT }} ${{ secrets.STAGE_SSH_USER }}@${{ secrets.STAGE_SSH_HOST }}"
|
||||
for i in 1 2 3 4 5 6; do
|
||||
sleep 5
|
||||
if $SSH "curl -fsS http://localhost:8080/health" 2>&1 | tee /tmp/health.out | grep -q '"status":"ok"'; then
|
||||
echo "Health OK"
|
||||
exit 0
|
||||
fi
|
||||
done
|
||||
echo "Health failed"
|
||||
cat /tmp/health.out || true
|
||||
exit 1
|
||||
|
||||
- name: Notify Telegram on success
|
||||
if: success()
|
||||
run: |
|
||||
curl -sS -X POST "https://api.telegram.org/bot${{ secrets.TELEGRAM_BOT_TOKEN }}/sendMessage" \
|
||||
--data-urlencode "chat_id=${{ secrets.TELEGRAM_CHAT_ID }}" \
|
||||
--data-urlencode "text=Deploy stage OK — commit ${GITHUB_SHA:0:7}. http://88.204.171.93:8081" \
|
||||
> /dev/null
|
||||
|
||||
- name: Notify Telegram on failure
|
||||
if: failure()
|
||||
run: |
|
||||
curl -sS -X POST "https://api.telegram.org/bot${{ secrets.TELEGRAM_BOT_TOKEN }}/sendMessage" \
|
||||
--data-urlencode "chat_id=${{ secrets.TELEGRAM_CHAT_ID }}" \
|
||||
--data-urlencode "text=Deploy stage FAILED — commit ${GITHUB_SHA:0:7}. See https://github.com/${{ github.repository }}/actions/runs/${{ github.run_id }}" \
|
||||
> /dev/null
|
||||
18
.github/workflows/notify.yml
vendored
Normal file
18
.github/workflows/notify.yml
vendored
Normal file
|
|
@ -0,0 +1,18 @@
|
|||
name: Notify CI failures
|
||||
|
||||
on:
|
||||
workflow_run:
|
||||
workflows: ["CI", "Docker Images"]
|
||||
types: [completed]
|
||||
|
||||
jobs:
|
||||
telegram:
|
||||
if: ${{ github.event.workflow_run.conclusion == 'failure' }}
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Ping Telegram
|
||||
run: |
|
||||
curl -sS -X POST "https://api.telegram.org/bot${{ secrets.TELEGRAM_BOT_TOKEN }}/sendMessage" \
|
||||
--data-urlencode "chat_id=${{ secrets.TELEGRAM_CHAT_ID }}" \
|
||||
--data-urlencode "text=CI FAILED: ${{ github.event.workflow_run.name }} on ${{ github.event.workflow_run.head_branch }} (${GITHUB_SHA:0:7}). https://github.com/${{ github.repository }}/actions/runs/${{ github.event.workflow_run.id }}" \
|
||||
> /dev/null
|
||||
|
|
@ -8,8 +8,9 @@ services:
|
|||
POSTGRES_USER: food_market
|
||||
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-food_market_dev}
|
||||
PGDATA: /var/lib/postgresql/data/pgdata
|
||||
# Stage VM already uses 5432 (host postgres) — map ours to 5434 to avoid clash.
|
||||
ports:
|
||||
- "5433:5432"
|
||||
- "127.0.0.1:5434:5432"
|
||||
volumes:
|
||||
- postgres-data:/var/lib/postgresql/data
|
||||
healthcheck:
|
||||
|
|
@ -28,8 +29,11 @@ services:
|
|||
environment:
|
||||
ASPNETCORE_ENVIRONMENT: Production
|
||||
ConnectionStrings__Default: Host=postgres;Port=5432;Database=food_market;Username=food_market;Password=${POSTGRES_PASSWORD:-food_market_dev}
|
||||
# Host port mapping: pick free ports on existing stage server (80/443 taken by
|
||||
# legacy nginx, 5000/5002/5005 taken by legacy .NET apps).
|
||||
ports:
|
||||
- "8080:8080"
|
||||
- "8080:8080" # api
|
||||
|
||||
volumes:
|
||||
- api-data:/app/App_Data
|
||||
- api-logs:/app/logs
|
||||
|
|
@ -41,7 +45,7 @@ services:
|
|||
depends_on:
|
||||
- api
|
||||
ports:
|
||||
- "80:80"
|
||||
- "8081:80" # web SPA, not on 80 (legacy nginx holds it)
|
||||
|
||||
volumes:
|
||||
postgres-data:
|
||||
|
|
|
|||
93
docs/stage-setup.md
Normal file
93
docs/stage-setup.md
Normal file
|
|
@ -0,0 +1,93 @@
|
|||
# Первичная настройка stage-сервера (88.204.171.93)
|
||||
|
||||
**Разовая процедура.** После этого деплой происходит автоматически на каждый push в `main`.
|
||||
|
||||
## Текущее состояние сервера (проверено)
|
||||
|
||||
- Ubuntu 24.04.3, 4 CPU, 15 ГБ RAM (8 ГБ свободно)
|
||||
- **Диск 19 ГБ, свободно 4 ГБ** ← узкое место, нужно следить
|
||||
- Docker 28.2.2 установлен ✓
|
||||
- PostgreSQL 14/16 на 5432 (используется существующими приложениями)
|
||||
- Порты 80/443 заняты legacy nginx
|
||||
- Порты 5000, 5002, 5005 заняты legacy .NET (food-market-server, calcman, makesales)
|
||||
- SSH: `nns@88.204.171.93:9393`
|
||||
|
||||
## Шаг 1 — выдать nns доступ к Docker (ОДНОРАЗОВО)
|
||||
|
||||
На сервере:
|
||||
```bash
|
||||
ssh -p 9393 nns@88.204.171.93
|
||||
sudo usermod -aG docker nns
|
||||
exit
|
||||
```
|
||||
|
||||
**Важно:** после этой команды разлогинься из SSH и залогинься снова — групповые права применяются только при новой сессии.
|
||||
|
||||
Проверь:
|
||||
```bash
|
||||
ssh -p 9393 nns@88.204.171.93 'docker ps'
|
||||
```
|
||||
Должно выдать список контейнеров (сейчас пустой) без permission denied.
|
||||
|
||||
## Шаг 2 — задать пароль для stage postgres
|
||||
|
||||
Генерим рандомный 32-символьный пароль и кладём его в GitHub Secrets:
|
||||
```bash
|
||||
# На твоём Mac
|
||||
PASS=$(openssl rand -base64 24 | tr -d '=+/' | head -c 32)
|
||||
gh secret set STAGE_POSTGRES_PASSWORD --repo nurdotnet/food-market --body "$PASS"
|
||||
# Сохрани его же в файл на всякий случай:
|
||||
echo "POSTGRES_PASSWORD=$PASS" > ~/.food-market-secrets/stage-postgres.env
|
||||
chmod 600 ~/.food-market-secrets/stage-postgres.env
|
||||
```
|
||||
|
||||
## Шаг 3 — проверить порты
|
||||
|
||||
Наш stage слушает:
|
||||
- **8080** — API (health: `curl http://88.204.171.93:8080/health`)
|
||||
- **8081** — Web (SPA с reverse-proxy на API)
|
||||
- **5434** — Postgres (только localhost, не наружу)
|
||||
|
||||
Проверь что эти порты ещё не заняты:
|
||||
```bash
|
||||
ssh -p 9393 nns@88.204.171.93 'ss -tlnp | grep -E "(8080|8081|5434)"'
|
||||
```
|
||||
Если пусто — всё ок.
|
||||
|
||||
## Шаг 4 — первый ручной деплой (для проверки)
|
||||
|
||||
После того как GitHub Actions собрал образы (это происходит автоматически при пуше), запусти workflow вручную:
|
||||
```bash
|
||||
gh workflow run deploy-stage.yml --repo nurdotnet/food-market
|
||||
# Смотри статус:
|
||||
gh run watch --repo nurdotnet/food-market
|
||||
```
|
||||
|
||||
После успеха откроется: http://88.204.171.93:8081 — это stage-админка.
|
||||
|
||||
## Мониторинг диска
|
||||
|
||||
Добавь cron на stage-сервере:
|
||||
```bash
|
||||
ssh -p 9393 nns@88.204.171.93
|
||||
crontab -e
|
||||
```
|
||||
Добавить строку:
|
||||
```
|
||||
0 */6 * * * /usr/bin/df -h / | awk '/\/$/ {if ($5+0 > 85) system("curl -sS -X POST https://api.telegram.org/bot$TG_TOKEN/sendMessage --data-urlencode chat_id=$TG_CHAT --data-urlencode text=\"Disk on stage: "$5" used\"")}'
|
||||
```
|
||||
(Подставь реальные TG_TOKEN и TG_CHAT, или используй `source ~/.food-market-secrets/telegram.env` в cron-wrapper.)
|
||||
|
||||
## Что происходит при каждом push в main
|
||||
|
||||
```
|
||||
push → Github Actions:
|
||||
1. CI (backend build + web build) — если упал, Telegram "CI FAILED"
|
||||
2. Docker Images (api + web → ghcr.io) — если упал, Telegram "CI FAILED"
|
||||
3. Deploy stage (после успешного Docker) →
|
||||
ssh nns@stage → docker compose pull → up -d → curl /health
|
||||
Если успешно — Telegram "Deploy stage OK — SHA — http://..."
|
||||
Если упало — Telegram "Deploy stage FAILED — ссылка на лог"
|
||||
```
|
||||
|
||||
Ты видишь уведомление в Telegram, открываешь stage, проверяешь, говоришь «мёрджим в prod» или «откатывай».
|
||||
Loading…
Reference in a new issue