docs: sync workflow progress and weekend handoff
This commit is contained in:
89
docs/operations/ha-runbook.md
Normal file
89
docs/operations/ha-runbook.md
Normal file
@@ -0,0 +1,89 @@
|
||||
# HA 部署与验证 Runbook(Phase 3 MVP)
|
||||
|
||||
本文档对应 `docker-compose.ha.yml`,用于本地/测试环境验证高可用基础能力。
|
||||
|
||||
## 1. 启动方式
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.yml -f docker-compose.ha.yml up -d
|
||||
```
|
||||
|
||||
说明:
|
||||
- 基础业务服务仍来自 `docker-compose.yml`。
|
||||
- `docker-compose.ha.yml` 覆盖了 `db`、`redis`,并新增 `db-replica`、`postgres-backup`、`redis-replica`、`redis-sentinel-*`。
|
||||
|
||||
## 2. 核心环境变量建议
|
||||
|
||||
在 `backend/.env`(或 shell 环境)中至少配置:
|
||||
|
||||
```env
|
||||
# PostgreSQL
|
||||
POSTGRES_USER=dreamweaver
|
||||
POSTGRES_PASSWORD=dreamweaver_password
|
||||
POSTGRES_DB=dreamweaver_db
|
||||
POSTGRES_REPMGR_PASSWORD=repmgr_password
|
||||
|
||||
# Redis Sentinel
|
||||
REDIS_SENTINEL_ENABLED=true
|
||||
REDIS_SENTINEL_NODES=redis-sentinel-1:26379,redis-sentinel-2:26379,redis-sentinel-3:26379
|
||||
REDIS_SENTINEL_MASTER_NAME=mymaster
|
||||
REDIS_SENTINEL_DB=0
|
||||
REDIS_SENTINEL_SOCKET_TIMEOUT=0.5
|
||||
|
||||
# 可选:若 Sentinel/Redis 设置了密码
|
||||
REDIS_SENTINEL_PASSWORD=
|
||||
|
||||
# 备份周期,默认 86400 秒(1 天)
|
||||
BACKUP_INTERVAL_SECONDS=86400
|
||||
```
|
||||
|
||||
## 3. 健康检查
|
||||
|
||||
### 3.1 PostgreSQL 主从
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.yml -f docker-compose.ha.yml ps
|
||||
docker exec -it dreamweaver_db_primary psql -U dreamweaver -d dreamweaver_db -c "select now();"
|
||||
docker exec -it dreamweaver_db_replica psql -U dreamweaver -d dreamweaver_db -c "select pg_is_in_recovery();"
|
||||
```
|
||||
|
||||
期望:
|
||||
- 主库可读写;
|
||||
- 从库 `pg_is_in_recovery()` 返回 `t`。
|
||||
|
||||
### 3.2 Redis Sentinel
|
||||
|
||||
```bash
|
||||
docker exec -it dreamweaver_redis_sentinel_1 redis-cli -p 26379 sentinel masters
|
||||
docker exec -it dreamweaver_redis_sentinel_1 redis-cli -p 26379 sentinel replicas mymaster
|
||||
```
|
||||
|
||||
期望:
|
||||
- `mymaster` 存在;
|
||||
- 至少 1 个 replica 被发现。
|
||||
|
||||
### 3.3 备份任务
|
||||
|
||||
```bash
|
||||
docker exec -it dreamweaver_postgres_backup sh -c "ls -lh /backups"
|
||||
```
|
||||
|
||||
期望:
|
||||
- `/backups` 下出现 `.dump` 文件;
|
||||
- 旧于 7 天的备份会被自动清理。
|
||||
|
||||
## 4. 故障切换演练(最小)
|
||||
|
||||
```bash
|
||||
# 模拟 Redis 主节点故障
|
||||
docker stop dreamweaver_redis_master
|
||||
|
||||
# 等待 Sentinel 选主后查看
|
||||
docker exec -it dreamweaver_redis_sentinel_1 redis-cli -p 26379 sentinel get-master-addr-by-name mymaster
|
||||
```
|
||||
|
||||
提示:应用与 Celery 已支持 Sentinel 配置。若未启用 Sentinel,仍可回退到 `REDIS_URL` / `CELERY_BROKER_URL` / `CELERY_RESULT_BACKEND` 直连模式。
|
||||
|
||||
## 5. 当前已知限制(下一步)
|
||||
|
||||
- PostgreSQL 侧当前仅完成主从拓扑,读写分离(PgBouncer/路由)待后续迭代。
|
||||
Reference in New Issue
Block a user