Compare commits

...

4 Commits

Author SHA1 Message Date
a037ea3064 feat: 邮件告警与每日报告优化(HTML+UTF-8、阈值实时告警、支付卡单检测、重试与状态机)
Some checks failed
Deploy Backend / deploy (push) Has been cancelled
2026-08-21 09:02:59 +08:00
3cce81321f test: 补充单元测试并修复 urlencoding 解码 bug
- auth.rs 新增 JWT 生成/验证回环、篡改拒绝、refresh token 随机性、临时 token 有效期测试
- payment.rs 新增套餐定义、URL 解码、支付宝表单解析(UTF-8/GBK)、RSA2 签名验签、UA 检测测试
- 修复 urlencoding 两个真实 bug:UTF-8 多字节序列乱码、不完整转义吞字符
- 移除 db.rs 悬空 doc 注释,修复 auth.rs 冗余引用(clippy 0 警告)
- cargo test 从 2 个增至 36 个全过
- 同步 IMPROVEMENTS.md(索引/重复查询/测试覆盖/公告系统状态)
2026-08-14 13:09:55 +08:00
366d62f262 feat: 部署时自动校验支付宝签名一致性 + 验签失败升级为 error + 密钥核对脚本 2026-08-10 16:06:02 +08:00
49d740e27c fix: 移除订单自动确认,仅依赖支付宝回调验签;rsa2_verify 支持裸 base64 DER 公钥;.env 取消 git 跟踪 2026-08-10 15:49:27 +08:00
13 changed files with 1011 additions and 170 deletions

19
.env
View File

@@ -1,19 +0,0 @@
# .env
DATABASE_URL=postgres://milkydata:44n6FdB8CdDAk5rk@154.37.213.24:5432/milkydata
WECHAT_APPID="wx5b00eb90621802f7"
WECHAT_SECRET="494efc513faa310bfba588bda2849bfd"
JWT_SECRET="your_super_secret_key"
SSL_KEY_PATH=/etc/ssl/private/private.key
SSL_CERT_PATH=/etc/ssl/certs/full_chain.pem
RUST_LOG=info
APP_VERSION="0.2.0"
FREE_USER_DATA_LIMIT=20
SSH_SERVER=root@1panel-server
# 支付宝生产环境配置
# 支付宝生产环境配置
ALIPAY_APP_ID=2021006148681106
ALIPAY_PRIVATE_KEY="MIIEvQIBADANBgkqhkiG9w0BAQEFAASCBKcwggSjAgEAAoIBAQCImX4Zj18whXYi6l29caq+ShjqT7JwWDbOoz/hEqZqPxQNu6KF59OswcUIv3It4CxJ0/kXsiC/pc6mrN9GedOAPIaz6YydgII/nwHl4OdPPxW8jeomBuiSmcbPcZ4AJTi1YTxrj//8YDIQInVc5/I4HPg+osN4qkqy+xrJ4vGrmyy+ClwHh3OxglGDLhbprH4WCgxKjwTDfR1A84w+pbrST1YaWI7sqqPhVxuYQQPjaLOpXMgrJu5ctnlFVTC27mY2wSIrxajB/6w1SQ+Z+Z149frk47PBOwL+xfKT4gwy3yj/t7WZStzH0gEhG7mXjc6qIjZ8uH5OxUZ+f6gMDpKBAgMBAAECggEAYnI87y7k65NEEOXXqTDCXWIJktTdRRpOZZ5B63QFe4z6ZutTti3d4CPGXQecwnNNd+fNHcbC01wsEvOD9sbWJxV3/zHSpbB4LmHqfEIacoKEp9+fYMiZ1Q3+uoKn3HL268GoSdz+IGUzr20mKE8Bwm8eegXT2qLgMx3FsLpecc3WXmwgMDI/alknO81+xZenLH1Y6+XdsoPVWsS3OY3gnzfYz40n1LaLCQiklBu6AUjSQupSUJ53rGtYpZnp+ImYwBVcYeFL0mZZVId/lzDI2K8eaM/RGVZYsU1vkP6CeSJzWhpr6ftSRRl3VJQou0ytLLX/YeUoXIhsfZ+x3cPKAQKBgQC9ub1z+T/+0lHZwm360otNHQNeEq3FYjnx1x7WOWTXCsAqVZizfmpUYSHoNZeNlyLpbXVsYp22xJmLBArB9LBMvjY1z2iOQJW7F46O6cY38AcgWC41YalLcrk2y4jny6rJZRMjvmDOM67SH01pzZQ6eGJpzSZpTj2Jm/tBsH7gGwKBgQC4UPK5a32ov9YuyzfQXcW341JFlTK2k2D1ewilYSZifpTDqdVpH1gSzGpfoXI7K6Z26xkx1n5rOIe+SxOKaNBj/5PJuEzmMVvUWkFU0IutoqVc2R6h4Rb+9q7llzAavFZ5Ssj1u9Q5Pjhc5Jr8zM6xXk55/4CuqYuBMCcte4fZkwKBgF/kVSEcxX+iBcSuRLAVYlrjYyRVBIdTEj5toCRedd1Kjb3X7YorRsrHHUAkdikWTor0y/iwD06lJ4Xf0SAe8EyZhwTDg/ECK6uYSHFrLshvBOsQJqrMZajAn2RWkye1BOktsPLL8lnGPYHuWE41ncm/tt4K1nM7HFdFA0U8U6QpAoGAe0gRgboG+6cMoJHmjCUciUmwDsCjggEDuk4H024umXSnFlRzXPPNGg3GHJ7E5BTnK+VGHdKlDhxqlvpqYIqI0EnzVNpD/TgIfvm7dvpiYspGJttGpOoEUpc3FnPQhiFxCYHe9pYidgNK+b8id3KZ0YMNpIYkUS8wE0HdAov8qqcCgYEAtRjXzj4JZ5we6aOqv4bcr7xLw4a7wzS99K5EcrbYRxvibUhelooQdgdY40FmdtfuY/4rpHC3R9/13aqcD/WSTFOdOsZAcfTi738y13h3dCy3FZisLYM7rgjUSnQnEGlyR0TYE4Yda490rpYJBjyQIE3EcIlWPerm8vh1wx1Rh90="
ALIPAY_ALIPAY_PUBLIC_KEY="MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAkhLqABUN1E2PCmOznQp+53T3x4j27I96hdatl6lWbCBPwyXnZVW4uYZ4kV+34+i2wCzEtMpUiNQaIsQXx2qD8x0d8szsj7pEZuY+xUUJTzm95fYO2nq4aaQ38T/kxdYrW12is8vwY83rlghdlkvVP2XZCuYoFq9UVieTu6LTDk6v91Mt2xr1Zw1Oj+yg8VpBFXkcS7ORJjiufWzCEoxzlb2KK2AGOLhdc8BJsBfrQ0JAqFB6b0uteUF21DDdPN/sblD8GoRAGuSpDhyQLG+0fDNdcIMyHpsq7bIDNjgYGS8RbKAtb6WyV6vriIemZ0fMvzN5Mpqv2jwCe/31ar//PwIDAQAB"
ALIPAY_GATEWAY="https://openapi.alipay.com/gateway.do"
APP_BASE_URL="https://dev.xmclassmate.top"

View File

@@ -280,18 +280,20 @@ DROP TABLE IF EXISTS invitation_codes;
- 完成时间2026-04-15
### 1.2 数据库操作优化
- [ ] **重复查询问题**
- [x] **重复查询问题**
- 位置:`insert_weather_data` 中多次查询用户信息
- 改进:合并查询或使用缓存
- 影响:减少数据库负载
- 状态:`get_user_quota` 已合并为单次查询(见 `db.rs` 顶部注释2026-08-11 复核无遗留 N+1
- [ ] **缺少数据库索引**
- 现状:未见索引定义
- 改进:添加常用查询字段索引
- [x] **缺少数据库索引**
- 状态:`migrations/006_add_performance_indexes.sql` 已添加
```sql
CREATE INDEX idx_weather_data_user_id ON weather_data(user_id);
CREATE INDEX idx_weather_data_date ON weather_data(date);
CREATE INDEX idx_weather_data_date ON weather_data(date DESC);
CREATE INDEX idx_weather_data_user_favorite ON weather_data(user_id, is_favorite);
CREATE INDEX idx_weather_data_user_date ON weather_data(user_id, date DESC);
CREATE INDEX idx_users_openid ON users(openid);
CREATE INDEX idx_payment_orders_status ON payment_orders(status);
```
### 1.3 错误处理改进
@@ -422,13 +424,13 @@ DROP TABLE IF EXISTS invitation_codes;
## 五、开发体验改进
### 5.1 测试覆盖
- [ ] **仅有一个基础编译测试**
- 现状:`tests/integration_test.rs` 只有 `assert_eq!(2 + 2, 4)`
- 改进:添加完整的测试套件
- 类型:
- 单元测试:工具函数、数据结构
- 集成测试API 端点
- 数据库测试CRUD 操作
- [~] **基础测试不足**2026-08-11 已补充单元测试)
- 现状:原仅有 `tests/integration_test.rs` 的编译测试和套餐一致性测试
- 已补充(`cargo test` 共 36 用例):
- `src/auth.rs`JWT 生成/验证回环、篡改拒绝、错误密钥拒绝、refresh token 随机性、临时 token 有效期
- `src/handlers/payment.rs`套餐定义、URL 解码(**修复了 UTF-8 乱码与吞字符两个真实 bug**、支付宝表单解析UTF-8/GBK、RSA2 签名/验签回环PEM + Base64 DER 密钥)、移动端 UA 检测、订单号唯一性
- `src/rate_limiter.rs`限流阈值、按客户端隔离、IP 提取(已有)
- 待做API 集成测试(需要测试数据库)
- [ ] **缺少 API 测试**
- 需求:验证 API 行为
@@ -1369,6 +1371,9 @@ async onRedeemCode() {
## 十四、公告系统
> **状态**:✅ 已实现2026-05`handlers/notifications.rs` + 迁移 `011_add_notifications.sql`,前端 `pkg-extra/notifications` 页面)
> 下方为原始设计文档,保留供参考。
### 功能概述
公告系统用于向用户发送系统公告,支持管理员发布、编辑、删除公告,用户查看公告列表和详情。

84
docs/OPS-ALERT.md Normal file
View File

@@ -0,0 +1,84 @@
# 服务器邮件告警与每日报告 — 部署指南
<!-- 更新时间2026-08-21 -->
## 概述
两封自动邮件,均通过 `msmtp` → 163 SMTP 发送(复用现有 `~/.msmtprc` 配置):
| 脚本 | 频率cron | 作用 |
| --- | --- | --- |
| `daily-report.sh` | 每日 08:00 | 服务器健康报告HTML系统/容器/SSL/服务/安全/备份 + **支付与会员统计** |
| `alert.sh` | 每 5 分钟 | **实时异常告警**:正常时静默,异常才发邮件(带防刷屏状态机) |
## 部署步骤
```bash
# 1. 上传三个文件到服务器root 用户)
scp scripts/mail-lib.sh scripts/daily-report.sh scripts/alert.sh root@<服务器>:/root/rust/scripts/
ssh root@<服务器> 'chmod +x /root/rust/scripts/*.sh && mkdir -p /root/rust/scripts/logs'
# 2. 配置 cronroot
ssh root@<服务器> 'crontab -e'
# 加入以下两行:
0 8 * * * /root/rust/scripts/daily-report.sh >> /root/rust/scripts/logs/daily-report.log 2>&1
*/5 * * * * /root/rust/scripts/alert.sh >> /root/rust/scripts/logs/alert.log 2>&1
```
## 验证
```bash
# 干跑告警(预览当前异常与将发送的邮件,不真正发送)
/root/rust/scripts/alert.sh --dry-run
# 手动发一封健康报告测试 msmtp 链路
/root/rust/scripts/daily-report.sh
# 查看邮件日志
tail -50 /root/rust/scripts/logs/mail.log
```
## alert.sh 检测项与阈值
| 检测项 | 警告 | 严重 |
| --- | --- | --- |
| 磁盘使用(/ | > 80% | > 90% |
| 内存使用 | > 80% | > 90% |
| 备份时效 | > 24h | > 48h或无备份 |
| 服务状态green/blue/nginx | — | 非 active |
| Docker 容器 | — | 存在非 Up 容器 / docker 不可用 |
| SSL 证书剩余 | < 30 天 | < 7 天 |
| 支付卡单pending > 30 分钟) | ≥ 1 笔(附订单号) | — |
| 数据库SELECT 1 | — | 不可用 |
### 防刷屏机制
状态文件 `.alert-state`(默认 `/root/rust/scripts/.alert-state`)记录每个检测项最近告警级别:
- 级别**上升** → 立即发邮件
- 级别**不变** → 冷却 `ALERT_COOLDOWN_HOURS`(默认 6 小时)后才再次提醒
- **恢复**正常 → 静默清除状态,不发邮件
## 可配置项(环境变量)
| 变量 | 默认值 | 说明 |
| --- | --- | --- |
| `MAIL_TO` | `m943790568@163.com` | 收件人,逗号分隔多地址 |
| `MAIL_FROM` | `ASD 监控 <m943790568@163.com>` | 发件人显示名 |
| `MAIL_ACCOUNT` | `default` | msmtp 账户名 |
| `MAIL_LOG_DIR` | `/root/rust/scripts/logs` | 日志目录mail.log / alert.log |
| `MAIL_RETRY` | `3` | 发送失败重试次数 |
| `DB_CONTAINER` | `1Panel-postgresql-FtMo` | PostgreSQL 容器名1Panel 安装) |
| `BACKUP_DIR` | `/root/rust/backups/milkydata` | 备份目录 |
| `STUCK_ORDER_MINUTES` | `30` | pending 订单视为卡单的时长 |
| `ALERT_STATE_FILE` | `/root/rust/scripts/.alert-state` | 状态文件路径 |
| `ALERT_COOLDOWN_HOURS` | `6` | 同级别再次提醒的冷却时间 |
## 注意事项
- 脚本在**服务器本机**运行cron 以 root 执行),数据库通过 `docker exec <容器> psql` 访问,
容器名与 `check-members.sh` / `send-notification.sh` 保持一致;如容器名不同,设置 `DB_CONTAINER` 覆盖。
- 邮件为 HTML + UTF-8无需额外编码配置msmtp 发送失败自动重试 3 次并记录日志。
- 单次采集失败(如 journalctl 无权限)不会阻断整封邮件,对应项显示 N/A。
- `alert.sh` 的卡单检测可直接发现「支付卡在确认页面」类问题pending 超 30 分钟即告警,
邮件中附订单号与创建时间,便于对照支付宝账单处理)。

View File

@@ -499,6 +499,85 @@ test_notify_url_domain() {
return 0
}
# ---------- 支付宝签名一致性校验 ----------
# 防止密钥不同步导致的 invalid-signature历史事故服务器私钥与控制台应用公钥不匹配
test_payment_signature() {
log_info "校验支付宝支付签名与本地密钥一致性..."
if [ ! -f "${SCRIPT_DIR}/.env" ] || ! grep -q "ALIPAY_PRIVATE_KEY" "${SCRIPT_DIR}/.env"; then
log_warn " 本地 .env 无 ALIPAY_PRIVATE_KEY跳过签名校验"
return 0
fi
local jwt
jwt=$(remote "curl -sk -X POST '${BASE_URL}/api/guest-login'" 2>/dev/null | python3 -c "import sys,json; print(json.load(sys.stdin).get('token',''))" 2>/dev/null)
if [ -z "$jwt" ]; then
log_error "获取 JWT 失败,无法校验支付签名"
return 1
fi
local form
form=$(remote "curl -sk --max-time 10 '${BASE_URL}/payment/page?package=monthly&jwt=${jwt}'" 2>/dev/null || true)
if ! echo "$form" | grep -q 'name="sign"'; then
log_warn " 支付表单未生成(可能为 Mock 支付模式),跳过签名校验"
return 0
fi
local result
result=$(echo "$form" | python3 -c "
import base64, re, html, sys
from cryptography.hazmat.primitives import serialization, hashes
from cryptography.hazmat.primitives.asymmetric import padding
priv_b64 = None
for line in open('${SCRIPT_DIR}/.env'):
if line.startswith('ALIPAY_PRIVATE_KEY='):
priv_b64 = line.strip().split('=', 1)[1]
break
if not priv_b64:
print('FAIL: 本地 .env 无 ALIPAY_PRIVATE_KEY')
sys.exit(1)
try:
priv = serialization.load_der_private_key(base64.b64decode(priv_b64), password=None)
except Exception:
try:
priv = serialization.load_pem_private_key(priv_b64.encode(), password=None)
except Exception as e:
print(f'FAIL: 私钥解析失败: {e}')
sys.exit(1)
content = sys.stdin.read()
fields = {}
for m in re.finditer(r'name=\"([^\"]+)\" value=\"([^\"]*)\"', content):
fields[m.group(1)] = html.unescape(m.group(2))
if 'sign' not in fields:
print('FAIL: 表单无 sign 字段')
sys.exit(1)
sign = fields.pop('sign')
sign_source = '&'.join(f'{k}={v}' for k, v in sorted(fields.items()))
try:
our_sign = base64.b64encode(priv.sign(sign_source.encode('utf-8'), padding.PKCS1v15(), hashes.SHA256())).decode()
except Exception as e:
print(f'FAIL: 签名失败: {e}')
sys.exit(1)
if our_sign == sign:
print('PASS')
else:
print('FAIL: 服务器签名与本地私钥不一致(密钥不同步!检查服务器 .env 与控制台应用公钥)')
sys.exit(1)
" 2>/dev/null)
if [ "$result" = "PASS" ]; then
log_info " 服务器签名与本地私钥一致 ✅"
return 0
else
log_error " ${result:-签名校验脚本执行失败}"
return 1
fi
}
# ---------- 主测试入口 ----------
run_tests() {
log_step "执行部署后测试..."
@@ -522,6 +601,7 @@ run_tests() {
"test_payment_refund:退款+会员撤销"
"test_payment_verify_detection:验证函数检测逻辑"
"test_notify_url_domain:支付回调域名校验"
"test_payment_signature:支付签名一致性"
)
local i=1 total=${#tests[@]}

257
scripts/alert.sh Normal file
View File

@@ -0,0 +1,257 @@
#!/bin/bash
# ===========================================
# alert.sh — 服务器异常实时告警(正常时静默,异常才发邮件)
#
# 检测项:
# 磁盘使用 >80% 警告 / >90% 严重
# 内存使用 >80% 警告 / >90% 严重
# 备份时效 >24h 警告 / >48h 严重(含备份缺失)
# 服务状态 green/blue/nginx 非 active → 严重
# Docker 容器非 Up → 严重
# SSL 剩余 <30 天警告 / <7 天严重
# 支付卡单 pending 且创建超过 30 分钟 → 警告(列出订单号)
# 数据库 不可用 → 严重(支付/会员功能受影响)
#
# 防刷屏状态机(.alert-state
# 级别上升 → 立即告警;级别不变 → 超过 ALERT_COOLDOWN_HOURS 再次提醒;
# 恢复正常 → 清除状态,不打扰。
#
# 部署:
# cp scripts/mail-lib.sh scripts/alert.sh → 服务器 /root/rust/scripts/
# cronroot:
# */5 * * * * /root/rust/scripts/alert.sh >> /root/rust/scripts/logs/alert.log 2>&1
#
# 用法:/root/rust/scripts/alert.sh [--dry-run]
# ===========================================
set -u
SCRIPT_DIR=$(cd "$(dirname "$0")" && pwd)
# shellcheck source=mail-lib.sh
source "${SCRIPT_DIR}/mail-lib.sh"
# ---------- 可配置项(环境变量可覆盖) ----------
: "${ALERT_STATE_FILE:=/root/rust/scripts/.alert-state}"
: "${ALERT_COOLDOWN_HOURS:=6}" # 同一级别再次提醒的冷却时间
: "${DB_CONTAINER:=1Panel-postgresql-FtMo}"
: "${DB_NAME:=milkydata}"
: "${DB_USER:=milkydata}"
: "${STUCK_ORDER_MINUTES:=30}" # pending 超过该时长视为卡单
: "${BACKUP_DIR:=/root/rust/backups/milkydata}" # 备份目录
DRY_RUN=false
[ "${1:-}" = "--dry-run" ] && DRY_RUN=true
# 日志
alert_log() { echo "[$(date '+%Y-%m-%d %H:%M:%S')] $*" >> "${MAIL_LOG_DIR}/alert.log" 2>/dev/null || true; }
[ "$DRY_RUN" = true ] && alert_log() { echo "[DRY-RUN] $*"; }
# ---------- 检测项 ----------
# 每项输出: 级别(0/1/2)|键名|标签|当前值|详情
# 级别: 0=正常 1=警告 2=严重
check_disk() {
local pct; pct=$(df -P / | awk 'NR==2{print $5}' | tr -d '%')
pct=${pct:-0}
local used total; used=$(df -h / | awk 'NR==2{print $3}'); total=$(df -h / | awk 'NR==2{print $2}')
local lvl=0
[ "$pct" -ge 90 ] && lvl=2
[ "$pct" -ge 80 ] && [ "$lvl" -eq 0 ] && lvl=1
echo "${lvl}|disk|磁盘使用|${used} / ${total} (${pct}%)|根分区已用 ${pct}%"
}
check_mem() {
local pct; pct=$(free | awk '/Mem/{printf "%.0f", $3/$2 * 100}')
pct=${pct:-0}
local used total; used=$(free -h | awk '/Mem/{print $3}'); total=$(free -h | awk '/Mem/{print $2}')
local lvl=0
[ "$pct" -ge 90 ] && lvl=2
[ "$pct" -ge 80 ] && [ "$lvl" -eq 0 ] && lvl=1
echo "${lvl}|mem|内存使用|${used} / ${total} (${pct}%)|内存已用 ${pct}%"
}
check_backup() {
local dir="$BACKUP_DIR"
local last; last=$(ls -1t "$dir" 2>/dev/null | head -1)
if [ -z "$last" ]; then
echo "2|backup|备份|无备份文件|目录 ${dir} 无任何备份"
return
fi
local age; age=$(( ($(date +%s) - $(stat -c %Y "$dir/$last" 2>/dev/null || echo 0)) / 3600 ))
local lvl=0
[ "$age" -ge 48 ] && lvl=2
[ "$age" -ge 24 ] && [ "$lvl" -eq 0 ] && lvl=1
echo "${lvl}|backup|备份时效|${age} 小时前(${last}|最新备份距今 ${age} 小时"
}
check_services() {
local bad=""
for svc in rust-backend-green rust-backend-blue nginx; do
local st; st=$(systemctl is-active "$svc" 2>/dev/null || echo "unknown")
[ "$st" = "active" ] || bad="${bad}${svc}=${st} "
done
if [ -n "$bad" ]; then
echo "2|services|服务状态|异常服务: ${bad}|systemd 服务未运行"
else
echo "0|services|服务状态|全部 active|green/blue/nginx 运行正常"
fi
}
check_docker() {
local out; out=$(docker ps --format '{{.Names}} {{.Status}}' 2>/dev/null)
if [ -z "$out" ]; then
if docker info >/dev/null 2>&1; then
echo "0|docker|Docker 容器|无运行容器|当前无容器在运行"
else
echo "2|docker|Docker|不可用|docker 命令执行失败"
fi
return
fi
local bad; bad=$(echo "$out" | grep -v " Up " || true)
if [ -n "$bad" ]; then
echo "2|docker|Docker 容器|$(echo "$out" | grep -c " Up ") 个正常|异常容器: $(echo "$bad" | awk '{print $1}' | tr '\n' ' ')"
else
echo "0|docker|Docker 容器|$(echo "$out" | grep -c " Up ") 个运行中|全部正常"
fi
}
check_ssl() {
local expiry; expiry=$(echo "" | openssl s_client -connect xmclassmate.top:443 -servername xmclassmate.top 2>&1 | openssl x509 -noout -enddate 2>/dev/null | cut -d= -f2)
if [ -z "$expiry" ]; then
echo "1|ssl|SSL 证书|检测失败|无法获取证书到期时间"
return
fi
local days; days=$(( ($(date -d "$expiry" +%s 2>/dev/null || echo 0) - $(date +%s)) / 86400 ))
local lvl=0
[ "$days" -lt 7 ] && lvl=2
[ "$days" -lt 30 ] && [ "$lvl" -eq 0 ] && lvl=1
echo "${lvl}|ssl|SSL 证书|剩余 ${days} 天(${expiry}|证书将于 ${expiry} 到期"
}
# 支付卡单检测pending 超过 STUCK_ORDER_MINUTES
check_stuck_orders() {
local count orders
count=$(docker exec "$DB_CONTAINER" psql -U "$DB_USER" -d "$DB_NAME" -At -c \
"SELECT count(*) FROM payment_orders WHERE status='pending' AND created_at < NOW() - INTERVAL '${STUCK_ORDER_MINUTES} minutes'" 2>/dev/null)
if [ -z "$count" ]; then
return # 数据库不可用由 check_db 负责
fi
if [ "$count" -gt 0 ]; then
orders=$(docker exec "$DB_CONTAINER" psql -U "$DB_USER" -d "$DB_NAME" -At -c \
"SELECT order_no || ' (用户' || user_id || ', ' || to_char(created_at, 'MM-DD HH24:MI') || ')' FROM payment_orders WHERE status='pending' AND created_at < NOW() - INTERVAL '${STUCK_ORDER_MINUTES} minutes' ORDER BY created_at" 2>/dev/null | tr '\n' '; ')
echo "1|stuck_orders|支付卡单|${count} 笔 pending 超 ${STUCK_ORDER_MINUTES} 分钟|${orders}"
else
echo "0|stuck_orders|支付卡单|无|无卡单"
fi
}
check_db() {
local ok; ok=$(docker exec "$DB_CONTAINER" psql -U "$DB_USER" -d "$DB_NAME" -At -c "SELECT 1" 2>/dev/null)
if [ -z "$ok" ]; then
echo "2|db|数据库|不可用|docker exec psql 失败(容器 ${DB_CONTAINER}"
else
echo "0|db|数据库|正常|SELECT 1 成功"
fi
}
# ---------- 状态文件管理 ----------
# 格式: key lvl last_alert_epoch
load_state() { [ -f "$ALERT_STATE_FILE" ] && cat "$ALERT_STATE_FILE" || true; }
save_state() {
local tmp="${ALERT_STATE_FILE}.tmp"
{ for k in "${!NEW_STATE[@]}"; do echo "$k ${NEW_STATE[$k]}"; done; } > "$tmp"
mv "$tmp" "$ALERT_STATE_FILE" 2>/dev/null || true
}
# 读取状态:$1=key, $2=字段序号1=级别 2=时间);状态行格式为 "key lvl time"
get_state() {
local line; line=$(load_state | awk -v k="$1" '$1==k')
[ -n "$line" ] || { echo "-1"; return; }
echo "$line" | awk -v f="$2" '{print $(f+1)}'
}
# ---------- 主流程 ----------
NOW=$(date +%s)
COOLDOWN_SEC=$((ALERT_COOLDOWN_HOURS * 3600))
# 收集检测结果(格式: level|key|label|value|detail
RESULTS=""
for check in check_disk check_mem check_backup check_services check_docker check_ssl check_db check_stuck_orders; do
r=$($check 2>/dev/null)
[ -n "$r" ] && RESULTS="${RESULTS}${r}\n"
done
declare -A NEW_STATE
ALERTS="" # 需要发邮件的条目
WARN_COUNT=0
CRIT_COUNT=0
while IFS='|' read -r lvl key label value detail; do
[ -z "$key" ] && continue
[ "$lvl" -eq 0 ] && continue # 正常项不处理(下面统一清除状态)
old_lvl=$(get_state "$key" 1)
old_time=$(get_state "$key" 2)
[ "$old_time" = "-1" ] && old_time=0
should_alert=false
if [ "$lvl" -gt "$old_lvl" ]; then
should_alert=true # 级别上升 → 立即告警
elif [ "$lvl" -eq "$old_lvl" ] && [ $((NOW - old_time)) -ge "$COOLDOWN_SEC" ]; then
should_alert=true # 同级别冷却期后再次提醒
fi
NEW_STATE[$key]="${lvl} ${NOW}"
if [ "$should_alert" = true ]; then
[ "$lvl" -eq 2 ] && CRIT_COUNT=$((CRIT_COUNT + 1))
[ "$lvl" -eq 1 ] && WARN_COUNT=$((WARN_COUNT + 1))
ALERTS="${ALERTS}<tr><td style='padding:8px 10px;border-bottom:1px solid #e2e8f0'>$( [ "$lvl" -eq 2 ] && echo "<span style='color:#dc2626'>严重</span>" || echo "<span style='color:#d97706'>警告</span>" )</td><td style='padding:8px 10px;border-bottom:1px solid #e2e8f0'>${label}</td><td style='padding:8px 10px;border-bottom:1px solid #e2e8f0'>${value}</td><td style='padding:8px 10px;border-bottom:1px solid #e2e8f0;color:#475569'>${detail}</td></tr>\n"
alert_log "告警 [${key}] 级别=${lvl}: ${label} ${value}${detail}"
fi
done < <(printf "%b" "$RESULTS")
# 恢复检测:状态文件中存在但本次为 0/未出现的项 → 清除
while read -r key old_lvl _; do
[ -z "$key" ] && continue
cur_lvl=$(printf "%b" "$RESULTS" | awk -F'|' -v k="$key" '$2==k{print $1; exit}')
if [ -z "$cur_lvl" ] || [ "$cur_lvl" -eq 0 ]; then
unset 'NEW_STATE[$key]' 2>/dev/null || true
alert_log "恢复: ${key} 已恢复正常,状态已清除"
fi
done < <(load_state)
save_state
# ---------- 发送告警邮件 ----------
if [ -n "$ALERTS" ]; then
if [ "$CRIT_COUNT" -gt 0 ]; then
SUBJECT="[ASD 告警] ${CRIT_COUNT} 项严重 / ${WARN_COUNT} 项警告 — $(date '+%m-%d %H:%M')"
else
SUBJECT="[ASD 告警] ${WARN_COUNT} 项警告 — $(date '+%m-%d %H:%M')"
fi
BODY=$(cat <<EOF
<html><body style="font-family:-apple-system,'PingFang SC','Microsoft YaHei',sans-serif;max-width:640px;margin:0 auto;padding:16px">
<h2 style="color:#0f172a;margin:0 0 4px">⚠️ ASD 服务器异常告警</h2>
<div style="color:#94a3b8;font-size:13px;margin-bottom:16px">$(date '+%Y-%m-%d %H:%M:%S') · $(hostname)</div>
<table style="border-collapse:collapse;width:100%;font-size:13px">
<tr><th style="padding:8px 10px;background:#f1f5f9;text-align:left;color:#64748b">级别</th><th style="padding:8px 10px;background:#f1f5f9;text-align:left;color:#64748b">项目</th><th style="padding:8px 10px;background:#f1f5f9;text-align:left;color:#64748b">当前值</th><th style="padding:8px 10px;background:#f1f5f9;text-align:left;color:#64748b">说明</th></tr>
$(printf "%b" "$ALERTS")
</table>
<div style="color:#94a3b8;font-size:12px;margin-top:16px">下次同类提醒将在 ${ALERT_COOLDOWN_HOURS} 小时后,恢复正常后不再打扰。</div>
</body></html>
EOF
)
if [ "$DRY_RUN" = true ]; then
echo "---- 将发送告警邮件 ----"; echo "主题: ${SUBJECT}"; echo "$BODY" | head -30
else
if mail_send "$SUBJECT" "$BODY"; then
alert_log "告警邮件已发送: ${SUBJECT}"
else
alert_log "告警邮件发送失败: ${SUBJECT}"
fi
fi
else
alert_log "本次检查无异常($(date '+%H:%M:%S')"
[ "$DRY_RUN" = true ] && echo "当前无异常,无需告警。"
fi

View File

@@ -1,116 +1,207 @@
#!/bin/bash
# 每日健康报告 — 发送格式化摘要邮件到管理员
# 部署cp 到服务器 /root/rust/scripts/daily-report.shcron 每日执行
# ===========================================
# 每日健康报告 — HTML 摘要邮件(含支付/会员统计)
#
# 部署:
# cp scripts/mail-lib.sh scripts/daily-report.sh → 服务器 /root/rust/scripts/
# chmod +x /root/rust/scripts/daily-report.sh
# cronroot:
# 0 8 * * * /root/rust/scripts/daily-report.sh >> /root/rust/scripts/logs/daily-report.log 2>&1
#
# 说明:单项采集失败自动降级为 N/A不会阻断整封邮件
# 发送失败自动重试并记录 /root/rust/scripts/logs/mail.log。
# ===========================================
set -e
# 不用 set -e:采集失败不应导致邮件发不出去
set -u
SCRIPT_DIR=$(cd "$(dirname "$0")" && pwd)
# shellcheck source=mail-lib.sh
source "${SCRIPT_DIR}/mail-lib.sh"
# ---------- 可配置项 ----------
: "${DB_CONTAINER:=1Panel-postgresql-FtMo}" # PostgreSQL 容器名
: "${DB_NAME:=milkydata}" # 数据库名
: "${DB_USER:=milkydata}" # 数据库用户
: "${BACKUP_DIR:=/root/rust/backups/milkydata}" # 备份目录
TO="m943790568@163.com"
DATE=$(date "+%Y-%m-%d")
HOST=$(hostname)
SUBJECT="[ASD 服务器报告] ${DATE}${HOST}"
# ---------- 采集数据 ----------
# ---------- 状态徽章 ----------
# badge <ok|warn|crit> <文本>
badge() {
case "$1" in
ok) echo "<span style='background:#16a34a;color:#fff;padding:1px 8px;border-radius:10px;font-size:12px'>$2</span>" ;;
warn) echo "<span style='background:#d97706;color:#fff;padding:1px 8px;border-radius:10px;font-size:12px'>$2</span>" ;;
crit) echo "<span style='background:#dc2626;color:#fff;padding:1px 8px;border-radius:10px;font-size:12px'>$2</span>" ;;
*) echo "<span style='background:#64748b;color:#fff;padding:1px 8px;border-radius:10px;font-size:12px'>$2</span>" ;;
esac
}
# 根据磁盘/内存百分比判定级别(参数: 数值,返回 ok/warn/crit/unknown
pct_level() {
local v="$1" w="${2:-80}" c="${3:-90}"
case "$v" in
''|*[!0-9]*) echo "unknown"; return ;;
esac
if [ "$v" -ge "$c" ]; then echo "crit"
elif [ "$v" -ge "$w" ]; then echo "warn"
else echo "ok"; fi
}
# ---------- 系统状态 ----------
UPTIME=$(uptime -p | sed 's/up //')
LOAD=$(uptime | awk -F'load average:' '{print $2}' | xargs)
LOAD1=${LOAD%%,*} # 1 分钟负载(可能是小数,取整数部分比较)
LOAD1_INT=${LOAD1%%.*}
DISK_USED=$(df -h / | awk 'NR==2{print $3}')
DISK_TOTAL=$(df -h / | awk 'NR==2{print $2}')
DISK_PCT=$(df -h / | awk 'NR==2{print $5}')
DISK_PCT_STR=$(df -h / | awk 'NR==2{print $5}')
DISK_PCT=${DISK_PCT_STR%\%}
MEM_USED=$(free -h | awk '/Mem/{print $3}')
MEM_TOTAL=$(free -h | awk '/Mem/{print $2}')
MEM_PCT=$(free | awk '/Mem/{printf "%.0f", $3/$2 * 100}')
CPU_TEMP=$(cat /sys/class/thermal/thermal_zone0/temp 2>/dev/null | awk '{printf "%.1f°C", $1/1000}' || echo "N/A")
# Docker
DOCKER_STATUS=$(docker ps --format '{{.Names}} {{.Status}}' 2>/dev/null || echo "Docker 不可用")
# ---------- Docker ----------
DOCKER_STATUS=$(docker ps --format '{{.Names}} {{.Status}}' 2>/dev/null || echo "")
DOCKER_DOWN=$(echo "$DOCKER_STATUS" | grep -vc "Up " || true)
if [ -z "$DOCKER_STATUS" ]; then
DOCKER_BODY="<span style='color:#dc2626'>Docker 不可用或无运行容器</span>"
elif [ "$DOCKER_DOWN" -gt 0 ]; then
DOCKER_BODY="$(echo "$DOCKER_STATUS" | awk '{print $1": "$2}' | sed 's/^/ /' | sed 's/&/\&amp;/g;s/</\&lt;/g;s/>/\&gt;/g' | tr '\n' '<br>')"
else
DOCKER_BODY="$(echo "$DOCKER_STATUS" | awk '{print $1" "$2}' | sed 's/&/\&amp;/g;s/</\&lt;/g;s/>/\&gt;/g' | tr '\n' '<br>')"
fi
# SSL 证书
# ---------- SSL 证书 ----------
SSL_EXPIRY=$(echo "" | openssl s_client -connect xmclassmate.top:443 -servername xmclassmate.top 2>&1 | openssl x509 -noout -enddate 2>/dev/null | cut -d= -f2 || echo "检测失败")
SSL_DAYS=$(( ($(date -d "$SSL_EXPIRY" +%s 2>/dev/null) - $(date +%s)) / 86400 )) || SSL_DAYS="?"
if [ "$SSL_EXPIRY" != "检测失败" ]; then
SSL_DAYS=$(( ($(date -d "$SSL_EXPIRY" +%s 2>/dev/null) - $(date +%s)) / 86400 ))
else
SSL_DAYS=9999
fi
# 服务状态
# ---------- 服务状态 ----------
GREEN_STATUS=$(systemctl is-active rust-backend-green.service 2>/dev/null || echo "unknown")
BLUE_STATUS=$(systemctl is-active rust-backend-blue.service 2>/dev/null || echo "unknown")
NGINX_STATUS=$(systemctl is-active nginx 2>/dev/null || echo "unknown")
# 最近备份
LAST_BACKUP=$(ls -1t /root/rust/backups/milkydata/ 2>/dev/null | head -1)
BACKUP_AGE="未知"
# ---------- 备份 ----------
LAST_BACKUP=$(ls -1t "$BACKUP_DIR/" 2>/dev/null | head -1)
BACKUP_AGE_HOURS=""
BACKUP_AGE_TEXT="未知"
if [ -n "$LAST_BACKUP" ]; then
BACKUP_TIME=$(stat -c %Y "/root/rust/backups/milkydata/$LAST_BACKUP" 2>/dev/null)
BACKUP_TIME=$(stat -c %Y "$BACKUP_DIR/$LAST_BACKUP" 2>/dev/null)
NOW=$(date +%s)
AGE_HOURS=$(( (NOW - BACKUP_TIME) / 3600 ))
if [ "$AGE_HOURS" -lt 24 ]; then
BACKUP_AGE="${AGE_HOURS}小时前"
elif [ "$AGE_HOURS" -lt 48 ]; then
BACKUP_AGE="⚠️ ${AGE_HOURS}小时前"
else
BACKUP_AGE="${AGE_HOURS}小时前"
fi
BACKUP_AGE_HOURS=$(( (NOW - BACKUP_TIME) / 3600 ))
BACKUP_AGE_TEXT="${BACKUP_AGE_HOURS} 小时前"
fi
# Fail2Ban 封禁统计
# ---------- 安全 ----------
SSH_BANS=$(sudo fail2ban-client status sshd 2>/dev/null | grep "Currently banned" | awk '{print $NF}' || echo "0")
NGINX_BANS=$(sudo fail2ban-client status nginx-botsearch 2>/dev/null | grep "Currently banned" | awk '{print $NF}' || echo "0")
# 近期错误数(昨天 journalctl
YESTERDAY_ERRORS=$(journalctl -u rust-backend-green.service --since "1 day ago" --priority err --no-pager 2>/dev/null | wc -l)
YESTERDAY_WARNS=$(journalctl -u rust-backend-green.service --since "1 day ago" --priority warning --no-pager 2>/dev/null | wc -l)
# 应用统计(昨天活跃用户)
# ---------- 应用概况(昨日) ----------
ACTIVE_USERS=$(journalctl -u rust-backend-green.service --since "1 day ago" --no-pager 2>/dev/null | grep -c "user_id=\|登录成功\|上传成功" || echo "0")
API_COUNT=$(journalctl -u rust-backend-green.service --since "1 day ago" --no-pager 2>/dev/null | grep -c "INFO\|WARN\|ERROR" || echo "0")
# ---------- 构建邮件正文 ----------
# ---------- 支付与会员(数据库统计,不可用则 N/A ----------
# 服务器上通过 docker exec 访问 PostgreSQL与 check-members.sh 同模式)
DB_QUERY() {
docker exec "$DB_CONTAINER" psql -U "$DB_USER" -d "$DB_NAME" -At -c "$1" 2>/dev/null || echo ""
}
DB_OK=false
if [ -n "$(DB_QUERY 'SELECT 1')" ]; then DB_OK=true; fi
SUBJECT="[ASD 服务器报告] ${DATE}${HOST}"
PAY_ORDERS_TODAY="N/A"; PAY_PAID_TODAY="N/A"; PAY_STUCK="N/A"
MEM_ACTIVE="N/A"; MEM_EXPIRE_7D="N/A"
if [ "$DB_OK" = true ]; then
PAY_ORDERS_TODAY=$(DB_QUERY "SELECT count(*) FROM payment_orders WHERE created_at::date = CURRENT_DATE")
PAY_PAID_TODAY=$(DB_QUERY "SELECT count(*) FROM payment_orders WHERE status='paid' AND paid_at::date = CURRENT_DATE")
PAY_STUCK=$(DB_QUERY "SELECT count(*) FROM payment_orders WHERE status='pending' AND created_at < NOW() - INTERVAL '30 minutes'")
MEM_ACTIVE=$(DB_QUERY "SELECT count(*) FROM users WHERE is_member AND (membership_expires_at IS NULL OR membership_expires_at > NOW())")
MEM_EXPIRE_7D=$(DB_QUERY "SELECT count(*) FROM users WHERE is_member AND membership_expires_at IS NOT NULL AND membership_expires_at BETWEEN NOW() AND NOW() + INTERVAL '7 days'")
fi
[ -z "$PAY_ORDERS_TODAY" ] && PAY_ORDERS_TODAY="N/A"
[ -z "$PAY_PAID_TODAY" ] && PAY_PAID_TODAY="N/A"
[ -z "$PAY_STUCK" ] && PAY_STUCK="N/A"
[ -z "$MEM_ACTIVE" ] && MEM_ACTIVE="N/A"
[ -z "$MEM_EXPIRE_7D" ] && MEM_EXPIRE_7D="N/A"
MAIL=$(cat <<EOF
===========================================
ASD 服务器健康报告
${DATE}
主机: ${HOST}
===========================================
# ---------- 组装 HTML ----------
🖥 系统状态
- 运行时长: ${UPTIME}
- 负载: ${LOAD}
- 磁盘: ${DISK_USED} / ${DISK_TOTAL} (${DISK_PCT})
- 内存: ${MEM_USED} / ${MEM_TOTAL} (${MEM_PCT})
- CPU 温度: ${CPU_TEMP}
# 行生成器: row <指标> <当前值> <级别> <附加说明?>
row() {
echo "<tr><td style='padding:6px 10px;border-bottom:1px solid #e2e8f0;color:#475569'>$1</td><td style='padding:6px 10px;border-bottom:1px solid #e2e8f0'>$2</td><td style='padding:6px 10px;border-bottom:1px solid #e2e8f0;text-align:center'>$(badge "$3" "$4")</td></tr>"
}
section_title() {
echo "<tr><td colspan='3' style='padding:10px;background:#f8fafc;font-weight:bold;color:#0f172a;border-bottom:1px solid #e2e8f0'>$1</td></tr>"
}
📦 容器
$(echo "$DOCKER_STATUS" | sed 's/^/ /')
STYLE="font-family:-apple-system,'PingFang SC','Microsoft YaHei',sans-serif;background:#fff;max-width:640px;margin:0 auto;padding:16px"
TH="padding:8px 10px;background:#f1f5f9;color:#64748b;text-align:left;font-size:12px;border-bottom:2px solid #e2e8f0"
🔐 SSL 证书
域名: xmclassmate.top
到期: ${SSL_EXPIRY}
剩余: ${SSL_DAYS} 天
BODY=$(cat <<EOF
<html><body style="${STYLE}">
<h2 style="color:#0f172a;margin:0 0 4px">ASD 服务器健康报告</h2>
<div style="color:#94a3b8;font-size:13px;margin-bottom:16px">${DATE} · ${HOST} · 发送于 $(date "+%H:%M")</div>
🔄 服务状态
- 后端 (green): ${GREEN_STATUS}
- 后端 (blue): ${BLUE_STATUS}
- nginx: ${NGINX_STATUS}
<table style="border-collapse:collapse;width:100%;font-size:13px">
<tr><th style="${TH}">指标</th><th style="${TH}">当前值</th><th style="${TH}" width="90">状态</th></tr>
🛡 安全
- SSH 封禁: ${SSH_BANS} 个 IP
- nginx 封禁: ${NGINX_BANS} 个 IP
- 昨日错误: ${YESTERDAY_ERRORS}
- 昨日警告: ${YESTERDAY_WARNS}
$(section_title "🖥 系统状态")
$(row "运行时长" "$UPTIME" ok "OK")
$(row "系统负载" "$LOAD" "$([ "$LOAD1_INT" -ge 4 ] && echo crit || { [ "$LOAD1_INT" -ge 2 ] && echo warn || echo ok; })" "$([ "$LOAD1_INT" -ge 4 ] && echo "超载" || { [ "$LOAD1_INT" -ge 2 ] && echo "偏高" || echo "正常"; })")
$(row "磁盘使用" "${DISK_USED} / ${DISK_TOTAL} (${DISK_PCT_STR})" "$(pct_level "$DISK_PCT")" "$(pct_level "$DISK_PCT" | tr 'a-z' 'A-Z')")
$(row "内存使用" "${MEM_USED} / ${MEM_TOTAL} (${MEM_PCT}%)" "$(pct_level "$MEM_PCT" 80 90)" "$(pct_level "$MEM_PCT" 80 90 | tr 'a-z' 'A-Z')")
$(row "CPU 温度" "${CPU_TEMP}" ok "OK")
📊 应用概况(昨日)
- 日志事件数: ${API_COUNT}
- 活跃用户操作: ${ACTIVE_USERS}
$(section_title "📦 容器与服务")
$(row "Docker 容器" "运行中 $(echo "$DOCKER_STATUS" | grep -c "Up " || echo 0) 个 / 异常 $(echo "$DOCKER_STATUS" | grep -vc "Up " || echo 0) 个" "$([ "$DOCKER_DOWN" -gt 0 ] && echo crit || echo ok)" "$([ "$DOCKER_DOWN" -gt 0 ] && echo "有异常" || echo "正常")")
$(row "后端 green" "$GREEN_STATUS" "$([ "$GREEN_STATUS" = "active" ] && echo ok || echo crit)" "$([ "$GREEN_STATUS" = "active" ] && echo "运行中" || echo "异常")")
$(row "后端 blue" "$BLUE_STATUS" "$([ "$BLUE_STATUS" = "active" ] && echo ok || echo crit)" "$([ "$BLUE_STATUS" = "active" ] && echo "运行中" || echo "异常")")
$(row "nginx" "$NGINX_STATUS" "$([ "$NGINX_STATUS" = "active" ] && echo ok || echo crit)" "$([ "$NGINX_STATUS" = "active" ] && echo "运行中" || echo "异常")")
<tr><td colspan="3" style="padding:6px 10px;color:#64748b;font-size:12px;background:#fbfdff">${DOCKER_BODY}</td></tr>
💾 备份
- 最新备份: ${LAST_BACKUP:-无}
- 备份时效: ${BACKUP_AGE}
$(section_title "🔐 SSL 与安全")
$(row "SSL 证书" "${SSL_EXPIRY}(剩余 ${SSL_DAYS} 天)" "$([ "$SSL_DAYS" -lt 7 ] && echo crit || { [ "$SSL_DAYS" -lt 30 ] && echo warn || echo ok; })" "$([ "$SSL_DAYS" -lt 7 ] && echo "即将过期" || { [ "$SSL_DAYS" -lt 30 ] && echo "注意续期" || echo "正常"; })")
$(row "Fail2ban SSH" "${SSH_BANS} 个 IP 被封禁" ok "OK")
$(row "Fail2ban nginx" "${NGINX_BANS} 个 IP 被封禁" ok "OK")
$(row "昨日日志错误" "$YESTERDAY_ERRORS 条" "$([ "$YESTERDAY_ERRORS" -gt 50 ] && echo warn || echo ok)" "$([ "$YESTERDAY_ERRORS" -gt 50 ] && echo "偏多" || echo "正常")")
$(row "昨日日志警告" "$YESTERDAY_WARNS 条" "$([ "$YESTERDAY_WARNS" -gt 200 ] && echo warn || echo ok)" "$([ "$YESTERDAY_WARNS" -gt 200 ] && echo "偏多" || echo "正常")")
---
ASD 自动监控 | $(date "+%Y-%m-%d %H:%M")
$(section_title "💾 备份")
$(row "最新备份" "${LAST_BACKUP:-无}" "$([ -n "$BACKUP_AGE_HOURS" ] && { [ "$BACKUP_AGE_HOURS" -ge 48 ] && echo crit || { [ "$BACKUP_AGE_HOURS" -ge 24 ] && echo warn || echo ok; }; } || echo crit)" "$([ -n "$BACKUP_AGE_HOURS" ] && { [ "$BACKUP_AGE_HOURS" -ge 48 ] && echo "严重过期" || { [ "$BACKUP_AGE_HOURS" -ge 24 ] && echo "⚠ 超24小时" || echo "${BACKUP_AGE_TEXT}"; }; } || echo "未知")")
$(row "备份时效" "${BACKUP_AGE_TEXT}" ok "OK")
$(section_title "📊 应用概况(昨日)")
$(row "日志事件数" "$API_COUNT 条" ok "OK")
$(row "活跃用户操作" "$ACTIVE_USERS 次" ok "OK")
$(section_title "💰 支付与会员")
$(row "今日新订单" "$PAY_ORDERS_TODAY 笔" ok "OK")
$(row "今日支付成功" "$PAY_PAID_TODAY 笔" ok "OK")
$(row "卡单pending>30分钟" "$PAY_STUCK 笔" "$([ "$PAY_STUCK" -gt 0 ] 2>/dev/null && echo crit || echo ok)" "$([ "$PAY_STUCK" -gt 0 ] 2>/dev/null && echo "需人工处理" || echo "无")")
$(row "活跃会员" "$MEM_ACTIVE 人" ok "OK")
$(row "7天内到期会员" "$MEM_EXPIRE_7D 人" "$([ "$MEM_EXPIRE_7D" -gt 0 ] 2>/dev/null && echo warn || echo ok)" "$([ "$MEM_EXPIRE_7D" -gt 0 ] 2>/dev/null && echo "可发续费提醒" || echo "无")")
$( [ "$DB_OK" = false ] && echo "<tr><td colspan='3' style='padding:6px 10px;color:#dc2626;font-size:12px;background:#fef2f2'>⚠ 数据库不可用,支付/会员统计缺失</td></tr>" )
</table>
<div style="color:#94a3b8;font-size:12px;margin-top:16px">ASD 自动监控 | $(date "+%Y-%m-%d %H:%M") | 异常项请关注 alert.sh 实时告警</div>
</body></html>
EOF
)
# ---------- 发送邮件 ----------
echo "$MAIL" | msmtp -a default "$TO"
echo "✅ 健康报告已发送到 ${TO}"
# ---------- 发送 ----------
if mail_send "$SUBJECT" "$BODY"; then
echo "✅ 健康报告已发送到 ${MAIL_TO}$(date '+%H:%M:%S')"
else
echo "❌ 健康报告发送失败,详见 ${MAIL_LOG_DIR}/mail.log"
exit 1
fi

70
scripts/mail-lib.sh Normal file
View File

@@ -0,0 +1,70 @@
#!/bin/bash
# ===========================================
# mail-lib.sh — 邮件发送共享库
# 供 daily-report.sh / alert.sh 复用:
# mail_send <主题> <HTML正文> (带重试 + 日志 + 轮转)
# ===========================================
# ---------- 可配置项(环境变量可覆盖) ----------
: "${MAIL_TO:=m943790568@163.com}" # 收件人,多个用逗号分隔
: "${MAIL_FROM:=ASD 监控 <m943790568@163.com>}" # 发件人显示名
: "${MAIL_ACCOUNT:=default}" # msmtp 账户(~/.msmtprc 中的 account
: "${MAIL_LOG_DIR:=/root/rust/scripts/logs}" # 日志目录
: "${MAIL_RETRY:=3}" # 发送重试次数
: "${MAIL_LOG_MAX_KB:=1024}" # mail.log 轮转阈值KB
mkdir -p "$MAIL_LOG_DIR" 2>/dev/null || true
# 写日志
mail_log() {
local level="$1"; shift
echo "[$(date '+%Y-%m-%d %H:%M:%S')] [${level}] $*" >> "${MAIL_LOG_DIR}/mail.log" 2>/dev/null || true
}
# 日志轮转:超过阈值保留末尾 200 行
mail_rotate_log() {
local f="${MAIL_LOG_DIR}/mail.log"
[ -f "$f" ] || return 0
local size_kb=$(( $(stat -c %s "$f" 2>/dev/null || echo 0) / 1024 ))
if [ "$size_kb" -gt "$MAIL_LOG_MAX_KB" ]; then
tail -n 200 "$f" > "$f.tmp" 2>/dev/null && mv "$f.tmp" "$f" 2>/dev/null || true
mail_log INFO "mail.log 已轮转"
fi
}
# 发送 HTML 邮件UTF-8失败自动重试
# 用法: mail_send "主题" "HTML 正文"
mail_send() {
local subject="$1" body="$2"
mail_rotate_log
local recipients headers
recipients=$(echo "$MAIL_TO" | tr ',' ' ')
headers=$(cat <<EOF
From: ${MAIL_FROM}
To: ${MAIL_TO}
Subject: ${subject}
MIME-Version: 1.0
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: 8bit
EOF
)
local attempt=1 ok=0
while [ "$attempt" -le "$MAIL_RETRY" ]; do
if { echo "$headers"; echo; echo "$body"; } | msmtp -a "$MAIL_ACCOUNT" $recipients 2>>"${MAIL_LOG_DIR}/mail.log"; then
ok=1
mail_log INFO "邮件发送成功: ${subject}${MAIL_TO}(第 ${attempt} 次尝试)"
break
fi
mail_log WARN "邮件发送失败(第 ${attempt}/${MAIL_RETRY} 次): ${subject}"
attempt=$((attempt + 1))
[ "$attempt" -le "$MAIL_RETRY" ] && sleep $((attempt * 5))
done
if [ "$ok" -ne 1 ]; then
mail_log ERROR "邮件最终发送失败: ${subject}"
return 1
fi
return 0
}

65
scripts/verify-alipay-keys.sh Executable file
View File

@@ -0,0 +1,65 @@
#!/bin/bash
# ===========================================
# 支付宝密钥一致性核对脚本
# 对比本地 .env 与服务器 green/blue .env 的应用私钥,
# 并从本地私钥导出应用公钥,供与支付宝控制台比对。
# 防止 invalid-signature密钥不同步问题复发。
# 用法: bash scripts/verify-alipay-keys.sh [--remote-host HOST]
# ===========================================
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
PROJECT_ROOT="$(dirname "$SCRIPT_DIR")"
ENV_FILE="$PROJECT_ROOT/.env"
REMOTE_HOST="${2:-aliyun-server}"
RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m'; NC='\033[0m'
info() { echo -e "${GREEN}[INFO]${NC} $1"; }
warn() { echo -e "${YELLOW}[WARN]${NC} $1"; }
error() { echo -e "${RED}[ERROR]${NC} $1" >&2; }
if [ ! -f "$ENV_FILE" ] || ! grep -q '^ALIPAY_PRIVATE_KEY=' "$ENV_FILE"; then
error "本地 ${ENV_FILE} 未找到 ALIPAY_PRIVATE_KEY"
exit 1
fi
LOCAL_KEY=$(grep '^ALIPAY_PRIVATE_KEY=' "$ENV_FILE" | head -1 | cut -d'=' -f2- | tr -d '\r\n ')
LOCAL_MD5=$(printf '%s' "$LOCAL_KEY" | md5sum | cut -d' ' -f1)
info "本地 .env 私钥 MD5: ${LOCAL_MD5}"
ALL_MATCH=true
for ENV in green blue; do
REMOTE_MD5=$(ssh "$REMOTE_HOST" "grep '^ALIPAY_PRIVATE_KEY=' /root/rust/rust_backend_${ENV}/.env | cut -c20- | tr -d '\n' | md5sum | cut -d' ' -f1" 2>/dev/null || echo "N/A")
if [ "$REMOTE_MD5" = "$LOCAL_MD5" ]; then
info "服务器 ${ENV} 私钥一致 ✅"
else
warn "服务器 ${ENV} 私钥不一致(本地=${LOCAL_MD5} 远端=${REMOTE_MD5}"
ALL_MATCH=false
fi
done
if [ "$ALL_MATCH" = true ]; then
info "本地与服务器密钥一致"
else
warn "存在不一致!请检查服务器 .env 或重新部署"
fi
echo ""
echo "========== 应用公钥(与控制台比对) =========="
python3 -c "
import base64
from cryptography.hazmat.primitives import serialization
key_b64 = '$LOCAL_KEY'
try:
priv = serialization.load_der_private_key(base64.b64decode(key_b64), password=None)
except Exception:
priv = serialization.load_pem_private_key(key_b64.encode(), password=None)
pub_der = priv.public_key().public_bytes(
encoding=serialization.Encoding.DER,
format=serialization.PublicFormat.SubjectPublicKeyInfo
)
print(base64.b64encode(pub_der).decode())
"
echo ""
info "请到支付宝开放平台 → 密钥工具 → 查看应用公钥,与上方公钥比对"

View File

@@ -124,3 +124,71 @@ pub fn verify_temp_token(token: &str, secret: &str) -> Result<TempTokenClaims, S
Ok(decoded.claims)
}
#[cfg(test)]
mod tests {
use super::*;
const TEST_SECRET: &str = "unit-test-secret-0123456789";
#[test]
fn test_generate_and_verify_token_roundtrip() {
let token = generate_token(42, "openid_test_123", 2, TEST_SECRET).unwrap();
let claims = verify_token(&token, TEST_SECRET).unwrap();
assert_eq!(claims.user_id, 42);
assert_eq!(claims.openid, "openid_test_123");
assert_eq!(claims.user_type, 2);
assert!(claims.exp > claims.iat, "过期时间应晚于签发时间");
assert!(claims.exp > Utc::now().timestamp(), "令牌应未过期");
}
#[test]
fn test_verify_token_rejects_wrong_secret() {
let token = generate_token(42, "openid", 2, TEST_SECRET).unwrap();
assert!(verify_token(&token, "wrong-secret").is_err());
}
#[test]
fn test_verify_token_rejects_tampered_payload() {
let token = generate_token(42, "openid", 2, TEST_SECRET).unwrap();
// 篡改 token 的载荷部分(第 2 段),应验证失败
let mut parts: Vec<&str> = token.split('.').collect();
assert_eq!(parts.len(), 3);
parts[1] = "eyJ1c2VyX2lkIjo5OTl9";
let tampered = parts.join(".");
assert!(verify_token(&tampered, TEST_SECRET).is_err());
}
#[test]
fn test_verify_token_rejects_garbage() {
assert!(verify_token("not-a-jwt", TEST_SECRET).is_err());
assert!(verify_token("", TEST_SECRET).is_err());
}
#[test]
fn test_generate_refresh_token_is_random_uuid() {
let a = generate_refresh_token(1, TEST_SECRET).unwrap();
let b = generate_refresh_token(1, TEST_SECRET).unwrap();
assert_ne!(a, b, "两次生成的 refresh token 不应相同");
// UUID v4 格式8-4-4-4-12 十六进制
let parts: Vec<&str> = a.split('-').collect();
assert_eq!(parts.len(), 5);
assert!(parts.iter().all(|p| !p.is_empty()));
}
#[test]
fn test_temp_token_roundtrip_and_expiry_window() {
let token = generate_temp_token("openid_share", 777, TEST_SECRET, 10).unwrap();
let claims = verify_temp_token(&token, TEST_SECRET).unwrap();
assert_eq!(claims.openid, "openid_share");
assert_eq!(claims.resource_id, 777);
// 10 分钟有效期exp - iat 约等于 600 秒(允许 ±2 秒误差)
let lifetime = claims.exp - claims.iat;
assert!((598..=602).contains(&lifetime), "临时 token 有效期应为 10 分钟,实际 {lifetime}s");
assert!(verify_temp_token(&token, "wrong-secret").is_err());
}
}

View File

@@ -370,47 +370,6 @@ pub async fn get_user_orders(
Ok(rows)
}
/// 扫描超过 10 分钟的待支付订单,尝试自动确认(用于回调重试)
/// 返回成功确认的订单数量
pub async fn check_and_retry_pending_orders(pool: &PgPool) -> Result<u64, AppError> {
// 查找超过 10 分钟仍为 pending 的订单
let stuck_orders: Vec<(String,)> = sqlx::query_as(
r#"
SELECT order_no FROM payment_orders
WHERE status = 'pending'
AND created_at < NOW() - INTERVAL '10 minutes'
AND created_at > NOW() - INTERVAL '24 hours'
ORDER BY created_at ASC
"#,
)
.fetch_all(pool)
.await
.map_err(|e| AppError::Database(format!("查询待处理订单失败: {}", e)))?;
if stuck_orders.is_empty() {
return Ok(0);
}
tracing::info!(
"发现 {} 个待处理订单,尝试自动确认...",
stuck_orders.len()
);
let mut confirmed: u64 = 0;
for (order_no,) in &stuck_orders {
match confirm_payment_order_by_orderno(pool, order_no).await {
Ok(_) => {
confirmed += 1;
tracing::info!("定时任务自动确认订单: {}", order_no);
}
Err(e) => {
tracing::warn!("定时任务无法确认订单 {}: {}", order_no, e);
}
}
}
Ok(confirmed)
}
/// 检查会员到期前 7 天的用户,发送即将到期通知
pub async fn check_member_expiry_soon(pool: &PgPool) -> Result<u64, AppError> {
let affected = sqlx::query(

View File

@@ -597,7 +597,7 @@ pub async fn web_generate_login_code(
ts % 0xFFFF,
&random_part[..4]
);
let display_code = format!("ASD-{}", &short_code[..7].to_uppercase());
let display_code = format!("ASD-{}", short_code[..7].to_uppercase());
let code_for_db = display_code.clone();
// 4. 存入临时表(关联 user_id

View File

@@ -153,8 +153,14 @@ fn rsa2_verify(content: &str, sign: &str, public_key_pem: &str) -> Result<bool,
use rsa::pkcs8::DecodePublicKey;
use rsa::RsaPublicKey;
use sha2::Digest;
let public_key =
RsaPublicKey::from_public_key_pem(public_key_pem)
let public_key = RsaPublicKey::from_public_key_pem(public_key_pem).or_else(|_| {
base64::Engine::decode(&base64::engine::general_purpose::STANDARD, public_key_pem)
.map_err(|e| format!("支付宝公钥 base64 解码失败: {}", e))
.and_then(|der| {
RsaPublicKey::from_public_key_der(&der)
.map_err(|e| format!("支付宝公钥 DER 解析失败: {}", e))
})
})
.map_err(|e| format!("支付宝公钥解析失败: {}", e))?;
let sig_bytes = base64::Engine::decode(&base64::engine::general_purpose::STANDARD, sign)
.map_err(|e| format!("签名 Base64 解码失败: {}", e))?;
@@ -954,24 +960,40 @@ fn parse_alipay_form(body: &[u8]) -> BTreeMap<String, String> {
/// 手动 URL 解码percent-decoding
/// 注意:`+` 始终解码为空格form-urlencoded 标准),而非仅在 `%` 转义之后
fn urlencoding(s: &str) -> String {
let mut result = String::with_capacity(s.len());
let mut chars = s.bytes().peekable();
while let Some(b) = chars.next() {
let input: Vec<u8> = s.bytes().collect();
let mut bytes: Vec<u8> = Vec::with_capacity(input.len());
let mut i = 0;
while i < input.len() {
let b = input[i];
if b == b'%' {
let hi = chars.next().and_then(hex_val);
let lo = chars.next().and_then(hex_val);
let hi = input.get(i + 1).and_then(|&c| hex_val(c));
let lo = input.get(i + 2).and_then(|&c| hex_val(c));
if let (Some(h), Some(l)) = (hi, lo) {
result.push((h << 4 | l) as char);
bytes.push(h << 4 | l);
i += 3;
continue;
}
// 不完整的转义:原样保留 '%',后续字符照常处理
}
if b == b'+' {
bytes.push(b' ');
} else {
result.push('%');
bytes.push(b);
}
} else if b == b'+' {
result.push(' ');
} else {
result.push(b as char);
i += 1;
}
// 按字节收集后统一解码:优先 UTF-8失败回退 GBK兼容支付宝回调编码
match String::from_utf8(bytes.clone()) {
Ok(s) => s,
Err(_) => match encoding_rs::Encoding::for_label(b"gbk") {
Some(enc) => {
let (s, _, _) = enc.decode(&bytes);
s.into_owned()
}
None => String::from_utf8_lossy(&bytes).into_owned(),
},
}
result
}
fn hex_val(c: u8) -> Option<u8> {
@@ -1021,7 +1043,7 @@ pub async fn alipay_notify(
.join("&");
if let Err(e) = rsa2_verify(&sign_source, &sign, &config.alipay_public_key) {
tracing::warn!("支付宝签名验证失败: {}", e);
tracing::error!("支付宝回调签名验证失败: {}", e);
return HttpResponse::Ok().body("fail");
}
@@ -1207,7 +1229,7 @@ pub async fn alipay_refund_notify(
.join("&");
if let Err(e) = rsa2_verify(&sign_source, &sign, &config.alipay_public_key) {
tracing::warn!("支付宝退款签名验证失败: {}", e);
tracing::error!("支付宝退款回调签名验证失败: {}", e);
return HttpResponse::Ok().body("fail");
}
@@ -1614,3 +1636,171 @@ pub async fn payment_login_status(
membership_expires_at,
}))
}
#[cfg(test)]
mod tests {
use super::*;
// ===== 套餐定义 =====
#[test]
fn test_get_package_info_known_packages() {
for (pkg, expected_amount, expected_days) in [
("monthly", 590, Some(30)),
("quarterly", 1680, Some(90)),
("half_year", 3190, Some(182)),
("yearly", 6020, Some(365)),
] {
let info = get_package_info(pkg).unwrap_or_else(|| panic!("套餐 {pkg} 应存在"));
assert_eq!(info.amount, expected_amount, "套餐 {pkg} 金额");
assert_eq!(info.days, expected_days, "套餐 {pkg} 天数");
}
}
#[test]
fn test_get_package_info_unknown_package() {
assert!(get_package_info("weekly").is_none());
assert!(get_package_info("lifetime").is_none());
assert!(get_package_info("").is_none());
}
// ===== URL 解码 =====
#[test]
fn test_urlencoding_decodes_percent_escapes() {
assert_eq!(urlencoding("hello%20world"), "hello world");
assert_eq!(urlencoding("%E4%B8%AD%E6%96%87"), "中文");
assert_eq!(urlencoding("a%2Bb"), "a+b");
}
#[test]
fn test_urlencoding_plus_means_space() {
assert_eq!(urlencoding("a+b+c"), "a b c");
}
#[test]
fn test_urlencoding_handles_invalid_escapes() {
// 不完整的 % 序列按字面保留
assert_eq!(urlencoding("100%"), "100%");
assert_eq!(urlencoding("%2"), "%2");
assert_eq!(urlencoding("%zz"), "%zz");
}
#[test]
fn test_hex_val() {
assert_eq!(hex_val(b'0'), Some(0));
assert_eq!(hex_val(b'9'), Some(9));
assert_eq!(hex_val(b'a'), Some(10));
assert_eq!(hex_val(b'f'), Some(15));
assert_eq!(hex_val(b'A'), Some(10));
assert_eq!(hex_val(b'F'), Some(15));
assert_eq!(hex_val(b'g'), None);
assert_eq!(hex_val(b'-'), None);
}
#[test]
fn test_parse_alipay_form_utf8() {
let body = b"out_trade_no=ASD20260101&trade_status=TRADE_SUCCESS&total_amount=5.90";
let map = parse_alipay_form(body);
assert_eq!(map.get("out_trade_no").map(String::as_str), Some("ASD20260101"));
assert_eq!(map.get("trade_status").map(String::as_str), Some("TRADE_SUCCESS"));
assert_eq!(map.get("total_amount").map(String::as_str), Some("5.90"));
}
#[test]
fn test_parse_alipay_form_gbk_fallback() {
// 模拟支付宝回调使用 GBK 编码(含中文 subject
let raw = "out_trade_no=ASD123&subject=包月会员";
let (gbk_bytes, _, _) = encoding_rs::GBK.encode(raw);
let map = parse_alipay_form(&gbk_bytes);
assert_eq!(map.get("out_trade_no").map(String::as_str), Some("ASD123"));
assert_eq!(map.get("subject").map(String::as_str), Some("包月会员"));
}
#[test]
fn test_parse_alipay_form_empty() {
let map = parse_alipay_form(b"");
assert!(map.is_empty());
}
// ===== RSA2 签名 / 验签 =====
fn generate_test_key_pair() -> (String, String) {
use rsa::pkcs8::{EncodePrivateKey, EncodePublicKey, LineEnding};
use rsa::RsaPrivateKey;
let mut rng = rand::thread_rng();
let private_key = RsaPrivateKey::new(&mut rng, 2048).expect("生成测试密钥失败");
let private_pem = private_key.to_pkcs8_pem(LineEnding::LF).unwrap().to_string();
let public_pem = private_key
.to_public_key()
.to_public_key_pem(LineEnding::LF)
.unwrap()
.to_string();
(private_pem, public_pem)
}
#[test]
fn test_rsa2_sign_verify_roundtrip() {
let (private_pem, public_pem) = generate_test_key_pair();
let content = "out_trade_no=ASD123&trade_status=TRADE_SUCCESS";
let sign = rsa2_sign(content, &private_pem).unwrap();
assert!(!sign.is_empty());
assert!(rsa2_verify(content, &sign, &public_pem).unwrap());
}
#[test]
fn test_rsa2_verify_rejects_tampered_content() {
let (private_pem, public_pem) = generate_test_key_pair();
let sign = rsa2_sign("original content", &private_pem).unwrap();
// 调用处语义:`if let Err(e) = rsa2_verify(...)` 即拒绝,篡改内容必须返回 Err
assert!(rsa2_verify("tampered content", &sign, &public_pem).is_err());
}
#[test]
fn test_rsa2_sign_supports_base64_der_key() {
use rsa::pkcs8::EncodePrivateKey;
use rsa::RsaPrivateKey;
// 支付宝密钥工具导出的是 Base64(DER),不是 PEM 文本
let mut rng = rand::thread_rng();
let private_key = RsaPrivateKey::new(&mut rng, 2048).expect("生成测试密钥失败");
let der = private_key.to_pkcs8_der().unwrap();
let b64_key = base64::Engine::encode(&base64::engine::general_purpose::STANDARD, der.as_bytes());
let sign = rsa2_sign("hello", &b64_key).unwrap();
assert!(!sign.is_empty());
}
#[test]
fn test_rsa2_sign_rejects_invalid_key() {
assert!(rsa2_sign("content", "not-a-key").is_err());
}
// ===== 移动端 UA 检测 =====
#[test]
fn test_is_mobile_ua() {
assert!(is_mobile_ua("Mozilla/5.0 (iPhone; CPU iPhone OS 17_0 like Mac OS X)"));
assert!(is_mobile_ua("Mozilla/5.0 (Linux; Android 14; Pixel 8)"));
assert!(is_mobile_ua("HarmonyOS Browser"));
assert!(is_mobile_ua("Mozilla/5.0 (iPad; CPU OS 17_0 like Mac OS X)"));
assert!(!is_mobile_ua("Mozilla/5.0 (Windows NT 10.0; Win64; x64) Chrome/126.0"));
assert!(!is_mobile_ua("Mozilla/5.0 (Macintosh; Intel Mac OS X 14_0) Safari/605.1.15"));
}
// ===== 订单号格式 =====
#[test]
fn test_order_no_format_is_unique() {
// create_order 内部生成格式ASD{timestamp}{random},这里验证生成逻辑的幂等约束:
// 同一时刻多次生成的订单号不应相同(随机后缀)
let mut seen = std::collections::HashSet::new();
for _ in 0..100 {
let no = format!("ASD{}{:04}", chrono::Utc::now().timestamp_millis(), rand::random::<u16>());
assert!(seen.insert(no.clone()), "订单号重复: {no}");
}
}
}

View File

@@ -244,22 +244,13 @@ async fn main() -> std::io::Result<()> {
Err(e) => warn!("启动时清理 web 登录码失败: {}", e),
}
// 后台定时扫描待支付订单(每 5 分钟)
let pool_clone = pool.clone();
actix_web::rt::spawn(async move {
let mut interval = actix_web::rt::time::interval(std::time::Duration::from_secs(300));
interval.tick().await; // 跳过立即执行
loop {
interval.tick().await;
tracing::info!("[定时任务] 开始扫描待支付订单...");
match db::check_and_retry_pending_orders(&pool_clone).await {
Ok(confirmed) => {
if confirmed > 0 {
tracing::info!("[定时任务] 自动确认了 {} 个待支付订单", confirmed);
}
}
Err(e) => tracing::warn!("[定时任务] 扫描待支付订单失败: {}", e),
}
tracing::info!("[定时任务] 开始...");
// 会员到期前 7 天提醒
if let Err(e) = db::check_member_expiry_soon(&pool_clone).await {