安全审计修复 + BUG 修复 + 新功能

审计基准 a3ae072。安全 14 项(严重 4、高危 10)全修:JWT 启动强制
校验(缺失/默认值/短于 32 位拒绝启动);默认 admin 改为
ADMIN_INITIAL_PASSWORD 必填(≥12 位);BMC/SMTP/飞书口令 Fernet
加密入库(新增 crypto.py,密钥走环境变量或 0600 文件,存量明文
启动时迁移),HostOut 与设置接口脱敏;InfluxDB 默认凭证清除、
Compose 改 ${VAR:?must be set}、start.sh 遇空/默认值退出;CORS
白名单;SPA 防遍历;生产关 /docs + 安全头;JWT 8h + jti + 登出/
改密(改密吊销全部旧 token);python-jose→PyJWT;requirements
全钉 == 版;SSRF 校验层(新增 ssrf.py);登录限流 5 次锁 15 分钟;
容器非 root、删源码挂载。

BUG 30 项全修:恢复误发、脏数据隔离、采集超时 30s→10s +
Semaphore(8) + 失败退避、SDR 解析、组件接口错文件、时区 8 小时、
InfluxDB 静默丢数据、密码明文返回、配置死变量、删 init_admin.py、
显式 commit、邮件 TLS/飞书 keyword、趋势功耗 fallback;剔除 -C 0
无认证与 IPMI 1.5 降级。

优化 11 项:按需查询、设置批量读、公共 resolve_total_power、
SQLite WAL/外键/索引、Dockerfile 加固、start.sh 轮询健康、
alert-trigger.sh 按真实接口重写、InfluxDB 超时、补测试 + README
重写(37 个端点)。

新功能 2 个:POST /api/auth/change-password(+ logout)、
GET /api/monitor/export CSV 导出;前端 Hosts/Settings 同步掩码展示。

验证:36/36 单元测试通过;py_compile 全过。未做真实 BMC/InfluxDB
联调与 Docker 构建验证。审计报告见 AUDIT.md 及三分报告。
This commit is contained in:
2026-11-16 12:30:00 +08:00
parent a3ae072975
commit fa09603025
35 changed files with 3601 additions and 835 deletions
+33 -25
View File
@@ -3,14 +3,18 @@
# ============================================================
# Copy this file to .env and modify values as needed:
# cp .env.example .env
#
# SECURITY: values marked REQUIRED below have no default. The stack refuses
# to start without them. Never commit a real .env file to git.
# ============================================================
# ----- InfluxDB Configuration -----
# InfluxDB admin username
INFLUXDB_USERNAME=admin
# InfluxDB admin password (minimum 8 characters)
INFLUXDB_PASSWORD=admin123456
# InfluxDB admin password (REQUIRED, minimum 8 characters).
# Leave empty here and set a strong password in your real .env file.
INFLUXDB_PASSWORD=
# InfluxDB organization name
INFLUXDB_ORG=ipmi-monitor
@@ -18,8 +22,9 @@ INFLUXDB_ORG=ipmi-monitor
# InfluxDB bucket name for storing metrics
INFLUXDB_BUCKET=metrics
# InfluxDB admin API token (change this in production!)
INFLUXDB_TOKEN=my-super-secret-token
# InfluxDB admin API token (REQUIRED).
# Leave empty here and set a strong random token in your real .env file.
INFLUXDB_TOKEN=
# InfluxDB host port mapping (host:container)
INFLUXDB_PORT=8086
@@ -32,6 +37,30 @@ COMPOSE_PROJECT_NAME=ipmi-monitor
# Backend server host port mapping (host:container)
BACKEND_PORT=8000
# Runtime environment: development or production. Production disables the
# interactive API docs (/docs, /redoc, /openapi.json).
ENVIRONMENT=production
# JWT signing secret (REQUIRED, at least 32 characters).
# Generate one, e.g.: python3 -c "import secrets; print(secrets.token_urlsafe(32))"
# The backend refuses to start without it.
JWT_SECRET_KEY=
# Password for the initial admin account (REQUIRED on first boot only, when
# the user table is empty; at least 12 characters). Ignored once any user
# exists. Change it after first login via the change-password API.
ADMIN_INITIAL_PASSWORD=
# Optional Fernet key used to encrypt BMC passwords, the SMTP password and
# the Feishu webhook URL stored in the database. When empty, the backend
# generates a key once and persists it next to the SQLite file.
# Generate one, e.g.: python3 -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"
CREDENTIAL_ENCRYPTION_KEY=
# Optional comma-separated CORS allow-list (e.g. https://monitor.example.com).
# Empty means CORS is disabled (same-origin deployment).
CORS_ORIGINS=
# SQLite database URL used by backend. The default points to /app/data inside
# the backend container, which is backed by the backend-data Docker volume.
DATABASE_URL=sqlite+aiosqlite:///./data/ipmi_monitor.db
@@ -51,16 +80,6 @@ POLL_INTERVAL=60
# Log level: DEBUG, INFO, WARNING, ERROR, CRITICAL
LOG_LEVEL=INFO
# ----- Alert Thresholds -----
# Temperature alert threshold in °C (0 to disable)
ALERT_TEMP_THRESHOLD=80
# Fan speed alert threshold in RPM (0 to disable, alert when below threshold)
ALERT_FAN_THRESHOLD=0
# Power consumption alert threshold in Watts (0 to disable, alert when above threshold)
ALERT_POWER_THRESHOLD=500
# ----- Email Alert Configuration -----
# SMTP server host (leave empty to disable email alerts)
SMTP_HOST=
@@ -73,14 +92,3 @@ SMTP_USER=
# SMTP authentication password
SMTP_PASSWORD=
# Sender email address for alerts
ALERT_EMAIL_FROM=
# Comma-separated list of recipient email addresses for alerts
ALERT_EMAIL_TO=
# ----- Webhook Alert Configuration -----
# Webhook URL for alert notifications (leave empty to disable)
# Supports Slack, DingTalk, WeChat Work, or generic webhooks
WEBHOOK_URL=
+279
View File
@@ -0,0 +1,279 @@
# IPMI Monitor — BUG 与优化审计报告
**审计范围:** `~/workspace/work/ipmi-monitor`,HEAD `a3ae072`,只读审计;后端约 4000 行(`backend/app/`),前端 Vue(`frontend/src/`,`dist/` 未审)。测试仅 `backend/tests/test_total_power_fallback.py` 一个文件(3 个用例,只覆盖功耗 fallback)。
## 数量统计
| 分组 | 数量 |
|---|---:|
| 已确认的 BUG(B1–B30) | 30 |
| 优化建议(O1–O16) | 16 |
| 合计 | 46 |
---
## 一、已确认的 BUG(30 项)
### B1 — 未达持续时间就“恢复”:瞬时超标会误发恢复通知
- **文件:** `backend/app/services/alert_engine.py:309-317`
- **问题:** `check_alerts()` 的 `else` 分支只判断 `key in _first_breach`,不判断 `_alert_active`。只要曾经超标(哪怕未满 `rule.duration`、从未真正告警),下一次不超标就调用 `_create_recovery_notification()`
- **复现/影响:** 温度短暂冲到 81℃ 又回落(`duration=300` 未满足),用户会收到一封“已恢复”邮件/飞书,却从未收到过告警,通知语义自相矛盾
- **修复:** 仅当 `_alert_active.get(key)` 为真时才发恢复通知;未触发告警时只清理 `_first_breach`
### B2 — 告警循环中一次脏数据会中断全部规则检查
- **文件:** `backend/app/services/alert_engine.py:272-276`
- **问题:** `metric_value = point["value"]` 无判空/类型校验;Influx 返回 `None` 或非数值时,`evaluate_rule()` 做 `None > threshold` 抛 `TypeError`,整个 `check_alerts()` 中断,后续规则/主机本轮都不检查(外层 `scheduler.py` 只记一条 `Alert engine error` 日志)
- **复现/影响:** 某台主机某传感器值为 `None`,全站告警本轮静默
- **修复:** 取值后 `if metric_value is None: continue`,并用 `float()` 转换、捕获异常后按单条规则隔离(`try/except` 包住每个 `(rule, host)` 循环体)
### B3 — 采集超时不重试且子进程可能泄漏
- **文件:** `backend/app/services/ipmi_collector.py:172, 207-215`
- **问题:** `_run_ipmitool_with_auth()` 用 `asyncio.wait_for(proc.communicate(), timeout=30)`,超时抛 `TimeoutError`;但 `collect_host()` 内层只捕获 `RuntimeError`,超时直接跳出 6 种认证方式的重试循环。且超时后未 `proc.kill()`/`proc.wait()`,被取消的 `communicate()` 可能留下僵尸 `ipmitool` 进程
- **复现/影响:** BMC 响应慢(30 秒超时)的主机本应试下一种认证方式,却直接判采集失败;多次超时后残留进程累积
- **修复:** 内层捕获 `(RuntimeError, TimeoutError, asyncio.TimeoutError)`;超时后 `proc.kill()` 并 `await proc.wait()`;或改用 `asyncio.timeout()` + 显式清理
### B4 — 单主机最坏采集 180 秒,超过 60 秒采集间隔,调度会积压/跳过
- **文件:** `backend/app/services/ipmi_collector.py:238-258`、`backend/app/services/scheduler.py:64-72`
- **问题:** 6 种认证方式 × 每次 30 秒超时 = 单主机最坏 180 秒;`COLLECT_INTERVAL=60`,且 `max_instances=1`(无 `misfire_grace_time`/`coalesce`)。一台离线主机会拖住整轮采集,后续轮次被丢弃或排队
- **复现/影响:** 1 台离线 + N 台在线时,在线主机的采集间隔被拉长到数分钟,`last_seen` 陈旧、Dashboard 显示离线
- **修复:** 每种方式超时改为 5~10 秒,总预算封顶(如单主机 30 秒);失败主机记入退避(如下一轮跳过);`add_job` 加 `coalesce=True, misfire_grace_time=30`
### B5 — 负温度被解析成正数
- **文件:** `backend/app/services/ipmi_collector.py:35-38`
- **问题:** `_TEMP_RE = (?P<val>[0-9.]+)\s*degrees?\s*C` 不含负号和小数点外的分隔符,`-5 degrees C` 匹配到 `5`,返回 `+5.0`;`1,200 RPM` 同理只取到 `1`
- **复现/影响:** 零下传感器、千分位读数全部错;阈值告警(低温告警)永远不触发
- **修复:** 改为 `(?P<val>-?[0-9][0-9,]*(\.[0-9]+)?)`,解析时去掉 `,` 再 `float()`
### B6 — 状态类传感器恒返回“正常”,电源离散值被当成功耗
- **文件:** `backend/app/services/ipmi_collector.py:63-77`
- **问题:** `_classify_metric()` 的 `status` 分支不接收 `status` 参数,名字含 `status` 即返回 `("status", 1.0)`,哪怕 SDR 状态是 `nc/nr`(严重/不可恢复);`power supply` 分支对 `Presence detected` 等非数值 `int(...,0)` 抛 `ValueError` 后硬编码 `0.0`,把“在位”记成 0W 的 `power` 指标,污染功耗统计
- **复现/影响:** 故障传感器在 Influx 里仍是 `1`;PSU 在位信息丢失,且多出 0/1W 的假功耗点
- **修复:** 把 `status` 传进 `_classify_metric`,按 `ok→1,其余→0`;离散电源传感器应归为 `status` 类型而非 `power`
### B7 — 非 `ok/ns/nc/nr/nd/na` 的状态行解析错位
- **文件:** `backend/app/services/ipmi_collector.py:95-118`
- **问题:** `parse_sdr_output()` 只认 6 个状态 token;状态为 `disabled` 等时 `status_idx` 为 `None`,回退到 legacy 分支取 `parts[1]` 当值——对 5 段新格式行,`parts[1]` 是传感器编号(如 `01h`),值和状态全错,该传感器直接丢失
- **复现/影响:** 含 `disabled` 传感器的 BMC,部分传感器在前端消失
- **修复:** 按段数分流:5 段按新格式(`name|id|status|entity|value`)解析,3 段按 legacy;状态 token 白名单扩充 `disabled/cr/na` 等,未知状态也按位置解析而非回退
### B8 — `component_parser` 直接丢弃 legacy 3 段 SDR 行
- **文件:** `backend/app/services/component_parser.py:42`
- **问题:** `if len(parts) < 4: continue`。legacy 格式 `CPU1 Temp | 45 degrees C | ok` 只有 3 段,整行跳过;`ipmi_collector.py` 的 `parse_sdr_output` 却支持 legacy
- **复现/影响:** 老 BMC 的组件(CPU/内存/PCIe/硬盘/风扇/PSU)全部显示 0,前端 Components 页为空
- **修复:** `len(parts) < 3` 才跳过;3 段时 `value=parts[1]、status=parts[2]、entity_id=""`
### B9 — 组件去重缺失 + PSU/硬盘在位判断恒为假
- **文件:** `backend/app/services/component_parser.py:339-430`
- **问题:** (1)`_parse_drive/_parse_fan/_parse_power_supply` 无去重字典(CPU/内存/PCIe 有),同一传感器出现两次即重复计数;(2)`_parse_power_supply` 要求值里含 `presence detected/present`,但 5 段行值是瓦数(如 `132 Watts`),恒为 `present=False`,PSU 在位永远 0;(3)`_parse_drive` 要求名字含 `_`,注释里说的 `Drive 0`(无下划线)永远匹配不上
- **复现/影响:** Components 页 PSU、硬盘计数错误;重复行导致总数虚高
- **修复:** 三类解析加 `seen` 字典去重;PSU/硬盘的在位判断改用 SDR 的 `status` 段(`ok` 即在位),而非值字段;`Drive 0` 用正则 `^Drive\s+\d+$` 匹配
### B10 — `/api/monitor/components/{host_id}` 读了一个从不写入的文件
- **文件:** `backend/app/routers/monitor.py:428-430`
- **问题:** 该端点读 `/tmp/ipmi_raw.txt`(无主机后缀),但全仓库没有任何地方写这个文件——`collect_host()` 写的是 `/tmp/ipmi_raw_{host.id}.txt`(`ipmi_collector.py:275`)。旁边的 `/grouped` 端点(`:470`)用的才是带后缀的正确路径
- **复现/影响:** 该端点永远返回 `{"error": "Raw IPMI data file not found"}`,且不分主机,任何主机都返回同一份(不存在的)数据
- **修复:** 改为 `Path(f"/tmp/ipmi_raw_{host_id}.txt")`,与 `/grouped` 一致;或删掉该重复端点
### B11 — 历史接口时区错 8 小时,`parse_dt` 容错分支本身会崩
- **文件:** `backend/app/routers/monitor.py:86-102, 383-396`
- **问题:** (1)无时区参数默认 `datetime.now()`(容器时区 Asia/Shanghai,UTC+8),而 `influxdb.py:120-127` 把 naive 时间当 UTC 查,导致默认查的是“未来 8 小时前 1 小时”的窗口,最新 1 小时数据查不到;(2)`parse_dt` 的 `except ValueError` 分支 `dt_str.split('+')[0].split('-')[0][:19]` 对 `2026-11-16T10:00:00+08:00` 会切成 `2026`,`fromisoformat("2026")` 再抛 `ValueError`,返回 500。同一函数在两个端点复制粘贴了两份
- **复现/影响:** History / Multi-history 默认时间范围查不到数据;传非法时间字符串直接 500
- **修复:** 默认值用 `datetime.now(timezone.utc)`;`parse_dt` 失败时返回 `None` 或 400,不要二次 `fromisoformat`;抽成公共函数放一处
### B12 — 历史/汇总接口无输入校验、无上限,可被打爆
- **文件:** `backend/app/routers/monitor.py:82, 145, 399`
- **问题:** `host_ids` 直接 `[int(x) for x in host_ids.split(',')]`,`abc` 即 500;`metric_type` 未校验,直接拼进 Flux(见 B20);`query_metrics()` 无 `limit`/`aggregateWindow`,30 天范围全量点一次返回,前端/内存都可能被打爆
- **复现/影响:** `GET /api/monitor/history/multi?host_ids=abc` 500;`time_range=30d` 的趋势接口返回数万点
- **修复:** 用 Pydantic/Query 校验 `host_ids` 与 `metric_type`(白名单);`host_ids` 限长(如 ≤50);Flux 加 `|> limit(n: 5000)` 或按窗口聚合
### B13 — `dashboard/totals` 算出的分组汇总被丢弃,且 `None` 值会 500
- **文件:** `backend/app/routers/monitor.py:219-302`
- **问题:** (1)`group_totals` 算完(`:277-296`)后 `return`(`:298-302`)只返回 `hosts` 和 `groups`(id/name),分组总功耗/平均温度根本没给前端,是死代码;(2)`total_power = m.get('value', 0)` 若为 `None`,`if total_power > 0` 抛 `TypeError`,整个 Dashboard 500;(3)`avg_temperature` 实际是各主机 `max_temperature` 的平均,命名误导
- **复现/影响:** 任一功耗传感器值为 `None` 即 Dashboard 全挂;分组汇总前端永远拿不到
- **修复:** `return` 里加上 `group_totals`;取值时 `value or 0` 并 `float()` 转换;改名 `avg_max_temperature` 或按真实均值算
### B14 — Influx 写入/查询失败被静默吞掉,采集仍报成功
- **文件:** `backend/app/influxdb.py:76-77, 99-100, 149-150`、`backend/app/services/ipmi_collector.py:321, 330-334`
- **问题:** `write_metric/write_metrics_batch/query_*` 全部 `except Exception: logger.exception` 后返回 `None/[]`;`collect_host()` 不检查返回值,照常更新 `last_seen` 并置 `success=True`
- **复现/影响:** InfluxDB 宕机期间数据全丢,但 `last_seen` 仍刷新、Dashboard 显示主机在线、告警无数据可查,无任何可见报错
- **修复:** 写入失败应抛异常或返回布尔值,`collect_host()` 失败时不更新 `last_seen`、置 `success=False`;查询失败与“确实无数据”要区分(抛异常 vs 空列表)
### B15 — 同步 InfluxDB 调用阻塞整个事件循环
- **文件:** `backend/app/influxdb.py:54-100`、`backend/app/routers/monitor.py:107, 149, 172, 219`、`backend/app/services/alert_engine.py:272`
- **问题:** `InfluxDBClient` 是同步客户端(`SYNCHRONOUS` 写),却在 `async` 路由/采集协程里直接调用;一次慢查询会卡住所有请求和采集
- **复现/影响:** Influx 慢(或超时无配置)时 API 全部卡死
- **修复:** 写入/查询包进 `asyncio.to_thread()`(邮件已用此模式,`email_notify.py` 有先例),或换异步客户端;给客户端配超时
### B16 — 告警持续时间/冷却/激活态全放进程内存,重启即丢、无清理
- **文件:** `backend/app/services/alert_engine.py:24-39, 282-317`
- **问题:** `_first_breach/_last_notify/_alert_active` 三个全局 dict:重启后持续计时归零(刚超标的告警要重新等满 `duration`);规则/主机删除、禁用后状态永不清理,无限增长;多 worker 部署时各进程状态不一致
- **复现/影响:** 重启后告警延迟一个 `duration`;长期运行内存缓慢增长
- **修复:** 持久化到 SQLite(`alert_state` 表)或至少在 `check_alerts()` 开头清理已删除/禁用规则的状态;`duration` 计时以 Influx 首次超标时间点为准
### B17 — 规则同时绑主机和分组时分组被静默忽略;绑定已禁用主机仍会告警
- **文件:** `backend/app/services/alert_engine.py:253-266`、`backend/app/routers/alerts.py:24-32`
- **问题:** `check_alerts()` 优先 `host_id`,`group_id` 被忽略;`host_id` 分支不滤 `enabled`,而 `group_id`/全部主机分支滤 `enabled`,行为不一致。`RuleCreate` 也未校验两者互斥
- **复现/影响:** 规则配了主机+分组,用户以为分组都覆盖,实际只查一台;禁用主机仍收告警
- **修复:** 建/改规则时校验 `host_id` 与 `group_id` 至多其一;`host_id` 分支同样只查 `enabled` 主机
### B18 — 告警规则创建无任何值域校验
- **文件:** `backend/app/routers/alerts.py:24-32`
- **问题:** `condition`、`metric_type` 无白名单,`duration` 无 `ge=0`,`threshold` 可为 NaN。非法 `condition` 在 `evaluate_rule()`(`alert_engine.py:46-62`)只记 warning 返回 `False`,规则静默永不触发
- **复现/影响:** 前端手误把 `condition` 写成 `greater`,规则建成功但永远不告警,无任何反馈
- **修复:** `condition: Literal["gt","lt","eq","gte","lte"]`,`metric_type` 白名单,`duration: Field(ge=0)`,`threshold` 拒绝 NaN/Inf
### B19 — 告警历史先落库、后通知,且“已激活”标记与落库脱钩
- **文件:** `backend/app/services/alert_engine.py:70-115, 282-296`
- **问题:** (1)`_create_alert_history()` 先 `commit` 历史再发邮件/飞书,`notify_type` 按开关填写(`both/email/feishu`),与实际是否发出无关——SMTP 未配、飞书返回非 0 码时历史仍写“已通知”;(2)`_alert_active[key]=True` 在 `if host_obj:` 之外,即使主机已删、未建任何历史,也被标记已激活,之后只在冷却后才重试
- **复现/影响:** 通知失败无法从历史里发现;主机删除间隙的一次告警永久丢失
- **修复:** 通知结果回写历史(如 `notify_status`);仅当历史创建成功才置 `_alert_active`
### B20 — 密码明文落库且经 API 明文返回;设置接口明文返回 SMTP 密码
- **文件:** `backend/app/routers/hosts.py:46-52`(`HostOut` 含 `ipmi_password`)、`backend/app/models.py:73-79`(明文列)、`backend/app/routers/settings.py:31-40`(`SettingsResponse` 含 `smtp_password`)
- **问题:** 主机列表/创建/更新接口把 BMC 密码原样返回给任何登录用户;设置接口同理返回 SMTP 密码。另 `init_admin.py:43` 与 `database.py:99-109` 的默认管理员密码为硬编码值(此处不引述具体值)
- **复现/影响:** 任一登录账号可导出全部 BMC 密码;日志/浏览器历史亦可能留存
- **修复:** `HostOut` 去掉密码字段(更新时空密码表示不改,已有此逻辑);DB 密码改用 Fernet/密钥加密存储;设置接口返回掩码(如 `********`),只在用户填写新值时更新
### B21 — 配置名错位:`.env`/Compose 里一半变量根本没人读
- **文件:** `backend/app/config.py:27-37`、`docker-compose.yml:44-60`、`.env.example:49-86`
- **问题:** 代码只认 `COLLECT_INTERVAL`,而 Compose/`.env.example` 传的是 `POLL_INTERVAL`;代码认 `SMTP_FROM/FEISHU_WEBHOOK_URL`,Compose 传的是 `ALERT_EMAIL_FROM/WEBHOOK_URL`;`ALERT_TEMP_THRESHOLD/ALERT_FAN_THRESHOLD/ALERT_POWER_THRESHOLD/ALERT_EMAIL_TO` 全仓库无人读取(告警只走 DB 里的规则)
- **复现/影响:** 用户按 `.env.example` 改采集间隔、阈值、收件人后毫无效果,误以为已生效
- **修复:** 二选一:`Settings` 增加别名(`POLL_INTERVAL` 等)或删掉 Compose/`.env.example` 里的无效变量并在 README 注明“阈值在页面规则里配”
### B22 — `init_admin.py` 根本跑不起来
- **文件:** `backend/app/init_admin.py:45-51`
- **问题:** `async with engine.begin() as conn:` 拿到的是 `AsyncConnection`,没有 `.add()` 方法(应为 `AsyncSession`),执行即抛 `AttributeError`;且同一文件混用 `passlib`,而 `database.py`/`auth.py` 用原生 `bcrypt`,哈希实现不一致
- **复现/影响:** 运行该脚本初始化管理员必失败(`database.py:init_db` 里的同类逻辑才是实际生效的,造成两份初始化逻辑并存)
- **修复:** 改用 `async_session` 写库,或删掉该脚本只保留 `init_db`;统一用 `auth.py` 的 `hash_password`
### B23 — 删除接口依赖 `get_db` 的隐式提交,行为脆弱且不一致
- **文件:** `backend/app/database.py:48-58`、`backend/app/routers/hosts.py:130-138`、`backend/app/routers/alerts.py:160-168`、`backend/app/routers/groups.py:105-113`
- **问题:** `get_db()` 在 `yield` 后统一 `commit()`;删除接口只 `await db.delete(...)`,不 `flush/commit`,靠依赖收尾提交。创建/更新接口却显式 `flush/refresh`。一旦依赖被改、或在非请求上下文调用,删除即丢失;且 `GET` 请求也会走一次无意义的 `commit()`
- **复现/影响:** 删除主机/规则/分组后若依赖逻辑变化即删不掉;读请求多一次提交开销
- **修复:** 每个写接口显式 `await db.commit()`;`get_db` 只 `yield`,异常时 `rollback`,去掉成功后自动提交
### B24 — 告警邮件无视 `smtp_use_tls`,非 465 端口强制 STARTTLS
- **文件:** `backend/app/services/email_notify.py:182-191, 238-247`
- **问题:** `_send()` 里 `if smtp_port != 465: server.starttls()` 无条件执行,`smtp_use_tls`(设置页可配)根本没传进来;25 端口明文服务器、或不支持 STARTTLS 的服务器会直接失败。测试邮件(`settings.py:180-189`)却正确按开关处理,两处行为不一致
- **复现/影响:** 用 25 端口明文 SMTP 的用户,测试邮件能通,真实告警邮件永远发不出
- **修复:** `send_alert_email/send_recovery_email` 接收并使用 `smtp_use_tls`;仅当开关开且非 465 时才 `starttls()`
### B25 — 飞书关键词安全校验永远通不过
- **文件:** `backend/app/services/alert_engine.py:126-150`、`backend/app/services/feishu_notify.py:13-80`
- **问题:** 设置里有 `feishu_keyword`,测试飞书(`settings.py`)会把它拼进消息,但真实告警只读 `feishu_webhook_url`,`send_feishu_alert/send_feishu_recovery` 签名里根本没有 keyword 参数
- **复现/影响:** 飞书机器人开启“关键词”安全设置时,真实告警被拒(仅记一条 warning),测试消息却能通,极具迷惑性
- **修复:** 告警路径同样读取 `feishu_keyword` 并拼进卡片文本(或 `send_feishu_*` 增加 `keyword` 参数)
### B26 — 测试邮件接口:端口解析在 `try` 外、SMTP 阻塞且可能泄漏连接
- **文件:** `backend/app/routers/settings.py:163, 175-191`
- **问题:** `smtp_port = int(await get_setting(db, "smtp_port"))` 在 `try` 之前,DB 里存了非数字即 500(`smtp_port` 是自由字符串字段,无校验);`smtplib.SMTP(host, port)` 同步调用且无 `timeout`,在 `async` 端点里可无限挂起;`server.quit()` 只在成功路径,`login/send_message` 抛异常即泄漏连接
- **复现/影响:** 设置页填错端口字符串即 500;SMTP 服务器不响应时整个 API 卡住
- **修复:** `int()` 移入 `try` 并校验范围;`SMTP(..., timeout=15)` 并用 `asyncio.to_thread()`;`quit()` 放 `finally`(或用 `with smtplib.SMTP(...) as server`)
### B27 — 告警邮件模板缺字段即崩,且与飞书通知互相复制
- **文件:** `backend/app/services/email_notify.py:166-168, 222-224`、`backend/app/services/feishu_notify.py:13-148`
- **问题:** `_PLAIN_TEMPLATE.format(**alert_info)` 缺任一 key 即 `KeyError`(`send_*` 外无 `try`,靠调用方兜底);HTML 模板用字符串 `replace`,缺字段会残留 `{{ ... }}`。邮件告警/恢复、飞书告警/恢复各约百行重复,仅模板不同
- **复现/影响:** 调用方少传一个字段,邮件直接不发;改一处模板要改两处
- **修复:** 用 `string.Template.safe_substitute` 或 Jinja2(依赖里已有 `jinja2`);抽公共 `_send_email/_send_feishu`
### B28 — SPA 兜底路由掩盖 API 404,且未防路径穿越
- **文件:** `backend/app/main.py:115-124`
- **问题:** `@app.get("/{full_path:path}")` 对所有未匹配路径返回 `index.html`(200),包括 `/api/xxx` 打错的路径,前端 `api.js` 收到 HTML 当 JSON 解析失败,报错信息失真;`file_path = _frontend_dist / full_path` 未 `resolve()`/校验 `is_relative_to(_frontend_dist)`,`../` 可拼出 dist 之外的路径
- **复现/影响:** API 404 变 200+HTML;存在越界读取静态目录外文件的风险
- **修复:** 兜底路由排除 `/api/` 前缀(返回 404);`file_path.resolve()` 后校验在 `_frontend_dist` 内,否则 404
### B29 — CORS `allow_origins=["*"]` 配 `allow_credentials=True`
- **文件:** `backend/app/main.py:65-71`
- **问题:** 通配符源 + 允许凭证组合,浏览器本应拒绝,实际等于把 JWT Bearer 的跨站风险敞开
- **修复:** 收敛为前端实际域名白名单;开发环境才放宽
### B30 — 趋势接口的功耗过滤把 PSU fallback 主机排除在外
- **文件:** `backend/app/routers/monitor.py:340-350`
- **问题:** `trend` 对 `power` 只保留 `Total_Power/Power/Pwr Consumption` 三个名字,`PSU{N}_PIN` 的点被丢弃——而这正是 Gooxi12 等无原生功耗传感器主机的唯一功耗来源(`component_parser.py` 的 fallback 和 `dashboard/totals` 都认 PSU_PIN)
- **复现/影响:** Gooxi12 的功耗趋势图永远是空的,与 Dashboard 总功耗有数矛盾
- **修复:** 趋势的功耗分支同样做“原生优先、否则 PSU_PIN 求和”的聚合(或在入库时多写一条合成的 `Total_Power` 点)
---
## 二、优化建议(16 项)
### O1 — 每次前端轮询触发 4×N 次 Influx 查询
- **文件:** `backend/app/influxdb.py:209-214`、`backend/app/routers/monitor.py:149, 172, 219`
- **问题:** `query_latest_grouped()` 对每台主机固定查 4 种类型(temperature/fan/voltage/power),即使调用方只要 1 种(如 `/summary/temperature`);Dashboard 每 10 秒轮询一次,50 台主机即每轮 200 次 Flux 查询
- **建议:** `query_latest_grouped(host_id, only: set[str] | None)` 按需查;或一次 Flux 用 `|> filter(fn: (r) => contains(value: r._measurement, set: [...]))` 合并
### O2 — 历史/趋势无分页、无降采样、无上限
- **文件:** `backend/app/influxdb.py:102-148`、`backend/app/routers/monitor.py:300-362`
- **问题:** Flux 无 `limit`/`aggregateWindow`;30 天趋势全量点返回,前端 `processTrendToSeries`(`Dashboard.vue:770-800`)再按时间戳去重取最大,内存与带宽浪费
- **建议:** 按 `time_range` 选聚合窗口(1h→1m,30d→1h),`aggregateWindow(every: v.windowPeriod, fn: mean)`,服务端限流 5000 点
### O3 — 多处 N+1 式逐条查库
- **文件:** `backend/app/services/alert_engine.py:126-132, 155-161`(SMTP 5 个 key 各 1 次查询,告警与恢复各一份)、`backend/app/routers/settings.py:104-108`(8 个 key 各 1 次)、`backend/app/routers/monitor.py:35-54`(Dashboard 4 次 `count`)
- **建议:** 一次 `select(...).where(key.in_(keys))` 批量取;Dashboard 用单条 `select(func.count(), ...)` 或一次聚合
### O4 — 采集无并发上限、无退避
- **文件:** `backend/app/services/scheduler.py:38-44`
- **问题:** `asyncio.as_completed([collect_host(h) for h in hosts])`,主机数无上限时同时起 N 个 `ipmitool` 子进程;失败主机每轮都重试 6 次
- **建议:** `asyncio.Semaphore(5~10)`;连续失败主机指数退避(如失败 3 次后跳过 5 轮)
### O5 — 总功耗 fallback 逻辑在三处各写一份,且 0 值语义不一致
- **文件:** `backend/app/services/component_parser.py:433-480`、`backend/app/routers/monitor.py:230-256`、`frontend/src/views/Sensors.vue:250-270, 380-400`
- **问题:** 后端两处把原生读数 `0` 当“缺失”进而用 PSU_PIN 求和,`Sensors.vue` 却把 `0` 当权威值(`priority` 已占位即不再累加),同一主机 Dashboard 与 Sensors 页总功耗不一致
- **建议:** 抽成后端唯一函数 `resolve_total_power(power_metrics)`,三处共用;明确 `0` 是否为有效读数(建议以“传感器存在即有效”为准)
### O6 — 重复代码遍布
- **文件:** `monitor.py:86-102` 与 `:383-396`(`parse_dt` 两份)、`ipmi_collector.py:149-175` 与 `:178-214`(`_run_ipmitool` 已标注 DEPRECATED 仍保留)、`hosts.py:146-190` 与 `collect_host` 的 6 元组 attempts(第三份)、`alert_engine.py` 的告警/恢复通知各两份
- **建议:** 抽公共函数/常量 `IPMI_AUTH_ATTEMPTS`、`parse_dt()`、`_notify()`
### O7 — 魔法数字散落各处
- **文件:** 前端 `Sensors.vue:234-240`(75/55℃、800/400W)、`Dashboard.vue:246`(轮询默认 10 秒)、`monitor.py:43`(在线阈值 2 倍间隔)、`alert_engine.py:39`(冷却 3 小时)、`ipmi_collector.py:172`(超时 30 秒)
- **建议:** 集中到 `config.py`/常量模块,温度阈值应与告警规则联动而非前端写死
### O8 — 前端轮询 10 秒 vs 后端采集 60 秒,且图表用轮询时间而非传感器时间
- **文件:** `frontend/src/views/Dashboard.vue:246, 714`、`frontend/src/views/Sensors.vue:427-430`、`frontend/src/views/Dashboard.vue:670-690`
- **问题:** 每 10 秒拉一次,但数据 60 秒才变一次,6 次里 5 次是重复查询;图表横轴用 `Date.now()` 拼点,数据陈旧时曲线仍“前进”,误导为实时
- **建议:** 默认轮询改为 30~60 秒(与 `COLLECT_INTERVAL` 对齐);图表用 Influx 返回的 `time` 而非 `Date.now()`;超 `2×COLLECT_INTERVAL` 未更新即断线显示
### O9 — SQLite 无 WAL、无外键、无关键索引
- **文件:** `backend/app/database.py:14-17`、`backend/app/models.py:77-127`
- **问题:** 未配 `journal_mode=WAL`/`busy_timeout`,采集(`last_seen` 更新)与告警写库并发时易 `database is locked`;SQLite 默认外键关闭,`ForeignKey` 不生效;`AlertHistory(host_id, rule_id, created_at)`、`Host(group_id)`、`AlertRule(host_id, group_id)` 均无索引,历史分页 `ORDER BY created_at DESC` 全表扫描
- **建议:** `create_async_engine(connect_args={"timeout": 30})` + 启动时 `PRAGMA journal_mode=WAL; PRAGMA foreign_keys=ON`;补索引;删除主机/分组时级联或先清关联规则/历史
### O10 — 认证:无登录限流、无改密/登出,JWT 有效期用 naive 时间
- **文件:** `backend/app/routers/auth.py:28-32, 62-66, 140-170`
- **问题:** 登录无速率限制/锁定,可暴力破解;无修改密码、登出(文档称有 logout,实际只有 login/verify/me)接口,默认管理员密码无法通过页面修改;`verify_password` 对非 bcrypt 格式哈希抛 `ValueError`(应为 401,却成 500);`create_access_token` 用 `datetime.now()`(本地 naive)算 `exp`,涉及时区歧义
- **建议:** 登录限流(如 5 次/分钟/IP+用户名);加改密接口与 token 吊销/黑名单;`verify_password` 捕获异常返回 `False`;`exp` 用 `datetime.now(timezone.utc)`
### O11 — Docker Compose 体积与一致性问题
- **文件:** `docker-compose.yml:1-70`、`docker-compose.dev.yml:1-66`、`backend/Dockerfile:31-38`
- **问题:** (1)`influxdb-data`、`backend-dev-data` 命名卷已声明但从未挂载,实际用 `./data/influxdb` 绑定目录,卷成摆设;(2)`dev` 用 `network_mode: host`,`ports` 声明失效,且 `PORT=8003` 与 Dockerfile 的 `HEALTHCHECK http://localhost:8000` 冲突;(3)`version: "3.8"` 已废弃;(4)后端无 `healthcheck`,只依赖 Influx 健康;(5)Influx 8086 直接映射到宿主机,无鉴权隔离
- **建议:** 删未用卷或改挂命名卷;`dev` 去掉 `host` 模式改端口映射;健康检查用 `${PORT:-8000}`;`version` 字段删除;生产不暴露 8086 或加防火墙
### O12 — Dockerfile:单 worker、无非 root 用户、健康检查写死端口
- **文件:** `backend/Dockerfile:1-38`
- **问题:** `uvicorn` 单进程(采集+API 同进程,一个慢请求即拖垮采集);以 root 运行;pip 镜像写死阿里云(海外/内网环境可能不可达);`HEALTHCHECK` 写死 8000,与 `${PORT}` 不一致
- **建议:** `uvicorn --workers 2`(注意 APScheduler 需单例,worker>1 时采集会重复——更稳妥是采集拆独立进程/只在 1 个 worker 跑调度);加 `USER app`;健康检查读 `$PORT`;镜像源改为构建参数
### O13 — `start.sh` 先 `down` 再 `up`,且只验 Influx 不验后端
- **文件:** `start.sh:76-77, 90-110`
- **问题:** 每次启动都 `docker compose down --remove-orphans`,会停掉正在跑的服务再重建(中断+重建镜像),且未跑 `scripts/preflight-data.sh` 的数据盘点;只等 Influx `/health`,后端是否起来、DB 是否为空无人验;`grep "^INFLUXDB_PORT="` 解析 `.env` 在值含空格/引号时出错
- **建议:** 默认用 `up -d --build`(不 `down`);启动后轮询 `/api/health`;接入 `preflight-data.sh`
### O14 — `alert-trigger.sh` 已完全失效
- **文件:** `alert-trigger.sh:3-10, 18-60`
- **问题:** 注释用法是 `--api-url ...`,代码却按位置参数 `$1/$2` 取,`--api-url` 会被当成 URL;调的 `/api/alerts` 端点不存在(实际是 `/api/alerts/history`,且需 Bearer 鉴权、返回分页对象而非数组);解析的 `.host/.severity/.sensor_type` 字段 API 从不返回;`ssh -o StrictHostKeyChecking=no root@...` 以 root 直连;无 `set -euo pipefail`、无 `jq` 存在性检查;`while read` 在管道子 shell 里,状态文件读写有竞态
- **建议:** 按 `--api-url/--ssh-key` 解析参数;改调 `/api/alerts/history?acknowledged=false` 并带 token;字段按实际返回改;SSH 改专用低权账号+密钥校验;或直接废弃该脚本
### O15 — Influx 客户端无超时、无重试、单例无锁
- **文件:** `backend/app/influxdb.py:19-44, 217-224`
- **问题:** `InfluxDBClient` 未配超时/重试;`get_client/get_write_api/get_query_api` 的懒创建无锁,并发首次调用可能建出两个客户端泄漏一个;`close()` 只在 lifespan 调,异常退出不保证执行
- **建议:** 建客户端时配 `timeout`;启动时(lifespan)预创建而非懒创建;写失败按指数退避重试 2~3 次
### O16 — 测试与文档缺口
- **文件:** `backend/tests/test_total_power_fallback.py`(唯一测试)、`README.md`
- **问题:** 仅 3 个用例覆盖功耗 fallback;`parse_sdr_output`、`evaluate_rule`(尤其 B1 的恢复逻辑)、`parse_dt`、时区换算、`component_parser` 的 legacy/去重分支全无测试;README 仍称前端为“纯静态 HTML/JS”,与 Vue 现状不符
- **建议:** 补单测:SDR 新/旧格式、负温度、超时、`eq` 浮点容差、恢复仅在已激活时、历史分页;README 更新技术栈与配置说明(尤其 B21 的无效变量)
## 最优先修的 5 项
B1(误发恢复)、B2(脏数据中断全部告警)、B14(Influx 宕机静默丢数据)、B20(密码明文返回)、B4+B3(采集超时拖垮调度)。其次是 B11(时区错 8 小时)、B10(组件接口读错文件)、B21(无效配置变量误导用户)。
+168
View File
@@ -0,0 +1,168 @@
# IPMI Monitor 功能审计报告
- 审计对象:`~/workspace/work/ipmi-monitor`,HEAD `a3ae072`,只读审计,未改任何文件
- 实际规模:后端约 3936 行(`backend/app/`:`main.py`、`config.py`、`database.py`、`influxdb.py`、`models.py`、`routers/` 6 个、`services/` 6 个);前端 Vue 3(`frontend/src/`,9 个视图 + 1 个组件);`frontend/dist/` 为构建产物,未审
- 总体结论:**核心链路(采集→存储→展示→按规则告警→邮件/飞书)是通的,但 README 明显滞后于代码**:README 的 API 表整表过期、技术栈表写错前端、通知渠道与环境变量多处宣称与实现不符;另有 8 个后端接口写好却无前端入口、1 个根脚本完全跑不通、若干“设置双轨制”和“临时文件依赖”等半成品
---
## 一、README 宣称功能逐项核对
### 1. 多服务器管理 — ✅ 已实现(但无批量操作)
- 依据:`backend/app/routers/hosts.py`(GET/POST/PUT/DELETE `/api/hosts`、启用开关、`POST /{id}/test` 连接测试,6 种 lanplus/lan 回退);`frontend/src/views/Hosts.vue`(增删改、按分组筛选、客户端搜索、连接测试);分组 CRUD 在 `backend/app/routers/groups.py` + `Groups.vue`
- 缺口:README“服务器管理”截图宣称的“批量操作”不存在——无批量启用/删除/测试/导入接口,前端也无多选操作
### 2. 实时监控(温度/风扇/电压/功耗)— ⚠️ 部分实现
- 已实现:`backend/app/services/scheduler.py`(APScheduler 定时 `collect_all_hosts`)+ `ipmi_collector.py`(`ipmitool sdr elist` 解析,`_classify_metric` 分 4 类)+ `influxdb.py`(`write_metrics_batch`);`Sensors.vue` 按 4 类指标展示最新值
- 部分原因:
- “实时”是 60 秒轮询,无 WebSocket/SSE;`Dashboard.vue`、`Sensors.vue` 的自动刷新默认都是关闭的,需手动打开
- 传感器“当前值”只读 InfluxDB 最近 1 小时(`influxdb.py: query_latest` 的 `range(start: -1h)`),超 1 小时无采集即显示空白而非“离线”
### 3. 历史趋势(InfluxDB)— ⚠️ 部分实现
- 已实现:`influxdb.py`(`query_metrics`/`query_trend`)、`monitor.py`(`GET /api/monitor/trend`,支持 1m–30d)、`Dashboard.vue`(温度/功耗两张 ECharts 趋势图,时间范围 5 分钟–30 天)
- 部分原因:
- `Sensors.vue`(697 行)只有最新值表格,**没有任何趋势图**;README 截图宣称的“电压稳定性图表”全站不存在,功耗趋势图只在 Dashboard 有,且只认 `Total_Power`/`Power`/`Pwr Consumption` 三个传感器名(`monitor.py` 的 `/trend`),无原生总功耗的机型(如 Gooxi12)在趋势图里是空的——而 Dashboard 汇总里刚修好的 PSU_PIN 求和 fallback(HEAD `a3ae072`)并没有同步到 `/trend`
- 无聚合/降采样、无 retention 策略配置,InfluxDB 会无限增长
### 4. 智能告警(可配阈值)— ⚠️ 部分实现
- 已实现:`routers/alerts.py`(规则 CRUD、按主机/分组/全局、条件 gt/lt/eq/gte/lte、`duration` 持续时长、3 小时通知冷却、恢复通知)、`services/alert_engine.py`(`check_alerts` 在每次采集后运行)、`AlertRules.vue`、`AlertHistory.vue`(分页、按主机/确认状态筛选、确认、批量确认)
- 部分原因:
- README 环境变量表里的 `ALERT_TEMP_THRESHOLD` / `ALERT_FAN_THRESHOLD` / `ALERT_POWER_THRESHOLD` 是**死变量**:`config.py` 里根本没有这三个字段,`docker-compose.yml` 传入后无人读取,全局默认阈值不存在,只有逐条建规则才生效;README“告警触发条件”一节描述的其实是已不存在的旧逻辑
- 无严重级别(severity);README 截图宣称的“告警静默操作”不存在,只有“确认”(acknowledge),且确认后规则仍会继续发恢复/重复通知
- 告警状态(`_first_breach`/`_alert_active`/`_last_notify`)全是 `alert_engine.py` 里的进程内 dict,重启即丢:重启后持续超标的告警会重新触发,持续时长计时归零
### 5. 多渠道通知(邮件、Webhook:Slack/钉钉/企业微信)— ⚠️ 部分实现
- 已实现(邮件 + 飞书):`services/email_notify.py`(HTML 邮件)、`services/feishu_notify.py`(飞书机器人)、`routers/settings.py`(DB 存储的 SMTP/飞书配置 + `POST /api/settings/test-email`、`/test-feishu`)、`Settings.vue`(飞书/邮件两个 Tab)
- 未实现(README 宣称的三个渠道一个都没有):
- Slack / 钉钉 / 企业微信:代码里无任何相关发送逻辑;README 让配的 `WEBHOOK_URL` 环境变量在 `config.py` 中不存在、**全代码库无人读取**,属于纯文档功能
- 反而是已实现的**飞书**在 README 功能列表里只字未提(只在任务描述里出现),README 与实现互相错位
- 另有设置双轨制:告警实际用的是 DB 里 `SystemSetting` 的 SMTP 配置(`alert_engine.py` 每次现查),`config.py` 的 `SMTP_HOST`/`SMTP_USER` 等环境变量只在 `email_notify.py` 无参回退时用,实际告警路径走不到;`test-email` 端点还绕开 `email_notify.py` 自己内联实现了一套 SMTP 发送
### 6. 一键部署(Docker Compose)— ✅ 已实现(带 2 个失效配置)
- 依据:`docker-compose.yml`(influxdb 2.7 + backend,healthcheck、命名卷 `backend-data`)、`start.sh`(6 步检查)、`scripts/preflight-data.sh`、README 的生产数据保护章节(`REQUIRE_EXISTING_DATABASE` / `MIN_EXPECTED_HOSTS`,`database.py: init_db` 真实校验)均与实现一致
- 失效配置:compose 传入的 `POLL_INTERVAL` 无效——`config.py` 的字段名是 `COLLECT_INTERVAL`,pydantic-settings 不会把 `POLL_INTERVAL` 映射过去,采集间隔恒为 60 秒,改 `.env` 也改不动;`ALERT_*`、`WEBHOOK_URL`、`ALERT_EMAIL_*` 同理是死变量(见第 4、5 条)
- 另:`INFLUXDB_BUCKET` 默认值两处不一致——`config.py` 默认 `ipmi_metrics`,`.env.example` / compose 默认 `metrics`,本地直接跑后端与 Docker 跑会写进不同 bucket
### 7. 响应式界面(纯静态前端,适配桌面和移动)— ⚠️ 部分实现(且技术描述错误)
- 部分实现:Element Plus 栅格在 Dashboard/Sensors 用了 `xs/sm/md/lg` 断点,`index.html` 有 viewport;但 `Components.vue` 统计行写死 `:span="6"`、各表格多列 `min-width` 固定,小屏需横向滚动,无移动端专项适配证据
- 描述错误:README 技术栈表写“前端 HTML / CSS / JavaScript(纯静态)”,实际是 **Vue 3.4 + Vue Router + Element Plus + ECharts + Vite**(`frontend/package.json`),且由 FastAPI 以 SPA 方式托管(`main.py` 的 catch-all 路由)
### 8. API 驱动(完整 RESTful API,支持二次开发)— ⚠️ 部分实现
- 已实现:FastAPI + Swagger(`/docs`)、JWT 鉴权(`routers/auth.py`:login/verify/me)、6 组路由共 33 个端点
- 部分原因:
- **README 的 API 表 14 个端点无一路径正确**:真实前缀是 `/api/hosts`、`/api/monitor/*`、`/api/alerts/*`、`/api/groups`、`/api/settings`、`/api/auth`,而 README 写的是 `/api/servers`、`/api/metrics/*`、`/api/dashboard`、`PUT /api/alerts/config`——按 README 调用的二次开发会全部 404
- 无 API Key / 服务账号机制,只有用户登录 JWT;无速率限制、无分页(除告警历史外所有列表接口都不分页)
### 9. README 截图章节宣称的页面 — ❌ 多处不存在
| 截图宣称 | 实际情况 |
|---|---|
| 服务器详情页(单台传感器列表、1h/6h/24h/7d 温度趋势、风扇/电压/功耗历史图) | **无此页面**:`frontend/src/router/index.js` 只有 9 条路由,无 `/hosts/:id` 详情页;单主机历史接口 `GET /api/monitor/history/{id}` 已写好但前端从未调用 |
| 仪表盘“最近告警摘要” | 只有未确认告警**数量**卡片(`monitor.py: /dashboard` 返回 `active_alerts` 计数),无摘要列表;Dashboard 不调告警历史接口 |
| 告警管理“通知渠道设置、告警确认/静默” | 通知渠道设置只有飞书+邮件(Settings.vue);静默不存在,只有确认 |
| 服务器管理“批量操作” | 不存在(见第 1 条) |
### 10. 其他 README 内容
- 快速开始/手动部署/systemd/FAQ:基本属实,但 `systemd` 示例未提需先建 DB 与默认账号;FAQ 没提本项目最常见的真实故障(`/tmp` 原始数据文件丢失导致组件页报错,见第二节)
- 技术栈表遗漏:数据库实际是 **SQLite(SQLAlchemy 存用户/主机/分组/规则/历史)+ InfluxDB** 两套,表里只写了 InfluxDB
---
## 二、缺失 / 半成品 / 断链功能清单
### A. 后端写了、前端没入口(共 8 个接口,白写)
| 接口 | 位置 | 说明 |
|---|---|---|
| `GET /api/monitor/realtime/{host_id}` | `routers/monitor.py` | 返回分组最新指标;无任何视图调用 |
| `GET /api/monitor/history/{host_id}` | 同上 | 单主机历史查询;无调用(详情页缺失的直接原因) |
| `GET /api/monitor/history/multi` | 同上 | 多主机历史对比;无调用 |
| `GET /api/monitor/sensors/{host_id}` | 同上 | 单主机全量传感器;无调用(Sensors 页走的是 `/summary` + `/dashboard/totals`) |
| `GET /api/hosts/{host_id}/status` | `routers/hosts.py` | 与 realtime 基本重复;无调用 |
| `GET /api/monitor/components/{host_id}`(非 grouped 版) | `routers/monitor.py` | 无调用,只有 `/grouped` 版被 Components.vue 用 |
| `POST /api/auth/verify`、`GET /api/auth/me` | `routers/auth.py` | 前端只用 login,登录态只靠 localStorage token + 路由守卫,未调 verify/me 校验 |
反向检查:前端所有 `api.*` 调用均有真实后端路由,**前端调不存在 API 的情况为零**;断链方向是反的(后端多余、README 过期)。
### B. README 宣称但代码里没有
1. `POST /api/servers/{id}/scan`(立即扫描):不存在;最接近的是 `POST /api/hosts/{id}/test`,但它只做 SDR 预览(返回前 20 条),**不写 InfluxDB、不更新 `last_seen`**,不能替代手动采集
2. Slack / 钉钉 / 企业微信通知(见一、5)
3. 全局阈值 `ALERT_*`(见一、4)、批量操作、告警静默、服务器详情页(见一、9)
### C. 半成品 / 脆弱实现(能跑,但随时会断)
1. **`alert-trigger.sh` 完全跑不通**:调 `GET /api/alerts`(无此路由,真实为 `/api/alerts/history` 且需 JWT 鉴权),解析 `.severity` / `.host` / `.sensor_type` 字段(真实字段是 `message` / `host_name` / `metric_name`,无 severity),默认端口 8003 与实际 8000 不符;且含 SSH 直连告警主机的逻辑,未被任何文档/部署引用
2. **组件在位监控依赖 `/tmp` 临时文件**:`ipmi_collector.py` 把每次采集的原始 SDR 写进 `/tmp/ipmi_raw_{host_id}.txt`,`monitor.py` 的组件接口直接读该文件;容器重启、`/tmp` 清理、或主机尚未采集过,都会返回 `error: Raw IPMI data file not found`,Components.vue 只弹错、页面留空。更甚者,非 grouped 版读的是**不分主机的** `/tmp/ipmi_raw.txt`(`monitor.py:430`),存在串主机数据的隐患。采集时写入 InfluxDB 的 `component` measurement(`ipmi_collector.py`)则**没有任何接口读回**,属于只写不读
3. **删主机留孤儿数据**:`models.py` 的外键无 `cascade`,`routers/hosts.py: delete_host` 只删 Host 行,不删其 AlertRule / AlertHistory,InfluxDB 历史也不清;删分组时有保护(组内有主机则 400),删主机没有等价保护
4. **认证与账号管理缺口**:全站只有一个 `admin` 账号(`database.py: init_db` 启动时自动创建,使用公开写死的默认口令——值略去,见代码);**无改密、无新增用户、无角色/只读账号**;`backend/app/init_admin.py` 是死代码(从未被 import,建账号逻辑与 `init_db` 重复);JWT 密钥、`CORS allow_origins=["*"] + allow_credentials=True`(`main.py`)均为宽松默认值
5. **采集间隔改不动**:`POLL_INTERVAL` 死变量(见一、6);设置页(Settings.vue)只有飞书+邮件两个 Tab,**采集间隔、全局阈值、Webhook、数据保留期均无界面可改**
6. **离线主机无告警**:在线状态只按 `last_seen` 在 Dashboard 算个数(`monitor.py: /dashboard`),主机掉线(采集失败)时没有任何告警规则能触发——`alert_engine.py` 只对有最新指标值的传感器做判断,`query_latest` 查不到就直接跳过
7. **测试覆盖极薄**:`backend/tests/` 仅 1 个测试文件(总功耗 fallback 回归),routers / alert_engine / auth / settings 均无测试
---
## 三、值得加的新功能(按价值从高到低)
### 1. 主机详情页 + 单传感器历史趋势图(价值最高:补最大断链,成本最低)
- **痛点**:README 承诺的详情页不存在;用户现在只能看全站汇总,想定位“某台机某传感器何时开始升温”没有任何入口;后端 `history` / `realtime` / `sensors` 三个接口已写好闲置
- **实现点**:前端 `router/index.js` 加 `/hosts/:id` 路由,新建 `views/HostDetail.vue`(复用 Dashboard 的 ECharts 趋势组件 + `Sensors.vue` 的表格),调现有 `GET /api/monitor/sensors/{id}`、`GET /api/monitor/history/{id}`、`GET /api/monitor/realtime/{id}`;Hosts.vue 表格加“详情”入口;后端零改动
### 2. 告警静默 / 维护窗口 + 严重级别
- **痛点**:上架、换件、固件升级期间必然触发误报,现在只能关规则(事后还得记得开)或忍受 3 小时一次的重复通知;无 severity 导致所有告警看起来一样重
- **实现点**:`models.py` 给 AlertRule 加 `severity`(info/warning/critical),新增 `Silence` 表(host/group/全局、起止时间、原因);`alert_engine.py: check_alerts` 在触发前查静默表;`routers/alerts.py` 加静默 CRUD;`AlertHistory.vue` 加“静默”操作和级别筛选;把 `_first_breach` 等内存状态顺带持久化到 DB(解决重启丢状态)
### 3. 通用 Webhook 通知(兑现 README 承诺的 Slack / 钉钉 / 企业微信)
- **痛点**:README 宣称三个渠道实际为零;`WEBHOOK_URL` 是死变量;飞书是唯一 IM 渠道,用户群在钉钉/企微的接不进来
- **实现点**:新建 `services/webhook_notify.py`(通用 POST JSON + 按渠道模板:钉钉 markdown、企业微信 markdown、Slack Block Kit、飞书沿用现有);`settings.py` 的 `DEFAULT_SETTINGS` 加 `webhook_url` / `webhook_type`,`Settings.vue` 加“Webhook”Tab 与测试按钮;`alert_engine.py` 在飞书分支旁加 webhook 分支;`config.py` 删掉无人读的 `WEBHOOK_URL` 或与之打通,二选一
### 4. 主机批量导入 / 批量操作
- **痛点**:`preflight-data.sh` 示例输出显示生产规模已达百台级(107 台),现在只能一台台在 Web 表单里加;无导出则换环境/备份全靠拷 SQLite 文件
- **实现点**:`routers/hosts.py` 加 `POST /api/hosts/batch`(CSV/JSON 导入,复用 `HostCreate` 校验)、`POST /api/hosts/batch-test`、`DELETE /api/hosts/batch`;`Hosts.vue` 加多选列 + 导入导出按钮;`models.py` 主机名/ipmi_host 加唯一约束防重复导入
### 5. 主机离线告警(采集失败即告警)
- **痛点**:现在最严重的故障(整机 BMC 掉线、断电)反而最安静——Dashboard 只把在线数减 1,无任何通知;`collect_host` 的失败只写日志
- **实现点**:`scheduler.py: collect_all_hosts` 收集每台的 `success` 后,对连续 N 次失败的主机写一条 AlertHistory(可复用现有 `notify_type`/`acknowledged` 流程,或新增 `metric_type='connectivity'` 的系统规则);`alert_engine.py` 加一条基于 `last_seen` 超时的兜底检查;Dashboard 离线卡片可点击下钻到离线主机列表
### 6. 用户管理:改密、多用户、只读角色
- **痛点**:单 admin 账号 + 公开默认口令 + 无改密入口,是当前最大的安全缺口;多人运维只能共用一个账号,无法审计谁改了规则
- **实现点**:`models.py: User` 加 `role`(admin/operator/viewer)、`routers/auth.py` 加 `POST /api/auth/change-password` 与用户 CRUD(admin 专属)、`routers/*` 的依赖按角色鉴权(viewer 禁写);前端 App.vue 用户菜单加“修改密码”,新增用户管理入口;删除死代码 `init_admin.py`,启动建账号逻辑只留 `database.py: init_db` 一处,并强制首次登录改密
### 7. 历史数据导出与报表(CSV 导出 + 每日摘要邮件)
- **痛点**:数据只进不出;故障复盘、容量规划、给领导汇报都只能截图;InfluxDB 里的数据除本系统外无人能用
- **实现点**:`routers/monitor.py` 加 `GET /api/monitor/export`(复用 `influxdb.py: query_metrics`,流式返回 CSV);Dashboard/Sensors 加“导出”按钮;`scheduler.py` 加每日定时任务,汇总昨日最高温/最大功耗/告警数,复用 `email_notify.py` 发摘要(与告警邮件模板分离)
### 8. 统一配置中心(修掉“环境变量与数据库设置双轨制”)
- **痛点**:`POLL_INTERVAL` 改不动、`ALERT_*` 不生效、SMTP 有环境变量和 DB 两套来源——这是本次审计发现的最多的一类问题根源,继续加功能只会越叠越乱
- **实现点**:`config.py` 明确划分:环境变量只管启动期(InfluxDB、DB 路径、JWT 密钥、端口),运行时可调项(采集间隔、全局默认阈值、通知渠道)全部收进 `SystemSetting` 表;`scheduler.py` 启动时读 DB 的采集间隔并支持热更新(APScheduler `reschedule_job`);`Settings.vue` 增加“采集与告警”Tab(间隔、默认阈值、冷却时间);README 环境变量表与之一一对齐,删掉死变量
### 9. InfluxDB 保留期与降采样管理
- **痛点**:无 retention 配置,百台规模下时序数据无限增长,迟早拖垮磁盘与查询;`query_latest` 只看 1 小时也与长期存储目标脱节
- **实现点**:`docker-compose.yml` / InfluxDB 初始化加 retention(如原始数据 30 天);`influxdb.py` 加降采样查询(`aggregateWindow` 按小时/天聚合,供 7d/30d 趋势图用,替代现在全量拉点);Settings 或 Dashboard 加存储用量展示(bucket 统计接口)
### 10. 采集健康诊断页
- **痛点**:采集失败只写后端日志,用户在界面上看不到“哪台机采失败、为什么失败”;排障要进容器翻日志;组件页报错(`/tmp` 文件丢失)也无任何自检提示
- **实现点**:持久化每次采集结果(新建 `CollectionLog` 表或复用 `last_seen` + 失败原因字段,替代 `/tmp/ipmi_raw_*.txt` 做组件数据源——组件数据改走 InfluxDB 的 `component` measurement 或 SQLite);`routers/monitor.py` 加 `GET /api/monitor/collection-status`;前端新增诊断视图或在 Hosts 页加“最后采集状态/失败原因”列,复用现有 `POST /api/hosts/{id}/test` 做一键诊断
---
**审计方法说明**:逐文件对照——README 14 个 API 端点逐一在 `routers/` 中检索、前端 9 个视图的全部 `api.*` 调用逐一反查后端路由、`.env.example` / `docker-compose.yml` 的每个变量在 `config.py` 与全代码库中检索引用;未做运行验证,结论均以静态代码为准。
+214
View File
@@ -0,0 +1,214 @@
# IPMI Monitor 安全审计报告
- 审计对象:`~/workspace/work/ipmi-monitor`,HEAD `a3ae072`
- 审计范围:后端 `backend/app/`(约 3936 行)、`docker-compose.yml` / `docker-compose.dev.yml` / `.env.example` / `start.sh` / `alert-trigger.sh` / `backend/Dockerfile` / `backend/requirements.txt` / `frontend/src/`
- 审计方法:只读静态审计;关键发现已对照源码逐项核实
- 报告不含任何真实密钥、密码、token 值,只写存在性与位置
## 结论
**不要用当前默认配置直接部署到生产。** 最致命的是两条链已闭环:① JWT 密钥是代码里的已知默认值,且部署链路根本不注入该变量——任何人都能伪造管理员 token;② 每次全新启动自动创建已知弱口令的 `admin` 账号。两者任一成立即等于无认证。再加上所有 BMC 的 IPMI 口令以明文存库、又经 API 明文回显,攻破一点即拿到全网服务器带外管理口令。
**已核查为阴性:** SQL 注入未发现——全部查询走 SQLAlchemy ORM 参数化,无原生 SQL / `text()` / 字符串拼接 SQL;命令注入未发现 `shell=True` / `os.system` / `eval`——`ipmitool` 走 `asyncio.create_subprocess_exec` 列表参数(但存在参数值经 argv 与日志泄露的问题,见 H-03);`.env` 未进 git 历史(`.gitignore` 已排除)。
## 分级统计
| 级别 | 数量 |
|---|---:|
| Critical(严重) | 4 |
| High(高危) | 10 |
| Medium(中危) | 10 |
| Low(低危) | 8 |
| 合计 | 32 |
## Critical(严重)
### C-01 JWT 密钥为代码内已知默认值,且部署链路从不设置它
- **位置:** `backend/app/config.py:43`(`JWT_SECRET_KEY` 默认值);`docker-compose.yml` 全文、`docker-compose.dev.yml` 全文、`.env.example` 全文——三处均无 `JWT_SECRET_KEY` 条目;`backend/app/routers/auth.py:62-72,75-86` 用该密钥签发/验签 HS256
- **原理:** `pydantic-settings` 未取到环境变量即回落到源码默认值;Compose 未传该变量、`.env.example` 也未提示设置。默认部署下密钥是公开可知的常量
- **影响:** 攻击者无需任何口令即可本地伪造 `{"sub":"admin"}` 的 JWT,通过全部受保护路由,等同管理员
- **修复:** 启动时强制校验:密钥缺失或等于默认值/长度 <32 字节则拒绝启动;用 `secrets.token_urlsafe(32)` 生成并经 `.env`(不进 git)注入;在 `docker-compose.yml` 与 `.env.example` 中加入 `JWT_SECRET_KEY=${JWT_SECRET_KEY:?must be set}`
### C-02 全新启动自动创建已知弱口令的默认管理员,且明文打印/记日志
- **位置:** `backend/app/database.py:95-112`(`init_db()` 内无 `admin` 则创建并 `logger.info` 打印账号口令);`backend/app/init_admin.py:42-55`(同样默认值,`print` 输出口令)
- **原理:** 默认口令为短弱口令、无强制改密、无首次登录改密流程、无过期;每次换新卷/新库都会重建该账号
- **影响:** 结合 C-01 或直接爆破,内网任何人登录即为唯一管理员;口令还落在容器日志里,可被日志采集方读取
- **修复:** 删除自动建号逻辑,改为首次部署由环境变量传入一次性强口令(`ADMIN_INITIAL_PASSWORD`,创建后立即失效)或交互式创建;强制首次登录改密;删除所有打印/记录口令的代码;已有部署立即改密并轮换 JWT 密钥
### C-03 全部 BMC 的 IPMI 口令明文存库,且经 API 明文回显给任何已认证用户
- **位置:** `backend/app/models.py:76`(`Host.ipmi_password` 为 `String(256)` 明文列);`backend/app/routers/hosts.py:46-52`(`HostOut` 含 `ipmi_password`)、`:62-86` 列表、`:88-103` 创建、`:106-126` 更新均原样返回;`backend/app/database.py` SQLite 文件本身亦无加密
- **原理:** 口令只做长度校验,无任何应用层加密;输出模型未做字段脱敏。前端 `Hosts.vue:66` 虽然只显示掩码,但 API 响应体里仍有明文
- **影响:** 任一 token 泄露(见 C-01/H-05)即可 `GET /api/hosts` 一次性拖走全网 BMC 口令,进而带外控制所有服务器(开关机、挂载镜像、读控制台);SQLite 文件一旦经 H-02 或卷泄露同样全量泄露
- **修复:** 立即从 `HostOut` 删除 `ipmi_password`(新建/更新接口只回 `has_password: bool`);库内用 Fernet/AES-GCM(密钥放环境变量,与 JWT 密钥分离)加密存储,内存中用后清零;轮换全部 BMC 口令;对 SQLite 文件设 `0600` 并考虑 SQLCipher
### C-04 InfluxDB 管理员口令与 Admin Token 在 Compose / 示例文件里有已知默认值,`start.sh` 会静默沿用
- **位置:** `docker-compose.yml:16`(`DOCKER_INFLUXDB_INIT_PASSWORD` 默认值)、`:19`(`DOCKER_INFLUXDB_INIT_ADMIN_TOKEN` 默认值)、`:43`(后端复用同一 token);`.env.example` 中 `INFLUXDB_PASSWORD` / `INFLUXDB_TOKEN` 同样为示例默认值;`start.sh:约 70-80 行`(无 `.env` 时 `cp .env.example .env`,只提示“请检查”而不强制改)
- **原理:** 用户按 README / `start.sh` 一路默认部署,即以公开可知的口令与 token 初始化 InfluxDB,且 `docker-compose.yml:9` 将 8086 映射到宿主机
- **影响:** 能访问 8086 者可用已知口令/token 登录 InfluxDB UI/API,读写/删除全部监控时序数据(含主机、传感器信息),并可作为横向立足点
- **修复:** 删除 Compose 中所有密钥类默认值,改为 `${VAR:?must be set}` 强制必填;`.env.example` 只留占位符;`start.sh` 在使用默认值时直接退出;已有部署轮换口令与 token,8086 不要对公网/办公网暴露
## High(高危)
### H-01 CORS `allow_origins=["*"]` 与 `allow_credentials=True` 并存
- **位置:** `backend/app/main.py:64-72`
- **原理:** 通配源 + 允许凭证是矛盾配置,浏览器对通配源本应忽略凭证,但一旦改为具体源或经代理放宽,即允许任意站点携带用户凭证跨域调用 API
- **影响:** 结合 H-05 的 localStorage token,恶意页面可借用户浏览器调用管理接口(删主机、改告警、改 SMTP/飞书配置)
- **修复:** 只允许明确的白名单源(如部署域名),`allow_methods`/`allow_headers` 收敛到必需项;生产用同源部署时可直接去掉该中间件
### H-02 SPA 静态文件服务存在路径遍历,可读任意文件(含 SQLite 库)
- **位置:** `backend/app/main.py:116-122`(`serve_spa`:`file_path = _frontend_dist / full_path` 后仅 `is_file()` 即 `FileResponse`,无 `resolve()` + `is_relative_to(_frontend_dist)` 校验)
- **原理:** 代码层面无任何防遍历措施,`../` 可跳出 `frontend/dist`
- **影响:** 若可绕过框架层归一化(需实测 `..` / `%2e%2e` 编码形态),可直接下载 `/app/data/ipmi_monitor.db`(C-03 的全部明文 BMC 口令 + 用户 bcrypt 哈希)及源码配置
- **修复:** `file_path = (_frontend_dist / full_path).resolve()` 后断言 `is_relative_to(_frontend_dist.resolve())`,否则 404;或改用 `StaticFiles` 挂载;SQLite 目录不得位于可被静态服务触及的路径
### H-03 IPMI 口令经进程参数与 DEBUG 日志泄露
- **位置:** `backend/app/services/ipmi_collector.py:149-176`(`_run_ipmitool`)、`:179-214`(`_run_ipmitool_with_auth`)、`:166,203`(`logger.debug("Running: %s", " ".join(cmd[:-5]) + " *** *** " + " ".join(cmd[-4:]))`)
- **原理:** 口令以 `-P <口令>` 出现在 `ipmitool` 的 argv 中,任何能 `ps` / 读 `/proc/<pid>/cmdline` 者可见;所谓脱敏是错的——已验证 `cmd[:-5]` 仍包含 `-P` 与口令(17 元命令的前 12 元即含口令位),`***` 只遮住了后半段无关参数。两函数均如此
- **影响:** 同机其他用户/容器逃逸后可直接捡到全部 BMC 口令;`LOG_LEVEL=DEBUG` 时口令进日志文件/日志平台
- **修复:** 改用 `IPMI_PASSWORD` 环境变量或 `-f` 口令文件(`0600`,用后删除)传口令,不再用 `-P`;删除该 debug 日志或只记录 `host` + `interface`,永不拼接完整命令
### H-04 全站明文 HTTP,无 TLS、无安全响应头,API 文档默认公开
- **位置:** `backend/Dockerfile:38`(`uvicorn … --host 0.0.0.0`,无 TLS);`backend/app/main.py:52-60`(`FastAPI()` 未关 `docs_url`/`redoc_url`/`openapi_url`,亦无 `HTTPSRedirect` / `TrustedHost` / HSTS / CSP / `X-Frame-Options` 等中间件);`start.sh` 末尾打印 `http://` 访问地址与 `/docs` 地址
- **原理:** 登录口令、JWT、IPMI 口令、SMTP 口令全部走明文 HTTP;`/docs`、`/redoc`、`/openapi.json` 未鉴权即可查看全部接口与模型
- **影响:** 局域网嗅探/中间人可直接窃取管理员口令与 token;文档泄露为攻击者提供完整接口清单
- **修复:** 前置反向代理做 TLS(或 uvicorn 配证书),`HSTS` + `Secure`;生产 `docs_url=None, redoc_url=None, openapi_url=None` 或仅内网开启;加 `TrustedHostMiddleware` 与安全头中间件
### H-05 JWT 有效期 24 小时,无 `jti`/`iat`、无吊销、无登出,token 存 localStorage
- **位置:** `backend/app/config.py:45`(`JWT_EXPIRATION_HOURS=24`);`backend/app/routers/auth.py:62-72`(仅 `sub` + `exp`,`datetime.now()` naive 时间)、`:75-86`(验签只认 `exp`)、`:178-190`(`/verify` 不查用户是否存在);模块文档称有 logout,实际无登出/吊销接口;`frontend/src/views/Login.vue:99-103` 与 `frontend/src/utils/api.js:15-18` 把 token 存 `localStorage` 并每次作 `Authorization: Bearer`
- **原理:** 被盗 token 在 24 小时内无法作废;改密、删用户均不影响已签发 token;localStorage 可被任意 XSS/第三方脚本读取;前端路由守卫 `frontend/src/router/index.js:69-72` 只查 token 存在与否,不验有效性
- **修复:** 访问 token 缩至 15 分钟 + refresh token(轮换、服务端可吊销);`jti` 入库做黑名单/白名单;提供登出与“全部会话失效”接口;改密后吊销全部 token;token 改存 `HttpOnly; Secure; SameSite=Strict` Cookie;前端守卫改为调 `/api/auth/me` 验真
### H-06 系统设置接口明文回显并明文存储 SMTP 口令与飞书 Webhook
- **位置:** `backend/app/routers/settings.py:104-122`(`GET /api/settings` 返回完整 `smtp_password` 与 `feishu_webhook_url`)、`:32-54`(响应/更新模型均含明文);`backend/app/models.py:36-42`(`SystemSetting.value` 为 `Text` 明文);`frontend/src/views/Settings.vue:199` 回填口令到表单
- **原理:** 密钥类配置与普通配置同表同字段存放、无加密、无掩码返回
- **影响:** 任一已认证调用即拖走邮箱口令与飞书机器人地址,可伪造告警、发钓鱼邮件、向外发通知
- **修复:** 密钥类设置单独加密列(AES-GCM);GET 只返回 `smtp_password_set: bool` / webhook 掩码(如末 4 位);更新时空值表示不改;轮换已泄露的 SMTP 口令与飞书 webhook
### H-07 `python-jose` 依赖存在已知 JWT 伪造类漏洞且已无人维护
- **位置:** `backend/requirements.txt`(`python-jose[cryptography]>=0.3.3`);`backend/app/routers/auth.py:8,69-81`(`from jose import jwt`)
- **原理:** `python-jose` 自 2021 年后基本停更;`>=0.3.3` 允许装到受影响版本:CVE-2024-33663(≤3.3.0,ECDSA/OpenSSH 密钥格式混淆)与 CVE-2026-120459(至 3.5.0 仍受影响、尚无修复版,空 HMAC 密钥可伪造 HS256 token,CVSS 9.1)。本项目 `INFLUXDB_TOKEN` 默认空(`config.py:22`)等空密钥配置习惯与后者风险同源,须确保 JWT 密钥永不为空
- **影响:** 密钥为空或特定格式时可绕过签名校验伪造任意用户 token
- **修复:** 迁移到积极维护的 `PyJWT>=2.10`(并显式 `algorithms=["HS256"]`、拒绝 `alg=none`);启动时校验密钥非空且长度足够;`pip-audit` 入 CI
### H-08 已认证 SSRF:IPMI 目标、飞书 Webhook、SMTP 主机均无任何校验
- **位置:** `backend/app/routers/hosts.py:28-31`(`HostCreate.ipmi_host` 只限长度,可填任意主机/IP 与 `ipmi_port` 1-65535);`backend/app/services/ipmi_collector.py:245-268` 与 `hosts.py:146-190`(`test` 端点按该目标发起 6 次 `ipmitool` 连接);`backend/app/routers/settings.py:200-242`(`test-feishu` 对任意 `webhook_url` 做 `httpx.post`,无域名/IP 白名单);`settings.py:154-192`(`test-email` 对任意 `smtp_host:smtp_port` 建连);`backend/app/services/feishu_notify.py:70,139`(告警时同样 POST 任意 URL)
- **原理:** 三处出站请求的目标完全由已认证用户控制,无内网 IP/元数据地址拦截、无域名白名单、无重定向限制
- **影响:** 可探测内网存活主机与端口、向内网服务 POST 告警内容、尝试访问 `169.254.169.254` 等元数据端点(结合 C-01/C-02,未授权者亦可利用);SMTP 主机还可用于内网端口扫描与凭证试探
- **修复:** `ipmi_host` 校验为 IP/主机名并拒绝回环/内网段(除非管理员显式放行网段);飞书 URL 只允许 `open.feishu.cn` / `open.larksuite.com` 等域名;SMTP 主机拒绝私网/元数据地址;httpx 禁用重定向到私网、设超时;出站统一走 SSRF 校验层
- **注(协调员核查):** IPMI 目标本身就是内网 BMC,禁止私网段会破坏产品核心用途。落地时对 `ipmi_host` 只拦截回环/链路本地/组播/元数据地址,允许 RFC1918 私网;飞书 URL 限制域名、SMTP 主机拦截回环/链路本地/元数据地址
### H-09 登录无任何限流、锁定与强度校验,可无限爆破
- **位置:** `backend/app/routers/auth.py:37-40`(`LoginRequest` 无长度/复杂度限制)、`:139-176`(登录逻辑无计数、无锁定、无延时);全仓无 `slowapi` / 限流中间件;`verify_password` 仅在用户存在时才执行 bcrypt(`auth.py:155`),不存在时直接 401,存在时多一次约数百毫秒 bcrypt,构成用户名枚举的时间侧信道
- **原理:** 结合 C-02 的已知弱口令,爆破成本极低
- **影响:** 字典爆破管理员口令;枚举有效用户名
- **修复:** 登录限流(如每 IP 每分钟 5 次、失败 5 次锁定 15 分钟)+ 全局速率限制;不存在用户时也执行一次虚拟 bcrypt 使耗时恒定;登录失败记审计日志;口令设最小长度 12 位
### H-10 容器以 root 运行,生产 Compose 还把源码目录挂进容器
- **位置:** `backend/Dockerfile` 全文无 `USER` 指令(默认 root);`docker-compose.yml:37-39`(`./backend/app:/app/app`、`./frontend:/app/frontend` 挂载);`backend/Dockerfile` 用 `pip config set global.trusted-host mirrors.aliyun.com`(HTTP 镜像源信任)
- **原理:** 容器逃逸即宿主机 root;源码挂载使容器内改代码可直接污染宿主机源码,宿主机改代码也可不经构建生效,生产与开发边界消失
- **影响:** 任一 RCE(如 H-02/H-08 链)放大为宿主机接管与持久化
- **修复:** `Dockerfile` 加专用非 root 用户(`USER appuser`)并 `chmod 0600` 敏感文件;生产 Compose 删除源码挂载,改用构建镜像;加 `read_only: true`、`cap_drop: [ALL]`、`no-new-privileges:true`;镜像源改用 HTTPS 并做哈希锁定
## Medium(中危)
### M-01 Flux 查询注入:`metric_type` 未校验即拼进 Flux 字符串
- **位置:** `backend/app/influxdb.py:138-146`(`query_metrics`)、`:171-180`(`query_latest`):`r["_measurement"] == "{metric_type}"` 直接插值;调用方 `backend/app/routers/monitor.py:76`(`/history/multi`)、`:122-124`(`/summary/{metric_type}`)、`:307`(`/trend`)、`:376`(`/history/{host_id}`)均未对 `metric_type` 做白名单校验。`host_id` 为 `int` 故安全,`bucket`/`org` 来自配置故安全
- **原理:** 传入含 `"`、`}` 的 `metric_type` 可闭合字符串、改写 Flux 过滤逻辑
- **影响:** 已认证用户可读取非预期 measurement 的数据、使查询报错/绕过过滤(数据越权与信息泄露)
- **修复:** 路由层用 `Literal["temperature","fan","voltage","power","component"]` 枚举校验;查询层对插值做转义或改用参数化 Flux;无效值直接 400
### M-02 IPMI 认证自动降级到最弱方式
- **位置:** `backend/app/services/ipmi_collector.py:245-268`(`attempts` 依次含 `["lanplus", ["-C", "3"]]`、`["lanplus", ["-C", "0"]]`(无认证)、`["lanplus", ["-A", "MD5"]]`、`["lan", ["-A", "MD5"]]`、`["lan", None]`(IPMI 1.5))
- **原理:** 强方式失败即自动降到无认证 / MD5 / IPMI 1.5,且每次采集、每次 `test` 都重复 6 次尝试
- **影响:** 凭据与会话降级为可破解/可窃听的弱协议;6 次尝试放大 BMC 侧爆破与账号锁定风险
- **修复:** 默认只用 `lanplus` + 强认证;降级需每主机显式勾选“允许弱协议”并在 UI 标红;失败即报错不再静默降级
### M-03 无 RBAC、无改密、无用户管理、无审计日志
- **位置:** `backend/app/models.py:20-31`(`User` 只有 `username`/`hashed_password`,无 `role`/`is_active`);全仓无改密/注册/用户列表/禁用接口;全仓无登录、配置变更、删主机等审计日志
- **原理:** 单一管理员模型,所有持 token 者权限等同;事后无法追溯谁改了什么
- **影响:** 无法做最小权限分离;口令泄露后无法只吊销一人;入侵难发现
- **修复:** 加 `role`(admin/operator/viewer)与只读角色;加改密与用户管理接口;关键操作写审计表(含操作者、IP、时间)
- **注:** 加 `role` 列需改数据库 schema,本次不做;改密接口与登录失败日志已落地,RBAC 与审计表待用户决策
### M-04 `/tmp` 原始 IPMI 数据文件权限过宽且跨主机串读
- **位置:** `backend/app/services/ipmi_collector.py:275-278`(`Path(f"/tmp/ipmi_raw_{host.id}.txt").write_text(raw)`,默认 `0644`,无清理);`backend/app/routers/monitor.py:424-430`(`/components/{host_id}` 读固定 `/tmp/ipmi_raw.txt`,与 `host_id` 无关——任意主机 ID 返回同一份数据);`:467-470`(`/grouped` 才按主机读)
- **原理:** 原始 SDR 含主机硬件细节,落盘权限过宽且无生命周期管理;一处接口取错文件
- **影响:** 同机其他用户可读硬件信息;调用方拿到别的主机组件数据(越权/误报)
- **修复:** 改写到应用数据目录并设 `0600`、按主机隔离、采集后定期清理;`/components/{host_id}` 改为按 `host_id` 取文件并鉴权归属;不要把敏感原始输出落盘,优先放内存/数据库
### M-05 报错与日志泄露内部细节
- **位置:** `backend/app/routers/hosts.py:186-190`(`test` 返回 `str(exc)` 含 `ipmitool` stderr);`backend/app/routers/monitor.py:449-455,530-536`(组件接口返回 `str(exc)`);`backend/app/routers/settings.py:192,237`(测试邮件/飞书失败返回 `str(e)`);`backend/app/influxdb.py` 多处 `logger.exception` 打完整堆栈
- **原理:** 异常直通前端,未做脱敏与分级
- **影响:** 泄露内部路径、主机名、服务版本与后端拓扑,辅助后续攻击
- **修复:** 对外只返回通用错误码与简短提示,详情只写服务端日志并脱敏;日志中过滤口令、token、内网地址
### M-06 SMTP 明文回退与证书校验不明确
- **位置:** `backend/app/services/email_notify.py:180-188,236-244` 与 `backend/app/routers/settings.py:180-188`:非 465 端口用 `smtplib.SMTP` + `starttls()`(未显式传 `SSLContext`),`smtp_use_tls=false` 时直接明文 `login`
- **原理:** 配置可关闭加密;STARTTLS 失败/被剥离时无强制校验与告警
- **影响:** 邮件口令与告警内容可被中间人窃听/篡改
- **修复:** 默认强制 TLS(465 用 `SMTP_SSL` 配默认校验上下文,587 用 `starttls(context=ssl.create_default_context())` 并校验失败即拒发);删除“关闭 TLS”选项或需管理员二次确认
### M-07 未校验输入与无上限查询可致 500 / DoS
- **位置:** `backend/app/routers/monitor.py:83,137`(`[int(x.strip()) for x in host_ids.split(',')]`,非法值抛 `ValueError` 未捕获 → 500);`:88-100,380-392`(`parse_dt` 回退分支同样可能抛错);`/history/multi` 对 `host_ids` 数量无上限;`list_hosts`(`hosts.py:62-86`)无分页;`dashboard_totals`/`trend`(`monitor.py:202-368`)每主机发 4 次以上 Influx 查询(N+1);`backend/app/services/scheduler.py:60-73` 每主机最多 6 次 ×30 秒超时、`max_instances=1` 易阻塞下一轮采集
- **原理:** 输入与扇出均无上限、无统一异常处理
- **影响:** 恶意/误操作请求可打满后端与 InfluxDB,拖垮采集调度
- **修复:** `host_ids` 限个数(如 ≤50)并校验为整数;统一异常处理返回 400;列表加分页;趋势/汇总改批量查询或缓存;采集超时与重试次数收敛
### M-08 依赖全部用 `>=` 下限、无锁定文件、无漏洞扫描
- **位置:** `backend/requirements.txt` 全文(14 个依赖全为 `>=`,无 `==`、无 `requirements.lock`、无哈希);`frontend/package.json` 依赖多为 `^` 范围;`jinja2>=3.1.2` 允许装到含 CVE-2024-22195 / CVE-2024-34064 等的 3.1.2(修复于 3.1.3+);`passlib[bcrypt]>=1.7.4` 已停更(与 bcrypt 4.x 存在已知兼容问题);`jinja2` 在代码中未见实际使用
- **原理:** 每次构建装到的版本不可复现,可能装到已知漏洞版;无扫描即无人发现
- **修复:** 全部依赖钉死 `==` 并生成 `pip freeze` / `uv.lock`,提交 lock 文件;升级 `jinja2>=3.1.6` 或删除未用依赖;`pip-audit` / `npm audit` 入 CI 并定期升级
### M-09 `alert-trigger.sh` 存在多个高危写法且调用了不存在的接口
- **位置:** `alert-trigger.sh:22`(无认证 `curl "$API_URL/api/alerts"`,但后端并无该 GET 路由,实际是 404/405);`:55-66`(`ssh -o StrictHostKeyChecking=no root@"$host"`);`:38,66`(主机名/消息直接拼进 `jq` 程序字符串 `".\"${host}_${sensor_type}\" // \"\""`);`:8`(状态文件用固定 `/tmp/ipmi-alert-state.json`)
- **原理:** 关闭主机密钥校验等于放弃中间人防护;以 root 直连 BMC/业务主机;jq 拼接存在注入;`/tmp` 固定文件可被抢占/篡改
- **影响:** 中间人可冒充目标主机;恶意告警内容可注入 jq/SSH 命令;脚本实际取不到告警却仍以 root 尝试连接
- **修复:** 改调真实存在的 `/api/alerts/history` 并带 JWT;`StrictHostKeyChecking` 改为 `accept-new` 并预置 known_hosts;用 `jq --arg` 传参;状态文件放用户家目录并设 `0600`;非必要不用 root,改用专用低权账号 + `sudo` 白名单
### M-10 InfluxDB Token 默认空、端口对外暴露
- **位置:** `backend/app/config.py:22`(`INFLUXDB_TOKEN: str = ""`);`docker-compose.yml:9`(`8086` 映射到宿主机);`backend/app/influxdb.py:20-30`(空 token 仍创建客户端,失败只在调用时暴露)
- **原理:** 配置缺省不报错,部署后才在写/读时失败;端口暴露扩大攻击面
- **影响:** 服务不可用时难定位;暴露的 8086 成为暴力破解与漏洞利用入口
- **修复:** 启动校验 token 非空;8086 只监听内网或不映射宿主机端口,管理走容器网络或反代鉴权
## Low(低危)
### L-01 三处口令哈希实现不统一、时间处理用 naive datetime
- **位置:** `backend/app/routers/auth.py:24-30`(直接 `bcrypt`)、`backend/app/database.py:90,100`(直接 `bcrypt`)、`backend/app/init_admin.py:17-20`(`passlib.CryptContext`);`auth.py:65`、`models.py`(`datetime.now()` 作 `created_at`/`last_login`/`exp`)
- **影响:** 成本参数与哈希格式可能漂移;naive 时间在夏令时/时区切换时导致 token 过期判断偏差
- **修复:** 统一用一处 `hash_password/verify_password`(`passlib` 或 `bcrypt` 二选一,成本 ≥12);全部时间改 `datetime.now(timezone.utc)`
### L-02 多处输入无格式校验
- **位置:** `backend/app/routers/auth.py:37-40`(登录字段无长度限制);`backend/app/routers/alerts.py:36`(`email_addresses` 为自由文本);`backend/app/routers/settings.py:60-63`(`TestEmailRequest.email: str`,未用 `EmailStr`)
- **影响:** 超长输入、非法邮箱可进库并在告警时触发异常/垃圾邮件
- **修复:** 登录字段加 `min_length/max_length`;邮箱用 `EmailStr` 并做去重与数量上限
### L-03 示例与文档固化默认口令,`start.sh` 不强制轮换
- **位置:** `.env.example`、`README.md:132-136`(配置表列出默认口令/token)、`start.sh`(复制示例后只提示、不校验、不退出)
- **影响:** 用户极易原样上线默认配置(见 C-04)
- **修复:** 文档与示例只写占位符;`start.sh` 增加“密钥是否为默认值/是否为空”检查,不通过即退出
### L-04 前端 token 存 localStorage、路由守卫只验存在性
- **位置:** `frontend/src/views/Login.vue:99-103`、`frontend/src/utils/api.js:15-18`、`frontend/src/router/index.js:69-82`。前端未发现 `v-html`/`innerHTML`(XSS 面相对小),但一旦引入含 XSS 的依赖或组件,token 即被盗
- **修复:** 见 H-05,改 HttpOnly Cookie;守卫改为先调 `/api/auth/me` 验真
### L-05 缺少安全头与版本信息泄露
- **位置:** `backend/app/main.py:52-60`(`FastAPI(title="IPMI Monitor", version="0.1.0")`,无安全头中间件);`/api/health`(`main.py:89-91`)未鉴权(本身合理,但会暴露服务存活)
- **修复:** 加 `X-Content-Type-Options: nosniff`、`X-Frame-Options: DENY`、CSP、HSTS;对外隐藏精确版本号
### L-06 开发配置混入生产部署
- **位置:** `docker-compose.dev.yml`(`network_mode: "host"`、端口 8003、源码挂载);`docker-compose.yml:67-69` 定义了 `influxdb-data` 命名卷却未使用(实际用 `./data/influxdb` 绑定挂载)
- **修复:** 开发与生产 Compose 彻底分离;删除未用卷定义;生产不用 `host` 网络模式
### L-07 `get_db` 对只读请求也提交事务
- **位置:** `backend/app/database.py:55-70`(`yield` 后无条件 `commit()`,异常才回滚)
- **影响:** GET 请求也可能把脏数据写盘;删除接口依赖该隐式提交,语义不清晰
- **修复:** 只读依赖不提交;写操作显式 `commit()`
### L-08 未用依赖与重复代码增加攻击面
- **位置:** `backend/requirements.txt`(`jinja2`、`aiofiles` 在代码中未见使用);`backend/app/services/ipmi_collector.py:149-176`(标注 DEPRECATED 的 `_run_ipmitool` 仍保留,与 `_run_ipmitool_with_auth` 重复)
- **修复:** 删除未用依赖与废弃函数,减小供应链与维护面
## 修复优先级建议
先做 C-01 + C-02 + C-03 + C-04(轮换 JWT 密钥、管理员口令、InfluxDB 口令/token、全部 BMC 口令与 SMTP/飞书密钥),再做 H-01/H-04/H-05(CORS、TLS、token 吊销与 Cookie 化)与 H-02/H-03,最后按 M 级逐项收敛。
+160
View File
@@ -0,0 +1,160 @@
# IPMI Monitor 审计报告
- 审计对象:`~/workspace/work/ipmi-monitor`,HEAD `a3ae072`
- 审计范围:后端 Python 3.11 + FastAPI + SQLAlchemy 2.0 + InfluxDB 2.7 + APScheduler(`backend/app/` 约 3936 行),前端 Vue 3(`frontend/src/`),Docker Compose 部署
- 审计方式:三路并行(安全 / BUG 与优化 / 功能缺口),关键发现已对照源码逐项核实;随后三路并行落地修复
- 分报告:[AUDIT-security.md](AUDIT-security.md)(安全 32 项)|[AUDIT-bugs.md](AUDIT-bugs.md)(BUG 30 项 + 优化 16 项)|[AUDIT-features.md](AUDIT-features.md)(功能核对 + 10 个新功能建议)
- 状态:**审计与修复均已完成,未提交、未推送**,只改工作区文件
## 总体结论
**核心链路是通的(采集→存储→展示→按规则告警→邮件/飞书),但默认配置不能直接上生产,且 README 明显滞后于代码。**
- **安全**:最致命的是两条已闭环的链——JWT 密钥是代码里的已知默认值且部署链路从不注入(任何人可伪造管理员 token);全新启动自动创建已知弱口令的 `admin` 账号。再加上全部 BMC 口令明文存库又经 API 明文回显,攻破一点即拿到全网服务器带外管理口令。以上 4 项严重 + 10 项高危已全部修复。
- **BUG**:30 项已确认,全部修复,其中 5 项最优先:告警恢复误发(B1)、脏数据中断全部告警检查(B2)、InfluxDB 宕机静默丢数据(B14)、密码明文返回(B20)、采集超时拖垮调度(B3+B4)。另有历史接口时区错 8 小时(B11)、组件接口读错文件(B10)、配置变量错位(B21)。
- **功能**:README 的 API 表 14 个端点无一路径正确;Slack/钉钉/企业微信、全局阈值、批量操作、告警静默、服务器详情页均为宣称但未实现;8 个后端接口写好却无前端入口;`alert-trigger.sh` 完全跑不通。README 已按真实代码重写。
## 分级统计
| 类别 | 严重 | 高危 | 中危 | 低危 | 小计 |
|---|---:|---:|---:|---:|---:|
| 安全问题 | 4 | 10 | 10 | 8 | 32 |
| 已确认 BUG | — | — | — | — | 30 |
| 优化建议 | — | — | — | — | 16 |
| 功能缺口(README 宣称未实现 + 断链) | — | — | — | — | 见 AUDIT-features.md 第二节 |
## 已修复(本次落地)
> 修复原则:不改行为(除安全修复本身要求的行为收敛外)、不动数据库 schema、不改 API 契约(新增端点与删除明文密码字段除外——后者正是安全修复本身)。每改一处均通过 `py_compile`;测试 36/36 通过(见“验证”)。**未提交、未推送**,只改工作区文件。
### 安全(严重/高危,14/14)
| 编号 | 修复 | 文件 |
|---|---|---|
| C-01 | 启动强制校验 JWT 密钥:缺失、等于旧默认值(以 SHA-256 摘要比对,代码与报告中均不写明文)或 <32 字符则拒绝启动;校验只在 lifespan 启动路径,不在 import 时执行;Compose 与 `.env.example` 改为必填 `${JWT_SECRET_KEY:?must be set}` | `backend/app/config.py`、`backend/app/main.py`、`docker-compose.yml`、`.env.example` |
| C-02 | 删除自动创建已知弱口令 admin 的逻辑与口令日志;库里无用户时要求 `ADMIN_INITIAL_PASSWORD` 环境变量(≥12 位),否则拒绝启动;删除死代码 `init_admin.py` | `backend/app/database.py`、`backend/app/init_admin.py`(删除) |
| C-03 | `HostOut` 删除 `ipmi_password`,改为 `has_password: bool`;BMC 口令入库前 Fernet 加密(密钥来自 `CREDENTIAL_ENCRYPTION_KEY` 或首次启动生成的 `data/.credential_key` 0600 文件),读取时解密;启动时对存量明文口令做一次性重加密迁移;采集器与 test 接口解密后内部使用 | `backend/app/crypto.py`(新增)、`backend/app/models.py`(未改字段)、`backend/app/routers/hosts.py`、`backend/app/services/ipmi_collector.py`、`backend/app/database.py` |
| C-04 | 删除 Compose 与 `.env.example` 中 InfluxDB 口令/token 的默认值,改为 `${VAR:?must be set}`;`start.sh` 检测到默认值/空值/口令 <8 位直接退出(实测退出码 1);顺手加了 `JWT_SECRET_KEY`(<32 位即退出)的同类检查 | `docker-compose.yml`、`.env.example`、`start.sh` |
| H-01 | CORS 改为白名单:`CORS_ORIGINS` 环境变量(逗号分隔),未设置时不启用 CORS 中间件(同源部署) | `backend/app/config.py`、`backend/app/main.py` |
| H-02 | SPA 静态服务 `resolve()` + `is_relative_to()` 防遍历;兜底路由排除 `/api/` 前缀(返回 404 JSON);`%2e%2e` 编码探针验证均返回 404 | `backend/app/main.py` |
| H-03 | ipmitool 口令改走 `-f` 临时口令文件(0600,用后删除),不再用 `-P`;删除拼接完整命令的 debug 日志,只记 host/interface | `backend/app/services/ipmi_collector.py` |
| H-04 | 生产默认关闭 API 文档(`ENVIRONMENT=production` 时 `docs_url/redoc_url/openapi_url=None`);加安全响应头中间件(nosniff / DENY / CSP / Referrer-Policy)。注:仓库内有 `frontend/dist` 时,`/docs` 会落到 SPA catch-all 返回 index.html(非 Swagger 内容),容器内无 dist 时返回 404 | `backend/app/main.py`、`backend/app/config.py` |
| H-05 | JWT 改用 `datetime.now(timezone.utc)`;默认有效期 24h → 8h;token 增加 `jti`;新增登出(进程内吊销)与改密接口(改密后经“改密时间戳”吊销该用户全部已发 token,含当前 token) | `backend/app/routers/auth.py`、`backend/app/config.py` |
| H-06 | `GET /api/settings` 不再返回 `smtp_password` 明文与完整飞书 URL,改为 `smtp_password_set: bool` + webhook 掩码(只显后 4 位);更新时空值表示不改;SMTP 口令与飞书 URL 入库前同样 Fernet 加密 | `backend/app/routers/settings.py`、`backend/app/crypto.py` |
| H-07 | `python-jose` → `PyJWT==2.10.1`;显式 `algorithms=["HS256"]`;`requirements.txt` 全部钉死 `==` 版本(版本逐个在 PyPI 核实存在且兼容 Python 3.11;passlib 1.7.4 与 bcrypt ≥4.1 不兼容,故 bcrypt 钉 4.0.1) | `backend/app/routers/auth.py`、`backend/requirements.txt` |
| H-08 | SSRF 校验层:`ipmi_host` 拒绝回环/链路本地/组播/元数据地址(允许 RFC1918 私网,因 BMC 本就在内网);飞书 URL 只允许 `open.feishu.cn` / `open.larksuite.com`;SMTP 主机同样拦截回环/链路本地/元数据地址 | `backend/app/ssrf.py`(新增)、`backend/app/routers/hosts.py`、`backend/app/routers/settings.py`、`backend/app/services/feishu_notify.py` |
| H-09 | 登录限流(内存计数:同一 IP+用户名 5 次失败锁定 15 分钟,第 6 次起 429);不存在用户时也执行一次虚拟 bcrypt 使耗时恒定;登录失败记日志 | `backend/app/routers/auth.py` |
| H-10 | Dockerfile 加非 root 用户(`USER appuser`);生产 Compose 删除源码目录挂载(只剩 `backend-data` 数据卷);pip 镜像源改用 HTTPS(去掉 `trusted-host`) | `backend/Dockerfile`、`docker-compose.yml` |
中危/低危安全项(H-02 关联的 B28/B29、Flux 注入防护等)随 B12/B18 的白名单校验、O9/O10 一并修复,详见 AUDIT-security.md 与下表。
### BUG(已确认,30/30)
| 编号 | 修复 |
|---|---|
| B1 | 恢复通知仅当 `_alert_active` 为真时发送 |
| B2 | `check_alerts` 对 `None`/非数值做隔离,单条规则异常不中断整轮 |
| B3 | 采集超时捕获 `TimeoutError` 继续试下一种认证;超时后 `proc.kill()` + `await proc.wait()` |
| B4 | 单次 ipmitool 超时 30s → 10s;调度加 `coalesce=True, misfire_grace_time=30`;采集加 `Semaphore(8)` 并发上限;连续失败 3 次后跳过 5 轮再试 |
| B5 | 温度/风扇/电压/功耗正则支持负号与千分位逗号 |
| B6 | `_classify_metric` 接收 `status` 参数,`ok→1、其余→0`;离散电源传感器归为 `status` 而非 `power` |
| B7 | `parse_sdr_output` 按段数分流(5 段新格式 / 3 段 legacy),状态 token 扩充 `disabled/cr` 等 |
| B8 | `component_parser` 调用前在 collector 内把 3 段 legacy 行展开为 5 段(`value=parts[1]、status=parts[2]`);未改 `component_parser.py` |
| B9 | drive/fan/psu 解析加 `seen` 去重;PSU/硬盘在位改用 SDR `status` 段;`Drive 0` 用正则匹配 |
| B10 | `/api/monitor/components/{host_id}` 改读 `/tmp/ipmi_raw_{host_id}.txt`,与 `/grouped` 一致;文件不存在返回 404 |
| B11 | `parse_dt` 抽成公共函数(`utils.py`);默认值用 `datetime.now(timezone.utc)`;解析失败返回 400 而非 500 |
| B12 | `host_ids` 校验为整数且限 ≤50 个;`metric_type` 白名单校验;查询加 `limit(5000)` |
| B13 | `dashboard/totals` 返回 `group_totals`;`None` 值用 `or 0` 防护;已查前端不读 `avg_temperature`,故改名 `avg_max_temperature` |
| B14 | `write_metrics_batch` 返回 bool;写入失败时 `collect_host` 不更新 `last_seen`、置 `success=False` |
| B15 | InfluxDB 同步调用包进 `asyncio.to_thread()`;客户端配超时(10000ms) |
| B16 | `check_alerts` 开头清理已删除/禁用规则的内存状态 |
| B17 | 建/改规则校验 `host_id` 与 `group_id` 至多其一;`host_id` 分支同样只查 `enabled` 主机 |
| B18 | `RuleCreate/RuleUpdate` 加值域校验:`condition` Literal、`metric_type` 白名单、`duration >= 0`、拒绝 NaN/Inf 阈值(同时堵住 Flux 注入点) |
| B19 | 仅当历史创建成功(主机存在)才置 `_alert_active` |
| B20 | 同 C-03/H-06(密码不再明文返回;设置接口掩码;告警路径 4 处解密点补齐,见下) |
| B21 | `config.py` 增加 `POLL_INTERVAL` 别名(`AliasChoices`);删除 Compose/`.env.example` 里的死变量 `ALERT_*` / `WEBHOOK_URL` / `ALERT_EMAIL_*`,改为正确变量名;`INFLUXDB_BUCKET` 默认值统一为 `metrics` |
| B22 | 删除 `init_admin.py`(死代码且跑不起来),只保留 `database.py: init_db` 一处建号逻辑 |
| B23 | 写接口显式 `await db.commit()`(alerts/groups/hosts/auth/settings 全部写接口);`get_db` 去掉成功后自动提交(只 yield,异常回滚) |
| B24 | `send_alert_email/send_recovery_email` 接收并使用 `smtp_use_tls`;仅开关开且非 465 时才 `starttls()`,配 `SSLContext`;465 用 `SMTP_SSL` |
| B25 | 告警路径读取 `feishu_keyword` 并拼进飞书卡片文本 |
| B26 | 测试邮件端口解析移入 `try` 并校验范围(1~65535);SMTP 调用包进 `asyncio.to_thread()` 并配 15s 超时;连接用 `with` 保证释放;新增 `email_notify.send_test_email` 供 settings 测试端点使用 |
| B27 | 邮件模板改用缺失字段安全的渲染(缺 key 填空串,不抛 `KeyError`);HTML 残留 `{{ ... }}` 清理 |
| B28 | 同 H-02(SPA 兜底排除 `/api/`,防遍历) |
| B29 | 同 H-01(CORS 收敛) |
| B30 | 趋势接口功耗分支同样做“原生优先、否则 PSU_PIN 求和”聚合 |
另:认证降级已剔除——`ipmi_collector.py` 与 `hosts.py` 的 test 接口均删除无认证(`-C 0`)与 IPMI 1.5(`lan`)尝试,只保留 lanplus 默认、`lanplus -C 3`、`lanplus -A MD5` 三种(见“未修复”第 6 项)。
衔接修复(收尾一路):加密落地后,`alert_engine.py` 告警/恢复两处路径的 4 个解密点(SMTP 口令 ×2、飞书 URL ×2)、`email_notify._smtp_settings` 与 `feishu_notify.send_feishu_alert` 入口的防御性解密已全部补上,告警通知链路不断。
### 优化(已做 11 项,另 2 项部分)
- O1:`query_latest_grouped(host_id, only=...)` 按需查询,不再每次查 4 种类型
- O3:设置批量读取(`get_settings_batch`),告警路径 SMTP 5 个 key + TLS 开关、飞书 2 个 key 一次取回
- O4:采集 `Semaphore(8)` 并发上限;连续失败主机记入退避(失败 3 次后跳过 5 轮)
- O5:抽公共 `resolve_total_power(power_metrics)`,Dashboard 汇总与趋势共用;`0` 视为有效读数(传感器存在即有效)
- O6:抽公共常量 `IPMI_AUTH_ATTEMPTS`、公共 `parse_dt()`;删除已标注 DEPRECATED 的 `_run_ipmitool`
- O9:SQLite 加 `journal_mode=WAL` + `foreign_keys=ON` + `busy_timeout`;`init_db` 里 `CREATE INDEX IF NOT EXISTS` 补关键索引
- O10:登录限流 + 改密/登出 + JWT 用 UTC 时间(同 H-05/H-09)
- O11:删除 Compose 未用卷定义与废弃 `version` 字段;Dockerfile 健康检查改用 `${PORT:-8000}`
- O12(部分):非 root 用户已做(同 H-10);单 worker 未动(见“未修复”第 8 项)
- O13:`start.sh` 默认不再 `down`(`up -d --build`),启动后轮询后端 `/api/health`
- O14:`alert-trigger.sh` 重写(仓库根目录,`scripts/` 下并无此文件):按 `--api-url/--token/--ssh-key` 解析参数,调真实 `/api/alerts/history?acknowledged=false` 带 JWT,`jq --arg` 传参,`StrictHostKeyChecking=accept-new`,状态文件放 `$HOME/.ipmi-monitor/` 并设 0600
- O15:InfluxDB 客户端配超时;启动时预创建(lifespan),懒创建加锁
- O16(部分):补测试 33 个新用例 + README 重写;CI 未加
未做的优化:O2(降采样未做;查询上限 5000 已随 B12 加上)、O7(魔法数字散落各处,未集中整理)、O8(前端轮询 10 秒 vs 后端采集 60 秒、图表用轮询时间而非传感器时间,未改)。
### 新功能(已实现 2 个)
1. **改密接口**(安全修复附带,AUDIT-features.md 第 6 项的一部分):`POST /api/auth/change-password`(需登录,旧密码校验 + 新密码 ≥12 位),改密后吊销该用户全部已发 token;另有 `POST /api/auth/logout`
2. **历史数据 CSV 导出**(AUDIT-features.md 第 7 项):`GET /api/monitor/export?host_ids=…&metric_type=…&start=…&end=…`,复用 `query_metrics`,流式返回 CSV(列:time,host_id,metric_type,metric_name,value,unit);供故障复盘与报表用
其余新功能(主机详情页、告警静默、通用 Webhook、批量导入、离线告警、RBAC、配置中心、保留期管理、采集诊断页)未做,只写进 AUDIT-features.md 待用户定。
### 前端同步
- `Hosts.vue`:密码列改读 `has_password`(已配置显示掩码,未配置显示“未配置”);编辑回填本来就是 `ipmi_password: undefined` + 提交时删空口令,未改
- `Settings.vue`:加载时改读 `feishu_webhook_masked` / `smtp_password_set` 做掩码展示(留空则不修改);提交时秘密字段留空表示不改;测试飞书/测试邮件的必填校验放行“已配置但本次未重填”;顺手修了测试邮件按钮 `@click` 指向 ref 而非处理函数的既有缺陷(原按钮点按无效,现已接线到 `testEmailSend`)
- `frontend/dist/` 未动、未重新构建:前端 `src` 改动后需用户自行 `npm run build` 刷新 dist
### README 更新
- API 表按真实路由重写(37 个端点,`/api/hosts`、`/api/monitor/*`、`/api/alerts/*`、`/api/groups`、`/api/settings`、`/api/auth`,含 logout、change-password、export)
- 技术栈表修正为 Vue 3.4 + Vue Router + Element Plus + ECharts + Vite,数据库补上 SQLite + InfluxDB 两套
- 环境变量表与 `config.py` 对齐,删除死变量说明,注明阈值在页面规则里配、采集间隔用 `COLLECT_INTERVAL`(兼容 `POLL_INTERVAL` 别名)、`JWT_SECRET_KEY` 与 `ADMIN_INITIAL_PASSWORD` 为首次启动必填
- 功能描述修正:仅邮件 + 飞书两种通知渠道(无 Slack/钉钉/企业微信)、采集为 60 秒轮询
## 改动文件清单
25 个修改 + 3 个新建模块 + 3 个新建测试 + 4 份报告(`git diff --stat`:25 files changed,+1962/−842):
- 修改:`.env.example`、`README.md`、`alert-trigger.sh`、`backend/Dockerfile`、`backend/app/config.py`、`backend/app/database.py`、`backend/app/influxdb.py`、`backend/app/main.py`、`backend/app/routers/alerts.py`、`backend/app/routers/auth.py`、`backend/app/routers/groups.py`、`backend/app/routers/hosts.py`、`backend/app/routers/monitor.py`、`backend/app/routers/settings.py`、`backend/app/services/alert_engine.py`、`backend/app/services/email_notify.py`、`backend/app/services/feishu_notify.py`、`backend/app/services/ipmi_collector.py`、`backend/app/services/scheduler.py`、`backend/requirements.txt`、`docker-compose.yml`、`frontend/src/views/Hosts.vue`、`frontend/src/views/Settings.vue`、`start.sh`(24 个,另含删除的 `backend/app/init_admin.py`)
- 新建:`backend/app/crypto.py`、`backend/app/ssrf.py`、`backend/app/utils.py`、`backend/tests/test_sdr_parser.py`、`backend/tests/test_alert_recovery.py`、`backend/tests/test_parse_dt.py`
- 报告:`AUDIT.md`、`AUDIT-security.md`、`AUDIT-bugs.md`、`AUDIT-features.md`
## 未修复 / 待用户决策
| # | 事项 | 原因 / 说明 |
|---|---|---|
| 1 | **轮换已泄露的凭证**:JWT 密钥、管理员口令、InfluxDB 口令与 token、全部 BMC 口令、SMTP 口令、飞书 webhook | 代码已修(不再用默认值),但已部署环境里正在用的值若曾以默认值上线,必须逐一轮换;BMC 口令需逐台改;轮换后旧加密口令需用新密钥重新加密入库(或直接在页面重填) |
| 2 | **TLS / 反向代理**:生产仍是明文 HTTP;`/docs` 已默认关闭,但 TLS 需要证书与域名,由用户部署反代(Nginx/Caddy)或配 uvicorn 证书 | 需要用户提供域名/证书或决定反代方案 |
| 3 | **Token 改存 HttpOnly Cookie + refresh token**:当前仍为 localStorage + 单 token(已缩至 8h、可登出吊销);完整方案需前后端协同改动 | 需要用户确认是否接受该改动面 |
| 4 | **RBAC 与审计表**:多用户、角色(admin/operator/viewer)、操作审计日志,需改数据库 schema(`User.role`、审计表) | 需改 schema,本次未动;待用户定 |
| 5 | **告警状态持久化**:`_first_breach/_last_notify/_alert_active` 仍在进程内存,重启丢状态;已加清理逻辑,完整持久化需新表 | 需新表,待用户定 |
| 6 | **IPMI 认证降级**:已去掉无认证(`-C 0`)与 IPMI 1.5(`lan`)两种尝试,只留 `lanplus` 默认、`lanplus -C 3`、`lanplus -A MD5`;极老 BMC 若只支持 IPMI 1.5 会采不到 | 若用户有只支持 IPMI 1.5 的老机器,需单独放行 |
| 7 | **8086 端口暴露**:InfluxDB 8086 仍映射到宿主机(`start.sh` 健康检查依赖它);彻底隔离需改 `start.sh` 用 `docker compose exec` 检查 | 待用户定是否关闭映射 |
| 8 | **Uvicorn 单 worker**:采集与 API 同进程;多 worker 会导致 APScheduler 重复采集,需拆采集为独立进程才能安全扩 worker | 架构改动,待用户定 |
| 9 | **告警静默 / 严重级别 / 通用 Webhook / 批量导入 / 离线告警 / 配置中心 / 保留期管理 / 采集诊断页 / 主机详情页**:见 AUDIT-features.md 第三节 | 待用户定优先级 |
| 10 | **规则列表未分页**:`GET /api/hosts` 已加可选 limit/offset,`GET /api/alerts/rules` 仍全量返回 | 小改动,可随下次迭代做 |
| 11 | **`frontend/dist` 已提交进仓库**:dist 与源码可能不一致,且每次构建污染 diff;需 `git rm -r --cached frontend/dist` + `.gitignore` 排除,构建改由 Dockerfile/CI 做 | 涉及 git 操作,本次未动,待用户定 |
| 12 | **前端需重新构建**:`src` 已同步新字段,但 `dist/` 未动;部署前需 `npm run build` | 用户部署时执行 |
## 验证结果
- `python3 -m py_compile`:全部后端 Python 文件通过(含 3 个新建模块与 3 个新建测试)
- 全量测试:`cd backend && python3 -m unittest discover -s tests`,**36/36 通过**(原有 3 个总功耗 fallback + 新增 33 个:SDR 解析 12、告警恢复 9、parse_dt 12)
- BUG 一路冒烟测试(`/tmp/ipmi_smoke.py`,stub 缺失依赖后跑纯逻辑):**35/35 通过**(B5 负数/逗号、B6 状态映射、B7 分段、B8 legacy 展开、B9 去重/在位、B11 非法时间、O5 的 0 值有效与 PSU 求和、H2 三组合、B26 端口越界、B27 缺字段渲染)
- 安全一路冒烟测试(`/tmp/ipmi-smoke/smoke.py`,venv + TestClient + 临时 SQLite):**60 项检查中 58 项通过**,另 2 项经核查是误报(`/docs`、`/openapi.json` 在生产模式下返回的是 SPA index.html 而非 Swagger 内容,`app.docs_url/openapi_url` 已确认为 None);覆盖启动校验拒绝空/短密钥、POLL_INTERVAL 别名、加密往返与历史明文兼容、密钥文件 0600、SSRF 允许/拒绝矩阵、5 次失败后 429、创建主机响应无口令且库内为密文、设置掩码、登出后 token 401、改密后旧 token 401、新口令可登录;路径遍历 `%2e%2e` 探针均返回 404;`start.sh` 用假 docker 桩验证,秘密为空时退出码 1,补齐后通过;`docker-compose.yml` YAML 解析通过
- 未做:真实 BMC 联调、真实 InfluxDB 联调、Docker 构建验证(本机无 Docker 运行时与真实依赖);`frontend` 未重新构建(`dist/` 不动,源码改动需用户自行 `npm run build`);`requirements.txt` 的钉死版本在 Python 3.12 venv 中验证安装可用,未在目标 Python 3.11 容器内实测安装(bcrypt 钉 4.0.1 是按 passlib 1.7.4 兼容性选的)
- 已知小边界:`parse_dt(" ")`(纯空白)会抛 `ValueError` 而非返回 `None`,路由层映射为 400,不影响使用;吊销集合与登录限流是进程内内存,多副本部署不共享、重启后清空,单进程部署无影响
+159 -122
View File
@@ -1,28 +1,42 @@
# IPMI Monitor
基于 Web 的 IPMI 服务器硬件监控系统,实时采集和展示服务器温度、风扇转速、电源状态等关键指标,支持告警通知。
基于 Web 的 IPMI 服务器硬件监控系统,定时采集服务器温度、风扇转速、电压、电源功耗等传感器指标,支持按规则告警(邮件 / 飞书通知)与历史数据查询、CSV 导出。
## 功能特性
- 🖥️ **多服务器管理** - 支持添加和管理多台 IPMI 服务器
- 📊 **实时监控** - 实时采集温度、风扇转速、电压、电源功耗等传感器数据
- 📈 **历史趋势** - 基于 InfluxDB 时序数据库,支持历史数据查询和趋势图表
- ⚠️ **智能告警** - 可配置温度、风扇、功耗等阈值,触发告警通知
- 📧 **多渠道通知** - 支持邮件、Webhook(Slack / 钉钉 / 企业微信)告警推送
- 🐳 **一键部署** - Docker Compose 一键部署,开箱即用
- 📱 **响应式界面** - 纯静态前端,适配桌面和移动设备
- 🔌 **API 驱动** - 完整的 RESTful API,支持二次开发
- 🖥️ **多服务器管理** — 添加、管理多台 IPMI 服务器,按分组组织,支持连接测试与启用开关
- 📊 **定时采集** — 每 60 秒(可配)用 `ipmitool sdr elist` 轮询 BMC,写入 InfluxDB(无 WebSocket/SSE 实时推送)
- 📈 **历史趋势** — 基于 InfluxDB 时序数据,仪表盘提供温度 / 功耗趋势图(ECharts),支持多主机历史查询与 CSV 导出
- ⚠️ **智能告警** — 逐条配置告警规则(条件 gt/lt/eq/gte/lte、持续时长、冷却 3 小时、恢复通知),按主机 / 分组 / 全局生效
- 📧 **通知渠道** — 邮件(SMTP)与飞书群机器人;配置存在数据库中,页面设置里维护
- 🐳 **一键部署** — Docker Compose 一键部署(InfluxDB 2.7 + 后端),开箱即用
- 📱 **响应式界面** — Vue 3 单页应用(SPA),由 FastAPI 托管 `frontend/dist` 构建产物
- 🔌 **API 驱动** — 完整的 RESTful API(JWT 鉴权),支持二次开发
## 技术栈
| 组件 | 技术 |
|------|------|
| 后端 | Python 3.11 + FastAPI + Uvicorn |
| 数据库 | InfluxDB 2.7 (时序数据库) |
| 前端 | HTML / CSS / JavaScript (纯静态) |
| 监控采集 | ipmitool |
| 数据库 | SQLite(SQLAlchemy:用户 / 主机 / 分组 / 告警规则 / 告警历史 / 系统设置)+ InfluxDB 2.7(时序数据) |
| 前端 | Vue 3.4 + Vue Router 4.3 + Element Plus 2.7 + ECharts 5.5 + Vite 5.4 |
| 监控采集 | ipmitool(`sdr elist`) |
| 容器化 | Docker + Docker Compose |
## 页面一览
| 路由 | 页面 | 说明 |
|------|------|------|
| `/` | 仪表盘 | 主机在线统计、总功耗 / 最高温度汇总、温度与功耗趋势图 |
| `/sensors` | 传感器 | 各主机最新传感器值(温度 / 风扇 / 电压 / 功耗) |
| `/components` | 组件 | CPU / 内存 / PCIe / 硬盘 / 风扇 / 电源在位状态与总功耗 |
| `/hosts` | 主机管理 | 主机增删改、分组筛选、连接测试 |
| `/groups` | 分组管理 | 分组增删改 |
| `/alert-rules` | 告警规则 | 规则增删改(阈值、条件、持续时长、通知渠道) |
| `/alert-history` | 告警历史 | 分页查询、按主机 / 确认状态筛选、确认 |
| `/settings` | 系统设置 | 飞书 Webhook 与 SMTP 配置、测试邮件 / 测试飞书消息 |
| `/login` | 登录 | 用户名 + 密码登录(JWT) |
## 快速开始(Docker Compose 一键部署)
### 前置要求
@@ -43,7 +57,7 @@ cd /opt/ipmi-monitor
```bash
cp .env.example .env
# 编辑 .env 文件,修改密码、Token 等敏感配置
# 编辑 .env 文件,填写首次启动必填项(见下表)与 InfluxDB 凭据
vim .env
```
@@ -58,18 +72,32 @@ chmod +x start.sh
docker compose up -d
```
4. **访问系统**
4. **首次登录并改密**
首次启动时用户表为空,后端用 `ADMIN_INITIAL_PASSWORD` 创建初始管理员账号(默认用户名 `admin`,见启动日志)。**登录后立即改密**:
```bash
# 登录取 token 后调用改密接口(新密码至少 12 位;改密后此前签发的所有 token 失效,需重新登录)
curl -X POST http://your-server:8000/api/auth/change-password \
-H "Authorization: Bearer <token>" \
-H "Content-Type: application/json" \
-d '{"old_password": "<初始密码>", "new_password": "<新密码>"}'
```
5. **访问系统**
- 前端界面: http://your-server:8000
- API 文档: http://your-server:8000/docs
- API 文档(仅非生产环境): http://your-server:8000/docs
- InfluxDB 管理界面: http://your-server:8086
5. **添加第一台服务器**
6. **添加第一台主机**
在 Web 界面中点击「添加服务器」,填写 IPMI 连接信息:
- 服务器名称(别名)
- IPMI IP 地址
- 用户名 / 密码
在 Web 界面「主机管理」中点击「添加主机」,填写 IPMI 连接信息:主机名称、IPMI 地址、端口(默认 623)、用户名 / 密码。
### 生产环境注意
- 生产关闭 API 文档:`ENVIRONMENT=production`(docker-compose 默认值)时 `/docs`、`/redoc`、`/openapi.json` 均不可用;开发环境(`ENVIRONMENT=development`)才开启。
- 前端改动后需自行构建:`frontend/src` 改动后不会自动生效,需在 `frontend/` 下执行 `npm run build` 重新生成 `frontend/dist`(后端以 SPA 方式托管该目录),再重启后端容器。
## 手动部署步骤
@@ -95,7 +123,7 @@ pip install -r requirements.txt
```bash
cp .env.example .env
# 编辑 .env,配置 InfluxDB 连接信息
# 编辑 .env,填写首次启动必填项(JWT_SECRET_KEY、ADMIN_INITIAL_PASSWORD)与 InfluxDB 连接信息
vim .env
```
@@ -127,31 +155,39 @@ WantedBy=multi-user.target
## 环境变量说明
与 `backend/app/config.py` 逐项对齐。标记 **首次启动必填** 的两项缺失时后端拒绝启动(`JWT_SECRET_KEY` 至少 32 位;`ADMIN_INITIAL_PASSWORD` 至少 12 位,仅用户表为空时使用,此后改动无效)。
| 变量名 | 默认值 | 说明 |
|--------|--------|------|
| `INFLUXDB_USERNAME` | `admin` | InfluxDB 管理员用户名 |
| `INFLUXDB_PASSWORD` | `admin123456` | InfluxDB 管理员密码 |
| `JWT_SECRET_KEY` | —(**首次启动必填**) | JWT 签名密钥,至少 32 位随机字符串 |
| `ADMIN_INITIAL_PASSWORD` | —(**首次启动必填**,仅首次建管理员时用) | 初始管理员密码,至少 12 位;首次登录后立即改密 |
| `ENVIRONMENT` | `development` | `development` 或 `production`;production 关闭 `/docs`、`/redoc`、`/openapi.json` |
| `DATABASE_URL` | `sqlite+aiosqlite:///./data/ipmi_monitor.db` | 后端 SQLite 地址,Docker 部署时位于 `/app/data` 命名卷中 |
| `REQUIRE_EXISTING_DATABASE` | `false` | 生产保护:为 `true` 时数据库文件不存在则拒绝启动 |
| `MIN_EXPECTED_HOSTS` | `0` | 生产保护:主机数低于该下限则拒绝启动(如 `50`) |
| `INFLUXDB_URL` | `http://localhost:8086` | InfluxDB 地址(Docker Compose 中为 `http://influxdb:8086`) |
| `INFLUXDB_TOKEN` | — | InfluxDB API Token |
| `INFLUXDB_ORG` | `ipmi-monitor` | InfluxDB 组织名称 |
| `INFLUXDB_BUCKET` | `metrics` | InfluxDB 存储桶名称 |
| `INFLUXDB_TOKEN` | `my-super-secret-token` | InfluxDB API Token |
| `INFLUXDB_PORT` | `8086` | InfluxDB 端口映射 |
| `COMPOSE_PROJECT_NAME` | `ipmi-monitor` | Docker Compose 项目名;会影响命名卷名称,生产环境必须保持稳定 |
| `BACKEND_PORT` | `8000` | 后端服务端口映射 |
| `DATABASE_URL` | `sqlite+aiosqlite:///./data/ipmi_monitor.db` | 后端 SQLite 数据库地址,Docker 部署时位于 `/app/data` 命名卷中 |
| `REQUIRE_EXISTING_DATABASE` | `false` | 是否要求启动前数据库文件已存在;新环境保持 `false`,已有生产数据环境建议设为 `true` |
| `MIN_EXPECTED_HOSTS` | `0` | 数据库最少主机数量保护;新环境保持 `0`,生产环境可设为预期下限,如 `50` |
| `POLL_INTERVAL` | `60` | 数据采集间隔(秒) |
| `COLLECT_INTERVAL` | `60` | 数据采集间隔(秒);兼容旧名 `POLL_INTERVAL` 别名 |
| `LOG_LEVEL` | `INFO` | 日志级别 |
| `ALERT_TEMP_THRESHOLD` | `80` | 温度告警阈值(°C) |
| `ALERT_FAN_THRESHOLD` | `0` | 风扇转速告警阈值(RPM,0 禁用) |
| `ALERT_POWER_THRESHOLD` | `500` | 功耗告警阈值(W,0 禁用) |
| `SMTP_HOST` | - | SMTP 服务器地址 |
| `CORS_ORIGINS` | —(空) | 逗号分隔的跨域白名单;为空表示不启用 CORS(同源部署) |
| `JWT_ALGORITHM` | `HS256` | JWT 签名算法 |
| `JWT_EXPIRATION_HOURS` | `8` | JWT 有效期(小时) |
| `CREDENTIAL_ENCRYPTION_KEY` | —(空) | 可选:加密入库口令(BMC 口令 / SMTP 口令 / 飞书 URL)的 Fernet 密钥;为空时后端首次启动自动生成并以 0600 权限保存在数据库目录旁 |
| `SMTP_HOST` | —(空) | SMTP 服务器地址(环境变量兜底;告警实际用数据库「系统设置」里的 SMTP 配置) |
| `SMTP_PORT` | `587` | SMTP 端口 |
| `SMTP_USER` | - | SMTP 用户名 |
| `SMTP_PASSWORD` | - | SMTP 密码 |
| `ALERT_EMAIL_FROM` | - | 告警发件邮箱 |
| `ALERT_EMAIL_TO` | - | 告警收件邮箱(逗号分隔) |
| `WEBHOOK_URL` | - | Webhook 告警 URL |
| `SMTP_USER` | —(空) | SMTP 用户名 |
| `SMTP_PASSWORD` | —(空) | SMTP 密码 |
| `SMTP_FROM` | —(空) | 发件人地址(留空则用 SMTP 用户名) |
| `FEISHU_WEBHOOK_URL` | —(空) | 飞书机器人 Webhook(环境变量兜底;通知实际用数据库「系统设置」里的值) |
Docker Compose 专用变量(不在 `config.py` 中,由 compose / InfluxDB 容器使用):`INFLUXDB_USERNAME`、`INFLUXDB_PASSWORD`、`INFLUXDB_PORT`(默认 `8086`)、`COMPOSE_PROJECT_NAME`(默认 `ipmi-monitor`,影响命名卷名称,生产环境保持稳定)、`BACKEND_PORT`(默认 `8000`)。
说明:
- **告警阈值在页面告警规则里逐条配置**(条件、阈值、持续时长、通知渠道);不存在全局 `ALERT_*` 环境变量,旧文档中的 `ALERT_TEMP_THRESHOLD` / `ALERT_FAN_THRESHOLD` / `ALERT_POWER_THRESHOLD`、`ALERT_EMAIL_FROM` / `ALERT_EMAIL_TO`、`WEBHOOK_URL` 均已被删除,无人读取。
- BMC 口令、SMTP 口令、飞书 Webhook URL 入库前用 Fernet 加密存储;API 永不返回这些秘密的明文(主机列表只返回 `has_password`,设置接口只返回 `smtp_password_set` 与 `feishu_webhook_masked` 后 4 位掩码),更新时秘密字段留空表示不改。
## 数据持久化与生产环境保护
@@ -246,106 +282,99 @@ rm -rf data/
## API 文档概览
启动服务后,访问 `http://your-server:8000/docs` 查看完整的 Swagger UI 文档。
启动服务后,非生产环境访问 `http://your-server:8000/docs` 查看完整的 Swagger UI 文档。除登录外所有接口均需 JWT 鉴权(`Authorization: Bearer <token>`)。
### 主要 API 端点
#### 健康检查
| 方法 | 路径 | 说明 |
|------|------|------|
| `GET` | `/api/health` | 健康检查 |
| `GET` | `/api/servers` | 获取所有服务器列表 |
| `POST` | `/api/servers` | 添加服务器 |
| `PUT` | `/api/servers/{id}` | 更新服务器信息 |
| `DELETE` | `/api/servers/{id}` | 删除服务器 |
| `POST` | `/api/servers/{id}/scan` | 立即扫描服务器传感器 |
| `GET` | `/api/servers/{id}/sensors` | 获取服务器传感器数据 |
| `GET` | `/api/servers/{id}/metrics` | 获取历史指标数据 |
| `GET` | `/api/metrics/temperature` | 获取所有服务器温度数据 |
| `GET` | `/api/metrics/fan` | 获取所有服务器风扇数据 |
| `GET` | `/api/metrics/power` | 获取所有服务器功耗数据 |
| `GET` | `/api/alerts` | 获取告警列表 |
| `PUT` | `/api/alerts/config` | 更新告警配置 |
| `GET` | `/api/dashboard` | 获取仪表盘汇总数据 |
| `GET` | `/api/health` | 健康检查,返回 `{"status": "ok"}` |
#### 认证(`backend/app/routers/auth.py`)
| 方法 | 路径 | 说明 |
|------|------|------|
| `POST` | `/api/auth/login` | 用户名 + 密码登录,返回 JWT |
| `POST` | `/api/auth/verify` | 校验 token 是否有效 |
| `POST` | `/api/auth/logout` | 吊销当前 token |
| `POST` | `/api/auth/change-password` | 改当前用户密码(新密码至少 12 位;成功后此前所有 token 失效) |
| `GET` | `/api/auth/me` | 取当前登录用户信息 |
#### 主机(`backend/app/routers/hosts.py`)
| 方法 | 路径 | 说明 |
|------|------|------|
| `GET` | `/api/hosts` | 主机列表(不返回口令明文,只返回 `has_password`) |
| `POST` | `/api/hosts` | 添加主机 |
| `PUT` | `/api/hosts/{host_id}` | 更新主机(口令留空表示不改) |
| `DELETE` | `/api/hosts/{host_id}` | 删除主机 |
| `POST` | `/api/hosts/{host_id}/test` | 连接测试 |
| `GET` | `/api/hosts/{host_id}/status` | 主机状态 |
#### 分组(`backend/app/routers/groups.py`)
| 方法 | 路径 | 说明 |
|------|------|------|
| `GET` | `/api/groups` | 分组列表 |
| `POST` | `/api/groups` | 新建分组 |
| `PUT` | `/api/groups/{group_id}` | 更新分组 |
| `DELETE` | `/api/groups/{group_id}` | 删除分组 |
#### 监控数据(`backend/app/routers/monitor.py`)
| 方法 | 路径 | 说明 |
|------|------|------|
| `GET` | `/api/monitor/dashboard` | 仪表盘汇总(主机数、在线数、未确认告警数) |
| `GET` | `/api/monitor/dashboard/totals` | 各主机总功耗 / 最高温度汇总(可按分组筛选) |
| `GET` | `/api/monitor/history/{host_id}` | 单主机历史指标 |
| `GET` | `/api/monitor/history/multi` | 多主机历史指标(`host_ids` 逗号分隔,最多 50 个) |
| `GET` | `/api/monitor/summary/{metric_type}` | 指定类型传感器的最新值列表 |
| `GET` | `/api/monitor/sensors/{host_id}` | 单主机全部传感器最新值 |
| `GET` | `/api/monitor/realtime/{host_id}` | 单主机最新指标(按类型分组) |
| `GET` | `/api/monitor/trend` | 分组趋势数据(时间范围 1m–30d) |
| `GET` | `/api/monitor/components/{host_id}` | 单主机组件在位状态 |
| `GET` | `/api/monitor/components/{host_id}/grouped` | 单主机组件在位状态(按在位 / 缺失分组) |
| `GET` | `/api/monitor/export` | 历史指标 CSV 导出(`host_ids`、`metric_type`、`start`、`end`;列:time,host_id,metric_type,metric_name,value,unit) |
#### 告警(`backend/app/routers/alerts.py`)
| 方法 | 路径 | 说明 |
|------|------|------|
| `GET` | `/api/alerts/rules` | 告警规则列表(可按主机 / 分组 / 指标类型筛选) |
| `POST` | `/api/alerts/rules` | 新建告警规则 |
| `PUT` | `/api/alerts/rules/{rule_id}` | 更新告警规则 |
| `DELETE` | `/api/alerts/rules/{rule_id}` | 删除告警规则 |
| `GET` | `/api/alerts/history` | 告警历史(分页,可按主机 / 规则 / 确认状态筛选) |
| `PUT` | `/api/alerts/history/{history_id}/acknowledge` | 确认一条告警 |
#### 系统设置(`backend/app/routers/settings.py`)
| 方法 | 路径 | 说明 |
|------|------|------|
| `GET` | `/api/settings` | 取系统设置(秘密只返回掩码 / 是否已配置) |
| `PUT` | `/api/settings` | 更新系统设置(秘密字段留空表示不改) |
| `POST` | `/api/settings/test-email` | 发送测试邮件 |
| `POST` | `/api/settings/test-feishu` | 发送飞书测试消息 |
## 告警配置说明
### 告警触发条件
- **温度告警**: 当任意温度传感器读数超过 `ALERT_TEMP_THRESHOLD` 时触发
- **风扇告警**: 当风扇转速低于 `ALERT_FAN_THRESHOLD` 时触发(设为 0 禁用)
- **功耗告警**: 当功耗超过 `ALERT_POWER_THRESHOLD` 时触发(设为 0 禁用)
阈值在「告警规则」页面逐条配置,每条规则指定:指标类型(temperature / fan / voltage / power / component / status)、传感器名、条件(大于 / 小于 / 等于 / 大于等于 / 小于等于)、阈值、持续时长(超标持续满该时长才触发)、适用范围(单主机 / 分组 / 全部主机)、通知渠道(邮件、飞书)与收件邮箱。触发后 3 小时内不重复通知;指标恢复正常后发送恢复通知(仅当该告警此前确实触发过)。
### 告警通知渠道
#### 邮件通知
在 `.env` 中配置 SMTP 信息即可启用:
在「系统设置 → 邮件通知」中配置 SMTP 信息(服务器、端口、用户名、密码、发件人、TLS),保存后用「发送测试邮件」验证。告警按规则的收件邮箱列表发送。
```env
SMTP_HOST=smtp.gmail.com
SMTP_PORT=587
SMTP_USER=your-email@gmail.com
SMTP_PASSWORD=your-app-password
ALERT_EMAIL_FROM=your-email@gmail.com
ALERT_EMAIL_TO=admin1@example.com,admin2@example.com
```
#### 飞书通知
#### Webhook 通知
在「系统设置 → 飞书通知」中配置群机器人 Webhook 地址(可配安全关键词),保存后用「发送测试消息」验证。告警以交互式卡片推送到飞书群。
支持标准 Webhook 格式,可对接以下平台:
- **Slack**: 填入 Slack Incoming Webhook URL
- **钉钉**: 填入钉钉机器人 Webhook URL
- **企业微信**: 填入企业微信机器人 Webhook URL
- **自定义**: 填入任意 Webhook URL,POST JSON 格式通知
```env
WEBHOOK_URL=https://hooks.slack.com/services/xxx/yyy/zzz
```
## 截图
### 仪表盘
> 📸 **仪表盘页面**
>
> 展示所有服务器的概览信息,包括:
> - 服务器在线/离线状态
> - 关键温度指标
> - 风扇运行状态
> - 实时功耗统计
> - 最近告警摘要
### 服务器详情
> 📸 **服务器详情页面**
>
> 单台服务器的详细监控数据:
> - 所有传感器列表及当前读数
> - 温度趋势图(1h / 6h / 24h / 7d)
> - 风扇转速历史图表
> - 电压稳定性图表
> - 功耗变化趋势
### 告警管理
> 📸 **告警管理页面**
>
> 告警记录和配置管理:
> - 历史告警列表
> - 告警阈值配置
> - 通知渠道设置
> - 告警确认/静默操作
### 服务器管理
> 📸 **服务器管理页面**
>
> 添加和管理监控服务器:
> - 服务器列表(名称、IP、状态)
> - 添加/编辑/删除服务器
> - 连接测试
> - 批量操作
> 仅支持以上两种渠道。Slack / 钉钉 / 企业微信没有实现,不要按旧文档配置。
## 常见问题
@@ -368,6 +397,14 @@ WEBHOOK_URL=https://hooks.slack.com/services/xxx/yyy/zzz
2. 确认容器网络可达 IPMI 地址
3. 查看后端日志排查具体错误
### 组件页提示原始数据文件缺失
组件在位状态读取采集时写下的 `/tmp/ipmi_raw_{host_id}.txt`;该文件丢失(如容器重启)时组件接口会报 404,等待下一轮采集(60 秒)后恢复。
### 前端改了代码页面没变化
后端托管的是 `frontend/dist` 构建产物,改 `frontend/src` 后需在 `frontend/` 下执行 `npm run build` 重新生成 `dist`,再重启后端。
## License
This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.
Regular → Executable
+98 -47
View File
@@ -1,71 +1,122 @@
#!/bin/bash
# IPMI告警触发器 - 检测到告警后自动SSH执行脚本
# 用法: ./alert-trigger.sh --api-url http://localhost:8003 --ssh-key ~/.ssh/id_rsa
# IPMI 告警触发器 - 轮询未确认告警,对涉及的主机执行远程诊断
#
# 用法:
# ./alert-trigger.sh --api-url http://localhost:8003 --token <API_TOKEN> \
# --ssh-key ~/.ssh/id_ed25519 [--ssh-user monitor] [--interval 30]
#
# 依赖: curl, jq
# 状态文件: $HOME/.ipmi-monitor/alert-state.json (0600)
set -euo pipefail
API_URL="${1:-http://localhost:8003}"
SSH_KEY="${2:-~/.ssh/id_rsa}"
CHECK_INTERVAL=30 # 检查间隔(秒)
STATE_FILE="/tmp/ipmi-alert-state.json"
API_URL="http://localhost:8003"
TOKEN=""
SSH_KEY="$HOME/.ssh/id_ed25519"
SSH_USER="monitor"
CHECK_INTERVAL=30
STATE_DIR="$HOME/.ipmi-monitor"
STATE_FILE="$STATE_DIR/alert-state.json"
# 初始化状态文件
usage() {
echo "用法: $0 --api-url <url> --token <token> --ssh-key <path> [--ssh-user <user>] [--interval <seconds>]"
exit 2
}
# 按 --flag value 解析参数(O14:旧版按位置参数取,--api-url 会被当成 URL)
while [ $# -gt 0 ]; do
case "$1" in
--api-url) API_URL="${2:?--api-url 需要一个值}"; shift 2 ;;
--token) TOKEN="${2:?--token 需要一个值}"; shift 2 ;;
--ssh-key) SSH_KEY="${2:?--ssh-key 需要一个值}"; shift 2 ;;
--ssh-user) SSH_USER="${2:?--ssh-user 需要一个值}"; shift 2 ;;
--interval) CHECK_INTERVAL="${2:?--interval 需要一个值}"; shift 2 ;;
-h|--help) usage ;;
*) echo "未知参数: $1" >&2; usage ;;
esac
done
if [ -z "$TOKEN" ]; then
echo "错误: 必须提供 --token(API Bearer token)" >&2
usage
fi
for cmd in curl jq; do
if ! command -v "$cmd" >/dev/null 2>&1; then
echo "错误: 缺少依赖 $cmd,请先安装" >&2
exit 1
fi
done
if [ ! -f "$SSH_KEY" ]; then
echo "错误: SSH 私钥不存在: $SSH_KEY" >&2
exit 1
fi
# 状态文件:目录 0700、文件 0600
mkdir -p "$STATE_DIR"
chmod 0700 "$STATE_DIR"
if [ ! -f "$STATE_FILE" ]; then
echo '{}' > "$STATE_FILE"
fi
chmod 0600 "$STATE_FILE"
echo "=== IPMI告警触发器 ==="
echo "=== IPMI 告警触发器 ==="
echo "API地址: $API_URL"
echo "SSH 用户: $SSH_USER"
echo "检查间隔: ${CHECK_INTERVAL}s"
echo ""
while true; do
# 获取当前告警
ALERTS=$(curl -s "$API_URL/api/alerts" 2>/dev/null)
if [ -z "$ALERTS" ] || [ "$ALERTS" = "[]" ]; then
sleep $CHECK_INTERVAL
# 真实接口:GET /api/alerts/history?acknowledged=false,Bearer 鉴权,
# 返回分页对象 {total, page, page_size, items: [...]}
RESPONSE=$(curl -sS --fail --max-time 15 \
-H "Authorization: Bearer $TOKEN" \
"$API_URL/api/alerts/history?acknowledged=false&page_size=100") || {
echo "[$(date '+%Y-%m-%d %H:%M:%S')] API 请求失败,${CHECK_INTERVAL}s 后重试" >&2
sleep "$CHECK_INTERVAL"
continue
fi
# 解析告警信息(使用jq)
echo "[$(date '+%Y-%m-%d %H:%M:%S')] 检测到告警:"
echo "$ALERTS" | jq -r '.[] | " - [\(.severity | ascii_upcase)] \(.host): \(.message)"' 2>/dev/null
# 遍历告警,按主机执行脚本
echo "$ALERTS" | jq -r '.[] | "\(.host)|\(.severity)|\(.sensor_type)|\(.message)"' 2>/dev/null | while IFS='|' read -r host severity sensor_type message; do
# 检查是否已经处理过这个告警(避免重复触发)
LAST_STATE=$(cat "$STATE_FILE" | jq -r ".\"${host}_${sensor_type}\" // \"\"" 2>/dev/null)
CURRENT_STATE="${message}"
if [ "$LAST_STATE" = "$CURRENT_STATE" ]; then
}
# jq 传参一律用 --arg,避免把变量拼进过滤器(注入风险)
mapfile -t ALERTS < <(echo "$RESPONSE" | jq -r '
(.items // [])[]
| [.id, (.host_name // ""), (.rule_name // ""), (.metric_name // ""), (.message // "")]
| @tsv')
for line in "${ALERTS[@]}"; do
IFS=$'\t' read -r alert_id host_name rule_name metric_name message <<< "$line"
[ -n "$alert_id" ] || continue
# 以告警 id 去重:已处理过的 id 不再触发
if jq -e --arg id "$alert_id" 'has($id)' "$STATE_FILE" >/dev/null; then
continue
fi
echo ""
echo ">>> 触发告警: $host | $severity | $sensor_type"
echo ">>> 消息: $message"
# 自动SSH到目标主机执行脚本
# 方式1: SSH到告警对应的主机(如果host是IP地址)
if [[ "$host" =~ ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo ">>> SSH到 $host 执行诊断脚本..."
ssh -i "$SSH_KEY" -o StrictHostKeyChecking=no -o ConnectTimeout=10 root@"$host" '
# 在这里写你要执行的脚本
echo "[$(date '+%Y-%m-%d %H:%M:%S')] 检测到未确认告警 #$alert_id: [$host_name] $rule_name - $message"
# host_name 是显示名不是 IP;这里只对 IP 格式的目标执行 SSH
target="$host_name"
if [[ "$target" =~ ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
echo ">>> SSH 到 $target 执行诊断脚本..."
ssh -i "$SSH_KEY" -o StrictHostKeyChecking=accept-new \
-o ConnectTimeout=10 "${SSH_USER}@${target}" '
echo "=== 主机诊断 ==="
hostname
uptime
free -h
df -h
# 添加你的自定义命令
' 2>&1
' 2>&1 || echo ">>> SSH 执行失败(主机 $target)" >&2
else
# 方式2: host是主机名,需要通过IPMI获取IP
echo ">>> 主机名: $host,跳过SSH(需要提供IP地址)"
echo ">>> 主机名 '$target' 不是 IP 地址,跳过 SSH(需提供 IP)"
fi
# 更新状态
TEMP=$(cat "$STATE_FILE")
echo "$TEMP" | jq ".\"${host}_${sensor_type}\" = \"$CURRENT_STATE\"" > "$STATE_FILE" 2>/dev/null
# 原子更新状态文件(jq --arg 传参,临时文件 + mv)
tmp_state="$(mktemp "$STATE_DIR/.state.XXXXXX")"
jq --arg id "$alert_id" --arg msg "$message" '.[$id] = $msg' \
"$STATE_FILE" > "$tmp_state"
chmod 0600 "$tmp_state"
mv "$tmp_state" "$STATE_FILE"
done
sleep $CHECK_INTERVAL
sleep "$CHECK_INTERVAL"
done
+11 -6
View File
@@ -13,19 +13,21 @@ RUN apt-get update && \
# Set working directory
WORKDIR /app
# Use pip mirror for faster downloads
RUN pip config set global.index-url https://mirrors.aliyun.com/pypi/simple/ && \
pip config set global.trusted-host mirrors.aliyun.com
# Use pip mirror for faster downloads (HTTPS only, no --trusted-host)
RUN pip config set global.index-url https://mirrors.aliyun.com/pypi/simple/
# Create a dedicated non-root user for the application
RUN groupadd -r appuser && useradd -r -g appuser -m appuser
# Copy and install Python dependencies
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
# Copy application code
COPY . .
# Copy application code (owned by the non-root user)
COPY --chown=appuser:appuser . .
# Create data directory for persistent storage
RUN mkdir -p /app/data
RUN mkdir -p /app/data && chown -R appuser:appuser /app/data
# Expose port
EXPOSE 8000
@@ -34,5 +36,8 @@ EXPOSE 8000
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
CMD curl -f http://localhost:8000/api/health || exit 1
# Run as the non-root user
USER appuser
# Start uvicorn server
CMD ["sh", "-c", "uvicorn app.main:app --host 0.0.0.0 --port ${PORT:-8000}"]
+73 -5
View File
@@ -1,12 +1,29 @@
"""Application configuration using pydantic-settings."""
import hashlib
from pydantic import AliasChoices, Field
from pydantic_settings import BaseSettings
from typing import Optional
# SHA-256 of the legacy hard-coded JWT placeholder that used to ship as the
# default value. The literal value itself is deliberately not stored here;
# startup validation rejects any key whose hash matches this digest.
_LEGACY_JWT_DEFAULT_SHA256 = (
"c99758db33af70fbf3588833c77678baa8d763407fcb82c58509e6332a63e21c"
)
# Minimum accepted JWT secret length (characters).
MIN_JWT_SECRET_LENGTH = 32
class Settings(BaseSettings):
"""Application settings loaded from environment variables."""
# Runtime environment: "development" or "production". Production disables
# the interactive API docs and enables stricter defaults.
ENVIRONMENT: str = "development"
# Database
DATABASE_URL: str = "sqlite+aiosqlite:///./data/ipmi_monitor.db"
# Production safety guard: when enabled, refuse to boot if the SQLite DB
@@ -21,10 +38,14 @@ class Settings(BaseSettings):
INFLUXDB_URL: str = "http://localhost:8086"
INFLUXDB_TOKEN: str = ""
INFLUXDB_ORG: str = "ipmi-monitor"
INFLUXDB_BUCKET: str = "ipmi_metrics"
INFLUXDB_BUCKET: str = "metrics"
# Collection
COLLECT_INTERVAL: int = 60 # seconds
# Collection (POLL_INTERVAL is the historical env-var name, kept as an
# alias so existing deployments keep working).
COLLECT_INTERVAL: int = Field(
default=60,
validation_alias=AliasChoices("COLLECT_INTERVAL", "POLL_INTERVAL"),
) # seconds
# SMTP / Email
SMTP_HOST: str = ""
@@ -39,10 +60,29 @@ class Settings(BaseSettings):
# Logging
LOG_LEVEL: str = "INFO"
# CORS: comma-separated list of allowed origins. Empty means CORS is not
# enabled at all (same-origin deployment).
CORS_ORIGINS: str = ""
# JWT Authentication
JWT_SECRET_KEY: str = "your-secret-key-change-this-in-production"
# SECURITY: no default value. JWT_SECRET_KEY must be injected via the
# environment (see .env.example / docker-compose.yml). Startup refuses
# to boot without a key of at least 32 characters (see
# validate_security_settings()).
JWT_SECRET_KEY: str = ""
JWT_ALGORITHM: str = "HS256"
JWT_EXPIRATION_HOURS: int = 24
JWT_EXPIRATION_HOURS: int = 8
# Initial admin account. Only consulted when the user table is empty:
# ADMIN_INITIAL_PASSWORD (at least 12 characters) is then required to
# create the first admin user at startup. Ignored once any user exists.
ADMIN_INITIAL_PASSWORD: str = ""
# Credential encryption (Fernet key for BMC passwords / SMTP password /
# Feishu webhook URL stored in the database). Optional: when unset, a key
# is generated once and persisted next to the SQLite file (see
# app/crypto.py).
CREDENTIAL_ENCRYPTION_KEY: str = ""
model_config = {
"env_file": ".env",
@@ -51,3 +91,31 @@ class Settings(BaseSettings):
settings = Settings()
def validate_security_settings(s: Optional[Settings] = None) -> None:
"""Fail fast on unsafe security configuration.
Called only from the application startup path (lifespan), never at
import time, so importing this module (e.g. in tests) stays side-effect
free. Raises RuntimeError when the JWT secret is missing, equals the
legacy shipped default, or is shorter than 32 characters.
"""
cfg = s if s is not None else settings
key = cfg.JWT_SECRET_KEY or ""
if not key:
raise RuntimeError(
"Refusing to start: JWT_SECRET_KEY is not set. Generate a secret "
"with at least 32 characters and provide it via the JWT_SECRET_KEY "
"environment variable (see .env.example)."
)
if hashlib.sha256(key.encode("utf-8")).hexdigest() == _LEGACY_JWT_DEFAULT_SHA256:
raise RuntimeError(
"Refusing to start: JWT_SECRET_KEY still equals the legacy shipped "
"default value. Generate a fresh secret of at least 32 characters."
)
if len(key) < MIN_JWT_SECRET_LENGTH:
raise RuntimeError(
"Refusing to start: JWT_SECRET_KEY must be at least "
f"{MIN_JWT_SECRET_LENGTH} characters long (got {len(key)})."
)
+130
View File
@@ -0,0 +1,130 @@
"""Application-layer encryption for stored credentials.
BMC (IPMI) passwords, the SMTP password and the Feishu webhook URL are stored
in the database encrypted with Fernet (AES-128-CBC + HMAC-SHA256, via the
``cryptography`` package) instead of plaintext.
Key resolution order:
1. ``CREDENTIAL_ENCRYPTION_KEY`` environment variable — a urlsafe base64
32-byte Fernet key (generate with
``python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"``).
2. Otherwise a key file ``.credential_key`` next to the SQLite database file
(created with mode 0600 on first use and persisted across restarts).
``decrypt_value`` deliberately returns undecryptable input unchanged so
pre-existing plaintext rows keep working until the one-time migration in
``app.database.init_db`` re-encrypts them.
"""
import base64
import binascii
import logging
import os
from pathlib import Path
from cryptography.fernet import Fernet, InvalidToken
from app.config import settings
logger = logging.getLogger(__name__)
_KEY_FILE_NAME = ".credential_key"
_fernet: Fernet | None = None
def _database_dir() -> Path:
"""Directory holding the SQLite file (fallback: ./data)."""
url = settings.DATABASE_URL or ""
for prefix in ("sqlite+aiosqlite:///", "sqlite:///"):
if url.startswith(prefix):
raw = url.removeprefix(prefix)
if raw and raw != ":memory:":
path = Path(raw)
if not path.is_absolute():
path = Path.cwd() / path
return path.parent
return Path.cwd() / "data"
def _load_or_create_key_file() -> bytes:
"""Read the persisted key file, creating it (0600) on first use."""
directory = _database_dir()
directory.mkdir(parents=True, exist_ok=True)
key_path = directory / _KEY_FILE_NAME
if key_path.exists():
raw = key_path.read_bytes().strip()
if raw:
return raw
logger.warning("Credential key file %s is empty; regenerating it", key_path)
key = Fernet.generate_key()
# Write atomically-ish with restrictive permissions from the start.
fd = os.open(key_path, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)
try:
os.write(fd, key)
finally:
os.close(fd)
try:
os.chmod(key_path, 0o600)
except OSError:
pass
logger.info("Generated credential encryption key file at %s", key_path)
return key
def get_fernet() -> Fernet:
"""Return the cached Fernet instance for credential encryption."""
global _fernet
if _fernet is not None:
return _fernet
env_key = (settings.CREDENTIAL_ENCRYPTION_KEY or "").strip()
if env_key:
try:
raw = base64.urlsafe_b64decode(env_key.encode("ascii"))
if len(raw) != 32:
raise ValueError("decoded key is not 32 bytes")
_fernet = Fernet(env_key.encode("ascii"))
return _fernet
except (ValueError, binascii.Error) as exc:
raise ValueError(
"CREDENTIAL_ENCRYPTION_KEY is not a valid urlsafe base64 "
"32-byte Fernet key"
) from exc
_fernet = Fernet(_load_or_create_key_file())
return _fernet
def encrypt_value(plaintext: str | None) -> str | None:
"""Encrypt a credential value. None/empty input is returned unchanged."""
if not plaintext:
return plaintext
token = get_fernet().encrypt(plaintext.encode("utf-8"))
return token.decode("ascii")
def decrypt_value(token: str | None) -> str | None:
"""Decrypt a credential value.
Returns the input unchanged when it is not a valid Fernet token (legacy
plaintext rows) or when it is None/empty.
"""
if not token:
return token
try:
return get_fernet().decrypt(token.encode("ascii")).decode("utf-8")
except (InvalidToken, ValueError):
# Legacy plaintext stored before encryption was introduced.
return token
def is_encrypted(value: str | None) -> bool:
"""Return True when *value* is a valid Fernet token for the current key."""
if not value:
return False
try:
get_fernet().decrypt(value.encode("ascii"))
return True
except (InvalidToken, ValueError):
return False
+97 -13
View File
@@ -1,11 +1,12 @@
"""SQLAlchemy async engine and session management."""
import logging
import os
from pathlib import Path
from sqlalchemy import event, func, select, text
from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine, async_sessionmaker
from sqlalchemy.orm import DeclarativeBase
from sqlalchemy import func, select
from app.config import settings
@@ -16,6 +17,21 @@ engine = create_async_engine(
echo=False,
)
def _set_sqlite_pragmas(dbapi_connection, connection_record):
"""Apply per-connection SQLite pragmas (WAL, FK enforcement, timeout)."""
cursor = dbapi_connection.cursor()
try:
cursor.execute("PRAGMA journal_mode=WAL")
cursor.execute("PRAGMA foreign_keys=ON")
cursor.execute("PRAGMA busy_timeout=5000")
finally:
cursor.close()
if settings.DATABASE_URL.startswith("sqlite"):
event.listens_for(engine.sync_engine, "connect")(_set_sqlite_pragmas)
async_session = async_sessionmaker(
engine,
class_=AsyncSession,
@@ -46,11 +62,15 @@ def _sqlite_database_path(database_url: str) -> Path | None:
async def get_db() -> AsyncSession: # type: ignore[misc]
"""FastAPI dependency that yields a database session."""
"""FastAPI dependency that yields a database session.
Read-only by default: this dependency never commits. Write endpoints
must call ``await db.commit()`` explicitly (and let the exception
propagate if the commit fails); on error the transaction is rolled back.
"""
async with async_session() as session:
try:
yield session
await session.commit()
except Exception:
await session.rollback()
raise
@@ -58,8 +78,53 @@ async def get_db() -> AsyncSession: # type: ignore[misc]
await session.close()
async def _create_indexes(conn) -> None:
"""Create indexes used by hot query paths (idempotent)."""
for name, table, column in [
("idx_alert_history_host_id", "alert_history", "host_id"),
("idx_alert_rules_host_id", "alert_rules", "host_id"),
("idx_hosts_group_id", "hosts", "group_id"),
("idx_system_settings_key", "system_settings", "key"),
]:
await conn.execute(
text(f"CREATE INDEX IF NOT EXISTS {name} ON {table} ({column})")
)
async def _migrate_host_passwords() -> None:
"""One-time migration: re-encrypt any plaintext IPMI passwords in place.
Rows that fail to migrate are logged and left untouched; a migration
failure must never block startup.
"""
from app.crypto import encrypt_value, is_encrypted
from app.models import Host
async with async_session() as session:
try:
result = await session.execute(select(Host))
hosts = result.scalars().all()
migrated = 0
for host in hosts:
try:
if host.ipmi_password and not is_encrypted(host.ipmi_password):
host.ipmi_password = encrypt_value(host.ipmi_password)
migrated += 1
except Exception as exc: # noqa: BLE001 - never block startup
logger.warning(
"Failed to migrate IPMI password for host %s (%s): %s",
host.id, host.name, exc,
)
if migrated:
await session.commit()
logger.info("Encrypted %d stored IPMI password(s)", migrated)
except Exception as exc: # noqa: BLE001 - never block startup
logger.warning("IPMI password migration skipped: %s", exc)
await session.rollback()
async def init_db() -> None:
"""Create tables and ensure default admin user exists."""
"""Create tables, indexes, and ensure an initial admin user exists."""
db_path = _sqlite_database_path(settings.DATABASE_URL)
if settings.REQUIRE_EXISTING_DATABASE and db_path is not None and not db_path.exists():
raise RuntimeError(
@@ -70,6 +135,7 @@ async def init_db() -> None:
async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
await _create_indexes(conn)
from app.models import Host
@@ -85,28 +151,46 @@ async def init_db() -> None:
"backend mounted a new/wrong Docker volume. Refusing to boot."
)
# 检查并创建默认管理员账号
# One-time migration of legacy plaintext BMC passwords (best effort).
await _migrate_host_passwords()
# Initial admin account: only when the user table is completely empty.
# The password must come from the ADMIN_INITIAL_PASSWORD environment
# variable (at least 12 characters); it is never hard-coded and never
# logged. Existing deployments (any user present) are unaffected.
from app.models import User
import bcrypt
async with async_session() as session:
try:
result = await session.execute(select(User).where(User.username == "admin"))
existing_user = result.scalar_one_or_none()
result = await session.execute(select(func.count()).select_from(User))
user_count = result.scalar() or 0
if not existing_user:
# 创建默认管理员账号
default_password = "admin123"
hashed_password = bcrypt.hashpw(default_password.encode('utf-8'), bcrypt.gensalt()).decode('utf-8')
if user_count == 0:
initial_password = (
settings.ADMIN_INITIAL_PASSWORD
or os.environ.get("ADMIN_INITIAL_PASSWORD", "")
)
if not initial_password or len(initial_password) < 12:
raise RuntimeError(
"Refusing to start: no users exist in the database and "
"ADMIN_INITIAL_PASSWORD is missing or shorter than 12 "
"characters. Set ADMIN_INITIAL_PASSWORD to a strong "
"password (at least 12 characters) for the initial "
"admin account."
)
hashed_password = bcrypt.hashpw(
initial_password.encode("utf-8"), bcrypt.gensalt()
).decode("utf-8")
new_user = User(
username="admin",
hashed_password=hashed_password
hashed_password=hashed_password,
)
session.add(new_user)
await session.commit()
logger.info("✓ 默认管理员账号创建成功: admin / admin123")
logger.info("已创建初始管理员账号")
else:
logger.info("✓ 管理员账号已存在")
except Exception as e:
+77 -6
View File
@@ -1,5 +1,6 @@
"""InfluxDB 2.x client wrapper for reading/writing IPMI metrics."""
import asyncio
import logging
from datetime import datetime, timezone
from typing import Any, Optional
@@ -11,6 +12,10 @@ from app.config import settings
logger = logging.getLogger(__name__)
# InfluxDB client timeouts (O15/B15): 10s connect/read/write so a slow or
# dead InfluxDB cannot hang the event loop forever.
INFLUX_TIMEOUT_MS = 10_000
_client: Optional[InfluxDBClient] = None
_write_api = None
_query_api = None
@@ -24,6 +29,7 @@ def get_client() -> InfluxDBClient:
url=settings.INFLUXDB_URL,
token=settings.INFLUXDB_TOKEN,
org=settings.INFLUXDB_ORG,
timeout=INFLUX_TIMEOUT_MS,
)
return _client
@@ -57,8 +63,12 @@ def write_metric(
metric_name: str,
value: float,
tags: Optional[dict[str, str]] = None,
) -> None:
"""Write a single metric point into InfluxDB."""
) -> bool:
"""Write a single metric point into InfluxDB.
Returns ``True`` on success, ``False`` on failure (B14) so callers can
tell a failed write apart from "no data".
"""
try:
point = (
Point(metric_type)
@@ -73,14 +83,18 @@ def write_metric(
get_write_api().write(bucket=settings.INFLUXDB_BUCKET, record=point)
logger.debug("Written metric %s/%s=%.2f for host %s", metric_type, metric_name, value, host_id)
return True
except Exception:
logger.exception("Failed to write metric to InfluxDB")
return False
def write_metrics_batch(points: list[dict[str, Any]]) -> None:
def write_metrics_batch(points: list[dict[str, Any]]) -> bool:
"""Write multiple metric points at once.
Each dict should contain: host_id, metric_type, metric_name, value, tags (optional).
Returns ``True`` on success, ``False`` on failure (B14).
"""
try:
influx_points: list[Point] = []
@@ -99,8 +113,54 @@ def write_metrics_batch(points: list[dict[str, Any]]) -> None:
get_write_api().write(bucket=settings.INFLUXDB_BUCKET, record=influx_points)
logger.debug("Batch written %d metrics", len(influx_points))
return True
except Exception:
logger.exception("Failed to batch-write metrics to InfluxDB")
return False
# ---------------------------------------------------------------------------
# Async wrappers (B15)
#
# InfluxDBClient is synchronous — never call it directly from async code.
# These wrappers run the blocking calls in a worker thread. The sync
# functions above stay available for the few remaining sync call sites.
# ---------------------------------------------------------------------------
async def awrite_metric(*args: Any, **kwargs: Any) -> bool:
return await asyncio.to_thread(write_metric, *args, **kwargs)
async def awrite_metrics_batch(points: list[dict[str, Any]]) -> bool:
return await asyncio.to_thread(write_metrics_batch, points)
async def aquery_metrics(
host_id: int,
metric_type: str,
start_time: datetime,
end_time: Optional[datetime] = None,
) -> list[dict[str, Any]]:
return await asyncio.to_thread(query_metrics, host_id, metric_type, start_time, end_time)
async def aquery_latest(host_id: int, metric_type: str) -> list[dict[str, Any]]:
return await asyncio.to_thread(query_latest, host_id, metric_type)
async def aquery_trend(
host_id: int,
metric_type: str,
start_time: datetime,
end_time: Optional[datetime] = None,
) -> list[dict[str, Any]]:
return await asyncio.to_thread(query_trend, host_id, metric_type, start_time, end_time)
async def aquery_latest_grouped(
host_id: int, only: Optional[set[str]] = None
) -> dict[str, list[dict[str, Any]]]:
return await asyncio.to_thread(query_latest_grouped, host_id, only)
# ---------------------------------------------------------------------------
@@ -140,6 +200,7 @@ from(bucket: "{settings.INFLUXDB_BUCKET}")
|> filter(fn: (r) => r["_measurement"] == "{metric_type}")
|> filter(fn: (r) => r["host_id"] == "{host_id}")
|> filter(fn: (r) => r["_field"] == "value")
|> limit(n: 5000)
"""
results: list[dict[str, Any]] = []
try:
@@ -206,10 +267,20 @@ def query_trend(
return query_metrics(host_id, metric_type, start_time, end_time)
def query_latest_grouped(host_id: int) -> dict[str, list[dict[str, Any]]]:
"""Return latest metrics grouped by metric_type for a host."""
def query_latest_grouped(
host_id: int, only: Optional[set[str]] = None
) -> dict[str, list[dict[str, Any]]]:
"""Return latest metrics grouped by metric_type for a host.
``only`` (O1) limits the query to the requested types instead of always
fetching all four — e.g. ``only={"temperature"}`` for the temperature
summary endpoint.
"""
types = ("temperature", "fan", "voltage", "power")
if only is not None:
types = tuple(t for t in types if t in only)
grouped: dict[str, list[dict[str, Any]]] = {}
for metric_type in ("temperature", "fan", "voltage", "power"):
for metric_type in types:
grouped[metric_type] = query_latest(host_id, metric_type)
return grouped
-86
View File
@@ -1,86 +0,0 @@
#!/usr/bin/env python3
"""
初始化管理员账号脚本
运行此脚本会在数据库中创建默认管理员账号
"""
import asyncio
import sys
from pathlib import Path
# 添加项目根目录到Python路径
sys.path.insert(0, str(Path(__file__).parent.parent))
from app.database import engine
from app.models import User, Base
from sqlalchemy import select
from passlib.context import CryptContext
pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")
async def init_admin_user():
"""创建默认管理员账号"""
# 创建所有表(如果不存在)
async with engine.begin() as conn:
await conn.run_sync(Base.metadata.create_all)
# 检查是否已存在管理员账号(使用独立的连接)
async with engine.connect() as conn:
result = await conn.execute(select(User).where(User.username == "admin"))
existing_user = result.scalar_one_or_none()
if existing_user:
print("✓ 管理员账号已存在,跳过创建")
print(f" 用户名: {existing_user.username}")
print(f" 创建时间: {existing_user.created_at}")
return
# 创建默认管理员账号
default_password = "admin123"
hashed_password = pwd_context.hash(default_password)
async with engine.begin() as conn:
new_user = User(
username="admin",
hashed_password=hashed_password
)
conn.add(new_user)
await conn.commit()
print("✓ 默认管理员账号创建成功")
print(f" 用户名: admin")
print(f" 密码: {default_password}")
print("\n⚠️ 重要提醒:")
print(" 1. 请立即登录系统修改默认密码")
print(" 2. 在生产环境中,请更改 JWT_SECRET_KEY 配置")
async def main():
"""主函数"""
try:
print("=" * 50)
print("IPMI监控系统 - 初始化管理员账号")
print("=" * 50)
print()
await init_admin_user()
print()
print("=" * 50)
print("初始化完成")
print("=" * 50)
except Exception as e:
print(f"\n✗ 错误: {e}")
import traceback
traceback.print_exc()
sys.exit(1)
finally:
await engine.dispose()
if __name__ == "__main__":
asyncio.run(main())
+45 -12
View File
@@ -7,9 +7,9 @@ from pathlib import Path
from fastapi import FastAPI, Request
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import FileResponse
from fastapi.responses import FileResponse, JSONResponse
from app.config import settings
from app.config import settings, validate_security_settings
from app.database import init_db
from app.routers import alerts, groups, hosts, monitor, auth
from app.routers import settings as settings_router
@@ -25,6 +25,9 @@ _data_dir.mkdir(parents=True, exist_ok=True)
async def lifespan(app: FastAPI):
"""Application lifespan: startup & shutdown hooks."""
# ---- Startup ----
# Fail fast on unsafe security configuration (missing / legacy / short
# JWT secret). Runs here, not at import time.
validate_security_settings()
logging.basicConfig(
level=getattr(logging, settings.LOG_LEVEL.upper(), logging.INFO),
format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
@@ -55,20 +58,44 @@ async def lifespan(app: FastAPI):
# Create FastAPI app
# ---------------------------------------------------------------------------
_is_production = settings.ENVIRONMENT.lower() == "production"
app = FastAPI(
title="IPMI Monitor",
version="0.1.0",
lifespan=lifespan,
# Never expose interactive API docs in production.
docs_url=None if _is_production else "/docs",
redoc_url=None if _is_production else "/redoc",
openapi_url=None if _is_production else "/openapi.json",
)
# CORS
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
@app.middleware("http")
async def security_headers(request: Request, call_next):
"""Add baseline security response headers to every response."""
response = await call_next(request)
response.headers["X-Content-Type-Options"] = "nosniff"
response.headers["X-Frame-Options"] = "DENY"
response.headers["Referrer-Policy"] = "strict-origin-when-cross-origin"
response.headers["Content-Security-Policy"] = (
"default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline'; "
"img-src 'self' data:; font-src 'self'; connect-src 'self'; "
"frame-ancestors 'none'; base-uri 'self'; form-action 'self'"
)
return response
# CORS: only enabled when an explicit origin allow-list is configured.
# Unconfigured means same-origin deployment and no CORS middleware at all.
_cors_origins = [o.strip() for o in settings.CORS_ORIGINS.split(",") if o.strip()]
if _cors_origins:
app.add_middleware(
CORSMiddleware,
allow_origins=_cors_origins,
allow_credentials=True,
allow_methods=["GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS"],
allow_headers=["Authorization", "Content-Type"],
)
# ---------------------------------------------------------------------------
# API routers
@@ -114,10 +141,16 @@ if _frontend_dist and _frontend_dist.is_dir():
@app.get("/{full_path:path}")
async def serve_spa(full_path: str):
from starlette.responses import Response
# Unknown /api/ paths are API 404s, never the SPA entry point.
if full_path == "api" or full_path.startswith("api/"):
return JSONResponse({"detail": "Not Found"}, status_code=404)
# Try to serve a real static file (e.g. /assets/index-xxx.js, favicon.ico)
if full_path:
file_path = _frontend_dist / full_path
dist_root = _frontend_dist.resolve()
file_path = (_frontend_dist / full_path).resolve()
# Path traversal guard: the resolved path must stay inside dist.
if not file_path.is_relative_to(dist_root):
return JSONResponse({"detail": "Not Found"}, status_code=404)
if file_path.is_file():
return FileResponse(str(file_path))
# Otherwise return index.html for Vue Router to handle
+56 -11
View File
@@ -1,11 +1,12 @@
"""Alert rules and history REST API."""
import logging
import math
from datetime import datetime
from typing import Optional
from typing import Literal, Optional
from fastapi import APIRouter, Depends, HTTPException, Query
from pydantic import BaseModel, Field
from pydantic import BaseModel, Field, field_validator
from sqlalchemy import func, select
from sqlalchemy.ext.asyncio import AsyncSession
@@ -17,39 +18,65 @@ from app.models import User as UserModel
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/alerts", tags=["alerts"])
# ---------------------------------------------------------------------------
# Schemas — Rules
# ---------------------------------------------------------------------------
# Shared value-domain validation (B18)
MetricType = Literal["temperature", "fan", "voltage", "power", "component", "status"]
Condition = Literal["gt", "lt", "eq", "gte", "lte"]
def _validate_threshold(value: Optional[float]) -> Optional[float]:
"""Reject NaN / infinite thresholds (B18)."""
if value is not None and (isinstance(value, float) and (math.isnan(value) or math.isinf(value))):
raise ValueError("threshold must be a finite number")
return value
class RuleCreate(BaseModel):
name: str = Field(..., max_length=128)
host_id: Optional[int] = None
group_id: Optional[int] = None
metric_type: str = Field(..., max_length=64) # temperature / voltage / fan / power
metric_type: MetricType
metric_name: str = Field(..., max_length=128)
condition: str = Field(..., max_length=16) # gt / lt / eq / gte / lte
condition: Condition
threshold: float
duration: int = Field(default=0, description="Seconds the condition must hold before alerting")
duration: int = Field(default=0, ge=0, description="Seconds the condition must hold before alerting")
enabled: bool = True
notify_email: bool = False
notify_feishu: bool = False
email_addresses: str = Field(default="", description="Comma-separated emails")
@field_validator("threshold")
@classmethod
def _check_threshold(cls, value: float) -> float:
return _validate_threshold(value) # type: ignore[return-value]
class RuleUpdate(BaseModel):
name: Optional[str] = Field(None, max_length=128)
host_id: Optional[int] = None
group_id: Optional[int] = None
metric_type: Optional[str] = Field(None, max_length=64)
metric_type: Optional[MetricType] = None
metric_name: Optional[str] = Field(None, max_length=128)
condition: Optional[str] = Field(None, max_length=16)
condition: Optional[Condition] = None
threshold: Optional[float] = None
duration: Optional[int] = None
duration: Optional[int] = Field(None, ge=0)
enabled: Optional[bool] = None
notify_email: Optional[bool] = None
notify_feishu: Optional[bool] = None
email_addresses: Optional[str] = None
@field_validator("threshold")
@classmethod
def _check_threshold(cls, value: Optional[float]) -> Optional[float]:
return _validate_threshold(value)
def _check_host_group_exclusive(
host_id: Optional[int], group_id: Optional[int]
) -> None:
"""A rule binds to at most one of host / group (B17)."""
if host_id is not None and group_id is not None:
raise HTTPException(400, "host_id and group_id are mutually exclusive")
class RuleOut(BaseModel):
id: int
@@ -120,6 +147,7 @@ async def create_rule(
db: AsyncSession = Depends(get_db),
current_user: UserModel = Depends(get_current_user)
):
_check_host_group_exclusive(body.host_id, body.group_id)
if body.host_id is not None:
host = await db.get(Host, body.host_id)
if not host:
@@ -133,6 +161,7 @@ async def create_rule(
db.add(rule)
await db.flush()
await db.refresh(rule)
await db.commit() # B23: explicit commit, do not rely on get_db auto-commit
return RuleOut.model_validate(rule)
@@ -148,11 +177,25 @@ async def update_rule(
raise HTTPException(404, "Rule not found")
update_data = body.model_dump(exclude_unset=True)
# B17: the merged host_id/group_id pair must still be exclusive.
new_host_id = update_data.get("host_id", rule.host_id)
new_group_id = update_data.get("group_id", rule.group_id)
_check_host_group_exclusive(new_host_id, new_group_id)
if update_data.get("host_id") is not None:
host = await db.get(Host, update_data["host_id"])
if not host:
raise HTTPException(404, "Host not found")
if update_data.get("group_id") is not None:
group = await db.get(HostGroup, update_data["group_id"])
if not group:
raise HTTPException(404, "Group not found")
for field, value in update_data.items():
setattr(rule, field, value)
await db.flush()
await db.refresh(rule)
await db.commit() # B23
return RuleOut.model_validate(rule)
@@ -166,6 +209,7 @@ async def delete_rule(
if not rule:
raise HTTPException(404, "Rule not found")
await db.delete(rule)
await db.commit() # B23
# ---------------------------------------------------------------------------
@@ -232,6 +276,7 @@ async def acknowledge_alert(
record.acknowledged = True
await db.flush()
await db.refresh(record)
await db.commit() # B23
d = HistoryOut.model_validate(record)
if record.host:
+177 -25
View File
@@ -1,13 +1,15 @@
"""Authentication endpoints: login, verify, logout."""
"""Authentication endpoints: login, verify, logout, change-password."""
from datetime import datetime, timedelta
import logging
import uuid
from datetime import datetime, timedelta, timezone
from typing import Optional
from fastapi import APIRouter, Depends, HTTPException, status
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
from jose import JWTError, jwt
import bcrypt
from pydantic import BaseModel
import jwt
from fastapi import APIRouter, Depends, HTTPException, Request, status
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
from pydantic import BaseModel, Field
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
@@ -15,6 +17,59 @@ from app.config import settings
from app.database import get_db
from app.models import User
logger = logging.getLogger(__name__)
# ---------------------------------------------------------------------------
# Token revocation (process-local)
# ---------------------------------------------------------------------------
# JTIs revoked by explicit logout. Password changes revoke every token
# issued before the change via the per-user timestamp below.
_REVOKED_JTIS: set[str] = set()
_PASSWORD_CHANGED_AT: dict[int, datetime] = {}
# ---------------------------------------------------------------------------
# Login rate limiting (process-local)
# ---------------------------------------------------------------------------
# (client IP, username) -> [consecutive failures, locked-until timestamp]
_LOGIN_FAILURES: dict[tuple[str, str], list] = {}
MAX_LOGIN_FAILURES = 5
LOGIN_LOCKOUT = timedelta(minutes=15)
# Pre-computed dummy hash so a login for a non-existent user costs the same
# bcrypt verification as a real one (no username-enumeration timing signal).
_DUMMY_HASH = bcrypt.hashpw(b"__ipmi_monitor_dummy__", bcrypt.gensalt()).decode("utf-8")
def _client_ip(request: Optional[Request]) -> str:
if request is not None and request.client is not None:
return request.client.host
return "unknown"
def _check_login_lockout(ip: str, username: str) -> None:
entry = _LOGIN_FAILURES.get((ip, username))
if entry and entry[1] is not None and datetime.now(timezone.utc) < entry[1]:
raise HTTPException(
status_code=status.HTTP_429_TOO_MANY_REQUESTS,
detail="Too many failed login attempts, please try again later",
headers={"Retry-After": str(int(LOGIN_LOCKOUT.total_seconds()))},
)
def _record_login_failure(ip: str, username: str) -> None:
entry = _LOGIN_FAILURES.setdefault((ip, username), [0, None])
entry[0] += 1
if entry[0] >= MAX_LOGIN_FAILURES:
entry[1] = datetime.now(timezone.utc) + LOGIN_LOCKOUT
logger.warning(
"Login locked for user %r from %s after %d failures",
username, ip, entry[0],
)
def _clear_login_failures(ip: str, username: str) -> None:
_LOGIN_FAILURES.pop((ip, username), None)
# ---------------------------------------------------------------------------
# Password hashing
@@ -35,8 +90,13 @@ def verify_password(plain_password: str, hashed_password: str) -> bool:
# ---------------------------------------------------------------------------
class LoginRequest(BaseModel):
username: str
password: str
username: str = Field(..., min_length=1, max_length=64)
password: str = Field(..., min_length=1, max_length=256)
class ChangePasswordRequest(BaseModel):
old_password: str = Field(..., min_length=1, max_length=256)
new_password: str = Field(..., min_length=12, max_length=256)
class TokenResponse(BaseModel):
@@ -56,33 +116,43 @@ class UserResponse(BaseModel):
# ---------------------------------------------------------------------------
# JWT utilities
# JWT utilities (PyJWT, explicit HS256)
# ---------------------------------------------------------------------------
def create_access_token(data: dict) -> str:
"""Create a JWT access token."""
"""Create a JWT access token (8h default, UTC, with jti + iat)."""
now = datetime.now(timezone.utc)
to_encode = data.copy()
expire = datetime.now() + timedelta(hours=settings.JWT_EXPIRATION_HOURS)
to_encode.update({"exp": expire})
encoded_jwt = jwt.encode(
to_encode.update({
"exp": now + timedelta(hours=settings.JWT_EXPIRATION_HOURS),
"iat": now,
"jti": uuid.uuid4().hex,
})
return jwt.encode(
to_encode,
settings.JWT_SECRET_KEY,
algorithm=settings.JWT_ALGORITHM
algorithm=settings.JWT_ALGORITHM,
)
return encoded_jwt
def verify_token(token: str) -> Optional[dict]:
"""Verify and decode a JWT token."""
"""Verify and decode a JWT token. Returns None when invalid/expired/revoked."""
try:
payload = jwt.decode(
token,
settings.JWT_SECRET_KEY,
algorithms=[settings.JWT_ALGORITHM]
algorithms=["HS256"],
)
return payload
except JWTError:
except jwt.InvalidTokenError:
return None
if payload.get("jti") in _REVOKED_JTIS:
return None
return payload
def revoke_token_jti(jti: Optional[str]) -> None:
if jti:
_REVOKED_JTIS.add(jti)
# ---------------------------------------------------------------------------
@@ -125,6 +195,22 @@ async def get_current_user(
headers={"WWW-Authenticate": "Bearer"},
)
# Tokens issued before the user's last password change are revoked.
changed_at = _PASSWORD_CHANGED_AT.get(user.id)
if changed_at is not None:
iat = payload.get("iat")
issued_at = (
datetime.fromtimestamp(iat, tz=timezone.utc)
if isinstance(iat, (int, float))
else None
)
if issued_at is not None and issued_at < changed_at:
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Token revoked (password changed)",
headers={"WWW-Authenticate": "Bearer"},
)
return user
@@ -137,7 +223,8 @@ router = APIRouter(prefix="/api/auth", tags=["Authentication"])
@router.post("/login", response_model=TokenResponse)
async def login(
request: LoginRequest,
body: LoginRequest,
request: Request,
db: AsyncSession = Depends(get_db)
):
"""
@@ -145,22 +232,36 @@ async def login(
Returns a JWT access token if credentials are valid.
"""
ip = _client_ip(request)
_check_login_lockout(ip, body.username)
# Find user by username
result = await db.execute(
select(User).where(User.username == request.username)
select(User).where(User.username == body.username)
)
user = result.scalar_one_or_none()
# Verify user exists and password is correct
if user is None or not verify_password(request.password, user.hashed_password):
# Constant-time behaviour: unknown users still pay one bcrypt check so
# the response time does not reveal whether the username exists.
password_ok = (
verify_password(body.password, user.hashed_password)
if user is not None
else verify_password(body.password, _DUMMY_HASH)
)
if user is None or not password_ok:
_record_login_failure(ip, body.username)
logger.warning("Failed login attempt for user %r from %s", body.username, ip)
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Invalid username or password",
headers={"WWW-Authenticate": "Bearer"},
)
# Update last login time
user.last_login = datetime.now()
_clear_login_failures(ip, body.username)
# Update last login time (stored naive UTC for the SQLite DateTime column)
user.last_login = datetime.now(timezone.utc).replace(tzinfo=None)
await db.commit()
# Create access token
@@ -191,6 +292,57 @@ async def verify_token_endpoint(
return VerifyResponse(valid=True, username=username)
@router.post("/logout")
async def logout(
credentials: HTTPAuthorizationCredentials = Depends(security),
current_user: User = Depends(get_current_user),
):
"""Revoke the token used for this request."""
payload = verify_token(credentials.credentials)
revoke_token_jti(payload.get("jti") if payload else None)
logger.info("User %r logged out", current_user.username)
return {"success": True, "message": "Logged out"}
@router.post("/change-password")
async def change_password(
body: ChangePasswordRequest,
credentials: HTTPAuthorizationCredentials = Depends(security),
current_user: User = Depends(get_current_user),
db: AsyncSession = Depends(get_db),
):
"""Change the current user's password.
The new password must be at least 12 characters. On success every token
issued before this change is revoked (including the current one).
"""
# NOTE: get_current_user loads the user in its own session, so re-fetch
# it in this endpoint's session before mutating.
user = await db.get(User, current_user.id)
if user is None:
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="User not found")
if not verify_password(body.old_password, user.hashed_password):
logger.warning(
"Failed password change attempt for user %r", current_user.username
)
raise HTTPException(
status_code=status.HTTP_401_UNAUTHORIZED,
detail="Current password is incorrect",
)
user.hashed_password = hash_password(body.new_password)
await db.commit()
# Revoke all previously issued tokens for this user.
_PASSWORD_CHANGED_AT[current_user.id] = datetime.now(timezone.utc)
payload = verify_token(credentials.credentials)
revoke_token_jti(payload.get("jti") if payload else None)
logger.info("User %r changed password; all prior tokens revoked", current_user.username)
return {"success": True, "message": "Password changed; please log in again"}
@router.get("/me", response_model=UserResponse)
async def get_me(current_user: User = Depends(get_current_user)):
"""
+3
View File
@@ -80,6 +80,7 @@ async def create_group(
db.add(group)
await db.flush()
await db.refresh(group)
await db.commit() # B23: explicit commit, do not rely on get_db auto-commit
return GroupOut.model_validate(group)
@@ -100,6 +101,7 @@ async def update_group(
await db.flush()
await db.refresh(group)
await db.commit() # B23
return GroupOut.model_validate(group)
@@ -120,3 +122,4 @@ async def delete_group(
raise HTTPException(400, "Group still has hosts – remove or reassign them first")
await db.delete(group)
await db.commit() # B23
+54 -15
View File
@@ -10,11 +10,13 @@ from pydantic import BaseModel, Field
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.crypto import decrypt_value, encrypt_value
from app.database import get_db
from app.models import Host, HostGroup
from app import influxdb as influxdb_module
from app.routers.auth import get_current_user
from app.models import User as UserModel
from app.ssrf import validate_ipmi_host
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/hosts", tags=["hosts"])
@@ -49,7 +51,8 @@ class HostOut(BaseModel):
ipmi_host: str
ipmi_port: int
ipmi_username: str
ipmi_password: str
# The BMC password is never returned via the API; only whether one is set.
has_password: bool = False
group_id: Optional[int]
group_name: Optional[str] = None
enabled: bool
@@ -59,6 +62,15 @@ class HostOut(BaseModel):
model_config = {"from_attributes": True}
def _to_host_out(host: Host) -> HostOut:
"""Build the API view of a host (password replaced by a boolean flag)."""
out = HostOut.model_validate(host)
out.has_password = bool(host.ipmi_password)
if host.group:
out.group_name = host.group.name
return out
# ---------------------------------------------------------------------------
# CRUD
# ---------------------------------------------------------------------------
@@ -66,22 +78,22 @@ class HostOut(BaseModel):
@router.get("", response_model=list[HostOut])
async def list_hosts(
group_id: Optional[int] = Query(None),
limit: Optional[int] = Query(None, ge=1, description="Max hosts to return"),
offset: int = Query(0, ge=0, description="Hosts to skip"),
db: AsyncSession = Depends(get_db),
current_user: UserModel = Depends(get_current_user),
):
stmt = select(Host).order_by(Host.id)
if group_id is not None:
stmt = stmt.where(Host.group_id == group_id)
if limit is not None:
stmt = stmt.limit(limit)
if offset:
stmt = stmt.offset(offset)
result = await db.execute(stmt)
hosts = result.scalars().all()
out: list[HostOut] = []
for h in hosts:
d = HostOut.model_validate(h)
if h.group:
d.group_name = h.group.name
out.append(d)
return out
return [_to_host_out(h) for h in hosts]
@router.post("", response_model=HostOut, status_code=201)
@@ -90,17 +102,26 @@ async def create_host(
db: AsyncSession = Depends(get_db),
current_user: UserModel = Depends(get_current_user)
):
try:
validate_ipmi_host(body.ipmi_host)
except ValueError as exc:
raise HTTPException(400, str(exc))
# Validate group_id if provided
if body.group_id is not None:
group = await db.get(HostGroup, body.group_id)
if not group:
raise HTTPException(404, "Group not found")
host = Host(**body.model_dump())
data = body.model_dump()
# Encrypt the BMC password before it reaches the database.
data["ipmi_password"] = encrypt_value(body.ipmi_password)
host = Host(**data)
db.add(host)
await db.flush()
await db.refresh(host)
return HostOut.model_validate(host)
await db.commit()
return _to_host_out(host)
@router.put("/{host_id}", response_model=HostOut)
@@ -115,15 +136,23 @@ async def update_host(
raise HTTPException(404, "Host not found")
update_data = body.model_dump(exclude_unset=True)
if "ipmi_host" in update_data and update_data["ipmi_host"]:
try:
validate_ipmi_host(update_data["ipmi_host"])
except ValueError as exc:
raise HTTPException(400, str(exc))
for field, value in update_data.items():
# 如果密码字段为空字符串或None,保留原密码不更新
if field == "ipmi_password" and (value is None or value == ""):
continue
if field == "ipmi_password":
value = encrypt_value(value)
setattr(host, field, value)
await db.flush()
await db.refresh(host)
return HostOut.model_validate(host)
await db.commit()
return _to_host_out(host)
@router.delete("/{host_id}", status_code=204)
@@ -136,6 +165,7 @@ async def delete_host(
if not host:
raise HTTPException(404, "Host not found")
await db.delete(host)
await db.commit()
# ---------------------------------------------------------------------------
@@ -153,8 +183,17 @@ async def test_ipmi_connection(
if not host:
raise HTTPException(404, "Host not found")
try:
validate_ipmi_host(host.ipmi_host)
except ValueError as exc:
raise HTTPException(400, str(exc))
from app.services.ipmi_collector import _run_ipmitool_with_auth, parse_sdr_output
# The collector needs the cleartext password; decrypt it here (internal
# use only — it is never sent back through the API).
ipmi_password = decrypt_value(host.ipmi_password)
try:
# Try different connection methods like collect_host does
raw = None
@@ -166,12 +205,12 @@ async def test_ipmi_connection(
["lan", ["-A", "MD5"]],
["lan", None],
]
for interface, extra_args in attempts:
try:
raw = await _run_ipmitool_with_auth(
host.ipmi_host, host.ipmi_port,
host.ipmi_username, host.ipmi_password,
host.ipmi_username, ipmi_password,
interface=interface,
extra_args=extra_args,
)
@@ -179,10 +218,10 @@ async def test_ipmi_connection(
break
except RuntimeError:
continue
if not raw:
raise RuntimeError("All connection attempts failed")
parsed = parse_sdr_output(raw)
return {
"success": True,
+200 -123
View File
@@ -1,12 +1,14 @@
"""Monitoring data REST API — dashboard, history, realtime."""
import csv
import io
import logging
import re
from datetime import datetime, timedelta, timezone
from pathlib import Path as PathLib
from typing import Optional, List
from fastapi import APIRouter, Depends, Query, Path
from fastapi import APIRouter, Depends, HTTPException, Query, Path
from fastapi.responses import StreamingResponse
from sqlalchemy import func, select
from sqlalchemy.ext.asyncio import AsyncSession
@@ -16,10 +18,58 @@ from app.models import AlertHistory, Host, Host as HostModel
from app.routers.auth import get_current_user
from app.models import User as UserModel
from app.services.component_parser import parse_component_presence, group_components_by_status
from app.utils import parse_dt, resolve_total_power, utc_now_naive
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/monitor", tags=["monitor"])
# Metric types accepted by the history/summary/trend endpoints (B12).
METRIC_TYPES = {"temperature", "fan", "voltage", "power", "component", "status"}
MAX_HOST_IDS = 50
def _parse_host_ids(host_ids: str) -> list[int]:
"""Parse a comma-separated host-id list (B12).
Raises 400 for non-integer ids or more than 50 ids.
"""
ids: list[int] = []
for raw in host_ids.split(","):
raw = raw.strip()
if not raw:
continue
try:
ids.append(int(raw))
except ValueError:
raise HTTPException(400, f"Invalid host id: {raw!r}")
if len(ids) > MAX_HOST_IDS:
raise HTTPException(400, f"Too many host ids (max {MAX_HOST_IDS})")
return ids
def _check_metric_type(metric_type: str) -> str:
"""Reject metric types outside the whitelist (B12)."""
if metric_type not in METRIC_TYPES:
raise HTTPException(400, f"Invalid metric_type: {metric_type!r}")
return metric_type
def _parse_time_range(
start: Optional[str], end: Optional[str]
) -> tuple[datetime, Optional[datetime]]:
"""Parse start/end query params (B11).
Bad values are a 400, never a 500. Defaults use UTC now (B11).
"""
try:
start_dt = parse_dt(start)
end_dt = parse_dt(end)
except ValueError as exc:
raise HTTPException(400, f"Invalid datetime: {exc}")
if start_dt is None:
start_dt = utc_now_naive() - timedelta(hours=1)
return start_dt, end_dt
# ---------------------------------------------------------------------------
# Dashboard
@@ -80,36 +130,18 @@ async def multi_host_history(
current_user: UserModel = Depends(get_current_user),
):
"""Query historical metrics for multiple hosts."""
host_id_list = [int(x.strip()) for x in host_ids.split(',') if x.strip()]
# Parse datetime with timezone handling
def parse_dt(dt_str: Optional[str]) -> Optional[datetime]:
if not dt_str:
return None
# Handle both 'Z' suffix and '+00:00' format
dt_str = dt_str.replace('Z', '+00:00')
try:
dt = datetime.fromisoformat(dt_str)
# If timezone-aware, convert to UTC naive datetime for InfluxDB queries
if dt.tzinfo is not None:
return dt.astimezone(timezone.utc).replace(tzinfo=None)
return dt
except ValueError:
# Fallback: strip timezone info
clean_str = dt_str.split('+')[0].split('-')[0][:19] if '+' in dt_str or dt_str.count('-') > 2 else dt_str[:19]
return datetime.fromisoformat(clean_str)
start_dt = parse_dt(start) or (datetime.now() - timedelta(hours=1))
end_dt = parse_dt(end)
host_id_list = _parse_host_ids(host_ids)
_check_metric_type(metric_type)
start_dt, end_dt = _parse_time_range(start, end)
all_data = {}
for hid in host_id_list:
data = influxdb_module.query_metrics(hid, metric_type, start_dt, end_dt)
data = await influxdb_module.aquery_metrics(hid, metric_type, start_dt, end_dt)
# Filter by metric_name if specified
if metric_name:
data = [d for d in data if d.get('metric_name') == metric_name]
all_data[str(hid)] = data
return {
"metric_type": metric_type,
"metric_name": metric_name,
@@ -119,6 +151,53 @@ async def multi_host_history(
}
@router.get("/export")
async def export_metrics(
host_ids: str = Query(..., description="Comma-separated host IDs, e.g. '1,2,3'"),
metric_type: str = Query("temperature", description="temperature/fan/voltage/power/component/status"),
start: Optional[str] = Query(None, description="ISO datetime, default 1h ago"),
end: Optional[str] = Query(None, description="ISO datetime, default now"),
current_user: UserModel = Depends(get_current_user),
):
"""Export historical metrics for multiple hosts as a CSV file.
Columns: time,host_id,metric_type,metric_name,value,unit.
Reuses the same host-id / metric-type / time-range validation as the
history endpoints (400 on bad input) and ``influxdb.query_metrics``
(via its async wrapper) for the data.
"""
host_id_list = _parse_host_ids(host_ids)
if not host_id_list:
raise HTTPException(400, "At least one host id is required")
_check_metric_type(metric_type)
start_dt, end_dt = _parse_time_range(start, end)
unit = _get_unit(metric_type)
buf = io.StringIO()
writer = csv.writer(buf)
writer.writerow(["time", "host_id", "metric_type", "metric_name", "value", "unit"])
for hid in host_id_list:
data = await influxdb_module.aquery_metrics(hid, metric_type, start_dt, end_dt)
for d in data:
writer.writerow([
d.get("time", ""),
hid,
metric_type,
d.get("metric_name", ""),
d.get("value", ""),
unit,
])
filename = (
f"ipmi_metrics_{metric_type}_{start_dt.strftime('%Y%m%d%H%M%S')}.csv"
)
return StreamingResponse(
iter([buf.getvalue()]),
media_type="text/csv",
headers={"Content-Disposition": f'attachment; filename="{filename}"'},
)
@router.get("/summary/{metric_type}")
async def metrics_summary(
metric_type: str = Path(..., description="temperature/fan/voltage/power"),
@@ -127,28 +206,30 @@ async def metrics_summary(
current_user: UserModel = Depends(get_current_user),
):
"""Get all sensors for specified hosts (not aggregated).
Returns ALL sensors per host, not aggregated values.
For dashboard totals, use /dashboard/totals endpoint instead.
"""
_check_metric_type(metric_type)
# Parse host IDs
host_id_list: Optional[List[int]] = None
if host_ids:
host_id_list = [int(x.strip()) for x in host_ids.split(',') if x.strip()]
host_id_list = _parse_host_ids(host_ids)
# Get enabled hosts
stmt = select(HostModel).where(HostModel.enabled.is_(True))
if host_id_list:
stmt = stmt.where(HostModel.id.in_(host_id_list))
result = await db.execute(stmt)
hosts = result.scalars().all()
# Return ALL sensors, not aggregated
# Return ALL sensors, not aggregated (only the requested type, O1)
all_sensors = []
for host in hosts:
grouped = influxdb_module.query_latest_grouped(host.id)
grouped = await influxdb_module.aquery_latest_grouped(host.id, only={metric_type})
metrics = grouped.get(metric_type, [])
for m in metrics:
all_sensors.append({
"host_id": host.id,
@@ -159,7 +240,7 @@ async def metrics_summary(
"time": m.get('time'),
"timestamp": m.get('timestamp'),
})
return {"metric_type": metric_type, "sensors": all_sensors}
@@ -169,8 +250,8 @@ async def host_sensors(
current_user: UserModel = Depends(get_current_user)
):
"""Get all sensors for a specific host with latest values."""
grouped = influxdb_module.query_latest_grouped(host_id)
grouped = await influxdb_module.aquery_latest_grouped(host_id)
sensors = []
for metric_type, metrics in grouped.items():
for m in metrics:
@@ -181,7 +262,7 @@ async def host_sensors(
"unit": _get_unit(metric_type),
"time": m.get('time'),
})
return {"host_id": host_id, "sensors": sensors}
@@ -209,42 +290,20 @@ async def dashboard_totals(
stmt = stmt.where(HostModel.group_id == group_id)
result = await db.execute(stmt)
hosts = result.scalars().all()
# Get groups for reference
from app.routers.groups import get_all_groups
groups = await get_all_groups(db)
host_totals = []
for host in hosts:
grouped = influxdb_module.query_latest_grouped(host.id)
grouped = await influxdb_module.aquery_latest_grouped(host.id)
# Get total power sensor value (not sum of all power sensors)
# Priority:
# 1. Native system-wide reading: Total_Power -> Power -> Pwr Consumption
# 2. Fallback: sum of all PSU{N}_PIN (PSU input power) when no native
# reading exists (covers hosts like Gooxi12 where BMC does not
# report a system Pwr Consumption sensor).
power_metrics = grouped.get("power", [])
total_power = 0
for sensor_name in ['Total_Power', 'Power', 'Pwr Consumption']:
for m in power_metrics:
if m.get('metric_name') == sensor_name:
total_power = m.get('value', 0)
break
if total_power > 0:
break
if total_power <= 0:
_psu_pin_re = re.compile(r'^PSU\d+_PIN$', re.IGNORECASE)
psu_pin_sum = 0.0
for m in power_metrics:
name = m.get('metric_name') or ''
if _psu_pin_re.match(name):
try:
psu_pin_sum += float(m.get('value', 0) or 0)
except (TypeError, ValueError):
continue
if psu_pin_sum > 0:
total_power = psu_pin_sum
# Total power via the shared resolver (O5): native system-wide
# reading first, otherwise the PSU{N}_PIN sum. A 0 reading from an
# existing sensor is valid and must not trigger the fallback.
total_power = resolve_total_power(grouped.get("power", []))
total_power = float(total_power) if total_power is not None else 0.0
# Get outlet temperature (exhaust air temperature)
temp_metrics = grouped.get("temperature", [])
@@ -258,9 +317,24 @@ async def dashboard_totals(
break
if outlet_temp is not None:
break
# Fallback to max temperature if no outlet sensor found
max_temp = outlet_temp if outlet_temp is not None else max((m.get('value', 0) for m in temp_metrics if m.get('value') is not None), default=0)
# Fallback to max temperature if no outlet sensor found (None-safe, B13)
if outlet_temp is not None:
try:
max_temp = float(outlet_temp)
except (TypeError, ValueError):
max_temp = 0.0
else:
temps = []
for m in temp_metrics:
v = m.get('value')
if v is None:
continue
try:
temps.append(float(v))
except (TypeError, ValueError):
continue
max_temp = max(temps) if temps else 0.0
host_totals.append({
"host_id": host.id,
"host_name": host.name,
@@ -272,7 +346,7 @@ async def dashboard_totals(
"max_temperature": round(max_temp, 1),
"last_seen": host.last_seen.isoformat() if host.last_seen else None,
})
# Group totals
group_totals = {}
for gt in host_totals:
@@ -282,22 +356,26 @@ async def dashboard_totals(
"group_id": gid,
"group_name": next((g["name"] for g in groups if g["id"] == gid), "Unknown"),
"total_power": 0,
"avg_temperature": 0,
"avg_max_temperature": 0,
"host_count": 0,
}
group_totals[gid]["total_power"] += gt["total_power"]
group_totals[gid]["avg_temperature"] += gt["max_temperature"]
group_totals[gid]["total_power"] += gt["total_power"] or 0
group_totals[gid]["avg_max_temperature"] += gt["max_temperature"] or 0
group_totals[gid]["host_count"] += 1
# Calculate averages
# Calculate averages. NOTE (B13): the field is named
# avg_max_temperature because it averages each host's max_temperature;
# the frontend does not read this field (Dashboard.vue computes its own
# average from max_temperature), so the rename is safe.
for gid in group_totals.values():
if gid["host_count"] > 0:
gid["avg_temperature"] = round(gid["avg_temperature"] / gid["host_count"], 1)
gid["avg_max_temperature"] = round(gid["avg_max_temperature"] / gid["host_count"], 1)
gid["total_power"] = round(gid["total_power"], 1)
return {
"hosts": host_totals,
"groups": [{"group_id": g["id"], "group_name": g["name"]} for g in groups],
"group_totals": list(group_totals.values()),
}
@@ -310,6 +388,9 @@ async def trend(
current_user: UserModel = Depends(get_current_user),
):
"""Get trend data for all hosts in a group for the specified time range."""
if metric_type not in ("temperature", "fan", "voltage", "power"):
raise HTTPException(400, f"Invalid metric_type: {metric_type!r}")
# Parse time range to timedelta
range_map = {
"default": timedelta(hours=1),
@@ -325,7 +406,7 @@ async def trend(
"30d": timedelta(days=30),
}
delta = range_map.get(time_range, timedelta(hours=1))
start_time = datetime.utcnow() - delta
start_time = datetime.now(timezone.utc).replace(tzinfo=None) - delta
# Get enabled hosts
stmt = select(HostModel).where(HostModel.enabled.is_(True))
@@ -337,22 +418,37 @@ async def trend(
# Query trend data for each host
trend_data = {}
for host in hosts:
metrics = influxdb_module.query_trend(host.id, metric_type, start_time)
metrics = await influxdb_module.aquery_trend(host.id, metric_type, start_time)
points = []
for m in metrics:
ts = m.get("timestamp") or m.get("time")
if not ts or m.get("value") is None:
continue
# For power metrics, only include total power sensors
if metric_type == "power":
metric_name = m.get("metric_name", "")
if metric_name not in ('Total_Power', 'Power', 'Pwr Consumption'):
if metric_type == "power":
# B30: same aggregation as the dashboard totals — per timestamp,
# native total-power sensor wins, otherwise sum PSU{N}_PIN.
by_ts: dict[str, list[dict]] = {}
for m in metrics:
ts = m.get("timestamp") or m.get("time")
if not ts or m.get("value") is None:
continue
points.append({
"timestamp": ts,
"metric_name": m.get("metric_name", ""),
"value": m["value"],
})
by_ts.setdefault(str(ts), []).append(m)
for ts, ts_metrics in by_ts.items():
total = resolve_total_power(ts_metrics)
if total is None:
continue
points.append({
"timestamp": ts,
"metric_name": "Total_Power",
"value": total,
})
points.sort(key=lambda p: p["timestamp"])
else:
for m in metrics:
ts = m.get("timestamp") or m.get("time")
if not ts or m.get("value") is None:
continue
points.append({
"timestamp": ts,
"metric_name": m.get("metric_name", ""),
"value": m["value"],
})
trend_data[str(host.id)] = {
"host_id": host.id,
"host_name": host.name,
@@ -379,24 +475,10 @@ async def history(
current_user: UserModel = Depends(get_current_user),
):
"""Query historical metrics for a host from InfluxDB."""
# Parse datetime with timezone handling
def parse_dt(dt_str: Optional[str]) -> Optional[datetime]:
if not dt_str:
return None
dt_str = dt_str.replace('Z', '+00:00')
try:
dt = datetime.fromisoformat(dt_str)
if dt.tzinfo is not None:
return dt.astimezone(timezone.utc).replace(tzinfo=None)
return dt
except ValueError:
clean_str = dt_str.split('+')[0].split('-')[0][:19] if '+' in dt_str or dt_str.count('-') > 2 else dt_str[:19]
return datetime.fromisoformat(clean_str)
start_dt = parse_dt(start) or (datetime.now() - timedelta(hours=1))
end_dt = parse_dt(end)
_check_metric_type(metric_type)
start_dt, end_dt = _parse_time_range(start, end)
data = influxdb_module.query_metrics(host_id, metric_type, start_dt, end_dt)
data = await influxdb_module.aquery_metrics(host_id, metric_type, start_dt, end_dt)
return {"host_id": host_id, "metric_type": metric_type, "data": data}
@@ -410,7 +492,7 @@ async def realtime(
current_user: UserModel = Depends(get_current_user)
):
"""Return the latest metrics for a host, grouped by type."""
grouped = influxdb_module.query_latest_grouped(host_id)
grouped = await influxdb_module.aquery_latest_grouped(host_id)
return {"host_id": host_id, "metrics": grouped}
@@ -425,16 +507,15 @@ async def component_presence(
):
"""Get component presence status for a host from raw IPMI data.
Reads /tmp/ipmi_raw.txt for parsing component status.
Reads /tmp/ipmi_raw_{host_id}.txt (B10) — the per-host file the
collector actually writes, same as the /grouped endpoint.
"""
raw_file = PathLib("/tmp/ipmi_raw.txt")
raw_file = PathLib(f"/tmp/ipmi_raw_{host_id}.txt")
if not raw_file.exists():
return {
"host_id": host_id,
"error": "Raw IPMI data file not found",
"components": None,
}
raise HTTPException(
404, f"Raw IPMI data file not found for host {host_id} — no collection yet?"
)
try:
raw_data = raw_file.read_text(encoding="utf-8")
@@ -454,11 +535,7 @@ async def component_presence(
}
except Exception as exc:
logger.error("Failed to parse component presence for host %s: %s", host_id, exc)
return {
"host_id": host_id,
"error": str(exc),
"components": None,
}
raise HTTPException(500, f"Failed to parse component presence: {exc}")
@router.get("/components/{host_id}/grouped")
+82 -29
View File
@@ -8,10 +8,12 @@ from pydantic import BaseModel, Field
from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app.crypto import decrypt_value, encrypt_value
from app.database import get_db
from app.models import SystemSetting
from app.routers.auth import get_current_user
from app.models import User as UserModel
from app.ssrf import validate_smtp_host, validate_webhook_url
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/settings", tags=["settings"])
@@ -28,18 +30,32 @@ DEFAULT_SETTINGS = {
"smtp_use_tls": "true",
}
# Settings whose values are secrets: encrypted at rest, never returned
# in full through the API.
SENSITIVE_SETTINGS = {"smtp_password", "feishu_webhook_url"}
def _mask_webhook_url(url: str) -> str:
"""Mask a webhook URL, revealing only the last 4 characters."""
if not url:
return ""
tail = url[-4:] if len(url) >= 4 else url
return f"****{tail}"
# ---------------------------------------------------------------------------
# Schemas
# ---------------------------------------------------------------------------
class SettingsResponse(BaseModel):
feishu_webhook_url: str = ""
# The full Feishu webhook URL is never returned; only a masked form.
feishu_webhook_masked: str = ""
feishu_keyword: str = ""
smtp_host: str = ""
smtp_port: str = "587"
smtp_user: str = ""
smtp_password: str = ""
# The SMTP password is never returned; only whether one is set.
smtp_password_set: bool = False
smtp_from: str = ""
smtp_use_tls: bool = True
@@ -68,18 +84,23 @@ class TestFeishuRequest(BaseModel):
# ---------------------------------------------------------------------------
async def get_setting(db: AsyncSession, key: str) -> str:
"""Get a setting value by key."""
"""Get a setting value by key (sensitive values are decrypted on read)."""
result = await db.execute(
select(SystemSetting).where(SystemSetting.key == key)
)
setting = result.scalar_one_or_none()
if setting:
return setting.value
value = setting.value or ""
if key in SENSITIVE_SETTINGS:
return decrypt_value(value) or ""
return value
return DEFAULT_SETTINGS.get(key, "")
async def set_setting(db: AsyncSession, key: str, value: str, description: str = ""):
"""Set a setting value."""
"""Set a setting value (sensitive values are encrypted before storage)."""
if key in SENSITIVE_SETTINGS:
value = encrypt_value(value) or ""
result = await db.execute(
select(SystemSetting).where(SystemSetting.key == key)
)
@@ -98,6 +119,24 @@ async def set_setting(db: AsyncSession, key: str, value: str, description: str =
await db.flush()
async def _build_response(db: AsyncSession) -> SettingsResponse:
"""Build the redacted settings response."""
values = {}
for key in DEFAULT_SETTINGS.keys():
values[key] = await get_setting(db, key)
return SettingsResponse(
feishu_webhook_masked=_mask_webhook_url(values.get("feishu_webhook_url", "")),
feishu_keyword=values.get("feishu_keyword", ""),
smtp_host=values.get("smtp_host", ""),
smtp_port=values.get("smtp_port", "587"),
smtp_user=values.get("smtp_user", ""),
smtp_password_set=bool(values.get("smtp_password", "")),
smtp_from=values.get("smtp_from", ""),
smtp_use_tls=values.get("smtp_use_tls", "true").lower() == "true",
)
# ---------------------------------------------------------------------------
# API Endpoints
# ---------------------------------------------------------------------------
@@ -107,15 +146,8 @@ async def get_settings(
db: AsyncSession = Depends(get_db),
current_user: UserModel = Depends(get_current_user),
):
"""Get all system settings."""
settings = {}
for key in DEFAULT_SETTINGS.keys():
settings[key] = await get_setting(db, key)
# Convert smtp_use_tls to boolean
settings["smtp_use_tls"] = settings.get("smtp_use_tls", "true").lower() == "true"
return SettingsResponse(**settings)
"""Get all system settings (secrets are redacted)."""
return await _build_response(db)
@router.put("", response_model=SettingsResponse)
@@ -124,28 +156,39 @@ async def update_settings(
db: AsyncSession = Depends(get_db),
current_user: UserModel = Depends(get_current_user),
):
"""Update system settings."""
"""Update system settings.
For the secret settings (``smtp_password``, ``feishu_webhook_url``) an
empty/absent value means "leave unchanged".
"""
update_data = body.model_dump(exclude_unset=True)
for key, value in update_data.items():
if value is not None:
# Convert boolean to string for storage
if isinstance(value, bool):
str_value = "true" if value else "false"
else:
str_value = str(value)
await set_setting(db, key, str_value)
if value is None:
continue
if key in SENSITIVE_SETTINGS and value == "":
continue
if key == "feishu_webhook_url":
try:
validate_webhook_url(str(value))
except ValueError as exc:
raise HTTPException(400, str(exc))
if key == "smtp_host" and value:
try:
validate_smtp_host(str(value))
except ValueError as exc:
raise HTTPException(400, str(exc))
# Convert boolean to string for storage
if isinstance(value, bool):
str_value = "true" if value else "false"
else:
str_value = str(value)
await set_setting(db, key, str_value)
await db.commit()
# Return updated settings
settings = {}
for key in DEFAULT_SETTINGS.keys():
settings[key] = await get_setting(db, key)
settings["smtp_use_tls"] = settings.get("smtp_use_tls", "true").lower() == "true"
return SettingsResponse(**settings)
return await _build_response(db)
@router.post("/test-email")
@@ -169,6 +212,11 @@ async def test_email(
if not smtp_host or not smtp_user or not smtp_password:
raise HTTPException(400, "SMTP配置不完整,请先配置SMTP服务器信息")
try:
validate_smtp_host(smtp_host)
except ValueError as exc:
raise HTTPException(400, str(exc))
try:
msg = MIMEMultipart()
msg['From'] = smtp_from or smtp_user
@@ -209,6 +257,11 @@ async def test_feishu(
if not webhook_url:
raise HTTPException(400, "飞书Webhook URL未配置")
try:
validate_webhook_url(webhook_url)
except ValueError as exc:
raise HTTPException(400, str(exc))
# Build message with keyword if configured
if keyword:
content = f"[IPMI监控测试] {keyword}\n这是一封来自IPMI监控系统的测试消息。\n如果您收到此消息,说明飞书Webhook配置正确。"
+164 -78
View File
@@ -12,8 +12,9 @@ from sqlalchemy import select
from sqlalchemy.ext.asyncio import AsyncSession
from app import influxdb as influxdb_module
from app.crypto import decrypt_value
from app.database import async_session
from app.models import AlertHistory, AlertRule, Host
from app.models import AlertHistory, AlertRule, Host, SystemSetting
logger = logging.getLogger(__name__)
@@ -39,6 +40,30 @@ _alert_active: dict[tuple[int, int], bool] = {}
NOTIFY_COOLDOWN_SECONDS = 3 * 60 * 60 # 10800 seconds
async def get_settings_batch(session: AsyncSession, keys: list[str]) -> dict[str, str]:
"""Fetch several SystemSetting values with a single query (O3).
Replaces the old pattern of one ``select`` per key (5 round-trips for
SMTP alone, repeated for alert and recovery paths). Missing keys map
to ``""``.
"""
if not keys:
return {}
result = await session.execute(
select(SystemSetting).where(SystemSetting.key.in_(keys))
)
settings_dict = {s.key: s.value for s in result.scalars().all()}
return {k: settings_dict.get(k, "") for k in keys}
def _smtp_use_tls_enabled(settings_dict: dict[str, str]) -> bool:
"""Interpret the stored ``smtp_use_tls`` flag (default True)."""
raw = settings_dict.get("smtp_use_tls", "true")
if isinstance(raw, bool):
return raw
return str(raw).strip().lower() in ("true", "1", "yes", "on")
# ---------------------------------------------------------------------------
# Rule evaluation
# ---------------------------------------------------------------------------
@@ -118,17 +143,18 @@ async def _create_alert_history(
if rule.notify_email:
try:
from app.services.email_notify import send_alert_email
# 从数据库读取SMTP配置
# 从数据库批量读取SMTP配置(一次查询取回全部 key,O3)
async with async_session() as session:
from sqlalchemy import select
from app.models import SystemSetting
settings_dict = {}
for key in ["smtp_host", "smtp_port", "smtp_user", "smtp_password", "smtp_from"]:
result = await session.execute(
select(SystemSetting).where(SystemSetting.key == key)
)
setting = result.scalar_one_or_none()
settings_dict[key] = setting.value if setting else ""
settings_dict = await get_settings_batch(
session,
["smtp_host", "smtp_port", "smtp_user", "smtp_password",
"smtp_from", "smtp_use_tls"],
)
# SMTP 口令入库时已 Fernet 加密,解密后再使用(历史明文原样返回)。
settings_dict["smtp_password"] = decrypt_value(
settings_dict.get("smtp_password", "")
) or ""
addresses = [e.strip() for e in rule.email_addresses.split(",") if e.strip()]
if addresses and settings_dict["smtp_host"] and settings_dict["smtp_user"]:
@@ -137,6 +163,7 @@ async def _create_alert_history(
subject=f"[IPMI Alert] {host.name} – {rule.metric_name}",
alert_info=alert_info,
smtp_config=settings_dict,
smtp_use_tls=_smtp_use_tls_enabled(settings_dict),
)
else:
logger.warning("Email SMTP not configured in settings, skipping alert for rule %d", rule.id)
@@ -146,18 +173,23 @@ async def _create_alert_history(
if rule.notify_feishu:
try:
from app.services.feishu_notify import send_feishu_alert
# 从数据库读取用户配置的飞书 webhook URL
# 从数据库批量读取飞书配置(B25:keyword 也要读)
async with async_session() as session:
from sqlalchemy import select
from app.models import SystemSetting
result = await session.execute(
select(SystemSetting).where(SystemSetting.key == "feishu_webhook_url")
feishu_settings = await get_settings_batch(
session, ["feishu_webhook_url", "feishu_keyword"]
)
setting = result.scalar_one_or_none()
url = setting.value if setting else ""
# 飞书 webhook URL 入库时已 Fernet 加密,解密后再使用。
url = decrypt_value(
feishu_settings.get("feishu_webhook_url", "")
) or ""
if url:
await send_feishu_alert(url, alert_info)
feishu_info = dict(alert_info)
keyword = feishu_settings.get("feishu_keyword", "")
if keyword:
# 飞书关键词安全校验要求关键词出现在卡片文本中
feishu_info["message"] = f"{feishu_info.get('message', '')}\n{keyword}"
await send_feishu_alert(url, feishu_info)
else:
logger.warning("Feishu webhook URL not configured in settings, skipping alert for rule %d", rule.id)
except Exception:
@@ -196,15 +228,16 @@ async def _create_recovery_notification(
try:
from app.services.email_notify import send_recovery_email
async with async_session() as session:
from sqlalchemy import select
from app.models import SystemSetting
settings_dict = {}
for key in ["smtp_host", "smtp_port", "smtp_user", "smtp_password", "smtp_from"]:
result = await session.execute(
select(SystemSetting).where(SystemSetting.key == key)
)
setting = result.scalar_one_or_none()
settings_dict[key] = setting.value if setting else ""
settings_dict = await get_settings_batch(
session,
["smtp_host", "smtp_port", "smtp_user", "smtp_password",
"smtp_from", "smtp_use_tls"],
)
# SMTP 口令入库时已 Fernet 加密,解密后再使用(历史明文原样返回)。
settings_dict["smtp_password"] = decrypt_value(
settings_dict.get("smtp_password", "")
) or ""
addresses = [e.strip() for e in rule.email_addresses.split(",") if e.strip()]
if addresses and settings_dict["smtp_host"] and settings_dict["smtp_user"]:
@@ -213,6 +246,7 @@ async def _create_recovery_notification(
subject=f"✅ IPMI Recovery: {host.name} – {rule.metric_name}",
recovery_info=recovery_info,
smtp_config=settings_dict,
smtp_use_tls=_smtp_use_tls_enabled(settings_dict),
)
else:
logger.warning("Email SMTP not configured, skipping recovery for rule %d", rule.id)
@@ -223,16 +257,20 @@ async def _create_recovery_notification(
try:
from app.services.feishu_notify import send_feishu_recovery
async with async_session() as session:
from sqlalchemy import select
from app.models import SystemSetting
result = await session.execute(
select(SystemSetting).where(SystemSetting.key == "feishu_webhook_url")
feishu_settings = await get_settings_batch(
session, ["feishu_webhook_url", "feishu_keyword"]
)
setting = result.scalar_one_or_none()
url = setting.value if setting else ""
# 飞书 webhook URL 入库时已 Fernet 加密,解密后再使用。
url = decrypt_value(
feishu_settings.get("feishu_webhook_url", "")
) or ""
if url:
await send_feishu_recovery(url, recovery_info)
feishu_info = dict(recovery_info)
keyword = feishu_settings.get("feishu_keyword", "")
if keyword:
feishu_info["message"] = f"{feishu_info.get('message', '')}\n{keyword}"
await send_feishu_recovery(url, feishu_info)
else:
logger.warning("Feishu webhook URL not configured, skipping recovery for rule %d", rule.id)
except Exception:
@@ -253,11 +291,22 @@ async def check_alerts() -> None:
if not rules:
return
# B16: drop in-memory state for rules that no longer exist or are
# disabled, so deleted/disabled rules stop accumulating state.
live_rule_ids = {r.id for r in rules}
for state in (_first_breach, _last_notify, _alert_active):
for key in [k for k in state if k[0] not in live_rule_ids]:
del state[key]
# Resolve hosts for rules (either specific host or all hosts in a group)
for rule in rules:
host_ids: list[int] = []
if rule.host_id:
host_ids = [rule.host_id]
# B17: a host-bound rule only applies while the host exists
# and is enabled — same as the group/all-hosts branches.
host_obj = await session.get(Host, rule.host_id)
if host_obj is not None and host_obj.enabled:
host_ids = [rule.host_id]
elif rule.group_id:
stmt_h = select(Host).where(Host.group_id == rule.group_id, Host.enabled.is_(True))
res_h = await session.execute(stmt_h)
@@ -269,49 +318,86 @@ async def check_alerts() -> None:
host_ids = [h.id for h in res_h.scalars().all()]
for hid in host_ids:
latest = influxdb_module.query_latest(hid, rule.metric_type)
for point in latest:
if point.get("metric_name") != rule.metric_name:
continue
metric_value = point["value"]
# B2: one bad rule/host must never abort the whole round.
try:
await _check_rule_for_host(session, rule, hid)
except Exception:
logger.exception(
"Alert check failed for rule %d host %d; continuing",
rule.id, hid,
)
breached = evaluate_rule(rule, metric_value)
key = (rule.id, hid)
if breached:
now = datetime.now()
async def _check_rule_for_host(
session: AsyncSession, rule: AlertRule, hid: int
) -> None:
"""Evaluate one rule against one host's latest metrics (B2 isolation)."""
latest = await influxdb_module.aquery_latest(hid, rule.metric_type)
for point in latest:
if point.get("metric_name") != rule.metric_name:
continue
try:
raw_value = point.get("value")
if raw_value is None:
continue
metric_value = float(raw_value)
except (TypeError, ValueError):
logger.warning(
"Non-numeric metric value %r for rule %d host %d; skipping",
point.get("value"), rule.id, hid,
)
continue
if key not in _first_breach:
# First time breaching — start the duration timer
_first_breach[key] = now
try:
breached = evaluate_rule(rule, metric_value)
except Exception:
logger.exception(
"Rule evaluation failed for rule %d host %d; skipping",
rule.id, hid,
)
continue
elapsed = (now - _first_breach[key]).total_seconds()
if elapsed >= rule.duration:
# Duration satisfied — check if we already have an active alert
if not _alert_active.get(key):
# No active alert yet → fire alert notification
host_obj = await session.get(Host, hid)
if host_obj:
await _create_alert_history(rule, host_obj, metric_value)
_alert_active[key] = True
_last_notify[key] = now
else:
# Alert already active — check cooldown for repeat notification
last_notify_time = _last_notify.get(key)
if last_notify_time and (now - last_notify_time).total_seconds() >= NOTIFY_COOLDOWN_SECONDS:
host_obj = await session.get(Host, hid)
if host_obj:
await _create_alert_history(rule, host_obj, metric_value)
_last_notify[key] = now
# Do NOT reset _first_breach — keep the original breach time
else:
# No longer breaching
if key in _first_breach:
# Was breaching before → now recovered, send recovery notification
host_obj = await session.get(Host, hid)
if host_obj:
await _create_recovery_notification(rule, host_obj, metric_value)
# Clear all state for this rule+host
_first_breach.pop(key, None)
_alert_active.pop(key, None)
_last_notify.pop(key, None)
key = (rule.id, hid)
if breached:
now = datetime.now()
if key not in _first_breach:
# First time breaching — start the duration timer
_first_breach[key] = now
elapsed = (now - _first_breach[key]).total_seconds()
if elapsed >= rule.duration:
# Duration satisfied — check if we already have an active alert
if not _alert_active.get(key):
# No active alert yet → fire alert notification
host_obj = await session.get(Host, hid)
if host_obj:
# B19: only mark the alert active once the history
# record was actually created (host still exists).
await _create_alert_history(rule, host_obj, metric_value)
_alert_active[key] = True
_last_notify[key] = now
else:
# Alert already active — check cooldown for repeat notification
last_notify_time = _last_notify.get(key)
if last_notify_time and (now - last_notify_time).total_seconds() >= NOTIFY_COOLDOWN_SECONDS:
host_obj = await session.get(Host, hid)
if host_obj:
await _create_alert_history(rule, host_obj, metric_value)
_last_notify[key] = now
# Do NOT reset _first_breach — keep the original breach time
else:
# No longer breaching
if key in _first_breach:
# B1: only send a recovery notification if an alert was
# actually fired. A transient breach that never satisfied
# `duration` just clears the timer silently.
if _alert_active.get(key):
host_obj = await session.get(Host, hid)
if host_obj:
await _create_recovery_notification(rule, host_obj, metric_value)
# Clear all state for this rule+host
_first_breach.pop(key, None)
_alert_active.pop(key, None)
_last_notify.pop(key, None)
+165 -69
View File
@@ -1,16 +1,26 @@
"""SMTP email notification service."""
import asyncio
import collections
import logging
import re
import smtplib
import ssl
from email.mime.multipart import MIMEMultipart
from email.mime.text import MIMEText
from typing import Any
from app.config import settings
from app.crypto import decrypt_value
logger = logging.getLogger(__name__)
# SMTP connect/read timeout for the blocking smtplib calls (B26).
SMTP_TIMEOUT_SECONDS = 15
# Matches a leftover "{{ placeholder }}" after rendering (B27).
_LEFTOVER_PLACEHOLDER_RE = re.compile(r"\{\{[^}]*\}\}")
# ---------------------------------------------------------------------------
# HTML email template
# ---------------------------------------------------------------------------
@@ -136,38 +146,86 @@ IPMI 恢复通知
"""
async def send_alert_email(
to_addresses: list[str],
subject: str,
alert_info: dict[str, Any],
smtp_config: dict[str, str] = None,
) -> None:
"""Send an alert email to *to_addresses* using the configured SMTP server."""
def _parse_smtp_port(raw: Any) -> int:
"""Parse and validate an SMTP port (B26).
# 优先使用传入的 smtp_config,否则回退到环境变量配置
Raises ``ValueError`` for non-numeric values or ports outside 1-65535.
Callers must do this inside a ``try`` so a bad stored value is a
logged warning, not a 500.
"""
port = int(str(raw).strip())
if not 1 <= port <= 65535:
raise ValueError(f"SMTP port out of range: {port}")
return port
def _parse_smtp_use_tls(raw: Any, default: bool = True) -> bool:
"""Interpret an smtp_use_tls flag from settings or a caller argument."""
if isinstance(raw, bool):
return raw
if raw is None:
return default
return str(raw).strip().lower() in ("true", "1", "yes", "on")
def _render_plain(template: str, info: dict[str, Any]) -> str:
"""Render a ``{key}`` plain-text template, missing keys become "" (B27)."""
safe = collections.defaultdict(str, {k: str(v) for k, v in info.items()})
return template.format_map(safe)
def _render_html(template: str, info: dict[str, Any]) -> str:
"""Render a ``{{ key }}`` HTML template and strip leftovers (B27)."""
html = template
for key, val in info.items():
html = html.replace("{{ " + key + " }}", str(val))
# Any placeholder not supplied by the caller is removed rather than
# leaking "{{ ... }}" into the sent email.
return _LEFTOVER_PLACEHOLDER_RE.sub("", html)
def _smtp_settings(smtp_config: dict[str, str] | None) -> tuple[str, int, str, str, str, bool]:
"""Resolve SMTP connection settings from a config dict or env settings.
Returns ``(host, port, user, password, from_addr, use_tls)``.
Raises ``ValueError`` when the configured port is invalid (B26).
"""
if smtp_config:
smtp_host = smtp_config.get("smtp_host", "")
smtp_port = int(smtp_config.get("smtp_port", "587"))
smtp_port = _parse_smtp_port(smtp_config.get("smtp_port", "587"))
smtp_user = smtp_config.get("smtp_user", "")
smtp_password = smtp_config.get("smtp_password", "")
# 口令入库时已 Fernet 加密,解密后再使用;历史明文原样返回。
smtp_password = decrypt_value(smtp_config.get("smtp_password", "")) or ""
smtp_from = smtp_config.get("smtp_from", "")
smtp_use_tls = _parse_smtp_use_tls(smtp_config.get("smtp_use_tls", True))
else:
smtp_host = settings.SMTP_HOST
smtp_port = settings.SMTP_PORT
smtp_port = _parse_smtp_port(settings.SMTP_PORT)
smtp_user = settings.SMTP_USER
smtp_password = settings.SMTP_PASSWORD
smtp_from = settings.SMTP_FROM
smtp_use_tls = True
return smtp_host, smtp_port, smtp_user, smtp_password, smtp_from, smtp_use_tls
if not smtp_host:
logger.warning("SMTP not configured – skipping email notification")
return
# Render HTML with simple string replacement
html = _ALERT_EMAIL_HTML
plain = _PLAIN_TEMPLATE.format(**alert_info)
for key, val in alert_info.items():
html = html.replace("{{ " + key + " }}", str(val))
def _send_mail(
to_addresses: list[str],
subject: str,
plain: str,
html: str,
smtp_host: str,
smtp_port: int,
smtp_user: str,
smtp_password: str,
smtp_from: str,
smtp_use_tls: bool,
) -> None:
"""Blocking SMTP send (runs inside ``asyncio.to_thread``).
Uses a ``with`` statement so the connection is always released (B26),
and only calls ``starttls()`` when TLS is enabled *and* the port is
not 465 (B24), with an explicit SSL context.
"""
msg = MIMEMultipart("alternative")
msg["Subject"] = subject
msg["From"] = smtp_from or smtp_user
@@ -175,22 +233,58 @@ async def send_alert_email(
msg.attach(MIMEText(plain, "plain", "utf-8"))
msg.attach(MIMEText(html, "html", "utf-8"))
def _send() -> None:
if smtp_port == 465:
smtp_cls = smtplib.SMTP_SSL
else:
smtp_cls = smtplib.SMTP
context = ssl.create_default_context()
if smtp_port == 465:
smtp_cls = smtplib.SMTP_SSL
server_kwargs: dict[str, Any] = {"context": context}
else:
smtp_cls = smtplib.SMTP
server_kwargs = {}
with smtp_cls(smtp_host, smtp_port, timeout=15) as server:
if smtp_port != 465:
server.starttls()
if smtp_user and smtp_password:
server.login(smtp_user, smtp_password)
server.sendmail(msg["From"], to_addresses, msg.as_string())
logger.info("Alert email sent to %s", to_addresses)
with smtp_cls(smtp_host, smtp_port, timeout=SMTP_TIMEOUT_SECONDS, **server_kwargs) as server:
if smtp_use_tls and smtp_port != 465:
server.starttls(context=context)
if smtp_user and smtp_password:
server.login(smtp_user, smtp_password)
server.sendmail(msg["From"], to_addresses, msg.as_string())
logger.info("Email sent to %s", to_addresses)
async def send_alert_email(
to_addresses: list[str],
subject: str,
alert_info: dict[str, Any],
smtp_config: dict[str, str] = None,
smtp_use_tls: bool | None = None,
) -> None:
"""Send an alert email to *to_addresses* using the configured SMTP server.
``smtp_use_tls`` (B24) may be passed explicitly or carried inside
``smtp_config``; an explicit argument wins.
"""
try:
smtp_host, smtp_port, smtp_user, smtp_password, smtp_from, cfg_use_tls = (
_smtp_settings(smtp_config)
)
except (ValueError, TypeError) as exc:
logger.warning("Invalid SMTP configuration, skipping alert email: %s", exc)
return
use_tls = cfg_use_tls if smtp_use_tls is None else smtp_use_tls
if not smtp_host:
logger.warning("SMTP not configured – skipping email notification")
return
# Render templates with missing-field-safe rendering (B27)
html = _render_html(_ALERT_EMAIL_HTML, alert_info)
plain = _render_plain(_PLAIN_TEMPLATE, alert_info)
# Run blocking SMTP call in a thread so it doesn't block the event loop
await asyncio.to_thread(_send)
await asyncio.to_thread(
_send_mail,
to_addresses, subject, plain, html,
smtp_host, smtp_port, smtp_user, smtp_password, smtp_from, use_tls,
)
async def send_recovery_email(
@@ -198,51 +292,53 @@ async def send_recovery_email(
subject: str,
recovery_info: dict[str, Any],
smtp_config: dict[str, str] = None,
smtp_use_tls: bool | None = None,
) -> None:
"""Send a recovery notification email to *to_addresses*."""
if smtp_config:
smtp_host = smtp_config.get("smtp_host", "")
smtp_port = int(smtp_config.get("smtp_port", "587"))
smtp_user = smtp_config.get("smtp_user", "")
smtp_password = smtp_config.get("smtp_password", "")
smtp_from = smtp_config.get("smtp_from", "")
else:
smtp_host = settings.SMTP_HOST
smtp_port = settings.SMTP_PORT
smtp_user = settings.SMTP_USER
smtp_password = settings.SMTP_PASSWORD
smtp_from = settings.SMTP_FROM
try:
smtp_host, smtp_port, smtp_user, smtp_password, smtp_from, cfg_use_tls = (
_smtp_settings(smtp_config)
)
except (ValueError, TypeError) as exc:
logger.warning("Invalid SMTP configuration, skipping recovery email: %s", exc)
return
use_tls = cfg_use_tls if smtp_use_tls is None else smtp_use_tls
if not smtp_host:
logger.warning("SMTP not configured – skipping recovery email")
return
# Render HTML with simple string replacement
html = _RECOVERY_EMAIL_HTML
plain = _RECOVERY_PLAIN_TEMPLATE.format(**recovery_info)
for key, val in recovery_info.items():
html = html.replace("{{ " + key + " }}", str(val))
html = _render_html(_RECOVERY_EMAIL_HTML, recovery_info)
plain = _render_plain(_RECOVERY_PLAIN_TEMPLATE, recovery_info)
msg = MIMEMultipart("alternative")
msg["Subject"] = subject
msg["From"] = smtp_from or smtp_user
msg["To"] = ", ".join(to_addresses)
msg.attach(MIMEText(plain, "plain", "utf-8"))
msg.attach(MIMEText(html, "html", "utf-8"))
await asyncio.to_thread(
_send_mail,
to_addresses, subject, plain, html,
smtp_host, smtp_port, smtp_user, smtp_password, smtp_from, use_tls,
)
def _send() -> None:
if smtp_port == 465:
smtp_cls = smtplib.SMTP_SSL
else:
smtp_cls = smtplib.SMTP
with smtp_cls(smtp_host, smtp_port, timeout=15) as server:
if smtp_port != 465:
server.starttls()
if smtp_user and smtp_password:
server.login(smtp_user, smtp_password)
server.sendmail(msg["From"], to_addresses, msg.as_string())
logger.info("Recovery email sent to %s", to_addresses)
async def send_test_email(
to_address: str,
smtp_config: dict[str, str] | None = None,
) -> None:
"""Send a test email using the same safe path as alert mail (B26).
await asyncio.to_thread(_send)
Intended for the settings test-email endpoint: the port is parsed
inside ``try`` and range-checked (1-65535), the blocking SMTP call
runs in a thread with a timeout, and the connection is released via
a ``with`` statement. Raises ``ValueError`` for a bad port and lets
SMTP errors propagate to the caller so the endpoint can report them.
"""
smtp_host, smtp_port, smtp_user, smtp_password, smtp_from, use_tls = (
_smtp_settings(smtp_config)
)
if not smtp_host:
raise ValueError("SMTP host is not configured")
plain = "This is a test email from IPMI Monitor. SMTP configuration works."
html = f"<p>{plain}</p>"
await asyncio.to_thread(
_send_mail,
[to_address], "[IPMI Monitor] Test Email", plain, html,
smtp_host, smtp_port, smtp_user, smtp_password, smtp_from, use_tls,
)
+19
View File
@@ -6,12 +6,25 @@ from typing import Any
import httpx
from app.config import settings
from app.crypto import decrypt_value
from app.ssrf import validate_webhook_url
logger = logging.getLogger(__name__)
def _checked_webhook_url(webhook_url: str) -> str | None:
"""Validate the webhook URL; log and return None when it is not allowed."""
try:
return validate_webhook_url(webhook_url)
except ValueError as exc:
logger.warning("Refusing to send Feishu alert to disallowed URL: %s", exc)
return None
async def send_feishu_alert(webhook_url: str, alert_info: dict[str, Any]) -> None:
"""Send an interactive card message to a Feishu group via custom robot webhook."""
# 传入的 URL 可能是入库的 Fernet 密文,解密后再使用;历史明文原样返回。
webhook_url = decrypt_value(webhook_url) or ""
host_name = alert_info.get("host_name", "N/A")
host_ip = alert_info.get("host_ip", "N/A")
@@ -67,6 +80,9 @@ async def send_feishu_alert(webhook_url: str, alert_info: dict[str, Any]) -> Non
}
try:
webhook_url = _checked_webhook_url(webhook_url)
if webhook_url is None:
return
async with httpx.AsyncClient(timeout=10) as client:
resp = await client.post(webhook_url, json=card_body)
resp.raise_for_status()
@@ -136,6 +152,9 @@ async def send_feishu_recovery(webhook_url: str, recovery_info: dict[str, Any])
}
try:
webhook_url = _checked_webhook_url(webhook_url)
if webhook_url is None:
return
async with httpx.AsyncClient(timeout=10) as client:
resp = await client.post(webhook_url, json=card_body)
resp.raise_for_status()
+268 -126
View File
@@ -5,7 +5,9 @@ Uses *ipmitool* over the LAN interface to gather sensor data from remote BMCs.
import asyncio
import logging
import os
import re
import tempfile
from datetime import datetime
from pathlib import Path
from typing import Any, Optional
@@ -15,6 +17,16 @@ from app.config import settings
logger = logging.getLogger(__name__)
# BMC password decryption (B20 companion fix). ``app.crypto`` is created by
# the parallel security fix; until it exists, fall back to the value as-is
# (historical plaintext compatibility — decrypt_value itself is specified
# to return the original value when decryption fails).
try:
from app.crypto import decrypt_value
except ImportError: # pragma: no cover - crypto module not yet present
def decrypt_value(value: str) -> str:
return value
# ---------------------------------------------------------------------------
# Output parsing helpers
# ---------------------------------------------------------------------------
@@ -31,49 +43,70 @@ _SDR_RE = re.compile(
r"^(?P<name>[^|]+?)\s*\|\s*(?P<value>.+?)\s*\|\s*(?P<status>\w+)"
)
# Numeric readings may be negative and/or use thousands separators
# (B5): "-5.0 degrees C", "1,234.5 RPM".
_NUM = r"-?[0-9][0-9,]*(?:\.[0-9]+)?"
# Common unit patterns
_TEMP_RE = re.compile(r"(?P<val>[0-9.]+)\s*degrees?\s*C", re.IGNORECASE)
_FAN_RE = re.compile(r"(?P<val>[0-9.]+)\s*RPM", re.IGNORECASE)
_VOLT_RE = re.compile(r"(?P<val>[0-9.]+)\s*Volts?", re.IGNORECASE)
_WATT_RE = re.compile(r"(?P<val>[0-9.]+)\s*Watts?", re.IGNORECASE)
_TEMP_RE = re.compile(r"(?P<val>" + _NUM + r")\s*degrees?\s*C", re.IGNORECASE)
_FAN_RE = re.compile(r"(?P<val>" + _NUM + r")\s*RPM", re.IGNORECASE)
_VOLT_RE = re.compile(r"(?P<val>" + _NUM + r")\s*Volts?", re.IGNORECASE)
_WATT_RE = re.compile(r"(?P<val>" + _NUM + r")\s*Watts?", re.IGNORECASE)
# SDR status tokens (B7). Beyond the original ok/ns/nc/nr/nd/na this covers
# disabled sensors and critical (cr) readings seen on real BMCs.
_STATUS_TOKENS = {
"ok", "ns", "nc", "nr", "nd", "na",
"disabled", "cr", "lnr", "lcr", "unc", "unr",
}
def _classify_metric(name: str, value_str: str) -> Optional[tuple[str, float]]:
"""Return (metric_type, numeric_value) or None if unrecognised."""
def _to_float(val_str: str) -> float:
"""Parse a numeric reading, tolerating thousands separators."""
return float(val_str.replace(",", "").strip())
def _classify_metric(
name: str, value_str: str, status: str = "ok"
) -> Optional[tuple[str, float]]:
"""Return (metric_type, numeric_value) or None if unrecognised.
``status`` is the SDR status segment (B6): status-class sensors map
``ok`` → 1.0 and anything else → 0.0 instead of always reporting 1.0.
"""
status_ok = (status or "").strip().lower() == "ok"
# Temperature
m = _TEMP_RE.search(value_str)
if m:
return ("temperature", float(m.group("val")))
return ("temperature", _to_float(m.group("val")))
# Fan speed
m = _FAN_RE.search(value_str)
if m:
return ("fan", float(m.group("val")))
return ("fan", _to_float(m.group("val")))
# Voltage
m = _VOLT_RE.search(value_str)
if m:
return ("voltage", float(m.group("val")))
return ("voltage", _to_float(m.group("val")))
# Power (Watts)
m = _WATT_RE.search(value_str)
if m:
return ("power", float(m.group("val")))
return ("power", _to_float(m.group("val")))
# Heuristic: name-based fallback for status / discrete sensors
name_lower = name.lower()
if "power" in name_lower and ("supply" in name_lower or "unit" in name_lower):
# Hex or discrete value — map 0x00 → 0 (off), anything else → 1 (on)
try:
val = float(int(value_str.strip(), 0))
except ValueError:
val = 0.0
return ("power", val)
# Discrete power-supply sensor (e.g. "0x00", "Presence detected"):
# this is presence/status information, not a wattage reading, so it
# belongs to the "status" class (B6) and must not pollute power stats.
return ("status", 1.0 if status_ok else 0.0)
if "status" in name_lower:
# Discrete — just store 1 for ok / 0 for anything else
return ("status", 1.0)
# Discrete — 1 for ok, 0 for anything else
return ("status", 1.0 if status_ok else 0.0)
return None
@@ -85,6 +118,11 @@ def parse_sdr_output(output: str) -> list[dict[str, Any]]:
Legacy: CPU1 Temp | 45 degrees C | ok
Newer: Temp | 01h | ok | 3.1 | 37 degrees C
Parsing is driven by segment count (B7): 5 segments = new format
(``name|id|status|entity|value``), 3 segments = legacy format
(``name|value|status``). Anything else falls back to locating a known
status token instead of misreading the sensor id as the value.
Returns a list of dicts:
{name, value_str, status, metric_type, numeric_value}
"""
@@ -93,36 +131,38 @@ def parse_sdr_output(output: str) -> list[dict[str, Any]]:
line = line.strip()
if not line:
continue
parts = [p.strip() for p in line.split('|')]
if len(parts) < 3:
continue
# Find status token (ok, ns, nc, nr, nd, na)
status_idx = None
for i, p in enumerate(parts):
if p in ('ok', 'ns', 'nc', 'nr', 'nd', 'na'):
status_idx = i
break
if status_idx and status_idx < len(parts) - 1:
parts = [p.strip() for p in line.split("|")]
if len(parts) == 5:
# New format: name | id | status | entity | value
name, status, value_str = parts[0], parts[2], parts[4]
elif len(parts) == 3:
# Legacy format: name | value | status
name, value_str, status = parts[0], parts[1], parts[2]
elif len(parts) >= 4:
# Unknown shape — find a known status token; never silently
# treat the sensor id (parts[1]) as the value.
status_idx = None
for i, p in enumerate(parts):
if p.lower() in _STATUS_TOKENS:
status_idx = i
break
if status_idx is None:
continue
name = parts[0]
status = parts[status_idx]
# Take the last part as the actual sensor value
value_str = parts[-1].strip()
value_str = parts[-1]
else:
# Legacy format: name | value | status
name = parts[0]
value_str = parts[1]
status = parts[2]
continue
entry: dict[str, Any] = {
"name": name,
"value_str": value_str,
"status": status,
}
classified = _classify_metric(name, value_str)
classified = _classify_metric(name, value_str, status)
if classified:
metric_type, numeric_value = classified
entry["metric_type"] = metric_type
@@ -142,75 +182,177 @@ def parse_sensor_output(output: str) -> list[dict[str, Any]]:
return parse_sdr_output(output)
# ---------------------------------------------------------------------------
# Component parsing helpers (B8/B9)
#
# component_parser.py drops legacy 3-segment SDR lines (it requires >= 4
# segments) and has no de-duplication / status-based presence for
# drive/fan/PSU. That module is owned by another fix track, so the
# collector normalises the raw output before handing it over and repairs
# the result afterwards instead of modifying it.
# ---------------------------------------------------------------------------
_DRIVE_RE = re.compile(r"^Drive\s+\d+$", re.IGNORECASE)
def _expand_legacy_sdr_lines(raw: str) -> str:
"""Rewrite legacy 3-segment SDR lines into 5-segment form.
``Name | value | status`` becomes ``Name | -- | status | -- | value``
so downstream component parsing (which needs >= 4 segments, with the
status at index 2 and the value last) sees the legacy rows too (B8).
"""
out_lines = []
for line in raw.splitlines():
parts = [p.strip() for p in line.split("|")]
if len(parts) == 3:
out_lines.append(f"{parts[0]} | -- | {parts[2]} | -- | {parts[1]}")
else:
out_lines.append(line)
return "\n".join(out_lines)
def _fixup_components(raw: str, components: dict[str, Any]) -> dict[str, Any]:
"""Repair drive/fan/PSU component results (B9).
- Adds ``Drive N`` sensors the component parser misses (its drive
matcher requires an underscore in the name).
- De-duplicates drive/fan/PSU items by sensor name (``seen`` set).
- Judges PSU / drive presence from the SDR ``status`` segment
(``ok`` = present) instead of the value field, which for 5-segment
rows holds a wattage like ``132 Watts`` and never contains
"presence detected".
"""
status_by_name: dict[str, str] = {}
for entry in parse_sdr_output(raw):
status_by_name.setdefault(entry["name"], entry["status"])
drive_cat = components.setdefault("drive", {"total": 0, "present": 0, "items": []})
existing_drive = {item.get("name") for item in drive_cat.get("items", [])}
for line in raw.splitlines():
parts = [p.strip() for p in line.split("|")]
if not parts:
continue
if _DRIVE_RE.match(parts[0]) and parts[0] not in existing_drive:
status = status_by_name.get(parts[0], "")
drive_cat["items"].append(
{
"name": parts[0],
"present": status.lower() == "ok",
"value": parts[-1] if len(parts) > 1 else "",
}
)
existing_drive.add(parts[0])
for category in ("drive", "fan", "power_supply"):
cat = components.get(category)
if not cat:
continue
seen: set[str] = set()
items: list[dict[str, Any]] = []
for item in cat.get("items", []):
name = item.get("name")
if name in seen:
continue
seen.add(name)
items.append(item)
cat["items"] = items
if category in ("drive", "power_supply"):
for item in items:
status = status_by_name.get(item.get("name", ""))
if status is not None:
item["present"] = status.lower() == "ok"
cat["total"] = len(items)
cat["present"] = sum(1 for i in items if i.get("present"))
return components
# ---------------------------------------------------------------------------
# Collection logic
# ---------------------------------------------------------------------------
async def _run_ipmitool(host: str, port: int, username: str, password: str, interface: str = "lanplus") -> str:
"""Execute ipmitool sdr elist and return raw stdout.
DEPRECATED: Use _run_ipmitool_with_auth instead for automatic auth retry.
This function is kept for backward compatibility.
"""
cmd = [
"ipmitool",
"-I", interface,
"-H", host,
"-U", username,
"-P", password,
"-p", str(port),
"-N", "5",
"-R", "1",
"sdr", "elist",
]
logger.debug("Running: %s", " ".join(cmd[:-5]) + " *** *** " + " ".join(cmd[-4:]))
proc = await asyncio.create_subprocess_exec(
*cmd,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=30)
if proc.returncode != 0:
err = stderr.decode(errors="replace").strip()
raise RuntimeError(f"ipmitool exited with code {proc.returncode}: {err}")
return stdout.decode(errors="replace")
# Single ipmitool call timeout (B4): 30s -> 10s so one slow BMC cannot
# stall a whole collection round.
IPMI_TIMEOUT_SECONDS = 10
# Authentication attempts (O6/H2). Only these three combinations are tried:
# lanplus default, lanplus -C 3, lanplus -A MD5. The no-auth (-C 0) and
# IPMI 1.5 (lan interface) attempts were removed — they downgrade security
# and are handled by the parallel security fix's scope for hosts.py.
IPMI_AUTH_ATTEMPTS: list[tuple[str, Optional[list[str]]]] = [
("lanplus", None),
("lanplus", ["-C", "3"]),
("lanplus", ["-A", "MD5"]),
]
async def _run_ipmitool_with_auth(
host: str, port: int, username: str, password: str,
host: str, port: int, username: str, password: str,
interface: str = "lanplus", extra_args: list = None
) -> str:
"""Execute ipmitool sdr elist with specified arguments."""
cmd = [
"ipmitool",
"-I", interface,
"-H", host,
"-U", username,
"-P", password,
]
# Add extra arguments (e.g., -C 3, -A MD5, etc.)
if extra_args:
cmd.extend(extra_args)
cmd.extend([
"-p", str(port),
"-N", "5",
"-R", "1",
"sdr", "elist",
])
logger.debug("Running: %s", " ".join(cmd[:-5]) + " *** *** " + " ".join(cmd[-4:]))
proc = await asyncio.create_subprocess_exec(
*cmd,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=30)
if proc.returncode != 0:
err = stderr.decode(errors="replace").strip()
raise RuntimeError(f"ipmitool exited with code {proc.returncode}: {err}")
return stdout.decode(errors="replace")
"""Execute ipmitool sdr elist with specified arguments.
The BMC password is passed via a temporary password file (``-f``,
mode 0600, deleted immediately after use — H-03) instead of the
``-P`` command-line argument, so it never appears in the process list
or in logs. Only the host, interface and command name are logged.
"""
pwd_file: Optional[str] = None
proc = None
try:
# Write the password to a 0600 temp file for `ipmitool -f`.
fd, pwd_file = tempfile.mkstemp(prefix="ipmi-pwd-", dir="/tmp")
os.fchmod(fd, 0o600)
with os.fdopen(fd, "w", encoding="utf-8") as f:
f.write(password)
cmd = [
"ipmitool",
"-I", interface,
"-H", host,
"-U", username,
"-f", pwd_file,
]
# Add extra arguments (e.g., -C 3, -A MD5, etc.)
if extra_args:
cmd.extend(extra_args)
cmd.extend([
"-p", str(port),
"-N", "5",
"-R", "1",
"sdr", "elist",
])
logger.debug("Running ipmitool sdr elist on %s via %s", host, interface)
proc = await asyncio.create_subprocess_exec(
*cmd,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE,
)
try:
stdout, stderr = await asyncio.wait_for(
proc.communicate(), timeout=IPMI_TIMEOUT_SECONDS
)
except (asyncio.TimeoutError, TimeoutError):
# B3: kill + reap the child so no zombie ipmitool is left behind.
proc.kill()
await proc.wait()
raise TimeoutError(
f"ipmitool timed out after {IPMI_TIMEOUT_SECONDS}s on {host} via {interface}"
)
if proc.returncode != 0:
err = stderr.decode(errors="replace").strip()
raise RuntimeError(f"ipmitool exited with code {proc.returncode}: {err}")
return stdout.decode(errors="replace")
finally:
if pwd_file:
try:
os.unlink(pwd_file)
except OSError:
pass
async def collect_host(host: Any) -> dict[str, Any]:
@@ -228,45 +370,36 @@ async def collect_host(host: Any) -> dict[str, Any]:
}
try:
# Try different connection methods:
# 1. lanplus (IPMI 2.0) - simplest, no extra args
# 2. lanplus with -C 3 (IPMI 2.0, user auth)
# 3. lanplus with -C 0 (IPMI 2.0, no auth)
# 4. lanplus with -A MD5 (IPMI 1.5 with MD5)
# 5. lan with -A MD5 (IPMI 1.5 fallback)
# 6. lan (IPMI 1.5) - simplest fallback
# Decrypt the stored BMC password (decrypt_value returns the
# original value when decryption fails — historical plaintext compat).
# The password is never logged.
password = decrypt_value(host.ipmi_password)
raw = None
last_error = None
attempts = [
["lanplus", None],
["lanplus", ["-C", "3"]],
["lanplus", ["-C", "0"]],
["lanplus", ["-A", "MD5"]],
["lan", ["-A", "MD5"]],
["lan", None],
]
for interface, extra_args in attempts:
for interface, extra_args in IPMI_AUTH_ATTEMPTS:
try:
raw = await _run_ipmitool_with_auth(
host.ipmi_host, host.ipmi_port,
host.ipmi_username, host.ipmi_password,
host.ipmi_username, password,
interface=interface,
extra_args=extra_args,
)
if raw:
auth_str = " ".join(extra_args) if extra_args else "default"
logger.info("Successfully connected to %s using %s with %s",
logger.info("Successfully connected to %s using %s with %s",
host.name, interface, auth_str)
break
except RuntimeError as e:
except (RuntimeError, TimeoutError, asyncio.TimeoutError) as e:
# B3: a timeout (or any auth failure) falls through to the
# next authentication combination instead of aborting.
last_error = str(e)
logger.debug("Failed with %s %s: %s", interface, extra_args, last_error)
if not raw:
raise RuntimeError(f"All connection attempts failed. Last error: {last_error}")
parsed = parse_sdr_output(raw)
result["raw_parsed"] = parsed
@@ -304,7 +437,8 @@ async def collect_host(host: Any) -> dict[str, Any]:
# Parse component presence and add to InfluxDB points
from app.services.component_parser import parse_component_presence
components = parse_component_presence(raw)
components = parse_component_presence(_expand_legacy_sdr_lines(raw))
components = _fixup_components(raw, components)
for category in ["cpu", "memory", "pcie", "drive"]:
cat_data = components.get(category, {})
for item in cat_data.get("items", []):
@@ -316,11 +450,19 @@ async def collect_host(host: Any) -> dict[str, Any]:
"tags": {"category": category, "present": str(item.get("present", False))},
})
# Write to InfluxDB in one batch
# Write to InfluxDB in one batch (B14/B15: async thread offload,
# and a failed write must not be reported as a success).
if influx_points:
influxdb_module.write_metrics_batch(influx_points)
write_ok = await influxdb_module.awrite_metrics_batch(influx_points)
if not write_ok:
result["error"] = "Failed to write metrics to InfluxDB"
logger.error(
"InfluxDB write failed for host %s (%s); not updating last_seen",
host.id, host.name,
)
return result
# Update last_seen timestamp
# Update last_seen timestamp (only reached when the write succeeded)
from app.database import async_session
from app.models import Host as HostModel
from sqlalchemy import update
+68 -4
View File
@@ -15,14 +15,32 @@ logger = logging.getLogger(__name__)
_scheduler: AsyncIOScheduler | None = None
# Concurrency cap for collection (B4/O4): at most 8 ipmitool subprocesses
# at once, no matter how many hosts are enabled.
_COLLECT_SEMAPHORE = asyncio.Semaphore(8)
# Failure backoff (B4/O4): a host that fails this many rounds in a row is
# skipped for the next N rounds before being retried.
BACKOFF_AFTER_FAILURES = 3
BACKOFF_SKIP_ROUNDS = 5
_consecutive_failures: dict[int, int] = {}
_skip_remaining: dict[int, int] = {}
async def _collect_one(host: Host) -> dict[str, Any]:
"""Collect one host under the global concurrency cap."""
from app.services.ipmi_collector import collect_host
async with _COLLECT_SEMAPHORE:
return await collect_host(host)
async def collect_all_hosts() -> list[dict[str, Any]]:
"""Collect IPMI data for every enabled host.
Returns a list of collection results (one per host).
"""
from app.services.ipmi_collector import collect_host
results: list[dict[str, Any]] = []
async with async_session() as session:
@@ -36,8 +54,29 @@ async def collect_all_hosts() -> list[dict[str, Any]]:
logger.info("Collecting IPMI data from %d host(s)", len(hosts))
# Run collections concurrently
tasks = [collect_host(h) for h in hosts]
# Backoff: skip hosts that failed repeatedly, counting the rounds down.
due_hosts: list[Host] = []
for host in hosts:
remaining = _skip_remaining.get(host.id, 0)
if remaining > 0:
_skip_remaining[host.id] = remaining - 1
logger.debug(
"Skipping host %s (%s): backing off for %d more round(s)",
host.id, host.name, remaining - 1,
)
results.append({
"host_id": host.id,
"host_name": host.name,
"success": False,
"skipped": True,
"error": f"Backing off after repeated failures ({remaining - 1} round(s) left)",
"metrics": [],
})
else:
due_hosts.append(host)
# Run collections concurrently, capped by the semaphore (B4/O4)
tasks = [_collect_one(h) for h in due_hosts]
for coro in asyncio.as_completed(tasks):
try:
res = await coro
@@ -45,6 +84,27 @@ async def collect_all_hosts() -> list[dict[str, Any]]:
except Exception:
logger.exception("Unexpected error during collection task")
# Update failure counters / backoff state from this round's outcomes.
for res in results:
if res.get("skipped"):
continue
hid = res.get("host_id")
if hid is None:
continue
if res.get("success"):
_consecutive_failures.pop(hid, None)
else:
failures = _consecutive_failures.get(hid, 0) + 1
if failures >= BACKOFF_AFTER_FAILURES:
_skip_remaining[hid] = BACKOFF_SKIP_ROUNDS
_consecutive_failures.pop(hid, None)
logger.warning(
"Host %s failed %d consecutive rounds; skipping next %d round(s)",
hid, failures, BACKOFF_SKIP_ROUNDS,
)
else:
_consecutive_failures[hid] = failures
# After collection, run alert checks
try:
from app.services.alert_engine import check_alerts
@@ -69,6 +129,10 @@ def start_scheduler() -> AsyncIOScheduler:
id="collect_all_hosts",
replace_existing=True,
max_instances=1,
# B4: coalesce missed runs into one and drop runs that are more
# than 30s late instead of letting them pile up behind a slow host.
coalesce=True,
misfire_grace_time=30,
)
_scheduler.start()
logger.info(
+122
View File
@@ -0,0 +1,122 @@
"""SSRF guards for outbound requests whose target is user-controlled.
Three user-controlled destinations exist in this app:
* ``ipmi_host`` — the BMC to poll. BMCs live on the internal network, so
RFC1918 private addresses and hostnames are legitimate; only the
dangerous classes are rejected (loopback, link-local incl. the
169.254.169.254 cloud-metadata address, multicast, reserved/unspecified).
* Feishu webhook URL — only the official Feishu/Lark open-platform hosts
over HTTPS.
* SMTP host — same dangerous-address rejection as the IPMI host (an
internal relay on a private address is legitimate).
All validators raise :class:`ValueError` on rejection; API callers turn that
into HTTP 400. Hostnames are validated syntactically only (no DNS lookup
here, so no resolution-time rebinding check is claimed).
"""
import ipaddress
import re
from urllib.parse import urlparse
# Cloud metadata endpoint (also covered by the link-local check, listed
# explicitly so the intent is obvious).
METADATA_IP = "169.254.169.254"
# Official Feishu / Lark open-platform webhook hosts.
FEISHU_WEBHOOK_HOSTS = ("open.feishu.cn", "open.larksuite.com")
_HOSTNAME_RE = re.compile(
r"^(?=.{1,253}$)(?!-)[A-Za-z0-9-]{1,63}(?<!-)"
r"(?:\.(?!-)[A-Za-z0-9-]{1,63}(?<!-))*$"
)
def _reject_dangerous_ip(ip: ipaddress.IPv4Address | ipaddress.IPv6Address, what: str) -> None:
"""Reject loopback / link-local / multicast / reserved / unspecified IPs."""
if ip.is_loopback:
raise ValueError(f"{what} must not be a loopback address: {ip}")
if ip.is_link_local:
raise ValueError(f"{what} must not be a link-local address: {ip}")
if ip.is_multicast:
raise ValueError(f"{what} must not be a multicast address: {ip}")
if ip.is_reserved or ip.is_unspecified:
raise ValueError(f"{what} must not be a reserved or unspecified address: {ip}")
if str(ip) == METADATA_IP:
raise ValueError(f"{what} must not be the cloud metadata address: {ip}")
def _validate_hostname(host: str, what: str) -> None:
lowered = host.lower()
if lowered in ("localhost",) or lowered.endswith(".localhost"):
raise ValueError(f"{what} must not be localhost")
if not _HOSTNAME_RE.match(host):
raise ValueError(f"{what} is not a valid hostname or IP address: {host!r}")
def validate_ipmi_host(host: str) -> str:
"""Validate an IPMI target host.
Allows RFC1918 private addresses and legitimate hostnames; rejects
loopback, link-local (incl. 169.254.169.254), multicast and reserved
addresses. Returns the stripped host on success.
"""
if not host or not isinstance(host, str):
raise ValueError("ipmi_host must not be empty")
host = host.strip()
if len(host) > 253:
raise ValueError("ipmi_host is too long")
try:
ip = ipaddress.ip_address(host)
except ValueError:
_validate_hostname(host, "ipmi_host")
return host
_reject_dangerous_ip(ip, "ipmi_host")
return host
def validate_webhook_url(url: str) -> str:
"""Validate a Feishu webhook URL.
Only ``https`` URLs on ``open.feishu.cn`` / ``open.larksuite.com`` (or
their subdomains) are allowed, with no embedded credentials and no
non-default port. Returns the URL on success.
"""
if not url or not isinstance(url, str):
raise ValueError("webhook URL must not be empty")
parsed = urlparse(url.strip())
if parsed.scheme.lower() != "https":
raise ValueError("webhook URL must use https")
host = (parsed.hostname or "").lower()
if not any(host == allowed or host.endswith("." + allowed)
for allowed in FEISHU_WEBHOOK_HOSTS):
raise ValueError(
"webhook URL host must be open.feishu.cn or open.larksuite.com"
)
if parsed.username or parsed.password:
raise ValueError("webhook URL must not contain credentials")
if parsed.port is not None and parsed.port != 443:
raise ValueError("webhook URL must not use a non-standard port")
return url.strip()
def validate_smtp_host(host: str) -> str:
"""Validate an SMTP host.
Rejects loopback, link-local (incl. 169.254.169.254), multicast and
reserved addresses; private-network relays and hostnames are allowed.
Returns the stripped host on success.
"""
if not host or not isinstance(host, str):
raise ValueError("smtp_host must not be empty")
host = host.strip()
if len(host) > 253:
raise ValueError("smtp_host is too long")
try:
ip = ipaddress.ip_address(host)
except ValueError:
_validate_hostname(host, "smtp_host")
return host
_reject_dangerous_ip(ip, "smtp_host")
return host
+84
View File
@@ -0,0 +1,84 @@
"""Shared helpers for the IPMI Monitor backend.
Small pure functions used by more than one module live here so the logic
is defined exactly once (parse_dt used to be copy-pasted in monitor.py,
total-power fallback used to be re-implemented in three places).
"""
import re
from datetime import datetime, timezone
from typing import Any, Optional
# ---------------------------------------------------------------------------
# Datetime parsing (B11)
# ---------------------------------------------------------------------------
def parse_dt(dt_str: Optional[str]) -> Optional[datetime]:
"""Parse an ISO-8601 datetime string into a naive UTC datetime.
Returns ``None`` for empty input. Raises :class:`ValueError` for an
unparseable string — callers should translate that into a 400 response
instead of letting it become a 500.
"""
if not dt_str:
return None
dt_str = dt_str.strip().replace("Z", "+00:00")
dt = datetime.fromisoformat(dt_str) # raises ValueError on bad input
if dt.tzinfo is not None:
return dt.astimezone(timezone.utc).replace(tzinfo=None)
return dt
def utc_now_naive() -> datetime:
"""Current time as a naive UTC datetime (matches InfluxDB query inputs)."""
return datetime.now(timezone.utc).replace(tzinfo=None)
# ---------------------------------------------------------------------------
# Total power resolution (O5 / B30)
# ---------------------------------------------------------------------------
# Sensors whose name indicates an overall system/host power reading.
NATIVE_TOTAL_POWER_NAMES = ("total_power", "power", "pwr consumption")
_PSU_PIN_RE = re.compile(r"^PSU\d+_PIN$", re.IGNORECASE)
def resolve_total_power(power_metrics: list[dict[str, Any]]) -> Optional[float]:
"""Resolve a host's total power from its ``power`` metrics.
Priority:
1. A native system-wide reading (``Total_Power`` / ``Power`` /
``Pwr Consumption``). A sensor that *exists* is authoritative —
a reading of ``0`` is a valid reading, not "missing".
2. Otherwise the sum of all ``PSU{N}_PIN`` (PSU input power), which
covers hosts whose BMC exposes no system-wide sensor (e.g. Gooxi12).
Returns ``None`` when no usable power metric exists at all.
"""
native_value: Optional[float] = None
psu_pin_sum = 0.0
psu_pin_seen = False
for m in power_metrics or []:
name = (m.get("metric_name") or "").strip()
raw_value = m.get("value")
if raw_value is None:
continue
try:
value = float(raw_value)
except (TypeError, ValueError):
continue
if name.lower() in NATIVE_TOTAL_POWER_NAMES:
# First native reading wins; 0.0 counts as a real reading.
if native_value is None:
native_value = value
elif _PSU_PIN_RE.match(name):
psu_pin_sum += value
psu_pin_seen = True
if native_value is not None:
return native_value
if psu_pin_seen:
return psu_pin_sum
return None
+20 -14
View File
@@ -1,14 +1,20 @@
fastapi>=0.104.1
uvicorn>=0.24.0
sqlalchemy>=2.0.23
aiosqlite>=0.19.0
influxdb-client>=1.38.0
httpx>=0.25.2
pydantic>=2.5.2
pydantic-settings>=2.1.0
apscheduler>=3.10.4
jinja2>=3.1.2
python-multipart>=0.0.6
aiofiles>=23.2.1
python-jose[cryptography]>=0.3.3
passlib[bcrypt]>=1.7.4
# Pinned exact versions (Python 3.11 compatible).
# PyJWT replaces the unmaintained python-jose (see H-07).
# cryptography is required by app/crypto.py (Fernet credential encryption).
# bcrypt is pinned to 4.0.1: passlib 1.7.4 is incompatible with bcrypt >= 4.1.
fastapi==0.115.12
uvicorn==0.34.0
sqlalchemy==2.0.36
aiosqlite==0.20.0
influxdb-client==1.49.0
httpx==0.28.1
pydantic==2.10.6
pydantic-settings==2.7.1
apscheduler==3.11.0
jinja2==3.1.6
python-multipart==0.0.20
aiofiles==24.1.0
PyJWT==2.10.1
passlib[bcrypt]==1.7.4
bcrypt==4.0.1
cryptography==44.0.3
+251
View File
@@ -0,0 +1,251 @@
"""Tests for alert recovery notifications (``app.services.alert_engine``).
- A recovery notification is sent only when an alert was actually fired
(``_alert_active`` set); a transient breach that never satisfied the
duration just clears its timer silently.
- Dirty metric data (``None`` / non-numeric values) must not abort the
whole check round: the bad point is skipped and the remaining points
are still evaluated.
Dependency note: this machine has no project third-party packages
installed (no ``sqlalchemy`` / ``pydantic`` / ``influxdb_client``), and
``alert_engine`` imports them at module level. The missing modules are
stubbed in ``sys.modules`` before the import — the same minimal-surface
approach as ``test_sdr_parser.py`` — so only the pure rule-evaluation
logic under test is exercised. InfluxDB access and the notification
dispatchers are replaced with recording fakes; no database, network or
SMTP/Feishu call is made.
"""
import sys
import types
import unittest
from datetime import datetime, timedelta
from types import SimpleNamespace
def _install_stubs() -> None:
"""Stub the heavy third-party / app modules ``alert_engine`` imports."""
if "sqlalchemy" not in sys.modules:
sqlalchemy_mod = types.ModuleType("sqlalchemy")
sqlalchemy_mod.select = lambda *args, **kwargs: None
sys.modules["sqlalchemy"] = sqlalchemy_mod
if "sqlalchemy.ext" not in sys.modules:
ext_mod = types.ModuleType("sqlalchemy.ext")
sys.modules["sqlalchemy.ext"] = ext_mod
sys.modules["sqlalchemy"].ext = ext_mod
if "sqlalchemy.ext.asyncio" not in sys.modules:
asyncio_mod = types.ModuleType("sqlalchemy.ext.asyncio")
class AsyncSession: # noqa: D101 - stub only
pass
asyncio_mod.AsyncSession = AsyncSession
sys.modules["sqlalchemy.ext.asyncio"] = asyncio_mod
sys.modules["sqlalchemy.ext"].asyncio = asyncio_mod
if "app.influxdb" not in sys.modules:
influx_mod = types.ModuleType("app.influxdb")
async def _aquery_latest(host_id, metric_type):
raise NotImplementedError("stubbed; patch per test")
influx_mod.aquery_latest = _aquery_latest
sys.modules["app.influxdb"] = influx_mod
if "app.database" not in sys.modules:
db_mod = types.ModuleType("app.database")
db_mod.async_session = None
sys.modules["app.database"] = db_mod
if "app.models" not in sys.modules:
models_mod = types.ModuleType("app.models")
class _ModelStub: # noqa: D101 - stub only
pass
for _name in ("AlertHistory", "AlertRule", "Host", "SystemSetting"):
setattr(models_mod, _name, _ModelStub)
sys.modules["app.models"] = models_mod
if "app.crypto" not in sys.modules:
# alert_engine imports decrypt_value directly; the real module
# needs ``cryptography`` (not installed here). These tests never
# exercise decryption (dispatchers are patched out), so an
# identity stand-in is sufficient.
crypto_mod = types.ModuleType("app.crypto")
crypto_mod.decrypt_value = lambda value: value
sys.modules["app.crypto"] = crypto_mod
_install_stubs()
from app.services import alert_engine
def _make_rule(**overrides):
"""A rule-like object: gt 80 on 'CPU Temp', duration 0 by default."""
data = {
"id": 1,
"name": "High Temp",
"host_id": None,
"group_id": None,
"metric_type": "temperature",
"metric_name": "CPU Temp",
"condition": "gt",
"threshold": 80.0,
"duration": 0,
"notify_email": True,
"notify_feishu": False,
"email_addresses": "a@example.com",
}
data.update(overrides)
return SimpleNamespace(**data)
class _FakeSession:
"""Minimal async session: only ``get`` is used on the recovery path."""
def __init__(self, host):
self._host = host
async def get(self, model, pk):
return self._host
class AlertRecoveryTest(unittest.IsolatedAsyncioTestCase):
def setUp(self):
self.host = SimpleNamespace(id=10, name="host-10")
self.session = _FakeSession(self.host)
self.rule = _make_rule()
self.key = (self.rule.id, self.host.id)
self.points = []
self.history_calls = []
self.recovery_calls = []
# Fresh in-process alert state for every test.
alert_engine._first_breach.clear()
alert_engine._alert_active.clear()
alert_engine._last_notify.clear()
# Patch the InfluxDB query and the notification dispatchers.
self._orig_latest = alert_engine.influxdb_module.aquery_latest
self._orig_history = alert_engine._create_alert_history
self._orig_recovery = alert_engine._create_recovery_notification
async def fake_latest(hid, metric_type):
return list(self.points)
async def record_history(rule, host, value):
self.history_calls.append((rule, host, value))
async def record_recovery(rule, host, value):
self.recovery_calls.append((rule, host, value))
alert_engine.influxdb_module.aquery_latest = fake_latest
alert_engine._create_alert_history = record_history
alert_engine._create_recovery_notification = record_recovery
def tearDown(self):
alert_engine.influxdb_module.aquery_latest = self._orig_latest
alert_engine._create_alert_history = self._orig_history
alert_engine._create_recovery_notification = self._orig_recovery
alert_engine._first_breach.clear()
alert_engine._alert_active.clear()
alert_engine._last_notify.clear()
async def _check(self):
await alert_engine._check_rule_for_host(
self.session, self.rule, self.host.id
)
def _mark_breaching(self, active: bool):
"""Pretend a breach timer is running; optionally an alert fired."""
alert_engine._first_breach[self.key] = datetime.now() - timedelta(
seconds=60
)
if active:
alert_engine._alert_active[self.key] = True
async def test_recovery_sent_when_alert_was_active(self):
"""Active alert + value back to normal -> exactly one recovery."""
self._mark_breaching(active=True)
self.points = [{"metric_name": "CPU Temp", "value": 30.0}]
await self._check()
self.assertEqual(len(self.recovery_calls), 1)
self.assertEqual(self.recovery_calls[0][2], 30.0)
# State is cleared after recovery.
self.assertNotIn(self.key, alert_engine._first_breach)
self.assertNotIn(self.key, alert_engine._alert_active)
async def test_no_recovery_when_breach_never_fired(self):
"""Transient breach (timer running, alert never fired) recovers
silently: no recovery notification, timer just cleared."""
self._mark_breaching(active=False)
self.points = [{"metric_name": "CPU Temp", "value": 30.0}]
await self._check()
self.assertEqual(self.recovery_calls, [])
self.assertNotIn(self.key, alert_engine._first_breach)
async def test_no_recovery_without_any_prior_breach(self):
"""Normal value with no breach state at all -> nothing happens."""
self.points = [{"metric_name": "CPU Temp", "value": 30.0}]
await self._check()
self.assertEqual(self.recovery_calls, [])
self.assertEqual(self.history_calls, [])
async def test_none_value_does_not_interrupt_round(self):
"""A None value is skipped; the following valid point in the same
round is still evaluated and triggers the recovery."""
self._mark_breaching(active=True)
self.points = [
{"metric_name": "CPU Temp", "value": None},
{"metric_name": "CPU Temp", "value": 30.0},
]
await self._check()
self.assertEqual(len(self.recovery_calls), 1)
async def test_none_value_alone_preserves_state(self):
"""A round with only a None point neither recovers nor clears the
active alert state."""
self._mark_breaching(active=True)
self.points = [{"metric_name": "CPU Temp", "value": None}]
await self._check()
self.assertEqual(self.recovery_calls, [])
self.assertTrue(alert_engine._alert_active.get(self.key))
async def test_non_numeric_value_does_not_interrupt_round(self):
"""A non-numeric value ('N/A') is skipped; the valid point after
it in the same round still triggers the recovery."""
self._mark_breaching(active=True)
self.points = [
{"metric_name": "CPU Temp", "value": "N/A"},
{"metric_name": "CPU Temp", "value": 30.0},
]
await self._check()
self.assertEqual(len(self.recovery_calls), 1)
async def test_non_numeric_value_alone_preserves_state(self):
self._mark_breaching(active=True)
self.points = [{"metric_name": "CPU Temp", "value": "not-a-number"}]
await self._check()
self.assertEqual(self.recovery_calls, [])
self.assertTrue(alert_engine._alert_active.get(self.key))
async def test_unrelated_metric_name_ignored(self):
"""Points for other sensors do not affect this rule's state."""
self._mark_breaching(active=True)
self.points = [{"metric_name": "FAN1", "value": 100.0}]
await self._check()
self.assertEqual(self.recovery_calls, [])
self.assertTrue(alert_engine._alert_active.get(self.key))
async def test_alert_fires_when_duration_satisfied(self):
"""Sanity check on the firing side: a breaching value with
duration 0 fires the alert and marks it active (so a later
recovery has something to recover from)."""
self.points = [{"metric_name": "CPU Temp", "value": 90.0}]
await self._check()
self.assertEqual(len(self.history_calls), 1)
self.assertTrue(alert_engine._alert_active.get(self.key))
self.assertEqual(self.recovery_calls, [])
if __name__ == "__main__":
unittest.main()
+87
View File
@@ -0,0 +1,87 @@
"""Tests for ``app.utils.parse_dt``.
``parse_dt`` is the shared datetime parser behind the monitor history /
export endpoints: valid input becomes a naive UTC datetime, empty input
is ``None`` (caller applies its default range), and invalid input raises
``ValueError`` — which the routers translate into a 400 response, never
a 500.
"""
import unittest
from datetime import datetime
from app.utils import parse_dt
class ParseDtTest(unittest.TestCase):
def test_valid_iso_with_t_separator(self):
self.assertEqual(
parse_dt("2026-11-16T10:00:00"), datetime(2026, 11, 16, 10, 0, 0)
)
def test_valid_iso_with_space_separator(self):
self.assertEqual(
parse_dt("2026-11-16 10:00:00"), datetime(2026, 11, 16, 10, 0, 0)
)
def test_valid_date_only(self):
self.assertEqual(parse_dt("2026-11-16"), datetime(2026, 11, 16, 0, 0, 0))
def test_valid_iso_with_fractional_seconds(self):
self.assertEqual(
parse_dt("2026-11-16T10:00:00.123456"),
datetime(2026, 11, 16, 10, 0, 0, 123456),
)
def test_zulu_suffix_converts_to_naive_utc(self):
"""'Z' means UTC already: the value is unchanged, tzinfo dropped."""
self.assertEqual(
parse_dt("2026-11-16T10:00:00Z"), datetime(2026, 11, 16, 10, 0, 0)
)
def test_positive_offset_converts_to_utc(self):
"""+08:00 input is shifted back 8 hours to UTC."""
self.assertEqual(
parse_dt("2026-11-16T18:00:00+08:00"),
datetime(2026, 11, 16, 10, 0, 0),
)
def test_negative_offset_converts_to_utc(self):
"""-05:00 input is shifted forward 5 hours to UTC."""
self.assertEqual(
parse_dt("2026-11-16T05:00:00-05:00"),
datetime(2026, 11, 16, 10, 0, 0),
)
def test_offset_can_shift_the_calendar_day(self):
"""+09:00 just after midnight UTC-converts to the previous day."""
self.assertEqual(
parse_dt("2026-11-17T01:00:00+09:00"),
datetime(2026, 11, 16, 16, 0, 0),
)
def test_result_is_always_naive(self):
for raw in ("2026-11-16T10:00:00Z", "2026-11-16T18:00:00+08:00"):
self.assertIsNone(parse_dt(raw).tzinfo)
def test_empty_input_returns_none(self):
self.assertIsNone(parse_dt(None))
self.assertIsNone(parse_dt(""))
def test_garbage_input_raises_value_error(self):
"""Invalid input raises ValueError (the routers map this to 400)."""
for raw in ("not-a-date", "2026-13-01", "2026-02-30", "16/11/2026",
"2026-11-16T25:00:00", "tomorrow", " "):
with self.subTest(raw=raw):
with self.assertRaises(ValueError):
parse_dt(raw)
def test_whitespace_is_tolerated(self):
self.assertEqual(
parse_dt(" 2026-11-16T10:00:00 "),
datetime(2026, 11, 16, 10, 0, 0),
)
if __name__ == "__main__":
unittest.main()
+151
View File
@@ -0,0 +1,151 @@
"""Tests for ``parse_sdr_output`` (``app.services.ipmi_collector``).
Covers the two SDR shapes (new 5-segment and legacy 3-segment), negative
readings, thousands separators and the disabled/critical status tokens.
Dependency note: this machine has no project third-party packages
installed (no ``influxdb_client`` / ``cryptography`` / ``pydantic``), and
``ipmi_collector`` imports ``app.influxdb`` and ``app.config`` at module
level. Like ``test_total_power_fallback`` (which only imports the
stdlib-only ``component_parser``), these tests keep the import surface
minimal: the two heavy modules are stubbed in ``sys.modules`` before the
import, so only the pure parsing function under test is exercised. The
``app.crypto`` import inside ``ipmi_collector`` is already guarded by a
``try/except ImportError`` there and falls back to the identity function.
"""
import sys
import types
import unittest
from types import SimpleNamespace
def _install_stubs() -> None:
"""Stub ``app.influxdb`` / ``app.config`` so ``ipmi_collector`` imports."""
if "app.influxdb" not in sys.modules:
sys.modules["app.influxdb"] = types.ModuleType("app.influxdb")
if "app.config" not in sys.modules:
config_mod = types.ModuleType("app.config")
config_mod.settings = SimpleNamespace(COLLECT_INTERVAL=60)
sys.modules["app.config"] = config_mod
_install_stubs()
from app.services.ipmi_collector import parse_sdr_output
class ParseSdrOutputTest(unittest.TestCase):
def test_new_format_five_segments(self):
"""New format: name | id | status | entity | value."""
result = parse_sdr_output("Temp | 01h | ok | 3.1 | 37 degrees C\n")
self.assertEqual(len(result), 1)
entry = result[0]
self.assertEqual(entry["name"], "Temp")
self.assertEqual(entry["status"], "ok")
self.assertEqual(entry["value_str"], "37 degrees C")
self.assertEqual(entry["metric_type"], "temperature")
self.assertEqual(entry["numeric_value"], 37.0)
def test_new_format_fan(self):
result = parse_sdr_output("Fan1A | 38h | ok | 7.1 | 7920 RPM\n")
self.assertEqual(result[0]["metric_type"], "fan")
self.assertEqual(result[0]["numeric_value"], 7920.0)
def test_legacy_format_three_segments(self):
"""Legacy format: name | value | status."""
result = parse_sdr_output("CPU1 Temp | 45 degrees C | ok\n")
self.assertEqual(len(result), 1)
entry = result[0]
self.assertEqual(entry["name"], "CPU1 Temp")
self.assertEqual(entry["status"], "ok")
self.assertEqual(entry["metric_type"], "temperature")
self.assertEqual(entry["numeric_value"], 45.0)
def test_legacy_format_fan_and_voltage(self):
result = parse_sdr_output(
"FAN1 | 5400 RPM | ok\n"
"Vcpu1 | 1.01 Volts | ok\n"
)
self.assertEqual(result[0]["metric_type"], "fan")
self.assertEqual(result[0]["numeric_value"], 5400.0)
self.assertEqual(result[1]["metric_type"], "voltage")
self.assertEqual(result[1]["numeric_value"], 1.01)
def test_negative_temperature(self):
"""Negative readings must keep their sign (both formats)."""
result = parse_sdr_output(
"Inlet Temp | 01h | ok | 3.1 | -5.0 degrees C\n"
"Ambient Temp | -2.5 degrees C | ok\n"
)
self.assertEqual(result[0]["metric_type"], "temperature")
self.assertEqual(result[0]["numeric_value"], -5.0)
self.assertEqual(result[1]["metric_type"], "temperature")
self.assertEqual(result[1]["numeric_value"], -2.5)
def test_thousands_separator_comma(self):
"""'1,234 RPM' / '1,234.5 RPM' parse as 1234 / 1234.5, not 1."""
result = parse_sdr_output(
"FAN1 | 38h | ok | 7.1 | 1,234 RPM\n"
"FAN2 | 39h | ok | 7.1 | 1,234.5 RPM\n"
)
self.assertEqual(result[0]["numeric_value"], 1234.0)
self.assertEqual(result[1]["numeric_value"], 1234.5)
def test_disabled_status_token(self):
"""A 'disabled' sensor still parses; its status is kept verbatim."""
result = parse_sdr_output("Fan2 | 39h | disabled | 7.1 | 0 RPM\n")
self.assertEqual(len(result), 1)
self.assertEqual(result[0]["status"], "disabled")
self.assertEqual(result[0]["metric_type"], "fan")
self.assertEqual(result[0]["numeric_value"], 0.0)
def test_critical_status_token(self):
"""A 'cr' (critical) reading still parses with its value."""
result = parse_sdr_output(
"CPU Temp | 01h | cr | 3.1 | 95 degrees C\n"
)
self.assertEqual(result[0]["status"], "cr")
self.assertEqual(result[0]["metric_type"], "temperature")
self.assertEqual(result[0]["numeric_value"], 95.0)
def test_unknown_shape_finds_status_token(self):
"""4-segment rows: locate the status token, never read the sensor
id (parts[1]) as the value."""
result = parse_sdr_output(
"Fan1 | 38h | ok | 7920 RPM\n"
"CPU Temp | 01h | cr | 95 degrees C\n"
)
self.assertEqual(result[0]["status"], "ok")
self.assertEqual(result[0]["numeric_value"], 7920.0)
self.assertEqual(result[1]["status"], "cr")
self.assertEqual(result[1]["numeric_value"], 95.0)
def test_discrete_status_sensor_uses_status(self):
"""Discrete power-supply sensors map ok -> 1.0, anything else -> 0.0."""
result = parse_sdr_output(
"Power Supply 1 | 0x00 | ok\n"
"Power Supply 2 | 0x00 | cr\n"
)
self.assertEqual(result[0]["metric_type"], "status")
self.assertEqual(result[0]["numeric_value"], 1.0)
self.assertEqual(result[1]["metric_type"], "status")
self.assertEqual(result[1]["numeric_value"], 0.0)
def test_blank_and_unparseable_lines_skipped(self):
result = parse_sdr_output(
"\n"
"CPU1 Temp | 45 degrees C | ok\n"
"garbage line without separators\n"
"Foo | bar\n"
)
self.assertEqual(len(result), 1)
self.assertEqual(result[0]["name"], "CPU1 Temp")
def test_empty_input_returns_empty_list(self):
self.assertEqual(parse_sdr_output(""), [])
self.assertEqual(parse_sdr_output(" \n \n"), [])
if __name__ == "__main__":
unittest.main()
+16 -11
View File
@@ -13,10 +13,11 @@ services:
- TZ=Asia/Shanghai
- DOCKER_INFLUXDB_INIT_MODE=setup
- DOCKER_INFLUXDB_INIT_USERNAME=${INFLUXDB_USERNAME:-admin}
- DOCKER_INFLUXDB_INIT_PASSWORD=${INFLUXDB_PASSWORD:-admin123456}
# Required: no default is shipped; Compose refuses to start without them.
- DOCKER_INFLUXDB_INIT_PASSWORD=${INFLUXDB_PASSWORD:?must be set}
- DOCKER_INFLUXDB_INIT_ORG=${INFLUXDB_ORG:-ipmi-monitor}
- DOCKER_INFLUXDB_INIT_BUCKET=${INFLUXDB_BUCKET:-metrics}
- DOCKER_INFLUXDB_INIT_ADMIN_TOKEN=${INFLUXDB_TOKEN:-my-super-secret-token}
- DOCKER_INFLUXDB_INIT_ADMIN_TOKEN=${INFLUXDB_TOKEN:?must be set}
restart: unless-stopped
healthcheck:
test: ["CMD", "influx", "ping"]
@@ -34,20 +35,27 @@ services:
ports:
- "${BACKEND_PORT:-8000}:8000"
volumes:
- ./backend/app:/app/app
- ./frontend:/app/frontend
- backend-data:/app/data
environment:
- TZ=Asia/Shanghai
- ENVIRONMENT=${ENVIRONMENT:-production}
# Required: the backend refuses to start without a JWT secret of at
# least 32 characters. Generate one and set it in .env.
- JWT_SECRET_KEY=${JWT_SECRET_KEY:?must be set}
# Required on first boot only (empty user table): password for the
# initial admin account (at least 12 characters). Ignored afterwards.
- ADMIN_INITIAL_PASSWORD=${ADMIN_INITIAL_PASSWORD:-}
# Optional: Fernet key for encrypting stored BMC/SMTP/Feishu secrets.
# When unset, the backend generates one and persists it next to the DB.
- CREDENTIAL_ENCRYPTION_KEY=${CREDENTIAL_ENCRYPTION_KEY:-}
# Optional: comma-separated CORS allow-list. Empty = no CORS (same-origin).
- CORS_ORIGINS=${CORS_ORIGINS:-}
- INFLUXDB_URL=http://influxdb:8086
- INFLUXDB_TOKEN=${INFLUXDB_TOKEN:-my-super-secret-token}
- INFLUXDB_TOKEN=${INFLUXDB_TOKEN:?must be set}
- INFLUXDB_ORG=${INFLUXDB_ORG:-ipmi-monitor}
- INFLUXDB_BUCKET=${INFLUXDB_BUCKET:-metrics}
- POLL_INTERVAL=${POLL_INTERVAL:-60}
- LOG_LEVEL=${LOG_LEVEL:-INFO}
- ALERT_TEMP_THRESHOLD=${ALERT_TEMP_THRESHOLD:-80}
- ALERT_FAN_THRESHOLD=${ALERT_FAN_THRESHOLD:-0}
- ALERT_POWER_THRESHOLD=${ALERT_POWER_THRESHOLD:-500}
- DATABASE_URL=${DATABASE_URL:-sqlite+aiosqlite:///./data/ipmi_monitor.db}
- REQUIRE_EXISTING_DATABASE=${REQUIRE_EXISTING_DATABASE:-false}
- MIN_EXPECTED_HOSTS=${MIN_EXPECTED_HOSTS:-0}
@@ -55,9 +63,6 @@ services:
- SMTP_PORT=${SMTP_PORT:-587}
- SMTP_USER=${SMTP_USER:-}
- SMTP_PASSWORD=${SMTP_PASSWORD:-}
- ALERT_EMAIL_FROM=${ALERT_EMAIL_FROM:-}
- ALERT_EMAIL_TO=${ALERT_EMAIL_TO:-}
- WEBHOOK_URL=${WEBHOOK_URL:-}
depends_on:
influxdb:
condition: service_healthy
+3 -1
View File
@@ -63,7 +63,9 @@
<el-table-column label="密码" min-width="110">
<template #default="{ row }">
<span class="password-mask">••••••••</span>
<!-- 后端不再返回 ipmi_password,只返回 has_password -->
<span v-if="row.has_password" class="password-mask">••••••••</span>
<span v-else class="text-muted">未配置</span>
</template>
</el-table-column>
+26 -7
View File
@@ -27,12 +27,18 @@
<el-form-item label="Webhook URL">
<el-input
v-model="feishuForm.feishu_webhook_url"
placeholder="https://open.feishu.cn/open-apis/bot/v2/hook/xxxxxxxx"
:placeholder="feishuWebhookMasked || 'https://open.feishu.cn/open-apis/bot/v2/hook/xxxxxxxx'"
clearable
/>
<div class="form-tip">
飞书群机器人 Webhook 地址,在飞书群设置中添加自定义机器人获取
</div>
<div v-if="feishuWebhookMasked" class="form-tip">
当前已配置:{{ feishuWebhookMasked }},留空则不修改
</div>
<div v-else class="form-tip">
当前未配置
</div>
</el-form-item>
<el-form-item label="安全关键词">
@@ -106,13 +112,16 @@
<el-input
v-model="emailForm.smtp_password"
type="password"
placeholder="邮箱密码或授权码"
:placeholder="smtpPasswordSet ? '已配置,留空则不修改' : '邮箱密码或授权码'"
show-password
clearable
/>
<div class="form-tip">
部分邮箱需要使用授权码而非登录密码,请在邮箱设置中生成
</div>
<div v-if="smtpPasswordSet" class="form-tip">
当前已配置,留空则不修改
</div>
</el-form-item>
<el-form-item label="发件人地址">
@@ -142,7 +151,7 @@
<el-button
type="info"
:loading="testingEmail"
@click="testEmail"
@click="testEmailSend"
:icon="Promotion"
>
发送测试邮件
@@ -170,6 +179,10 @@ const testEmail = ref('')
const feishuFormRef = ref()
const emailFormRef = ref()
// 后端不再返回秘密明文,只返回掩码/是否已配置;秘密输入框留空表示不改。
const feishuWebhookMasked = ref('')
const smtpPasswordSet = ref(false)
const feishuForm = reactive({
feishu_webhook_url: '',
feishu_keyword: '',
@@ -188,15 +201,20 @@ const emailForm = reactive({
const loadSettings = async () => {
try {
const response = await api.get('/api/settings')
// 后端不再返回 feishu_webhook_url / smtp_password 明文:
// 飞书只给掩码(后 4 位可见),SMTP 只给是否已配置。
// 秘密输入框保持留空,提交时留空表示不改。
feishuWebhookMasked.value = response.feishu_webhook_masked || ''
smtpPasswordSet.value = !!response.smtp_password_set
Object.assign(feishuForm, {
feishu_webhook_url: response.feishu_webhook_url || '',
feishu_webhook_url: '',
feishu_keyword: response.feishu_keyword || '',
})
Object.assign(emailForm, {
smtp_host: response.smtp_host || '',
smtp_port: parseInt(response.smtp_port) || 587,
smtp_user: response.smtp_user || '',
smtp_password: response.smtp_password || '',
smtp_password: '',
smtp_from: response.smtp_from || '',
smtp_use_tls: response.smtp_use_tls !== false,
})
@@ -232,7 +250,8 @@ const saveSettings = async () => {
// 测试飞书
const testFeishu = async () => {
if (!feishuForm.feishu_webhook_url) {
// 本次未重填也可用已配置的 Webhook(后端存着解密后的值)
if (!feishuForm.feishu_webhook_url && !feishuWebhookMasked.value) {
ElMessage.warning('请先配置飞书Webhook URL')
return
}
@@ -254,7 +273,7 @@ const testEmailSend = async () => {
ElMessage.warning('请输入测试邮箱地址')
return
}
if (!emailForm.smtp_host || !emailForm.smtp_user || !emailForm.smtp_password) {
if (!emailForm.smtp_host || !emailForm.smtp_user || (!emailForm.smtp_password && !smtpPasswordSet.value)) {
ElMessage.warning('请先配置SMTP服务器信息')
return
}
+41
View File
@@ -71,6 +71,47 @@ else
echo -e "${GREEN} ✓ .env file found${NC}"
fi
# ----- Step 3b: Reject missing / placeholder secrets -----
echo -e "${YELLOW}[3b/6] Checking required secrets in .env...${NC}"
get_env_value() {
grep -E "^$1=" .env 2>/dev/null | tail -n 1 | cut -d'=' -f2- | tr -d '\r' | sed -e 's/^["'\'']//' -e 's/["'\'']$//'
}
looks_like_placeholder() {
local value="$1"
local lowered
lowered=$(printf '%s' "$value" | tr '[:upper:]' '[:lower:]')
case "$lowered" in
*changeme*|*example*|*placeholder*|*default*|*todo*) return 0 ;;
esac
return 1
}
INFLUXDB_PASSWORD=$(get_env_value INFLUXDB_PASSWORD)
INFLUXDB_TOKEN=$(get_env_value INFLUXDB_TOKEN)
JWT_SECRET_KEY=$(get_env_value JWT_SECRET_KEY)
if [ -z "$INFLUXDB_PASSWORD" ] || looks_like_placeholder "$INFLUXDB_PASSWORD" || [ "${#INFLUXDB_PASSWORD}" -lt 8 ]; then
echo -e "${RED}Error: INFLUXDB_PASSWORD is missing, too short (< 8 chars), or still a placeholder.${NC}"
echo "Set a strong password in .env before starting."
exit 1
fi
if [ -z "$INFLUXDB_TOKEN" ] || looks_like_placeholder "$INFLUXDB_TOKEN"; then
echo -e "${RED}Error: INFLUXDB_TOKEN is missing or still a placeholder.${NC}"
echo "Set a strong random token in .env before starting."
exit 1
fi
if [ -z "$JWT_SECRET_KEY" ] || looks_like_placeholder "$JWT_SECRET_KEY" || [ "${#JWT_SECRET_KEY}" -lt 32 ]; then
echo -e "${RED}Error: JWT_SECRET_KEY is missing, too short (< 32 chars), or still a placeholder.${NC}"
echo "Generate one (e.g. python3 -c \"import secrets; print(secrets.token_urlsafe(32))\") and set it in .env."
exit 1
fi
echo -e "${GREEN} ✓ Required secrets are set${NC}"
# ----- Step 4: Start Docker Compose services -----
echo -e "${YELLOW}[4/6] Starting Docker Compose services...${NC}"