安全审计修复 + BUG 修复 + 新功能
审计基准 a3ae072。安全 14 项(严重 4、高危 10)全修:JWT 启动强制
校验(缺失/默认值/短于 32 位拒绝启动);默认 admin 改为
ADMIN_INITIAL_PASSWORD 必填(≥12 位);BMC/SMTP/飞书口令 Fernet
加密入库(新增 crypto.py,密钥走环境变量或 0600 文件,存量明文
启动时迁移),HostOut 与设置接口脱敏;InfluxDB 默认凭证清除、
Compose 改 ${VAR:?must be set}、start.sh 遇空/默认值退出;CORS
白名单;SPA 防遍历;生产关 /docs + 安全头;JWT 8h + jti + 登出/
改密(改密吊销全部旧 token);python-jose→PyJWT;requirements
全钉 == 版;SSRF 校验层(新增 ssrf.py);登录限流 5 次锁 15 分钟;
容器非 root、删源码挂载。
BUG 30 项全修:恢复误发、脏数据隔离、采集超时 30s→10s +
Semaphore(8) + 失败退避、SDR 解析、组件接口错文件、时区 8 小时、
InfluxDB 静默丢数据、密码明文返回、配置死变量、删 init_admin.py、
显式 commit、邮件 TLS/飞书 keyword、趋势功耗 fallback;剔除 -C 0
无认证与 IPMI 1.5 降级。
优化 11 项:按需查询、设置批量读、公共 resolve_total_power、
SQLite WAL/外键/索引、Dockerfile 加固、start.sh 轮询健康、
alert-trigger.sh 按真实接口重写、InfluxDB 超时、补测试 + README
重写(37 个端点)。
新功能 2 个:POST /api/auth/change-password(+ logout)、
GET /api/monitor/export CSV 导出;前端 Hosts/Settings 同步掩码展示。
验证:36/36 单元测试通过;py_compile 全过。未做真实 BMC/InfluxDB
联调与 Docker 构建验证。审计报告见 AUDIT.md 及三分报告。
This commit is contained in:
+33
-25
@@ -3,14 +3,18 @@
|
||||
# ============================================================
|
||||
# Copy this file to .env and modify values as needed:
|
||||
# cp .env.example .env
|
||||
#
|
||||
# SECURITY: values marked REQUIRED below have no default. The stack refuses
|
||||
# to start without them. Never commit a real .env file to git.
|
||||
# ============================================================
|
||||
|
||||
# ----- InfluxDB Configuration -----
|
||||
# InfluxDB admin username
|
||||
INFLUXDB_USERNAME=admin
|
||||
|
||||
# InfluxDB admin password (minimum 8 characters)
|
||||
INFLUXDB_PASSWORD=admin123456
|
||||
# InfluxDB admin password (REQUIRED, minimum 8 characters).
|
||||
# Leave empty here and set a strong password in your real .env file.
|
||||
INFLUXDB_PASSWORD=
|
||||
|
||||
# InfluxDB organization name
|
||||
INFLUXDB_ORG=ipmi-monitor
|
||||
@@ -18,8 +22,9 @@ INFLUXDB_ORG=ipmi-monitor
|
||||
# InfluxDB bucket name for storing metrics
|
||||
INFLUXDB_BUCKET=metrics
|
||||
|
||||
# InfluxDB admin API token (change this in production!)
|
||||
INFLUXDB_TOKEN=my-super-secret-token
|
||||
# InfluxDB admin API token (REQUIRED).
|
||||
# Leave empty here and set a strong random token in your real .env file.
|
||||
INFLUXDB_TOKEN=
|
||||
|
||||
# InfluxDB host port mapping (host:container)
|
||||
INFLUXDB_PORT=8086
|
||||
@@ -32,6 +37,30 @@ COMPOSE_PROJECT_NAME=ipmi-monitor
|
||||
# Backend server host port mapping (host:container)
|
||||
BACKEND_PORT=8000
|
||||
|
||||
# Runtime environment: development or production. Production disables the
|
||||
# interactive API docs (/docs, /redoc, /openapi.json).
|
||||
ENVIRONMENT=production
|
||||
|
||||
# JWT signing secret (REQUIRED, at least 32 characters).
|
||||
# Generate one, e.g.: python3 -c "import secrets; print(secrets.token_urlsafe(32))"
|
||||
# The backend refuses to start without it.
|
||||
JWT_SECRET_KEY=
|
||||
|
||||
# Password for the initial admin account (REQUIRED on first boot only, when
|
||||
# the user table is empty; at least 12 characters). Ignored once any user
|
||||
# exists. Change it after first login via the change-password API.
|
||||
ADMIN_INITIAL_PASSWORD=
|
||||
|
||||
# Optional Fernet key used to encrypt BMC passwords, the SMTP password and
|
||||
# the Feishu webhook URL stored in the database. When empty, the backend
|
||||
# generates a key once and persists it next to the SQLite file.
|
||||
# Generate one, e.g.: python3 -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"
|
||||
CREDENTIAL_ENCRYPTION_KEY=
|
||||
|
||||
# Optional comma-separated CORS allow-list (e.g. https://monitor.example.com).
|
||||
# Empty means CORS is disabled (same-origin deployment).
|
||||
CORS_ORIGINS=
|
||||
|
||||
# SQLite database URL used by backend. The default points to /app/data inside
|
||||
# the backend container, which is backed by the backend-data Docker volume.
|
||||
DATABASE_URL=sqlite+aiosqlite:///./data/ipmi_monitor.db
|
||||
@@ -51,16 +80,6 @@ POLL_INTERVAL=60
|
||||
# Log level: DEBUG, INFO, WARNING, ERROR, CRITICAL
|
||||
LOG_LEVEL=INFO
|
||||
|
||||
# ----- Alert Thresholds -----
|
||||
# Temperature alert threshold in °C (0 to disable)
|
||||
ALERT_TEMP_THRESHOLD=80
|
||||
|
||||
# Fan speed alert threshold in RPM (0 to disable, alert when below threshold)
|
||||
ALERT_FAN_THRESHOLD=0
|
||||
|
||||
# Power consumption alert threshold in Watts (0 to disable, alert when above threshold)
|
||||
ALERT_POWER_THRESHOLD=500
|
||||
|
||||
# ----- Email Alert Configuration -----
|
||||
# SMTP server host (leave empty to disable email alerts)
|
||||
SMTP_HOST=
|
||||
@@ -73,14 +92,3 @@ SMTP_USER=
|
||||
|
||||
# SMTP authentication password
|
||||
SMTP_PASSWORD=
|
||||
|
||||
# Sender email address for alerts
|
||||
ALERT_EMAIL_FROM=
|
||||
|
||||
# Comma-separated list of recipient email addresses for alerts
|
||||
ALERT_EMAIL_TO=
|
||||
|
||||
# ----- Webhook Alert Configuration -----
|
||||
# Webhook URL for alert notifications (leave empty to disable)
|
||||
# Supports Slack, DingTalk, WeChat Work, or generic webhooks
|
||||
WEBHOOK_URL=
|
||||
|
||||
+279
@@ -0,0 +1,279 @@
|
||||
# IPMI Monitor — BUG 与优化审计报告
|
||||
|
||||
**审计范围:** `~/workspace/work/ipmi-monitor`,HEAD `a3ae072`,只读审计;后端约 4000 行(`backend/app/`),前端 Vue(`frontend/src/`,`dist/` 未审)。测试仅 `backend/tests/test_total_power_fallback.py` 一个文件(3 个用例,只覆盖功耗 fallback)。
|
||||
|
||||
## 数量统计
|
||||
|
||||
| 分组 | 数量 |
|
||||
|---|---:|
|
||||
| 已确认的 BUG(B1–B30) | 30 |
|
||||
| 优化建议(O1–O16) | 16 |
|
||||
| 合计 | 46 |
|
||||
|
||||
---
|
||||
|
||||
## 一、已确认的 BUG(30 项)
|
||||
|
||||
### B1 — 未达持续时间就“恢复”:瞬时超标会误发恢复通知
|
||||
- **文件:** `backend/app/services/alert_engine.py:309-317`
|
||||
- **问题:** `check_alerts()` 的 `else` 分支只判断 `key in _first_breach`,不判断 `_alert_active`。只要曾经超标(哪怕未满 `rule.duration`、从未真正告警),下一次不超标就调用 `_create_recovery_notification()`
|
||||
- **复现/影响:** 温度短暂冲到 81℃ 又回落(`duration=300` 未满足),用户会收到一封“已恢复”邮件/飞书,却从未收到过告警,通知语义自相矛盾
|
||||
- **修复:** 仅当 `_alert_active.get(key)` 为真时才发恢复通知;未触发告警时只清理 `_first_breach`
|
||||
|
||||
### B2 — 告警循环中一次脏数据会中断全部规则检查
|
||||
- **文件:** `backend/app/services/alert_engine.py:272-276`
|
||||
- **问题:** `metric_value = point["value"]` 无判空/类型校验;Influx 返回 `None` 或非数值时,`evaluate_rule()` 做 `None > threshold` 抛 `TypeError`,整个 `check_alerts()` 中断,后续规则/主机本轮都不检查(外层 `scheduler.py` 只记一条 `Alert engine error` 日志)
|
||||
- **复现/影响:** 某台主机某传感器值为 `None`,全站告警本轮静默
|
||||
- **修复:** 取值后 `if metric_value is None: continue`,并用 `float()` 转换、捕获异常后按单条规则隔离(`try/except` 包住每个 `(rule, host)` 循环体)
|
||||
|
||||
### B3 — 采集超时不重试且子进程可能泄漏
|
||||
- **文件:** `backend/app/services/ipmi_collector.py:172, 207-215`
|
||||
- **问题:** `_run_ipmitool_with_auth()` 用 `asyncio.wait_for(proc.communicate(), timeout=30)`,超时抛 `TimeoutError`;但 `collect_host()` 内层只捕获 `RuntimeError`,超时直接跳出 6 种认证方式的重试循环。且超时后未 `proc.kill()`/`proc.wait()`,被取消的 `communicate()` 可能留下僵尸 `ipmitool` 进程
|
||||
- **复现/影响:** BMC 响应慢(30 秒超时)的主机本应试下一种认证方式,却直接判采集失败;多次超时后残留进程累积
|
||||
- **修复:** 内层捕获 `(RuntimeError, TimeoutError, asyncio.TimeoutError)`;超时后 `proc.kill()` 并 `await proc.wait()`;或改用 `asyncio.timeout()` + 显式清理
|
||||
|
||||
### B4 — 单主机最坏采集 180 秒,超过 60 秒采集间隔,调度会积压/跳过
|
||||
- **文件:** `backend/app/services/ipmi_collector.py:238-258`、`backend/app/services/scheduler.py:64-72`
|
||||
- **问题:** 6 种认证方式 × 每次 30 秒超时 = 单主机最坏 180 秒;`COLLECT_INTERVAL=60`,且 `max_instances=1`(无 `misfire_grace_time`/`coalesce`)。一台离线主机会拖住整轮采集,后续轮次被丢弃或排队
|
||||
- **复现/影响:** 1 台离线 + N 台在线时,在线主机的采集间隔被拉长到数分钟,`last_seen` 陈旧、Dashboard 显示离线
|
||||
- **修复:** 每种方式超时改为 5~10 秒,总预算封顶(如单主机 30 秒);失败主机记入退避(如下一轮跳过);`add_job` 加 `coalesce=True, misfire_grace_time=30`
|
||||
|
||||
### B5 — 负温度被解析成正数
|
||||
- **文件:** `backend/app/services/ipmi_collector.py:35-38`
|
||||
- **问题:** `_TEMP_RE = (?P<val>[0-9.]+)\s*degrees?\s*C` 不含负号和小数点外的分隔符,`-5 degrees C` 匹配到 `5`,返回 `+5.0`;`1,200 RPM` 同理只取到 `1`
|
||||
- **复现/影响:** 零下传感器、千分位读数全部错;阈值告警(低温告警)永远不触发
|
||||
- **修复:** 改为 `(?P<val>-?[0-9][0-9,]*(\.[0-9]+)?)`,解析时去掉 `,` 再 `float()`
|
||||
|
||||
### B6 — 状态类传感器恒返回“正常”,电源离散值被当成功耗
|
||||
- **文件:** `backend/app/services/ipmi_collector.py:63-77`
|
||||
- **问题:** `_classify_metric()` 的 `status` 分支不接收 `status` 参数,名字含 `status` 即返回 `("status", 1.0)`,哪怕 SDR 状态是 `nc/nr`(严重/不可恢复);`power supply` 分支对 `Presence detected` 等非数值 `int(...,0)` 抛 `ValueError` 后硬编码 `0.0`,把“在位”记成 0W 的 `power` 指标,污染功耗统计
|
||||
- **复现/影响:** 故障传感器在 Influx 里仍是 `1`;PSU 在位信息丢失,且多出 0/1W 的假功耗点
|
||||
- **修复:** 把 `status` 传进 `_classify_metric`,按 `ok→1,其余→0`;离散电源传感器应归为 `status` 类型而非 `power`
|
||||
|
||||
### B7 — 非 `ok/ns/nc/nr/nd/na` 的状态行解析错位
|
||||
- **文件:** `backend/app/services/ipmi_collector.py:95-118`
|
||||
- **问题:** `parse_sdr_output()` 只认 6 个状态 token;状态为 `disabled` 等时 `status_idx` 为 `None`,回退到 legacy 分支取 `parts[1]` 当值——对 5 段新格式行,`parts[1]` 是传感器编号(如 `01h`),值和状态全错,该传感器直接丢失
|
||||
- **复现/影响:** 含 `disabled` 传感器的 BMC,部分传感器在前端消失
|
||||
- **修复:** 按段数分流:5 段按新格式(`name|id|status|entity|value`)解析,3 段按 legacy;状态 token 白名单扩充 `disabled/cr/na` 等,未知状态也按位置解析而非回退
|
||||
|
||||
### B8 — `component_parser` 直接丢弃 legacy 3 段 SDR 行
|
||||
- **文件:** `backend/app/services/component_parser.py:42`
|
||||
- **问题:** `if len(parts) < 4: continue`。legacy 格式 `CPU1 Temp | 45 degrees C | ok` 只有 3 段,整行跳过;`ipmi_collector.py` 的 `parse_sdr_output` 却支持 legacy
|
||||
- **复现/影响:** 老 BMC 的组件(CPU/内存/PCIe/硬盘/风扇/PSU)全部显示 0,前端 Components 页为空
|
||||
- **修复:** `len(parts) < 3` 才跳过;3 段时 `value=parts[1]、status=parts[2]、entity_id=""`
|
||||
|
||||
### B9 — 组件去重缺失 + PSU/硬盘在位判断恒为假
|
||||
- **文件:** `backend/app/services/component_parser.py:339-430`
|
||||
- **问题:** (1)`_parse_drive/_parse_fan/_parse_power_supply` 无去重字典(CPU/内存/PCIe 有),同一传感器出现两次即重复计数;(2)`_parse_power_supply` 要求值里含 `presence detected/present`,但 5 段行值是瓦数(如 `132 Watts`),恒为 `present=False`,PSU 在位永远 0;(3)`_parse_drive` 要求名字含 `_`,注释里说的 `Drive 0`(无下划线)永远匹配不上
|
||||
- **复现/影响:** Components 页 PSU、硬盘计数错误;重复行导致总数虚高
|
||||
- **修复:** 三类解析加 `seen` 字典去重;PSU/硬盘的在位判断改用 SDR 的 `status` 段(`ok` 即在位),而非值字段;`Drive 0` 用正则 `^Drive\s+\d+$` 匹配
|
||||
|
||||
### B10 — `/api/monitor/components/{host_id}` 读了一个从不写入的文件
|
||||
- **文件:** `backend/app/routers/monitor.py:428-430`
|
||||
- **问题:** 该端点读 `/tmp/ipmi_raw.txt`(无主机后缀),但全仓库没有任何地方写这个文件——`collect_host()` 写的是 `/tmp/ipmi_raw_{host.id}.txt`(`ipmi_collector.py:275`)。旁边的 `/grouped` 端点(`:470`)用的才是带后缀的正确路径
|
||||
- **复现/影响:** 该端点永远返回 `{"error": "Raw IPMI data file not found"}`,且不分主机,任何主机都返回同一份(不存在的)数据
|
||||
- **修复:** 改为 `Path(f"/tmp/ipmi_raw_{host_id}.txt")`,与 `/grouped` 一致;或删掉该重复端点
|
||||
|
||||
### B11 — 历史接口时区错 8 小时,`parse_dt` 容错分支本身会崩
|
||||
- **文件:** `backend/app/routers/monitor.py:86-102, 383-396`
|
||||
- **问题:** (1)无时区参数默认 `datetime.now()`(容器时区 Asia/Shanghai,UTC+8),而 `influxdb.py:120-127` 把 naive 时间当 UTC 查,导致默认查的是“未来 8 小时前 1 小时”的窗口,最新 1 小时数据查不到;(2)`parse_dt` 的 `except ValueError` 分支 `dt_str.split('+')[0].split('-')[0][:19]` 对 `2026-11-16T10:00:00+08:00` 会切成 `2026`,`fromisoformat("2026")` 再抛 `ValueError`,返回 500。同一函数在两个端点复制粘贴了两份
|
||||
- **复现/影响:** History / Multi-history 默认时间范围查不到数据;传非法时间字符串直接 500
|
||||
- **修复:** 默认值用 `datetime.now(timezone.utc)`;`parse_dt` 失败时返回 `None` 或 400,不要二次 `fromisoformat`;抽成公共函数放一处
|
||||
|
||||
### B12 — 历史/汇总接口无输入校验、无上限,可被打爆
|
||||
- **文件:** `backend/app/routers/monitor.py:82, 145, 399`
|
||||
- **问题:** `host_ids` 直接 `[int(x) for x in host_ids.split(',')]`,`abc` 即 500;`metric_type` 未校验,直接拼进 Flux(见 B20);`query_metrics()` 无 `limit`/`aggregateWindow`,30 天范围全量点一次返回,前端/内存都可能被打爆
|
||||
- **复现/影响:** `GET /api/monitor/history/multi?host_ids=abc` 500;`time_range=30d` 的趋势接口返回数万点
|
||||
- **修复:** 用 Pydantic/Query 校验 `host_ids` 与 `metric_type`(白名单);`host_ids` 限长(如 ≤50);Flux 加 `|> limit(n: 5000)` 或按窗口聚合
|
||||
|
||||
### B13 — `dashboard/totals` 算出的分组汇总被丢弃,且 `None` 值会 500
|
||||
- **文件:** `backend/app/routers/monitor.py:219-302`
|
||||
- **问题:** (1)`group_totals` 算完(`:277-296`)后 `return`(`:298-302`)只返回 `hosts` 和 `groups`(id/name),分组总功耗/平均温度根本没给前端,是死代码;(2)`total_power = m.get('value', 0)` 若为 `None`,`if total_power > 0` 抛 `TypeError`,整个 Dashboard 500;(3)`avg_temperature` 实际是各主机 `max_temperature` 的平均,命名误导
|
||||
- **复现/影响:** 任一功耗传感器值为 `None` 即 Dashboard 全挂;分组汇总前端永远拿不到
|
||||
- **修复:** `return` 里加上 `group_totals`;取值时 `value or 0` 并 `float()` 转换;改名 `avg_max_temperature` 或按真实均值算
|
||||
|
||||
### B14 — Influx 写入/查询失败被静默吞掉,采集仍报成功
|
||||
- **文件:** `backend/app/influxdb.py:76-77, 99-100, 149-150`、`backend/app/services/ipmi_collector.py:321, 330-334`
|
||||
- **问题:** `write_metric/write_metrics_batch/query_*` 全部 `except Exception: logger.exception` 后返回 `None/[]`;`collect_host()` 不检查返回值,照常更新 `last_seen` 并置 `success=True`
|
||||
- **复现/影响:** InfluxDB 宕机期间数据全丢,但 `last_seen` 仍刷新、Dashboard 显示主机在线、告警无数据可查,无任何可见报错
|
||||
- **修复:** 写入失败应抛异常或返回布尔值,`collect_host()` 失败时不更新 `last_seen`、置 `success=False`;查询失败与“确实无数据”要区分(抛异常 vs 空列表)
|
||||
|
||||
### B15 — 同步 InfluxDB 调用阻塞整个事件循环
|
||||
- **文件:** `backend/app/influxdb.py:54-100`、`backend/app/routers/monitor.py:107, 149, 172, 219`、`backend/app/services/alert_engine.py:272`
|
||||
- **问题:** `InfluxDBClient` 是同步客户端(`SYNCHRONOUS` 写),却在 `async` 路由/采集协程里直接调用;一次慢查询会卡住所有请求和采集
|
||||
- **复现/影响:** Influx 慢(或超时无配置)时 API 全部卡死
|
||||
- **修复:** 写入/查询包进 `asyncio.to_thread()`(邮件已用此模式,`email_notify.py` 有先例),或换异步客户端;给客户端配超时
|
||||
|
||||
### B16 — 告警持续时间/冷却/激活态全放进程内存,重启即丢、无清理
|
||||
- **文件:** `backend/app/services/alert_engine.py:24-39, 282-317`
|
||||
- **问题:** `_first_breach/_last_notify/_alert_active` 三个全局 dict:重启后持续计时归零(刚超标的告警要重新等满 `duration`);规则/主机删除、禁用后状态永不清理,无限增长;多 worker 部署时各进程状态不一致
|
||||
- **复现/影响:** 重启后告警延迟一个 `duration`;长期运行内存缓慢增长
|
||||
- **修复:** 持久化到 SQLite(`alert_state` 表)或至少在 `check_alerts()` 开头清理已删除/禁用规则的状态;`duration` 计时以 Influx 首次超标时间点为准
|
||||
|
||||
### B17 — 规则同时绑主机和分组时分组被静默忽略;绑定已禁用主机仍会告警
|
||||
- **文件:** `backend/app/services/alert_engine.py:253-266`、`backend/app/routers/alerts.py:24-32`
|
||||
- **问题:** `check_alerts()` 优先 `host_id`,`group_id` 被忽略;`host_id` 分支不滤 `enabled`,而 `group_id`/全部主机分支滤 `enabled`,行为不一致。`RuleCreate` 也未校验两者互斥
|
||||
- **复现/影响:** 规则配了主机+分组,用户以为分组都覆盖,实际只查一台;禁用主机仍收告警
|
||||
- **修复:** 建/改规则时校验 `host_id` 与 `group_id` 至多其一;`host_id` 分支同样只查 `enabled` 主机
|
||||
|
||||
### B18 — 告警规则创建无任何值域校验
|
||||
- **文件:** `backend/app/routers/alerts.py:24-32`
|
||||
- **问题:** `condition`、`metric_type` 无白名单,`duration` 无 `ge=0`,`threshold` 可为 NaN。非法 `condition` 在 `evaluate_rule()`(`alert_engine.py:46-62`)只记 warning 返回 `False`,规则静默永不触发
|
||||
- **复现/影响:** 前端手误把 `condition` 写成 `greater`,规则建成功但永远不告警,无任何反馈
|
||||
- **修复:** `condition: Literal["gt","lt","eq","gte","lte"]`,`metric_type` 白名单,`duration: Field(ge=0)`,`threshold` 拒绝 NaN/Inf
|
||||
|
||||
### B19 — 告警历史先落库、后通知,且“已激活”标记与落库脱钩
|
||||
- **文件:** `backend/app/services/alert_engine.py:70-115, 282-296`
|
||||
- **问题:** (1)`_create_alert_history()` 先 `commit` 历史再发邮件/飞书,`notify_type` 按开关填写(`both/email/feishu`),与实际是否发出无关——SMTP 未配、飞书返回非 0 码时历史仍写“已通知”;(2)`_alert_active[key]=True` 在 `if host_obj:` 之外,即使主机已删、未建任何历史,也被标记已激活,之后只在冷却后才重试
|
||||
- **复现/影响:** 通知失败无法从历史里发现;主机删除间隙的一次告警永久丢失
|
||||
- **修复:** 通知结果回写历史(如 `notify_status`);仅当历史创建成功才置 `_alert_active`
|
||||
|
||||
### B20 — 密码明文落库且经 API 明文返回;设置接口明文返回 SMTP 密码
|
||||
- **文件:** `backend/app/routers/hosts.py:46-52`(`HostOut` 含 `ipmi_password`)、`backend/app/models.py:73-79`(明文列)、`backend/app/routers/settings.py:31-40`(`SettingsResponse` 含 `smtp_password`)
|
||||
- **问题:** 主机列表/创建/更新接口把 BMC 密码原样返回给任何登录用户;设置接口同理返回 SMTP 密码。另 `init_admin.py:43` 与 `database.py:99-109` 的默认管理员密码为硬编码值(此处不引述具体值)
|
||||
- **复现/影响:** 任一登录账号可导出全部 BMC 密码;日志/浏览器历史亦可能留存
|
||||
- **修复:** `HostOut` 去掉密码字段(更新时空密码表示不改,已有此逻辑);DB 密码改用 Fernet/密钥加密存储;设置接口返回掩码(如 `********`),只在用户填写新值时更新
|
||||
|
||||
### B21 — 配置名错位:`.env`/Compose 里一半变量根本没人读
|
||||
- **文件:** `backend/app/config.py:27-37`、`docker-compose.yml:44-60`、`.env.example:49-86`
|
||||
- **问题:** 代码只认 `COLLECT_INTERVAL`,而 Compose/`.env.example` 传的是 `POLL_INTERVAL`;代码认 `SMTP_FROM/FEISHU_WEBHOOK_URL`,Compose 传的是 `ALERT_EMAIL_FROM/WEBHOOK_URL`;`ALERT_TEMP_THRESHOLD/ALERT_FAN_THRESHOLD/ALERT_POWER_THRESHOLD/ALERT_EMAIL_TO` 全仓库无人读取(告警只走 DB 里的规则)
|
||||
- **复现/影响:** 用户按 `.env.example` 改采集间隔、阈值、收件人后毫无效果,误以为已生效
|
||||
- **修复:** 二选一:`Settings` 增加别名(`POLL_INTERVAL` 等)或删掉 Compose/`.env.example` 里的无效变量并在 README 注明“阈值在页面规则里配”
|
||||
|
||||
### B22 — `init_admin.py` 根本跑不起来
|
||||
- **文件:** `backend/app/init_admin.py:45-51`
|
||||
- **问题:** `async with engine.begin() as conn:` 拿到的是 `AsyncConnection`,没有 `.add()` 方法(应为 `AsyncSession`),执行即抛 `AttributeError`;且同一文件混用 `passlib`,而 `database.py`/`auth.py` 用原生 `bcrypt`,哈希实现不一致
|
||||
- **复现/影响:** 运行该脚本初始化管理员必失败(`database.py:init_db` 里的同类逻辑才是实际生效的,造成两份初始化逻辑并存)
|
||||
- **修复:** 改用 `async_session` 写库,或删掉该脚本只保留 `init_db`;统一用 `auth.py` 的 `hash_password`
|
||||
|
||||
### B23 — 删除接口依赖 `get_db` 的隐式提交,行为脆弱且不一致
|
||||
- **文件:** `backend/app/database.py:48-58`、`backend/app/routers/hosts.py:130-138`、`backend/app/routers/alerts.py:160-168`、`backend/app/routers/groups.py:105-113`
|
||||
- **问题:** `get_db()` 在 `yield` 后统一 `commit()`;删除接口只 `await db.delete(...)`,不 `flush/commit`,靠依赖收尾提交。创建/更新接口却显式 `flush/refresh`。一旦依赖被改、或在非请求上下文调用,删除即丢失;且 `GET` 请求也会走一次无意义的 `commit()`
|
||||
- **复现/影响:** 删除主机/规则/分组后若依赖逻辑变化即删不掉;读请求多一次提交开销
|
||||
- **修复:** 每个写接口显式 `await db.commit()`;`get_db` 只 `yield`,异常时 `rollback`,去掉成功后自动提交
|
||||
|
||||
### B24 — 告警邮件无视 `smtp_use_tls`,非 465 端口强制 STARTTLS
|
||||
- **文件:** `backend/app/services/email_notify.py:182-191, 238-247`
|
||||
- **问题:** `_send()` 里 `if smtp_port != 465: server.starttls()` 无条件执行,`smtp_use_tls`(设置页可配)根本没传进来;25 端口明文服务器、或不支持 STARTTLS 的服务器会直接失败。测试邮件(`settings.py:180-189`)却正确按开关处理,两处行为不一致
|
||||
- **复现/影响:** 用 25 端口明文 SMTP 的用户,测试邮件能通,真实告警邮件永远发不出
|
||||
- **修复:** `send_alert_email/send_recovery_email` 接收并使用 `smtp_use_tls`;仅当开关开且非 465 时才 `starttls()`
|
||||
|
||||
### B25 — 飞书关键词安全校验永远通不过
|
||||
- **文件:** `backend/app/services/alert_engine.py:126-150`、`backend/app/services/feishu_notify.py:13-80`
|
||||
- **问题:** 设置里有 `feishu_keyword`,测试飞书(`settings.py`)会把它拼进消息,但真实告警只读 `feishu_webhook_url`,`send_feishu_alert/send_feishu_recovery` 签名里根本没有 keyword 参数
|
||||
- **复现/影响:** 飞书机器人开启“关键词”安全设置时,真实告警被拒(仅记一条 warning),测试消息却能通,极具迷惑性
|
||||
- **修复:** 告警路径同样读取 `feishu_keyword` 并拼进卡片文本(或 `send_feishu_*` 增加 `keyword` 参数)
|
||||
|
||||
### B26 — 测试邮件接口:端口解析在 `try` 外、SMTP 阻塞且可能泄漏连接
|
||||
- **文件:** `backend/app/routers/settings.py:163, 175-191`
|
||||
- **问题:** `smtp_port = int(await get_setting(db, "smtp_port"))` 在 `try` 之前,DB 里存了非数字即 500(`smtp_port` 是自由字符串字段,无校验);`smtplib.SMTP(host, port)` 同步调用且无 `timeout`,在 `async` 端点里可无限挂起;`server.quit()` 只在成功路径,`login/send_message` 抛异常即泄漏连接
|
||||
- **复现/影响:** 设置页填错端口字符串即 500;SMTP 服务器不响应时整个 API 卡住
|
||||
- **修复:** `int()` 移入 `try` 并校验范围;`SMTP(..., timeout=15)` 并用 `asyncio.to_thread()`;`quit()` 放 `finally`(或用 `with smtplib.SMTP(...) as server`)
|
||||
|
||||
### B27 — 告警邮件模板缺字段即崩,且与飞书通知互相复制
|
||||
- **文件:** `backend/app/services/email_notify.py:166-168, 222-224`、`backend/app/services/feishu_notify.py:13-148`
|
||||
- **问题:** `_PLAIN_TEMPLATE.format(**alert_info)` 缺任一 key 即 `KeyError`(`send_*` 外无 `try`,靠调用方兜底);HTML 模板用字符串 `replace`,缺字段会残留 `{{ ... }}`。邮件告警/恢复、飞书告警/恢复各约百行重复,仅模板不同
|
||||
- **复现/影响:** 调用方少传一个字段,邮件直接不发;改一处模板要改两处
|
||||
- **修复:** 用 `string.Template.safe_substitute` 或 Jinja2(依赖里已有 `jinja2`);抽公共 `_send_email/_send_feishu`
|
||||
|
||||
### B28 — SPA 兜底路由掩盖 API 404,且未防路径穿越
|
||||
- **文件:** `backend/app/main.py:115-124`
|
||||
- **问题:** `@app.get("/{full_path:path}")` 对所有未匹配路径返回 `index.html`(200),包括 `/api/xxx` 打错的路径,前端 `api.js` 收到 HTML 当 JSON 解析失败,报错信息失真;`file_path = _frontend_dist / full_path` 未 `resolve()`/校验 `is_relative_to(_frontend_dist)`,`../` 可拼出 dist 之外的路径
|
||||
- **复现/影响:** API 404 变 200+HTML;存在越界读取静态目录外文件的风险
|
||||
- **修复:** 兜底路由排除 `/api/` 前缀(返回 404);`file_path.resolve()` 后校验在 `_frontend_dist` 内,否则 404
|
||||
|
||||
### B29 — CORS `allow_origins=["*"]` 配 `allow_credentials=True`
|
||||
- **文件:** `backend/app/main.py:65-71`
|
||||
- **问题:** 通配符源 + 允许凭证组合,浏览器本应拒绝,实际等于把 JWT Bearer 的跨站风险敞开
|
||||
- **修复:** 收敛为前端实际域名白名单;开发环境才放宽
|
||||
|
||||
### B30 — 趋势接口的功耗过滤把 PSU fallback 主机排除在外
|
||||
- **文件:** `backend/app/routers/monitor.py:340-350`
|
||||
- **问题:** `trend` 对 `power` 只保留 `Total_Power/Power/Pwr Consumption` 三个名字,`PSU{N}_PIN` 的点被丢弃——而这正是 Gooxi12 等无原生功耗传感器主机的唯一功耗来源(`component_parser.py` 的 fallback 和 `dashboard/totals` 都认 PSU_PIN)
|
||||
- **复现/影响:** Gooxi12 的功耗趋势图永远是空的,与 Dashboard 总功耗有数矛盾
|
||||
- **修复:** 趋势的功耗分支同样做“原生优先、否则 PSU_PIN 求和”的聚合(或在入库时多写一条合成的 `Total_Power` 点)
|
||||
|
||||
---
|
||||
|
||||
## 二、优化建议(16 项)
|
||||
|
||||
### O1 — 每次前端轮询触发 4×N 次 Influx 查询
|
||||
- **文件:** `backend/app/influxdb.py:209-214`、`backend/app/routers/monitor.py:149, 172, 219`
|
||||
- **问题:** `query_latest_grouped()` 对每台主机固定查 4 种类型(temperature/fan/voltage/power),即使调用方只要 1 种(如 `/summary/temperature`);Dashboard 每 10 秒轮询一次,50 台主机即每轮 200 次 Flux 查询
|
||||
- **建议:** `query_latest_grouped(host_id, only: set[str] | None)` 按需查;或一次 Flux 用 `|> filter(fn: (r) => contains(value: r._measurement, set: [...]))` 合并
|
||||
|
||||
### O2 — 历史/趋势无分页、无降采样、无上限
|
||||
- **文件:** `backend/app/influxdb.py:102-148`、`backend/app/routers/monitor.py:300-362`
|
||||
- **问题:** Flux 无 `limit`/`aggregateWindow`;30 天趋势全量点返回,前端 `processTrendToSeries`(`Dashboard.vue:770-800`)再按时间戳去重取最大,内存与带宽浪费
|
||||
- **建议:** 按 `time_range` 选聚合窗口(1h→1m,30d→1h),`aggregateWindow(every: v.windowPeriod, fn: mean)`,服务端限流 5000 点
|
||||
|
||||
### O3 — 多处 N+1 式逐条查库
|
||||
- **文件:** `backend/app/services/alert_engine.py:126-132, 155-161`(SMTP 5 个 key 各 1 次查询,告警与恢复各一份)、`backend/app/routers/settings.py:104-108`(8 个 key 各 1 次)、`backend/app/routers/monitor.py:35-54`(Dashboard 4 次 `count`)
|
||||
- **建议:** 一次 `select(...).where(key.in_(keys))` 批量取;Dashboard 用单条 `select(func.count(), ...)` 或一次聚合
|
||||
|
||||
### O4 — 采集无并发上限、无退避
|
||||
- **文件:** `backend/app/services/scheduler.py:38-44`
|
||||
- **问题:** `asyncio.as_completed([collect_host(h) for h in hosts])`,主机数无上限时同时起 N 个 `ipmitool` 子进程;失败主机每轮都重试 6 次
|
||||
- **建议:** `asyncio.Semaphore(5~10)`;连续失败主机指数退避(如失败 3 次后跳过 5 轮)
|
||||
|
||||
### O5 — 总功耗 fallback 逻辑在三处各写一份,且 0 值语义不一致
|
||||
- **文件:** `backend/app/services/component_parser.py:433-480`、`backend/app/routers/monitor.py:230-256`、`frontend/src/views/Sensors.vue:250-270, 380-400`
|
||||
- **问题:** 后端两处把原生读数 `0` 当“缺失”进而用 PSU_PIN 求和,`Sensors.vue` 却把 `0` 当权威值(`priority` 已占位即不再累加),同一主机 Dashboard 与 Sensors 页总功耗不一致
|
||||
- **建议:** 抽成后端唯一函数 `resolve_total_power(power_metrics)`,三处共用;明确 `0` 是否为有效读数(建议以“传感器存在即有效”为准)
|
||||
|
||||
### O6 — 重复代码遍布
|
||||
- **文件:** `monitor.py:86-102` 与 `:383-396`(`parse_dt` 两份)、`ipmi_collector.py:149-175` 与 `:178-214`(`_run_ipmitool` 已标注 DEPRECATED 仍保留)、`hosts.py:146-190` 与 `collect_host` 的 6 元组 attempts(第三份)、`alert_engine.py` 的告警/恢复通知各两份
|
||||
- **建议:** 抽公共函数/常量 `IPMI_AUTH_ATTEMPTS`、`parse_dt()`、`_notify()`
|
||||
|
||||
### O7 — 魔法数字散落各处
|
||||
- **文件:** 前端 `Sensors.vue:234-240`(75/55℃、800/400W)、`Dashboard.vue:246`(轮询默认 10 秒)、`monitor.py:43`(在线阈值 2 倍间隔)、`alert_engine.py:39`(冷却 3 小时)、`ipmi_collector.py:172`(超时 30 秒)
|
||||
- **建议:** 集中到 `config.py`/常量模块,温度阈值应与告警规则联动而非前端写死
|
||||
|
||||
### O8 — 前端轮询 10 秒 vs 后端采集 60 秒,且图表用轮询时间而非传感器时间
|
||||
- **文件:** `frontend/src/views/Dashboard.vue:246, 714`、`frontend/src/views/Sensors.vue:427-430`、`frontend/src/views/Dashboard.vue:670-690`
|
||||
- **问题:** 每 10 秒拉一次,但数据 60 秒才变一次,6 次里 5 次是重复查询;图表横轴用 `Date.now()` 拼点,数据陈旧时曲线仍“前进”,误导为实时
|
||||
- **建议:** 默认轮询改为 30~60 秒(与 `COLLECT_INTERVAL` 对齐);图表用 Influx 返回的 `time` 而非 `Date.now()`;超 `2×COLLECT_INTERVAL` 未更新即断线显示
|
||||
|
||||
### O9 — SQLite 无 WAL、无外键、无关键索引
|
||||
- **文件:** `backend/app/database.py:14-17`、`backend/app/models.py:77-127`
|
||||
- **问题:** 未配 `journal_mode=WAL`/`busy_timeout`,采集(`last_seen` 更新)与告警写库并发时易 `database is locked`;SQLite 默认外键关闭,`ForeignKey` 不生效;`AlertHistory(host_id, rule_id, created_at)`、`Host(group_id)`、`AlertRule(host_id, group_id)` 均无索引,历史分页 `ORDER BY created_at DESC` 全表扫描
|
||||
- **建议:** `create_async_engine(connect_args={"timeout": 30})` + 启动时 `PRAGMA journal_mode=WAL; PRAGMA foreign_keys=ON`;补索引;删除主机/分组时级联或先清关联规则/历史
|
||||
|
||||
### O10 — 认证:无登录限流、无改密/登出,JWT 有效期用 naive 时间
|
||||
- **文件:** `backend/app/routers/auth.py:28-32, 62-66, 140-170`
|
||||
- **问题:** 登录无速率限制/锁定,可暴力破解;无修改密码、登出(文档称有 logout,实际只有 login/verify/me)接口,默认管理员密码无法通过页面修改;`verify_password` 对非 bcrypt 格式哈希抛 `ValueError`(应为 401,却成 500);`create_access_token` 用 `datetime.now()`(本地 naive)算 `exp`,涉及时区歧义
|
||||
- **建议:** 登录限流(如 5 次/分钟/IP+用户名);加改密接口与 token 吊销/黑名单;`verify_password` 捕获异常返回 `False`;`exp` 用 `datetime.now(timezone.utc)`
|
||||
|
||||
### O11 — Docker Compose 体积与一致性问题
|
||||
- **文件:** `docker-compose.yml:1-70`、`docker-compose.dev.yml:1-66`、`backend/Dockerfile:31-38`
|
||||
- **问题:** (1)`influxdb-data`、`backend-dev-data` 命名卷已声明但从未挂载,实际用 `./data/influxdb` 绑定目录,卷成摆设;(2)`dev` 用 `network_mode: host`,`ports` 声明失效,且 `PORT=8003` 与 Dockerfile 的 `HEALTHCHECK http://localhost:8000` 冲突;(3)`version: "3.8"` 已废弃;(4)后端无 `healthcheck`,只依赖 Influx 健康;(5)Influx 8086 直接映射到宿主机,无鉴权隔离
|
||||
- **建议:** 删未用卷或改挂命名卷;`dev` 去掉 `host` 模式改端口映射;健康检查用 `${PORT:-8000}`;`version` 字段删除;生产不暴露 8086 或加防火墙
|
||||
|
||||
### O12 — Dockerfile:单 worker、无非 root 用户、健康检查写死端口
|
||||
- **文件:** `backend/Dockerfile:1-38`
|
||||
- **问题:** `uvicorn` 单进程(采集+API 同进程,一个慢请求即拖垮采集);以 root 运行;pip 镜像写死阿里云(海外/内网环境可能不可达);`HEALTHCHECK` 写死 8000,与 `${PORT}` 不一致
|
||||
- **建议:** `uvicorn --workers 2`(注意 APScheduler 需单例,worker>1 时采集会重复——更稳妥是采集拆独立进程/只在 1 个 worker 跑调度);加 `USER app`;健康检查读 `$PORT`;镜像源改为构建参数
|
||||
|
||||
### O13 — `start.sh` 先 `down` 再 `up`,且只验 Influx 不验后端
|
||||
- **文件:** `start.sh:76-77, 90-110`
|
||||
- **问题:** 每次启动都 `docker compose down --remove-orphans`,会停掉正在跑的服务再重建(中断+重建镜像),且未跑 `scripts/preflight-data.sh` 的数据盘点;只等 Influx `/health`,后端是否起来、DB 是否为空无人验;`grep "^INFLUXDB_PORT="` 解析 `.env` 在值含空格/引号时出错
|
||||
- **建议:** 默认用 `up -d --build`(不 `down`);启动后轮询 `/api/health`;接入 `preflight-data.sh`
|
||||
|
||||
### O14 — `alert-trigger.sh` 已完全失效
|
||||
- **文件:** `alert-trigger.sh:3-10, 18-60`
|
||||
- **问题:** 注释用法是 `--api-url ...`,代码却按位置参数 `$1/$2` 取,`--api-url` 会被当成 URL;调的 `/api/alerts` 端点不存在(实际是 `/api/alerts/history`,且需 Bearer 鉴权、返回分页对象而非数组);解析的 `.host/.severity/.sensor_type` 字段 API 从不返回;`ssh -o StrictHostKeyChecking=no root@...` 以 root 直连;无 `set -euo pipefail`、无 `jq` 存在性检查;`while read` 在管道子 shell 里,状态文件读写有竞态
|
||||
- **建议:** 按 `--api-url/--ssh-key` 解析参数;改调 `/api/alerts/history?acknowledged=false` 并带 token;字段按实际返回改;SSH 改专用低权账号+密钥校验;或直接废弃该脚本
|
||||
|
||||
### O15 — Influx 客户端无超时、无重试、单例无锁
|
||||
- **文件:** `backend/app/influxdb.py:19-44, 217-224`
|
||||
- **问题:** `InfluxDBClient` 未配超时/重试;`get_client/get_write_api/get_query_api` 的懒创建无锁,并发首次调用可能建出两个客户端泄漏一个;`close()` 只在 lifespan 调,异常退出不保证执行
|
||||
- **建议:** 建客户端时配 `timeout`;启动时(lifespan)预创建而非懒创建;写失败按指数退避重试 2~3 次
|
||||
|
||||
### O16 — 测试与文档缺口
|
||||
- **文件:** `backend/tests/test_total_power_fallback.py`(唯一测试)、`README.md`
|
||||
- **问题:** 仅 3 个用例覆盖功耗 fallback;`parse_sdr_output`、`evaluate_rule`(尤其 B1 的恢复逻辑)、`parse_dt`、时区换算、`component_parser` 的 legacy/去重分支全无测试;README 仍称前端为“纯静态 HTML/JS”,与 Vue 现状不符
|
||||
- **建议:** 补单测:SDR 新/旧格式、负温度、超时、`eq` 浮点容差、恢复仅在已激活时、历史分页;README 更新技术栈与配置说明(尤其 B21 的无效变量)
|
||||
|
||||
## 最优先修的 5 项
|
||||
|
||||
B1(误发恢复)、B2(脏数据中断全部告警)、B14(Influx 宕机静默丢数据)、B20(密码明文返回)、B4+B3(采集超时拖垮调度)。其次是 B11(时区错 8 小时)、B10(组件接口读错文件)、B21(无效配置变量误导用户)。
|
||||
@@ -0,0 +1,168 @@
|
||||
# IPMI Monitor 功能审计报告
|
||||
|
||||
- 审计对象:`~/workspace/work/ipmi-monitor`,HEAD `a3ae072`,只读审计,未改任何文件
|
||||
- 实际规模:后端约 3936 行(`backend/app/`:`main.py`、`config.py`、`database.py`、`influxdb.py`、`models.py`、`routers/` 6 个、`services/` 6 个);前端 Vue 3(`frontend/src/`,9 个视图 + 1 个组件);`frontend/dist/` 为构建产物,未审
|
||||
- 总体结论:**核心链路(采集→存储→展示→按规则告警→邮件/飞书)是通的,但 README 明显滞后于代码**:README 的 API 表整表过期、技术栈表写错前端、通知渠道与环境变量多处宣称与实现不符;另有 8 个后端接口写好却无前端入口、1 个根脚本完全跑不通、若干“设置双轨制”和“临时文件依赖”等半成品
|
||||
|
||||
---
|
||||
|
||||
## 一、README 宣称功能逐项核对
|
||||
|
||||
### 1. 多服务器管理 — ✅ 已实现(但无批量操作)
|
||||
|
||||
- 依据:`backend/app/routers/hosts.py`(GET/POST/PUT/DELETE `/api/hosts`、启用开关、`POST /{id}/test` 连接测试,6 种 lanplus/lan 回退);`frontend/src/views/Hosts.vue`(增删改、按分组筛选、客户端搜索、连接测试);分组 CRUD 在 `backend/app/routers/groups.py` + `Groups.vue`
|
||||
- 缺口:README“服务器管理”截图宣称的“批量操作”不存在——无批量启用/删除/测试/导入接口,前端也无多选操作
|
||||
|
||||
### 2. 实时监控(温度/风扇/电压/功耗)— ⚠️ 部分实现
|
||||
|
||||
- 已实现:`backend/app/services/scheduler.py`(APScheduler 定时 `collect_all_hosts`)+ `ipmi_collector.py`(`ipmitool sdr elist` 解析,`_classify_metric` 分 4 类)+ `influxdb.py`(`write_metrics_batch`);`Sensors.vue` 按 4 类指标展示最新值
|
||||
- 部分原因:
|
||||
- “实时”是 60 秒轮询,无 WebSocket/SSE;`Dashboard.vue`、`Sensors.vue` 的自动刷新默认都是关闭的,需手动打开
|
||||
- 传感器“当前值”只读 InfluxDB 最近 1 小时(`influxdb.py: query_latest` 的 `range(start: -1h)`),超 1 小时无采集即显示空白而非“离线”
|
||||
|
||||
### 3. 历史趋势(InfluxDB)— ⚠️ 部分实现
|
||||
|
||||
- 已实现:`influxdb.py`(`query_metrics`/`query_trend`)、`monitor.py`(`GET /api/monitor/trend`,支持 1m–30d)、`Dashboard.vue`(温度/功耗两张 ECharts 趋势图,时间范围 5 分钟–30 天)
|
||||
- 部分原因:
|
||||
- `Sensors.vue`(697 行)只有最新值表格,**没有任何趋势图**;README 截图宣称的“电压稳定性图表”全站不存在,功耗趋势图只在 Dashboard 有,且只认 `Total_Power`/`Power`/`Pwr Consumption` 三个传感器名(`monitor.py` 的 `/trend`),无原生总功耗的机型(如 Gooxi12)在趋势图里是空的——而 Dashboard 汇总里刚修好的 PSU_PIN 求和 fallback(HEAD `a3ae072`)并没有同步到 `/trend`
|
||||
- 无聚合/降采样、无 retention 策略配置,InfluxDB 会无限增长
|
||||
|
||||
### 4. 智能告警(可配阈值)— ⚠️ 部分实现
|
||||
|
||||
- 已实现:`routers/alerts.py`(规则 CRUD、按主机/分组/全局、条件 gt/lt/eq/gte/lte、`duration` 持续时长、3 小时通知冷却、恢复通知)、`services/alert_engine.py`(`check_alerts` 在每次采集后运行)、`AlertRules.vue`、`AlertHistory.vue`(分页、按主机/确认状态筛选、确认、批量确认)
|
||||
- 部分原因:
|
||||
- README 环境变量表里的 `ALERT_TEMP_THRESHOLD` / `ALERT_FAN_THRESHOLD` / `ALERT_POWER_THRESHOLD` 是**死变量**:`config.py` 里根本没有这三个字段,`docker-compose.yml` 传入后无人读取,全局默认阈值不存在,只有逐条建规则才生效;README“告警触发条件”一节描述的其实是已不存在的旧逻辑
|
||||
- 无严重级别(severity);README 截图宣称的“告警静默操作”不存在,只有“确认”(acknowledge),且确认后规则仍会继续发恢复/重复通知
|
||||
- 告警状态(`_first_breach`/`_alert_active`/`_last_notify`)全是 `alert_engine.py` 里的进程内 dict,重启即丢:重启后持续超标的告警会重新触发,持续时长计时归零
|
||||
|
||||
### 5. 多渠道通知(邮件、Webhook:Slack/钉钉/企业微信)— ⚠️ 部分实现
|
||||
|
||||
- 已实现(邮件 + 飞书):`services/email_notify.py`(HTML 邮件)、`services/feishu_notify.py`(飞书机器人)、`routers/settings.py`(DB 存储的 SMTP/飞书配置 + `POST /api/settings/test-email`、`/test-feishu`)、`Settings.vue`(飞书/邮件两个 Tab)
|
||||
- 未实现(README 宣称的三个渠道一个都没有):
|
||||
- Slack / 钉钉 / 企业微信:代码里无任何相关发送逻辑;README 让配的 `WEBHOOK_URL` 环境变量在 `config.py` 中不存在、**全代码库无人读取**,属于纯文档功能
|
||||
- 反而是已实现的**飞书**在 README 功能列表里只字未提(只在任务描述里出现),README 与实现互相错位
|
||||
- 另有设置双轨制:告警实际用的是 DB 里 `SystemSetting` 的 SMTP 配置(`alert_engine.py` 每次现查),`config.py` 的 `SMTP_HOST`/`SMTP_USER` 等环境变量只在 `email_notify.py` 无参回退时用,实际告警路径走不到;`test-email` 端点还绕开 `email_notify.py` 自己内联实现了一套 SMTP 发送
|
||||
|
||||
### 6. 一键部署(Docker Compose)— ✅ 已实现(带 2 个失效配置)
|
||||
|
||||
- 依据:`docker-compose.yml`(influxdb 2.7 + backend,healthcheck、命名卷 `backend-data`)、`start.sh`(6 步检查)、`scripts/preflight-data.sh`、README 的生产数据保护章节(`REQUIRE_EXISTING_DATABASE` / `MIN_EXPECTED_HOSTS`,`database.py: init_db` 真实校验)均与实现一致
|
||||
- 失效配置:compose 传入的 `POLL_INTERVAL` 无效——`config.py` 的字段名是 `COLLECT_INTERVAL`,pydantic-settings 不会把 `POLL_INTERVAL` 映射过去,采集间隔恒为 60 秒,改 `.env` 也改不动;`ALERT_*`、`WEBHOOK_URL`、`ALERT_EMAIL_*` 同理是死变量(见第 4、5 条)
|
||||
- 另:`INFLUXDB_BUCKET` 默认值两处不一致——`config.py` 默认 `ipmi_metrics`,`.env.example` / compose 默认 `metrics`,本地直接跑后端与 Docker 跑会写进不同 bucket
|
||||
|
||||
### 7. 响应式界面(纯静态前端,适配桌面和移动)— ⚠️ 部分实现(且技术描述错误)
|
||||
|
||||
- 部分实现:Element Plus 栅格在 Dashboard/Sensors 用了 `xs/sm/md/lg` 断点,`index.html` 有 viewport;但 `Components.vue` 统计行写死 `:span="6"`、各表格多列 `min-width` 固定,小屏需横向滚动,无移动端专项适配证据
|
||||
- 描述错误:README 技术栈表写“前端 HTML / CSS / JavaScript(纯静态)”,实际是 **Vue 3.4 + Vue Router + Element Plus + ECharts + Vite**(`frontend/package.json`),且由 FastAPI 以 SPA 方式托管(`main.py` 的 catch-all 路由)
|
||||
|
||||
### 8. API 驱动(完整 RESTful API,支持二次开发)— ⚠️ 部分实现
|
||||
|
||||
- 已实现:FastAPI + Swagger(`/docs`)、JWT 鉴权(`routers/auth.py`:login/verify/me)、6 组路由共 33 个端点
|
||||
- 部分原因:
|
||||
- **README 的 API 表 14 个端点无一路径正确**:真实前缀是 `/api/hosts`、`/api/monitor/*`、`/api/alerts/*`、`/api/groups`、`/api/settings`、`/api/auth`,而 README 写的是 `/api/servers`、`/api/metrics/*`、`/api/dashboard`、`PUT /api/alerts/config`——按 README 调用的二次开发会全部 404
|
||||
- 无 API Key / 服务账号机制,只有用户登录 JWT;无速率限制、无分页(除告警历史外所有列表接口都不分页)
|
||||
|
||||
### 9. README 截图章节宣称的页面 — ❌ 多处不存在
|
||||
|
||||
| 截图宣称 | 实际情况 |
|
||||
|---|---|
|
||||
| 服务器详情页(单台传感器列表、1h/6h/24h/7d 温度趋势、风扇/电压/功耗历史图) | **无此页面**:`frontend/src/router/index.js` 只有 9 条路由,无 `/hosts/:id` 详情页;单主机历史接口 `GET /api/monitor/history/{id}` 已写好但前端从未调用 |
|
||||
| 仪表盘“最近告警摘要” | 只有未确认告警**数量**卡片(`monitor.py: /dashboard` 返回 `active_alerts` 计数),无摘要列表;Dashboard 不调告警历史接口 |
|
||||
| 告警管理“通知渠道设置、告警确认/静默” | 通知渠道设置只有飞书+邮件(Settings.vue);静默不存在,只有确认 |
|
||||
| 服务器管理“批量操作” | 不存在(见第 1 条) |
|
||||
|
||||
### 10. 其他 README 内容
|
||||
|
||||
- 快速开始/手动部署/systemd/FAQ:基本属实,但 `systemd` 示例未提需先建 DB 与默认账号;FAQ 没提本项目最常见的真实故障(`/tmp` 原始数据文件丢失导致组件页报错,见第二节)
|
||||
- 技术栈表遗漏:数据库实际是 **SQLite(SQLAlchemy 存用户/主机/分组/规则/历史)+ InfluxDB** 两套,表里只写了 InfluxDB
|
||||
|
||||
---
|
||||
|
||||
## 二、缺失 / 半成品 / 断链功能清单
|
||||
|
||||
### A. 后端写了、前端没入口(共 8 个接口,白写)
|
||||
|
||||
| 接口 | 位置 | 说明 |
|
||||
|---|---|---|
|
||||
| `GET /api/monitor/realtime/{host_id}` | `routers/monitor.py` | 返回分组最新指标;无任何视图调用 |
|
||||
| `GET /api/monitor/history/{host_id}` | 同上 | 单主机历史查询;无调用(详情页缺失的直接原因) |
|
||||
| `GET /api/monitor/history/multi` | 同上 | 多主机历史对比;无调用 |
|
||||
| `GET /api/monitor/sensors/{host_id}` | 同上 | 单主机全量传感器;无调用(Sensors 页走的是 `/summary` + `/dashboard/totals`) |
|
||||
| `GET /api/hosts/{host_id}/status` | `routers/hosts.py` | 与 realtime 基本重复;无调用 |
|
||||
| `GET /api/monitor/components/{host_id}`(非 grouped 版) | `routers/monitor.py` | 无调用,只有 `/grouped` 版被 Components.vue 用 |
|
||||
| `POST /api/auth/verify`、`GET /api/auth/me` | `routers/auth.py` | 前端只用 login,登录态只靠 localStorage token + 路由守卫,未调 verify/me 校验 |
|
||||
|
||||
反向检查:前端所有 `api.*` 调用均有真实后端路由,**前端调不存在 API 的情况为零**;断链方向是反的(后端多余、README 过期)。
|
||||
|
||||
### B. README 宣称但代码里没有
|
||||
|
||||
1. `POST /api/servers/{id}/scan`(立即扫描):不存在;最接近的是 `POST /api/hosts/{id}/test`,但它只做 SDR 预览(返回前 20 条),**不写 InfluxDB、不更新 `last_seen`**,不能替代手动采集
|
||||
2. Slack / 钉钉 / 企业微信通知(见一、5)
|
||||
3. 全局阈值 `ALERT_*`(见一、4)、批量操作、告警静默、服务器详情页(见一、9)
|
||||
|
||||
### C. 半成品 / 脆弱实现(能跑,但随时会断)
|
||||
|
||||
1. **`alert-trigger.sh` 完全跑不通**:调 `GET /api/alerts`(无此路由,真实为 `/api/alerts/history` 且需 JWT 鉴权),解析 `.severity` / `.host` / `.sensor_type` 字段(真实字段是 `message` / `host_name` / `metric_name`,无 severity),默认端口 8003 与实际 8000 不符;且含 SSH 直连告警主机的逻辑,未被任何文档/部署引用
|
||||
2. **组件在位监控依赖 `/tmp` 临时文件**:`ipmi_collector.py` 把每次采集的原始 SDR 写进 `/tmp/ipmi_raw_{host_id}.txt`,`monitor.py` 的组件接口直接读该文件;容器重启、`/tmp` 清理、或主机尚未采集过,都会返回 `error: Raw IPMI data file not found`,Components.vue 只弹错、页面留空。更甚者,非 grouped 版读的是**不分主机的** `/tmp/ipmi_raw.txt`(`monitor.py:430`),存在串主机数据的隐患。采集时写入 InfluxDB 的 `component` measurement(`ipmi_collector.py`)则**没有任何接口读回**,属于只写不读
|
||||
3. **删主机留孤儿数据**:`models.py` 的外键无 `cascade`,`routers/hosts.py: delete_host` 只删 Host 行,不删其 AlertRule / AlertHistory,InfluxDB 历史也不清;删分组时有保护(组内有主机则 400),删主机没有等价保护
|
||||
4. **认证与账号管理缺口**:全站只有一个 `admin` 账号(`database.py: init_db` 启动时自动创建,使用公开写死的默认口令——值略去,见代码);**无改密、无新增用户、无角色/只读账号**;`backend/app/init_admin.py` 是死代码(从未被 import,建账号逻辑与 `init_db` 重复);JWT 密钥、`CORS allow_origins=["*"] + allow_credentials=True`(`main.py`)均为宽松默认值
|
||||
5. **采集间隔改不动**:`POLL_INTERVAL` 死变量(见一、6);设置页(Settings.vue)只有飞书+邮件两个 Tab,**采集间隔、全局阈值、Webhook、数据保留期均无界面可改**
|
||||
6. **离线主机无告警**:在线状态只按 `last_seen` 在 Dashboard 算个数(`monitor.py: /dashboard`),主机掉线(采集失败)时没有任何告警规则能触发——`alert_engine.py` 只对有最新指标值的传感器做判断,`query_latest` 查不到就直接跳过
|
||||
7. **测试覆盖极薄**:`backend/tests/` 仅 1 个测试文件(总功耗 fallback 回归),routers / alert_engine / auth / settings 均无测试
|
||||
|
||||
---
|
||||
|
||||
## 三、值得加的新功能(按价值从高到低)
|
||||
|
||||
### 1. 主机详情页 + 单传感器历史趋势图(价值最高:补最大断链,成本最低)
|
||||
|
||||
- **痛点**:README 承诺的详情页不存在;用户现在只能看全站汇总,想定位“某台机某传感器何时开始升温”没有任何入口;后端 `history` / `realtime` / `sensors` 三个接口已写好闲置
|
||||
- **实现点**:前端 `router/index.js` 加 `/hosts/:id` 路由,新建 `views/HostDetail.vue`(复用 Dashboard 的 ECharts 趋势组件 + `Sensors.vue` 的表格),调现有 `GET /api/monitor/sensors/{id}`、`GET /api/monitor/history/{id}`、`GET /api/monitor/realtime/{id}`;Hosts.vue 表格加“详情”入口;后端零改动
|
||||
|
||||
### 2. 告警静默 / 维护窗口 + 严重级别
|
||||
|
||||
- **痛点**:上架、换件、固件升级期间必然触发误报,现在只能关规则(事后还得记得开)或忍受 3 小时一次的重复通知;无 severity 导致所有告警看起来一样重
|
||||
- **实现点**:`models.py` 给 AlertRule 加 `severity`(info/warning/critical),新增 `Silence` 表(host/group/全局、起止时间、原因);`alert_engine.py: check_alerts` 在触发前查静默表;`routers/alerts.py` 加静默 CRUD;`AlertHistory.vue` 加“静默”操作和级别筛选;把 `_first_breach` 等内存状态顺带持久化到 DB(解决重启丢状态)
|
||||
|
||||
### 3. 通用 Webhook 通知(兑现 README 承诺的 Slack / 钉钉 / 企业微信)
|
||||
|
||||
- **痛点**:README 宣称三个渠道实际为零;`WEBHOOK_URL` 是死变量;飞书是唯一 IM 渠道,用户群在钉钉/企微的接不进来
|
||||
- **实现点**:新建 `services/webhook_notify.py`(通用 POST JSON + 按渠道模板:钉钉 markdown、企业微信 markdown、Slack Block Kit、飞书沿用现有);`settings.py` 的 `DEFAULT_SETTINGS` 加 `webhook_url` / `webhook_type`,`Settings.vue` 加“Webhook”Tab 与测试按钮;`alert_engine.py` 在飞书分支旁加 webhook 分支;`config.py` 删掉无人读的 `WEBHOOK_URL` 或与之打通,二选一
|
||||
|
||||
### 4. 主机批量导入 / 批量操作
|
||||
|
||||
- **痛点**:`preflight-data.sh` 示例输出显示生产规模已达百台级(107 台),现在只能一台台在 Web 表单里加;无导出则换环境/备份全靠拷 SQLite 文件
|
||||
- **实现点**:`routers/hosts.py` 加 `POST /api/hosts/batch`(CSV/JSON 导入,复用 `HostCreate` 校验)、`POST /api/hosts/batch-test`、`DELETE /api/hosts/batch`;`Hosts.vue` 加多选列 + 导入导出按钮;`models.py` 主机名/ipmi_host 加唯一约束防重复导入
|
||||
|
||||
### 5. 主机离线告警(采集失败即告警)
|
||||
|
||||
- **痛点**:现在最严重的故障(整机 BMC 掉线、断电)反而最安静——Dashboard 只把在线数减 1,无任何通知;`collect_host` 的失败只写日志
|
||||
- **实现点**:`scheduler.py: collect_all_hosts` 收集每台的 `success` 后,对连续 N 次失败的主机写一条 AlertHistory(可复用现有 `notify_type`/`acknowledged` 流程,或新增 `metric_type='connectivity'` 的系统规则);`alert_engine.py` 加一条基于 `last_seen` 超时的兜底检查;Dashboard 离线卡片可点击下钻到离线主机列表
|
||||
|
||||
### 6. 用户管理:改密、多用户、只读角色
|
||||
|
||||
- **痛点**:单 admin 账号 + 公开默认口令 + 无改密入口,是当前最大的安全缺口;多人运维只能共用一个账号,无法审计谁改了规则
|
||||
- **实现点**:`models.py: User` 加 `role`(admin/operator/viewer)、`routers/auth.py` 加 `POST /api/auth/change-password` 与用户 CRUD(admin 专属)、`routers/*` 的依赖按角色鉴权(viewer 禁写);前端 App.vue 用户菜单加“修改密码”,新增用户管理入口;删除死代码 `init_admin.py`,启动建账号逻辑只留 `database.py: init_db` 一处,并强制首次登录改密
|
||||
|
||||
### 7. 历史数据导出与报表(CSV 导出 + 每日摘要邮件)
|
||||
|
||||
- **痛点**:数据只进不出;故障复盘、容量规划、给领导汇报都只能截图;InfluxDB 里的数据除本系统外无人能用
|
||||
- **实现点**:`routers/monitor.py` 加 `GET /api/monitor/export`(复用 `influxdb.py: query_metrics`,流式返回 CSV);Dashboard/Sensors 加“导出”按钮;`scheduler.py` 加每日定时任务,汇总昨日最高温/最大功耗/告警数,复用 `email_notify.py` 发摘要(与告警邮件模板分离)
|
||||
|
||||
### 8. 统一配置中心(修掉“环境变量与数据库设置双轨制”)
|
||||
|
||||
- **痛点**:`POLL_INTERVAL` 改不动、`ALERT_*` 不生效、SMTP 有环境变量和 DB 两套来源——这是本次审计发现的最多的一类问题根源,继续加功能只会越叠越乱
|
||||
- **实现点**:`config.py` 明确划分:环境变量只管启动期(InfluxDB、DB 路径、JWT 密钥、端口),运行时可调项(采集间隔、全局默认阈值、通知渠道)全部收进 `SystemSetting` 表;`scheduler.py` 启动时读 DB 的采集间隔并支持热更新(APScheduler `reschedule_job`);`Settings.vue` 增加“采集与告警”Tab(间隔、默认阈值、冷却时间);README 环境变量表与之一一对齐,删掉死变量
|
||||
|
||||
### 9. InfluxDB 保留期与降采样管理
|
||||
|
||||
- **痛点**:无 retention 配置,百台规模下时序数据无限增长,迟早拖垮磁盘与查询;`query_latest` 只看 1 小时也与长期存储目标脱节
|
||||
- **实现点**:`docker-compose.yml` / InfluxDB 初始化加 retention(如原始数据 30 天);`influxdb.py` 加降采样查询(`aggregateWindow` 按小时/天聚合,供 7d/30d 趋势图用,替代现在全量拉点);Settings 或 Dashboard 加存储用量展示(bucket 统计接口)
|
||||
|
||||
### 10. 采集健康诊断页
|
||||
|
||||
- **痛点**:采集失败只写后端日志,用户在界面上看不到“哪台机采失败、为什么失败”;排障要进容器翻日志;组件页报错(`/tmp` 文件丢失)也无任何自检提示
|
||||
- **实现点**:持久化每次采集结果(新建 `CollectionLog` 表或复用 `last_seen` + 失败原因字段,替代 `/tmp/ipmi_raw_*.txt` 做组件数据源——组件数据改走 InfluxDB 的 `component` measurement 或 SQLite);`routers/monitor.py` 加 `GET /api/monitor/collection-status`;前端新增诊断视图或在 Hosts 页加“最后采集状态/失败原因”列,复用现有 `POST /api/hosts/{id}/test` 做一键诊断
|
||||
|
||||
---
|
||||
|
||||
**审计方法说明**:逐文件对照——README 14 个 API 端点逐一在 `routers/` 中检索、前端 9 个视图的全部 `api.*` 调用逐一反查后端路由、`.env.example` / `docker-compose.yml` 的每个变量在 `config.py` 与全代码库中检索引用;未做运行验证,结论均以静态代码为准。
|
||||
@@ -0,0 +1,214 @@
|
||||
# IPMI Monitor 安全审计报告
|
||||
|
||||
- 审计对象:`~/workspace/work/ipmi-monitor`,HEAD `a3ae072`
|
||||
- 审计范围:后端 `backend/app/`(约 3936 行)、`docker-compose.yml` / `docker-compose.dev.yml` / `.env.example` / `start.sh` / `alert-trigger.sh` / `backend/Dockerfile` / `backend/requirements.txt` / `frontend/src/`
|
||||
- 审计方法:只读静态审计;关键发现已对照源码逐项核实
|
||||
- 报告不含任何真实密钥、密码、token 值,只写存在性与位置
|
||||
|
||||
## 结论
|
||||
|
||||
**不要用当前默认配置直接部署到生产。** 最致命的是两条链已闭环:① JWT 密钥是代码里的已知默认值,且部署链路根本不注入该变量——任何人都能伪造管理员 token;② 每次全新启动自动创建已知弱口令的 `admin` 账号。两者任一成立即等于无认证。再加上所有 BMC 的 IPMI 口令以明文存库、又经 API 明文回显,攻破一点即拿到全网服务器带外管理口令。
|
||||
|
||||
**已核查为阴性:** SQL 注入未发现——全部查询走 SQLAlchemy ORM 参数化,无原生 SQL / `text()` / 字符串拼接 SQL;命令注入未发现 `shell=True` / `os.system` / `eval`——`ipmitool` 走 `asyncio.create_subprocess_exec` 列表参数(但存在参数值经 argv 与日志泄露的问题,见 H-03);`.env` 未进 git 历史(`.gitignore` 已排除)。
|
||||
|
||||
## 分级统计
|
||||
|
||||
| 级别 | 数量 |
|
||||
|---|---:|
|
||||
| Critical(严重) | 4 |
|
||||
| High(高危) | 10 |
|
||||
| Medium(中危) | 10 |
|
||||
| Low(低危) | 8 |
|
||||
| 合计 | 32 |
|
||||
|
||||
## Critical(严重)
|
||||
|
||||
### C-01 JWT 密钥为代码内已知默认值,且部署链路从不设置它
|
||||
- **位置:** `backend/app/config.py:43`(`JWT_SECRET_KEY` 默认值);`docker-compose.yml` 全文、`docker-compose.dev.yml` 全文、`.env.example` 全文——三处均无 `JWT_SECRET_KEY` 条目;`backend/app/routers/auth.py:62-72,75-86` 用该密钥签发/验签 HS256
|
||||
- **原理:** `pydantic-settings` 未取到环境变量即回落到源码默认值;Compose 未传该变量、`.env.example` 也未提示设置。默认部署下密钥是公开可知的常量
|
||||
- **影响:** 攻击者无需任何口令即可本地伪造 `{"sub":"admin"}` 的 JWT,通过全部受保护路由,等同管理员
|
||||
- **修复:** 启动时强制校验:密钥缺失或等于默认值/长度 <32 字节则拒绝启动;用 `secrets.token_urlsafe(32)` 生成并经 `.env`(不进 git)注入;在 `docker-compose.yml` 与 `.env.example` 中加入 `JWT_SECRET_KEY=${JWT_SECRET_KEY:?must be set}`
|
||||
|
||||
### C-02 全新启动自动创建已知弱口令的默认管理员,且明文打印/记日志
|
||||
- **位置:** `backend/app/database.py:95-112`(`init_db()` 内无 `admin` 则创建并 `logger.info` 打印账号口令);`backend/app/init_admin.py:42-55`(同样默认值,`print` 输出口令)
|
||||
- **原理:** 默认口令为短弱口令、无强制改密、无首次登录改密流程、无过期;每次换新卷/新库都会重建该账号
|
||||
- **影响:** 结合 C-01 或直接爆破,内网任何人登录即为唯一管理员;口令还落在容器日志里,可被日志采集方读取
|
||||
- **修复:** 删除自动建号逻辑,改为首次部署由环境变量传入一次性强口令(`ADMIN_INITIAL_PASSWORD`,创建后立即失效)或交互式创建;强制首次登录改密;删除所有打印/记录口令的代码;已有部署立即改密并轮换 JWT 密钥
|
||||
|
||||
### C-03 全部 BMC 的 IPMI 口令明文存库,且经 API 明文回显给任何已认证用户
|
||||
- **位置:** `backend/app/models.py:76`(`Host.ipmi_password` 为 `String(256)` 明文列);`backend/app/routers/hosts.py:46-52`(`HostOut` 含 `ipmi_password`)、`:62-86` 列表、`:88-103` 创建、`:106-126` 更新均原样返回;`backend/app/database.py` SQLite 文件本身亦无加密
|
||||
- **原理:** 口令只做长度校验,无任何应用层加密;输出模型未做字段脱敏。前端 `Hosts.vue:66` 虽然只显示掩码,但 API 响应体里仍有明文
|
||||
- **影响:** 任一 token 泄露(见 C-01/H-05)即可 `GET /api/hosts` 一次性拖走全网 BMC 口令,进而带外控制所有服务器(开关机、挂载镜像、读控制台);SQLite 文件一旦经 H-02 或卷泄露同样全量泄露
|
||||
- **修复:** 立即从 `HostOut` 删除 `ipmi_password`(新建/更新接口只回 `has_password: bool`);库内用 Fernet/AES-GCM(密钥放环境变量,与 JWT 密钥分离)加密存储,内存中用后清零;轮换全部 BMC 口令;对 SQLite 文件设 `0600` 并考虑 SQLCipher
|
||||
|
||||
### C-04 InfluxDB 管理员口令与 Admin Token 在 Compose / 示例文件里有已知默认值,`start.sh` 会静默沿用
|
||||
- **位置:** `docker-compose.yml:16`(`DOCKER_INFLUXDB_INIT_PASSWORD` 默认值)、`:19`(`DOCKER_INFLUXDB_INIT_ADMIN_TOKEN` 默认值)、`:43`(后端复用同一 token);`.env.example` 中 `INFLUXDB_PASSWORD` / `INFLUXDB_TOKEN` 同样为示例默认值;`start.sh:约 70-80 行`(无 `.env` 时 `cp .env.example .env`,只提示“请检查”而不强制改)
|
||||
- **原理:** 用户按 README / `start.sh` 一路默认部署,即以公开可知的口令与 token 初始化 InfluxDB,且 `docker-compose.yml:9` 将 8086 映射到宿主机
|
||||
- **影响:** 能访问 8086 者可用已知口令/token 登录 InfluxDB UI/API,读写/删除全部监控时序数据(含主机、传感器信息),并可作为横向立足点
|
||||
- **修复:** 删除 Compose 中所有密钥类默认值,改为 `${VAR:?must be set}` 强制必填;`.env.example` 只留占位符;`start.sh` 在使用默认值时直接退出;已有部署轮换口令与 token,8086 不要对公网/办公网暴露
|
||||
|
||||
## High(高危)
|
||||
|
||||
### H-01 CORS `allow_origins=["*"]` 与 `allow_credentials=True` 并存
|
||||
- **位置:** `backend/app/main.py:64-72`
|
||||
- **原理:** 通配源 + 允许凭证是矛盾配置,浏览器对通配源本应忽略凭证,但一旦改为具体源或经代理放宽,即允许任意站点携带用户凭证跨域调用 API
|
||||
- **影响:** 结合 H-05 的 localStorage token,恶意页面可借用户浏览器调用管理接口(删主机、改告警、改 SMTP/飞书配置)
|
||||
- **修复:** 只允许明确的白名单源(如部署域名),`allow_methods`/`allow_headers` 收敛到必需项;生产用同源部署时可直接去掉该中间件
|
||||
|
||||
### H-02 SPA 静态文件服务存在路径遍历,可读任意文件(含 SQLite 库)
|
||||
- **位置:** `backend/app/main.py:116-122`(`serve_spa`:`file_path = _frontend_dist / full_path` 后仅 `is_file()` 即 `FileResponse`,无 `resolve()` + `is_relative_to(_frontend_dist)` 校验)
|
||||
- **原理:** 代码层面无任何防遍历措施,`../` 可跳出 `frontend/dist`
|
||||
- **影响:** 若可绕过框架层归一化(需实测 `..` / `%2e%2e` 编码形态),可直接下载 `/app/data/ipmi_monitor.db`(C-03 的全部明文 BMC 口令 + 用户 bcrypt 哈希)及源码配置
|
||||
- **修复:** `file_path = (_frontend_dist / full_path).resolve()` 后断言 `is_relative_to(_frontend_dist.resolve())`,否则 404;或改用 `StaticFiles` 挂载;SQLite 目录不得位于可被静态服务触及的路径
|
||||
|
||||
### H-03 IPMI 口令经进程参数与 DEBUG 日志泄露
|
||||
- **位置:** `backend/app/services/ipmi_collector.py:149-176`(`_run_ipmitool`)、`:179-214`(`_run_ipmitool_with_auth`)、`:166,203`(`logger.debug("Running: %s", " ".join(cmd[:-5]) + " *** *** " + " ".join(cmd[-4:]))`)
|
||||
- **原理:** 口令以 `-P <口令>` 出现在 `ipmitool` 的 argv 中,任何能 `ps` / 读 `/proc/<pid>/cmdline` 者可见;所谓脱敏是错的——已验证 `cmd[:-5]` 仍包含 `-P` 与口令(17 元命令的前 12 元即含口令位),`***` 只遮住了后半段无关参数。两函数均如此
|
||||
- **影响:** 同机其他用户/容器逃逸后可直接捡到全部 BMC 口令;`LOG_LEVEL=DEBUG` 时口令进日志文件/日志平台
|
||||
- **修复:** 改用 `IPMI_PASSWORD` 环境变量或 `-f` 口令文件(`0600`,用后删除)传口令,不再用 `-P`;删除该 debug 日志或只记录 `host` + `interface`,永不拼接完整命令
|
||||
|
||||
### H-04 全站明文 HTTP,无 TLS、无安全响应头,API 文档默认公开
|
||||
- **位置:** `backend/Dockerfile:38`(`uvicorn … --host 0.0.0.0`,无 TLS);`backend/app/main.py:52-60`(`FastAPI()` 未关 `docs_url`/`redoc_url`/`openapi_url`,亦无 `HTTPSRedirect` / `TrustedHost` / HSTS / CSP / `X-Frame-Options` 等中间件);`start.sh` 末尾打印 `http://` 访问地址与 `/docs` 地址
|
||||
- **原理:** 登录口令、JWT、IPMI 口令、SMTP 口令全部走明文 HTTP;`/docs`、`/redoc`、`/openapi.json` 未鉴权即可查看全部接口与模型
|
||||
- **影响:** 局域网嗅探/中间人可直接窃取管理员口令与 token;文档泄露为攻击者提供完整接口清单
|
||||
- **修复:** 前置反向代理做 TLS(或 uvicorn 配证书),`HSTS` + `Secure`;生产 `docs_url=None, redoc_url=None, openapi_url=None` 或仅内网开启;加 `TrustedHostMiddleware` 与安全头中间件
|
||||
|
||||
### H-05 JWT 有效期 24 小时,无 `jti`/`iat`、无吊销、无登出,token 存 localStorage
|
||||
- **位置:** `backend/app/config.py:45`(`JWT_EXPIRATION_HOURS=24`);`backend/app/routers/auth.py:62-72`(仅 `sub` + `exp`,`datetime.now()` naive 时间)、`:75-86`(验签只认 `exp`)、`:178-190`(`/verify` 不查用户是否存在);模块文档称有 logout,实际无登出/吊销接口;`frontend/src/views/Login.vue:99-103` 与 `frontend/src/utils/api.js:15-18` 把 token 存 `localStorage` 并每次作 `Authorization: Bearer`
|
||||
- **原理:** 被盗 token 在 24 小时内无法作废;改密、删用户均不影响已签发 token;localStorage 可被任意 XSS/第三方脚本读取;前端路由守卫 `frontend/src/router/index.js:69-72` 只查 token 存在与否,不验有效性
|
||||
- **修复:** 访问 token 缩至 15 分钟 + refresh token(轮换、服务端可吊销);`jti` 入库做黑名单/白名单;提供登出与“全部会话失效”接口;改密后吊销全部 token;token 改存 `HttpOnly; Secure; SameSite=Strict` Cookie;前端守卫改为调 `/api/auth/me` 验真
|
||||
|
||||
### H-06 系统设置接口明文回显并明文存储 SMTP 口令与飞书 Webhook
|
||||
- **位置:** `backend/app/routers/settings.py:104-122`(`GET /api/settings` 返回完整 `smtp_password` 与 `feishu_webhook_url`)、`:32-54`(响应/更新模型均含明文);`backend/app/models.py:36-42`(`SystemSetting.value` 为 `Text` 明文);`frontend/src/views/Settings.vue:199` 回填口令到表单
|
||||
- **原理:** 密钥类配置与普通配置同表同字段存放、无加密、无掩码返回
|
||||
- **影响:** 任一已认证调用即拖走邮箱口令与飞书机器人地址,可伪造告警、发钓鱼邮件、向外发通知
|
||||
- **修复:** 密钥类设置单独加密列(AES-GCM);GET 只返回 `smtp_password_set: bool` / webhook 掩码(如末 4 位);更新时空值表示不改;轮换已泄露的 SMTP 口令与飞书 webhook
|
||||
|
||||
### H-07 `python-jose` 依赖存在已知 JWT 伪造类漏洞且已无人维护
|
||||
- **位置:** `backend/requirements.txt`(`python-jose[cryptography]>=0.3.3`);`backend/app/routers/auth.py:8,69-81`(`from jose import jwt`)
|
||||
- **原理:** `python-jose` 自 2021 年后基本停更;`>=0.3.3` 允许装到受影响版本:CVE-2024-33663(≤3.3.0,ECDSA/OpenSSH 密钥格式混淆)与 CVE-2026-120459(至 3.5.0 仍受影响、尚无修复版,空 HMAC 密钥可伪造 HS256 token,CVSS 9.1)。本项目 `INFLUXDB_TOKEN` 默认空(`config.py:22`)等空密钥配置习惯与后者风险同源,须确保 JWT 密钥永不为空
|
||||
- **影响:** 密钥为空或特定格式时可绕过签名校验伪造任意用户 token
|
||||
- **修复:** 迁移到积极维护的 `PyJWT>=2.10`(并显式 `algorithms=["HS256"]`、拒绝 `alg=none`);启动时校验密钥非空且长度足够;`pip-audit` 入 CI
|
||||
|
||||
### H-08 已认证 SSRF:IPMI 目标、飞书 Webhook、SMTP 主机均无任何校验
|
||||
- **位置:** `backend/app/routers/hosts.py:28-31`(`HostCreate.ipmi_host` 只限长度,可填任意主机/IP 与 `ipmi_port` 1-65535);`backend/app/services/ipmi_collector.py:245-268` 与 `hosts.py:146-190`(`test` 端点按该目标发起 6 次 `ipmitool` 连接);`backend/app/routers/settings.py:200-242`(`test-feishu` 对任意 `webhook_url` 做 `httpx.post`,无域名/IP 白名单);`settings.py:154-192`(`test-email` 对任意 `smtp_host:smtp_port` 建连);`backend/app/services/feishu_notify.py:70,139`(告警时同样 POST 任意 URL)
|
||||
- **原理:** 三处出站请求的目标完全由已认证用户控制,无内网 IP/元数据地址拦截、无域名白名单、无重定向限制
|
||||
- **影响:** 可探测内网存活主机与端口、向内网服务 POST 告警内容、尝试访问 `169.254.169.254` 等元数据端点(结合 C-01/C-02,未授权者亦可利用);SMTP 主机还可用于内网端口扫描与凭证试探
|
||||
- **修复:** `ipmi_host` 校验为 IP/主机名并拒绝回环/内网段(除非管理员显式放行网段);飞书 URL 只允许 `open.feishu.cn` / `open.larksuite.com` 等域名;SMTP 主机拒绝私网/元数据地址;httpx 禁用重定向到私网、设超时;出站统一走 SSRF 校验层
|
||||
- **注(协调员核查):** IPMI 目标本身就是内网 BMC,禁止私网段会破坏产品核心用途。落地时对 `ipmi_host` 只拦截回环/链路本地/组播/元数据地址,允许 RFC1918 私网;飞书 URL 限制域名、SMTP 主机拦截回环/链路本地/元数据地址
|
||||
|
||||
### H-09 登录无任何限流、锁定与强度校验,可无限爆破
|
||||
- **位置:** `backend/app/routers/auth.py:37-40`(`LoginRequest` 无长度/复杂度限制)、`:139-176`(登录逻辑无计数、无锁定、无延时);全仓无 `slowapi` / 限流中间件;`verify_password` 仅在用户存在时才执行 bcrypt(`auth.py:155`),不存在时直接 401,存在时多一次约数百毫秒 bcrypt,构成用户名枚举的时间侧信道
|
||||
- **原理:** 结合 C-02 的已知弱口令,爆破成本极低
|
||||
- **影响:** 字典爆破管理员口令;枚举有效用户名
|
||||
- **修复:** 登录限流(如每 IP 每分钟 5 次、失败 5 次锁定 15 分钟)+ 全局速率限制;不存在用户时也执行一次虚拟 bcrypt 使耗时恒定;登录失败记审计日志;口令设最小长度 12 位
|
||||
|
||||
### H-10 容器以 root 运行,生产 Compose 还把源码目录挂进容器
|
||||
- **位置:** `backend/Dockerfile` 全文无 `USER` 指令(默认 root);`docker-compose.yml:37-39`(`./backend/app:/app/app`、`./frontend:/app/frontend` 挂载);`backend/Dockerfile` 用 `pip config set global.trusted-host mirrors.aliyun.com`(HTTP 镜像源信任)
|
||||
- **原理:** 容器逃逸即宿主机 root;源码挂载使容器内改代码可直接污染宿主机源码,宿主机改代码也可不经构建生效,生产与开发边界消失
|
||||
- **影响:** 任一 RCE(如 H-02/H-08 链)放大为宿主机接管与持久化
|
||||
- **修复:** `Dockerfile` 加专用非 root 用户(`USER appuser`)并 `chmod 0600` 敏感文件;生产 Compose 删除源码挂载,改用构建镜像;加 `read_only: true`、`cap_drop: [ALL]`、`no-new-privileges:true`;镜像源改用 HTTPS 并做哈希锁定
|
||||
|
||||
## Medium(中危)
|
||||
|
||||
### M-01 Flux 查询注入:`metric_type` 未校验即拼进 Flux 字符串
|
||||
- **位置:** `backend/app/influxdb.py:138-146`(`query_metrics`)、`:171-180`(`query_latest`):`r["_measurement"] == "{metric_type}"` 直接插值;调用方 `backend/app/routers/monitor.py:76`(`/history/multi`)、`:122-124`(`/summary/{metric_type}`)、`:307`(`/trend`)、`:376`(`/history/{host_id}`)均未对 `metric_type` 做白名单校验。`host_id` 为 `int` 故安全,`bucket`/`org` 来自配置故安全
|
||||
- **原理:** 传入含 `"`、`}` 的 `metric_type` 可闭合字符串、改写 Flux 过滤逻辑
|
||||
- **影响:** 已认证用户可读取非预期 measurement 的数据、使查询报错/绕过过滤(数据越权与信息泄露)
|
||||
- **修复:** 路由层用 `Literal["temperature","fan","voltage","power","component"]` 枚举校验;查询层对插值做转义或改用参数化 Flux;无效值直接 400
|
||||
|
||||
### M-02 IPMI 认证自动降级到最弱方式
|
||||
- **位置:** `backend/app/services/ipmi_collector.py:245-268`(`attempts` 依次含 `["lanplus", ["-C", "3"]]`、`["lanplus", ["-C", "0"]]`(无认证)、`["lanplus", ["-A", "MD5"]]`、`["lan", ["-A", "MD5"]]`、`["lan", None]`(IPMI 1.5))
|
||||
- **原理:** 强方式失败即自动降到无认证 / MD5 / IPMI 1.5,且每次采集、每次 `test` 都重复 6 次尝试
|
||||
- **影响:** 凭据与会话降级为可破解/可窃听的弱协议;6 次尝试放大 BMC 侧爆破与账号锁定风险
|
||||
- **修复:** 默认只用 `lanplus` + 强认证;降级需每主机显式勾选“允许弱协议”并在 UI 标红;失败即报错不再静默降级
|
||||
|
||||
### M-03 无 RBAC、无改密、无用户管理、无审计日志
|
||||
- **位置:** `backend/app/models.py:20-31`(`User` 只有 `username`/`hashed_password`,无 `role`/`is_active`);全仓无改密/注册/用户列表/禁用接口;全仓无登录、配置变更、删主机等审计日志
|
||||
- **原理:** 单一管理员模型,所有持 token 者权限等同;事后无法追溯谁改了什么
|
||||
- **影响:** 无法做最小权限分离;口令泄露后无法只吊销一人;入侵难发现
|
||||
- **修复:** 加 `role`(admin/operator/viewer)与只读角色;加改密与用户管理接口;关键操作写审计表(含操作者、IP、时间)
|
||||
- **注:** 加 `role` 列需改数据库 schema,本次不做;改密接口与登录失败日志已落地,RBAC 与审计表待用户决策
|
||||
|
||||
### M-04 `/tmp` 原始 IPMI 数据文件权限过宽且跨主机串读
|
||||
- **位置:** `backend/app/services/ipmi_collector.py:275-278`(`Path(f"/tmp/ipmi_raw_{host.id}.txt").write_text(raw)`,默认 `0644`,无清理);`backend/app/routers/monitor.py:424-430`(`/components/{host_id}` 读固定 `/tmp/ipmi_raw.txt`,与 `host_id` 无关——任意主机 ID 返回同一份数据);`:467-470`(`/grouped` 才按主机读)
|
||||
- **原理:** 原始 SDR 含主机硬件细节,落盘权限过宽且无生命周期管理;一处接口取错文件
|
||||
- **影响:** 同机其他用户可读硬件信息;调用方拿到别的主机组件数据(越权/误报)
|
||||
- **修复:** 改写到应用数据目录并设 `0600`、按主机隔离、采集后定期清理;`/components/{host_id}` 改为按 `host_id` 取文件并鉴权归属;不要把敏感原始输出落盘,优先放内存/数据库
|
||||
|
||||
### M-05 报错与日志泄露内部细节
|
||||
- **位置:** `backend/app/routers/hosts.py:186-190`(`test` 返回 `str(exc)` 含 `ipmitool` stderr);`backend/app/routers/monitor.py:449-455,530-536`(组件接口返回 `str(exc)`);`backend/app/routers/settings.py:192,237`(测试邮件/飞书失败返回 `str(e)`);`backend/app/influxdb.py` 多处 `logger.exception` 打完整堆栈
|
||||
- **原理:** 异常直通前端,未做脱敏与分级
|
||||
- **影响:** 泄露内部路径、主机名、服务版本与后端拓扑,辅助后续攻击
|
||||
- **修复:** 对外只返回通用错误码与简短提示,详情只写服务端日志并脱敏;日志中过滤口令、token、内网地址
|
||||
|
||||
### M-06 SMTP 明文回退与证书校验不明确
|
||||
- **位置:** `backend/app/services/email_notify.py:180-188,236-244` 与 `backend/app/routers/settings.py:180-188`:非 465 端口用 `smtplib.SMTP` + `starttls()`(未显式传 `SSLContext`),`smtp_use_tls=false` 时直接明文 `login`
|
||||
- **原理:** 配置可关闭加密;STARTTLS 失败/被剥离时无强制校验与告警
|
||||
- **影响:** 邮件口令与告警内容可被中间人窃听/篡改
|
||||
- **修复:** 默认强制 TLS(465 用 `SMTP_SSL` 配默认校验上下文,587 用 `starttls(context=ssl.create_default_context())` 并校验失败即拒发);删除“关闭 TLS”选项或需管理员二次确认
|
||||
|
||||
### M-07 未校验输入与无上限查询可致 500 / DoS
|
||||
- **位置:** `backend/app/routers/monitor.py:83,137`(`[int(x.strip()) for x in host_ids.split(',')]`,非法值抛 `ValueError` 未捕获 → 500);`:88-100,380-392`(`parse_dt` 回退分支同样可能抛错);`/history/multi` 对 `host_ids` 数量无上限;`list_hosts`(`hosts.py:62-86`)无分页;`dashboard_totals`/`trend`(`monitor.py:202-368`)每主机发 4 次以上 Influx 查询(N+1);`backend/app/services/scheduler.py:60-73` 每主机最多 6 次 ×30 秒超时、`max_instances=1` 易阻塞下一轮采集
|
||||
- **原理:** 输入与扇出均无上限、无统一异常处理
|
||||
- **影响:** 恶意/误操作请求可打满后端与 InfluxDB,拖垮采集调度
|
||||
- **修复:** `host_ids` 限个数(如 ≤50)并校验为整数;统一异常处理返回 400;列表加分页;趋势/汇总改批量查询或缓存;采集超时与重试次数收敛
|
||||
|
||||
### M-08 依赖全部用 `>=` 下限、无锁定文件、无漏洞扫描
|
||||
- **位置:** `backend/requirements.txt` 全文(14 个依赖全为 `>=`,无 `==`、无 `requirements.lock`、无哈希);`frontend/package.json` 依赖多为 `^` 范围;`jinja2>=3.1.2` 允许装到含 CVE-2024-22195 / CVE-2024-34064 等的 3.1.2(修复于 3.1.3+);`passlib[bcrypt]>=1.7.4` 已停更(与 bcrypt 4.x 存在已知兼容问题);`jinja2` 在代码中未见实际使用
|
||||
- **原理:** 每次构建装到的版本不可复现,可能装到已知漏洞版;无扫描即无人发现
|
||||
- **修复:** 全部依赖钉死 `==` 并生成 `pip freeze` / `uv.lock`,提交 lock 文件;升级 `jinja2>=3.1.6` 或删除未用依赖;`pip-audit` / `npm audit` 入 CI 并定期升级
|
||||
|
||||
### M-09 `alert-trigger.sh` 存在多个高危写法且调用了不存在的接口
|
||||
- **位置:** `alert-trigger.sh:22`(无认证 `curl "$API_URL/api/alerts"`,但后端并无该 GET 路由,实际是 404/405);`:55-66`(`ssh -o StrictHostKeyChecking=no root@"$host"`);`:38,66`(主机名/消息直接拼进 `jq` 程序字符串 `".\"${host}_${sensor_type}\" // \"\""`);`:8`(状态文件用固定 `/tmp/ipmi-alert-state.json`)
|
||||
- **原理:** 关闭主机密钥校验等于放弃中间人防护;以 root 直连 BMC/业务主机;jq 拼接存在注入;`/tmp` 固定文件可被抢占/篡改
|
||||
- **影响:** 中间人可冒充目标主机;恶意告警内容可注入 jq/SSH 命令;脚本实际取不到告警却仍以 root 尝试连接
|
||||
- **修复:** 改调真实存在的 `/api/alerts/history` 并带 JWT;`StrictHostKeyChecking` 改为 `accept-new` 并预置 known_hosts;用 `jq --arg` 传参;状态文件放用户家目录并设 `0600`;非必要不用 root,改用专用低权账号 + `sudo` 白名单
|
||||
|
||||
### M-10 InfluxDB Token 默认空、端口对外暴露
|
||||
- **位置:** `backend/app/config.py:22`(`INFLUXDB_TOKEN: str = ""`);`docker-compose.yml:9`(`8086` 映射到宿主机);`backend/app/influxdb.py:20-30`(空 token 仍创建客户端,失败只在调用时暴露)
|
||||
- **原理:** 配置缺省不报错,部署后才在写/读时失败;端口暴露扩大攻击面
|
||||
- **影响:** 服务不可用时难定位;暴露的 8086 成为暴力破解与漏洞利用入口
|
||||
- **修复:** 启动校验 token 非空;8086 只监听内网或不映射宿主机端口,管理走容器网络或反代鉴权
|
||||
|
||||
## Low(低危)
|
||||
|
||||
### L-01 三处口令哈希实现不统一、时间处理用 naive datetime
|
||||
- **位置:** `backend/app/routers/auth.py:24-30`(直接 `bcrypt`)、`backend/app/database.py:90,100`(直接 `bcrypt`)、`backend/app/init_admin.py:17-20`(`passlib.CryptContext`);`auth.py:65`、`models.py`(`datetime.now()` 作 `created_at`/`last_login`/`exp`)
|
||||
- **影响:** 成本参数与哈希格式可能漂移;naive 时间在夏令时/时区切换时导致 token 过期判断偏差
|
||||
- **修复:** 统一用一处 `hash_password/verify_password`(`passlib` 或 `bcrypt` 二选一,成本 ≥12);全部时间改 `datetime.now(timezone.utc)`
|
||||
|
||||
### L-02 多处输入无格式校验
|
||||
- **位置:** `backend/app/routers/auth.py:37-40`(登录字段无长度限制);`backend/app/routers/alerts.py:36`(`email_addresses` 为自由文本);`backend/app/routers/settings.py:60-63`(`TestEmailRequest.email: str`,未用 `EmailStr`)
|
||||
- **影响:** 超长输入、非法邮箱可进库并在告警时触发异常/垃圾邮件
|
||||
- **修复:** 登录字段加 `min_length/max_length`;邮箱用 `EmailStr` 并做去重与数量上限
|
||||
|
||||
### L-03 示例与文档固化默认口令,`start.sh` 不强制轮换
|
||||
- **位置:** `.env.example`、`README.md:132-136`(配置表列出默认口令/token)、`start.sh`(复制示例后只提示、不校验、不退出)
|
||||
- **影响:** 用户极易原样上线默认配置(见 C-04)
|
||||
- **修复:** 文档与示例只写占位符;`start.sh` 增加“密钥是否为默认值/是否为空”检查,不通过即退出
|
||||
|
||||
### L-04 前端 token 存 localStorage、路由守卫只验存在性
|
||||
- **位置:** `frontend/src/views/Login.vue:99-103`、`frontend/src/utils/api.js:15-18`、`frontend/src/router/index.js:69-82`。前端未发现 `v-html`/`innerHTML`(XSS 面相对小),但一旦引入含 XSS 的依赖或组件,token 即被盗
|
||||
- **修复:** 见 H-05,改 HttpOnly Cookie;守卫改为先调 `/api/auth/me` 验真
|
||||
|
||||
### L-05 缺少安全头与版本信息泄露
|
||||
- **位置:** `backend/app/main.py:52-60`(`FastAPI(title="IPMI Monitor", version="0.1.0")`,无安全头中间件);`/api/health`(`main.py:89-91`)未鉴权(本身合理,但会暴露服务存活)
|
||||
- **修复:** 加 `X-Content-Type-Options: nosniff`、`X-Frame-Options: DENY`、CSP、HSTS;对外隐藏精确版本号
|
||||
|
||||
### L-06 开发配置混入生产部署
|
||||
- **位置:** `docker-compose.dev.yml`(`network_mode: "host"`、端口 8003、源码挂载);`docker-compose.yml:67-69` 定义了 `influxdb-data` 命名卷却未使用(实际用 `./data/influxdb` 绑定挂载)
|
||||
- **修复:** 开发与生产 Compose 彻底分离;删除未用卷定义;生产不用 `host` 网络模式
|
||||
|
||||
### L-07 `get_db` 对只读请求也提交事务
|
||||
- **位置:** `backend/app/database.py:55-70`(`yield` 后无条件 `commit()`,异常才回滚)
|
||||
- **影响:** GET 请求也可能把脏数据写盘;删除接口依赖该隐式提交,语义不清晰
|
||||
- **修复:** 只读依赖不提交;写操作显式 `commit()`
|
||||
|
||||
### L-08 未用依赖与重复代码增加攻击面
|
||||
- **位置:** `backend/requirements.txt`(`jinja2`、`aiofiles` 在代码中未见使用);`backend/app/services/ipmi_collector.py:149-176`(标注 DEPRECATED 的 `_run_ipmitool` 仍保留,与 `_run_ipmitool_with_auth` 重复)
|
||||
- **修复:** 删除未用依赖与废弃函数,减小供应链与维护面
|
||||
|
||||
## 修复优先级建议
|
||||
|
||||
先做 C-01 + C-02 + C-03 + C-04(轮换 JWT 密钥、管理员口令、InfluxDB 口令/token、全部 BMC 口令与 SMTP/飞书密钥),再做 H-01/H-04/H-05(CORS、TLS、token 吊销与 Cookie 化)与 H-02/H-03,最后按 M 级逐项收敛。
|
||||
@@ -0,0 +1,160 @@
|
||||
# IPMI Monitor 审计报告
|
||||
|
||||
- 审计对象:`~/workspace/work/ipmi-monitor`,HEAD `a3ae072`
|
||||
- 审计范围:后端 Python 3.11 + FastAPI + SQLAlchemy 2.0 + InfluxDB 2.7 + APScheduler(`backend/app/` 约 3936 行),前端 Vue 3(`frontend/src/`),Docker Compose 部署
|
||||
- 审计方式:三路并行(安全 / BUG 与优化 / 功能缺口),关键发现已对照源码逐项核实;随后三路并行落地修复
|
||||
- 分报告:[AUDIT-security.md](AUDIT-security.md)(安全 32 项)|[AUDIT-bugs.md](AUDIT-bugs.md)(BUG 30 项 + 优化 16 项)|[AUDIT-features.md](AUDIT-features.md)(功能核对 + 10 个新功能建议)
|
||||
- 状态:**审计与修复均已完成,未提交、未推送**,只改工作区文件
|
||||
|
||||
## 总体结论
|
||||
|
||||
**核心链路是通的(采集→存储→展示→按规则告警→邮件/飞书),但默认配置不能直接上生产,且 README 明显滞后于代码。**
|
||||
|
||||
- **安全**:最致命的是两条已闭环的链——JWT 密钥是代码里的已知默认值且部署链路从不注入(任何人可伪造管理员 token);全新启动自动创建已知弱口令的 `admin` 账号。再加上全部 BMC 口令明文存库又经 API 明文回显,攻破一点即拿到全网服务器带外管理口令。以上 4 项严重 + 10 项高危已全部修复。
|
||||
- **BUG**:30 项已确认,全部修复,其中 5 项最优先:告警恢复误发(B1)、脏数据中断全部告警检查(B2)、InfluxDB 宕机静默丢数据(B14)、密码明文返回(B20)、采集超时拖垮调度(B3+B4)。另有历史接口时区错 8 小时(B11)、组件接口读错文件(B10)、配置变量错位(B21)。
|
||||
- **功能**:README 的 API 表 14 个端点无一路径正确;Slack/钉钉/企业微信、全局阈值、批量操作、告警静默、服务器详情页均为宣称但未实现;8 个后端接口写好却无前端入口;`alert-trigger.sh` 完全跑不通。README 已按真实代码重写。
|
||||
|
||||
## 分级统计
|
||||
|
||||
| 类别 | 严重 | 高危 | 中危 | 低危 | 小计 |
|
||||
|---|---:|---:|---:|---:|---:|
|
||||
| 安全问题 | 4 | 10 | 10 | 8 | 32 |
|
||||
| 已确认 BUG | — | — | — | — | 30 |
|
||||
| 优化建议 | — | — | — | — | 16 |
|
||||
| 功能缺口(README 宣称未实现 + 断链) | — | — | — | — | 见 AUDIT-features.md 第二节 |
|
||||
|
||||
## 已修复(本次落地)
|
||||
|
||||
> 修复原则:不改行为(除安全修复本身要求的行为收敛外)、不动数据库 schema、不改 API 契约(新增端点与删除明文密码字段除外——后者正是安全修复本身)。每改一处均通过 `py_compile`;测试 36/36 通过(见“验证”)。**未提交、未推送**,只改工作区文件。
|
||||
|
||||
### 安全(严重/高危,14/14)
|
||||
|
||||
| 编号 | 修复 | 文件 |
|
||||
|---|---|---|
|
||||
| C-01 | 启动强制校验 JWT 密钥:缺失、等于旧默认值(以 SHA-256 摘要比对,代码与报告中均不写明文)或 <32 字符则拒绝启动;校验只在 lifespan 启动路径,不在 import 时执行;Compose 与 `.env.example` 改为必填 `${JWT_SECRET_KEY:?must be set}` | `backend/app/config.py`、`backend/app/main.py`、`docker-compose.yml`、`.env.example` |
|
||||
| C-02 | 删除自动创建已知弱口令 admin 的逻辑与口令日志;库里无用户时要求 `ADMIN_INITIAL_PASSWORD` 环境变量(≥12 位),否则拒绝启动;删除死代码 `init_admin.py` | `backend/app/database.py`、`backend/app/init_admin.py`(删除) |
|
||||
| C-03 | `HostOut` 删除 `ipmi_password`,改为 `has_password: bool`;BMC 口令入库前 Fernet 加密(密钥来自 `CREDENTIAL_ENCRYPTION_KEY` 或首次启动生成的 `data/.credential_key` 0600 文件),读取时解密;启动时对存量明文口令做一次性重加密迁移;采集器与 test 接口解密后内部使用 | `backend/app/crypto.py`(新增)、`backend/app/models.py`(未改字段)、`backend/app/routers/hosts.py`、`backend/app/services/ipmi_collector.py`、`backend/app/database.py` |
|
||||
| C-04 | 删除 Compose 与 `.env.example` 中 InfluxDB 口令/token 的默认值,改为 `${VAR:?must be set}`;`start.sh` 检测到默认值/空值/口令 <8 位直接退出(实测退出码 1);顺手加了 `JWT_SECRET_KEY`(<32 位即退出)的同类检查 | `docker-compose.yml`、`.env.example`、`start.sh` |
|
||||
| H-01 | CORS 改为白名单:`CORS_ORIGINS` 环境变量(逗号分隔),未设置时不启用 CORS 中间件(同源部署) | `backend/app/config.py`、`backend/app/main.py` |
|
||||
| H-02 | SPA 静态服务 `resolve()` + `is_relative_to()` 防遍历;兜底路由排除 `/api/` 前缀(返回 404 JSON);`%2e%2e` 编码探针验证均返回 404 | `backend/app/main.py` |
|
||||
| H-03 | ipmitool 口令改走 `-f` 临时口令文件(0600,用后删除),不再用 `-P`;删除拼接完整命令的 debug 日志,只记 host/interface | `backend/app/services/ipmi_collector.py` |
|
||||
| H-04 | 生产默认关闭 API 文档(`ENVIRONMENT=production` 时 `docs_url/redoc_url/openapi_url=None`);加安全响应头中间件(nosniff / DENY / CSP / Referrer-Policy)。注:仓库内有 `frontend/dist` 时,`/docs` 会落到 SPA catch-all 返回 index.html(非 Swagger 内容),容器内无 dist 时返回 404 | `backend/app/main.py`、`backend/app/config.py` |
|
||||
| H-05 | JWT 改用 `datetime.now(timezone.utc)`;默认有效期 24h → 8h;token 增加 `jti`;新增登出(进程内吊销)与改密接口(改密后经“改密时间戳”吊销该用户全部已发 token,含当前 token) | `backend/app/routers/auth.py`、`backend/app/config.py` |
|
||||
| H-06 | `GET /api/settings` 不再返回 `smtp_password` 明文与完整飞书 URL,改为 `smtp_password_set: bool` + webhook 掩码(只显后 4 位);更新时空值表示不改;SMTP 口令与飞书 URL 入库前同样 Fernet 加密 | `backend/app/routers/settings.py`、`backend/app/crypto.py` |
|
||||
| H-07 | `python-jose` → `PyJWT==2.10.1`;显式 `algorithms=["HS256"]`;`requirements.txt` 全部钉死 `==` 版本(版本逐个在 PyPI 核实存在且兼容 Python 3.11;passlib 1.7.4 与 bcrypt ≥4.1 不兼容,故 bcrypt 钉 4.0.1) | `backend/app/routers/auth.py`、`backend/requirements.txt` |
|
||||
| H-08 | SSRF 校验层:`ipmi_host` 拒绝回环/链路本地/组播/元数据地址(允许 RFC1918 私网,因 BMC 本就在内网);飞书 URL 只允许 `open.feishu.cn` / `open.larksuite.com`;SMTP 主机同样拦截回环/链路本地/元数据地址 | `backend/app/ssrf.py`(新增)、`backend/app/routers/hosts.py`、`backend/app/routers/settings.py`、`backend/app/services/feishu_notify.py` |
|
||||
| H-09 | 登录限流(内存计数:同一 IP+用户名 5 次失败锁定 15 分钟,第 6 次起 429);不存在用户时也执行一次虚拟 bcrypt 使耗时恒定;登录失败记日志 | `backend/app/routers/auth.py` |
|
||||
| H-10 | Dockerfile 加非 root 用户(`USER appuser`);生产 Compose 删除源码目录挂载(只剩 `backend-data` 数据卷);pip 镜像源改用 HTTPS(去掉 `trusted-host`) | `backend/Dockerfile`、`docker-compose.yml` |
|
||||
|
||||
中危/低危安全项(H-02 关联的 B28/B29、Flux 注入防护等)随 B12/B18 的白名单校验、O9/O10 一并修复,详见 AUDIT-security.md 与下表。
|
||||
|
||||
### BUG(已确认,30/30)
|
||||
|
||||
| 编号 | 修复 |
|
||||
|---|---|
|
||||
| B1 | 恢复通知仅当 `_alert_active` 为真时发送 |
|
||||
| B2 | `check_alerts` 对 `None`/非数值做隔离,单条规则异常不中断整轮 |
|
||||
| B3 | 采集超时捕获 `TimeoutError` 继续试下一种认证;超时后 `proc.kill()` + `await proc.wait()` |
|
||||
| B4 | 单次 ipmitool 超时 30s → 10s;调度加 `coalesce=True, misfire_grace_time=30`;采集加 `Semaphore(8)` 并发上限;连续失败 3 次后跳过 5 轮再试 |
|
||||
| B5 | 温度/风扇/电压/功耗正则支持负号与千分位逗号 |
|
||||
| B6 | `_classify_metric` 接收 `status` 参数,`ok→1、其余→0`;离散电源传感器归为 `status` 而非 `power` |
|
||||
| B7 | `parse_sdr_output` 按段数分流(5 段新格式 / 3 段 legacy),状态 token 扩充 `disabled/cr` 等 |
|
||||
| B8 | `component_parser` 调用前在 collector 内把 3 段 legacy 行展开为 5 段(`value=parts[1]、status=parts[2]`);未改 `component_parser.py` |
|
||||
| B9 | drive/fan/psu 解析加 `seen` 去重;PSU/硬盘在位改用 SDR `status` 段;`Drive 0` 用正则匹配 |
|
||||
| B10 | `/api/monitor/components/{host_id}` 改读 `/tmp/ipmi_raw_{host_id}.txt`,与 `/grouped` 一致;文件不存在返回 404 |
|
||||
| B11 | `parse_dt` 抽成公共函数(`utils.py`);默认值用 `datetime.now(timezone.utc)`;解析失败返回 400 而非 500 |
|
||||
| B12 | `host_ids` 校验为整数且限 ≤50 个;`metric_type` 白名单校验;查询加 `limit(5000)` |
|
||||
| B13 | `dashboard/totals` 返回 `group_totals`;`None` 值用 `or 0` 防护;已查前端不读 `avg_temperature`,故改名 `avg_max_temperature` |
|
||||
| B14 | `write_metrics_batch` 返回 bool;写入失败时 `collect_host` 不更新 `last_seen`、置 `success=False` |
|
||||
| B15 | InfluxDB 同步调用包进 `asyncio.to_thread()`;客户端配超时(10000ms) |
|
||||
| B16 | `check_alerts` 开头清理已删除/禁用规则的内存状态 |
|
||||
| B17 | 建/改规则校验 `host_id` 与 `group_id` 至多其一;`host_id` 分支同样只查 `enabled` 主机 |
|
||||
| B18 | `RuleCreate/RuleUpdate` 加值域校验:`condition` Literal、`metric_type` 白名单、`duration >= 0`、拒绝 NaN/Inf 阈值(同时堵住 Flux 注入点) |
|
||||
| B19 | 仅当历史创建成功(主机存在)才置 `_alert_active` |
|
||||
| B20 | 同 C-03/H-06(密码不再明文返回;设置接口掩码;告警路径 4 处解密点补齐,见下) |
|
||||
| B21 | `config.py` 增加 `POLL_INTERVAL` 别名(`AliasChoices`);删除 Compose/`.env.example` 里的死变量 `ALERT_*` / `WEBHOOK_URL` / `ALERT_EMAIL_*`,改为正确变量名;`INFLUXDB_BUCKET` 默认值统一为 `metrics` |
|
||||
| B22 | 删除 `init_admin.py`(死代码且跑不起来),只保留 `database.py: init_db` 一处建号逻辑 |
|
||||
| B23 | 写接口显式 `await db.commit()`(alerts/groups/hosts/auth/settings 全部写接口);`get_db` 去掉成功后自动提交(只 yield,异常回滚) |
|
||||
| B24 | `send_alert_email/send_recovery_email` 接收并使用 `smtp_use_tls`;仅开关开且非 465 时才 `starttls()`,配 `SSLContext`;465 用 `SMTP_SSL` |
|
||||
| B25 | 告警路径读取 `feishu_keyword` 并拼进飞书卡片文本 |
|
||||
| B26 | 测试邮件端口解析移入 `try` 并校验范围(1~65535);SMTP 调用包进 `asyncio.to_thread()` 并配 15s 超时;连接用 `with` 保证释放;新增 `email_notify.send_test_email` 供 settings 测试端点使用 |
|
||||
| B27 | 邮件模板改用缺失字段安全的渲染(缺 key 填空串,不抛 `KeyError`);HTML 残留 `{{ ... }}` 清理 |
|
||||
| B28 | 同 H-02(SPA 兜底排除 `/api/`,防遍历) |
|
||||
| B29 | 同 H-01(CORS 收敛) |
|
||||
| B30 | 趋势接口功耗分支同样做“原生优先、否则 PSU_PIN 求和”聚合 |
|
||||
|
||||
另:认证降级已剔除——`ipmi_collector.py` 与 `hosts.py` 的 test 接口均删除无认证(`-C 0`)与 IPMI 1.5(`lan`)尝试,只保留 lanplus 默认、`lanplus -C 3`、`lanplus -A MD5` 三种(见“未修复”第 6 项)。
|
||||
|
||||
衔接修复(收尾一路):加密落地后,`alert_engine.py` 告警/恢复两处路径的 4 个解密点(SMTP 口令 ×2、飞书 URL ×2)、`email_notify._smtp_settings` 与 `feishu_notify.send_feishu_alert` 入口的防御性解密已全部补上,告警通知链路不断。
|
||||
|
||||
### 优化(已做 11 项,另 2 项部分)
|
||||
|
||||
- O1:`query_latest_grouped(host_id, only=...)` 按需查询,不再每次查 4 种类型
|
||||
- O3:设置批量读取(`get_settings_batch`),告警路径 SMTP 5 个 key + TLS 开关、飞书 2 个 key 一次取回
|
||||
- O4:采集 `Semaphore(8)` 并发上限;连续失败主机记入退避(失败 3 次后跳过 5 轮)
|
||||
- O5:抽公共 `resolve_total_power(power_metrics)`,Dashboard 汇总与趋势共用;`0` 视为有效读数(传感器存在即有效)
|
||||
- O6:抽公共常量 `IPMI_AUTH_ATTEMPTS`、公共 `parse_dt()`;删除已标注 DEPRECATED 的 `_run_ipmitool`
|
||||
- O9:SQLite 加 `journal_mode=WAL` + `foreign_keys=ON` + `busy_timeout`;`init_db` 里 `CREATE INDEX IF NOT EXISTS` 补关键索引
|
||||
- O10:登录限流 + 改密/登出 + JWT 用 UTC 时间(同 H-05/H-09)
|
||||
- O11:删除 Compose 未用卷定义与废弃 `version` 字段;Dockerfile 健康检查改用 `${PORT:-8000}`
|
||||
- O12(部分):非 root 用户已做(同 H-10);单 worker 未动(见“未修复”第 8 项)
|
||||
- O13:`start.sh` 默认不再 `down`(`up -d --build`),启动后轮询后端 `/api/health`
|
||||
- O14:`alert-trigger.sh` 重写(仓库根目录,`scripts/` 下并无此文件):按 `--api-url/--token/--ssh-key` 解析参数,调真实 `/api/alerts/history?acknowledged=false` 带 JWT,`jq --arg` 传参,`StrictHostKeyChecking=accept-new`,状态文件放 `$HOME/.ipmi-monitor/` 并设 0600
|
||||
- O15:InfluxDB 客户端配超时;启动时预创建(lifespan),懒创建加锁
|
||||
- O16(部分):补测试 33 个新用例 + README 重写;CI 未加
|
||||
|
||||
未做的优化:O2(降采样未做;查询上限 5000 已随 B12 加上)、O7(魔法数字散落各处,未集中整理)、O8(前端轮询 10 秒 vs 后端采集 60 秒、图表用轮询时间而非传感器时间,未改)。
|
||||
|
||||
### 新功能(已实现 2 个)
|
||||
|
||||
1. **改密接口**(安全修复附带,AUDIT-features.md 第 6 项的一部分):`POST /api/auth/change-password`(需登录,旧密码校验 + 新密码 ≥12 位),改密后吊销该用户全部已发 token;另有 `POST /api/auth/logout`
|
||||
2. **历史数据 CSV 导出**(AUDIT-features.md 第 7 项):`GET /api/monitor/export?host_ids=…&metric_type=…&start=…&end=…`,复用 `query_metrics`,流式返回 CSV(列:time,host_id,metric_type,metric_name,value,unit);供故障复盘与报表用
|
||||
|
||||
其余新功能(主机详情页、告警静默、通用 Webhook、批量导入、离线告警、RBAC、配置中心、保留期管理、采集诊断页)未做,只写进 AUDIT-features.md 待用户定。
|
||||
|
||||
### 前端同步
|
||||
|
||||
- `Hosts.vue`:密码列改读 `has_password`(已配置显示掩码,未配置显示“未配置”);编辑回填本来就是 `ipmi_password: undefined` + 提交时删空口令,未改
|
||||
- `Settings.vue`:加载时改读 `feishu_webhook_masked` / `smtp_password_set` 做掩码展示(留空则不修改);提交时秘密字段留空表示不改;测试飞书/测试邮件的必填校验放行“已配置但本次未重填”;顺手修了测试邮件按钮 `@click` 指向 ref 而非处理函数的既有缺陷(原按钮点按无效,现已接线到 `testEmailSend`)
|
||||
- `frontend/dist/` 未动、未重新构建:前端 `src` 改动后需用户自行 `npm run build` 刷新 dist
|
||||
|
||||
### README 更新
|
||||
|
||||
- API 表按真实路由重写(37 个端点,`/api/hosts`、`/api/monitor/*`、`/api/alerts/*`、`/api/groups`、`/api/settings`、`/api/auth`,含 logout、change-password、export)
|
||||
- 技术栈表修正为 Vue 3.4 + Vue Router + Element Plus + ECharts + Vite,数据库补上 SQLite + InfluxDB 两套
|
||||
- 环境变量表与 `config.py` 对齐,删除死变量说明,注明阈值在页面规则里配、采集间隔用 `COLLECT_INTERVAL`(兼容 `POLL_INTERVAL` 别名)、`JWT_SECRET_KEY` 与 `ADMIN_INITIAL_PASSWORD` 为首次启动必填
|
||||
- 功能描述修正:仅邮件 + 飞书两种通知渠道(无 Slack/钉钉/企业微信)、采集为 60 秒轮询
|
||||
|
||||
## 改动文件清单
|
||||
|
||||
25 个修改 + 3 个新建模块 + 3 个新建测试 + 4 份报告(`git diff --stat`:25 files changed,+1962/−842):
|
||||
|
||||
- 修改:`.env.example`、`README.md`、`alert-trigger.sh`、`backend/Dockerfile`、`backend/app/config.py`、`backend/app/database.py`、`backend/app/influxdb.py`、`backend/app/main.py`、`backend/app/routers/alerts.py`、`backend/app/routers/auth.py`、`backend/app/routers/groups.py`、`backend/app/routers/hosts.py`、`backend/app/routers/monitor.py`、`backend/app/routers/settings.py`、`backend/app/services/alert_engine.py`、`backend/app/services/email_notify.py`、`backend/app/services/feishu_notify.py`、`backend/app/services/ipmi_collector.py`、`backend/app/services/scheduler.py`、`backend/requirements.txt`、`docker-compose.yml`、`frontend/src/views/Hosts.vue`、`frontend/src/views/Settings.vue`、`start.sh`(24 个,另含删除的 `backend/app/init_admin.py`)
|
||||
- 新建:`backend/app/crypto.py`、`backend/app/ssrf.py`、`backend/app/utils.py`、`backend/tests/test_sdr_parser.py`、`backend/tests/test_alert_recovery.py`、`backend/tests/test_parse_dt.py`
|
||||
- 报告:`AUDIT.md`、`AUDIT-security.md`、`AUDIT-bugs.md`、`AUDIT-features.md`
|
||||
|
||||
## 未修复 / 待用户决策
|
||||
|
||||
| # | 事项 | 原因 / 说明 |
|
||||
|---|---|---|
|
||||
| 1 | **轮换已泄露的凭证**:JWT 密钥、管理员口令、InfluxDB 口令与 token、全部 BMC 口令、SMTP 口令、飞书 webhook | 代码已修(不再用默认值),但已部署环境里正在用的值若曾以默认值上线,必须逐一轮换;BMC 口令需逐台改;轮换后旧加密口令需用新密钥重新加密入库(或直接在页面重填) |
|
||||
| 2 | **TLS / 反向代理**:生产仍是明文 HTTP;`/docs` 已默认关闭,但 TLS 需要证书与域名,由用户部署反代(Nginx/Caddy)或配 uvicorn 证书 | 需要用户提供域名/证书或决定反代方案 |
|
||||
| 3 | **Token 改存 HttpOnly Cookie + refresh token**:当前仍为 localStorage + 单 token(已缩至 8h、可登出吊销);完整方案需前后端协同改动 | 需要用户确认是否接受该改动面 |
|
||||
| 4 | **RBAC 与审计表**:多用户、角色(admin/operator/viewer)、操作审计日志,需改数据库 schema(`User.role`、审计表) | 需改 schema,本次未动;待用户定 |
|
||||
| 5 | **告警状态持久化**:`_first_breach/_last_notify/_alert_active` 仍在进程内存,重启丢状态;已加清理逻辑,完整持久化需新表 | 需新表,待用户定 |
|
||||
| 6 | **IPMI 认证降级**:已去掉无认证(`-C 0`)与 IPMI 1.5(`lan`)两种尝试,只留 `lanplus` 默认、`lanplus -C 3`、`lanplus -A MD5`;极老 BMC 若只支持 IPMI 1.5 会采不到 | 若用户有只支持 IPMI 1.5 的老机器,需单独放行 |
|
||||
| 7 | **8086 端口暴露**:InfluxDB 8086 仍映射到宿主机(`start.sh` 健康检查依赖它);彻底隔离需改 `start.sh` 用 `docker compose exec` 检查 | 待用户定是否关闭映射 |
|
||||
| 8 | **Uvicorn 单 worker**:采集与 API 同进程;多 worker 会导致 APScheduler 重复采集,需拆采集为独立进程才能安全扩 worker | 架构改动,待用户定 |
|
||||
| 9 | **告警静默 / 严重级别 / 通用 Webhook / 批量导入 / 离线告警 / 配置中心 / 保留期管理 / 采集诊断页 / 主机详情页**:见 AUDIT-features.md 第三节 | 待用户定优先级 |
|
||||
| 10 | **规则列表未分页**:`GET /api/hosts` 已加可选 limit/offset,`GET /api/alerts/rules` 仍全量返回 | 小改动,可随下次迭代做 |
|
||||
| 11 | **`frontend/dist` 已提交进仓库**:dist 与源码可能不一致,且每次构建污染 diff;需 `git rm -r --cached frontend/dist` + `.gitignore` 排除,构建改由 Dockerfile/CI 做 | 涉及 git 操作,本次未动,待用户定 |
|
||||
| 12 | **前端需重新构建**:`src` 已同步新字段,但 `dist/` 未动;部署前需 `npm run build` | 用户部署时执行 |
|
||||
|
||||
## 验证结果
|
||||
|
||||
- `python3 -m py_compile`:全部后端 Python 文件通过(含 3 个新建模块与 3 个新建测试)
|
||||
- 全量测试:`cd backend && python3 -m unittest discover -s tests`,**36/36 通过**(原有 3 个总功耗 fallback + 新增 33 个:SDR 解析 12、告警恢复 9、parse_dt 12)
|
||||
- BUG 一路冒烟测试(`/tmp/ipmi_smoke.py`,stub 缺失依赖后跑纯逻辑):**35/35 通过**(B5 负数/逗号、B6 状态映射、B7 分段、B8 legacy 展开、B9 去重/在位、B11 非法时间、O5 的 0 值有效与 PSU 求和、H2 三组合、B26 端口越界、B27 缺字段渲染)
|
||||
- 安全一路冒烟测试(`/tmp/ipmi-smoke/smoke.py`,venv + TestClient + 临时 SQLite):**60 项检查中 58 项通过**,另 2 项经核查是误报(`/docs`、`/openapi.json` 在生产模式下返回的是 SPA index.html 而非 Swagger 内容,`app.docs_url/openapi_url` 已确认为 None);覆盖启动校验拒绝空/短密钥、POLL_INTERVAL 别名、加密往返与历史明文兼容、密钥文件 0600、SSRF 允许/拒绝矩阵、5 次失败后 429、创建主机响应无口令且库内为密文、设置掩码、登出后 token 401、改密后旧 token 401、新口令可登录;路径遍历 `%2e%2e` 探针均返回 404;`start.sh` 用假 docker 桩验证,秘密为空时退出码 1,补齐后通过;`docker-compose.yml` YAML 解析通过
|
||||
- 未做:真实 BMC 联调、真实 InfluxDB 联调、Docker 构建验证(本机无 Docker 运行时与真实依赖);`frontend` 未重新构建(`dist/` 不动,源码改动需用户自行 `npm run build`);`requirements.txt` 的钉死版本在 Python 3.12 venv 中验证安装可用,未在目标 Python 3.11 容器内实测安装(bcrypt 钉 4.0.1 是按 passlib 1.7.4 兼容性选的)
|
||||
- 已知小边界:`parse_dt(" ")`(纯空白)会抛 `ValueError` 而非返回 `None`,路由层映射为 400,不影响使用;吊销集合与登录限流是进程内内存,多副本部署不共享、重启后清空,单进程部署无影响
|
||||
@@ -1,28 +1,42 @@
|
||||
# IPMI Monitor
|
||||
|
||||
基于 Web 的 IPMI 服务器硬件监控系统,实时采集和展示服务器温度、风扇转速、电源状态等关键指标,支持告警通知。
|
||||
基于 Web 的 IPMI 服务器硬件监控系统,定时采集服务器温度、风扇转速、电压、电源功耗等传感器指标,支持按规则告警(邮件 / 飞书通知)与历史数据查询、CSV 导出。
|
||||
|
||||
## 功能特性
|
||||
|
||||
- 🖥️ **多服务器管理** - 支持添加和管理多台 IPMI 服务器
|
||||
- 📊 **实时监控** - 实时采集温度、风扇转速、电压、电源功耗等传感器数据
|
||||
- 📈 **历史趋势** - 基于 InfluxDB 时序数据库,支持历史数据查询和趋势图表
|
||||
- ⚠️ **智能告警** - 可配置温度、风扇、功耗等阈值,触发告警通知
|
||||
- 📧 **多渠道通知** - 支持邮件、Webhook(Slack / 钉钉 / 企业微信)告警推送
|
||||
- 🐳 **一键部署** - Docker Compose 一键部署,开箱即用
|
||||
- 📱 **响应式界面** - 纯静态前端,适配桌面和移动设备
|
||||
- 🔌 **API 驱动** - 完整的 RESTful API,支持二次开发
|
||||
- 🖥️ **多服务器管理** — 添加、管理多台 IPMI 服务器,按分组组织,支持连接测试与启用开关
|
||||
- 📊 **定时采集** — 每 60 秒(可配)用 `ipmitool sdr elist` 轮询 BMC,写入 InfluxDB(无 WebSocket/SSE 实时推送)
|
||||
- 📈 **历史趋势** — 基于 InfluxDB 时序数据,仪表盘提供温度 / 功耗趋势图(ECharts),支持多主机历史查询与 CSV 导出
|
||||
- ⚠️ **智能告警** — 逐条配置告警规则(条件 gt/lt/eq/gte/lte、持续时长、冷却 3 小时、恢复通知),按主机 / 分组 / 全局生效
|
||||
- 📧 **通知渠道** — 邮件(SMTP)与飞书群机器人;配置存在数据库中,页面设置里维护
|
||||
- 🐳 **一键部署** — Docker Compose 一键部署(InfluxDB 2.7 + 后端),开箱即用
|
||||
- 📱 **响应式界面** — Vue 3 单页应用(SPA),由 FastAPI 托管 `frontend/dist` 构建产物
|
||||
- 🔌 **API 驱动** — 完整的 RESTful API(JWT 鉴权),支持二次开发
|
||||
|
||||
## 技术栈
|
||||
|
||||
| 组件 | 技术 |
|
||||
|------|------|
|
||||
| 后端 | Python 3.11 + FastAPI + Uvicorn |
|
||||
| 数据库 | InfluxDB 2.7 (时序数据库) |
|
||||
| 前端 | HTML / CSS / JavaScript (纯静态) |
|
||||
| 监控采集 | ipmitool |
|
||||
| 数据库 | SQLite(SQLAlchemy:用户 / 主机 / 分组 / 告警规则 / 告警历史 / 系统设置)+ InfluxDB 2.7(时序数据) |
|
||||
| 前端 | Vue 3.4 + Vue Router 4.3 + Element Plus 2.7 + ECharts 5.5 + Vite 5.4 |
|
||||
| 监控采集 | ipmitool(`sdr elist`) |
|
||||
| 容器化 | Docker + Docker Compose |
|
||||
|
||||
## 页面一览
|
||||
|
||||
| 路由 | 页面 | 说明 |
|
||||
|------|------|------|
|
||||
| `/` | 仪表盘 | 主机在线统计、总功耗 / 最高温度汇总、温度与功耗趋势图 |
|
||||
| `/sensors` | 传感器 | 各主机最新传感器值(温度 / 风扇 / 电压 / 功耗) |
|
||||
| `/components` | 组件 | CPU / 内存 / PCIe / 硬盘 / 风扇 / 电源在位状态与总功耗 |
|
||||
| `/hosts` | 主机管理 | 主机增删改、分组筛选、连接测试 |
|
||||
| `/groups` | 分组管理 | 分组增删改 |
|
||||
| `/alert-rules` | 告警规则 | 规则增删改(阈值、条件、持续时长、通知渠道) |
|
||||
| `/alert-history` | 告警历史 | 分页查询、按主机 / 确认状态筛选、确认 |
|
||||
| `/settings` | 系统设置 | 飞书 Webhook 与 SMTP 配置、测试邮件 / 测试飞书消息 |
|
||||
| `/login` | 登录 | 用户名 + 密码登录(JWT) |
|
||||
|
||||
## 快速开始(Docker Compose 一键部署)
|
||||
|
||||
### 前置要求
|
||||
@@ -43,7 +57,7 @@ cd /opt/ipmi-monitor
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
# 编辑 .env 文件,修改密码、Token 等敏感配置
|
||||
# 编辑 .env 文件,填写首次启动必填项(见下表)与 InfluxDB 凭据
|
||||
vim .env
|
||||
```
|
||||
|
||||
@@ -58,18 +72,32 @@ chmod +x start.sh
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
4. **访问系统**
|
||||
4. **首次登录并改密**
|
||||
|
||||
首次启动时用户表为空,后端用 `ADMIN_INITIAL_PASSWORD` 创建初始管理员账号(默认用户名 `admin`,见启动日志)。**登录后立即改密**:
|
||||
|
||||
```bash
|
||||
# 登录取 token 后调用改密接口(新密码至少 12 位;改密后此前签发的所有 token 失效,需重新登录)
|
||||
curl -X POST http://your-server:8000/api/auth/change-password \
|
||||
-H "Authorization: Bearer <token>" \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{"old_password": "<初始密码>", "new_password": "<新密码>"}'
|
||||
```
|
||||
|
||||
5. **访问系统**
|
||||
|
||||
- 前端界面: http://your-server:8000
|
||||
- API 文档: http://your-server:8000/docs
|
||||
- API 文档(仅非生产环境): http://your-server:8000/docs
|
||||
- InfluxDB 管理界面: http://your-server:8086
|
||||
|
||||
5. **添加第一台服务器**
|
||||
6. **添加第一台主机**
|
||||
|
||||
在 Web 界面中点击「添加服务器」,填写 IPMI 连接信息:
|
||||
- 服务器名称(别名)
|
||||
- IPMI IP 地址
|
||||
- 用户名 / 密码
|
||||
在 Web 界面「主机管理」中点击「添加主机」,填写 IPMI 连接信息:主机名称、IPMI 地址、端口(默认 623)、用户名 / 密码。
|
||||
|
||||
### 生产环境注意
|
||||
|
||||
- 生产关闭 API 文档:`ENVIRONMENT=production`(docker-compose 默认值)时 `/docs`、`/redoc`、`/openapi.json` 均不可用;开发环境(`ENVIRONMENT=development`)才开启。
|
||||
- 前端改动后需自行构建:`frontend/src` 改动后不会自动生效,需在 `frontend/` 下执行 `npm run build` 重新生成 `frontend/dist`(后端以 SPA 方式托管该目录),再重启后端容器。
|
||||
|
||||
## 手动部署步骤
|
||||
|
||||
@@ -95,7 +123,7 @@ pip install -r requirements.txt
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
# 编辑 .env,配置 InfluxDB 连接信息
|
||||
# 编辑 .env,填写首次启动必填项(JWT_SECRET_KEY、ADMIN_INITIAL_PASSWORD)与 InfluxDB 连接信息
|
||||
vim .env
|
||||
```
|
||||
|
||||
@@ -127,31 +155,39 @@ WantedBy=multi-user.target
|
||||
|
||||
## 环境变量说明
|
||||
|
||||
与 `backend/app/config.py` 逐项对齐。标记 **首次启动必填** 的两项缺失时后端拒绝启动(`JWT_SECRET_KEY` 至少 32 位;`ADMIN_INITIAL_PASSWORD` 至少 12 位,仅用户表为空时使用,此后改动无效)。
|
||||
|
||||
| 变量名 | 默认值 | 说明 |
|
||||
|--------|--------|------|
|
||||
| `INFLUXDB_USERNAME` | `admin` | InfluxDB 管理员用户名 |
|
||||
| `INFLUXDB_PASSWORD` | `admin123456` | InfluxDB 管理员密码 |
|
||||
| `JWT_SECRET_KEY` | —(**首次启动必填**) | JWT 签名密钥,至少 32 位随机字符串 |
|
||||
| `ADMIN_INITIAL_PASSWORD` | —(**首次启动必填**,仅首次建管理员时用) | 初始管理员密码,至少 12 位;首次登录后立即改密 |
|
||||
| `ENVIRONMENT` | `development` | `development` 或 `production`;production 关闭 `/docs`、`/redoc`、`/openapi.json` |
|
||||
| `DATABASE_URL` | `sqlite+aiosqlite:///./data/ipmi_monitor.db` | 后端 SQLite 地址,Docker 部署时位于 `/app/data` 命名卷中 |
|
||||
| `REQUIRE_EXISTING_DATABASE` | `false` | 生产保护:为 `true` 时数据库文件不存在则拒绝启动 |
|
||||
| `MIN_EXPECTED_HOSTS` | `0` | 生产保护:主机数低于该下限则拒绝启动(如 `50`) |
|
||||
| `INFLUXDB_URL` | `http://localhost:8086` | InfluxDB 地址(Docker Compose 中为 `http://influxdb:8086`) |
|
||||
| `INFLUXDB_TOKEN` | — | InfluxDB API Token |
|
||||
| `INFLUXDB_ORG` | `ipmi-monitor` | InfluxDB 组织名称 |
|
||||
| `INFLUXDB_BUCKET` | `metrics` | InfluxDB 存储桶名称 |
|
||||
| `INFLUXDB_TOKEN` | `my-super-secret-token` | InfluxDB API Token |
|
||||
| `INFLUXDB_PORT` | `8086` | InfluxDB 端口映射 |
|
||||
| `COMPOSE_PROJECT_NAME` | `ipmi-monitor` | Docker Compose 项目名;会影响命名卷名称,生产环境必须保持稳定 |
|
||||
| `BACKEND_PORT` | `8000` | 后端服务端口映射 |
|
||||
| `DATABASE_URL` | `sqlite+aiosqlite:///./data/ipmi_monitor.db` | 后端 SQLite 数据库地址,Docker 部署时位于 `/app/data` 命名卷中 |
|
||||
| `REQUIRE_EXISTING_DATABASE` | `false` | 是否要求启动前数据库文件已存在;新环境保持 `false`,已有生产数据环境建议设为 `true` |
|
||||
| `MIN_EXPECTED_HOSTS` | `0` | 数据库最少主机数量保护;新环境保持 `0`,生产环境可设为预期下限,如 `50` |
|
||||
| `POLL_INTERVAL` | `60` | 数据采集间隔(秒) |
|
||||
| `COLLECT_INTERVAL` | `60` | 数据采集间隔(秒);兼容旧名 `POLL_INTERVAL` 别名 |
|
||||
| `LOG_LEVEL` | `INFO` | 日志级别 |
|
||||
| `ALERT_TEMP_THRESHOLD` | `80` | 温度告警阈值(°C) |
|
||||
| `ALERT_FAN_THRESHOLD` | `0` | 风扇转速告警阈值(RPM,0 禁用) |
|
||||
| `ALERT_POWER_THRESHOLD` | `500` | 功耗告警阈值(W,0 禁用) |
|
||||
| `SMTP_HOST` | - | SMTP 服务器地址 |
|
||||
| `CORS_ORIGINS` | —(空) | 逗号分隔的跨域白名单;为空表示不启用 CORS(同源部署) |
|
||||
| `JWT_ALGORITHM` | `HS256` | JWT 签名算法 |
|
||||
| `JWT_EXPIRATION_HOURS` | `8` | JWT 有效期(小时) |
|
||||
| `CREDENTIAL_ENCRYPTION_KEY` | —(空) | 可选:加密入库口令(BMC 口令 / SMTP 口令 / 飞书 URL)的 Fernet 密钥;为空时后端首次启动自动生成并以 0600 权限保存在数据库目录旁 |
|
||||
| `SMTP_HOST` | —(空) | SMTP 服务器地址(环境变量兜底;告警实际用数据库「系统设置」里的 SMTP 配置) |
|
||||
| `SMTP_PORT` | `587` | SMTP 端口 |
|
||||
| `SMTP_USER` | - | SMTP 用户名 |
|
||||
| `SMTP_PASSWORD` | - | SMTP 密码 |
|
||||
| `ALERT_EMAIL_FROM` | - | 告警发件邮箱 |
|
||||
| `ALERT_EMAIL_TO` | - | 告警收件邮箱(逗号分隔) |
|
||||
| `WEBHOOK_URL` | - | Webhook 告警 URL |
|
||||
| `SMTP_USER` | —(空) | SMTP 用户名 |
|
||||
| `SMTP_PASSWORD` | —(空) | SMTP 密码 |
|
||||
| `SMTP_FROM` | —(空) | 发件人地址(留空则用 SMTP 用户名) |
|
||||
| `FEISHU_WEBHOOK_URL` | —(空) | 飞书机器人 Webhook(环境变量兜底;通知实际用数据库「系统设置」里的值) |
|
||||
|
||||
Docker Compose 专用变量(不在 `config.py` 中,由 compose / InfluxDB 容器使用):`INFLUXDB_USERNAME`、`INFLUXDB_PASSWORD`、`INFLUXDB_PORT`(默认 `8086`)、`COMPOSE_PROJECT_NAME`(默认 `ipmi-monitor`,影响命名卷名称,生产环境保持稳定)、`BACKEND_PORT`(默认 `8000`)。
|
||||
|
||||
说明:
|
||||
|
||||
- **告警阈值在页面告警规则里逐条配置**(条件、阈值、持续时长、通知渠道);不存在全局 `ALERT_*` 环境变量,旧文档中的 `ALERT_TEMP_THRESHOLD` / `ALERT_FAN_THRESHOLD` / `ALERT_POWER_THRESHOLD`、`ALERT_EMAIL_FROM` / `ALERT_EMAIL_TO`、`WEBHOOK_URL` 均已被删除,无人读取。
|
||||
- BMC 口令、SMTP 口令、飞书 Webhook URL 入库前用 Fernet 加密存储;API 永不返回这些秘密的明文(主机列表只返回 `has_password`,设置接口只返回 `smtp_password_set` 与 `feishu_webhook_masked` 后 4 位掩码),更新时秘密字段留空表示不改。
|
||||
|
||||
## 数据持久化与生产环境保护
|
||||
|
||||
@@ -246,106 +282,99 @@ rm -rf data/
|
||||
|
||||
## API 文档概览
|
||||
|
||||
启动服务后,访问 `http://your-server:8000/docs` 查看完整的 Swagger UI 文档。
|
||||
启动服务后,非生产环境访问 `http://your-server:8000/docs` 查看完整的 Swagger UI 文档。除登录外所有接口均需 JWT 鉴权(`Authorization: Bearer <token>`)。
|
||||
|
||||
### 主要 API 端点
|
||||
|
||||
#### 健康检查
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
|------|------|------|
|
||||
| `GET` | `/api/health` | 健康检查 |
|
||||
| `GET` | `/api/servers` | 获取所有服务器列表 |
|
||||
| `POST` | `/api/servers` | 添加服务器 |
|
||||
| `PUT` | `/api/servers/{id}` | 更新服务器信息 |
|
||||
| `DELETE` | `/api/servers/{id}` | 删除服务器 |
|
||||
| `POST` | `/api/servers/{id}/scan` | 立即扫描服务器传感器 |
|
||||
| `GET` | `/api/servers/{id}/sensors` | 获取服务器传感器数据 |
|
||||
| `GET` | `/api/servers/{id}/metrics` | 获取历史指标数据 |
|
||||
| `GET` | `/api/metrics/temperature` | 获取所有服务器温度数据 |
|
||||
| `GET` | `/api/metrics/fan` | 获取所有服务器风扇数据 |
|
||||
| `GET` | `/api/metrics/power` | 获取所有服务器功耗数据 |
|
||||
| `GET` | `/api/alerts` | 获取告警列表 |
|
||||
| `PUT` | `/api/alerts/config` | 更新告警配置 |
|
||||
| `GET` | `/api/dashboard` | 获取仪表盘汇总数据 |
|
||||
| `GET` | `/api/health` | 健康检查,返回 `{"status": "ok"}` |
|
||||
|
||||
#### 认证(`backend/app/routers/auth.py`)
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
|------|------|------|
|
||||
| `POST` | `/api/auth/login` | 用户名 + 密码登录,返回 JWT |
|
||||
| `POST` | `/api/auth/verify` | 校验 token 是否有效 |
|
||||
| `POST` | `/api/auth/logout` | 吊销当前 token |
|
||||
| `POST` | `/api/auth/change-password` | 改当前用户密码(新密码至少 12 位;成功后此前所有 token 失效) |
|
||||
| `GET` | `/api/auth/me` | 取当前登录用户信息 |
|
||||
|
||||
#### 主机(`backend/app/routers/hosts.py`)
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
|------|------|------|
|
||||
| `GET` | `/api/hosts` | 主机列表(不返回口令明文,只返回 `has_password`) |
|
||||
| `POST` | `/api/hosts` | 添加主机 |
|
||||
| `PUT` | `/api/hosts/{host_id}` | 更新主机(口令留空表示不改) |
|
||||
| `DELETE` | `/api/hosts/{host_id}` | 删除主机 |
|
||||
| `POST` | `/api/hosts/{host_id}/test` | 连接测试 |
|
||||
| `GET` | `/api/hosts/{host_id}/status` | 主机状态 |
|
||||
|
||||
#### 分组(`backend/app/routers/groups.py`)
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
|------|------|------|
|
||||
| `GET` | `/api/groups` | 分组列表 |
|
||||
| `POST` | `/api/groups` | 新建分组 |
|
||||
| `PUT` | `/api/groups/{group_id}` | 更新分组 |
|
||||
| `DELETE` | `/api/groups/{group_id}` | 删除分组 |
|
||||
|
||||
#### 监控数据(`backend/app/routers/monitor.py`)
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
|------|------|------|
|
||||
| `GET` | `/api/monitor/dashboard` | 仪表盘汇总(主机数、在线数、未确认告警数) |
|
||||
| `GET` | `/api/monitor/dashboard/totals` | 各主机总功耗 / 最高温度汇总(可按分组筛选) |
|
||||
| `GET` | `/api/monitor/history/{host_id}` | 单主机历史指标 |
|
||||
| `GET` | `/api/monitor/history/multi` | 多主机历史指标(`host_ids` 逗号分隔,最多 50 个) |
|
||||
| `GET` | `/api/monitor/summary/{metric_type}` | 指定类型传感器的最新值列表 |
|
||||
| `GET` | `/api/monitor/sensors/{host_id}` | 单主机全部传感器最新值 |
|
||||
| `GET` | `/api/monitor/realtime/{host_id}` | 单主机最新指标(按类型分组) |
|
||||
| `GET` | `/api/monitor/trend` | 分组趋势数据(时间范围 1m–30d) |
|
||||
| `GET` | `/api/monitor/components/{host_id}` | 单主机组件在位状态 |
|
||||
| `GET` | `/api/monitor/components/{host_id}/grouped` | 单主机组件在位状态(按在位 / 缺失分组) |
|
||||
| `GET` | `/api/monitor/export` | 历史指标 CSV 导出(`host_ids`、`metric_type`、`start`、`end`;列:time,host_id,metric_type,metric_name,value,unit) |
|
||||
|
||||
#### 告警(`backend/app/routers/alerts.py`)
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
|------|------|------|
|
||||
| `GET` | `/api/alerts/rules` | 告警规则列表(可按主机 / 分组 / 指标类型筛选) |
|
||||
| `POST` | `/api/alerts/rules` | 新建告警规则 |
|
||||
| `PUT` | `/api/alerts/rules/{rule_id}` | 更新告警规则 |
|
||||
| `DELETE` | `/api/alerts/rules/{rule_id}` | 删除告警规则 |
|
||||
| `GET` | `/api/alerts/history` | 告警历史(分页,可按主机 / 规则 / 确认状态筛选) |
|
||||
| `PUT` | `/api/alerts/history/{history_id}/acknowledge` | 确认一条告警 |
|
||||
|
||||
#### 系统设置(`backend/app/routers/settings.py`)
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
|------|------|------|
|
||||
| `GET` | `/api/settings` | 取系统设置(秘密只返回掩码 / 是否已配置) |
|
||||
| `PUT` | `/api/settings` | 更新系统设置(秘密字段留空表示不改) |
|
||||
| `POST` | `/api/settings/test-email` | 发送测试邮件 |
|
||||
| `POST` | `/api/settings/test-feishu` | 发送飞书测试消息 |
|
||||
|
||||
## 告警配置说明
|
||||
|
||||
### 告警触发条件
|
||||
|
||||
- **温度告警**: 当任意温度传感器读数超过 `ALERT_TEMP_THRESHOLD` 时触发
|
||||
- **风扇告警**: 当风扇转速低于 `ALERT_FAN_THRESHOLD` 时触发(设为 0 禁用)
|
||||
- **功耗告警**: 当功耗超过 `ALERT_POWER_THRESHOLD` 时触发(设为 0 禁用)
|
||||
阈值在「告警规则」页面逐条配置,每条规则指定:指标类型(temperature / fan / voltage / power / component / status)、传感器名、条件(大于 / 小于 / 等于 / 大于等于 / 小于等于)、阈值、持续时长(超标持续满该时长才触发)、适用范围(单主机 / 分组 / 全部主机)、通知渠道(邮件、飞书)与收件邮箱。触发后 3 小时内不重复通知;指标恢复正常后发送恢复通知(仅当该告警此前确实触发过)。
|
||||
|
||||
### 告警通知渠道
|
||||
|
||||
#### 邮件通知
|
||||
|
||||
在 `.env` 中配置 SMTP 信息即可启用:
|
||||
在「系统设置 → 邮件通知」中配置 SMTP 信息(服务器、端口、用户名、密码、发件人、TLS),保存后用「发送测试邮件」验证。告警按规则的收件邮箱列表发送。
|
||||
|
||||
```env
|
||||
SMTP_HOST=smtp.gmail.com
|
||||
SMTP_PORT=587
|
||||
SMTP_USER=your-email@gmail.com
|
||||
SMTP_PASSWORD=your-app-password
|
||||
ALERT_EMAIL_FROM=your-email@gmail.com
|
||||
ALERT_EMAIL_TO=admin1@example.com,admin2@example.com
|
||||
```
|
||||
#### 飞书通知
|
||||
|
||||
#### Webhook 通知
|
||||
在「系统设置 → 飞书通知」中配置群机器人 Webhook 地址(可配安全关键词),保存后用「发送测试消息」验证。告警以交互式卡片推送到飞书群。
|
||||
|
||||
支持标准 Webhook 格式,可对接以下平台:
|
||||
|
||||
- **Slack**: 填入 Slack Incoming Webhook URL
|
||||
- **钉钉**: 填入钉钉机器人 Webhook URL
|
||||
- **企业微信**: 填入企业微信机器人 Webhook URL
|
||||
- **自定义**: 填入任意 Webhook URL,POST JSON 格式通知
|
||||
|
||||
```env
|
||||
WEBHOOK_URL=https://hooks.slack.com/services/xxx/yyy/zzz
|
||||
```
|
||||
|
||||
## 截图
|
||||
|
||||
### 仪表盘
|
||||
|
||||
> 📸 **仪表盘页面**
|
||||
>
|
||||
> 展示所有服务器的概览信息,包括:
|
||||
> - 服务器在线/离线状态
|
||||
> - 关键温度指标
|
||||
> - 风扇运行状态
|
||||
> - 实时功耗统计
|
||||
> - 最近告警摘要
|
||||
|
||||
### 服务器详情
|
||||
|
||||
> 📸 **服务器详情页面**
|
||||
>
|
||||
> 单台服务器的详细监控数据:
|
||||
> - 所有传感器列表及当前读数
|
||||
> - 温度趋势图(1h / 6h / 24h / 7d)
|
||||
> - 风扇转速历史图表
|
||||
> - 电压稳定性图表
|
||||
> - 功耗变化趋势
|
||||
|
||||
### 告警管理
|
||||
|
||||
> 📸 **告警管理页面**
|
||||
>
|
||||
> 告警记录和配置管理:
|
||||
> - 历史告警列表
|
||||
> - 告警阈值配置
|
||||
> - 通知渠道设置
|
||||
> - 告警确认/静默操作
|
||||
|
||||
### 服务器管理
|
||||
|
||||
> 📸 **服务器管理页面**
|
||||
>
|
||||
> 添加和管理监控服务器:
|
||||
> - 服务器列表(名称、IP、状态)
|
||||
> - 添加/编辑/删除服务器
|
||||
> - 连接测试
|
||||
> - 批量操作
|
||||
> 仅支持以上两种渠道。Slack / 钉钉 / 企业微信没有实现,不要按旧文档配置。
|
||||
|
||||
## 常见问题
|
||||
|
||||
@@ -368,6 +397,14 @@ WEBHOOK_URL=https://hooks.slack.com/services/xxx/yyy/zzz
|
||||
2. 确认容器网络可达 IPMI 地址
|
||||
3. 查看后端日志排查具体错误
|
||||
|
||||
### 组件页提示原始数据文件缺失
|
||||
|
||||
组件在位状态读取采集时写下的 `/tmp/ipmi_raw_{host_id}.txt`;该文件丢失(如容器重启)时组件接口会报 404,等待下一轮采集(60 秒)后恢复。
|
||||
|
||||
### 前端改了代码页面没变化
|
||||
|
||||
后端托管的是 `frontend/dist` 构建产物,改 `frontend/src` 后需在 `frontend/` 下执行 `npm run build` 重新生成 `dist`,再重启后端。
|
||||
|
||||
## License
|
||||
|
||||
This project is licensed under the MIT License. See the [LICENSE](LICENSE) file for details.
|
||||
|
||||
Regular → Executable
+98
-47
@@ -1,71 +1,122 @@
|
||||
#!/bin/bash
|
||||
# IPMI告警触发器 - 检测到告警后自动SSH执行脚本
|
||||
# 用法: ./alert-trigger.sh --api-url http://localhost:8003 --ssh-key ~/.ssh/id_rsa
|
||||
# IPMI 告警触发器 - 轮询未确认告警,对涉及的主机执行远程诊断
|
||||
#
|
||||
# 用法:
|
||||
# ./alert-trigger.sh --api-url http://localhost:8003 --token <API_TOKEN> \
|
||||
# --ssh-key ~/.ssh/id_ed25519 [--ssh-user monitor] [--interval 30]
|
||||
#
|
||||
# 依赖: curl, jq
|
||||
# 状态文件: $HOME/.ipmi-monitor/alert-state.json (0600)
|
||||
set -euo pipefail
|
||||
|
||||
API_URL="${1:-http://localhost:8003}"
|
||||
SSH_KEY="${2:-~/.ssh/id_rsa}"
|
||||
CHECK_INTERVAL=30 # 检查间隔(秒)
|
||||
STATE_FILE="/tmp/ipmi-alert-state.json"
|
||||
API_URL="http://localhost:8003"
|
||||
TOKEN=""
|
||||
SSH_KEY="$HOME/.ssh/id_ed25519"
|
||||
SSH_USER="monitor"
|
||||
CHECK_INTERVAL=30
|
||||
STATE_DIR="$HOME/.ipmi-monitor"
|
||||
STATE_FILE="$STATE_DIR/alert-state.json"
|
||||
|
||||
# 初始化状态文件
|
||||
usage() {
|
||||
echo "用法: $0 --api-url <url> --token <token> --ssh-key <path> [--ssh-user <user>] [--interval <seconds>]"
|
||||
exit 2
|
||||
}
|
||||
|
||||
# 按 --flag value 解析参数(O14:旧版按位置参数取,--api-url 会被当成 URL)
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--api-url) API_URL="${2:?--api-url 需要一个值}"; shift 2 ;;
|
||||
--token) TOKEN="${2:?--token 需要一个值}"; shift 2 ;;
|
||||
--ssh-key) SSH_KEY="${2:?--ssh-key 需要一个值}"; shift 2 ;;
|
||||
--ssh-user) SSH_USER="${2:?--ssh-user 需要一个值}"; shift 2 ;;
|
||||
--interval) CHECK_INTERVAL="${2:?--interval 需要一个值}"; shift 2 ;;
|
||||
-h|--help) usage ;;
|
||||
*) echo "未知参数: $1" >&2; usage ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [ -z "$TOKEN" ]; then
|
||||
echo "错误: 必须提供 --token(API Bearer token)" >&2
|
||||
usage
|
||||
fi
|
||||
|
||||
for cmd in curl jq; do
|
||||
if ! command -v "$cmd" >/dev/null 2>&1; then
|
||||
echo "错误: 缺少依赖 $cmd,请先安装" >&2
|
||||
exit 1
|
||||
fi
|
||||
done
|
||||
|
||||
if [ ! -f "$SSH_KEY" ]; then
|
||||
echo "错误: SSH 私钥不存在: $SSH_KEY" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# 状态文件:目录 0700、文件 0600
|
||||
mkdir -p "$STATE_DIR"
|
||||
chmod 0700 "$STATE_DIR"
|
||||
if [ ! -f "$STATE_FILE" ]; then
|
||||
echo '{}' > "$STATE_FILE"
|
||||
fi
|
||||
chmod 0600 "$STATE_FILE"
|
||||
|
||||
echo "=== IPMI告警触发器 ==="
|
||||
echo "=== IPMI 告警触发器 ==="
|
||||
echo "API地址: $API_URL"
|
||||
echo "SSH 用户: $SSH_USER"
|
||||
echo "检查间隔: ${CHECK_INTERVAL}s"
|
||||
echo ""
|
||||
|
||||
while true; do
|
||||
# 获取当前告警
|
||||
ALERTS=$(curl -s "$API_URL/api/alerts" 2>/dev/null)
|
||||
|
||||
if [ -z "$ALERTS" ] || [ "$ALERTS" = "[]" ]; then
|
||||
sleep $CHECK_INTERVAL
|
||||
# 真实接口:GET /api/alerts/history?acknowledged=false,Bearer 鉴权,
|
||||
# 返回分页对象 {total, page, page_size, items: [...]}
|
||||
RESPONSE=$(curl -sS --fail --max-time 15 \
|
||||
-H "Authorization: Bearer $TOKEN" \
|
||||
"$API_URL/api/alerts/history?acknowledged=false&page_size=100") || {
|
||||
echo "[$(date '+%Y-%m-%d %H:%M:%S')] API 请求失败,${CHECK_INTERVAL}s 后重试" >&2
|
||||
sleep "$CHECK_INTERVAL"
|
||||
continue
|
||||
fi
|
||||
|
||||
# 解析告警信息(使用jq)
|
||||
echo "[$(date '+%Y-%m-%d %H:%M:%S')] 检测到告警:"
|
||||
echo "$ALERTS" | jq -r '.[] | " - [\(.severity | ascii_upcase)] \(.host): \(.message)"' 2>/dev/null
|
||||
|
||||
# 遍历告警,按主机执行脚本
|
||||
echo "$ALERTS" | jq -r '.[] | "\(.host)|\(.severity)|\(.sensor_type)|\(.message)"' 2>/dev/null | while IFS='|' read -r host severity sensor_type message; do
|
||||
# 检查是否已经处理过这个告警(避免重复触发)
|
||||
LAST_STATE=$(cat "$STATE_FILE" | jq -r ".\"${host}_${sensor_type}\" // \"\"" 2>/dev/null)
|
||||
CURRENT_STATE="${message}"
|
||||
|
||||
if [ "$LAST_STATE" = "$CURRENT_STATE" ]; then
|
||||
}
|
||||
|
||||
# jq 传参一律用 --arg,避免把变量拼进过滤器(注入风险)
|
||||
mapfile -t ALERTS < <(echo "$RESPONSE" | jq -r '
|
||||
(.items // [])[]
|
||||
| [.id, (.host_name // ""), (.rule_name // ""), (.metric_name // ""), (.message // "")]
|
||||
| @tsv')
|
||||
|
||||
for line in "${ALERTS[@]}"; do
|
||||
IFS=$'\t' read -r alert_id host_name rule_name metric_name message <<< "$line"
|
||||
[ -n "$alert_id" ] || continue
|
||||
|
||||
# 以告警 id 去重:已处理过的 id 不再触发
|
||||
if jq -e --arg id "$alert_id" 'has($id)' "$STATE_FILE" >/dev/null; then
|
||||
continue
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo ">>> 触发告警: $host | $severity | $sensor_type"
|
||||
echo ">>> 消息: $message"
|
||||
|
||||
# 自动SSH到目标主机执行脚本
|
||||
# 方式1: SSH到告警对应的主机(如果host是IP地址)
|
||||
if [[ "$host" =~ ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
|
||||
echo ">>> SSH到 $host 执行诊断脚本..."
|
||||
ssh -i "$SSH_KEY" -o StrictHostKeyChecking=no -o ConnectTimeout=10 root@"$host" '
|
||||
# 在这里写你要执行的脚本
|
||||
|
||||
echo "[$(date '+%Y-%m-%d %H:%M:%S')] 检测到未确认告警 #$alert_id: [$host_name] $rule_name - $message"
|
||||
|
||||
# host_name 是显示名不是 IP;这里只对 IP 格式的目标执行 SSH
|
||||
target="$host_name"
|
||||
if [[ "$target" =~ ^[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+$ ]]; then
|
||||
echo ">>> SSH 到 $target 执行诊断脚本..."
|
||||
ssh -i "$SSH_KEY" -o StrictHostKeyChecking=accept-new \
|
||||
-o ConnectTimeout=10 "${SSH_USER}@${target}" '
|
||||
echo "=== 主机诊断 ==="
|
||||
hostname
|
||||
uptime
|
||||
free -h
|
||||
df -h
|
||||
# 添加你的自定义命令
|
||||
' 2>&1
|
||||
' 2>&1 || echo ">>> SSH 执行失败(主机 $target)" >&2
|
||||
else
|
||||
# 方式2: host是主机名,需要通过IPMI获取IP
|
||||
echo ">>> 主机名: $host,跳过SSH(需要提供IP地址)"
|
||||
echo ">>> 主机名 '$target' 不是 IP 地址,跳过 SSH(需提供 IP)"
|
||||
fi
|
||||
|
||||
# 更新状态
|
||||
TEMP=$(cat "$STATE_FILE")
|
||||
echo "$TEMP" | jq ".\"${host}_${sensor_type}\" = \"$CURRENT_STATE\"" > "$STATE_FILE" 2>/dev/null
|
||||
|
||||
# 原子更新状态文件(jq --arg 传参,临时文件 + mv)
|
||||
tmp_state="$(mktemp "$STATE_DIR/.state.XXXXXX")"
|
||||
jq --arg id "$alert_id" --arg msg "$message" '.[$id] = $msg' \
|
||||
"$STATE_FILE" > "$tmp_state"
|
||||
chmod 0600 "$tmp_state"
|
||||
mv "$tmp_state" "$STATE_FILE"
|
||||
done
|
||||
|
||||
sleep $CHECK_INTERVAL
|
||||
|
||||
sleep "$CHECK_INTERVAL"
|
||||
done
|
||||
|
||||
+11
-6
@@ -13,19 +13,21 @@ RUN apt-get update && \
|
||||
# Set working directory
|
||||
WORKDIR /app
|
||||
|
||||
# Use pip mirror for faster downloads
|
||||
RUN pip config set global.index-url https://mirrors.aliyun.com/pypi/simple/ && \
|
||||
pip config set global.trusted-host mirrors.aliyun.com
|
||||
# Use pip mirror for faster downloads (HTTPS only, no --trusted-host)
|
||||
RUN pip config set global.index-url https://mirrors.aliyun.com/pypi/simple/
|
||||
|
||||
# Create a dedicated non-root user for the application
|
||||
RUN groupadd -r appuser && useradd -r -g appuser -m appuser
|
||||
|
||||
# Copy and install Python dependencies
|
||||
COPY requirements.txt .
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
|
||||
# Copy application code
|
||||
COPY . .
|
||||
# Copy application code (owned by the non-root user)
|
||||
COPY --chown=appuser:appuser . .
|
||||
|
||||
# Create data directory for persistent storage
|
||||
RUN mkdir -p /app/data
|
||||
RUN mkdir -p /app/data && chown -R appuser:appuser /app/data
|
||||
|
||||
# Expose port
|
||||
EXPOSE 8000
|
||||
@@ -34,5 +36,8 @@ EXPOSE 8000
|
||||
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
|
||||
CMD curl -f http://localhost:8000/api/health || exit 1
|
||||
|
||||
# Run as the non-root user
|
||||
USER appuser
|
||||
|
||||
# Start uvicorn server
|
||||
CMD ["sh", "-c", "uvicorn app.main:app --host 0.0.0.0 --port ${PORT:-8000}"]
|
||||
|
||||
+73
-5
@@ -1,12 +1,29 @@
|
||||
"""Application configuration using pydantic-settings."""
|
||||
|
||||
import hashlib
|
||||
|
||||
from pydantic import AliasChoices, Field
|
||||
from pydantic_settings import BaseSettings
|
||||
from typing import Optional
|
||||
|
||||
# SHA-256 of the legacy hard-coded JWT placeholder that used to ship as the
|
||||
# default value. The literal value itself is deliberately not stored here;
|
||||
# startup validation rejects any key whose hash matches this digest.
|
||||
_LEGACY_JWT_DEFAULT_SHA256 = (
|
||||
"c99758db33af70fbf3588833c77678baa8d763407fcb82c58509e6332a63e21c"
|
||||
)
|
||||
|
||||
# Minimum accepted JWT secret length (characters).
|
||||
MIN_JWT_SECRET_LENGTH = 32
|
||||
|
||||
|
||||
class Settings(BaseSettings):
|
||||
"""Application settings loaded from environment variables."""
|
||||
|
||||
# Runtime environment: "development" or "production". Production disables
|
||||
# the interactive API docs and enables stricter defaults.
|
||||
ENVIRONMENT: str = "development"
|
||||
|
||||
# Database
|
||||
DATABASE_URL: str = "sqlite+aiosqlite:///./data/ipmi_monitor.db"
|
||||
# Production safety guard: when enabled, refuse to boot if the SQLite DB
|
||||
@@ -21,10 +38,14 @@ class Settings(BaseSettings):
|
||||
INFLUXDB_URL: str = "http://localhost:8086"
|
||||
INFLUXDB_TOKEN: str = ""
|
||||
INFLUXDB_ORG: str = "ipmi-monitor"
|
||||
INFLUXDB_BUCKET: str = "ipmi_metrics"
|
||||
INFLUXDB_BUCKET: str = "metrics"
|
||||
|
||||
# Collection
|
||||
COLLECT_INTERVAL: int = 60 # seconds
|
||||
# Collection (POLL_INTERVAL is the historical env-var name, kept as an
|
||||
# alias so existing deployments keep working).
|
||||
COLLECT_INTERVAL: int = Field(
|
||||
default=60,
|
||||
validation_alias=AliasChoices("COLLECT_INTERVAL", "POLL_INTERVAL"),
|
||||
) # seconds
|
||||
|
||||
# SMTP / Email
|
||||
SMTP_HOST: str = ""
|
||||
@@ -39,10 +60,29 @@ class Settings(BaseSettings):
|
||||
# Logging
|
||||
LOG_LEVEL: str = "INFO"
|
||||
|
||||
# CORS: comma-separated list of allowed origins. Empty means CORS is not
|
||||
# enabled at all (same-origin deployment).
|
||||
CORS_ORIGINS: str = ""
|
||||
|
||||
# JWT Authentication
|
||||
JWT_SECRET_KEY: str = "your-secret-key-change-this-in-production"
|
||||
# SECURITY: no default value. JWT_SECRET_KEY must be injected via the
|
||||
# environment (see .env.example / docker-compose.yml). Startup refuses
|
||||
# to boot without a key of at least 32 characters (see
|
||||
# validate_security_settings()).
|
||||
JWT_SECRET_KEY: str = ""
|
||||
JWT_ALGORITHM: str = "HS256"
|
||||
JWT_EXPIRATION_HOURS: int = 24
|
||||
JWT_EXPIRATION_HOURS: int = 8
|
||||
|
||||
# Initial admin account. Only consulted when the user table is empty:
|
||||
# ADMIN_INITIAL_PASSWORD (at least 12 characters) is then required to
|
||||
# create the first admin user at startup. Ignored once any user exists.
|
||||
ADMIN_INITIAL_PASSWORD: str = ""
|
||||
|
||||
# Credential encryption (Fernet key for BMC passwords / SMTP password /
|
||||
# Feishu webhook URL stored in the database). Optional: when unset, a key
|
||||
# is generated once and persisted next to the SQLite file (see
|
||||
# app/crypto.py).
|
||||
CREDENTIAL_ENCRYPTION_KEY: str = ""
|
||||
|
||||
model_config = {
|
||||
"env_file": ".env",
|
||||
@@ -51,3 +91,31 @@ class Settings(BaseSettings):
|
||||
|
||||
|
||||
settings = Settings()
|
||||
|
||||
|
||||
def validate_security_settings(s: Optional[Settings] = None) -> None:
|
||||
"""Fail fast on unsafe security configuration.
|
||||
|
||||
Called only from the application startup path (lifespan), never at
|
||||
import time, so importing this module (e.g. in tests) stays side-effect
|
||||
free. Raises RuntimeError when the JWT secret is missing, equals the
|
||||
legacy shipped default, or is shorter than 32 characters.
|
||||
"""
|
||||
cfg = s if s is not None else settings
|
||||
key = cfg.JWT_SECRET_KEY or ""
|
||||
if not key:
|
||||
raise RuntimeError(
|
||||
"Refusing to start: JWT_SECRET_KEY is not set. Generate a secret "
|
||||
"with at least 32 characters and provide it via the JWT_SECRET_KEY "
|
||||
"environment variable (see .env.example)."
|
||||
)
|
||||
if hashlib.sha256(key.encode("utf-8")).hexdigest() == _LEGACY_JWT_DEFAULT_SHA256:
|
||||
raise RuntimeError(
|
||||
"Refusing to start: JWT_SECRET_KEY still equals the legacy shipped "
|
||||
"default value. Generate a fresh secret of at least 32 characters."
|
||||
)
|
||||
if len(key) < MIN_JWT_SECRET_LENGTH:
|
||||
raise RuntimeError(
|
||||
"Refusing to start: JWT_SECRET_KEY must be at least "
|
||||
f"{MIN_JWT_SECRET_LENGTH} characters long (got {len(key)})."
|
||||
)
|
||||
|
||||
@@ -0,0 +1,130 @@
|
||||
"""Application-layer encryption for stored credentials.
|
||||
|
||||
BMC (IPMI) passwords, the SMTP password and the Feishu webhook URL are stored
|
||||
in the database encrypted with Fernet (AES-128-CBC + HMAC-SHA256, via the
|
||||
``cryptography`` package) instead of plaintext.
|
||||
|
||||
Key resolution order:
|
||||
1. ``CREDENTIAL_ENCRYPTION_KEY`` environment variable — a urlsafe base64
|
||||
32-byte Fernet key (generate with
|
||||
``python -c "from cryptography.fernet import Fernet; print(Fernet.generate_key().decode())"``).
|
||||
2. Otherwise a key file ``.credential_key`` next to the SQLite database file
|
||||
(created with mode 0600 on first use and persisted across restarts).
|
||||
|
||||
``decrypt_value`` deliberately returns undecryptable input unchanged so
|
||||
pre-existing plaintext rows keep working until the one-time migration in
|
||||
``app.database.init_db`` re-encrypts them.
|
||||
"""
|
||||
|
||||
import base64
|
||||
import binascii
|
||||
import logging
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
from cryptography.fernet import Fernet, InvalidToken
|
||||
|
||||
from app.config import settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_KEY_FILE_NAME = ".credential_key"
|
||||
|
||||
_fernet: Fernet | None = None
|
||||
|
||||
|
||||
def _database_dir() -> Path:
|
||||
"""Directory holding the SQLite file (fallback: ./data)."""
|
||||
url = settings.DATABASE_URL or ""
|
||||
for prefix in ("sqlite+aiosqlite:///", "sqlite:///"):
|
||||
if url.startswith(prefix):
|
||||
raw = url.removeprefix(prefix)
|
||||
if raw and raw != ":memory:":
|
||||
path = Path(raw)
|
||||
if not path.is_absolute():
|
||||
path = Path.cwd() / path
|
||||
return path.parent
|
||||
return Path.cwd() / "data"
|
||||
|
||||
|
||||
def _load_or_create_key_file() -> bytes:
|
||||
"""Read the persisted key file, creating it (0600) on first use."""
|
||||
directory = _database_dir()
|
||||
directory.mkdir(parents=True, exist_ok=True)
|
||||
key_path = directory / _KEY_FILE_NAME
|
||||
if key_path.exists():
|
||||
raw = key_path.read_bytes().strip()
|
||||
if raw:
|
||||
return raw
|
||||
logger.warning("Credential key file %s is empty; regenerating it", key_path)
|
||||
key = Fernet.generate_key()
|
||||
# Write atomically-ish with restrictive permissions from the start.
|
||||
fd = os.open(key_path, os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)
|
||||
try:
|
||||
os.write(fd, key)
|
||||
finally:
|
||||
os.close(fd)
|
||||
try:
|
||||
os.chmod(key_path, 0o600)
|
||||
except OSError:
|
||||
pass
|
||||
logger.info("Generated credential encryption key file at %s", key_path)
|
||||
return key
|
||||
|
||||
|
||||
def get_fernet() -> Fernet:
|
||||
"""Return the cached Fernet instance for credential encryption."""
|
||||
global _fernet
|
||||
if _fernet is not None:
|
||||
return _fernet
|
||||
|
||||
env_key = (settings.CREDENTIAL_ENCRYPTION_KEY or "").strip()
|
||||
if env_key:
|
||||
try:
|
||||
raw = base64.urlsafe_b64decode(env_key.encode("ascii"))
|
||||
if len(raw) != 32:
|
||||
raise ValueError("decoded key is not 32 bytes")
|
||||
_fernet = Fernet(env_key.encode("ascii"))
|
||||
return _fernet
|
||||
except (ValueError, binascii.Error) as exc:
|
||||
raise ValueError(
|
||||
"CREDENTIAL_ENCRYPTION_KEY is not a valid urlsafe base64 "
|
||||
"32-byte Fernet key"
|
||||
) from exc
|
||||
|
||||
_fernet = Fernet(_load_or_create_key_file())
|
||||
return _fernet
|
||||
|
||||
|
||||
def encrypt_value(plaintext: str | None) -> str | None:
|
||||
"""Encrypt a credential value. None/empty input is returned unchanged."""
|
||||
if not plaintext:
|
||||
return plaintext
|
||||
token = get_fernet().encrypt(plaintext.encode("utf-8"))
|
||||
return token.decode("ascii")
|
||||
|
||||
|
||||
def decrypt_value(token: str | None) -> str | None:
|
||||
"""Decrypt a credential value.
|
||||
|
||||
Returns the input unchanged when it is not a valid Fernet token (legacy
|
||||
plaintext rows) or when it is None/empty.
|
||||
"""
|
||||
if not token:
|
||||
return token
|
||||
try:
|
||||
return get_fernet().decrypt(token.encode("ascii")).decode("utf-8")
|
||||
except (InvalidToken, ValueError):
|
||||
# Legacy plaintext stored before encryption was introduced.
|
||||
return token
|
||||
|
||||
|
||||
def is_encrypted(value: str | None) -> bool:
|
||||
"""Return True when *value* is a valid Fernet token for the current key."""
|
||||
if not value:
|
||||
return False
|
||||
try:
|
||||
get_fernet().decrypt(value.encode("ascii"))
|
||||
return True
|
||||
except (InvalidToken, ValueError):
|
||||
return False
|
||||
+97
-13
@@ -1,11 +1,12 @@
|
||||
"""SQLAlchemy async engine and session management."""
|
||||
|
||||
import logging
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
from sqlalchemy import event, func, select, text
|
||||
from sqlalchemy.ext.asyncio import AsyncSession, create_async_engine, async_sessionmaker
|
||||
from sqlalchemy.orm import DeclarativeBase
|
||||
from sqlalchemy import func, select
|
||||
|
||||
from app.config import settings
|
||||
|
||||
@@ -16,6 +17,21 @@ engine = create_async_engine(
|
||||
echo=False,
|
||||
)
|
||||
|
||||
|
||||
def _set_sqlite_pragmas(dbapi_connection, connection_record):
|
||||
"""Apply per-connection SQLite pragmas (WAL, FK enforcement, timeout)."""
|
||||
cursor = dbapi_connection.cursor()
|
||||
try:
|
||||
cursor.execute("PRAGMA journal_mode=WAL")
|
||||
cursor.execute("PRAGMA foreign_keys=ON")
|
||||
cursor.execute("PRAGMA busy_timeout=5000")
|
||||
finally:
|
||||
cursor.close()
|
||||
|
||||
|
||||
if settings.DATABASE_URL.startswith("sqlite"):
|
||||
event.listens_for(engine.sync_engine, "connect")(_set_sqlite_pragmas)
|
||||
|
||||
async_session = async_sessionmaker(
|
||||
engine,
|
||||
class_=AsyncSession,
|
||||
@@ -46,11 +62,15 @@ def _sqlite_database_path(database_url: str) -> Path | None:
|
||||
|
||||
|
||||
async def get_db() -> AsyncSession: # type: ignore[misc]
|
||||
"""FastAPI dependency that yields a database session."""
|
||||
"""FastAPI dependency that yields a database session.
|
||||
|
||||
Read-only by default: this dependency never commits. Write endpoints
|
||||
must call ``await db.commit()`` explicitly (and let the exception
|
||||
propagate if the commit fails); on error the transaction is rolled back.
|
||||
"""
|
||||
async with async_session() as session:
|
||||
try:
|
||||
yield session
|
||||
await session.commit()
|
||||
except Exception:
|
||||
await session.rollback()
|
||||
raise
|
||||
@@ -58,8 +78,53 @@ async def get_db() -> AsyncSession: # type: ignore[misc]
|
||||
await session.close()
|
||||
|
||||
|
||||
async def _create_indexes(conn) -> None:
|
||||
"""Create indexes used by hot query paths (idempotent)."""
|
||||
for name, table, column in [
|
||||
("idx_alert_history_host_id", "alert_history", "host_id"),
|
||||
("idx_alert_rules_host_id", "alert_rules", "host_id"),
|
||||
("idx_hosts_group_id", "hosts", "group_id"),
|
||||
("idx_system_settings_key", "system_settings", "key"),
|
||||
]:
|
||||
await conn.execute(
|
||||
text(f"CREATE INDEX IF NOT EXISTS {name} ON {table} ({column})")
|
||||
)
|
||||
|
||||
|
||||
async def _migrate_host_passwords() -> None:
|
||||
"""One-time migration: re-encrypt any plaintext IPMI passwords in place.
|
||||
|
||||
Rows that fail to migrate are logged and left untouched; a migration
|
||||
failure must never block startup.
|
||||
"""
|
||||
from app.crypto import encrypt_value, is_encrypted
|
||||
from app.models import Host
|
||||
|
||||
async with async_session() as session:
|
||||
try:
|
||||
result = await session.execute(select(Host))
|
||||
hosts = result.scalars().all()
|
||||
migrated = 0
|
||||
for host in hosts:
|
||||
try:
|
||||
if host.ipmi_password and not is_encrypted(host.ipmi_password):
|
||||
host.ipmi_password = encrypt_value(host.ipmi_password)
|
||||
migrated += 1
|
||||
except Exception as exc: # noqa: BLE001 - never block startup
|
||||
logger.warning(
|
||||
"Failed to migrate IPMI password for host %s (%s): %s",
|
||||
host.id, host.name, exc,
|
||||
)
|
||||
if migrated:
|
||||
await session.commit()
|
||||
logger.info("Encrypted %d stored IPMI password(s)", migrated)
|
||||
except Exception as exc: # noqa: BLE001 - never block startup
|
||||
logger.warning("IPMI password migration skipped: %s", exc)
|
||||
await session.rollback()
|
||||
|
||||
|
||||
async def init_db() -> None:
|
||||
"""Create tables and ensure default admin user exists."""
|
||||
"""Create tables, indexes, and ensure an initial admin user exists."""
|
||||
db_path = _sqlite_database_path(settings.DATABASE_URL)
|
||||
if settings.REQUIRE_EXISTING_DATABASE and db_path is not None and not db_path.exists():
|
||||
raise RuntimeError(
|
||||
@@ -70,6 +135,7 @@ async def init_db() -> None:
|
||||
|
||||
async with engine.begin() as conn:
|
||||
await conn.run_sync(Base.metadata.create_all)
|
||||
await _create_indexes(conn)
|
||||
|
||||
from app.models import Host
|
||||
|
||||
@@ -85,28 +151,46 @@ async def init_db() -> None:
|
||||
"backend mounted a new/wrong Docker volume. Refusing to boot."
|
||||
)
|
||||
|
||||
# 检查并创建默认管理员账号
|
||||
# One-time migration of legacy plaintext BMC passwords (best effort).
|
||||
await _migrate_host_passwords()
|
||||
|
||||
# Initial admin account: only when the user table is completely empty.
|
||||
# The password must come from the ADMIN_INITIAL_PASSWORD environment
|
||||
# variable (at least 12 characters); it is never hard-coded and never
|
||||
# logged. Existing deployments (any user present) are unaffected.
|
||||
from app.models import User
|
||||
import bcrypt
|
||||
|
||||
async with async_session() as session:
|
||||
try:
|
||||
result = await session.execute(select(User).where(User.username == "admin"))
|
||||
existing_user = result.scalar_one_or_none()
|
||||
result = await session.execute(select(func.count()).select_from(User))
|
||||
user_count = result.scalar() or 0
|
||||
|
||||
if not existing_user:
|
||||
# 创建默认管理员账号
|
||||
default_password = "admin123"
|
||||
hashed_password = bcrypt.hashpw(default_password.encode('utf-8'), bcrypt.gensalt()).decode('utf-8')
|
||||
if user_count == 0:
|
||||
initial_password = (
|
||||
settings.ADMIN_INITIAL_PASSWORD
|
||||
or os.environ.get("ADMIN_INITIAL_PASSWORD", "")
|
||||
)
|
||||
if not initial_password or len(initial_password) < 12:
|
||||
raise RuntimeError(
|
||||
"Refusing to start: no users exist in the database and "
|
||||
"ADMIN_INITIAL_PASSWORD is missing or shorter than 12 "
|
||||
"characters. Set ADMIN_INITIAL_PASSWORD to a strong "
|
||||
"password (at least 12 characters) for the initial "
|
||||
"admin account."
|
||||
)
|
||||
hashed_password = bcrypt.hashpw(
|
||||
initial_password.encode("utf-8"), bcrypt.gensalt()
|
||||
).decode("utf-8")
|
||||
|
||||
new_user = User(
|
||||
username="admin",
|
||||
hashed_password=hashed_password
|
||||
hashed_password=hashed_password,
|
||||
)
|
||||
session.add(new_user)
|
||||
await session.commit()
|
||||
|
||||
logger.info("✓ 默认管理员账号创建成功: admin / admin123")
|
||||
logger.info("已创建初始管理员账号")
|
||||
else:
|
||||
logger.info("✓ 管理员账号已存在")
|
||||
except Exception as e:
|
||||
|
||||
+77
-6
@@ -1,5 +1,6 @@
|
||||
"""InfluxDB 2.x client wrapper for reading/writing IPMI metrics."""
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
from datetime import datetime, timezone
|
||||
from typing import Any, Optional
|
||||
@@ -11,6 +12,10 @@ from app.config import settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# InfluxDB client timeouts (O15/B15): 10s connect/read/write so a slow or
|
||||
# dead InfluxDB cannot hang the event loop forever.
|
||||
INFLUX_TIMEOUT_MS = 10_000
|
||||
|
||||
_client: Optional[InfluxDBClient] = None
|
||||
_write_api = None
|
||||
_query_api = None
|
||||
@@ -24,6 +29,7 @@ def get_client() -> InfluxDBClient:
|
||||
url=settings.INFLUXDB_URL,
|
||||
token=settings.INFLUXDB_TOKEN,
|
||||
org=settings.INFLUXDB_ORG,
|
||||
timeout=INFLUX_TIMEOUT_MS,
|
||||
)
|
||||
return _client
|
||||
|
||||
@@ -57,8 +63,12 @@ def write_metric(
|
||||
metric_name: str,
|
||||
value: float,
|
||||
tags: Optional[dict[str, str]] = None,
|
||||
) -> None:
|
||||
"""Write a single metric point into InfluxDB."""
|
||||
) -> bool:
|
||||
"""Write a single metric point into InfluxDB.
|
||||
|
||||
Returns ``True`` on success, ``False`` on failure (B14) so callers can
|
||||
tell a failed write apart from "no data".
|
||||
"""
|
||||
try:
|
||||
point = (
|
||||
Point(metric_type)
|
||||
@@ -73,14 +83,18 @@ def write_metric(
|
||||
|
||||
get_write_api().write(bucket=settings.INFLUXDB_BUCKET, record=point)
|
||||
logger.debug("Written metric %s/%s=%.2f for host %s", metric_type, metric_name, value, host_id)
|
||||
return True
|
||||
except Exception:
|
||||
logger.exception("Failed to write metric to InfluxDB")
|
||||
return False
|
||||
|
||||
|
||||
def write_metrics_batch(points: list[dict[str, Any]]) -> None:
|
||||
def write_metrics_batch(points: list[dict[str, Any]]) -> bool:
|
||||
"""Write multiple metric points at once.
|
||||
|
||||
Each dict should contain: host_id, metric_type, metric_name, value, tags (optional).
|
||||
|
||||
Returns ``True`` on success, ``False`` on failure (B14).
|
||||
"""
|
||||
try:
|
||||
influx_points: list[Point] = []
|
||||
@@ -99,8 +113,54 @@ def write_metrics_batch(points: list[dict[str, Any]]) -> None:
|
||||
|
||||
get_write_api().write(bucket=settings.INFLUXDB_BUCKET, record=influx_points)
|
||||
logger.debug("Batch written %d metrics", len(influx_points))
|
||||
return True
|
||||
except Exception:
|
||||
logger.exception("Failed to batch-write metrics to InfluxDB")
|
||||
return False
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Async wrappers (B15)
|
||||
#
|
||||
# InfluxDBClient is synchronous — never call it directly from async code.
|
||||
# These wrappers run the blocking calls in a worker thread. The sync
|
||||
# functions above stay available for the few remaining sync call sites.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
async def awrite_metric(*args: Any, **kwargs: Any) -> bool:
|
||||
return await asyncio.to_thread(write_metric, *args, **kwargs)
|
||||
|
||||
|
||||
async def awrite_metrics_batch(points: list[dict[str, Any]]) -> bool:
|
||||
return await asyncio.to_thread(write_metrics_batch, points)
|
||||
|
||||
|
||||
async def aquery_metrics(
|
||||
host_id: int,
|
||||
metric_type: str,
|
||||
start_time: datetime,
|
||||
end_time: Optional[datetime] = None,
|
||||
) -> list[dict[str, Any]]:
|
||||
return await asyncio.to_thread(query_metrics, host_id, metric_type, start_time, end_time)
|
||||
|
||||
|
||||
async def aquery_latest(host_id: int, metric_type: str) -> list[dict[str, Any]]:
|
||||
return await asyncio.to_thread(query_latest, host_id, metric_type)
|
||||
|
||||
|
||||
async def aquery_trend(
|
||||
host_id: int,
|
||||
metric_type: str,
|
||||
start_time: datetime,
|
||||
end_time: Optional[datetime] = None,
|
||||
) -> list[dict[str, Any]]:
|
||||
return await asyncio.to_thread(query_trend, host_id, metric_type, start_time, end_time)
|
||||
|
||||
|
||||
async def aquery_latest_grouped(
|
||||
host_id: int, only: Optional[set[str]] = None
|
||||
) -> dict[str, list[dict[str, Any]]]:
|
||||
return await asyncio.to_thread(query_latest_grouped, host_id, only)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -140,6 +200,7 @@ from(bucket: "{settings.INFLUXDB_BUCKET}")
|
||||
|> filter(fn: (r) => r["_measurement"] == "{metric_type}")
|
||||
|> filter(fn: (r) => r["host_id"] == "{host_id}")
|
||||
|> filter(fn: (r) => r["_field"] == "value")
|
||||
|> limit(n: 5000)
|
||||
"""
|
||||
results: list[dict[str, Any]] = []
|
||||
try:
|
||||
@@ -206,10 +267,20 @@ def query_trend(
|
||||
return query_metrics(host_id, metric_type, start_time, end_time)
|
||||
|
||||
|
||||
def query_latest_grouped(host_id: int) -> dict[str, list[dict[str, Any]]]:
|
||||
"""Return latest metrics grouped by metric_type for a host."""
|
||||
def query_latest_grouped(
|
||||
host_id: int, only: Optional[set[str]] = None
|
||||
) -> dict[str, list[dict[str, Any]]]:
|
||||
"""Return latest metrics grouped by metric_type for a host.
|
||||
|
||||
``only`` (O1) limits the query to the requested types instead of always
|
||||
fetching all four — e.g. ``only={"temperature"}`` for the temperature
|
||||
summary endpoint.
|
||||
"""
|
||||
types = ("temperature", "fan", "voltage", "power")
|
||||
if only is not None:
|
||||
types = tuple(t for t in types if t in only)
|
||||
grouped: dict[str, list[dict[str, Any]]] = {}
|
||||
for metric_type in ("temperature", "fan", "voltage", "power"):
|
||||
for metric_type in types:
|
||||
grouped[metric_type] = query_latest(host_id, metric_type)
|
||||
return grouped
|
||||
|
||||
|
||||
@@ -1,86 +0,0 @@
|
||||
#!/usr/bin/env python3
|
||||
"""
|
||||
初始化管理员账号脚本
|
||||
运行此脚本会在数据库中创建默认管理员账号
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# 添加项目根目录到Python路径
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent))
|
||||
|
||||
from app.database import engine
|
||||
from app.models import User, Base
|
||||
from sqlalchemy import select
|
||||
from passlib.context import CryptContext
|
||||
|
||||
|
||||
pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")
|
||||
|
||||
|
||||
async def init_admin_user():
|
||||
"""创建默认管理员账号"""
|
||||
|
||||
# 创建所有表(如果不存在)
|
||||
async with engine.begin() as conn:
|
||||
await conn.run_sync(Base.metadata.create_all)
|
||||
|
||||
# 检查是否已存在管理员账号(使用独立的连接)
|
||||
async with engine.connect() as conn:
|
||||
result = await conn.execute(select(User).where(User.username == "admin"))
|
||||
existing_user = result.scalar_one_or_none()
|
||||
|
||||
if existing_user:
|
||||
print("✓ 管理员账号已存在,跳过创建")
|
||||
print(f" 用户名: {existing_user.username}")
|
||||
print(f" 创建时间: {existing_user.created_at}")
|
||||
return
|
||||
|
||||
# 创建默认管理员账号
|
||||
default_password = "admin123"
|
||||
hashed_password = pwd_context.hash(default_password)
|
||||
|
||||
async with engine.begin() as conn:
|
||||
new_user = User(
|
||||
username="admin",
|
||||
hashed_password=hashed_password
|
||||
)
|
||||
conn.add(new_user)
|
||||
await conn.commit()
|
||||
|
||||
print("✓ 默认管理员账号创建成功")
|
||||
print(f" 用户名: admin")
|
||||
print(f" 密码: {default_password}")
|
||||
print("\n⚠️ 重要提醒:")
|
||||
print(" 1. 请立即登录系统修改默认密码")
|
||||
print(" 2. 在生产环境中,请更改 JWT_SECRET_KEY 配置")
|
||||
|
||||
|
||||
async def main():
|
||||
"""主函数"""
|
||||
try:
|
||||
print("=" * 50)
|
||||
print("IPMI监控系统 - 初始化管理员账号")
|
||||
print("=" * 50)
|
||||
print()
|
||||
|
||||
await init_admin_user()
|
||||
|
||||
print()
|
||||
print("=" * 50)
|
||||
print("初始化完成")
|
||||
print("=" * 50)
|
||||
|
||||
except Exception as e:
|
||||
print(f"\n✗ 错误: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
sys.exit(1)
|
||||
finally:
|
||||
await engine.dispose()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(main())
|
||||
+45
-12
@@ -7,9 +7,9 @@ from pathlib import Path
|
||||
|
||||
from fastapi import FastAPI, Request
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from fastapi.responses import FileResponse
|
||||
from fastapi.responses import FileResponse, JSONResponse
|
||||
|
||||
from app.config import settings
|
||||
from app.config import settings, validate_security_settings
|
||||
from app.database import init_db
|
||||
from app.routers import alerts, groups, hosts, monitor, auth
|
||||
from app.routers import settings as settings_router
|
||||
@@ -25,6 +25,9 @@ _data_dir.mkdir(parents=True, exist_ok=True)
|
||||
async def lifespan(app: FastAPI):
|
||||
"""Application lifespan: startup & shutdown hooks."""
|
||||
# ---- Startup ----
|
||||
# Fail fast on unsafe security configuration (missing / legacy / short
|
||||
# JWT secret). Runs here, not at import time.
|
||||
validate_security_settings()
|
||||
logging.basicConfig(
|
||||
level=getattr(logging, settings.LOG_LEVEL.upper(), logging.INFO),
|
||||
format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
|
||||
@@ -55,20 +58,44 @@ async def lifespan(app: FastAPI):
|
||||
# Create FastAPI app
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_is_production = settings.ENVIRONMENT.lower() == "production"
|
||||
|
||||
app = FastAPI(
|
||||
title="IPMI Monitor",
|
||||
version="0.1.0",
|
||||
lifespan=lifespan,
|
||||
# Never expose interactive API docs in production.
|
||||
docs_url=None if _is_production else "/docs",
|
||||
redoc_url=None if _is_production else "/redoc",
|
||||
openapi_url=None if _is_production else "/openapi.json",
|
||||
)
|
||||
|
||||
# CORS
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=["*"],
|
||||
allow_credentials=True,
|
||||
allow_methods=["*"],
|
||||
allow_headers=["*"],
|
||||
)
|
||||
|
||||
@app.middleware("http")
|
||||
async def security_headers(request: Request, call_next):
|
||||
"""Add baseline security response headers to every response."""
|
||||
response = await call_next(request)
|
||||
response.headers["X-Content-Type-Options"] = "nosniff"
|
||||
response.headers["X-Frame-Options"] = "DENY"
|
||||
response.headers["Referrer-Policy"] = "strict-origin-when-cross-origin"
|
||||
response.headers["Content-Security-Policy"] = (
|
||||
"default-src 'self'; script-src 'self'; style-src 'self' 'unsafe-inline'; "
|
||||
"img-src 'self' data:; font-src 'self'; connect-src 'self'; "
|
||||
"frame-ancestors 'none'; base-uri 'self'; form-action 'self'"
|
||||
)
|
||||
return response
|
||||
|
||||
# CORS: only enabled when an explicit origin allow-list is configured.
|
||||
# Unconfigured means same-origin deployment and no CORS middleware at all.
|
||||
_cors_origins = [o.strip() for o in settings.CORS_ORIGINS.split(",") if o.strip()]
|
||||
if _cors_origins:
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=_cors_origins,
|
||||
allow_credentials=True,
|
||||
allow_methods=["GET", "POST", "PUT", "DELETE", "PATCH", "OPTIONS"],
|
||||
allow_headers=["Authorization", "Content-Type"],
|
||||
)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# API routers
|
||||
@@ -114,10 +141,16 @@ if _frontend_dist and _frontend_dist.is_dir():
|
||||
|
||||
@app.get("/{full_path:path}")
|
||||
async def serve_spa(full_path: str):
|
||||
from starlette.responses import Response
|
||||
# Unknown /api/ paths are API 404s, never the SPA entry point.
|
||||
if full_path == "api" or full_path.startswith("api/"):
|
||||
return JSONResponse({"detail": "Not Found"}, status_code=404)
|
||||
# Try to serve a real static file (e.g. /assets/index-xxx.js, favicon.ico)
|
||||
if full_path:
|
||||
file_path = _frontend_dist / full_path
|
||||
dist_root = _frontend_dist.resolve()
|
||||
file_path = (_frontend_dist / full_path).resolve()
|
||||
# Path traversal guard: the resolved path must stay inside dist.
|
||||
if not file_path.is_relative_to(dist_root):
|
||||
return JSONResponse({"detail": "Not Found"}, status_code=404)
|
||||
if file_path.is_file():
|
||||
return FileResponse(str(file_path))
|
||||
# Otherwise return index.html for Vue Router to handle
|
||||
|
||||
@@ -1,11 +1,12 @@
|
||||
"""Alert rules and history REST API."""
|
||||
|
||||
import logging
|
||||
import math
|
||||
from datetime import datetime
|
||||
from typing import Optional
|
||||
from typing import Literal, Optional
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query
|
||||
from pydantic import BaseModel, Field
|
||||
from pydantic import BaseModel, Field, field_validator
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
@@ -17,39 +18,65 @@ from app.models import User as UserModel
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter(prefix="/api/alerts", tags=["alerts"])
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Schemas — Rules
|
||||
# ---------------------------------------------------------------------------
|
||||
# Shared value-domain validation (B18)
|
||||
MetricType = Literal["temperature", "fan", "voltage", "power", "component", "status"]
|
||||
Condition = Literal["gt", "lt", "eq", "gte", "lte"]
|
||||
|
||||
|
||||
def _validate_threshold(value: Optional[float]) -> Optional[float]:
|
||||
"""Reject NaN / infinite thresholds (B18)."""
|
||||
if value is not None and (isinstance(value, float) and (math.isnan(value) or math.isinf(value))):
|
||||
raise ValueError("threshold must be a finite number")
|
||||
return value
|
||||
|
||||
|
||||
class RuleCreate(BaseModel):
|
||||
name: str = Field(..., max_length=128)
|
||||
host_id: Optional[int] = None
|
||||
group_id: Optional[int] = None
|
||||
metric_type: str = Field(..., max_length=64) # temperature / voltage / fan / power
|
||||
metric_type: MetricType
|
||||
metric_name: str = Field(..., max_length=128)
|
||||
condition: str = Field(..., max_length=16) # gt / lt / eq / gte / lte
|
||||
condition: Condition
|
||||
threshold: float
|
||||
duration: int = Field(default=0, description="Seconds the condition must hold before alerting")
|
||||
duration: int = Field(default=0, ge=0, description="Seconds the condition must hold before alerting")
|
||||
enabled: bool = True
|
||||
notify_email: bool = False
|
||||
notify_feishu: bool = False
|
||||
email_addresses: str = Field(default="", description="Comma-separated emails")
|
||||
|
||||
@field_validator("threshold")
|
||||
@classmethod
|
||||
def _check_threshold(cls, value: float) -> float:
|
||||
return _validate_threshold(value) # type: ignore[return-value]
|
||||
|
||||
|
||||
class RuleUpdate(BaseModel):
|
||||
name: Optional[str] = Field(None, max_length=128)
|
||||
host_id: Optional[int] = None
|
||||
group_id: Optional[int] = None
|
||||
metric_type: Optional[str] = Field(None, max_length=64)
|
||||
metric_type: Optional[MetricType] = None
|
||||
metric_name: Optional[str] = Field(None, max_length=128)
|
||||
condition: Optional[str] = Field(None, max_length=16)
|
||||
condition: Optional[Condition] = None
|
||||
threshold: Optional[float] = None
|
||||
duration: Optional[int] = None
|
||||
duration: Optional[int] = Field(None, ge=0)
|
||||
enabled: Optional[bool] = None
|
||||
notify_email: Optional[bool] = None
|
||||
notify_feishu: Optional[bool] = None
|
||||
email_addresses: Optional[str] = None
|
||||
|
||||
@field_validator("threshold")
|
||||
@classmethod
|
||||
def _check_threshold(cls, value: Optional[float]) -> Optional[float]:
|
||||
return _validate_threshold(value)
|
||||
|
||||
|
||||
def _check_host_group_exclusive(
|
||||
host_id: Optional[int], group_id: Optional[int]
|
||||
) -> None:
|
||||
"""A rule binds to at most one of host / group (B17)."""
|
||||
if host_id is not None and group_id is not None:
|
||||
raise HTTPException(400, "host_id and group_id are mutually exclusive")
|
||||
|
||||
|
||||
class RuleOut(BaseModel):
|
||||
id: int
|
||||
@@ -120,6 +147,7 @@ async def create_rule(
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: UserModel = Depends(get_current_user)
|
||||
):
|
||||
_check_host_group_exclusive(body.host_id, body.group_id)
|
||||
if body.host_id is not None:
|
||||
host = await db.get(Host, body.host_id)
|
||||
if not host:
|
||||
@@ -133,6 +161,7 @@ async def create_rule(
|
||||
db.add(rule)
|
||||
await db.flush()
|
||||
await db.refresh(rule)
|
||||
await db.commit() # B23: explicit commit, do not rely on get_db auto-commit
|
||||
return RuleOut.model_validate(rule)
|
||||
|
||||
|
||||
@@ -148,11 +177,25 @@ async def update_rule(
|
||||
raise HTTPException(404, "Rule not found")
|
||||
|
||||
update_data = body.model_dump(exclude_unset=True)
|
||||
# B17: the merged host_id/group_id pair must still be exclusive.
|
||||
new_host_id = update_data.get("host_id", rule.host_id)
|
||||
new_group_id = update_data.get("group_id", rule.group_id)
|
||||
_check_host_group_exclusive(new_host_id, new_group_id)
|
||||
if update_data.get("host_id") is not None:
|
||||
host = await db.get(Host, update_data["host_id"])
|
||||
if not host:
|
||||
raise HTTPException(404, "Host not found")
|
||||
if update_data.get("group_id") is not None:
|
||||
group = await db.get(HostGroup, update_data["group_id"])
|
||||
if not group:
|
||||
raise HTTPException(404, "Group not found")
|
||||
|
||||
for field, value in update_data.items():
|
||||
setattr(rule, field, value)
|
||||
|
||||
await db.flush()
|
||||
await db.refresh(rule)
|
||||
await db.commit() # B23
|
||||
return RuleOut.model_validate(rule)
|
||||
|
||||
|
||||
@@ -166,6 +209,7 @@ async def delete_rule(
|
||||
if not rule:
|
||||
raise HTTPException(404, "Rule not found")
|
||||
await db.delete(rule)
|
||||
await db.commit() # B23
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -232,6 +276,7 @@ async def acknowledge_alert(
|
||||
record.acknowledged = True
|
||||
await db.flush()
|
||||
await db.refresh(record)
|
||||
await db.commit() # B23
|
||||
|
||||
d = HistoryOut.model_validate(record)
|
||||
if record.host:
|
||||
|
||||
+177
-25
@@ -1,13 +1,15 @@
|
||||
"""Authentication endpoints: login, verify, logout."""
|
||||
"""Authentication endpoints: login, verify, logout, change-password."""
|
||||
|
||||
from datetime import datetime, timedelta
|
||||
import logging
|
||||
import uuid
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from typing import Optional
|
||||
|
||||
from fastapi import APIRouter, Depends, HTTPException, status
|
||||
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
|
||||
from jose import JWTError, jwt
|
||||
import bcrypt
|
||||
from pydantic import BaseModel
|
||||
import jwt
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request, status
|
||||
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
|
||||
from pydantic import BaseModel, Field
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
@@ -15,6 +17,59 @@ from app.config import settings
|
||||
from app.database import get_db
|
||||
from app.models import User
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Token revocation (process-local)
|
||||
# ---------------------------------------------------------------------------
|
||||
# JTIs revoked by explicit logout. Password changes revoke every token
|
||||
# issued before the change via the per-user timestamp below.
|
||||
_REVOKED_JTIS: set[str] = set()
|
||||
_PASSWORD_CHANGED_AT: dict[int, datetime] = {}
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Login rate limiting (process-local)
|
||||
# ---------------------------------------------------------------------------
|
||||
# (client IP, username) -> [consecutive failures, locked-until timestamp]
|
||||
_LOGIN_FAILURES: dict[tuple[str, str], list] = {}
|
||||
MAX_LOGIN_FAILURES = 5
|
||||
LOGIN_LOCKOUT = timedelta(minutes=15)
|
||||
|
||||
# Pre-computed dummy hash so a login for a non-existent user costs the same
|
||||
# bcrypt verification as a real one (no username-enumeration timing signal).
|
||||
_DUMMY_HASH = bcrypt.hashpw(b"__ipmi_monitor_dummy__", bcrypt.gensalt()).decode("utf-8")
|
||||
|
||||
|
||||
def _client_ip(request: Optional[Request]) -> str:
|
||||
if request is not None and request.client is not None:
|
||||
return request.client.host
|
||||
return "unknown"
|
||||
|
||||
|
||||
def _check_login_lockout(ip: str, username: str) -> None:
|
||||
entry = _LOGIN_FAILURES.get((ip, username))
|
||||
if entry and entry[1] is not None and datetime.now(timezone.utc) < entry[1]:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_429_TOO_MANY_REQUESTS,
|
||||
detail="Too many failed login attempts, please try again later",
|
||||
headers={"Retry-After": str(int(LOGIN_LOCKOUT.total_seconds()))},
|
||||
)
|
||||
|
||||
|
||||
def _record_login_failure(ip: str, username: str) -> None:
|
||||
entry = _LOGIN_FAILURES.setdefault((ip, username), [0, None])
|
||||
entry[0] += 1
|
||||
if entry[0] >= MAX_LOGIN_FAILURES:
|
||||
entry[1] = datetime.now(timezone.utc) + LOGIN_LOCKOUT
|
||||
logger.warning(
|
||||
"Login locked for user %r from %s after %d failures",
|
||||
username, ip, entry[0],
|
||||
)
|
||||
|
||||
|
||||
def _clear_login_failures(ip: str, username: str) -> None:
|
||||
_LOGIN_FAILURES.pop((ip, username), None)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Password hashing
|
||||
@@ -35,8 +90,13 @@ def verify_password(plain_password: str, hashed_password: str) -> bool:
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class LoginRequest(BaseModel):
|
||||
username: str
|
||||
password: str
|
||||
username: str = Field(..., min_length=1, max_length=64)
|
||||
password: str = Field(..., min_length=1, max_length=256)
|
||||
|
||||
|
||||
class ChangePasswordRequest(BaseModel):
|
||||
old_password: str = Field(..., min_length=1, max_length=256)
|
||||
new_password: str = Field(..., min_length=12, max_length=256)
|
||||
|
||||
|
||||
class TokenResponse(BaseModel):
|
||||
@@ -56,33 +116,43 @@ class UserResponse(BaseModel):
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# JWT utilities
|
||||
# JWT utilities (PyJWT, explicit HS256)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def create_access_token(data: dict) -> str:
|
||||
"""Create a JWT access token."""
|
||||
"""Create a JWT access token (8h default, UTC, with jti + iat)."""
|
||||
now = datetime.now(timezone.utc)
|
||||
to_encode = data.copy()
|
||||
expire = datetime.now() + timedelta(hours=settings.JWT_EXPIRATION_HOURS)
|
||||
to_encode.update({"exp": expire})
|
||||
encoded_jwt = jwt.encode(
|
||||
to_encode.update({
|
||||
"exp": now + timedelta(hours=settings.JWT_EXPIRATION_HOURS),
|
||||
"iat": now,
|
||||
"jti": uuid.uuid4().hex,
|
||||
})
|
||||
return jwt.encode(
|
||||
to_encode,
|
||||
settings.JWT_SECRET_KEY,
|
||||
algorithm=settings.JWT_ALGORITHM
|
||||
algorithm=settings.JWT_ALGORITHM,
|
||||
)
|
||||
return encoded_jwt
|
||||
|
||||
|
||||
def verify_token(token: str) -> Optional[dict]:
|
||||
"""Verify and decode a JWT token."""
|
||||
"""Verify and decode a JWT token. Returns None when invalid/expired/revoked."""
|
||||
try:
|
||||
payload = jwt.decode(
|
||||
token,
|
||||
settings.JWT_SECRET_KEY,
|
||||
algorithms=[settings.JWT_ALGORITHM]
|
||||
algorithms=["HS256"],
|
||||
)
|
||||
return payload
|
||||
except JWTError:
|
||||
except jwt.InvalidTokenError:
|
||||
return None
|
||||
if payload.get("jti") in _REVOKED_JTIS:
|
||||
return None
|
||||
return payload
|
||||
|
||||
|
||||
def revoke_token_jti(jti: Optional[str]) -> None:
|
||||
if jti:
|
||||
_REVOKED_JTIS.add(jti)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -125,6 +195,22 @@ async def get_current_user(
|
||||
headers={"WWW-Authenticate": "Bearer"},
|
||||
)
|
||||
|
||||
# Tokens issued before the user's last password change are revoked.
|
||||
changed_at = _PASSWORD_CHANGED_AT.get(user.id)
|
||||
if changed_at is not None:
|
||||
iat = payload.get("iat")
|
||||
issued_at = (
|
||||
datetime.fromtimestamp(iat, tz=timezone.utc)
|
||||
if isinstance(iat, (int, float))
|
||||
else None
|
||||
)
|
||||
if issued_at is not None and issued_at < changed_at:
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||
detail="Token revoked (password changed)",
|
||||
headers={"WWW-Authenticate": "Bearer"},
|
||||
)
|
||||
|
||||
return user
|
||||
|
||||
|
||||
@@ -137,7 +223,8 @@ router = APIRouter(prefix="/api/auth", tags=["Authentication"])
|
||||
|
||||
@router.post("/login", response_model=TokenResponse)
|
||||
async def login(
|
||||
request: LoginRequest,
|
||||
body: LoginRequest,
|
||||
request: Request,
|
||||
db: AsyncSession = Depends(get_db)
|
||||
):
|
||||
"""
|
||||
@@ -145,22 +232,36 @@ async def login(
|
||||
|
||||
Returns a JWT access token if credentials are valid.
|
||||
"""
|
||||
ip = _client_ip(request)
|
||||
_check_login_lockout(ip, body.username)
|
||||
|
||||
# Find user by username
|
||||
result = await db.execute(
|
||||
select(User).where(User.username == request.username)
|
||||
select(User).where(User.username == body.username)
|
||||
)
|
||||
user = result.scalar_one_or_none()
|
||||
|
||||
# Verify user exists and password is correct
|
||||
if user is None or not verify_password(request.password, user.hashed_password):
|
||||
# Constant-time behaviour: unknown users still pay one bcrypt check so
|
||||
# the response time does not reveal whether the username exists.
|
||||
password_ok = (
|
||||
verify_password(body.password, user.hashed_password)
|
||||
if user is not None
|
||||
else verify_password(body.password, _DUMMY_HASH)
|
||||
)
|
||||
|
||||
if user is None or not password_ok:
|
||||
_record_login_failure(ip, body.username)
|
||||
logger.warning("Failed login attempt for user %r from %s", body.username, ip)
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||
detail="Invalid username or password",
|
||||
headers={"WWW-Authenticate": "Bearer"},
|
||||
)
|
||||
|
||||
# Update last login time
|
||||
user.last_login = datetime.now()
|
||||
_clear_login_failures(ip, body.username)
|
||||
|
||||
# Update last login time (stored naive UTC for the SQLite DateTime column)
|
||||
user.last_login = datetime.now(timezone.utc).replace(tzinfo=None)
|
||||
await db.commit()
|
||||
|
||||
# Create access token
|
||||
@@ -191,6 +292,57 @@ async def verify_token_endpoint(
|
||||
return VerifyResponse(valid=True, username=username)
|
||||
|
||||
|
||||
@router.post("/logout")
|
||||
async def logout(
|
||||
credentials: HTTPAuthorizationCredentials = Depends(security),
|
||||
current_user: User = Depends(get_current_user),
|
||||
):
|
||||
"""Revoke the token used for this request."""
|
||||
payload = verify_token(credentials.credentials)
|
||||
revoke_token_jti(payload.get("jti") if payload else None)
|
||||
logger.info("User %r logged out", current_user.username)
|
||||
return {"success": True, "message": "Logged out"}
|
||||
|
||||
|
||||
@router.post("/change-password")
|
||||
async def change_password(
|
||||
body: ChangePasswordRequest,
|
||||
credentials: HTTPAuthorizationCredentials = Depends(security),
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
):
|
||||
"""Change the current user's password.
|
||||
|
||||
The new password must be at least 12 characters. On success every token
|
||||
issued before this change is revoked (including the current one).
|
||||
"""
|
||||
# NOTE: get_current_user loads the user in its own session, so re-fetch
|
||||
# it in this endpoint's session before mutating.
|
||||
user = await db.get(User, current_user.id)
|
||||
if user is None:
|
||||
raise HTTPException(status_code=status.HTTP_401_UNAUTHORIZED, detail="User not found")
|
||||
|
||||
if not verify_password(body.old_password, user.hashed_password):
|
||||
logger.warning(
|
||||
"Failed password change attempt for user %r", current_user.username
|
||||
)
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_401_UNAUTHORIZED,
|
||||
detail="Current password is incorrect",
|
||||
)
|
||||
|
||||
user.hashed_password = hash_password(body.new_password)
|
||||
await db.commit()
|
||||
|
||||
# Revoke all previously issued tokens for this user.
|
||||
_PASSWORD_CHANGED_AT[current_user.id] = datetime.now(timezone.utc)
|
||||
payload = verify_token(credentials.credentials)
|
||||
revoke_token_jti(payload.get("jti") if payload else None)
|
||||
|
||||
logger.info("User %r changed password; all prior tokens revoked", current_user.username)
|
||||
return {"success": True, "message": "Password changed; please log in again"}
|
||||
|
||||
|
||||
@router.get("/me", response_model=UserResponse)
|
||||
async def get_me(current_user: User = Depends(get_current_user)):
|
||||
"""
|
||||
|
||||
@@ -80,6 +80,7 @@ async def create_group(
|
||||
db.add(group)
|
||||
await db.flush()
|
||||
await db.refresh(group)
|
||||
await db.commit() # B23: explicit commit, do not rely on get_db auto-commit
|
||||
return GroupOut.model_validate(group)
|
||||
|
||||
|
||||
@@ -100,6 +101,7 @@ async def update_group(
|
||||
|
||||
await db.flush()
|
||||
await db.refresh(group)
|
||||
await db.commit() # B23
|
||||
return GroupOut.model_validate(group)
|
||||
|
||||
|
||||
@@ -120,3 +122,4 @@ async def delete_group(
|
||||
raise HTTPException(400, "Group still has hosts – remove or reassign them first")
|
||||
|
||||
await db.delete(group)
|
||||
await db.commit() # B23
|
||||
|
||||
@@ -10,11 +10,13 @@ from pydantic import BaseModel, Field
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.crypto import decrypt_value, encrypt_value
|
||||
from app.database import get_db
|
||||
from app.models import Host, HostGroup
|
||||
from app import influxdb as influxdb_module
|
||||
from app.routers.auth import get_current_user
|
||||
from app.models import User as UserModel
|
||||
from app.ssrf import validate_ipmi_host
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter(prefix="/api/hosts", tags=["hosts"])
|
||||
@@ -49,7 +51,8 @@ class HostOut(BaseModel):
|
||||
ipmi_host: str
|
||||
ipmi_port: int
|
||||
ipmi_username: str
|
||||
ipmi_password: str
|
||||
# The BMC password is never returned via the API; only whether one is set.
|
||||
has_password: bool = False
|
||||
group_id: Optional[int]
|
||||
group_name: Optional[str] = None
|
||||
enabled: bool
|
||||
@@ -59,6 +62,15 @@ class HostOut(BaseModel):
|
||||
model_config = {"from_attributes": True}
|
||||
|
||||
|
||||
def _to_host_out(host: Host) -> HostOut:
|
||||
"""Build the API view of a host (password replaced by a boolean flag)."""
|
||||
out = HostOut.model_validate(host)
|
||||
out.has_password = bool(host.ipmi_password)
|
||||
if host.group:
|
||||
out.group_name = host.group.name
|
||||
return out
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# CRUD
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -66,22 +78,22 @@ class HostOut(BaseModel):
|
||||
@router.get("", response_model=list[HostOut])
|
||||
async def list_hosts(
|
||||
group_id: Optional[int] = Query(None),
|
||||
limit: Optional[int] = Query(None, ge=1, description="Max hosts to return"),
|
||||
offset: int = Query(0, ge=0, description="Hosts to skip"),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: UserModel = Depends(get_current_user),
|
||||
):
|
||||
stmt = select(Host).order_by(Host.id)
|
||||
if group_id is not None:
|
||||
stmt = stmt.where(Host.group_id == group_id)
|
||||
if limit is not None:
|
||||
stmt = stmt.limit(limit)
|
||||
if offset:
|
||||
stmt = stmt.offset(offset)
|
||||
result = await db.execute(stmt)
|
||||
hosts = result.scalars().all()
|
||||
|
||||
out: list[HostOut] = []
|
||||
for h in hosts:
|
||||
d = HostOut.model_validate(h)
|
||||
if h.group:
|
||||
d.group_name = h.group.name
|
||||
out.append(d)
|
||||
return out
|
||||
return [_to_host_out(h) for h in hosts]
|
||||
|
||||
|
||||
@router.post("", response_model=HostOut, status_code=201)
|
||||
@@ -90,17 +102,26 @@ async def create_host(
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: UserModel = Depends(get_current_user)
|
||||
):
|
||||
try:
|
||||
validate_ipmi_host(body.ipmi_host)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(400, str(exc))
|
||||
|
||||
# Validate group_id if provided
|
||||
if body.group_id is not None:
|
||||
group = await db.get(HostGroup, body.group_id)
|
||||
if not group:
|
||||
raise HTTPException(404, "Group not found")
|
||||
|
||||
host = Host(**body.model_dump())
|
||||
data = body.model_dump()
|
||||
# Encrypt the BMC password before it reaches the database.
|
||||
data["ipmi_password"] = encrypt_value(body.ipmi_password)
|
||||
host = Host(**data)
|
||||
db.add(host)
|
||||
await db.flush()
|
||||
await db.refresh(host)
|
||||
return HostOut.model_validate(host)
|
||||
await db.commit()
|
||||
return _to_host_out(host)
|
||||
|
||||
|
||||
@router.put("/{host_id}", response_model=HostOut)
|
||||
@@ -115,15 +136,23 @@ async def update_host(
|
||||
raise HTTPException(404, "Host not found")
|
||||
|
||||
update_data = body.model_dump(exclude_unset=True)
|
||||
if "ipmi_host" in update_data and update_data["ipmi_host"]:
|
||||
try:
|
||||
validate_ipmi_host(update_data["ipmi_host"])
|
||||
except ValueError as exc:
|
||||
raise HTTPException(400, str(exc))
|
||||
for field, value in update_data.items():
|
||||
# 如果密码字段为空字符串或None,保留原密码不更新
|
||||
if field == "ipmi_password" and (value is None or value == ""):
|
||||
continue
|
||||
if field == "ipmi_password":
|
||||
value = encrypt_value(value)
|
||||
setattr(host, field, value)
|
||||
|
||||
await db.flush()
|
||||
await db.refresh(host)
|
||||
return HostOut.model_validate(host)
|
||||
await db.commit()
|
||||
return _to_host_out(host)
|
||||
|
||||
|
||||
@router.delete("/{host_id}", status_code=204)
|
||||
@@ -136,6 +165,7 @@ async def delete_host(
|
||||
if not host:
|
||||
raise HTTPException(404, "Host not found")
|
||||
await db.delete(host)
|
||||
await db.commit()
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -153,8 +183,17 @@ async def test_ipmi_connection(
|
||||
if not host:
|
||||
raise HTTPException(404, "Host not found")
|
||||
|
||||
try:
|
||||
validate_ipmi_host(host.ipmi_host)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(400, str(exc))
|
||||
|
||||
from app.services.ipmi_collector import _run_ipmitool_with_auth, parse_sdr_output
|
||||
|
||||
# The collector needs the cleartext password; decrypt it here (internal
|
||||
# use only — it is never sent back through the API).
|
||||
ipmi_password = decrypt_value(host.ipmi_password)
|
||||
|
||||
try:
|
||||
# Try different connection methods like collect_host does
|
||||
raw = None
|
||||
@@ -166,12 +205,12 @@ async def test_ipmi_connection(
|
||||
["lan", ["-A", "MD5"]],
|
||||
["lan", None],
|
||||
]
|
||||
|
||||
|
||||
for interface, extra_args in attempts:
|
||||
try:
|
||||
raw = await _run_ipmitool_with_auth(
|
||||
host.ipmi_host, host.ipmi_port,
|
||||
host.ipmi_username, host.ipmi_password,
|
||||
host.ipmi_username, ipmi_password,
|
||||
interface=interface,
|
||||
extra_args=extra_args,
|
||||
)
|
||||
@@ -179,10 +218,10 @@ async def test_ipmi_connection(
|
||||
break
|
||||
except RuntimeError:
|
||||
continue
|
||||
|
||||
|
||||
if not raw:
|
||||
raise RuntimeError("All connection attempts failed")
|
||||
|
||||
|
||||
parsed = parse_sdr_output(raw)
|
||||
return {
|
||||
"success": True,
|
||||
|
||||
+200
-123
@@ -1,12 +1,14 @@
|
||||
"""Monitoring data REST API — dashboard, history, realtime."""
|
||||
|
||||
import csv
|
||||
import io
|
||||
import logging
|
||||
import re
|
||||
from datetime import datetime, timedelta, timezone
|
||||
from pathlib import Path as PathLib
|
||||
from typing import Optional, List
|
||||
|
||||
from fastapi import APIRouter, Depends, Query, Path
|
||||
from fastapi import APIRouter, Depends, HTTPException, Query, Path
|
||||
from fastapi.responses import StreamingResponse
|
||||
from sqlalchemy import func, select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
@@ -16,10 +18,58 @@ from app.models import AlertHistory, Host, Host as HostModel
|
||||
from app.routers.auth import get_current_user
|
||||
from app.models import User as UserModel
|
||||
from app.services.component_parser import parse_component_presence, group_components_by_status
|
||||
from app.utils import parse_dt, resolve_total_power, utc_now_naive
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter(prefix="/api/monitor", tags=["monitor"])
|
||||
|
||||
# Metric types accepted by the history/summary/trend endpoints (B12).
|
||||
METRIC_TYPES = {"temperature", "fan", "voltage", "power", "component", "status"}
|
||||
MAX_HOST_IDS = 50
|
||||
|
||||
|
||||
def _parse_host_ids(host_ids: str) -> list[int]:
|
||||
"""Parse a comma-separated host-id list (B12).
|
||||
|
||||
Raises 400 for non-integer ids or more than 50 ids.
|
||||
"""
|
||||
ids: list[int] = []
|
||||
for raw in host_ids.split(","):
|
||||
raw = raw.strip()
|
||||
if not raw:
|
||||
continue
|
||||
try:
|
||||
ids.append(int(raw))
|
||||
except ValueError:
|
||||
raise HTTPException(400, f"Invalid host id: {raw!r}")
|
||||
if len(ids) > MAX_HOST_IDS:
|
||||
raise HTTPException(400, f"Too many host ids (max {MAX_HOST_IDS})")
|
||||
return ids
|
||||
|
||||
|
||||
def _check_metric_type(metric_type: str) -> str:
|
||||
"""Reject metric types outside the whitelist (B12)."""
|
||||
if metric_type not in METRIC_TYPES:
|
||||
raise HTTPException(400, f"Invalid metric_type: {metric_type!r}")
|
||||
return metric_type
|
||||
|
||||
|
||||
def _parse_time_range(
|
||||
start: Optional[str], end: Optional[str]
|
||||
) -> tuple[datetime, Optional[datetime]]:
|
||||
"""Parse start/end query params (B11).
|
||||
|
||||
Bad values are a 400, never a 500. Defaults use UTC now (B11).
|
||||
"""
|
||||
try:
|
||||
start_dt = parse_dt(start)
|
||||
end_dt = parse_dt(end)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(400, f"Invalid datetime: {exc}")
|
||||
if start_dt is None:
|
||||
start_dt = utc_now_naive() - timedelta(hours=1)
|
||||
return start_dt, end_dt
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Dashboard
|
||||
@@ -80,36 +130,18 @@ async def multi_host_history(
|
||||
current_user: UserModel = Depends(get_current_user),
|
||||
):
|
||||
"""Query historical metrics for multiple hosts."""
|
||||
host_id_list = [int(x.strip()) for x in host_ids.split(',') if x.strip()]
|
||||
|
||||
# Parse datetime with timezone handling
|
||||
def parse_dt(dt_str: Optional[str]) -> Optional[datetime]:
|
||||
if not dt_str:
|
||||
return None
|
||||
# Handle both 'Z' suffix and '+00:00' format
|
||||
dt_str = dt_str.replace('Z', '+00:00')
|
||||
try:
|
||||
dt = datetime.fromisoformat(dt_str)
|
||||
# If timezone-aware, convert to UTC naive datetime for InfluxDB queries
|
||||
if dt.tzinfo is not None:
|
||||
return dt.astimezone(timezone.utc).replace(tzinfo=None)
|
||||
return dt
|
||||
except ValueError:
|
||||
# Fallback: strip timezone info
|
||||
clean_str = dt_str.split('+')[0].split('-')[0][:19] if '+' in dt_str or dt_str.count('-') > 2 else dt_str[:19]
|
||||
return datetime.fromisoformat(clean_str)
|
||||
|
||||
start_dt = parse_dt(start) or (datetime.now() - timedelta(hours=1))
|
||||
end_dt = parse_dt(end)
|
||||
|
||||
host_id_list = _parse_host_ids(host_ids)
|
||||
_check_metric_type(metric_type)
|
||||
start_dt, end_dt = _parse_time_range(start, end)
|
||||
|
||||
all_data = {}
|
||||
for hid in host_id_list:
|
||||
data = influxdb_module.query_metrics(hid, metric_type, start_dt, end_dt)
|
||||
data = await influxdb_module.aquery_metrics(hid, metric_type, start_dt, end_dt)
|
||||
# Filter by metric_name if specified
|
||||
if metric_name:
|
||||
data = [d for d in data if d.get('metric_name') == metric_name]
|
||||
all_data[str(hid)] = data
|
||||
|
||||
|
||||
return {
|
||||
"metric_type": metric_type,
|
||||
"metric_name": metric_name,
|
||||
@@ -119,6 +151,53 @@ async def multi_host_history(
|
||||
}
|
||||
|
||||
|
||||
@router.get("/export")
|
||||
async def export_metrics(
|
||||
host_ids: str = Query(..., description="Comma-separated host IDs, e.g. '1,2,3'"),
|
||||
metric_type: str = Query("temperature", description="temperature/fan/voltage/power/component/status"),
|
||||
start: Optional[str] = Query(None, description="ISO datetime, default 1h ago"),
|
||||
end: Optional[str] = Query(None, description="ISO datetime, default now"),
|
||||
current_user: UserModel = Depends(get_current_user),
|
||||
):
|
||||
"""Export historical metrics for multiple hosts as a CSV file.
|
||||
|
||||
Columns: time,host_id,metric_type,metric_name,value,unit.
|
||||
Reuses the same host-id / metric-type / time-range validation as the
|
||||
history endpoints (400 on bad input) and ``influxdb.query_metrics``
|
||||
(via its async wrapper) for the data.
|
||||
"""
|
||||
host_id_list = _parse_host_ids(host_ids)
|
||||
if not host_id_list:
|
||||
raise HTTPException(400, "At least one host id is required")
|
||||
_check_metric_type(metric_type)
|
||||
start_dt, end_dt = _parse_time_range(start, end)
|
||||
|
||||
unit = _get_unit(metric_type)
|
||||
buf = io.StringIO()
|
||||
writer = csv.writer(buf)
|
||||
writer.writerow(["time", "host_id", "metric_type", "metric_name", "value", "unit"])
|
||||
for hid in host_id_list:
|
||||
data = await influxdb_module.aquery_metrics(hid, metric_type, start_dt, end_dt)
|
||||
for d in data:
|
||||
writer.writerow([
|
||||
d.get("time", ""),
|
||||
hid,
|
||||
metric_type,
|
||||
d.get("metric_name", ""),
|
||||
d.get("value", ""),
|
||||
unit,
|
||||
])
|
||||
|
||||
filename = (
|
||||
f"ipmi_metrics_{metric_type}_{start_dt.strftime('%Y%m%d%H%M%S')}.csv"
|
||||
)
|
||||
return StreamingResponse(
|
||||
iter([buf.getvalue()]),
|
||||
media_type="text/csv",
|
||||
headers={"Content-Disposition": f'attachment; filename="{filename}"'},
|
||||
)
|
||||
|
||||
|
||||
@router.get("/summary/{metric_type}")
|
||||
async def metrics_summary(
|
||||
metric_type: str = Path(..., description="temperature/fan/voltage/power"),
|
||||
@@ -127,28 +206,30 @@ async def metrics_summary(
|
||||
current_user: UserModel = Depends(get_current_user),
|
||||
):
|
||||
"""Get all sensors for specified hosts (not aggregated).
|
||||
|
||||
|
||||
Returns ALL sensors per host, not aggregated values.
|
||||
For dashboard totals, use /dashboard/totals endpoint instead.
|
||||
"""
|
||||
_check_metric_type(metric_type)
|
||||
|
||||
# Parse host IDs
|
||||
host_id_list: Optional[List[int]] = None
|
||||
if host_ids:
|
||||
host_id_list = [int(x.strip()) for x in host_ids.split(',') if x.strip()]
|
||||
|
||||
host_id_list = _parse_host_ids(host_ids)
|
||||
|
||||
# Get enabled hosts
|
||||
stmt = select(HostModel).where(HostModel.enabled.is_(True))
|
||||
if host_id_list:
|
||||
stmt = stmt.where(HostModel.id.in_(host_id_list))
|
||||
result = await db.execute(stmt)
|
||||
hosts = result.scalars().all()
|
||||
|
||||
# Return ALL sensors, not aggregated
|
||||
|
||||
# Return ALL sensors, not aggregated (only the requested type, O1)
|
||||
all_sensors = []
|
||||
for host in hosts:
|
||||
grouped = influxdb_module.query_latest_grouped(host.id)
|
||||
grouped = await influxdb_module.aquery_latest_grouped(host.id, only={metric_type})
|
||||
metrics = grouped.get(metric_type, [])
|
||||
|
||||
|
||||
for m in metrics:
|
||||
all_sensors.append({
|
||||
"host_id": host.id,
|
||||
@@ -159,7 +240,7 @@ async def metrics_summary(
|
||||
"time": m.get('time'),
|
||||
"timestamp": m.get('timestamp'),
|
||||
})
|
||||
|
||||
|
||||
return {"metric_type": metric_type, "sensors": all_sensors}
|
||||
|
||||
|
||||
@@ -169,8 +250,8 @@ async def host_sensors(
|
||||
current_user: UserModel = Depends(get_current_user)
|
||||
):
|
||||
"""Get all sensors for a specific host with latest values."""
|
||||
grouped = influxdb_module.query_latest_grouped(host_id)
|
||||
|
||||
grouped = await influxdb_module.aquery_latest_grouped(host_id)
|
||||
|
||||
sensors = []
|
||||
for metric_type, metrics in grouped.items():
|
||||
for m in metrics:
|
||||
@@ -181,7 +262,7 @@ async def host_sensors(
|
||||
"unit": _get_unit(metric_type),
|
||||
"time": m.get('time'),
|
||||
})
|
||||
|
||||
|
||||
return {"host_id": host_id, "sensors": sensors}
|
||||
|
||||
|
||||
@@ -209,42 +290,20 @@ async def dashboard_totals(
|
||||
stmt = stmt.where(HostModel.group_id == group_id)
|
||||
result = await db.execute(stmt)
|
||||
hosts = result.scalars().all()
|
||||
|
||||
|
||||
# Get groups for reference
|
||||
from app.routers.groups import get_all_groups
|
||||
groups = await get_all_groups(db)
|
||||
|
||||
|
||||
host_totals = []
|
||||
for host in hosts:
|
||||
grouped = influxdb_module.query_latest_grouped(host.id)
|
||||
grouped = await influxdb_module.aquery_latest_grouped(host.id)
|
||||
|
||||
# Get total power sensor value (not sum of all power sensors)
|
||||
# Priority:
|
||||
# 1. Native system-wide reading: Total_Power -> Power -> Pwr Consumption
|
||||
# 2. Fallback: sum of all PSU{N}_PIN (PSU input power) when no native
|
||||
# reading exists (covers hosts like Gooxi12 where BMC does not
|
||||
# report a system Pwr Consumption sensor).
|
||||
power_metrics = grouped.get("power", [])
|
||||
total_power = 0
|
||||
for sensor_name in ['Total_Power', 'Power', 'Pwr Consumption']:
|
||||
for m in power_metrics:
|
||||
if m.get('metric_name') == sensor_name:
|
||||
total_power = m.get('value', 0)
|
||||
break
|
||||
if total_power > 0:
|
||||
break
|
||||
if total_power <= 0:
|
||||
_psu_pin_re = re.compile(r'^PSU\d+_PIN$', re.IGNORECASE)
|
||||
psu_pin_sum = 0.0
|
||||
for m in power_metrics:
|
||||
name = m.get('metric_name') or ''
|
||||
if _psu_pin_re.match(name):
|
||||
try:
|
||||
psu_pin_sum += float(m.get('value', 0) or 0)
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
if psu_pin_sum > 0:
|
||||
total_power = psu_pin_sum
|
||||
# Total power via the shared resolver (O5): native system-wide
|
||||
# reading first, otherwise the PSU{N}_PIN sum. A 0 reading from an
|
||||
# existing sensor is valid and must not trigger the fallback.
|
||||
total_power = resolve_total_power(grouped.get("power", []))
|
||||
total_power = float(total_power) if total_power is not None else 0.0
|
||||
|
||||
# Get outlet temperature (exhaust air temperature)
|
||||
temp_metrics = grouped.get("temperature", [])
|
||||
@@ -258,9 +317,24 @@ async def dashboard_totals(
|
||||
break
|
||||
if outlet_temp is not None:
|
||||
break
|
||||
# Fallback to max temperature if no outlet sensor found
|
||||
max_temp = outlet_temp if outlet_temp is not None else max((m.get('value', 0) for m in temp_metrics if m.get('value') is not None), default=0)
|
||||
|
||||
# Fallback to max temperature if no outlet sensor found (None-safe, B13)
|
||||
if outlet_temp is not None:
|
||||
try:
|
||||
max_temp = float(outlet_temp)
|
||||
except (TypeError, ValueError):
|
||||
max_temp = 0.0
|
||||
else:
|
||||
temps = []
|
||||
for m in temp_metrics:
|
||||
v = m.get('value')
|
||||
if v is None:
|
||||
continue
|
||||
try:
|
||||
temps.append(float(v))
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
max_temp = max(temps) if temps else 0.0
|
||||
|
||||
host_totals.append({
|
||||
"host_id": host.id,
|
||||
"host_name": host.name,
|
||||
@@ -272,7 +346,7 @@ async def dashboard_totals(
|
||||
"max_temperature": round(max_temp, 1),
|
||||
"last_seen": host.last_seen.isoformat() if host.last_seen else None,
|
||||
})
|
||||
|
||||
|
||||
# Group totals
|
||||
group_totals = {}
|
||||
for gt in host_totals:
|
||||
@@ -282,22 +356,26 @@ async def dashboard_totals(
|
||||
"group_id": gid,
|
||||
"group_name": next((g["name"] for g in groups if g["id"] == gid), "Unknown"),
|
||||
"total_power": 0,
|
||||
"avg_temperature": 0,
|
||||
"avg_max_temperature": 0,
|
||||
"host_count": 0,
|
||||
}
|
||||
group_totals[gid]["total_power"] += gt["total_power"]
|
||||
group_totals[gid]["avg_temperature"] += gt["max_temperature"]
|
||||
group_totals[gid]["total_power"] += gt["total_power"] or 0
|
||||
group_totals[gid]["avg_max_temperature"] += gt["max_temperature"] or 0
|
||||
group_totals[gid]["host_count"] += 1
|
||||
|
||||
# Calculate averages
|
||||
|
||||
# Calculate averages. NOTE (B13): the field is named
|
||||
# avg_max_temperature because it averages each host's max_temperature;
|
||||
# the frontend does not read this field (Dashboard.vue computes its own
|
||||
# average from max_temperature), so the rename is safe.
|
||||
for gid in group_totals.values():
|
||||
if gid["host_count"] > 0:
|
||||
gid["avg_temperature"] = round(gid["avg_temperature"] / gid["host_count"], 1)
|
||||
gid["avg_max_temperature"] = round(gid["avg_max_temperature"] / gid["host_count"], 1)
|
||||
gid["total_power"] = round(gid["total_power"], 1)
|
||||
|
||||
|
||||
return {
|
||||
"hosts": host_totals,
|
||||
"groups": [{"group_id": g["id"], "group_name": g["name"]} for g in groups],
|
||||
"group_totals": list(group_totals.values()),
|
||||
}
|
||||
|
||||
|
||||
@@ -310,6 +388,9 @@ async def trend(
|
||||
current_user: UserModel = Depends(get_current_user),
|
||||
):
|
||||
"""Get trend data for all hosts in a group for the specified time range."""
|
||||
if metric_type not in ("temperature", "fan", "voltage", "power"):
|
||||
raise HTTPException(400, f"Invalid metric_type: {metric_type!r}")
|
||||
|
||||
# Parse time range to timedelta
|
||||
range_map = {
|
||||
"default": timedelta(hours=1),
|
||||
@@ -325,7 +406,7 @@ async def trend(
|
||||
"30d": timedelta(days=30),
|
||||
}
|
||||
delta = range_map.get(time_range, timedelta(hours=1))
|
||||
start_time = datetime.utcnow() - delta
|
||||
start_time = datetime.now(timezone.utc).replace(tzinfo=None) - delta
|
||||
|
||||
# Get enabled hosts
|
||||
stmt = select(HostModel).where(HostModel.enabled.is_(True))
|
||||
@@ -337,22 +418,37 @@ async def trend(
|
||||
# Query trend data for each host
|
||||
trend_data = {}
|
||||
for host in hosts:
|
||||
metrics = influxdb_module.query_trend(host.id, metric_type, start_time)
|
||||
metrics = await influxdb_module.aquery_trend(host.id, metric_type, start_time)
|
||||
points = []
|
||||
for m in metrics:
|
||||
ts = m.get("timestamp") or m.get("time")
|
||||
if not ts or m.get("value") is None:
|
||||
continue
|
||||
# For power metrics, only include total power sensors
|
||||
if metric_type == "power":
|
||||
metric_name = m.get("metric_name", "")
|
||||
if metric_name not in ('Total_Power', 'Power', 'Pwr Consumption'):
|
||||
if metric_type == "power":
|
||||
# B30: same aggregation as the dashboard totals — per timestamp,
|
||||
# native total-power sensor wins, otherwise sum PSU{N}_PIN.
|
||||
by_ts: dict[str, list[dict]] = {}
|
||||
for m in metrics:
|
||||
ts = m.get("timestamp") or m.get("time")
|
||||
if not ts or m.get("value") is None:
|
||||
continue
|
||||
points.append({
|
||||
"timestamp": ts,
|
||||
"metric_name": m.get("metric_name", ""),
|
||||
"value": m["value"],
|
||||
})
|
||||
by_ts.setdefault(str(ts), []).append(m)
|
||||
for ts, ts_metrics in by_ts.items():
|
||||
total = resolve_total_power(ts_metrics)
|
||||
if total is None:
|
||||
continue
|
||||
points.append({
|
||||
"timestamp": ts,
|
||||
"metric_name": "Total_Power",
|
||||
"value": total,
|
||||
})
|
||||
points.sort(key=lambda p: p["timestamp"])
|
||||
else:
|
||||
for m in metrics:
|
||||
ts = m.get("timestamp") or m.get("time")
|
||||
if not ts or m.get("value") is None:
|
||||
continue
|
||||
points.append({
|
||||
"timestamp": ts,
|
||||
"metric_name": m.get("metric_name", ""),
|
||||
"value": m["value"],
|
||||
})
|
||||
trend_data[str(host.id)] = {
|
||||
"host_id": host.id,
|
||||
"host_name": host.name,
|
||||
@@ -379,24 +475,10 @@ async def history(
|
||||
current_user: UserModel = Depends(get_current_user),
|
||||
):
|
||||
"""Query historical metrics for a host from InfluxDB."""
|
||||
# Parse datetime with timezone handling
|
||||
def parse_dt(dt_str: Optional[str]) -> Optional[datetime]:
|
||||
if not dt_str:
|
||||
return None
|
||||
dt_str = dt_str.replace('Z', '+00:00')
|
||||
try:
|
||||
dt = datetime.fromisoformat(dt_str)
|
||||
if dt.tzinfo is not None:
|
||||
return dt.astimezone(timezone.utc).replace(tzinfo=None)
|
||||
return dt
|
||||
except ValueError:
|
||||
clean_str = dt_str.split('+')[0].split('-')[0][:19] if '+' in dt_str or dt_str.count('-') > 2 else dt_str[:19]
|
||||
return datetime.fromisoformat(clean_str)
|
||||
|
||||
start_dt = parse_dt(start) or (datetime.now() - timedelta(hours=1))
|
||||
end_dt = parse_dt(end)
|
||||
_check_metric_type(metric_type)
|
||||
start_dt, end_dt = _parse_time_range(start, end)
|
||||
|
||||
data = influxdb_module.query_metrics(host_id, metric_type, start_dt, end_dt)
|
||||
data = await influxdb_module.aquery_metrics(host_id, metric_type, start_dt, end_dt)
|
||||
return {"host_id": host_id, "metric_type": metric_type, "data": data}
|
||||
|
||||
|
||||
@@ -410,7 +492,7 @@ async def realtime(
|
||||
current_user: UserModel = Depends(get_current_user)
|
||||
):
|
||||
"""Return the latest metrics for a host, grouped by type."""
|
||||
grouped = influxdb_module.query_latest_grouped(host_id)
|
||||
grouped = await influxdb_module.aquery_latest_grouped(host_id)
|
||||
return {"host_id": host_id, "metrics": grouped}
|
||||
|
||||
|
||||
@@ -425,16 +507,15 @@ async def component_presence(
|
||||
):
|
||||
"""Get component presence status for a host from raw IPMI data.
|
||||
|
||||
Reads /tmp/ipmi_raw.txt for parsing component status.
|
||||
Reads /tmp/ipmi_raw_{host_id}.txt (B10) — the per-host file the
|
||||
collector actually writes, same as the /grouped endpoint.
|
||||
"""
|
||||
raw_file = PathLib("/tmp/ipmi_raw.txt")
|
||||
raw_file = PathLib(f"/tmp/ipmi_raw_{host_id}.txt")
|
||||
|
||||
if not raw_file.exists():
|
||||
return {
|
||||
"host_id": host_id,
|
||||
"error": "Raw IPMI data file not found",
|
||||
"components": None,
|
||||
}
|
||||
raise HTTPException(
|
||||
404, f"Raw IPMI data file not found for host {host_id} — no collection yet?"
|
||||
)
|
||||
|
||||
try:
|
||||
raw_data = raw_file.read_text(encoding="utf-8")
|
||||
@@ -454,11 +535,7 @@ async def component_presence(
|
||||
}
|
||||
except Exception as exc:
|
||||
logger.error("Failed to parse component presence for host %s: %s", host_id, exc)
|
||||
return {
|
||||
"host_id": host_id,
|
||||
"error": str(exc),
|
||||
"components": None,
|
||||
}
|
||||
raise HTTPException(500, f"Failed to parse component presence: {exc}")
|
||||
|
||||
|
||||
@router.get("/components/{host_id}/grouped")
|
||||
|
||||
@@ -8,10 +8,12 @@ from pydantic import BaseModel, Field
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.crypto import decrypt_value, encrypt_value
|
||||
from app.database import get_db
|
||||
from app.models import SystemSetting
|
||||
from app.routers.auth import get_current_user
|
||||
from app.models import User as UserModel
|
||||
from app.ssrf import validate_smtp_host, validate_webhook_url
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
router = APIRouter(prefix="/api/settings", tags=["settings"])
|
||||
@@ -28,18 +30,32 @@ DEFAULT_SETTINGS = {
|
||||
"smtp_use_tls": "true",
|
||||
}
|
||||
|
||||
# Settings whose values are secrets: encrypted at rest, never returned
|
||||
# in full through the API.
|
||||
SENSITIVE_SETTINGS = {"smtp_password", "feishu_webhook_url"}
|
||||
|
||||
|
||||
def _mask_webhook_url(url: str) -> str:
|
||||
"""Mask a webhook URL, revealing only the last 4 characters."""
|
||||
if not url:
|
||||
return ""
|
||||
tail = url[-4:] if len(url) >= 4 else url
|
||||
return f"****{tail}"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Schemas
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
class SettingsResponse(BaseModel):
|
||||
feishu_webhook_url: str = ""
|
||||
# The full Feishu webhook URL is never returned; only a masked form.
|
||||
feishu_webhook_masked: str = ""
|
||||
feishu_keyword: str = ""
|
||||
smtp_host: str = ""
|
||||
smtp_port: str = "587"
|
||||
smtp_user: str = ""
|
||||
smtp_password: str = ""
|
||||
# The SMTP password is never returned; only whether one is set.
|
||||
smtp_password_set: bool = False
|
||||
smtp_from: str = ""
|
||||
smtp_use_tls: bool = True
|
||||
|
||||
@@ -68,18 +84,23 @@ class TestFeishuRequest(BaseModel):
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
async def get_setting(db: AsyncSession, key: str) -> str:
|
||||
"""Get a setting value by key."""
|
||||
"""Get a setting value by key (sensitive values are decrypted on read)."""
|
||||
result = await db.execute(
|
||||
select(SystemSetting).where(SystemSetting.key == key)
|
||||
)
|
||||
setting = result.scalar_one_or_none()
|
||||
if setting:
|
||||
return setting.value
|
||||
value = setting.value or ""
|
||||
if key in SENSITIVE_SETTINGS:
|
||||
return decrypt_value(value) or ""
|
||||
return value
|
||||
return DEFAULT_SETTINGS.get(key, "")
|
||||
|
||||
|
||||
async def set_setting(db: AsyncSession, key: str, value: str, description: str = ""):
|
||||
"""Set a setting value."""
|
||||
"""Set a setting value (sensitive values are encrypted before storage)."""
|
||||
if key in SENSITIVE_SETTINGS:
|
||||
value = encrypt_value(value) or ""
|
||||
result = await db.execute(
|
||||
select(SystemSetting).where(SystemSetting.key == key)
|
||||
)
|
||||
@@ -98,6 +119,24 @@ async def set_setting(db: AsyncSession, key: str, value: str, description: str =
|
||||
await db.flush()
|
||||
|
||||
|
||||
async def _build_response(db: AsyncSession) -> SettingsResponse:
|
||||
"""Build the redacted settings response."""
|
||||
values = {}
|
||||
for key in DEFAULT_SETTINGS.keys():
|
||||
values[key] = await get_setting(db, key)
|
||||
|
||||
return SettingsResponse(
|
||||
feishu_webhook_masked=_mask_webhook_url(values.get("feishu_webhook_url", "")),
|
||||
feishu_keyword=values.get("feishu_keyword", ""),
|
||||
smtp_host=values.get("smtp_host", ""),
|
||||
smtp_port=values.get("smtp_port", "587"),
|
||||
smtp_user=values.get("smtp_user", ""),
|
||||
smtp_password_set=bool(values.get("smtp_password", "")),
|
||||
smtp_from=values.get("smtp_from", ""),
|
||||
smtp_use_tls=values.get("smtp_use_tls", "true").lower() == "true",
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# API Endpoints
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -107,15 +146,8 @@ async def get_settings(
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: UserModel = Depends(get_current_user),
|
||||
):
|
||||
"""Get all system settings."""
|
||||
settings = {}
|
||||
for key in DEFAULT_SETTINGS.keys():
|
||||
settings[key] = await get_setting(db, key)
|
||||
|
||||
# Convert smtp_use_tls to boolean
|
||||
settings["smtp_use_tls"] = settings.get("smtp_use_tls", "true").lower() == "true"
|
||||
|
||||
return SettingsResponse(**settings)
|
||||
"""Get all system settings (secrets are redacted)."""
|
||||
return await _build_response(db)
|
||||
|
||||
|
||||
@router.put("", response_model=SettingsResponse)
|
||||
@@ -124,28 +156,39 @@ async def update_settings(
|
||||
db: AsyncSession = Depends(get_db),
|
||||
current_user: UserModel = Depends(get_current_user),
|
||||
):
|
||||
"""Update system settings."""
|
||||
"""Update system settings.
|
||||
|
||||
For the secret settings (``smtp_password``, ``feishu_webhook_url``) an
|
||||
empty/absent value means "leave unchanged".
|
||||
"""
|
||||
update_data = body.model_dump(exclude_unset=True)
|
||||
|
||||
for key, value in update_data.items():
|
||||
if value is not None:
|
||||
# Convert boolean to string for storage
|
||||
if isinstance(value, bool):
|
||||
str_value = "true" if value else "false"
|
||||
else:
|
||||
str_value = str(value)
|
||||
await set_setting(db, key, str_value)
|
||||
if value is None:
|
||||
continue
|
||||
if key in SENSITIVE_SETTINGS and value == "":
|
||||
continue
|
||||
if key == "feishu_webhook_url":
|
||||
try:
|
||||
validate_webhook_url(str(value))
|
||||
except ValueError as exc:
|
||||
raise HTTPException(400, str(exc))
|
||||
if key == "smtp_host" and value:
|
||||
try:
|
||||
validate_smtp_host(str(value))
|
||||
except ValueError as exc:
|
||||
raise HTTPException(400, str(exc))
|
||||
# Convert boolean to string for storage
|
||||
if isinstance(value, bool):
|
||||
str_value = "true" if value else "false"
|
||||
else:
|
||||
str_value = str(value)
|
||||
await set_setting(db, key, str_value)
|
||||
|
||||
await db.commit()
|
||||
|
||||
# Return updated settings
|
||||
settings = {}
|
||||
for key in DEFAULT_SETTINGS.keys():
|
||||
settings[key] = await get_setting(db, key)
|
||||
|
||||
settings["smtp_use_tls"] = settings.get("smtp_use_tls", "true").lower() == "true"
|
||||
|
||||
return SettingsResponse(**settings)
|
||||
return await _build_response(db)
|
||||
|
||||
|
||||
@router.post("/test-email")
|
||||
@@ -169,6 +212,11 @@ async def test_email(
|
||||
if not smtp_host or not smtp_user or not smtp_password:
|
||||
raise HTTPException(400, "SMTP配置不完整,请先配置SMTP服务器信息")
|
||||
|
||||
try:
|
||||
validate_smtp_host(smtp_host)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(400, str(exc))
|
||||
|
||||
try:
|
||||
msg = MIMEMultipart()
|
||||
msg['From'] = smtp_from or smtp_user
|
||||
@@ -209,6 +257,11 @@ async def test_feishu(
|
||||
if not webhook_url:
|
||||
raise HTTPException(400, "飞书Webhook URL未配置")
|
||||
|
||||
try:
|
||||
validate_webhook_url(webhook_url)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(400, str(exc))
|
||||
|
||||
# Build message with keyword if configured
|
||||
if keyword:
|
||||
content = f"[IPMI监控测试] {keyword}\n这是一封来自IPMI监控系统的测试消息。\n如果您收到此消息,说明飞书Webhook配置正确。"
|
||||
|
||||
@@ -12,8 +12,9 @@ from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app import influxdb as influxdb_module
|
||||
from app.crypto import decrypt_value
|
||||
from app.database import async_session
|
||||
from app.models import AlertHistory, AlertRule, Host
|
||||
from app.models import AlertHistory, AlertRule, Host, SystemSetting
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -39,6 +40,30 @@ _alert_active: dict[tuple[int, int], bool] = {}
|
||||
NOTIFY_COOLDOWN_SECONDS = 3 * 60 * 60 # 10800 seconds
|
||||
|
||||
|
||||
async def get_settings_batch(session: AsyncSession, keys: list[str]) -> dict[str, str]:
|
||||
"""Fetch several SystemSetting values with a single query (O3).
|
||||
|
||||
Replaces the old pattern of one ``select`` per key (5 round-trips for
|
||||
SMTP alone, repeated for alert and recovery paths). Missing keys map
|
||||
to ``""``.
|
||||
"""
|
||||
if not keys:
|
||||
return {}
|
||||
result = await session.execute(
|
||||
select(SystemSetting).where(SystemSetting.key.in_(keys))
|
||||
)
|
||||
settings_dict = {s.key: s.value for s in result.scalars().all()}
|
||||
return {k: settings_dict.get(k, "") for k in keys}
|
||||
|
||||
|
||||
def _smtp_use_tls_enabled(settings_dict: dict[str, str]) -> bool:
|
||||
"""Interpret the stored ``smtp_use_tls`` flag (default True)."""
|
||||
raw = settings_dict.get("smtp_use_tls", "true")
|
||||
if isinstance(raw, bool):
|
||||
return raw
|
||||
return str(raw).strip().lower() in ("true", "1", "yes", "on")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Rule evaluation
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -118,17 +143,18 @@ async def _create_alert_history(
|
||||
if rule.notify_email:
|
||||
try:
|
||||
from app.services.email_notify import send_alert_email
|
||||
# 从数据库读取SMTP配置
|
||||
# 从数据库批量读取SMTP配置(一次查询取回全部 key,O3)
|
||||
async with async_session() as session:
|
||||
from sqlalchemy import select
|
||||
from app.models import SystemSetting
|
||||
settings_dict = {}
|
||||
for key in ["smtp_host", "smtp_port", "smtp_user", "smtp_password", "smtp_from"]:
|
||||
result = await session.execute(
|
||||
select(SystemSetting).where(SystemSetting.key == key)
|
||||
)
|
||||
setting = result.scalar_one_or_none()
|
||||
settings_dict[key] = setting.value if setting else ""
|
||||
settings_dict = await get_settings_batch(
|
||||
session,
|
||||
["smtp_host", "smtp_port", "smtp_user", "smtp_password",
|
||||
"smtp_from", "smtp_use_tls"],
|
||||
)
|
||||
|
||||
# SMTP 口令入库时已 Fernet 加密,解密后再使用(历史明文原样返回)。
|
||||
settings_dict["smtp_password"] = decrypt_value(
|
||||
settings_dict.get("smtp_password", "")
|
||||
) or ""
|
||||
|
||||
addresses = [e.strip() for e in rule.email_addresses.split(",") if e.strip()]
|
||||
if addresses and settings_dict["smtp_host"] and settings_dict["smtp_user"]:
|
||||
@@ -137,6 +163,7 @@ async def _create_alert_history(
|
||||
subject=f"[IPMI Alert] {host.name} – {rule.metric_name}",
|
||||
alert_info=alert_info,
|
||||
smtp_config=settings_dict,
|
||||
smtp_use_tls=_smtp_use_tls_enabled(settings_dict),
|
||||
)
|
||||
else:
|
||||
logger.warning("Email SMTP not configured in settings, skipping alert for rule %d", rule.id)
|
||||
@@ -146,18 +173,23 @@ async def _create_alert_history(
|
||||
if rule.notify_feishu:
|
||||
try:
|
||||
from app.services.feishu_notify import send_feishu_alert
|
||||
# 从数据库读取用户配置的飞书 webhook URL
|
||||
# 从数据库批量读取飞书配置(B25:keyword 也要读)
|
||||
async with async_session() as session:
|
||||
from sqlalchemy import select
|
||||
from app.models import SystemSetting
|
||||
result = await session.execute(
|
||||
select(SystemSetting).where(SystemSetting.key == "feishu_webhook_url")
|
||||
feishu_settings = await get_settings_batch(
|
||||
session, ["feishu_webhook_url", "feishu_keyword"]
|
||||
)
|
||||
setting = result.scalar_one_or_none()
|
||||
url = setting.value if setting else ""
|
||||
# 飞书 webhook URL 入库时已 Fernet 加密,解密后再使用。
|
||||
url = decrypt_value(
|
||||
feishu_settings.get("feishu_webhook_url", "")
|
||||
) or ""
|
||||
|
||||
if url:
|
||||
await send_feishu_alert(url, alert_info)
|
||||
feishu_info = dict(alert_info)
|
||||
keyword = feishu_settings.get("feishu_keyword", "")
|
||||
if keyword:
|
||||
# 飞书关键词安全校验要求关键词出现在卡片文本中
|
||||
feishu_info["message"] = f"{feishu_info.get('message', '')}\n{keyword}"
|
||||
await send_feishu_alert(url, feishu_info)
|
||||
else:
|
||||
logger.warning("Feishu webhook URL not configured in settings, skipping alert for rule %d", rule.id)
|
||||
except Exception:
|
||||
@@ -196,15 +228,16 @@ async def _create_recovery_notification(
|
||||
try:
|
||||
from app.services.email_notify import send_recovery_email
|
||||
async with async_session() as session:
|
||||
from sqlalchemy import select
|
||||
from app.models import SystemSetting
|
||||
settings_dict = {}
|
||||
for key in ["smtp_host", "smtp_port", "smtp_user", "smtp_password", "smtp_from"]:
|
||||
result = await session.execute(
|
||||
select(SystemSetting).where(SystemSetting.key == key)
|
||||
)
|
||||
setting = result.scalar_one_or_none()
|
||||
settings_dict[key] = setting.value if setting else ""
|
||||
settings_dict = await get_settings_batch(
|
||||
session,
|
||||
["smtp_host", "smtp_port", "smtp_user", "smtp_password",
|
||||
"smtp_from", "smtp_use_tls"],
|
||||
)
|
||||
|
||||
# SMTP 口令入库时已 Fernet 加密,解密后再使用(历史明文原样返回)。
|
||||
settings_dict["smtp_password"] = decrypt_value(
|
||||
settings_dict.get("smtp_password", "")
|
||||
) or ""
|
||||
|
||||
addresses = [e.strip() for e in rule.email_addresses.split(",") if e.strip()]
|
||||
if addresses and settings_dict["smtp_host"] and settings_dict["smtp_user"]:
|
||||
@@ -213,6 +246,7 @@ async def _create_recovery_notification(
|
||||
subject=f"✅ IPMI Recovery: {host.name} – {rule.metric_name}",
|
||||
recovery_info=recovery_info,
|
||||
smtp_config=settings_dict,
|
||||
smtp_use_tls=_smtp_use_tls_enabled(settings_dict),
|
||||
)
|
||||
else:
|
||||
logger.warning("Email SMTP not configured, skipping recovery for rule %d", rule.id)
|
||||
@@ -223,16 +257,20 @@ async def _create_recovery_notification(
|
||||
try:
|
||||
from app.services.feishu_notify import send_feishu_recovery
|
||||
async with async_session() as session:
|
||||
from sqlalchemy import select
|
||||
from app.models import SystemSetting
|
||||
result = await session.execute(
|
||||
select(SystemSetting).where(SystemSetting.key == "feishu_webhook_url")
|
||||
feishu_settings = await get_settings_batch(
|
||||
session, ["feishu_webhook_url", "feishu_keyword"]
|
||||
)
|
||||
setting = result.scalar_one_or_none()
|
||||
url = setting.value if setting else ""
|
||||
# 飞书 webhook URL 入库时已 Fernet 加密,解密后再使用。
|
||||
url = decrypt_value(
|
||||
feishu_settings.get("feishu_webhook_url", "")
|
||||
) or ""
|
||||
|
||||
if url:
|
||||
await send_feishu_recovery(url, recovery_info)
|
||||
feishu_info = dict(recovery_info)
|
||||
keyword = feishu_settings.get("feishu_keyword", "")
|
||||
if keyword:
|
||||
feishu_info["message"] = f"{feishu_info.get('message', '')}\n{keyword}"
|
||||
await send_feishu_recovery(url, feishu_info)
|
||||
else:
|
||||
logger.warning("Feishu webhook URL not configured, skipping recovery for rule %d", rule.id)
|
||||
except Exception:
|
||||
@@ -253,11 +291,22 @@ async def check_alerts() -> None:
|
||||
if not rules:
|
||||
return
|
||||
|
||||
# B16: drop in-memory state for rules that no longer exist or are
|
||||
# disabled, so deleted/disabled rules stop accumulating state.
|
||||
live_rule_ids = {r.id for r in rules}
|
||||
for state in (_first_breach, _last_notify, _alert_active):
|
||||
for key in [k for k in state if k[0] not in live_rule_ids]:
|
||||
del state[key]
|
||||
|
||||
# Resolve hosts for rules (either specific host or all hosts in a group)
|
||||
for rule in rules:
|
||||
host_ids: list[int] = []
|
||||
if rule.host_id:
|
||||
host_ids = [rule.host_id]
|
||||
# B17: a host-bound rule only applies while the host exists
|
||||
# and is enabled — same as the group/all-hosts branches.
|
||||
host_obj = await session.get(Host, rule.host_id)
|
||||
if host_obj is not None and host_obj.enabled:
|
||||
host_ids = [rule.host_id]
|
||||
elif rule.group_id:
|
||||
stmt_h = select(Host).where(Host.group_id == rule.group_id, Host.enabled.is_(True))
|
||||
res_h = await session.execute(stmt_h)
|
||||
@@ -269,49 +318,86 @@ async def check_alerts() -> None:
|
||||
host_ids = [h.id for h in res_h.scalars().all()]
|
||||
|
||||
for hid in host_ids:
|
||||
latest = influxdb_module.query_latest(hid, rule.metric_type)
|
||||
for point in latest:
|
||||
if point.get("metric_name") != rule.metric_name:
|
||||
continue
|
||||
metric_value = point["value"]
|
||||
# B2: one bad rule/host must never abort the whole round.
|
||||
try:
|
||||
await _check_rule_for_host(session, rule, hid)
|
||||
except Exception:
|
||||
logger.exception(
|
||||
"Alert check failed for rule %d host %d; continuing",
|
||||
rule.id, hid,
|
||||
)
|
||||
|
||||
breached = evaluate_rule(rule, metric_value)
|
||||
key = (rule.id, hid)
|
||||
|
||||
if breached:
|
||||
now = datetime.now()
|
||||
async def _check_rule_for_host(
|
||||
session: AsyncSession, rule: AlertRule, hid: int
|
||||
) -> None:
|
||||
"""Evaluate one rule against one host's latest metrics (B2 isolation)."""
|
||||
latest = await influxdb_module.aquery_latest(hid, rule.metric_type)
|
||||
for point in latest:
|
||||
if point.get("metric_name") != rule.metric_name:
|
||||
continue
|
||||
try:
|
||||
raw_value = point.get("value")
|
||||
if raw_value is None:
|
||||
continue
|
||||
metric_value = float(raw_value)
|
||||
except (TypeError, ValueError):
|
||||
logger.warning(
|
||||
"Non-numeric metric value %r for rule %d host %d; skipping",
|
||||
point.get("value"), rule.id, hid,
|
||||
)
|
||||
continue
|
||||
|
||||
if key not in _first_breach:
|
||||
# First time breaching — start the duration timer
|
||||
_first_breach[key] = now
|
||||
try:
|
||||
breached = evaluate_rule(rule, metric_value)
|
||||
except Exception:
|
||||
logger.exception(
|
||||
"Rule evaluation failed for rule %d host %d; skipping",
|
||||
rule.id, hid,
|
||||
)
|
||||
continue
|
||||
|
||||
elapsed = (now - _first_breach[key]).total_seconds()
|
||||
if elapsed >= rule.duration:
|
||||
# Duration satisfied — check if we already have an active alert
|
||||
if not _alert_active.get(key):
|
||||
# No active alert yet → fire alert notification
|
||||
host_obj = await session.get(Host, hid)
|
||||
if host_obj:
|
||||
await _create_alert_history(rule, host_obj, metric_value)
|
||||
_alert_active[key] = True
|
||||
_last_notify[key] = now
|
||||
else:
|
||||
# Alert already active — check cooldown for repeat notification
|
||||
last_notify_time = _last_notify.get(key)
|
||||
if last_notify_time and (now - last_notify_time).total_seconds() >= NOTIFY_COOLDOWN_SECONDS:
|
||||
host_obj = await session.get(Host, hid)
|
||||
if host_obj:
|
||||
await _create_alert_history(rule, host_obj, metric_value)
|
||||
_last_notify[key] = now
|
||||
# Do NOT reset _first_breach — keep the original breach time
|
||||
else:
|
||||
# No longer breaching
|
||||
if key in _first_breach:
|
||||
# Was breaching before → now recovered, send recovery notification
|
||||
host_obj = await session.get(Host, hid)
|
||||
if host_obj:
|
||||
await _create_recovery_notification(rule, host_obj, metric_value)
|
||||
# Clear all state for this rule+host
|
||||
_first_breach.pop(key, None)
|
||||
_alert_active.pop(key, None)
|
||||
_last_notify.pop(key, None)
|
||||
key = (rule.id, hid)
|
||||
|
||||
if breached:
|
||||
now = datetime.now()
|
||||
|
||||
if key not in _first_breach:
|
||||
# First time breaching — start the duration timer
|
||||
_first_breach[key] = now
|
||||
|
||||
elapsed = (now - _first_breach[key]).total_seconds()
|
||||
if elapsed >= rule.duration:
|
||||
# Duration satisfied — check if we already have an active alert
|
||||
if not _alert_active.get(key):
|
||||
# No active alert yet → fire alert notification
|
||||
host_obj = await session.get(Host, hid)
|
||||
if host_obj:
|
||||
# B19: only mark the alert active once the history
|
||||
# record was actually created (host still exists).
|
||||
await _create_alert_history(rule, host_obj, metric_value)
|
||||
_alert_active[key] = True
|
||||
_last_notify[key] = now
|
||||
else:
|
||||
# Alert already active — check cooldown for repeat notification
|
||||
last_notify_time = _last_notify.get(key)
|
||||
if last_notify_time and (now - last_notify_time).total_seconds() >= NOTIFY_COOLDOWN_SECONDS:
|
||||
host_obj = await session.get(Host, hid)
|
||||
if host_obj:
|
||||
await _create_alert_history(rule, host_obj, metric_value)
|
||||
_last_notify[key] = now
|
||||
# Do NOT reset _first_breach — keep the original breach time
|
||||
else:
|
||||
# No longer breaching
|
||||
if key in _first_breach:
|
||||
# B1: only send a recovery notification if an alert was
|
||||
# actually fired. A transient breach that never satisfied
|
||||
# `duration` just clears the timer silently.
|
||||
if _alert_active.get(key):
|
||||
host_obj = await session.get(Host, hid)
|
||||
if host_obj:
|
||||
await _create_recovery_notification(rule, host_obj, metric_value)
|
||||
# Clear all state for this rule+host
|
||||
_first_breach.pop(key, None)
|
||||
_alert_active.pop(key, None)
|
||||
_last_notify.pop(key, None)
|
||||
|
||||
@@ -1,16 +1,26 @@
|
||||
"""SMTP email notification service."""
|
||||
|
||||
import asyncio
|
||||
import collections
|
||||
import logging
|
||||
import re
|
||||
import smtplib
|
||||
import ssl
|
||||
from email.mime.multipart import MIMEMultipart
|
||||
from email.mime.text import MIMEText
|
||||
from typing import Any
|
||||
|
||||
from app.config import settings
|
||||
from app.crypto import decrypt_value
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# SMTP connect/read timeout for the blocking smtplib calls (B26).
|
||||
SMTP_TIMEOUT_SECONDS = 15
|
||||
|
||||
# Matches a leftover "{{ placeholder }}" after rendering (B27).
|
||||
_LEFTOVER_PLACEHOLDER_RE = re.compile(r"\{\{[^}]*\}\}")
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# HTML email template
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -136,38 +146,86 @@ IPMI 恢复通知
|
||||
"""
|
||||
|
||||
|
||||
async def send_alert_email(
|
||||
to_addresses: list[str],
|
||||
subject: str,
|
||||
alert_info: dict[str, Any],
|
||||
smtp_config: dict[str, str] = None,
|
||||
) -> None:
|
||||
"""Send an alert email to *to_addresses* using the configured SMTP server."""
|
||||
def _parse_smtp_port(raw: Any) -> int:
|
||||
"""Parse and validate an SMTP port (B26).
|
||||
|
||||
# 优先使用传入的 smtp_config,否则回退到环境变量配置
|
||||
Raises ``ValueError`` for non-numeric values or ports outside 1-65535.
|
||||
Callers must do this inside a ``try`` so a bad stored value is a
|
||||
logged warning, not a 500.
|
||||
"""
|
||||
port = int(str(raw).strip())
|
||||
if not 1 <= port <= 65535:
|
||||
raise ValueError(f"SMTP port out of range: {port}")
|
||||
return port
|
||||
|
||||
|
||||
def _parse_smtp_use_tls(raw: Any, default: bool = True) -> bool:
|
||||
"""Interpret an smtp_use_tls flag from settings or a caller argument."""
|
||||
if isinstance(raw, bool):
|
||||
return raw
|
||||
if raw is None:
|
||||
return default
|
||||
return str(raw).strip().lower() in ("true", "1", "yes", "on")
|
||||
|
||||
|
||||
def _render_plain(template: str, info: dict[str, Any]) -> str:
|
||||
"""Render a ``{key}`` plain-text template, missing keys become "" (B27)."""
|
||||
safe = collections.defaultdict(str, {k: str(v) for k, v in info.items()})
|
||||
return template.format_map(safe)
|
||||
|
||||
|
||||
def _render_html(template: str, info: dict[str, Any]) -> str:
|
||||
"""Render a ``{{ key }}`` HTML template and strip leftovers (B27)."""
|
||||
html = template
|
||||
for key, val in info.items():
|
||||
html = html.replace("{{ " + key + " }}", str(val))
|
||||
# Any placeholder not supplied by the caller is removed rather than
|
||||
# leaking "{{ ... }}" into the sent email.
|
||||
return _LEFTOVER_PLACEHOLDER_RE.sub("", html)
|
||||
|
||||
|
||||
def _smtp_settings(smtp_config: dict[str, str] | None) -> tuple[str, int, str, str, str, bool]:
|
||||
"""Resolve SMTP connection settings from a config dict or env settings.
|
||||
|
||||
Returns ``(host, port, user, password, from_addr, use_tls)``.
|
||||
Raises ``ValueError`` when the configured port is invalid (B26).
|
||||
"""
|
||||
if smtp_config:
|
||||
smtp_host = smtp_config.get("smtp_host", "")
|
||||
smtp_port = int(smtp_config.get("smtp_port", "587"))
|
||||
smtp_port = _parse_smtp_port(smtp_config.get("smtp_port", "587"))
|
||||
smtp_user = smtp_config.get("smtp_user", "")
|
||||
smtp_password = smtp_config.get("smtp_password", "")
|
||||
# 口令入库时已 Fernet 加密,解密后再使用;历史明文原样返回。
|
||||
smtp_password = decrypt_value(smtp_config.get("smtp_password", "")) or ""
|
||||
smtp_from = smtp_config.get("smtp_from", "")
|
||||
smtp_use_tls = _parse_smtp_use_tls(smtp_config.get("smtp_use_tls", True))
|
||||
else:
|
||||
smtp_host = settings.SMTP_HOST
|
||||
smtp_port = settings.SMTP_PORT
|
||||
smtp_port = _parse_smtp_port(settings.SMTP_PORT)
|
||||
smtp_user = settings.SMTP_USER
|
||||
smtp_password = settings.SMTP_PASSWORD
|
||||
smtp_from = settings.SMTP_FROM
|
||||
smtp_use_tls = True
|
||||
return smtp_host, smtp_port, smtp_user, smtp_password, smtp_from, smtp_use_tls
|
||||
|
||||
if not smtp_host:
|
||||
logger.warning("SMTP not configured – skipping email notification")
|
||||
return
|
||||
|
||||
# Render HTML with simple string replacement
|
||||
html = _ALERT_EMAIL_HTML
|
||||
plain = _PLAIN_TEMPLATE.format(**alert_info)
|
||||
for key, val in alert_info.items():
|
||||
html = html.replace("{{ " + key + " }}", str(val))
|
||||
def _send_mail(
|
||||
to_addresses: list[str],
|
||||
subject: str,
|
||||
plain: str,
|
||||
html: str,
|
||||
smtp_host: str,
|
||||
smtp_port: int,
|
||||
smtp_user: str,
|
||||
smtp_password: str,
|
||||
smtp_from: str,
|
||||
smtp_use_tls: bool,
|
||||
) -> None:
|
||||
"""Blocking SMTP send (runs inside ``asyncio.to_thread``).
|
||||
|
||||
Uses a ``with`` statement so the connection is always released (B26),
|
||||
and only calls ``starttls()`` when TLS is enabled *and* the port is
|
||||
not 465 (B24), with an explicit SSL context.
|
||||
"""
|
||||
msg = MIMEMultipart("alternative")
|
||||
msg["Subject"] = subject
|
||||
msg["From"] = smtp_from or smtp_user
|
||||
@@ -175,22 +233,58 @@ async def send_alert_email(
|
||||
msg.attach(MIMEText(plain, "plain", "utf-8"))
|
||||
msg.attach(MIMEText(html, "html", "utf-8"))
|
||||
|
||||
def _send() -> None:
|
||||
if smtp_port == 465:
|
||||
smtp_cls = smtplib.SMTP_SSL
|
||||
else:
|
||||
smtp_cls = smtplib.SMTP
|
||||
context = ssl.create_default_context()
|
||||
if smtp_port == 465:
|
||||
smtp_cls = smtplib.SMTP_SSL
|
||||
server_kwargs: dict[str, Any] = {"context": context}
|
||||
else:
|
||||
smtp_cls = smtplib.SMTP
|
||||
server_kwargs = {}
|
||||
|
||||
with smtp_cls(smtp_host, smtp_port, timeout=15) as server:
|
||||
if smtp_port != 465:
|
||||
server.starttls()
|
||||
if smtp_user and smtp_password:
|
||||
server.login(smtp_user, smtp_password)
|
||||
server.sendmail(msg["From"], to_addresses, msg.as_string())
|
||||
logger.info("Alert email sent to %s", to_addresses)
|
||||
with smtp_cls(smtp_host, smtp_port, timeout=SMTP_TIMEOUT_SECONDS, **server_kwargs) as server:
|
||||
if smtp_use_tls and smtp_port != 465:
|
||||
server.starttls(context=context)
|
||||
if smtp_user and smtp_password:
|
||||
server.login(smtp_user, smtp_password)
|
||||
server.sendmail(msg["From"], to_addresses, msg.as_string())
|
||||
logger.info("Email sent to %s", to_addresses)
|
||||
|
||||
|
||||
async def send_alert_email(
|
||||
to_addresses: list[str],
|
||||
subject: str,
|
||||
alert_info: dict[str, Any],
|
||||
smtp_config: dict[str, str] = None,
|
||||
smtp_use_tls: bool | None = None,
|
||||
) -> None:
|
||||
"""Send an alert email to *to_addresses* using the configured SMTP server.
|
||||
|
||||
``smtp_use_tls`` (B24) may be passed explicitly or carried inside
|
||||
``smtp_config``; an explicit argument wins.
|
||||
"""
|
||||
try:
|
||||
smtp_host, smtp_port, smtp_user, smtp_password, smtp_from, cfg_use_tls = (
|
||||
_smtp_settings(smtp_config)
|
||||
)
|
||||
except (ValueError, TypeError) as exc:
|
||||
logger.warning("Invalid SMTP configuration, skipping alert email: %s", exc)
|
||||
return
|
||||
use_tls = cfg_use_tls if smtp_use_tls is None else smtp_use_tls
|
||||
|
||||
if not smtp_host:
|
||||
logger.warning("SMTP not configured – skipping email notification")
|
||||
return
|
||||
|
||||
# Render templates with missing-field-safe rendering (B27)
|
||||
html = _render_html(_ALERT_EMAIL_HTML, alert_info)
|
||||
plain = _render_plain(_PLAIN_TEMPLATE, alert_info)
|
||||
|
||||
# Run blocking SMTP call in a thread so it doesn't block the event loop
|
||||
await asyncio.to_thread(_send)
|
||||
await asyncio.to_thread(
|
||||
_send_mail,
|
||||
to_addresses, subject, plain, html,
|
||||
smtp_host, smtp_port, smtp_user, smtp_password, smtp_from, use_tls,
|
||||
)
|
||||
|
||||
|
||||
async def send_recovery_email(
|
||||
@@ -198,51 +292,53 @@ async def send_recovery_email(
|
||||
subject: str,
|
||||
recovery_info: dict[str, Any],
|
||||
smtp_config: dict[str, str] = None,
|
||||
smtp_use_tls: bool | None = None,
|
||||
) -> None:
|
||||
"""Send a recovery notification email to *to_addresses*."""
|
||||
|
||||
if smtp_config:
|
||||
smtp_host = smtp_config.get("smtp_host", "")
|
||||
smtp_port = int(smtp_config.get("smtp_port", "587"))
|
||||
smtp_user = smtp_config.get("smtp_user", "")
|
||||
smtp_password = smtp_config.get("smtp_password", "")
|
||||
smtp_from = smtp_config.get("smtp_from", "")
|
||||
else:
|
||||
smtp_host = settings.SMTP_HOST
|
||||
smtp_port = settings.SMTP_PORT
|
||||
smtp_user = settings.SMTP_USER
|
||||
smtp_password = settings.SMTP_PASSWORD
|
||||
smtp_from = settings.SMTP_FROM
|
||||
try:
|
||||
smtp_host, smtp_port, smtp_user, smtp_password, smtp_from, cfg_use_tls = (
|
||||
_smtp_settings(smtp_config)
|
||||
)
|
||||
except (ValueError, TypeError) as exc:
|
||||
logger.warning("Invalid SMTP configuration, skipping recovery email: %s", exc)
|
||||
return
|
||||
use_tls = cfg_use_tls if smtp_use_tls is None else smtp_use_tls
|
||||
|
||||
if not smtp_host:
|
||||
logger.warning("SMTP not configured – skipping recovery email")
|
||||
return
|
||||
|
||||
# Render HTML with simple string replacement
|
||||
html = _RECOVERY_EMAIL_HTML
|
||||
plain = _RECOVERY_PLAIN_TEMPLATE.format(**recovery_info)
|
||||
for key, val in recovery_info.items():
|
||||
html = html.replace("{{ " + key + " }}", str(val))
|
||||
html = _render_html(_RECOVERY_EMAIL_HTML, recovery_info)
|
||||
plain = _render_plain(_RECOVERY_PLAIN_TEMPLATE, recovery_info)
|
||||
|
||||
msg = MIMEMultipart("alternative")
|
||||
msg["Subject"] = subject
|
||||
msg["From"] = smtp_from or smtp_user
|
||||
msg["To"] = ", ".join(to_addresses)
|
||||
msg.attach(MIMEText(plain, "plain", "utf-8"))
|
||||
msg.attach(MIMEText(html, "html", "utf-8"))
|
||||
await asyncio.to_thread(
|
||||
_send_mail,
|
||||
to_addresses, subject, plain, html,
|
||||
smtp_host, smtp_port, smtp_user, smtp_password, smtp_from, use_tls,
|
||||
)
|
||||
|
||||
def _send() -> None:
|
||||
if smtp_port == 465:
|
||||
smtp_cls = smtplib.SMTP_SSL
|
||||
else:
|
||||
smtp_cls = smtplib.SMTP
|
||||
|
||||
with smtp_cls(smtp_host, smtp_port, timeout=15) as server:
|
||||
if smtp_port != 465:
|
||||
server.starttls()
|
||||
if smtp_user and smtp_password:
|
||||
server.login(smtp_user, smtp_password)
|
||||
server.sendmail(msg["From"], to_addresses, msg.as_string())
|
||||
logger.info("Recovery email sent to %s", to_addresses)
|
||||
async def send_test_email(
|
||||
to_address: str,
|
||||
smtp_config: dict[str, str] | None = None,
|
||||
) -> None:
|
||||
"""Send a test email using the same safe path as alert mail (B26).
|
||||
|
||||
await asyncio.to_thread(_send)
|
||||
Intended for the settings test-email endpoint: the port is parsed
|
||||
inside ``try`` and range-checked (1-65535), the blocking SMTP call
|
||||
runs in a thread with a timeout, and the connection is released via
|
||||
a ``with`` statement. Raises ``ValueError`` for a bad port and lets
|
||||
SMTP errors propagate to the caller so the endpoint can report them.
|
||||
"""
|
||||
smtp_host, smtp_port, smtp_user, smtp_password, smtp_from, use_tls = (
|
||||
_smtp_settings(smtp_config)
|
||||
)
|
||||
if not smtp_host:
|
||||
raise ValueError("SMTP host is not configured")
|
||||
plain = "This is a test email from IPMI Monitor. SMTP configuration works."
|
||||
html = f"<p>{plain}</p>"
|
||||
await asyncio.to_thread(
|
||||
_send_mail,
|
||||
[to_address], "[IPMI Monitor] Test Email", plain, html,
|
||||
smtp_host, smtp_port, smtp_user, smtp_password, smtp_from, use_tls,
|
||||
)
|
||||
|
||||
@@ -6,12 +6,25 @@ from typing import Any
|
||||
import httpx
|
||||
|
||||
from app.config import settings
|
||||
from app.crypto import decrypt_value
|
||||
from app.ssrf import validate_webhook_url
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def _checked_webhook_url(webhook_url: str) -> str | None:
|
||||
"""Validate the webhook URL; log and return None when it is not allowed."""
|
||||
try:
|
||||
return validate_webhook_url(webhook_url)
|
||||
except ValueError as exc:
|
||||
logger.warning("Refusing to send Feishu alert to disallowed URL: %s", exc)
|
||||
return None
|
||||
|
||||
|
||||
async def send_feishu_alert(webhook_url: str, alert_info: dict[str, Any]) -> None:
|
||||
"""Send an interactive card message to a Feishu group via custom robot webhook."""
|
||||
# 传入的 URL 可能是入库的 Fernet 密文,解密后再使用;历史明文原样返回。
|
||||
webhook_url = decrypt_value(webhook_url) or ""
|
||||
|
||||
host_name = alert_info.get("host_name", "N/A")
|
||||
host_ip = alert_info.get("host_ip", "N/A")
|
||||
@@ -67,6 +80,9 @@ async def send_feishu_alert(webhook_url: str, alert_info: dict[str, Any]) -> Non
|
||||
}
|
||||
|
||||
try:
|
||||
webhook_url = _checked_webhook_url(webhook_url)
|
||||
if webhook_url is None:
|
||||
return
|
||||
async with httpx.AsyncClient(timeout=10) as client:
|
||||
resp = await client.post(webhook_url, json=card_body)
|
||||
resp.raise_for_status()
|
||||
@@ -136,6 +152,9 @@ async def send_feishu_recovery(webhook_url: str, recovery_info: dict[str, Any])
|
||||
}
|
||||
|
||||
try:
|
||||
webhook_url = _checked_webhook_url(webhook_url)
|
||||
if webhook_url is None:
|
||||
return
|
||||
async with httpx.AsyncClient(timeout=10) as client:
|
||||
resp = await client.post(webhook_url, json=card_body)
|
||||
resp.raise_for_status()
|
||||
|
||||
@@ -5,7 +5,9 @@ Uses *ipmitool* over the LAN interface to gather sensor data from remote BMCs.
|
||||
|
||||
import asyncio
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import tempfile
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any, Optional
|
||||
@@ -15,6 +17,16 @@ from app.config import settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# BMC password decryption (B20 companion fix). ``app.crypto`` is created by
|
||||
# the parallel security fix; until it exists, fall back to the value as-is
|
||||
# (historical plaintext compatibility — decrypt_value itself is specified
|
||||
# to return the original value when decryption fails).
|
||||
try:
|
||||
from app.crypto import decrypt_value
|
||||
except ImportError: # pragma: no cover - crypto module not yet present
|
||||
def decrypt_value(value: str) -> str:
|
||||
return value
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Output parsing helpers
|
||||
# ---------------------------------------------------------------------------
|
||||
@@ -31,49 +43,70 @@ _SDR_RE = re.compile(
|
||||
r"^(?P<name>[^|]+?)\s*\|\s*(?P<value>.+?)\s*\|\s*(?P<status>\w+)"
|
||||
)
|
||||
|
||||
# Numeric readings may be negative and/or use thousands separators
|
||||
# (B5): "-5.0 degrees C", "1,234.5 RPM".
|
||||
_NUM = r"-?[0-9][0-9,]*(?:\.[0-9]+)?"
|
||||
|
||||
# Common unit patterns
|
||||
_TEMP_RE = re.compile(r"(?P<val>[0-9.]+)\s*degrees?\s*C", re.IGNORECASE)
|
||||
_FAN_RE = re.compile(r"(?P<val>[0-9.]+)\s*RPM", re.IGNORECASE)
|
||||
_VOLT_RE = re.compile(r"(?P<val>[0-9.]+)\s*Volts?", re.IGNORECASE)
|
||||
_WATT_RE = re.compile(r"(?P<val>[0-9.]+)\s*Watts?", re.IGNORECASE)
|
||||
_TEMP_RE = re.compile(r"(?P<val>" + _NUM + r")\s*degrees?\s*C", re.IGNORECASE)
|
||||
_FAN_RE = re.compile(r"(?P<val>" + _NUM + r")\s*RPM", re.IGNORECASE)
|
||||
_VOLT_RE = re.compile(r"(?P<val>" + _NUM + r")\s*Volts?", re.IGNORECASE)
|
||||
_WATT_RE = re.compile(r"(?P<val>" + _NUM + r")\s*Watts?", re.IGNORECASE)
|
||||
|
||||
# SDR status tokens (B7). Beyond the original ok/ns/nc/nr/nd/na this covers
|
||||
# disabled sensors and critical (cr) readings seen on real BMCs.
|
||||
_STATUS_TOKENS = {
|
||||
"ok", "ns", "nc", "nr", "nd", "na",
|
||||
"disabled", "cr", "lnr", "lcr", "unc", "unr",
|
||||
}
|
||||
|
||||
|
||||
def _classify_metric(name: str, value_str: str) -> Optional[tuple[str, float]]:
|
||||
"""Return (metric_type, numeric_value) or None if unrecognised."""
|
||||
def _to_float(val_str: str) -> float:
|
||||
"""Parse a numeric reading, tolerating thousands separators."""
|
||||
return float(val_str.replace(",", "").strip())
|
||||
|
||||
|
||||
def _classify_metric(
|
||||
name: str, value_str: str, status: str = "ok"
|
||||
) -> Optional[tuple[str, float]]:
|
||||
"""Return (metric_type, numeric_value) or None if unrecognised.
|
||||
|
||||
``status`` is the SDR status segment (B6): status-class sensors map
|
||||
``ok`` → 1.0 and anything else → 0.0 instead of always reporting 1.0.
|
||||
"""
|
||||
status_ok = (status or "").strip().lower() == "ok"
|
||||
|
||||
# Temperature
|
||||
m = _TEMP_RE.search(value_str)
|
||||
if m:
|
||||
return ("temperature", float(m.group("val")))
|
||||
return ("temperature", _to_float(m.group("val")))
|
||||
|
||||
# Fan speed
|
||||
m = _FAN_RE.search(value_str)
|
||||
if m:
|
||||
return ("fan", float(m.group("val")))
|
||||
return ("fan", _to_float(m.group("val")))
|
||||
|
||||
# Voltage
|
||||
m = _VOLT_RE.search(value_str)
|
||||
if m:
|
||||
return ("voltage", float(m.group("val")))
|
||||
return ("voltage", _to_float(m.group("val")))
|
||||
|
||||
# Power (Watts)
|
||||
m = _WATT_RE.search(value_str)
|
||||
if m:
|
||||
return ("power", float(m.group("val")))
|
||||
return ("power", _to_float(m.group("val")))
|
||||
|
||||
# Heuristic: name-based fallback for status / discrete sensors
|
||||
name_lower = name.lower()
|
||||
if "power" in name_lower and ("supply" in name_lower or "unit" in name_lower):
|
||||
# Hex or discrete value — map 0x00 → 0 (off), anything else → 1 (on)
|
||||
try:
|
||||
val = float(int(value_str.strip(), 0))
|
||||
except ValueError:
|
||||
val = 0.0
|
||||
return ("power", val)
|
||||
# Discrete power-supply sensor (e.g. "0x00", "Presence detected"):
|
||||
# this is presence/status information, not a wattage reading, so it
|
||||
# belongs to the "status" class (B6) and must not pollute power stats.
|
||||
return ("status", 1.0 if status_ok else 0.0)
|
||||
|
||||
if "status" in name_lower:
|
||||
# Discrete — just store 1 for ok / 0 for anything else
|
||||
return ("status", 1.0)
|
||||
# Discrete — 1 for ok, 0 for anything else
|
||||
return ("status", 1.0 if status_ok else 0.0)
|
||||
|
||||
return None
|
||||
|
||||
@@ -85,6 +118,11 @@ def parse_sdr_output(output: str) -> list[dict[str, Any]]:
|
||||
Legacy: CPU1 Temp | 45 degrees C | ok
|
||||
Newer: Temp | 01h | ok | 3.1 | 37 degrees C
|
||||
|
||||
Parsing is driven by segment count (B7): 5 segments = new format
|
||||
(``name|id|status|entity|value``), 3 segments = legacy format
|
||||
(``name|value|status``). Anything else falls back to locating a known
|
||||
status token instead of misreading the sensor id as the value.
|
||||
|
||||
Returns a list of dicts:
|
||||
{name, value_str, status, metric_type, numeric_value}
|
||||
"""
|
||||
@@ -93,36 +131,38 @@ def parse_sdr_output(output: str) -> list[dict[str, Any]]:
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
|
||||
parts = [p.strip() for p in line.split('|')]
|
||||
if len(parts) < 3:
|
||||
continue
|
||||
|
||||
# Find status token (ok, ns, nc, nr, nd, na)
|
||||
status_idx = None
|
||||
for i, p in enumerate(parts):
|
||||
if p in ('ok', 'ns', 'nc', 'nr', 'nd', 'na'):
|
||||
status_idx = i
|
||||
break
|
||||
|
||||
if status_idx and status_idx < len(parts) - 1:
|
||||
|
||||
parts = [p.strip() for p in line.split("|")]
|
||||
|
||||
if len(parts) == 5:
|
||||
# New format: name | id | status | entity | value
|
||||
name, status, value_str = parts[0], parts[2], parts[4]
|
||||
elif len(parts) == 3:
|
||||
# Legacy format: name | value | status
|
||||
name, value_str, status = parts[0], parts[1], parts[2]
|
||||
elif len(parts) >= 4:
|
||||
# Unknown shape — find a known status token; never silently
|
||||
# treat the sensor id (parts[1]) as the value.
|
||||
status_idx = None
|
||||
for i, p in enumerate(parts):
|
||||
if p.lower() in _STATUS_TOKENS:
|
||||
status_idx = i
|
||||
break
|
||||
if status_idx is None:
|
||||
continue
|
||||
name = parts[0]
|
||||
status = parts[status_idx]
|
||||
# Take the last part as the actual sensor value
|
||||
value_str = parts[-1].strip()
|
||||
value_str = parts[-1]
|
||||
else:
|
||||
# Legacy format: name | value | status
|
||||
name = parts[0]
|
||||
value_str = parts[1]
|
||||
status = parts[2]
|
||||
|
||||
continue
|
||||
|
||||
entry: dict[str, Any] = {
|
||||
"name": name,
|
||||
"value_str": value_str,
|
||||
"status": status,
|
||||
}
|
||||
|
||||
classified = _classify_metric(name, value_str)
|
||||
classified = _classify_metric(name, value_str, status)
|
||||
if classified:
|
||||
metric_type, numeric_value = classified
|
||||
entry["metric_type"] = metric_type
|
||||
@@ -142,75 +182,177 @@ def parse_sensor_output(output: str) -> list[dict[str, Any]]:
|
||||
return parse_sdr_output(output)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Component parsing helpers (B8/B9)
|
||||
#
|
||||
# component_parser.py drops legacy 3-segment SDR lines (it requires >= 4
|
||||
# segments) and has no de-duplication / status-based presence for
|
||||
# drive/fan/PSU. That module is owned by another fix track, so the
|
||||
# collector normalises the raw output before handing it over and repairs
|
||||
# the result afterwards instead of modifying it.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
_DRIVE_RE = re.compile(r"^Drive\s+\d+$", re.IGNORECASE)
|
||||
|
||||
|
||||
def _expand_legacy_sdr_lines(raw: str) -> str:
|
||||
"""Rewrite legacy 3-segment SDR lines into 5-segment form.
|
||||
|
||||
``Name | value | status`` becomes ``Name | -- | status | -- | value``
|
||||
so downstream component parsing (which needs >= 4 segments, with the
|
||||
status at index 2 and the value last) sees the legacy rows too (B8).
|
||||
"""
|
||||
out_lines = []
|
||||
for line in raw.splitlines():
|
||||
parts = [p.strip() for p in line.split("|")]
|
||||
if len(parts) == 3:
|
||||
out_lines.append(f"{parts[0]} | -- | {parts[2]} | -- | {parts[1]}")
|
||||
else:
|
||||
out_lines.append(line)
|
||||
return "\n".join(out_lines)
|
||||
|
||||
|
||||
def _fixup_components(raw: str, components: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Repair drive/fan/PSU component results (B9).
|
||||
|
||||
- Adds ``Drive N`` sensors the component parser misses (its drive
|
||||
matcher requires an underscore in the name).
|
||||
- De-duplicates drive/fan/PSU items by sensor name (``seen`` set).
|
||||
- Judges PSU / drive presence from the SDR ``status`` segment
|
||||
(``ok`` = present) instead of the value field, which for 5-segment
|
||||
rows holds a wattage like ``132 Watts`` and never contains
|
||||
"presence detected".
|
||||
"""
|
||||
status_by_name: dict[str, str] = {}
|
||||
for entry in parse_sdr_output(raw):
|
||||
status_by_name.setdefault(entry["name"], entry["status"])
|
||||
|
||||
drive_cat = components.setdefault("drive", {"total": 0, "present": 0, "items": []})
|
||||
existing_drive = {item.get("name") for item in drive_cat.get("items", [])}
|
||||
for line in raw.splitlines():
|
||||
parts = [p.strip() for p in line.split("|")]
|
||||
if not parts:
|
||||
continue
|
||||
if _DRIVE_RE.match(parts[0]) and parts[0] not in existing_drive:
|
||||
status = status_by_name.get(parts[0], "")
|
||||
drive_cat["items"].append(
|
||||
{
|
||||
"name": parts[0],
|
||||
"present": status.lower() == "ok",
|
||||
"value": parts[-1] if len(parts) > 1 else "",
|
||||
}
|
||||
)
|
||||
existing_drive.add(parts[0])
|
||||
|
||||
for category in ("drive", "fan", "power_supply"):
|
||||
cat = components.get(category)
|
||||
if not cat:
|
||||
continue
|
||||
seen: set[str] = set()
|
||||
items: list[dict[str, Any]] = []
|
||||
for item in cat.get("items", []):
|
||||
name = item.get("name")
|
||||
if name in seen:
|
||||
continue
|
||||
seen.add(name)
|
||||
items.append(item)
|
||||
cat["items"] = items
|
||||
if category in ("drive", "power_supply"):
|
||||
for item in items:
|
||||
status = status_by_name.get(item.get("name", ""))
|
||||
if status is not None:
|
||||
item["present"] = status.lower() == "ok"
|
||||
cat["total"] = len(items)
|
||||
cat["present"] = sum(1 for i in items if i.get("present"))
|
||||
|
||||
return components
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Collection logic
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
async def _run_ipmitool(host: str, port: int, username: str, password: str, interface: str = "lanplus") -> str:
|
||||
"""Execute ipmitool sdr elist and return raw stdout.
|
||||
|
||||
DEPRECATED: Use _run_ipmitool_with_auth instead for automatic auth retry.
|
||||
This function is kept for backward compatibility.
|
||||
"""
|
||||
cmd = [
|
||||
"ipmitool",
|
||||
"-I", interface,
|
||||
"-H", host,
|
||||
"-U", username,
|
||||
"-P", password,
|
||||
"-p", str(port),
|
||||
"-N", "5",
|
||||
"-R", "1",
|
||||
"sdr", "elist",
|
||||
]
|
||||
logger.debug("Running: %s", " ".join(cmd[:-5]) + " *** *** " + " ".join(cmd[-4:]))
|
||||
proc = await asyncio.create_subprocess_exec(
|
||||
*cmd,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.PIPE,
|
||||
)
|
||||
stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=30)
|
||||
if proc.returncode != 0:
|
||||
err = stderr.decode(errors="replace").strip()
|
||||
raise RuntimeError(f"ipmitool exited with code {proc.returncode}: {err}")
|
||||
return stdout.decode(errors="replace")
|
||||
# Single ipmitool call timeout (B4): 30s -> 10s so one slow BMC cannot
|
||||
# stall a whole collection round.
|
||||
IPMI_TIMEOUT_SECONDS = 10
|
||||
|
||||
# Authentication attempts (O6/H2). Only these three combinations are tried:
|
||||
# lanplus default, lanplus -C 3, lanplus -A MD5. The no-auth (-C 0) and
|
||||
# IPMI 1.5 (lan interface) attempts were removed — they downgrade security
|
||||
# and are handled by the parallel security fix's scope for hosts.py.
|
||||
IPMI_AUTH_ATTEMPTS: list[tuple[str, Optional[list[str]]]] = [
|
||||
("lanplus", None),
|
||||
("lanplus", ["-C", "3"]),
|
||||
("lanplus", ["-A", "MD5"]),
|
||||
]
|
||||
|
||||
|
||||
async def _run_ipmitool_with_auth(
|
||||
host: str, port: int, username: str, password: str,
|
||||
host: str, port: int, username: str, password: str,
|
||||
interface: str = "lanplus", extra_args: list = None
|
||||
) -> str:
|
||||
"""Execute ipmitool sdr elist with specified arguments."""
|
||||
cmd = [
|
||||
"ipmitool",
|
||||
"-I", interface,
|
||||
"-H", host,
|
||||
"-U", username,
|
||||
"-P", password,
|
||||
]
|
||||
|
||||
# Add extra arguments (e.g., -C 3, -A MD5, etc.)
|
||||
if extra_args:
|
||||
cmd.extend(extra_args)
|
||||
|
||||
cmd.extend([
|
||||
"-p", str(port),
|
||||
"-N", "5",
|
||||
"-R", "1",
|
||||
"sdr", "elist",
|
||||
])
|
||||
|
||||
logger.debug("Running: %s", " ".join(cmd[:-5]) + " *** *** " + " ".join(cmd[-4:]))
|
||||
proc = await asyncio.create_subprocess_exec(
|
||||
*cmd,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.PIPE,
|
||||
)
|
||||
stdout, stderr = await asyncio.wait_for(proc.communicate(), timeout=30)
|
||||
if proc.returncode != 0:
|
||||
err = stderr.decode(errors="replace").strip()
|
||||
raise RuntimeError(f"ipmitool exited with code {proc.returncode}: {err}")
|
||||
return stdout.decode(errors="replace")
|
||||
"""Execute ipmitool sdr elist with specified arguments.
|
||||
|
||||
The BMC password is passed via a temporary password file (``-f``,
|
||||
mode 0600, deleted immediately after use — H-03) instead of the
|
||||
``-P`` command-line argument, so it never appears in the process list
|
||||
or in logs. Only the host, interface and command name are logged.
|
||||
"""
|
||||
pwd_file: Optional[str] = None
|
||||
proc = None
|
||||
try:
|
||||
# Write the password to a 0600 temp file for `ipmitool -f`.
|
||||
fd, pwd_file = tempfile.mkstemp(prefix="ipmi-pwd-", dir="/tmp")
|
||||
os.fchmod(fd, 0o600)
|
||||
with os.fdopen(fd, "w", encoding="utf-8") as f:
|
||||
f.write(password)
|
||||
|
||||
cmd = [
|
||||
"ipmitool",
|
||||
"-I", interface,
|
||||
"-H", host,
|
||||
"-U", username,
|
||||
"-f", pwd_file,
|
||||
]
|
||||
|
||||
# Add extra arguments (e.g., -C 3, -A MD5, etc.)
|
||||
if extra_args:
|
||||
cmd.extend(extra_args)
|
||||
|
||||
cmd.extend([
|
||||
"-p", str(port),
|
||||
"-N", "5",
|
||||
"-R", "1",
|
||||
"sdr", "elist",
|
||||
])
|
||||
|
||||
logger.debug("Running ipmitool sdr elist on %s via %s", host, interface)
|
||||
proc = await asyncio.create_subprocess_exec(
|
||||
*cmd,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.PIPE,
|
||||
)
|
||||
try:
|
||||
stdout, stderr = await asyncio.wait_for(
|
||||
proc.communicate(), timeout=IPMI_TIMEOUT_SECONDS
|
||||
)
|
||||
except (asyncio.TimeoutError, TimeoutError):
|
||||
# B3: kill + reap the child so no zombie ipmitool is left behind.
|
||||
proc.kill()
|
||||
await proc.wait()
|
||||
raise TimeoutError(
|
||||
f"ipmitool timed out after {IPMI_TIMEOUT_SECONDS}s on {host} via {interface}"
|
||||
)
|
||||
if proc.returncode != 0:
|
||||
err = stderr.decode(errors="replace").strip()
|
||||
raise RuntimeError(f"ipmitool exited with code {proc.returncode}: {err}")
|
||||
return stdout.decode(errors="replace")
|
||||
finally:
|
||||
if pwd_file:
|
||||
try:
|
||||
os.unlink(pwd_file)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
async def collect_host(host: Any) -> dict[str, Any]:
|
||||
@@ -228,45 +370,36 @@ async def collect_host(host: Any) -> dict[str, Any]:
|
||||
}
|
||||
|
||||
try:
|
||||
# Try different connection methods:
|
||||
# 1. lanplus (IPMI 2.0) - simplest, no extra args
|
||||
# 2. lanplus with -C 3 (IPMI 2.0, user auth)
|
||||
# 3. lanplus with -C 0 (IPMI 2.0, no auth)
|
||||
# 4. lanplus with -A MD5 (IPMI 1.5 with MD5)
|
||||
# 5. lan with -A MD5 (IPMI 1.5 fallback)
|
||||
# 6. lan (IPMI 1.5) - simplest fallback
|
||||
# Decrypt the stored BMC password (decrypt_value returns the
|
||||
# original value when decryption fails — historical plaintext compat).
|
||||
# The password is never logged.
|
||||
password = decrypt_value(host.ipmi_password)
|
||||
|
||||
raw = None
|
||||
last_error = None
|
||||
|
||||
attempts = [
|
||||
["lanplus", None],
|
||||
["lanplus", ["-C", "3"]],
|
||||
["lanplus", ["-C", "0"]],
|
||||
["lanplus", ["-A", "MD5"]],
|
||||
["lan", ["-A", "MD5"]],
|
||||
["lan", None],
|
||||
]
|
||||
|
||||
for interface, extra_args in attempts:
|
||||
|
||||
for interface, extra_args in IPMI_AUTH_ATTEMPTS:
|
||||
try:
|
||||
raw = await _run_ipmitool_with_auth(
|
||||
host.ipmi_host, host.ipmi_port,
|
||||
host.ipmi_username, host.ipmi_password,
|
||||
host.ipmi_username, password,
|
||||
interface=interface,
|
||||
extra_args=extra_args,
|
||||
)
|
||||
if raw:
|
||||
auth_str = " ".join(extra_args) if extra_args else "default"
|
||||
logger.info("Successfully connected to %s using %s with %s",
|
||||
logger.info("Successfully connected to %s using %s with %s",
|
||||
host.name, interface, auth_str)
|
||||
break
|
||||
except RuntimeError as e:
|
||||
except (RuntimeError, TimeoutError, asyncio.TimeoutError) as e:
|
||||
# B3: a timeout (or any auth failure) falls through to the
|
||||
# next authentication combination instead of aborting.
|
||||
last_error = str(e)
|
||||
logger.debug("Failed with %s %s: %s", interface, extra_args, last_error)
|
||||
|
||||
|
||||
if not raw:
|
||||
raise RuntimeError(f"All connection attempts failed. Last error: {last_error}")
|
||||
|
||||
|
||||
parsed = parse_sdr_output(raw)
|
||||
result["raw_parsed"] = parsed
|
||||
|
||||
@@ -304,7 +437,8 @@ async def collect_host(host: Any) -> dict[str, Any]:
|
||||
|
||||
# Parse component presence and add to InfluxDB points
|
||||
from app.services.component_parser import parse_component_presence
|
||||
components = parse_component_presence(raw)
|
||||
components = parse_component_presence(_expand_legacy_sdr_lines(raw))
|
||||
components = _fixup_components(raw, components)
|
||||
for category in ["cpu", "memory", "pcie", "drive"]:
|
||||
cat_data = components.get(category, {})
|
||||
for item in cat_data.get("items", []):
|
||||
@@ -316,11 +450,19 @@ async def collect_host(host: Any) -> dict[str, Any]:
|
||||
"tags": {"category": category, "present": str(item.get("present", False))},
|
||||
})
|
||||
|
||||
# Write to InfluxDB in one batch
|
||||
# Write to InfluxDB in one batch (B14/B15: async thread offload,
|
||||
# and a failed write must not be reported as a success).
|
||||
if influx_points:
|
||||
influxdb_module.write_metrics_batch(influx_points)
|
||||
write_ok = await influxdb_module.awrite_metrics_batch(influx_points)
|
||||
if not write_ok:
|
||||
result["error"] = "Failed to write metrics to InfluxDB"
|
||||
logger.error(
|
||||
"InfluxDB write failed for host %s (%s); not updating last_seen",
|
||||
host.id, host.name,
|
||||
)
|
||||
return result
|
||||
|
||||
# Update last_seen timestamp
|
||||
# Update last_seen timestamp (only reached when the write succeeded)
|
||||
from app.database import async_session
|
||||
from app.models import Host as HostModel
|
||||
from sqlalchemy import update
|
||||
|
||||
@@ -15,14 +15,32 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
_scheduler: AsyncIOScheduler | None = None
|
||||
|
||||
# Concurrency cap for collection (B4/O4): at most 8 ipmitool subprocesses
|
||||
# at once, no matter how many hosts are enabled.
|
||||
_COLLECT_SEMAPHORE = asyncio.Semaphore(8)
|
||||
|
||||
# Failure backoff (B4/O4): a host that fails this many rounds in a row is
|
||||
# skipped for the next N rounds before being retried.
|
||||
BACKOFF_AFTER_FAILURES = 3
|
||||
BACKOFF_SKIP_ROUNDS = 5
|
||||
|
||||
_consecutive_failures: dict[int, int] = {}
|
||||
_skip_remaining: dict[int, int] = {}
|
||||
|
||||
|
||||
async def _collect_one(host: Host) -> dict[str, Any]:
|
||||
"""Collect one host under the global concurrency cap."""
|
||||
from app.services.ipmi_collector import collect_host
|
||||
|
||||
async with _COLLECT_SEMAPHORE:
|
||||
return await collect_host(host)
|
||||
|
||||
|
||||
async def collect_all_hosts() -> list[dict[str, Any]]:
|
||||
"""Collect IPMI data for every enabled host.
|
||||
|
||||
Returns a list of collection results (one per host).
|
||||
"""
|
||||
from app.services.ipmi_collector import collect_host
|
||||
|
||||
results: list[dict[str, Any]] = []
|
||||
|
||||
async with async_session() as session:
|
||||
@@ -36,8 +54,29 @@ async def collect_all_hosts() -> list[dict[str, Any]]:
|
||||
|
||||
logger.info("Collecting IPMI data from %d host(s)", len(hosts))
|
||||
|
||||
# Run collections concurrently
|
||||
tasks = [collect_host(h) for h in hosts]
|
||||
# Backoff: skip hosts that failed repeatedly, counting the rounds down.
|
||||
due_hosts: list[Host] = []
|
||||
for host in hosts:
|
||||
remaining = _skip_remaining.get(host.id, 0)
|
||||
if remaining > 0:
|
||||
_skip_remaining[host.id] = remaining - 1
|
||||
logger.debug(
|
||||
"Skipping host %s (%s): backing off for %d more round(s)",
|
||||
host.id, host.name, remaining - 1,
|
||||
)
|
||||
results.append({
|
||||
"host_id": host.id,
|
||||
"host_name": host.name,
|
||||
"success": False,
|
||||
"skipped": True,
|
||||
"error": f"Backing off after repeated failures ({remaining - 1} round(s) left)",
|
||||
"metrics": [],
|
||||
})
|
||||
else:
|
||||
due_hosts.append(host)
|
||||
|
||||
# Run collections concurrently, capped by the semaphore (B4/O4)
|
||||
tasks = [_collect_one(h) for h in due_hosts]
|
||||
for coro in asyncio.as_completed(tasks):
|
||||
try:
|
||||
res = await coro
|
||||
@@ -45,6 +84,27 @@ async def collect_all_hosts() -> list[dict[str, Any]]:
|
||||
except Exception:
|
||||
logger.exception("Unexpected error during collection task")
|
||||
|
||||
# Update failure counters / backoff state from this round's outcomes.
|
||||
for res in results:
|
||||
if res.get("skipped"):
|
||||
continue
|
||||
hid = res.get("host_id")
|
||||
if hid is None:
|
||||
continue
|
||||
if res.get("success"):
|
||||
_consecutive_failures.pop(hid, None)
|
||||
else:
|
||||
failures = _consecutive_failures.get(hid, 0) + 1
|
||||
if failures >= BACKOFF_AFTER_FAILURES:
|
||||
_skip_remaining[hid] = BACKOFF_SKIP_ROUNDS
|
||||
_consecutive_failures.pop(hid, None)
|
||||
logger.warning(
|
||||
"Host %s failed %d consecutive rounds; skipping next %d round(s)",
|
||||
hid, failures, BACKOFF_SKIP_ROUNDS,
|
||||
)
|
||||
else:
|
||||
_consecutive_failures[hid] = failures
|
||||
|
||||
# After collection, run alert checks
|
||||
try:
|
||||
from app.services.alert_engine import check_alerts
|
||||
@@ -69,6 +129,10 @@ def start_scheduler() -> AsyncIOScheduler:
|
||||
id="collect_all_hosts",
|
||||
replace_existing=True,
|
||||
max_instances=1,
|
||||
# B4: coalesce missed runs into one and drop runs that are more
|
||||
# than 30s late instead of letting them pile up behind a slow host.
|
||||
coalesce=True,
|
||||
misfire_grace_time=30,
|
||||
)
|
||||
_scheduler.start()
|
||||
logger.info(
|
||||
|
||||
@@ -0,0 +1,122 @@
|
||||
"""SSRF guards for outbound requests whose target is user-controlled.
|
||||
|
||||
Three user-controlled destinations exist in this app:
|
||||
|
||||
* ``ipmi_host`` — the BMC to poll. BMCs live on the internal network, so
|
||||
RFC1918 private addresses and hostnames are legitimate; only the
|
||||
dangerous classes are rejected (loopback, link-local incl. the
|
||||
169.254.169.254 cloud-metadata address, multicast, reserved/unspecified).
|
||||
* Feishu webhook URL — only the official Feishu/Lark open-platform hosts
|
||||
over HTTPS.
|
||||
* SMTP host — same dangerous-address rejection as the IPMI host (an
|
||||
internal relay on a private address is legitimate).
|
||||
|
||||
All validators raise :class:`ValueError` on rejection; API callers turn that
|
||||
into HTTP 400. Hostnames are validated syntactically only (no DNS lookup
|
||||
here, so no resolution-time rebinding check is claimed).
|
||||
"""
|
||||
|
||||
import ipaddress
|
||||
import re
|
||||
from urllib.parse import urlparse
|
||||
|
||||
# Cloud metadata endpoint (also covered by the link-local check, listed
|
||||
# explicitly so the intent is obvious).
|
||||
METADATA_IP = "169.254.169.254"
|
||||
|
||||
# Official Feishu / Lark open-platform webhook hosts.
|
||||
FEISHU_WEBHOOK_HOSTS = ("open.feishu.cn", "open.larksuite.com")
|
||||
|
||||
_HOSTNAME_RE = re.compile(
|
||||
r"^(?=.{1,253}$)(?!-)[A-Za-z0-9-]{1,63}(?<!-)"
|
||||
r"(?:\.(?!-)[A-Za-z0-9-]{1,63}(?<!-))*$"
|
||||
)
|
||||
|
||||
|
||||
def _reject_dangerous_ip(ip: ipaddress.IPv4Address | ipaddress.IPv6Address, what: str) -> None:
|
||||
"""Reject loopback / link-local / multicast / reserved / unspecified IPs."""
|
||||
if ip.is_loopback:
|
||||
raise ValueError(f"{what} must not be a loopback address: {ip}")
|
||||
if ip.is_link_local:
|
||||
raise ValueError(f"{what} must not be a link-local address: {ip}")
|
||||
if ip.is_multicast:
|
||||
raise ValueError(f"{what} must not be a multicast address: {ip}")
|
||||
if ip.is_reserved or ip.is_unspecified:
|
||||
raise ValueError(f"{what} must not be a reserved or unspecified address: {ip}")
|
||||
if str(ip) == METADATA_IP:
|
||||
raise ValueError(f"{what} must not be the cloud metadata address: {ip}")
|
||||
|
||||
|
||||
def _validate_hostname(host: str, what: str) -> None:
|
||||
lowered = host.lower()
|
||||
if lowered in ("localhost",) or lowered.endswith(".localhost"):
|
||||
raise ValueError(f"{what} must not be localhost")
|
||||
if not _HOSTNAME_RE.match(host):
|
||||
raise ValueError(f"{what} is not a valid hostname or IP address: {host!r}")
|
||||
|
||||
|
||||
def validate_ipmi_host(host: str) -> str:
|
||||
"""Validate an IPMI target host.
|
||||
|
||||
Allows RFC1918 private addresses and legitimate hostnames; rejects
|
||||
loopback, link-local (incl. 169.254.169.254), multicast and reserved
|
||||
addresses. Returns the stripped host on success.
|
||||
"""
|
||||
if not host or not isinstance(host, str):
|
||||
raise ValueError("ipmi_host must not be empty")
|
||||
host = host.strip()
|
||||
if len(host) > 253:
|
||||
raise ValueError("ipmi_host is too long")
|
||||
try:
|
||||
ip = ipaddress.ip_address(host)
|
||||
except ValueError:
|
||||
_validate_hostname(host, "ipmi_host")
|
||||
return host
|
||||
_reject_dangerous_ip(ip, "ipmi_host")
|
||||
return host
|
||||
|
||||
|
||||
def validate_webhook_url(url: str) -> str:
|
||||
"""Validate a Feishu webhook URL.
|
||||
|
||||
Only ``https`` URLs on ``open.feishu.cn`` / ``open.larksuite.com`` (or
|
||||
their subdomains) are allowed, with no embedded credentials and no
|
||||
non-default port. Returns the URL on success.
|
||||
"""
|
||||
if not url or not isinstance(url, str):
|
||||
raise ValueError("webhook URL must not be empty")
|
||||
parsed = urlparse(url.strip())
|
||||
if parsed.scheme.lower() != "https":
|
||||
raise ValueError("webhook URL must use https")
|
||||
host = (parsed.hostname or "").lower()
|
||||
if not any(host == allowed or host.endswith("." + allowed)
|
||||
for allowed in FEISHU_WEBHOOK_HOSTS):
|
||||
raise ValueError(
|
||||
"webhook URL host must be open.feishu.cn or open.larksuite.com"
|
||||
)
|
||||
if parsed.username or parsed.password:
|
||||
raise ValueError("webhook URL must not contain credentials")
|
||||
if parsed.port is not None and parsed.port != 443:
|
||||
raise ValueError("webhook URL must not use a non-standard port")
|
||||
return url.strip()
|
||||
|
||||
|
||||
def validate_smtp_host(host: str) -> str:
|
||||
"""Validate an SMTP host.
|
||||
|
||||
Rejects loopback, link-local (incl. 169.254.169.254), multicast and
|
||||
reserved addresses; private-network relays and hostnames are allowed.
|
||||
Returns the stripped host on success.
|
||||
"""
|
||||
if not host or not isinstance(host, str):
|
||||
raise ValueError("smtp_host must not be empty")
|
||||
host = host.strip()
|
||||
if len(host) > 253:
|
||||
raise ValueError("smtp_host is too long")
|
||||
try:
|
||||
ip = ipaddress.ip_address(host)
|
||||
except ValueError:
|
||||
_validate_hostname(host, "smtp_host")
|
||||
return host
|
||||
_reject_dangerous_ip(ip, "smtp_host")
|
||||
return host
|
||||
@@ -0,0 +1,84 @@
|
||||
"""Shared helpers for the IPMI Monitor backend.
|
||||
|
||||
Small pure functions used by more than one module live here so the logic
|
||||
is defined exactly once (parse_dt used to be copy-pasted in monitor.py,
|
||||
total-power fallback used to be re-implemented in three places).
|
||||
"""
|
||||
|
||||
import re
|
||||
from datetime import datetime, timezone
|
||||
from typing import Any, Optional
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Datetime parsing (B11)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def parse_dt(dt_str: Optional[str]) -> Optional[datetime]:
|
||||
"""Parse an ISO-8601 datetime string into a naive UTC datetime.
|
||||
|
||||
Returns ``None`` for empty input. Raises :class:`ValueError` for an
|
||||
unparseable string — callers should translate that into a 400 response
|
||||
instead of letting it become a 500.
|
||||
"""
|
||||
if not dt_str:
|
||||
return None
|
||||
dt_str = dt_str.strip().replace("Z", "+00:00")
|
||||
dt = datetime.fromisoformat(dt_str) # raises ValueError on bad input
|
||||
if dt.tzinfo is not None:
|
||||
return dt.astimezone(timezone.utc).replace(tzinfo=None)
|
||||
return dt
|
||||
|
||||
|
||||
def utc_now_naive() -> datetime:
|
||||
"""Current time as a naive UTC datetime (matches InfluxDB query inputs)."""
|
||||
return datetime.now(timezone.utc).replace(tzinfo=None)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Total power resolution (O5 / B30)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# Sensors whose name indicates an overall system/host power reading.
|
||||
NATIVE_TOTAL_POWER_NAMES = ("total_power", "power", "pwr consumption")
|
||||
|
||||
_PSU_PIN_RE = re.compile(r"^PSU\d+_PIN$", re.IGNORECASE)
|
||||
|
||||
|
||||
def resolve_total_power(power_metrics: list[dict[str, Any]]) -> Optional[float]:
|
||||
"""Resolve a host's total power from its ``power`` metrics.
|
||||
|
||||
Priority:
|
||||
1. A native system-wide reading (``Total_Power`` / ``Power`` /
|
||||
``Pwr Consumption``). A sensor that *exists* is authoritative —
|
||||
a reading of ``0`` is a valid reading, not "missing".
|
||||
2. Otherwise the sum of all ``PSU{N}_PIN`` (PSU input power), which
|
||||
covers hosts whose BMC exposes no system-wide sensor (e.g. Gooxi12).
|
||||
|
||||
Returns ``None`` when no usable power metric exists at all.
|
||||
"""
|
||||
native_value: Optional[float] = None
|
||||
psu_pin_sum = 0.0
|
||||
psu_pin_seen = False
|
||||
|
||||
for m in power_metrics or []:
|
||||
name = (m.get("metric_name") or "").strip()
|
||||
raw_value = m.get("value")
|
||||
if raw_value is None:
|
||||
continue
|
||||
try:
|
||||
value = float(raw_value)
|
||||
except (TypeError, ValueError):
|
||||
continue
|
||||
if name.lower() in NATIVE_TOTAL_POWER_NAMES:
|
||||
# First native reading wins; 0.0 counts as a real reading.
|
||||
if native_value is None:
|
||||
native_value = value
|
||||
elif _PSU_PIN_RE.match(name):
|
||||
psu_pin_sum += value
|
||||
psu_pin_seen = True
|
||||
|
||||
if native_value is not None:
|
||||
return native_value
|
||||
if psu_pin_seen:
|
||||
return psu_pin_sum
|
||||
return None
|
||||
+20
-14
@@ -1,14 +1,20 @@
|
||||
fastapi>=0.104.1
|
||||
uvicorn>=0.24.0
|
||||
sqlalchemy>=2.0.23
|
||||
aiosqlite>=0.19.0
|
||||
influxdb-client>=1.38.0
|
||||
httpx>=0.25.2
|
||||
pydantic>=2.5.2
|
||||
pydantic-settings>=2.1.0
|
||||
apscheduler>=3.10.4
|
||||
jinja2>=3.1.2
|
||||
python-multipart>=0.0.6
|
||||
aiofiles>=23.2.1
|
||||
python-jose[cryptography]>=0.3.3
|
||||
passlib[bcrypt]>=1.7.4
|
||||
# Pinned exact versions (Python 3.11 compatible).
|
||||
# PyJWT replaces the unmaintained python-jose (see H-07).
|
||||
# cryptography is required by app/crypto.py (Fernet credential encryption).
|
||||
# bcrypt is pinned to 4.0.1: passlib 1.7.4 is incompatible with bcrypt >= 4.1.
|
||||
fastapi==0.115.12
|
||||
uvicorn==0.34.0
|
||||
sqlalchemy==2.0.36
|
||||
aiosqlite==0.20.0
|
||||
influxdb-client==1.49.0
|
||||
httpx==0.28.1
|
||||
pydantic==2.10.6
|
||||
pydantic-settings==2.7.1
|
||||
apscheduler==3.11.0
|
||||
jinja2==3.1.6
|
||||
python-multipart==0.0.20
|
||||
aiofiles==24.1.0
|
||||
PyJWT==2.10.1
|
||||
passlib[bcrypt]==1.7.4
|
||||
bcrypt==4.0.1
|
||||
cryptography==44.0.3
|
||||
|
||||
@@ -0,0 +1,251 @@
|
||||
"""Tests for alert recovery notifications (``app.services.alert_engine``).
|
||||
|
||||
- A recovery notification is sent only when an alert was actually fired
|
||||
(``_alert_active`` set); a transient breach that never satisfied the
|
||||
duration just clears its timer silently.
|
||||
- Dirty metric data (``None`` / non-numeric values) must not abort the
|
||||
whole check round: the bad point is skipped and the remaining points
|
||||
are still evaluated.
|
||||
|
||||
Dependency note: this machine has no project third-party packages
|
||||
installed (no ``sqlalchemy`` / ``pydantic`` / ``influxdb_client``), and
|
||||
``alert_engine`` imports them at module level. The missing modules are
|
||||
stubbed in ``sys.modules`` before the import — the same minimal-surface
|
||||
approach as ``test_sdr_parser.py`` — so only the pure rule-evaluation
|
||||
logic under test is exercised. InfluxDB access and the notification
|
||||
dispatchers are replaced with recording fakes; no database, network or
|
||||
SMTP/Feishu call is made.
|
||||
"""
|
||||
|
||||
import sys
|
||||
import types
|
||||
import unittest
|
||||
from datetime import datetime, timedelta
|
||||
from types import SimpleNamespace
|
||||
|
||||
|
||||
def _install_stubs() -> None:
|
||||
"""Stub the heavy third-party / app modules ``alert_engine`` imports."""
|
||||
if "sqlalchemy" not in sys.modules:
|
||||
sqlalchemy_mod = types.ModuleType("sqlalchemy")
|
||||
sqlalchemy_mod.select = lambda *args, **kwargs: None
|
||||
sys.modules["sqlalchemy"] = sqlalchemy_mod
|
||||
if "sqlalchemy.ext" not in sys.modules:
|
||||
ext_mod = types.ModuleType("sqlalchemy.ext")
|
||||
sys.modules["sqlalchemy.ext"] = ext_mod
|
||||
sys.modules["sqlalchemy"].ext = ext_mod
|
||||
if "sqlalchemy.ext.asyncio" not in sys.modules:
|
||||
asyncio_mod = types.ModuleType("sqlalchemy.ext.asyncio")
|
||||
|
||||
class AsyncSession: # noqa: D101 - stub only
|
||||
pass
|
||||
|
||||
asyncio_mod.AsyncSession = AsyncSession
|
||||
sys.modules["sqlalchemy.ext.asyncio"] = asyncio_mod
|
||||
sys.modules["sqlalchemy.ext"].asyncio = asyncio_mod
|
||||
if "app.influxdb" not in sys.modules:
|
||||
influx_mod = types.ModuleType("app.influxdb")
|
||||
|
||||
async def _aquery_latest(host_id, metric_type):
|
||||
raise NotImplementedError("stubbed; patch per test")
|
||||
|
||||
influx_mod.aquery_latest = _aquery_latest
|
||||
sys.modules["app.influxdb"] = influx_mod
|
||||
if "app.database" not in sys.modules:
|
||||
db_mod = types.ModuleType("app.database")
|
||||
db_mod.async_session = None
|
||||
sys.modules["app.database"] = db_mod
|
||||
if "app.models" not in sys.modules:
|
||||
models_mod = types.ModuleType("app.models")
|
||||
|
||||
class _ModelStub: # noqa: D101 - stub only
|
||||
pass
|
||||
|
||||
for _name in ("AlertHistory", "AlertRule", "Host", "SystemSetting"):
|
||||
setattr(models_mod, _name, _ModelStub)
|
||||
sys.modules["app.models"] = models_mod
|
||||
if "app.crypto" not in sys.modules:
|
||||
# alert_engine imports decrypt_value directly; the real module
|
||||
# needs ``cryptography`` (not installed here). These tests never
|
||||
# exercise decryption (dispatchers are patched out), so an
|
||||
# identity stand-in is sufficient.
|
||||
crypto_mod = types.ModuleType("app.crypto")
|
||||
crypto_mod.decrypt_value = lambda value: value
|
||||
sys.modules["app.crypto"] = crypto_mod
|
||||
|
||||
|
||||
_install_stubs()
|
||||
|
||||
from app.services import alert_engine
|
||||
|
||||
|
||||
def _make_rule(**overrides):
|
||||
"""A rule-like object: gt 80 on 'CPU Temp', duration 0 by default."""
|
||||
data = {
|
||||
"id": 1,
|
||||
"name": "High Temp",
|
||||
"host_id": None,
|
||||
"group_id": None,
|
||||
"metric_type": "temperature",
|
||||
"metric_name": "CPU Temp",
|
||||
"condition": "gt",
|
||||
"threshold": 80.0,
|
||||
"duration": 0,
|
||||
"notify_email": True,
|
||||
"notify_feishu": False,
|
||||
"email_addresses": "a@example.com",
|
||||
}
|
||||
data.update(overrides)
|
||||
return SimpleNamespace(**data)
|
||||
|
||||
|
||||
class _FakeSession:
|
||||
"""Minimal async session: only ``get`` is used on the recovery path."""
|
||||
|
||||
def __init__(self, host):
|
||||
self._host = host
|
||||
|
||||
async def get(self, model, pk):
|
||||
return self._host
|
||||
|
||||
|
||||
class AlertRecoveryTest(unittest.IsolatedAsyncioTestCase):
|
||||
def setUp(self):
|
||||
self.host = SimpleNamespace(id=10, name="host-10")
|
||||
self.session = _FakeSession(self.host)
|
||||
self.rule = _make_rule()
|
||||
self.key = (self.rule.id, self.host.id)
|
||||
self.points = []
|
||||
self.history_calls = []
|
||||
self.recovery_calls = []
|
||||
|
||||
# Fresh in-process alert state for every test.
|
||||
alert_engine._first_breach.clear()
|
||||
alert_engine._alert_active.clear()
|
||||
alert_engine._last_notify.clear()
|
||||
|
||||
# Patch the InfluxDB query and the notification dispatchers.
|
||||
self._orig_latest = alert_engine.influxdb_module.aquery_latest
|
||||
self._orig_history = alert_engine._create_alert_history
|
||||
self._orig_recovery = alert_engine._create_recovery_notification
|
||||
|
||||
async def fake_latest(hid, metric_type):
|
||||
return list(self.points)
|
||||
|
||||
async def record_history(rule, host, value):
|
||||
self.history_calls.append((rule, host, value))
|
||||
|
||||
async def record_recovery(rule, host, value):
|
||||
self.recovery_calls.append((rule, host, value))
|
||||
|
||||
alert_engine.influxdb_module.aquery_latest = fake_latest
|
||||
alert_engine._create_alert_history = record_history
|
||||
alert_engine._create_recovery_notification = record_recovery
|
||||
|
||||
def tearDown(self):
|
||||
alert_engine.influxdb_module.aquery_latest = self._orig_latest
|
||||
alert_engine._create_alert_history = self._orig_history
|
||||
alert_engine._create_recovery_notification = self._orig_recovery
|
||||
alert_engine._first_breach.clear()
|
||||
alert_engine._alert_active.clear()
|
||||
alert_engine._last_notify.clear()
|
||||
|
||||
async def _check(self):
|
||||
await alert_engine._check_rule_for_host(
|
||||
self.session, self.rule, self.host.id
|
||||
)
|
||||
|
||||
def _mark_breaching(self, active: bool):
|
||||
"""Pretend a breach timer is running; optionally an alert fired."""
|
||||
alert_engine._first_breach[self.key] = datetime.now() - timedelta(
|
||||
seconds=60
|
||||
)
|
||||
if active:
|
||||
alert_engine._alert_active[self.key] = True
|
||||
|
||||
async def test_recovery_sent_when_alert_was_active(self):
|
||||
"""Active alert + value back to normal -> exactly one recovery."""
|
||||
self._mark_breaching(active=True)
|
||||
self.points = [{"metric_name": "CPU Temp", "value": 30.0}]
|
||||
await self._check()
|
||||
self.assertEqual(len(self.recovery_calls), 1)
|
||||
self.assertEqual(self.recovery_calls[0][2], 30.0)
|
||||
# State is cleared after recovery.
|
||||
self.assertNotIn(self.key, alert_engine._first_breach)
|
||||
self.assertNotIn(self.key, alert_engine._alert_active)
|
||||
|
||||
async def test_no_recovery_when_breach_never_fired(self):
|
||||
"""Transient breach (timer running, alert never fired) recovers
|
||||
silently: no recovery notification, timer just cleared."""
|
||||
self._mark_breaching(active=False)
|
||||
self.points = [{"metric_name": "CPU Temp", "value": 30.0}]
|
||||
await self._check()
|
||||
self.assertEqual(self.recovery_calls, [])
|
||||
self.assertNotIn(self.key, alert_engine._first_breach)
|
||||
|
||||
async def test_no_recovery_without_any_prior_breach(self):
|
||||
"""Normal value with no breach state at all -> nothing happens."""
|
||||
self.points = [{"metric_name": "CPU Temp", "value": 30.0}]
|
||||
await self._check()
|
||||
self.assertEqual(self.recovery_calls, [])
|
||||
self.assertEqual(self.history_calls, [])
|
||||
|
||||
async def test_none_value_does_not_interrupt_round(self):
|
||||
"""A None value is skipped; the following valid point in the same
|
||||
round is still evaluated and triggers the recovery."""
|
||||
self._mark_breaching(active=True)
|
||||
self.points = [
|
||||
{"metric_name": "CPU Temp", "value": None},
|
||||
{"metric_name": "CPU Temp", "value": 30.0},
|
||||
]
|
||||
await self._check()
|
||||
self.assertEqual(len(self.recovery_calls), 1)
|
||||
|
||||
async def test_none_value_alone_preserves_state(self):
|
||||
"""A round with only a None point neither recovers nor clears the
|
||||
active alert state."""
|
||||
self._mark_breaching(active=True)
|
||||
self.points = [{"metric_name": "CPU Temp", "value": None}]
|
||||
await self._check()
|
||||
self.assertEqual(self.recovery_calls, [])
|
||||
self.assertTrue(alert_engine._alert_active.get(self.key))
|
||||
|
||||
async def test_non_numeric_value_does_not_interrupt_round(self):
|
||||
"""A non-numeric value ('N/A') is skipped; the valid point after
|
||||
it in the same round still triggers the recovery."""
|
||||
self._mark_breaching(active=True)
|
||||
self.points = [
|
||||
{"metric_name": "CPU Temp", "value": "N/A"},
|
||||
{"metric_name": "CPU Temp", "value": 30.0},
|
||||
]
|
||||
await self._check()
|
||||
self.assertEqual(len(self.recovery_calls), 1)
|
||||
|
||||
async def test_non_numeric_value_alone_preserves_state(self):
|
||||
self._mark_breaching(active=True)
|
||||
self.points = [{"metric_name": "CPU Temp", "value": "not-a-number"}]
|
||||
await self._check()
|
||||
self.assertEqual(self.recovery_calls, [])
|
||||
self.assertTrue(alert_engine._alert_active.get(self.key))
|
||||
|
||||
async def test_unrelated_metric_name_ignored(self):
|
||||
"""Points for other sensors do not affect this rule's state."""
|
||||
self._mark_breaching(active=True)
|
||||
self.points = [{"metric_name": "FAN1", "value": 100.0}]
|
||||
await self._check()
|
||||
self.assertEqual(self.recovery_calls, [])
|
||||
self.assertTrue(alert_engine._alert_active.get(self.key))
|
||||
|
||||
async def test_alert_fires_when_duration_satisfied(self):
|
||||
"""Sanity check on the firing side: a breaching value with
|
||||
duration 0 fires the alert and marks it active (so a later
|
||||
recovery has something to recover from)."""
|
||||
self.points = [{"metric_name": "CPU Temp", "value": 90.0}]
|
||||
await self._check()
|
||||
self.assertEqual(len(self.history_calls), 1)
|
||||
self.assertTrue(alert_engine._alert_active.get(self.key))
|
||||
self.assertEqual(self.recovery_calls, [])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,87 @@
|
||||
"""Tests for ``app.utils.parse_dt``.
|
||||
|
||||
``parse_dt`` is the shared datetime parser behind the monitor history /
|
||||
export endpoints: valid input becomes a naive UTC datetime, empty input
|
||||
is ``None`` (caller applies its default range), and invalid input raises
|
||||
``ValueError`` — which the routers translate into a 400 response, never
|
||||
a 500.
|
||||
"""
|
||||
|
||||
import unittest
|
||||
from datetime import datetime
|
||||
|
||||
from app.utils import parse_dt
|
||||
|
||||
|
||||
class ParseDtTest(unittest.TestCase):
|
||||
def test_valid_iso_with_t_separator(self):
|
||||
self.assertEqual(
|
||||
parse_dt("2026-11-16T10:00:00"), datetime(2026, 11, 16, 10, 0, 0)
|
||||
)
|
||||
|
||||
def test_valid_iso_with_space_separator(self):
|
||||
self.assertEqual(
|
||||
parse_dt("2026-11-16 10:00:00"), datetime(2026, 11, 16, 10, 0, 0)
|
||||
)
|
||||
|
||||
def test_valid_date_only(self):
|
||||
self.assertEqual(parse_dt("2026-11-16"), datetime(2026, 11, 16, 0, 0, 0))
|
||||
|
||||
def test_valid_iso_with_fractional_seconds(self):
|
||||
self.assertEqual(
|
||||
parse_dt("2026-11-16T10:00:00.123456"),
|
||||
datetime(2026, 11, 16, 10, 0, 0, 123456),
|
||||
)
|
||||
|
||||
def test_zulu_suffix_converts_to_naive_utc(self):
|
||||
"""'Z' means UTC already: the value is unchanged, tzinfo dropped."""
|
||||
self.assertEqual(
|
||||
parse_dt("2026-11-16T10:00:00Z"), datetime(2026, 11, 16, 10, 0, 0)
|
||||
)
|
||||
|
||||
def test_positive_offset_converts_to_utc(self):
|
||||
"""+08:00 input is shifted back 8 hours to UTC."""
|
||||
self.assertEqual(
|
||||
parse_dt("2026-11-16T18:00:00+08:00"),
|
||||
datetime(2026, 11, 16, 10, 0, 0),
|
||||
)
|
||||
|
||||
def test_negative_offset_converts_to_utc(self):
|
||||
"""-05:00 input is shifted forward 5 hours to UTC."""
|
||||
self.assertEqual(
|
||||
parse_dt("2026-11-16T05:00:00-05:00"),
|
||||
datetime(2026, 11, 16, 10, 0, 0),
|
||||
)
|
||||
|
||||
def test_offset_can_shift_the_calendar_day(self):
|
||||
"""+09:00 just after midnight UTC-converts to the previous day."""
|
||||
self.assertEqual(
|
||||
parse_dt("2026-11-17T01:00:00+09:00"),
|
||||
datetime(2026, 11, 16, 16, 0, 0),
|
||||
)
|
||||
|
||||
def test_result_is_always_naive(self):
|
||||
for raw in ("2026-11-16T10:00:00Z", "2026-11-16T18:00:00+08:00"):
|
||||
self.assertIsNone(parse_dt(raw).tzinfo)
|
||||
|
||||
def test_empty_input_returns_none(self):
|
||||
self.assertIsNone(parse_dt(None))
|
||||
self.assertIsNone(parse_dt(""))
|
||||
|
||||
def test_garbage_input_raises_value_error(self):
|
||||
"""Invalid input raises ValueError (the routers map this to 400)."""
|
||||
for raw in ("not-a-date", "2026-13-01", "2026-02-30", "16/11/2026",
|
||||
"2026-11-16T25:00:00", "tomorrow", " "):
|
||||
with self.subTest(raw=raw):
|
||||
with self.assertRaises(ValueError):
|
||||
parse_dt(raw)
|
||||
|
||||
def test_whitespace_is_tolerated(self):
|
||||
self.assertEqual(
|
||||
parse_dt(" 2026-11-16T10:00:00 "),
|
||||
datetime(2026, 11, 16, 10, 0, 0),
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
@@ -0,0 +1,151 @@
|
||||
"""Tests for ``parse_sdr_output`` (``app.services.ipmi_collector``).
|
||||
|
||||
Covers the two SDR shapes (new 5-segment and legacy 3-segment), negative
|
||||
readings, thousands separators and the disabled/critical status tokens.
|
||||
|
||||
Dependency note: this machine has no project third-party packages
|
||||
installed (no ``influxdb_client`` / ``cryptography`` / ``pydantic``), and
|
||||
``ipmi_collector`` imports ``app.influxdb`` and ``app.config`` at module
|
||||
level. Like ``test_total_power_fallback`` (which only imports the
|
||||
stdlib-only ``component_parser``), these tests keep the import surface
|
||||
minimal: the two heavy modules are stubbed in ``sys.modules`` before the
|
||||
import, so only the pure parsing function under test is exercised. The
|
||||
``app.crypto`` import inside ``ipmi_collector`` is already guarded by a
|
||||
``try/except ImportError`` there and falls back to the identity function.
|
||||
"""
|
||||
|
||||
import sys
|
||||
import types
|
||||
import unittest
|
||||
from types import SimpleNamespace
|
||||
|
||||
|
||||
def _install_stubs() -> None:
|
||||
"""Stub ``app.influxdb`` / ``app.config`` so ``ipmi_collector`` imports."""
|
||||
if "app.influxdb" not in sys.modules:
|
||||
sys.modules["app.influxdb"] = types.ModuleType("app.influxdb")
|
||||
if "app.config" not in sys.modules:
|
||||
config_mod = types.ModuleType("app.config")
|
||||
config_mod.settings = SimpleNamespace(COLLECT_INTERVAL=60)
|
||||
sys.modules["app.config"] = config_mod
|
||||
|
||||
|
||||
_install_stubs()
|
||||
|
||||
from app.services.ipmi_collector import parse_sdr_output
|
||||
|
||||
|
||||
class ParseSdrOutputTest(unittest.TestCase):
|
||||
def test_new_format_five_segments(self):
|
||||
"""New format: name | id | status | entity | value."""
|
||||
result = parse_sdr_output("Temp | 01h | ok | 3.1 | 37 degrees C\n")
|
||||
self.assertEqual(len(result), 1)
|
||||
entry = result[0]
|
||||
self.assertEqual(entry["name"], "Temp")
|
||||
self.assertEqual(entry["status"], "ok")
|
||||
self.assertEqual(entry["value_str"], "37 degrees C")
|
||||
self.assertEqual(entry["metric_type"], "temperature")
|
||||
self.assertEqual(entry["numeric_value"], 37.0)
|
||||
|
||||
def test_new_format_fan(self):
|
||||
result = parse_sdr_output("Fan1A | 38h | ok | 7.1 | 7920 RPM\n")
|
||||
self.assertEqual(result[0]["metric_type"], "fan")
|
||||
self.assertEqual(result[0]["numeric_value"], 7920.0)
|
||||
|
||||
def test_legacy_format_three_segments(self):
|
||||
"""Legacy format: name | value | status."""
|
||||
result = parse_sdr_output("CPU1 Temp | 45 degrees C | ok\n")
|
||||
self.assertEqual(len(result), 1)
|
||||
entry = result[0]
|
||||
self.assertEqual(entry["name"], "CPU1 Temp")
|
||||
self.assertEqual(entry["status"], "ok")
|
||||
self.assertEqual(entry["metric_type"], "temperature")
|
||||
self.assertEqual(entry["numeric_value"], 45.0)
|
||||
|
||||
def test_legacy_format_fan_and_voltage(self):
|
||||
result = parse_sdr_output(
|
||||
"FAN1 | 5400 RPM | ok\n"
|
||||
"Vcpu1 | 1.01 Volts | ok\n"
|
||||
)
|
||||
self.assertEqual(result[0]["metric_type"], "fan")
|
||||
self.assertEqual(result[0]["numeric_value"], 5400.0)
|
||||
self.assertEqual(result[1]["metric_type"], "voltage")
|
||||
self.assertEqual(result[1]["numeric_value"], 1.01)
|
||||
|
||||
def test_negative_temperature(self):
|
||||
"""Negative readings must keep their sign (both formats)."""
|
||||
result = parse_sdr_output(
|
||||
"Inlet Temp | 01h | ok | 3.1 | -5.0 degrees C\n"
|
||||
"Ambient Temp | -2.5 degrees C | ok\n"
|
||||
)
|
||||
self.assertEqual(result[0]["metric_type"], "temperature")
|
||||
self.assertEqual(result[0]["numeric_value"], -5.0)
|
||||
self.assertEqual(result[1]["metric_type"], "temperature")
|
||||
self.assertEqual(result[1]["numeric_value"], -2.5)
|
||||
|
||||
def test_thousands_separator_comma(self):
|
||||
"""'1,234 RPM' / '1,234.5 RPM' parse as 1234 / 1234.5, not 1."""
|
||||
result = parse_sdr_output(
|
||||
"FAN1 | 38h | ok | 7.1 | 1,234 RPM\n"
|
||||
"FAN2 | 39h | ok | 7.1 | 1,234.5 RPM\n"
|
||||
)
|
||||
self.assertEqual(result[0]["numeric_value"], 1234.0)
|
||||
self.assertEqual(result[1]["numeric_value"], 1234.5)
|
||||
|
||||
def test_disabled_status_token(self):
|
||||
"""A 'disabled' sensor still parses; its status is kept verbatim."""
|
||||
result = parse_sdr_output("Fan2 | 39h | disabled | 7.1 | 0 RPM\n")
|
||||
self.assertEqual(len(result), 1)
|
||||
self.assertEqual(result[0]["status"], "disabled")
|
||||
self.assertEqual(result[0]["metric_type"], "fan")
|
||||
self.assertEqual(result[0]["numeric_value"], 0.0)
|
||||
|
||||
def test_critical_status_token(self):
|
||||
"""A 'cr' (critical) reading still parses with its value."""
|
||||
result = parse_sdr_output(
|
||||
"CPU Temp | 01h | cr | 3.1 | 95 degrees C\n"
|
||||
)
|
||||
self.assertEqual(result[0]["status"], "cr")
|
||||
self.assertEqual(result[0]["metric_type"], "temperature")
|
||||
self.assertEqual(result[0]["numeric_value"], 95.0)
|
||||
|
||||
def test_unknown_shape_finds_status_token(self):
|
||||
"""4-segment rows: locate the status token, never read the sensor
|
||||
id (parts[1]) as the value."""
|
||||
result = parse_sdr_output(
|
||||
"Fan1 | 38h | ok | 7920 RPM\n"
|
||||
"CPU Temp | 01h | cr | 95 degrees C\n"
|
||||
)
|
||||
self.assertEqual(result[0]["status"], "ok")
|
||||
self.assertEqual(result[0]["numeric_value"], 7920.0)
|
||||
self.assertEqual(result[1]["status"], "cr")
|
||||
self.assertEqual(result[1]["numeric_value"], 95.0)
|
||||
|
||||
def test_discrete_status_sensor_uses_status(self):
|
||||
"""Discrete power-supply sensors map ok -> 1.0, anything else -> 0.0."""
|
||||
result = parse_sdr_output(
|
||||
"Power Supply 1 | 0x00 | ok\n"
|
||||
"Power Supply 2 | 0x00 | cr\n"
|
||||
)
|
||||
self.assertEqual(result[0]["metric_type"], "status")
|
||||
self.assertEqual(result[0]["numeric_value"], 1.0)
|
||||
self.assertEqual(result[1]["metric_type"], "status")
|
||||
self.assertEqual(result[1]["numeric_value"], 0.0)
|
||||
|
||||
def test_blank_and_unparseable_lines_skipped(self):
|
||||
result = parse_sdr_output(
|
||||
"\n"
|
||||
"CPU1 Temp | 45 degrees C | ok\n"
|
||||
"garbage line without separators\n"
|
||||
"Foo | bar\n"
|
||||
)
|
||||
self.assertEqual(len(result), 1)
|
||||
self.assertEqual(result[0]["name"], "CPU1 Temp")
|
||||
|
||||
def test_empty_input_returns_empty_list(self):
|
||||
self.assertEqual(parse_sdr_output(""), [])
|
||||
self.assertEqual(parse_sdr_output(" \n \n"), [])
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
unittest.main()
|
||||
+16
-11
@@ -13,10 +13,11 @@ services:
|
||||
- TZ=Asia/Shanghai
|
||||
- DOCKER_INFLUXDB_INIT_MODE=setup
|
||||
- DOCKER_INFLUXDB_INIT_USERNAME=${INFLUXDB_USERNAME:-admin}
|
||||
- DOCKER_INFLUXDB_INIT_PASSWORD=${INFLUXDB_PASSWORD:-admin123456}
|
||||
# Required: no default is shipped; Compose refuses to start without them.
|
||||
- DOCKER_INFLUXDB_INIT_PASSWORD=${INFLUXDB_PASSWORD:?must be set}
|
||||
- DOCKER_INFLUXDB_INIT_ORG=${INFLUXDB_ORG:-ipmi-monitor}
|
||||
- DOCKER_INFLUXDB_INIT_BUCKET=${INFLUXDB_BUCKET:-metrics}
|
||||
- DOCKER_INFLUXDB_INIT_ADMIN_TOKEN=${INFLUXDB_TOKEN:-my-super-secret-token}
|
||||
- DOCKER_INFLUXDB_INIT_ADMIN_TOKEN=${INFLUXDB_TOKEN:?must be set}
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
test: ["CMD", "influx", "ping"]
|
||||
@@ -34,20 +35,27 @@ services:
|
||||
ports:
|
||||
- "${BACKEND_PORT:-8000}:8000"
|
||||
volumes:
|
||||
- ./backend/app:/app/app
|
||||
- ./frontend:/app/frontend
|
||||
- backend-data:/app/data
|
||||
environment:
|
||||
- TZ=Asia/Shanghai
|
||||
- ENVIRONMENT=${ENVIRONMENT:-production}
|
||||
# Required: the backend refuses to start without a JWT secret of at
|
||||
# least 32 characters. Generate one and set it in .env.
|
||||
- JWT_SECRET_KEY=${JWT_SECRET_KEY:?must be set}
|
||||
# Required on first boot only (empty user table): password for the
|
||||
# initial admin account (at least 12 characters). Ignored afterwards.
|
||||
- ADMIN_INITIAL_PASSWORD=${ADMIN_INITIAL_PASSWORD:-}
|
||||
# Optional: Fernet key for encrypting stored BMC/SMTP/Feishu secrets.
|
||||
# When unset, the backend generates one and persists it next to the DB.
|
||||
- CREDENTIAL_ENCRYPTION_KEY=${CREDENTIAL_ENCRYPTION_KEY:-}
|
||||
# Optional: comma-separated CORS allow-list. Empty = no CORS (same-origin).
|
||||
- CORS_ORIGINS=${CORS_ORIGINS:-}
|
||||
- INFLUXDB_URL=http://influxdb:8086
|
||||
- INFLUXDB_TOKEN=${INFLUXDB_TOKEN:-my-super-secret-token}
|
||||
- INFLUXDB_TOKEN=${INFLUXDB_TOKEN:?must be set}
|
||||
- INFLUXDB_ORG=${INFLUXDB_ORG:-ipmi-monitor}
|
||||
- INFLUXDB_BUCKET=${INFLUXDB_BUCKET:-metrics}
|
||||
- POLL_INTERVAL=${POLL_INTERVAL:-60}
|
||||
- LOG_LEVEL=${LOG_LEVEL:-INFO}
|
||||
- ALERT_TEMP_THRESHOLD=${ALERT_TEMP_THRESHOLD:-80}
|
||||
- ALERT_FAN_THRESHOLD=${ALERT_FAN_THRESHOLD:-0}
|
||||
- ALERT_POWER_THRESHOLD=${ALERT_POWER_THRESHOLD:-500}
|
||||
- DATABASE_URL=${DATABASE_URL:-sqlite+aiosqlite:///./data/ipmi_monitor.db}
|
||||
- REQUIRE_EXISTING_DATABASE=${REQUIRE_EXISTING_DATABASE:-false}
|
||||
- MIN_EXPECTED_HOSTS=${MIN_EXPECTED_HOSTS:-0}
|
||||
@@ -55,9 +63,6 @@ services:
|
||||
- SMTP_PORT=${SMTP_PORT:-587}
|
||||
- SMTP_USER=${SMTP_USER:-}
|
||||
- SMTP_PASSWORD=${SMTP_PASSWORD:-}
|
||||
- ALERT_EMAIL_FROM=${ALERT_EMAIL_FROM:-}
|
||||
- ALERT_EMAIL_TO=${ALERT_EMAIL_TO:-}
|
||||
- WEBHOOK_URL=${WEBHOOK_URL:-}
|
||||
depends_on:
|
||||
influxdb:
|
||||
condition: service_healthy
|
||||
|
||||
@@ -63,7 +63,9 @@
|
||||
|
||||
<el-table-column label="密码" min-width="110">
|
||||
<template #default="{ row }">
|
||||
<span class="password-mask">••••••••</span>
|
||||
<!-- 后端不再返回 ipmi_password,只返回 has_password -->
|
||||
<span v-if="row.has_password" class="password-mask">••••••••</span>
|
||||
<span v-else class="text-muted">未配置</span>
|
||||
</template>
|
||||
</el-table-column>
|
||||
|
||||
|
||||
@@ -27,12 +27,18 @@
|
||||
<el-form-item label="Webhook URL">
|
||||
<el-input
|
||||
v-model="feishuForm.feishu_webhook_url"
|
||||
placeholder="https://open.feishu.cn/open-apis/bot/v2/hook/xxxxxxxx"
|
||||
:placeholder="feishuWebhookMasked || 'https://open.feishu.cn/open-apis/bot/v2/hook/xxxxxxxx'"
|
||||
clearable
|
||||
/>
|
||||
<div class="form-tip">
|
||||
飞书群机器人 Webhook 地址,在飞书群设置中添加自定义机器人获取
|
||||
</div>
|
||||
<div v-if="feishuWebhookMasked" class="form-tip">
|
||||
当前已配置:{{ feishuWebhookMasked }},留空则不修改
|
||||
</div>
|
||||
<div v-else class="form-tip">
|
||||
当前未配置
|
||||
</div>
|
||||
</el-form-item>
|
||||
|
||||
<el-form-item label="安全关键词">
|
||||
@@ -106,13 +112,16 @@
|
||||
<el-input
|
||||
v-model="emailForm.smtp_password"
|
||||
type="password"
|
||||
placeholder="邮箱密码或授权码"
|
||||
:placeholder="smtpPasswordSet ? '已配置,留空则不修改' : '邮箱密码或授权码'"
|
||||
show-password
|
||||
clearable
|
||||
/>
|
||||
<div class="form-tip">
|
||||
部分邮箱需要使用授权码而非登录密码,请在邮箱设置中生成
|
||||
</div>
|
||||
<div v-if="smtpPasswordSet" class="form-tip">
|
||||
当前已配置,留空则不修改
|
||||
</div>
|
||||
</el-form-item>
|
||||
|
||||
<el-form-item label="发件人地址">
|
||||
@@ -142,7 +151,7 @@
|
||||
<el-button
|
||||
type="info"
|
||||
:loading="testingEmail"
|
||||
@click="testEmail"
|
||||
@click="testEmailSend"
|
||||
:icon="Promotion"
|
||||
>
|
||||
发送测试邮件
|
||||
@@ -170,6 +179,10 @@ const testEmail = ref('')
|
||||
const feishuFormRef = ref()
|
||||
const emailFormRef = ref()
|
||||
|
||||
// 后端不再返回秘密明文,只返回掩码/是否已配置;秘密输入框留空表示不改。
|
||||
const feishuWebhookMasked = ref('')
|
||||
const smtpPasswordSet = ref(false)
|
||||
|
||||
const feishuForm = reactive({
|
||||
feishu_webhook_url: '',
|
||||
feishu_keyword: '',
|
||||
@@ -188,15 +201,20 @@ const emailForm = reactive({
|
||||
const loadSettings = async () => {
|
||||
try {
|
||||
const response = await api.get('/api/settings')
|
||||
// 后端不再返回 feishu_webhook_url / smtp_password 明文:
|
||||
// 飞书只给掩码(后 4 位可见),SMTP 只给是否已配置。
|
||||
// 秘密输入框保持留空,提交时留空表示不改。
|
||||
feishuWebhookMasked.value = response.feishu_webhook_masked || ''
|
||||
smtpPasswordSet.value = !!response.smtp_password_set
|
||||
Object.assign(feishuForm, {
|
||||
feishu_webhook_url: response.feishu_webhook_url || '',
|
||||
feishu_webhook_url: '',
|
||||
feishu_keyword: response.feishu_keyword || '',
|
||||
})
|
||||
Object.assign(emailForm, {
|
||||
smtp_host: response.smtp_host || '',
|
||||
smtp_port: parseInt(response.smtp_port) || 587,
|
||||
smtp_user: response.smtp_user || '',
|
||||
smtp_password: response.smtp_password || '',
|
||||
smtp_password: '',
|
||||
smtp_from: response.smtp_from || '',
|
||||
smtp_use_tls: response.smtp_use_tls !== false,
|
||||
})
|
||||
@@ -232,7 +250,8 @@ const saveSettings = async () => {
|
||||
|
||||
// 测试飞书
|
||||
const testFeishu = async () => {
|
||||
if (!feishuForm.feishu_webhook_url) {
|
||||
// 本次未重填也可用已配置的 Webhook(后端存着解密后的值)
|
||||
if (!feishuForm.feishu_webhook_url && !feishuWebhookMasked.value) {
|
||||
ElMessage.warning('请先配置飞书Webhook URL')
|
||||
return
|
||||
}
|
||||
@@ -254,7 +273,7 @@ const testEmailSend = async () => {
|
||||
ElMessage.warning('请输入测试邮箱地址')
|
||||
return
|
||||
}
|
||||
if (!emailForm.smtp_host || !emailForm.smtp_user || !emailForm.smtp_password) {
|
||||
if (!emailForm.smtp_host || !emailForm.smtp_user || (!emailForm.smtp_password && !smtpPasswordSet.value)) {
|
||||
ElMessage.warning('请先配置SMTP服务器信息')
|
||||
return
|
||||
}
|
||||
|
||||
@@ -71,6 +71,47 @@ else
|
||||
echo -e "${GREEN} ✓ .env file found${NC}"
|
||||
fi
|
||||
|
||||
# ----- Step 3b: Reject missing / placeholder secrets -----
|
||||
echo -e "${YELLOW}[3b/6] Checking required secrets in .env...${NC}"
|
||||
|
||||
get_env_value() {
|
||||
grep -E "^$1=" .env 2>/dev/null | tail -n 1 | cut -d'=' -f2- | tr -d '\r' | sed -e 's/^["'\'']//' -e 's/["'\'']$//'
|
||||
}
|
||||
|
||||
looks_like_placeholder() {
|
||||
local value="$1"
|
||||
local lowered
|
||||
lowered=$(printf '%s' "$value" | tr '[:upper:]' '[:lower:]')
|
||||
case "$lowered" in
|
||||
*changeme*|*example*|*placeholder*|*default*|*todo*) return 0 ;;
|
||||
esac
|
||||
return 1
|
||||
}
|
||||
|
||||
INFLUXDB_PASSWORD=$(get_env_value INFLUXDB_PASSWORD)
|
||||
INFLUXDB_TOKEN=$(get_env_value INFLUXDB_TOKEN)
|
||||
JWT_SECRET_KEY=$(get_env_value JWT_SECRET_KEY)
|
||||
|
||||
if [ -z "$INFLUXDB_PASSWORD" ] || looks_like_placeholder "$INFLUXDB_PASSWORD" || [ "${#INFLUXDB_PASSWORD}" -lt 8 ]; then
|
||||
echo -e "${RED}Error: INFLUXDB_PASSWORD is missing, too short (< 8 chars), or still a placeholder.${NC}"
|
||||
echo "Set a strong password in .env before starting."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ -z "$INFLUXDB_TOKEN" ] || looks_like_placeholder "$INFLUXDB_TOKEN"; then
|
||||
echo -e "${RED}Error: INFLUXDB_TOKEN is missing or still a placeholder.${NC}"
|
||||
echo "Set a strong random token in .env before starting."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
if [ -z "$JWT_SECRET_KEY" ] || looks_like_placeholder "$JWT_SECRET_KEY" || [ "${#JWT_SECRET_KEY}" -lt 32 ]; then
|
||||
echo -e "${RED}Error: JWT_SECRET_KEY is missing, too short (< 32 chars), or still a placeholder.${NC}"
|
||||
echo "Generate one (e.g. python3 -c \"import secrets; print(secrets.token_urlsafe(32))\") and set it in .env."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo -e "${GREEN} ✓ Required secrets are set${NC}"
|
||||
|
||||
# ----- Step 4: Start Docker Compose services -----
|
||||
echo -e "${YELLOW}[4/6] Starting Docker Compose services...${NC}"
|
||||
|
||||
|
||||
Reference in New Issue
Block a user