401 lines
13 KiB
Markdown
401 lines
13 KiB
Markdown
# 检查点操作详细日志指南
|
||
|
||
## 📋 概述
|
||
|
||
已为检查点功能添加详细的日志输出，让你能够实时查看每张表的数据变化和操作进度。
|
||
|
||
---
|
||
|
||
## 🔍 日志特点
|
||
|
||
### 日志级别
|
||
- **INFO** - 正常操作信息
|
||
- **WARNING** - 重要警告（恢复操作）
|
||
- **ERROR** - 错误信息
|
||
|
||
### 日志格式
|
||
```
|
||
2025-10-30 14:30:15 [INFO] lawrisk.services.licensing_repo: [CHECKPOINT] Starting checkpoint creation: checkpoint_20251030_143015
|
||
```
|
||
|
||
### 关键标识
|
||
- `[CHECKPOINT]` - 检查点相关操作
|
||
- `[API]` - API 请求信息
|
||
- `[1/8]` - 表恢复进度 (当前表/总表数)
|
||
|
||
---
|
||
|
||
## 📊 创建 Checkpoint 日志示例
|
||
|
||
```
|
||
================================================================================
|
||
[CHECKPOINT] Starting checkpoint creation: checkpoint_20251030_143015
|
||
[CHECKPOINT] Description: 测试备份
|
||
================================================================================
|
||
[CHECKPOINT] Found 8 tables to backup: regions, themes, business_scopes, permits, risks, region_themes, region_scopes, region_theme_permits
|
||
[CHECKPOINT] [1/8] Processing table: regions
|
||
[CHECKPOINT] Backing up table: regions
|
||
[CHECKPOINT] Backup complete: regions - 5 rows, 3 columns
|
||
[CHECKPOINT] [1/8] Table regions backed up: 5 rows
|
||
[CHECKPOINT] [2/8] Processing table: themes
|
||
[CHECKPOINT] Backing up table: themes
|
||
[CHECKPOINT] Backup complete: themes - 12 rows, 3 columns
|
||
[CHECKPOINT] [2/8] Table themes backed up: 12 rows
|
||
...
|
||
[CHECKPOINT] All tables backed up successfully, committing transaction...
|
||
[CHECKPOINT] Transaction committed
|
||
[CHECKPOINT] Saving checkpoint file: data/checkpoints/checkpoint_20251030_143015.json
|
||
================================================================================
|
||
[CHECKPOINT] Checkpoint creation COMPLETED: checkpoint_20251030_143015
|
||
[CHECKPOINT] Total rows backed up: 1247
|
||
[CHECKPOINT] File: data/checkpoints/checkpoint_20251030_143015.json
|
||
================================================================================
|
||
```
|
||
|
||
---
|
||
|
||
## 📈 恢复 Checkpoint 日志示例
|
||
|
||
```
|
||
================================================================================
|
||
[API] Received checkpoint restore request: checkpoint_20251030_143015
|
||
[API] Request parameters: {'create_auto_backup': 'true'}
|
||
[API] Auto-backup enabled: True
|
||
[API] Starting restore operation...
|
||
================================================================================
|
||
[CHECKPOINT] WARNING: Starting restore operation: checkpoint_20251030_143015
|
||
[CHECKPOINT] This will OVERWRITE all existing data in the database!
|
||
================================================================================
|
||
[CHECKPOINT] Creating auto-backup before restore...
|
||
================================================================================
|
||
[CHECKPOINT] Starting checkpoint creation: auto_backup_before_restore_checkpoint_20251030_143015
|
||
...
|
||
[CHECKPOINT] Auto-backup created: auto_backup_before_restore_checkpoint_20251030_143015
|
||
[CHECKPOINT] Auto-backup contains 1247 rows
|
||
================================================================================
|
||
[CHECKPOINT] Checkpoint details:
|
||
[CHECKPOINT] ID: checkpoint_20251030_143015
|
||
[CHECKPOINT] Tables: 8
|
||
[CHECKPOINT] Total rows: 1247
|
||
[CHECKPOINT] Auto-backup: Yes (auto_backup_before_restore_checkpoint_20251030_143015)
|
||
================================================================================
|
||
[CHECKPOINT] Building table dependency graph...
|
||
[CHECKPOINT] Found 6 table dependencies
|
||
[CHECKPOINT] Calculating restore order...
|
||
[CHECKPOINT] Restore order: risks -> permits -> business_scopes -> themes -> regions -> region_themes -> region_scopes -> region_theme_permits
|
||
[CHECKPOINT] Acquiring exclusive locks on all tables...
|
||
[CHECKPOINT] All tables locked exclusively
|
||
================================================================================
|
||
[CHECKPOINT] Starting restore process...
|
||
================================================================================
|
||
[CHECKPOINT] [1/8] Preparing to restore table: risks
|
||
[CHECKPOINT] Truncating table: risks
|
||
[CHECKPOINT] Restoring 100 rows into risks (columns: id, risk_content, legal_basis, document_no, summary)
|
||
[CHECKPOINT] Progress: risks - 100/100 rows inserted
|
||
[CHECKPOINT] Restore complete: risks - 100 rows successfully inserted
|
||
[CHECKPOINT] [1/8] Table risks restored: 100 rows
|
||
|
||
[CHECKPOINT] [2/8] Preparing to restore table: permits
|
||
[CHECKPOINT] Truncating table: permits
|
||
[CHECKPOINT] Restoring 50 rows into permits (columns: id, name)
|
||
[CHECKPOINT] Progress: permits - 50/50 rows inserted
|
||
[CHECKPOINT] Restore complete: permits - 50 rows successfully inserted
|
||
[CHECKPOINT] [2/8] Table permits restored: 50 rows
|
||
|
||
...
|
||
[CHECKPOINT] [8/8] Preparing to restore table: region_theme_permits
|
||
[CHECKPOINT] Truncating table: region_theme_permits
|
||
[CHECKPOINT] Restoring 800 rows into region_theme_permits (columns: region_id, theme_id, permit_id)
|
||
[CHECKPOINT] Progress: region_theme_permits - 100/800 rows inserted
|
||
[CHECKPOINT] Progress: region_theme_permits - 200/800 rows inserted
|
||
[CHECKPOINT] Progress: region_theme_permits - 300/800 rows inserted
|
||
...
|
||
[CHECKPOINT] Progress: region_theme_permits - 800/800 rows inserted
|
||
[CHECKPOINT] Restore complete: region_theme_permits - 800 rows successfully inserted
|
||
[CHECKPOINT] [8/8] Table region_theme_permits restored: 800 rows
|
||
|
||
================================================================================
|
||
[CHECKPOINT] All tables restored successfully, committing transaction...
|
||
[CHECKPOINT] Transaction committed successfully
|
||
================================================================================
|
||
================================================================================
|
||
[CHECKPOINT] RESTORE COMPLETED SUCCESSFULLY: checkpoint_20251030_143015
|
||
[CHECKPOINT] Tables restored: 8/8
|
||
[CHECKPOINT] Total rows restored: 1247
|
||
[CHECKPOINT] Auto-backup available: auto_backup_before_restore_checkpoint_20251030_143015
|
||
================================================================================
|
||
[API] Restore completed successfully
|
||
[API] Tables restored: 8
|
||
[API] Total rows restored: 1247
|
||
```
|
||
|
||
---
|
||
|
||
## 🚀 如何查看日志
|
||
|
||
### 方法1: 查看控制台输出
|
||
|
||
启动应用时，所有日志会直接输出到控制台：
|
||
|
||
```bash
|
||
python app.py
|
||
```
|
||
|
||
### 方法2: 查看日志文件
|
||
|
||
如果将日志重定向到文件：
|
||
|
||
```bash
|
||
python app.py > checkpoint.log 2>&1
|
||
tail -f checkpoint.log
|
||
```
|
||
|
||
### 方法3: 实时监控日志
|
||
|
||
```bash
|
||
# Linux/Mac
|
||
tail -f /path/to/app.log | grep -E "(CHECKPOINT|API)"
|
||
|
||
# Windows (PowerShell)
|
||
Get-Content app.log -Wait | Where-Object { $_ -match "(CHECKPOINT|API)" }
|
||
```
|
||
|
||
---
|
||
|
||
## 📝 日志信息说明
|
||
|
||
### 创建 Checkpoint 时
|
||
|
||
| 日志内容 | 含义 |
|
||
|---------|------|
|
||
| `[CHECKPOINT] Starting checkpoint creation` | 开始创建检查点 |
|
||
| `[CHECKPOINT] Found X tables` | 找到X个表 |
|
||
| `[CHECKPOINT] [i/N] Processing table` | 正在处理第i个表，共N个表 |
|
||
| `[CHECKPOINT] Backup complete: table_name - X rows` | 表备份完成，X行数据 |
|
||
| `[CHECKPOINT] Transaction committed` | 事务提交成功 |
|
||
| `[CHECKPOINT] Checkpoint creation COMPLETED` | 检查点创建完成 |
|
||
|
||
### 恢复 Checkpoint 时
|
||
|
||
| 日志内容 | 含义 |
|
||
|---------|------|
|
||
| `[CHECKPOINT] WARNING: Starting restore` | 开始恢复操作警告 |
|
||
| `[CHECKPOINT] Creating auto-backup` | 创建自动备份 |
|
||
| `[CHECKPOINT] Auto-backup created` | 自动备份创建完成 |
|
||
| `[CHECKPOINT] Building table dependency graph` | 构建表依赖关系 |
|
||
| `[CHECKPOINT] Calculating restore order` | 计算恢复顺序 |
|
||
| `[CHECKPOINT] Restore order: ...` | 显示恢复顺序 |
|
||
| `[CHECKPOINT] Acquiring exclusive locks` | 获取表级锁 |
|
||
| `[CHECKPOINT] [i/N] Preparing to restore table` | 准备恢复第i个表 |
|
||
| `[CHECKPOINT] Truncating table` | 清空表数据 |
|
||
| `[CHECKPOINT] Restoring X rows into table` | 开始恢复X行数据 |
|
||
| `[CHECKPOINT] Progress: table - Y/Z rows inserted` | 恢复进度：已插入Y行，共Z行 |
|
||
| `[CHECKPOINT] Restore complete: table - X rows` | 表恢复完成，共X行 |
|
||
| `[CHECKPOINT] Table table_name restored: X rows` | 表恢复统计信息 |
|
||
| `[CHECKPOINT] RESTORE COMPLETED SUCCESSFULLY` | 恢复操作成功完成 |
|
||
|
||
### 错误日志
|
||
|
||
| 日志内容 | 含义 |
|
||
|---------|------|
|
||
| `[CHECKPOINT] ERROR: ...` | 发生错误 |
|
||
| `[CHECKPOINT] Restore FAILED` | 恢复失败 |
|
||
| `[CHECKPOINT] Transaction rolled back` | 事务已回滚 |
|
||
| `[CHECKPOINT] Recovery suggestion:` | 恢复建议（使用自动备份） |
|
||
|
||
---
|
||
|
||
## 🔧 故障排除
|
||
|
||
### 问题1: 日志看不到输出
|
||
|
||
**原因**: 日志被缓冲或重定向
|
||
|
||
**解决方案**:
|
||
1. 确保使用了 `-u` 参数运行Python（禁用缓冲）
|
||
```bash
|
||
python -u app.py
|
||
```
|
||
|
||
2. 或者设置环境变量
|
||
```bash
|
||
export PYTHONUNBUFFERED=1
|
||
python app.py
|
||
```
|
||
|
||
### 问题2: 日志信息不完整
|
||
|
||
**原因**: 日志级别设置过高
|
||
|
||
**解决方案**:
|
||
确保日志级别设置为INFO或DEBUG：
|
||
|
||
```python
|
||
import logging
|
||
logging.getLogger().setLevel(logging.INFO)
|
||
```
|
||
|
||
### 问题3: 恢复操作卡住
|
||
|
||
**原因**: 可能正在处理大表，或者等待锁
|
||
|
||
**解决方案**:
|
||
1. 查看日志中的进度信息
|
||
2. 等待表锁释放
|
||
3. 检查是否有其他会话占用表锁
|
||
|
||
---
|
||
|
||
## 📊 性能监控
|
||
|
||
通过日志可以监控：
|
||
|
||
1. **备份/恢复速度**
|
||
```
|
||
每行数据处理时间 = 总时间 / 行数
|
||
```
|
||
|
||
2. **表大小分布**
|
||
```
|
||
查看每个表的行数
|
||
```
|
||
|
||
3. **依赖关系正确性**
|
||
```
|
||
验证恢复顺序是否正确
|
||
```
|
||
|
||
4. **事务效率**
|
||
```
|
||
开始时间 -> 提交时间 = 事务持续时间
|
||
```
|
||
|
||
---
|
||
|
||
## ⚡ 快速查看命令
|
||
|
||
### 查看最新日志
|
||
```bash
|
||
tail -100 checkpoint.log | grep "CHECKPOINT"
|
||
```
|
||
|
||
### 统计处理的行数
|
||
```bash
|
||
grep "rows inserted" checkpoint.log | awk '{print $6}'
|
||
```
|
||
|
||
### 查看错误信息
|
||
```bash
|
||
grep -i "error\|failed" checkpoint.log
|
||
```
|
||
|
||
### 监控实时进度
|
||
```bash
|
||
tail -f checkpoint.log | grep "Progress:"
|
||
```
|
||
|
||
---
|
||
|
||
**注意**: 所有日志都会立即输出到控制台，无需等待操作完成即可查看进度。
|
||
|
||
---
|
||
|
||
## 🧱 针对许可风险的精细化快照
|
||
|
||
> 新增：基于 `region_permit_risk_vw` 视图的原子检查点
|
||
|
||
当用户编辑某条许可风险记录（唯一键为 `region_id + permit_id + risk_id`）时，可调用以下工具函数先写入快照：
|
||
|
||
```python
|
||
from lawrisk.services.licensing_repo import create_permit_risk_snapshot
|
||
|
||
snapshot_meta = create_permit_risk_snapshot(
|
||
region_id="7c2b…",
|
||
permit_id="b93d…",
|
||
risk_id="a01f…",
|
||
edited_by="alice",
|
||
change_summary="调整法律依据与责任主体"
|
||
)
|
||
```
|
||
|
||
返回示例：
|
||
|
||
```python
|
||
{
|
||
"snapshot_id": "5c4b…",
|
||
"permit_risk_key": "7c2b…::b93d…::a01f…",
|
||
"version": 3,
|
||
"created_at": "2024-05-18T02:41:55.210412+00:00",
|
||
"edited_by": "alice",
|
||
"change_summary": "调整法律依据与责任主体",
|
||
"payload": {...完整的视图行...}
|
||
}
|
||
```
|
||
|
||
- 快照数据存放在 `permit_risk_snapshots` 表中，`payload` 记录视图的完整列。
|
||
- 使用 `list_permit_risk_snapshots(region_id=..., permit_id=..., risk_id=...)` 查看历史版本。
|
||
- 通过 `get_permit_risk_snapshot(snapshot_id)` 获取任意版本的详细内容，以便比对或手工回滚。
|
||
|
||
### 带快照的原子修改
|
||
|
||
```python
|
||
from lawrisk.services.licensing_repo import update_permit_risk_record
|
||
|
||
result = update_permit_risk_record(
|
||
region_id="7c2b…",
|
||
permit_id="b93d…",
|
||
risk_id="a01f…",
|
||
risk_content="新的风险提示要点",
|
||
legal_basis="《某条例》第三条",
|
||
permit_status="正常受理",
|
||
edited_by="alice",
|
||
change_summary="补充法规依据并更新状态"
|
||
)
|
||
|
||
print(result["snapshot"]["version"]) # 修改前版本
|
||
print(result["current"]["risk_content"]) # 修改后视图内容
|
||
```
|
||
|
||
- 函数会先锁定 `region_permit_risks` 目标行，写入快照，再执行字段更新。
|
||
- `risk_*` 参数对应 `risks` 表字段；`permit_*` 参数对应 `region_permit_details`。
|
||
- 可传入 `None` 将字段清空；若不想修改某列则省略参数。
|
||
|
||
### 管理端快照列表 API
|
||
|
||
通过管理端路由 `/fs-ai-asistant/api/workflow/lawrisk/admin/permit-risk-snapshots` 获取快照历史，支持查询参数：
|
||
|
||
- `region_id` / `region`：按区域筛选
|
||
- `permit_id` / `permit`：按许可筛选
|
||
- `edited_by`：按编辑人筛选
|
||
- `limit`（默认20，最大200）、`offset`
|
||
|
||
返回结构：
|
||
|
||
```json
|
||
{
|
||
"success": true,
|
||
"data": {
|
||
"snapshots": [
|
||
{
|
||
"snapshot_id": "…",
|
||
"region_name": "佛山市禅城区",
|
||
"permit_name": "食品经营许可-餐饮服务",
|
||
"risk_content": "1. 禁止在居民住宅楼…",
|
||
"permit_status": "正常受理",
|
||
"edited_by": "alice",
|
||
"change_summary": "补充法规依据",
|
||
"created_at": "2024-05-18T02:41:55.210412+00:00",
|
||
"version": 3
|
||
}
|
||
],
|
||
"pagination": {
|
||
"limit": 20,
|
||
"offset": 0,
|
||
"total": 128
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
可以直接在“检查点管理”按钮中调用此接口，将 `snapshots` 列表渲染成表格并展示“受影响许可”“编辑人”“创建时间”等信息。
|