diff --git a/CHANGELOG.md b/CHANGELOG.md index 7626d0a..70e95cd 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,7 @@ Versioning follows [Semantic Versioning](https://semver.org/). - **CI runs remaining unwired suites** — `test-p0-friction.ps1` on the Windows leg of `routing-tests` (Windows PowerShell 5.1); `case-review/tests/test_review_case.py` in the Linux `case-contract` job. `test-workflow-title-safety.ps1` was already wired. ### Fixed +- **IDA MCP HTTP stall** — `run-supervisor.py` patches stock `idalib_supervisor` onto `ThreadingHTTPServer` and accepts Streamable HTTP GET `/mcp` (patch failure is skipped, supervisor still starts). `watchdog.ps1` / `start.ps1` replace a *managed* supervisor if `tools/list` stays dead for more than 3 minutes; GUI and in-flight `idb_open` stay busy/reuse. New `recover.ps1` is an immediate `-Force` path. Never `taskkill`s `ida.exe`. - **Windows PowerShell 5.1 encoding** — added a UTF-8 BOM to five non-ASCII `.ps1` scripts (`skills/scripts/verify-doc-facts.ps1`, `apk-reverse/scripts/frida-run.ps1`, `apk-reverse/scripts/rebuild-sign-install.ps1`, `ida-reverse/scripts/start.ps1`, `radare2/scripts/recon.ps1`). Without a BOM, PS 5.1 parses these files as the system ANSI codepage and garbles their Chinese / em-dash string literals; `verify-doc-facts.ps1` was failing four checks under 5.1 (CI only ran it under `pwsh`, which defaults to UTF-8). CI now guards every non-ASCII `.ps1` for a BOM. ### Changed diff --git a/skills/ida-reverse/LOCAL-SETUP.md b/skills/ida-reverse/LOCAL-SETUP.md index 5070140..470d058 100644 --- a/skills/ida-reverse/LOCAL-SETUP.md +++ b/skills/ida-reverse/LOCAL-SETUP.md @@ -36,7 +36,8 @@ python -m ida_pro_mcp --install --transport streamable-http --scope global | 脚本 | 作用 | |------|------| | `scripts/start.ps1` | 健康则 `OK::reuse`；端口在听但 RPC 超时视为忙，不杀；只在无人监听或缺 `py_eval` 时替换 managed supervisor；永不杀 `ida.exe` | -| `scripts/watchdog.ps1` | 每分钟巡检；忙/健康则 reuse；只有 down/stale 才调 `start.ps1` | +| `scripts/watchdog.ps1` | 每分钟巡检；健康 reuse；GUI/新开库 busy reuse；**managed supervisor 超过 3 分钟仍答不上 tools/list 则 `-Force` 拉起** | +| `scripts/recover.ps1` | 立刻 `-Force` 重启 supervisor（不杀 `ida.exe`）。HTTP 客户端把 `idapro` 标成 error 时用这个 | | `scripts/install-autostart.ps1` | 注册计划任务 `reverse-skill-ida-mcp`（登录 + 每分钟） | | `scripts/start-gui.ps1` | idalib license 失败时开 GUI 插件 | | `scripts/open.ps1` | HTTP 直调 `idb_open`，绕过部分客户端 schema 校验 | diff --git a/skills/ida-reverse/SKILL.md b/skills/ida-reverse/SKILL.md index fbc00e9..72c0cce 100644 --- a/skills/ida-reverse/SKILL.md +++ b/skills/ida-reverse/SKILL.md @@ -56,7 +56,8 @@ description: | - 第一次 `open.ps1` 超时后，idalib 的 python worker 子进程可能变成孤儿，咬着 `.id0`/`.id1`/`.nam` 不放 - 后续任何工具或手动拖入 IDA GUI 都会报"权限不足" - **禁止** `taskkill /F /T` 杀进程树——`/T` 会把 GUI `ida.exe` 子进程一起干掉 - - **解决办法**：`start.ps1` 只在端口无人监听、或 `tools/list` 快速返回但缺 `py_eval`（旧 supervisor）时替换 managed supervisor；RPC 超时且 13337 仍在听视为忙，不杀 + - **解决办法**：`start.ps1` 只在端口无人监听、或 `tools/list` 快速返回但缺 `py_eval`（旧 supervisor）时替换 managed supervisor；RPC 超时且 13337 仍在听、且进程是 GUI 或开库未满 3 分钟，视为忙，不杀 + - **死锁例外**：managed supervisor 超过 3 分钟仍答不上 `tools/list` 时，`watchdog.ps1` / `start.ps1` 用 `-Force` 只替换 supervisor，仍不杀 `ida.exe` - **兜底**：`open.ps1` 检测到旧库被锁自动复制到 Temp 并加 GUID 前缀 8. **带自动分析打开看起来像卡死** @@ -69,7 +70,12 @@ description: | - Cursor/Claude 的 `type: http` 不会代为拉起进程；旧计划任务只在登录时跑一次 - `pythonw` 无控制台，崩溃时 Application 日志也是空的 - **解决办法**：`start.ps1` 默认健康则复用；`watchdog.ps1` 每分钟巡检；日志在 `%LOCALAPPDATA%\reverse-skill\ida-mcp\` - - 安装：`scripts/install-autostart.ps1`。Cursor 若启动时端口还没起来，仍需在 MCP 面板手动刷新一次 + - 安装：`scripts/install-autostart.ps1`。HTTP 客户端若启动时端口还没起来，仍需在 MCP 面板手动刷新一次 + +10. **Streamable HTTP GET `/mcp` 会卡住单线程 supervisor** + - 部分 HTTP MCP 客户端会对 `/mcp` 发长连接 GET（SSE）。stock `idalib_supervisor` 用 `background=False` 的 `HTTPServer`，一次只处理一个请求 + - 结果：`tools/list` 超时，客户端把 `idapro` 标成 error + - **解决办法**：`run-supervisor.py` 把 HTTP 换成 `ThreadingHTTPServer` 并接受 GET `/mcp`；补丁失败则跳过并仍启动 supervisor。卡住时用 `scripts/recover.ps1`（立刻 `-Force`） ### 工作流程原则 @@ -89,8 +95,9 @@ description: | - 自动解析 `IDADIR`（环境变量 / 便携版桌面路径 / 常见安装路径） - 优先用 IDA 自带 `Python314\python.exe -m ida_pro_mcp.idalib_supervisor` - 默认先探测 `http://127.0.0.1:13337/mcp`，健康则输出 `OK::reuse` 并退出 -- 13337 在听但 `tools/list` 超时 → `WARN:busy` / `OK:busy:reuse`，**不杀**（supervisor 单线程，开库时无法回包） -- 仅在端口无人监听、或快速返回且缺 `py_eval` 时替换 managed supervisor；**永不杀 `ida.exe`，不用 `taskkill /T`** +- 13337 在听但 `tools/list` 超时 → `WARN:busy` / `OK:busy:reuse`，**不杀**（开库或 GUI 占用时无法回包） +- managed supervisor **超过 3 分钟**仍答不上 `tools/list` → 视为死锁，输出 `INFO:deadlock` 并 `-Force` 替换 supervisor +- 仅在端口无人监听、缺 `py_eval`、或上述死锁时替换 managed supervisor；**永不杀 `ida.exe`，不用 `taskkill /T`** - GUI 占用 13337 时输出 `WARN:gui_busy` 并退出，不另起 supervisor - 成功输出 `OK:<工具数>`（当前约 66），失败输出 `ERR:timeout` - supervisor 日志：`%LOCALAPPDATA%\reverse-skill\ida-mcp\supervisor.log` @@ -101,9 +108,10 @@ description: | powershell -File "\ida-reverse\scripts\start.ps1" ``` -### watchdog.ps1 / install-autostart.ps1 — 保活 +### watchdog.ps1 / recover.ps1 / install-autostart.ps1 — 保活 -- `watchdog.ps1`：探测 13337，健康则 `OK::reuse`，挂了才调用 `start.ps1` +- `watchdog.ps1`：探测 13337；健康 reuse；GUI/新开库 busy reuse；managed supervisor 超过 3 分钟仍无 `tools/list` 则 `start.ps1 -Force` +- `recover.ps1`：立刻 `start.ps1 -Force`（不杀 `ida.exe`）。HTTP 客户端把 `idapro` 标成 error 时用这个 - `install-autostart.ps1`：注册计划任务 `reverse-skill-ida-mcp`（登录 + 每分钟） - 日志：`%LOCALAPPDATA%\reverse-skill\ida-mcp\watchdog.log` diff --git a/skills/ida-reverse/scripts/recover.ps1 b/skills/ida-reverse/scripts/recover.ps1 new file mode 100644 index 0000000..24ea719 --- /dev/null +++ b/skills/ida-reverse/scripts/recover.ps1 @@ -0,0 +1,20 @@ +<# +.SYNOPSIS +Force-restart IDA Pro MCP HTTP (127.0.0.1:13337). + +.DESCRIPTION +Use when Cursor user-idapro is error / tools/list hangs. +Never kills ida.exe. Replaces only the managed supervisor. + +Usage: + powershell -File recover.ps1 +#> +param( + [int]$Port = 13337, + [int]$WaitSeconds = 45 +) + +$ErrorActionPreference = 'Stop' +$startScript = Join-Path $PSScriptRoot 'start.ps1' +& $startScript -Port $Port -Force -WaitSeconds $WaitSeconds +exit $LASTEXITCODE diff --git a/skills/ida-reverse/scripts/run-supervisor.py b/skills/ida-reverse/scripts/run-supervisor.py index 7fd92ce..e73a1be 100644 --- a/skills/ida-reverse/scripts/run-supervisor.py +++ b/skills/ida-reverse/scripts/run-supervisor.py @@ -37,10 +37,94 @@ def main() -> None: flush=True, ) + try: + _patch_streamable_http() + except Exception as exc: + print(f"WARN: streamable-http patch skipped: {exc}", flush=True) + from ida_pro_mcp.idalib_supervisor import main as supervisor_main supervisor_main() +def _patch_streamable_http() -> None: + """HTTP MCP clients use Streamable HTTP on /mcp. + + Stock idalib_supervisor serves with background=False → HTTPServer + (one request at a time). A GET /sse or GET /mcp that stays open + then makes tools/list hang, and the client marks the server error. + GET /mcp also returned 405, which breaks live tool discovery. + """ + import select + import socket + import sys + import time + import uuid + from urllib.parse import urlparse + + # Vendored zeromcp lives under site-packages/ida_pro_mcp/ida_mcp. + # Do not import ida_pro_mcp.ida_mcp (that pulls idaapi). + import ida_pro_mcp.idalib_supervisor as _sup + + zm_dir = Path(_sup.__file__).resolve().parent / "ida_mcp" + sys.path.insert(0, str(zm_dir)) + try: + import zeromcp.mcp as zm # type: ignore + finally: + sys.path.remove(str(zm_dir)) + + zm.HTTPServer = zm.ThreadingHTTPServer + + orig_get = zm.McpHttpRequestHandler.do_GET + + def do_get(self): + if not self._check_api_request(): + return + if urlparse(self.path).path != "/mcp": + return orig_get(self) + + session_id = self.headers.get("Mcp-Session-Id") or str(uuid.uuid4()) + self.mcp_server.register_http_session(session_id) + conn = zm._McpSseConnection(self.wfile) + self.mcp_server._sse_connections[conn.session_id] = conn + try: + self.send_response(200) + self.send_header("Content-Type", "text/event-stream") + self.send_header("Cache-Control", "no-cache") + self.send_header("Connection", "keep-alive") + self.send_header("Mcp-Session-Id", session_id) + self.send_cors_headers() + self.end_headers() + sock = self.connection + if sock and hasattr(sock, "settimeout"): + try: + sock.settimeout(1.0) + except OSError: + pass + last_ping = time.time() + while conn.alive and self.mcp_server._running: + now = time.time() + if sock: + try: + readable, _, _ = select.select([sock], [], [], 1.0) + if readable and sock.recv(1, socket.MSG_PEEK) == b"": + break + except (OSError, socket.error, ConnectionResetError, BrokenPipeError): + break + else: + time.sleep(1.0) + if now - last_ping >= 30: + try: + conn.send_event("ping", "") + except (OSError, BrokenPipeError): + break + last_ping = now + finally: + conn.alive = False + self.mcp_server._sse_connections.pop(conn.session_id, None) + + zm.McpHttpRequestHandler.do_GET = do_get + + if __name__ == "__main__": main() diff --git a/skills/ida-reverse/scripts/start.ps1 b/skills/ida-reverse/scripts/start.ps1 index 4d0b6f8..7132bd6 100644 --- a/skills/ida-reverse/scripts/start.ps1 +++ b/skills/ida-reverse/scripts/start.ps1 @@ -351,9 +351,23 @@ if (-not $Force) { exit 0 } if ($probe.Status -eq 'busy') { - Write-Output ("WARN:busy:port={0} pid={1}" -f $Port, ($probe.Owners -join ',')) - Write-Output 'HINT: 13337 is listening but tools/list timed out (likely idb_open). Not killing supervisor.' - exit 0 + $deadlock = $false + $now = Get-Date + foreach ($ownerPid in @($probe.Owners)) { + if (Test-IdaGuiProcess -ProcessId $ownerPid) { continue } + if (-not (Test-ManagedSupervisorProcess -ProcessId $ownerPid)) { continue } + $proc = Get-IdaMcpProcessInfo -ProcessId $ownerPid + if ($proc -and $proc.CreationDate -and (($now - $proc.CreationDate).TotalSeconds -ge 180)) { + $deadlock = $true + break + } + } + if (-not $deadlock) { + Write-Output ("WARN:busy:port={0} pid={1}" -f $Port, ($probe.Owners -join ',')) + Write-Output 'HINT: 13337 is listening but tools/list timed out (likely idb_open). Not killing supervisor.' + exit 0 + } + Write-Output ("INFO:deadlock:port={0} pid={1} (tools/list dead >180s, replacing)" -f $Port, ($probe.Owners -join ',')) } } diff --git a/skills/ida-reverse/scripts/watchdog.ps1 b/skills/ida-reverse/scripts/watchdog.ps1 index 02f2933..8ad185f 100644 --- a/skills/ida-reverse/scripts/watchdog.ps1 +++ b/skills/ida-reverse/scripts/watchdog.ps1 @@ -1,10 +1,12 @@ <# .SYNOPSIS -Ensure IDA Pro MCP HTTP is healthy; start it only when down. +Ensure IDA Pro MCP HTTP is healthy; start or replace it when needed. .DESCRIPTION One-shot health check used by the scheduled task. Safe to run every minute: -a live server is reused, a dead listener is replaced via start.ps1. +a live server is reused; GUI / fresh idb_open busy is reused; a managed +supervisor older than 3 minutes that cannot answer tools/list is replaced +via start.ps1 -Force; a dead listener is started via start.ps1. Usage: powershell -File watchdog.ps1 @@ -64,12 +66,54 @@ function Probe-IdaMcp { return @{ Status = 'stale'; Count = $count } } } catch {} - # Listen + RPC timeout = busy (single-threaded supervisor during idb_open). - # Only "nothing listening" or a quick stale (no py_eval) list is down. + # Listen + RPC timeout: GUI or a fresh supervisor during idb_open = busy. + # A managed supervisor older than 3 minutes that cannot answer tools/list + # is a deadlock (Streamable HTTP GET /mcp holding single-thread HTTPServer). if ($owners.Count -gt 0) { - return @{ Status = 'busy'; Count = 0 } + return @{ Status = 'busy'; Count = 0; Owners = $owners } } - return @{ Status = 'down'; Count = 0 } + return @{ Status = 'down'; Count = 0; Owners = @() } +} + +function Get-IdaMcpProcessInfo { + param([int]$ProcessId) + return Get-CimInstance Win32_Process -Filter "ProcessId=$ProcessId" -ErrorAction SilentlyContinue +} + +function Test-IdaGuiProcess { + param([int]$ProcessId) + $proc = Get-IdaMcpProcessInfo -ProcessId $ProcessId + if (-not $proc) { return $false } + return [string]$proc.Name -match '(?i)^(ida|ida64|idaq|idaq64)\.exe$' +} + +function Test-ManagedSupervisorProcess { + param([int]$ProcessId) + $proc = Get-IdaMcpProcessInfo -ProcessId $ProcessId + if (-not $proc) { return $false } + $name = [string]$proc.Name + $cmd = [string]$proc.CommandLine + if ($name -match '(?i)^(ida|ida64|idaq|idaq64)\.exe$') { return $false } + if ($name -match '(?i)^(python|pythonw|cmd)\.exe$') { + return $cmd -match '(?i)(run-supervisor\.py|idalib_supervisor|idalib-mcp|ida-pro-mcp)' + } + return $name -match '(?i)^(idalib-mcp|ida-pro-mcp|idalib_supervisor)' +} + +function Test-DeadlockedSupervisor { + param([int[]]$Owners, [int]$MaxAgeSeconds = 180) + $gui = @($Owners | Where-Object { Test-IdaGuiProcess -ProcessId $_ }) + if ($gui.Count -gt 0) { return $false } + $now = Get-Date + $managed = @($Owners | Where-Object { Test-ManagedSupervisorProcess -ProcessId $_ }) + if ($managed.Count -eq 0) { return $false } + foreach ($pid in $managed) { + $proc = Get-IdaMcpProcessInfo -ProcessId $pid + if (-not $proc -or -not $proc.CreationDate) { continue } + $age = ($now - $proc.CreationDate).TotalSeconds + if ($age -ge $MaxAgeSeconds) { return $true } + } + return $false } $probe = Probe-IdaMcp -Port $Port @@ -78,6 +122,20 @@ if ($probe.Status -eq 'healthy') { exit 0 } if ($probe.Status -eq 'busy') { + $owners = @($probe.Owners) + if (Test-DeadlockedSupervisor -Owners $owners) { + Write-WatchLog ("INFO:deadlock pids={0}, calling start.ps1 -Force" -f ($owners -join ',')) + $startScript = Join-Path $PSScriptRoot 'start.ps1' + $startOutput = & $startScript -Port $Port -Force + foreach ($line in @($startOutput)) { + Write-WatchLog "START:$line" + } + $last = (@($startOutput) | Where-Object { $_ -match '^(OK|ERR|WARN):' } | Select-Object -Last 1) + if ($last -match '^(OK:|WARN:gui_busy|WARN:busy)') { + exit 0 + } + exit 1 + } Write-WatchLog 'OK:busy:reuse' exit 0 } diff --git a/skills/scripts/test-parse-contracts.ps1 b/skills/scripts/test-parse-contracts.ps1 index bada38c..7ec57ad 100644 --- a/skills/scripts/test-parse-contracts.ps1 +++ b/skills/scripts/test-parse-contracts.ps1 @@ -46,6 +46,70 @@ try { Remove-Item -Recurse -Force $tmp -ErrorAction SilentlyContinue } +$idaDir = Join-Path $skillsRoot 'ida-reverse/scripts' +$idaFiles = @{ + 'start.ps1' = (Join-Path $idaDir 'start.ps1') + 'watchdog.ps1' = (Join-Path $idaDir 'watchdog.ps1') + 'recover.ps1' = (Join-Path $idaDir 'recover.ps1') + 'run-supervisor.py' = (Join-Path $idaDir 'run-supervisor.py') +} +foreach ($name in @($idaFiles.Keys)) { + $p = $idaFiles[$name] + if (-not (Test-Path -LiteralPath $p)) { Bad ("missing {0}" -f $name); continue } + if ($name -eq 'run-supervisor.py') { continue } + $errs = $null + $tokens = $null + $null = [System.Management.Automation.Language.Parser]::ParseFile($p, [ref]$tokens, [ref]$errs) + if ($errs -and $errs.Count -gt 0) { + Bad ("parse fail {0}: {1}" -f $name, $errs[0].Message) + } else { + Ok ("parse {0}" -f $name) + } +} + +$recover = Get-Content -LiteralPath $idaFiles['recover.ps1'] -Raw -ErrorAction SilentlyContinue +if ($recover -match '(?i)start\.ps1' -and $recover -match '-Force') { + Ok 'recover.ps1 calls start.ps1 -Force' +} else { + Bad 'recover.ps1 must invoke start.ps1 -Force' +} +if ($recover -match '(?im)^\s*(&\s*)?taskkill(\.exe)?\s+[^\n]*/T\b') { + Bad 'recover.ps1 must not taskkill /T' +} else { + Ok 'recover.ps1 has no taskkill /T' +} + +$start = Get-Content -LiteralPath $idaFiles['start.ps1'] -Raw +if ($start -match 'TotalSeconds\s+-ge\s+180' -and $start -match 'INFO:deadlock') { + Ok 'start.ps1 deadlock replace after 180s' +} else { + Bad 'start.ps1 missing 180s deadlock replace' +} +if ($start -match '(?im)^\s*(&\s*)?taskkill(\.exe)?\s+[^\n]*/T\b') { + Bad 'start.ps1 must not taskkill /T' +} else { + Ok 'start.ps1 has no taskkill /T' +} + +$watch = Get-Content -LiteralPath $idaFiles['watchdog.ps1'] -Raw +if ($watch -match 'Test-DeadlockedSupervisor' -and $watch -match '-Force') { + Ok 'watchdog.ps1 Force-replaces deadlocked supervisor' +} else { + Bad 'watchdog.ps1 missing deadlock -Force path' +} + +$py = Get-Content -LiteralPath $idaFiles['run-supervisor.py'] -Raw +if ($py -match 'ThreadingHTTPServer' -and $py -match '_patch_streamable_http') { + Ok 'run-supervisor.py patches ThreadingHTTPServer' +} else { + Bad 'run-supervisor.py missing streamable-http patch' +} +if ($py -match 'streamable-http patch skipped') { + Ok 'run-supervisor.py skips a failed HTTP patch' +} else { + Bad 'run-supervisor.py must fail-open when the HTTP patch errors' +} + if ($fail.Count -gt 0) { Write-Host ("FAILED {0}" -f $fail.Count) -ForegroundColor Red $fail | ForEach-Object { Write-Host " - $_" }