fix(debugger): report debugger process exit instead of timing out (#115)

## What this changes

When the debugger process exits, the session now notices at once and
raises
`DebuggerExitedError` with the exit code and the debugger's last output,
instead
of waiting out the full timeout and then reporting a timeout.

- `debug_session.py`: when the reader thread reaches EOF, it marks the
session as
exited, publishes the last output it collected, and sets `ready_event`
so
waiters wake up. Every marker waiter (`_send_marked`,
`_wait_for_prompt`,
`_wait_for_break_locked`) and every stdin write checks for the exit and
raises
`DebuggerExitedError`, a `DebuggerError` subclass, so the server's
existing
error handling is unchanged. The probes (`_still_running`,
`_resume_took_effect`,
`_break_in_and_resync`) pass this error through rather than treating it
as
"target still running". Once the debugger has exited, `send_command`
fails fast
  and says to close the session and open a new one.
- `kd_session.py`: a new `_on_debugger_exit` hook releases the
connect-banner
wait, so kd exiting before it connects is reported at once with kd's own
  message.
- Nonzero exit codes are shown in hex (e.g. `0xC0000005`, `0x8007274C`),
since
  they are usually NTSTATUS/HRESULT values.

## Why

The reader loop returned silently at EOF, and nothing woke the waiting
thread.
So whenever cdb/kd exited - `q`, `.kill` on a dump, a failed `-remote`
or `-k`
connect, or the debugger itself crashing - every pending call sat out
its whole
timeout (up to 300s for `wait_for_break`) and then reported the wrong
cause:

- `run_cdb_command("q")` -> `Command timed out after N seconds: q
(session may
  need a manual break-in)`
- `open_cdb_remote` with a mistyped server -> `Debugger initialization
timed out`,
  and cdb's actual reason was thrown away
- `open_kd_session` with a bad `-k` string or a busy KDNET port -> a
connect
  timeout that hid kd's error

## How it was verified

Against real cdb (WinDbg 1.2606.22001.0, Store install):

| Case | Before | After |
|---|---|---|
| `q` on `DemoCrash1.exe.7088.dmp` | full timeout, then "timed out" |
fails at once: `Debugger process exited (exit code 0) while running
'q'.` |
| `open_cdb_remote` to an unreachable `tcp:Port=5999,Server=127.0.0.1` |
60s, then "Debugger initialization timed out" | 2.3s: `exit code
0x8007274C`, plus `DebugConnect failed, Win32 error 0n10060 ...` |

New hermetic tests, using the fake cdb/kd:

- the debugger exits during a command (`q`), and later commands fail
fast
- the debugger crashes mid-command; the error carries the exit status
and its last output
- the debugger exits during startup; the error carries its message
- `wait_for_break` ends when the debugger exits
- kd exits before the connect banner

Full suite: `uv run pytest src/mcp_windbg/tests/` - 116 passed, 2
skipped (the
kernel scenarios, since no `MCP_WINDBG_KERNEL_CONNECTION` target was
configured).

Co-authored-by: Sven Scharmentke <sven.scharmentke@liasoft.de>
This commit is contained in:
Ramana Reddy Polaka
2026-09-28 15:45:57 -07:00
committed by GitHub
parent c8e2355696
commit 5bf4213313
5 changed files with 222 additions and 24 deletions
+5
View File
@@ -19,6 +19,11 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
output already read, so the timeout error still shows useful diagnostics from
runaway commands such as SOS `!clrstack`. The retained output is capped at
2,000 lines and 64 KiB (#112, #114). Thanks to @Oscar-Williams for the fix.
- A debugger process that exits (`q`, a failed `-remote` or `-k` connect, or
cdb/kd crashing) is now reported at once with its exit code and last output,
instead of every pending command, `wait_for_break`, or session open sitting out
its full timeout and then misreporting a timeout (#115). Thanks to
@RamanaIntel for the fix.
## [1.3.0] - 2026-09-10
+94 -18
View File
@@ -24,6 +24,10 @@ Two robustness properties this base guarantees:
cannot answer, so the command would "time out" and the resulting CTRL+BREAK
would halt the target we just released. They are written bare instead, and
``wait_for_break`` picks up whatever the target prints when it does stop.
- **Exit is noticed, not timed out.** When the debugger process exits (``q``, a
failed ``-remote``/``-k`` connect, cdb itself crashing), the reader sees EOF
and wakes whoever is waiting, who reports the exit status and the debugger's
last output instead of sitting out the timeout and blaming the command.
"""
from __future__ import annotations
@@ -71,6 +75,9 @@ BREAK_IN_PROBE_TIMEOUT = 2
# headroom for a loaded link.
RESUME_CONFIRM_TIMEOUT = 0.5
# How many of the debugger's final output lines an exit error carries.
_EXIT_TAIL_LINES = 20
class DebuggerError(Exception):
"""Raised for any debugger session failure (launch, timeout, I/O)."""
@@ -80,6 +87,10 @@ class DebuggerError(Exception):
self.partial_output = list(partial_output or [])
class DebuggerExitedError(DebuggerError):
"""The debugger process exited; the session cannot be used again."""
class _ReleaseOnExit:
"""Context manager that releases an already-acquired lock on exit."""
@@ -260,6 +271,8 @@ class DebuggerSession:
self._target_running = False
#: Set by shutdown so a parked wait stops rather than outliving the session.
self._closing = False
#: Set by the reader at EOF: the debugger is gone and no marker can land.
self._debugger_exited = False
try:
creationflags = 0
@@ -301,6 +314,9 @@ class DebuggerSession:
"""Reach the first usable prompt. Overridden by kernel sessions."""
try:
self._wait_for_prompt(self.timeout)
except DebuggerExitedError:
self.shutdown()
raise
except DebuggerError:
self.shutdown()
raise DebuggerError("Debugger initialization timed out")
@@ -309,6 +325,10 @@ class DebuggerSession:
"""Called (under ``self.lock``) for every output line. Kernel uses it
to notice the ``Connected to target`` banner."""
def _on_debugger_exit(self) -> None:
"""Called by the reader once the debugger's output ends. Kernel uses it
to stop waiting for a connect banner that can no longer come."""
# -- Reader thread ----------------------------------------------------
def _read_output(self) -> None:
@@ -343,6 +363,55 @@ class DebuggerSession:
except (IOError, ValueError, AttributeError) as e:
if self.verbose:
print(f"Debugger output reader error: {e}")
finally:
# Publish what the debugger printed last - often the only record of
# why it exited - and wake any waiter rather than let it time out.
with self.lock:
self._debugger_exited = True
self.output_lines = buffer
self._on_debugger_exit()
self.ready_event.set()
def _exited_message(self, doing: str, tail: List[str]) -> str:
process = self.process
code = None
if process is not None:
try:
code = process.wait(timeout=2)
except Exception:
code = process.poll()
if code is None:
status = ""
elif code == 0:
status = " (exit code 0)"
else:
# Windows exit codes are often NTSTATUS values, e.g. 0xC0000005.
status = f" (exit code 0x{code & 0xFFFFFFFF:08X})"
message = f"Debugger process exited{status} {doing}."
if tail:
message += "\nLast debugger output:\n" + "\n".join(tail[-_EXIT_TAIL_LINES:])
return message
def _raise_if_exited(self, doing: str) -> None:
"""Raise DebuggerExitedError if the debugger exited with the current
marker still pending - i.e. it can never land."""
with self.lock:
if not self._debugger_exited or self._expected_marker is None:
return
self._expected_marker = None
tail, self.output_lines = self.output_lines, []
raise DebuggerExitedError(self._exited_message(doing, tail))
def _write_input(self, text: str, doing: str, failure: str) -> None:
"""Write to the debugger's stdin, reporting an exit rather than a bare
I/O error when the write failed because the process is gone."""
try:
self.process.stdin.write(text)
self.process.stdin.flush()
except (IOError, ValueError, AttributeError) as e:
self.reader_thread.join(timeout=1) # let the reader observe EOF
self._raise_if_exited(doing)
raise DebuggerError(f"{failure}: {e}")
# -- Command protocol -------------------------------------------------
@@ -430,13 +499,13 @@ class DebuggerSession:
self.ready_event.clear()
with self.lock:
self._expected_marker = marker
try:
self.process.stdin.write(f".echo {marker}\n")
self.process.stdin.flush()
except (IOError, ValueError, AttributeError) as e:
raise DebuggerError(f"Failed to communicate with debugger: {e}")
doing = "before reaching a prompt"
self._raise_if_exited(doing)
self._write_input(f".echo {marker}\n", doing, "Failed to communicate with debugger")
if not self.ready_event.wait(timeout or self.timeout):
landed = self.ready_event.wait(timeout or self.timeout)
self._raise_if_exited(doing)
if not landed:
raise DebuggerError("Timed out waiting for debugger prompt")
#: Commands that hand the target back its CPU and do not return to a prompt
@@ -521,6 +590,11 @@ class DebuggerSession:
"""
if not self.process:
raise DebuggerError("Debugger process is not running")
if self._debugger_exited:
raise DebuggerExitedError(
"Debugger process has exited; this session can no longer be used. "
"Close it and open a new one."
)
cmd_timeout = timeout or self.timeout
@@ -545,16 +619,14 @@ class DebuggerSession:
with self.lock:
self.output_lines = []
self._expected_marker = marker
try:
self.process.stdin.write(f"{command}\n.echo {marker}\n")
self.process.stdin.flush()
except (IOError, ValueError, AttributeError) as e:
raise DebuggerError(f"Failed to send command: {e}")
doing = f"while running '{command}'"
self._raise_if_exited(doing)
self._write_input(f"{command}\n.echo {marker}\n", doing, "Failed to send command")
landed = self.ready_event.wait(cmd_timeout)
if self._closing:
raise DebuggerError("Session was closed while the command was running")
self._raise_if_exited(doing)
if not landed and not self._marker_landed():
partial_output = self._snapshot_partial_output()
resynced = self._abort_running_command()
@@ -764,6 +836,8 @@ class DebuggerSession:
"""
try:
self._wait_for_prompt(RESUME_CONFIRM_TIMEOUT)
except DebuggerExitedError:
raise
except DebuggerError:
return not self._abandon_marker()
return False
@@ -792,6 +866,8 @@ class DebuggerSession:
raise DebuggerError(f"Failed to break into the running target: {e}")
try:
self._wait_for_prompt(min(10, max(3, self.timeout)))
except DebuggerExitedError:
raise
except DebuggerError:
if not self._abandon_marker():
raise DebuggerError(
@@ -812,6 +888,8 @@ class DebuggerSession:
"""
try:
self._wait_for_prompt(BREAK_IN_PROBE_TIMEOUT)
except DebuggerExitedError:
raise
except DebuggerError:
return not self._abandon_marker()
return False
@@ -851,17 +929,15 @@ class DebuggerSession:
with self.lock:
self.output_lines = []
self._expected_marker = marker
try:
self.process.stdin.write(f".echo {marker}\n")
self.process.stdin.flush()
except (IOError, ValueError, AttributeError) as e:
raise DebuggerError(f"Failed to communicate with debugger: {e}")
doing = "while waiting for the target to stop"
self._raise_if_exited(doing)
self._write_input(f".echo {marker}\n", doing, "Failed to communicate with debugger")
wait = timeout or DEFAULT_WAIT_FOR_BREAK_TIMEOUT
landed = self.ready_event.wait(wait)
if self._closing:
raise DebuggerError("Session was closed while waiting for the target to stop")
self._raise_if_exited(doing)
if not landed and not self._abandon_marker():
raise DebuggerError(
f"Target did not stop within {wait} seconds and is still running. "
+13
View File
@@ -25,6 +25,7 @@ from typing import Optional
from .debug_session import (
DebuggerError,
DebuggerExitedError,
DebuggerSession,
build_debugger_args,
find_executable,
@@ -122,6 +123,9 @@ class KDSession(DebuggerSession):
if any(banner in line for banner in KERNEL_CONNECTED_BANNERS):
self._connected_event.set()
def _on_debugger_exit(self) -> None:
self._connected_event.set()
def _startup(self) -> None:
"""Wait for the target to connect, break in, then reach the prompt."""
if not self._connected_event.wait(self.timeout):
@@ -131,6 +135,12 @@ class KDSession(DebuggerSession):
"Is the target booted with debugging enabled and transmitting "
"on this transport? (kd reports 'no_debuggee' until it is.)"
)
if self._debugger_exited:
message = self._exited_message(
"before the kernel target connected", self._take_output()
)
self.shutdown()
raise DebuggerExitedError(message)
# Target is connected but running; break in to reach a prompt.
try:
self.process.send_signal(signal.CTRL_BREAK_EVENT)
@@ -139,6 +149,9 @@ class KDSession(DebuggerSession):
raise KDError(f"Failed to break into the kernel target: {e}")
try:
self._wait_for_prompt(self.timeout)
except DebuggerExitedError:
self.shutdown()
raise
except DebuggerError:
self.shutdown()
raise KDError("Kernel debugger did not reach a prompt after break-in")
+84 -3
View File
@@ -19,7 +19,7 @@ import time
import pytest
from mcp_windbg import debug_session
from mcp_windbg.debug_session import DebuggerError, DebuggerSession
from mcp_windbg.debug_session import DebuggerError, DebuggerExitedError, DebuggerSession
_STOP = object()
@@ -82,6 +82,7 @@ class _FakeProc:
self.stdout = _FakeStdout(self)
self._alive = True
self.pid = 4321
self.returncode = 0
self.signals: list = []
self.running = False
self._queued: list = []
@@ -149,8 +150,17 @@ class _FakeProc:
for line in queued:
self._handle(line)
def exit_with(self, *lines: str, code: int = 0):
"""The debugger prints *lines* and exits on its own - a failed -remote
connect, a bad -k string, or cdb itself crashing."""
for line in lines:
self._out.put(line)
self.returncode = code
self._alive = False
self._out.put(_STOP)
def poll(self):
return None if self._alive else 0
return None if self._alive else self.returncode
def send_signal(self, sig):
self.signals.append(sig)
@@ -162,7 +172,7 @@ class _FakeProc:
self._out.put(_STOP)
def wait(self, timeout=None):
return 0
return self.returncode
class _Session(DebuggerSession):
@@ -258,6 +268,77 @@ def test_partial_timeout_output_has_line_and_size_limits(make_session):
assert lines[-1].startswith("[partial output truncated;")
# -- debugger process exit ------------------------------------------------
#
# Once cdb/kd exits, no marker can ever arrive. Every waiter used to sit out its
# full timeout and then blame the command ("timed out"), hiding the real cause.
def test_a_command_that_ends_the_debugger_fails_fast_not_on_timeout(make_session):
session, _ = make_session(timeout=30)
began = time.monotonic()
with pytest.raises(DebuggerExitedError) as exc:
session.send_command("q")
assert time.monotonic() - began < 5
assert "exited" in str(exc.value)
assert "timed out" not in str(exc.value).lower()
with pytest.raises(DebuggerExitedError, match="open a new one"):
session.send_command("k")
def test_a_debugger_crash_mid_command_reports_the_exit_status(make_session):
session, proc = make_session(timeout=30)
proc._swallow = True # the command never completes on its own
timer = threading.Timer(
0.1, lambda: proc.exit_with("[ComMethodFrame: 000000c0]", code=0xC0000005)
)
timer.start()
began = time.monotonic()
try:
with pytest.raises(DebuggerExitedError) as exc:
session.send_command("!clrstack")
finally:
timer.cancel()
assert time.monotonic() - began < 5
assert "0xC0000005" in str(exc.value)
# What it printed last is often the only clue to why it died.
assert "[ComMethodFrame: 000000c0]" in str(exc.value)
def test_a_debugger_that_exits_during_startup_reports_its_last_words(monkeypatch):
"""A mistyped -remote string makes cdb print why and exit at once; the
caller should see that, not 'initialization timed out' a minute later."""
proc = _FakeProc()
proc.exit_with(
"Unable to connect to server 'tcp:Port=5005,Server=nohost'",
"Win32 error 0n10061",
code=1,
)
monkeypatch.setattr(debug_session.subprocess, "Popen", lambda *a, **k: proc)
began = time.monotonic()
with pytest.raises(DebuggerExitedError) as exc:
_LiveSession(debugger_path="fake", launch_args=["fake"], timeout=30, verbose=False)
assert time.monotonic() - began < 5
assert "Unable to connect to server" in str(exc.value)
assert "initialization timed out" not in str(exc.value)
def test_wait_for_break_ends_when_the_debugger_exits(make_session):
session, proc = make_session(live=True)
session.send_command("g")
timer = threading.Timer(0.1, lambda: proc.exit_with("Server went away", code=1))
timer.start()
began = time.monotonic()
try:
with pytest.raises(DebuggerExitedError) as exc:
session.wait_for_break(timeout=30)
finally:
timer.cancel()
assert time.monotonic() - began < 5
assert "Server went away" in str(exc.value)
# -- go-class commands ----------------------------------------------------
#
# The bug these cover: 'g' hands the CPU to the target, after which the debugger
+26 -3
View File
@@ -16,10 +16,12 @@ from __future__ import annotations
import os
import queue
import signal
import time
import pytest
from mcp_windbg import debug_session, kd_session
from mcp_windbg.debug_session import DebuggerExitedError
from mcp_windbg.kd_session import KDError, KDSession
_STOP = object()
@@ -67,9 +69,10 @@ class _FakeKd:
Args:
connect: emit the ``Connected to target`` banner (as a live target does).
answer_markers: echo ``.echo <marker>`` back, i.e. reach a prompt.
exit_lines: print these and exit at once, as kd does on a bad ``-k``.
"""
def __init__(self, *, connect: bool = True, answer_markers: bool = True):
def __init__(self, *, connect: bool = True, answer_markers: bool = True, exit_lines=None):
self._out: "queue.Queue" = queue.Queue()
self._answer_markers = answer_markers
self.stdin = _FakeStdin(self)
@@ -81,6 +84,10 @@ class _FakeKd:
self._out.put(
"Connected to target 172.16.2.189 on port 50005 on local IP 172.16.2.183."
)
if exit_lines:
for line in exit_lines:
self._out.put(line)
self.terminate()
def _feed(self, text: str):
for line in text.splitlines():
@@ -113,8 +120,8 @@ def launch(monkeypatch):
procs: list[_FakeKd] = []
launched: list[list[str]] = []
def _factory(*, connect=True, answer_markers=True, **kwargs):
proc = _FakeKd(connect=connect, answer_markers=answer_markers)
def _factory(*, connect=True, answer_markers=True, exit_lines=None, **kwargs):
proc = _FakeKd(connect=connect, answer_markers=answer_markers, exit_lines=exit_lines)
procs.append(proc)
def _popen(args, **_):
@@ -176,6 +183,22 @@ def test_startup_raises_when_prompt_never_arrives_after_break_in(launch):
assert "did not reach a prompt" in str(exc.value).lower()
def test_startup_fails_fast_when_kd_exits_before_connecting(launch):
"""A bad -k string or a busy KDNET port makes kd print why and exit. That
must surface at once, not as a connect timeout that hides kd's message."""
factory, _, _ = launch
began = time.monotonic()
with pytest.raises(DebuggerExitedError) as exc:
factory(
connect=False,
exit_lines=["KDNET: failed to bind to port 50005 - address already in use"],
timeout=30,
)
assert time.monotonic() - began < 5
assert "before the kernel target connected" in str(exc.value)
assert "failed to bind to port 50005" in str(exc.value)
def test_init_without_connection_string_raises():
with pytest.raises(ValueError, match="kernel_connection must be provided"):
KDSession(kernel_connection="")