Skip to content

feat(runstore): stream run progress over Server-Sent Events - #925

Open
HugoXK wants to merge 10 commits into
ThinkFlowLab:mainfrom
HugoXK:agent/922-runstore-sse
Open

HugoXK wants to merge 10 commits into
ThinkFlowLab:mainfrom
HugoXK:agent/922-runstore-sse

Conversation

@HugoXK

@HugoXK HugoXK commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #922

What changed

Adds GET /api/runs/<run_id>/events to the existing read-only run-store
server. It is a Server-Sent Events stream:

  • an initial status event from status.json;
  • one round event per new rounds.jsonl line, with the round index as the
    event id, so a reconnect can send Last-Event-ID and resume after the last
    round it saw;
  • a final status event when the run reaches done, failed or cancelled,
    then the stream closes;
  • retry: 1000 and periodic : keep-alive comments;
  • the same loopback-only, read-only, loopback-CORS boundary as the JSON
    endpoints, and the same run-id traversal checks.

The stream polls rounds.jsonl (default every EVENT_POLL_SECONDS = 0.5) and
does not add a write path, an auth change or a background thread to the run
itself. serve_http(..., event_poll=...) is a wall-clock knob, used by the
tests to keep them fast.

Verification at the tip

Tip SHA: 900f69a

Ran:

  • pytest -q tests/test_run_events.py tests/test_run_panel.py → 18 passed
  • python -m tools.gen_api_docs --check → up to date
  • pytest -q → 5 failed, all reproduced on the base 1aaf1bd0 in this macOS
    environment before the change:
    test_dynamic_cheatsheet::test_the_shared_runner_really_is_ungated...,
    test_era_algotune::test_compilation_is_not_charged_wherever_the_author_put_it,
    test_gskill::test_a_memory_hungry_candidate_costs_its_score_not_the_run,
    test_integrations::test_the_launcher_install_writes_ignores_a_shadowing_cwd,
    test_porous_molecules::test_every_heavy_atom_gets_a_position...
    (CI runs Linux; these are environment-sensitive, not touched by this diff.)

Not run:

  • a browser EventSource client — the offline tests read the stream with
    http.client; the protocol is covered at the transport level, no browser is
    needed.

@HugoXK HugoXK added area:cli The agentdescent command line (cli.py, doctor, plan/evolve/status/apply) area:runstore Run store, ledger, checkpoint, resume and replay enhancement New feature or request labels Nov 10, 2026

@Birfy Birfy left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: CHANGES REQUESTED

Tip SHA: 900f69a
Ran (on a trial merge of this tip into main 1aaf1bd):

  • python3 .github/scripts/risk_gate.py --repo ThinkFlowLab/agentdescent --pr 925 → T3 (path agentdescent/runstore.py)
  • pytest -q tests/test_run_events.py tests/test_run_panel.py → 18 passed
  • pytest -q (full suite) → no failures through ~81% before my 600 s local timeout; CI test (3.9–3.13) still running at review time
    Not run: a browser EventSource client; the full suite to completion locally.

Blocking

  1. Unrelated CNAME file at the repo root (agentdescent.hugoxk.online). It isn't part of this feature and isn't in the commit 900f69a; it comes in from the fork's history (the Merge branch 'Birfy:main' into main merge on the fork's main). Merged into this repo, a root CNAME can repoint the project's GitHub Pages to a personal domain. Please drop it from the PR. The cleanest way is to branch from current ThinkFlowLab/agentdescent main (not the fork's main) and cherry-pick 900f69a.
  2. PR body still says Tip SHA: PENDING and pytest -q → PENDING. Please fill in the real tip and full-suite result once (1) is fixed.

Non-blocking (correctness/design)

  • A stalled run streams forever. If the run process dies without writing a terminal status, state stays running and the handler thread loops until the client disconnects. A cheap guard would be to emit a terminal status (or close) when status.json's pid is no longer alive, the same way status / list_runs detect a stale run, or to cap the stream with a max idle time so the client reconnects.
  • Each poll re-parses the whole rounds.jsonl. That's fine for tens of rounds, but tracking the byte offset (or at least only parsing lines past the last-seen count) would keep long runs O(new lines) per tick.
  • Status change and the rounds that caused it are emitted in status-then-rounds order. On the terminal tick, the final rounds are emitted after the status was read but before return, which is right. On a non-terminal state change, though, a client sees status before the rounds that preceded it. Consider reading rounds first, then status, so event order matches what happened.
  • Last-Event-ID with a negative or garbage value silently falls back to replay-all. That's reasonable, but a one-line test would pin it.
  • Keep-alive interval (15 s) is hard-coded while the poll interval is configurable; making it a module constant next to EVENT_POLL_SECONDS would let tests cover the keep-alive branch, which no test exercises today.
  • _events duplicates the run-id traversal check in do_GET; hoisting it into one helper would keep the two endpoints from drifting.

Good

The read-only, loopback-only CORS boundary matches the JSON endpoints. X-Accel-Buffering: no, retry: and the round-index ids are the right SSE details, and the tests drive a real server on an ephemeral port with a small event_poll. The docs change in docs/plugins.md merges cleanly onto main as a small paragraph, so it's fine as is.

Risk tier T3: this PR needs Danyang's explicit merge click even after a PASS; it won't be auto-merged.

@github-actions github-actions Bot added the risk:T3 Risk tier T3 (core loop, release, process; needs Danyang's click), set by the controller label Nov 10, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:cli The agentdescent command line (cli.py, doctor, plan/evolve/status/apply) area:runstore Run store, ledger, checkpoint, resume and replay enhancement New feature or request risk:T3 Risk tier T3 (core loop, release, process; needs Danyang's click), set by the controller

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(runstore): stream run progress over SSE

2 participants