Skip to content

feat: implement portable outbound HTTP and request lifecycle contracts - #275

Open
aram356 wants to merge 595 commits into
mainfrom
docs/outbound-http-spec
Open

aram356 wants to merge 595 commits into
mainfrom
docs/outbound-http-spec

Conversation

@aram356

@aram356 aram356 commented Jun 8, 2026 •

Copy link
Copy Markdown
Collaborator

PRfection phase: implementation

Current Review Checkpoint

Latest repair: 4cbabfee. Verification, separate issue replies and re-review handoff.

Issues #505 and #506 are implemented: effective Fastly endpoint validation now fails at manifest/macro boundaries, and managed setup failures before custom callbacks attempt canonical redacted 500 responses while preserving the original typed error and the no-second-send rule. Demo/generated examples are aligned.

The complete local prescribed suite and 35 additional workflow checks pass. Committed-head focused tests, strict Clippy and all six new Viceroy controls pass. The separate seven-case first-response assembly-attestation blocker and independent current-head acceptance remain open. Hosted/manual checkboxes are unchanged; no deployment, provider promotion or merge occurred. Fresh GitHub checks are separate.

Earlier sections below are historical checkpoints, not fresh validation or approval of this head.

Summary

Startup Fixture Validation Closeout (2026-11-09)

Latest pushed head: a7d7fd0f.
The test responder now owns its ephemeral listener continuously and publishes bounded PID/nonce/endpoint metadata before the first prescribed request. First-response attestation, sticky failures and the original absolute startup deadline remain intact. Production outbound behavior, hard-cut APIs, pins, limits and capability levels are unchanged.

  • Full prescribed suite: exit 0, including all four adapter CI suites, fresh generated-project compilation, demo tests and all three demo WASM checks.
  • Default-parallel verification: 2,700 workspace tests across 33 binaries and 653 harness tests passed, zero failed; one/three existing ignored respectively. All five root Rust gates, harness Clippy/formatting, docs format/lint/build and contract guards pass.
  • Rebuilt and explicitly selected the existing audited Viceroy 0.21.0 repair; all 136 upstream library tests pass. No installed tool or repository pin was replaced.
  • Inventory remains 255 criteria / 514 legs, qualification false. The earlier missing-runtime prerequisite is closed. Historical refusal, timing misses and intermittent workerd shutdown captures remain unexplained and are not waived by this passing sample.

Details: audit and plan. Manual hosted checks, independent current-head approval, freeze and durable qualification remain open. Existing manual checkboxes and review threads are unchanged. Earlier checkpoint results below are retained as history.

Signed-Origin Image Cache Policy Clarification (2026-11-09)

Pushed c8fd1939 under AG's direct approval. This carries the migration requirements from Trusted Server #1516 without changing production APIs, transport, SDK/runtime pins, deadlines, limits or capability levels.

  • Preserve per-request policy: signed HEAD preflight and separately signed raw error-body GET use Bypass without optimizer options; final optimized GET uses PlatformDefault with options. Signing, preflight decisions and private/no-store error rendering stay application policy. No automatic extra fetch or second transport.
  • Strengthen existing EZ-8 acceptance 08/09, including cold functional GETs without raw/debug warming, ordinary repeated-GET caching/optimizer metadata and bodyless private 403/404 HEAD errors. Original deployment results are references, not candidate qualification or a replacement performance baseline. Separate approved performance warmups remain intact; standalone cold HEAD paths remain unqualified.
  • Refresh exactly three inventory rows and current hashes, retaining 255 criteria / 514 legs, every binding/approval/comparison flag, and the original 140 titles / 223 scenarios. No criterion or environment identity was added, removed or reordered.
  • Add a co-located facade/native conversion regression for exact method, escaped target, authority, fixture signing headers and cache/options independence. Extend existing SDK cache-selection coverage to HEAD/GET in Send/Batch. All three focused SDK tests pass on the committed head using explicitly selected installed Viceroy 0.21.0; an intentional no-pass mutation fails, and restoring production mapping passes.
  • All five primary Rust gates pass (2,700 workspace tests / 33 binaries / zero failed), as do the 30 cutover tests, manifest/coverage guards, strict Fastly WASM and harness Clippy, docs/legacy checks, generated-app gate and demo tests.
  • Broader validation remains blocked. A standalone harness run has 649 passed / one startup identity-test connection-refusal failure. The full prescribed two-thread script has 650 passed / zero failed / three ignored, passes Axum integration, then exits 1 because its selected temporary repaired Viceroy executable is missing. The installed executable was not substituted for that gate. Both failures remain retained and unclassified; no retry credit, fixture weakening or qualification promotion.

Plan/checkpoint and verification boundaries. Hosted S3/optimizer checks, previous timing/shutdown failures, candidate freeze, durable Q5 publication and independent acceptance remain open. Fresh GitHub checks are separate. No deployment, thread resolution, dismissal or merge. Maintainer checkboxes are unchanged.

Earlier Checkpoint: Native Timer And Shutdown Investigation (2026-11-08)

Pushed 25114e66. This is a narrow Axum timer repair with co-located tests and audit/changelog updates; public APIs, SDK/runtime pins, capabilities and acceptance limits are unchanged.

  • Fix a deterministically reproduced native re-anchoring window: sampling the paired clock and then starting a relative sleep extended the native target by an intervening delay. Anchor first and use sleep_until; a paused-time regression fails at 120 ms before the fix and passes at the original 100 ms target afterward. Native early success remains only a wakeup, not logical expiry. Runtime/driver failures and overflow remain typed.
  • Retain real workerd stacks from two idle TERM failures, matching owned PID/parent, shutdown window and native executable UUID. KJ waits in kevent; this does not distinguish signal notification from stalled drain propagation. Four separate JavaScript-only controls pass but do not prove bootstrap causation. Temporary tracing/profiling code was removed after retaining captures; no new script, CI probe or profiler dependency ships.
  • Validation remains blocked overall. All five primary Rust gates, 2,700 workspace tests / 33 binaries / zero failed, 650 harness tests / zero failed / three opt-in ignored, and all 30 additional workflow format/lint/WASM/docs checks pass. The default-parallel workspace run passes too; Spin CI, generated-app builds and demo checks pass independently.
  • The full prescribed RUST_TEST_THREADS=2 bash scripts/run_tests.sh exits 1: Axum and Fastly pass, then Cloudflare's real routing probe requires SIGKILL after the unchanged five-second TERM grace. Preserve run 18e64816e16e8f68-663-0; no retry is substituted. Four focused native-timer tests also pass on the committed head.
  • The historical 107.941125 / 112.950416 ms versus 105 ms failures remain unexplained; the narrow conversion proof is not their root-cause proof. Workerd shutdown, independent review, manual hosted gates, strict qualification, candidate freeze and durable Q5 publication remain open.

Detailed investigation and evidence limits. New-head hosted checks are separate; older green checks are not reused. No deployment, capability promotion, thread resolution, review dismissal or merge. Maintainer checkboxes are unchanged.

Earlier Checkpoint: Local Reuse And Shutdown Closeout (2026-11-08)

Pushed ebab57ea. This is a test-harness/documentation checkpoint, not a production API or runtime-policy change.

  • Fix the custom Fastly initialization probe by waiting for the original guest's native reuse registration and matching pending-handle wait before each remaining prescribed callback. Keep all five requests, same-guest assertions and the original 500 ms windows. Nine deterministic regressions cover invalid/stale/partial/late witnesses and final SDK expiry; the complete Fastly smoke run passes on the committed head.
  • Retain direct-workerd routing inputs, native logs and process identity on startup/shutdown failure. A fail-first regression proves retention. One opt-in Rust diagnostic compares an idle real fixture with the original requests, with no new CI phase or script.
  • Workerd shutdown is not fixed: an idle Apple ARM64 workerd 1.20260415.1 needs SIGKILL after the unchanged five-second TERM grace. Other samples pass; response work is not necessary for the failure. Its cause is unconfirmed and no native stack or controller repair is established.

Verification is mixed. The full prescribed RUST_TEST_THREADS=2 bash scripts/run_tests.sh passes, including all four adapter suites, generated app, demo and documentation guards. Harness: 650 passed / zero failed / three opt-in tests ignored. All 30 additional workflow format/lint/WASM/docs commands and the other four primary Rust gates pass.

However, separate default-parallel and two-thread workspace runs both exit 101 on the unchanged Axum pending-upload target: 107.941125 ms and 112.950416 ms versus 105 ms. Both failures are retained, not dismissed or called pre-existing. Reduced parallelism is not an established repair; no threshold, assertion or qualification gate was relaxed.

Investigation and evidence limits. Fresh GitHub checks belong to this head and must be read separately; earlier green checks are not substituted. Reviewer acceptance, manual hosted checks, strict timing/cancellation gaps, candidate freeze and durable Q5 remain open. No merge-ready claim, deployment, capability promotion, thread resolution, review dismissal or merge.

This current status supersedes the historical checkpoints below. The maintainer's hosted checkboxes are unchanged.

Earlier Checkpoint: Fastly Callback-Ownership CI Repair (2026-11-08)

Pushed 26612053, a two-file test/changelog repair approved by AG. The original CI failure was reproduced unchanged on repetition 6; forcing a typed timer error reproduces it deterministically. The test incorrectly treated per-exchange timer destruction as callback closure, although an early connection error must release that timer immediately.

The repaired test observes the managed callback scope and exactly-once callback-family resource release, covers standalone/wrapped helpers with pending/failed timers, and retains the post-callback batch Internal assertion. Existing deterministic pending-driver cleanup tests remain unchanged. No production code, APIs, pins, deadlines, limits, assertions, capability levels or qualification gates were relaxed.

  • Focused verification: 30 consecutive functional-patch repetitions; final Fastly SDK suite 37 passed / zero failed, runtime units 322 passed / zero failed.
  • Clean committed-head verification: all five primary Rust gates and all 30 additional format/lint/WASM/docs commands pass. Workspace: 2698 passed / zero failed / one existing ignored / 33 binaries; harness: 640 passed / zero failed / two existing ignored.
  • Broader verification remains blocked. One committed-head RUST_TEST_THREADS=2 bash scripts/run_tests.sh attempt exited 1 when the Cloudflare ingress fixture required SIGKILL after the unchanged five-second workerd shutdown grace. The unchanged retry exited 1 because Fastly smoke returned 2: custom_lazy_initialization_recovery was unverified because all five callbacks were not observed in one guest. These failures are retained; their causes are not established and are not labelled pre-existing, waived, or converted to passing results.
  • The preceding working-tree full suite passed before the lint-only binding rename; it does not replace the failed committed-head evidence above. Initial disk exhaustion was addressed by removing only unused build targets. Strict WASM lint caught a shadowed local binding, which was renamed without suppression.

Fresh GitHub CI is running separately at 26612053. This is not an all-green or merge-ready claim. Existing review dispositions, manual hosted checkboxes, strict timing/cancellation gaps, candidate freeze and Q4/Q5 qualification remain open; no deployment, capability promotion, thread resolution or merge occurred.

Approved Qualification Amendments (2026-11-08)

Implemented in dcbf622f under AG's direct approval:

  • Scope HTTP/3 qualification to each selected adapter/deployment, preserving the original Fastly HTTP/1.1/HTTP/2 legs. Keep unresolved support required; approve unsupported N/A only through an evidenced, attributed single-leg decision. Inventory: 255 criteria / 514 legs, with every original leg retained.
  • Apply Fastly SDK parity only to content-bearing buffered responses; require strict HEAD/204/205/304 suppression. Separate six parity comparisons from twelve bodiless reference observations while retaining all nine historical differences.
  • Local framing passes all four adapters: 80 production cases, eight clean-EOF HTTP/1 HEAD checks, six Fastly parity passes and twelve checked references. Production outbound behavior/APIs, deadlines, limits, audited pins and capability levels are unchanged.
  • Verification passes the full prescribed suite with two Rust test threads, five Rust gates, and 30 additional format/lint/WASM/docs commands. Earlier OS/timing/output-mode failures remain disclosed in the verification record, not dismissed or relabeled.

The current handoff records current-head checks, per-thread replies and remaining review decisions. Older checkpoints below retain their historical status; they are not current completion claims.

  • Maintainer: qualify each selected deployment's HTTP/3 leg or approve its evidenced single-leg N/A.
  • Maintainer: obtain hosted content-bearing compression/parity and actual bodiless payload/completion proof.
  • Reviewers: approve the current head and settle the eight retained external dispositions. No merge is authorized by implementation approval.

TS Reposted Review Repairs

The six reposted TS findings are repaired in 6f0cc74c, with a separate fixed reply and executed evidence in each original thread. Reviewer acceptance remains pending; no thread was resolved by the author, and other reviewers' threads are untouched.

The scoped changes complete nonblocking Fastly buffered readiness through head/data/EOF and cooperative yields, add routed AC2 expiry coverage and bindings, restore the positive Rust deferral regression, and correct the remaining checklist, idle-policy and acceptance-reference text. Public outbound contracts, hard-cut APIs, SDK/runtime pins and qualification limits are unchanged.

Validation at the exact committed head passed: all five primary Rust gates; the complete bash scripts/run_tests.sh; and the 33-command lint/build/docs matrix covering adapter feature selections, all three WASM targets, fixtures and demo. Workspace: 2685 passed, zero failed, one existing ignored, 33 binaries. Harness: 626 passed, zero failed, two existing ignored. All four local adapter integration suites and the generated-project gate passed. The deferral spelling mutation failed as intended and was restored before the commit.

The initial terminal-stdin interruption and the local Cloudflare tool-version failure are retained; the final committed-head run used noninteractive stdin and the ingress fixture's lock-matched wasm-bindgen. Fresh GitHub checks are running; they are not declared fully green. This is local validation, not a frozen or hosted qualification record.

Qualification remains open: the strict local lifetime run retained cold Begin at 8548 us against the unchanged 7000 us bound. SDK parity, unknown optimizer blocking horizon, manual hosted evidence, candidate freeze and Q4/Q5 are not waived. No deployment or capability promotion was performed. The manual hosted checklist below is unchanged.

Implements the portable outbound HTTP and request-lifecycle contracts across core and the Axum, Cloudflare, Fastly, and Spin adapters.

  • Adds buffered and streamed outbound requests/responses plus a hard-cut, completion-driven batch API: start_batch_until, indexed next, explicit cancel, ordered collect, and the send_all_until convenience path.
  • Clarifies that start_batch_until takes an observation cutoff: reaching it stops observation even when individual slots retain budget, while earlier cutoffs remain a supported partial-result operation.
  • Replaces the memory-only adapter seam with typed PlatformMetadata facts for memory ceilings, live inbound request population, and host-ingress accounting, preserving source and explicit unknown reasons.
  • Adds checked, scope-aware memory-envelope validation with Fits, Exceeds, and typed Indeterminate outcomes; publishes canonical Axum, Cloudflare, Fastly, generic Spin, and Akamai Functions profiles.
  • Hard-cuts retired platform-memory APIs, updates the demo and generator, and records exact signature migrations in the unreleased changelog.
  • Makes batch termination explicit as Completed or Cutoff. Only Cutoff permits unresolved slots in a successful result. Explicit adapter failure, premature driver EOF, and duplicate or out-of-range indices return OutboundBatchFailure, preserving the precise error and every terminal slot collected before failure.
  • Preserves per-slot elapsed time and failure provenance. A batch cutoff keeps completed slots and leaves unresolved slots explicit instead of converting them into request timeouts.
  • Adds explicit cache-bypass and wire-authority policies. URI routing, SNI, and certificate identity remain independent where the provider supports that separation.
  • Enforces independent encoded-transport, decoded-output, and final-buffer limits; header count/byte limits; Brotli window and decoder-state policy; opt-in response rechunking; and typed timeout, transport, protocol, and content-coding failures.
  • Uses fixed category-only public error messages so provider diagnostics, URLs, and credentials cannot become wire-visible.
  • Carries one application-owned monotonic clock through ingress, outbound dispatch and deferred streams, returned response drains, configuration extraction, and response egress. MonotonicClock::deadline_after anchors relative cutoffs to that clock. Equality expiry wins and backwards time fails closed.
  • Adds route-aware ingress admission on every standard adapter, stable route metadata, request-start-relative deadlines, opaque request-owned grants, and bounded 404/405 fallback reads with application-selected overflow/timeout responses.
  • Gives every response-producing admission decision one non-clone ResponseEgressCompletion, transferred directly through PreparedIngress, ResponseEgressEnvelope, and the sole ResponseEgressAttempt. Terminal state is stored before the response-scoped completion and global observer run, each exactly once and behind independent panic boundaries. No response-extension carrier or envelope extraction bypass remains.
  • Adds deterministic ResponseEgressCompletion::join composition so independent response-owned resources share one exactly-once terminal report, and hard-cuts detached normalized errors to Send { completion, deadline } | Abort with request-start-relative bounded deadlines.
  • Adds explicit non-response AdmissionDecision::Abort. Axum closes the owned HTTP/1 connection before body polling; Cloudflare, Fastly, and Spin return a fixed provider error at their strongest available pre-response boundary and remain conservatively classified BestEffort.
  • Hard-cuts application assembly to fallible Hooks::configure plus core-owned App::build::<A>(platform). Successful hooks must preserve the complete adapter-selected metadata; core rejects changed facts instead of restoring them. Typed configuration failures propagate unchanged and stop serving before request handling. The generated scaffold keeps finite ingress policy with core-owned empty lifecycle tokens; the checked-in demo retains the full application-owned resource-composition example.
  • Implements adapter-owned response delivery through each platform's strongest boundary: Hyper connection ownership on Axum, a Workers writer plus AbortController, Fastly's low-level stream_to_client, and Spin's WASI response/body/result writers.
  • Adds a closed Fastly lifecycle runner that keeps request conversion, routed execution, response finalization, framing, streaming, and platform delivery under adapter ownership.
  • Fastly dispatches eligible slots before harvesting and advances heads/bodies through one cooperative readiness multiplexer, preserving positional ownership, original deadlines, bounded raw reads, fair rotation and documented provider limitations.
  • Preserves duplicate response fields, canonical host authority, bodyless response framing, strict content-length validation, hop-by-hop stripping, and source release on every terminal path.
  • Hard-cuts superseded APIs and configuration error variants. The checked-in demo, generated applications, templates, guides, capability matrices, CI sentinels, and legacy guards use only the current contracts.

Current Checkpoint: Event-Driven Workers Cancellation

Head: 9b31a697, including main cf2c57b1. Implementation and evidence boundaries.

The PR is ready for review, not draft, but remains unqualified. Historical checkpoint references to draft status describe their then-current state; they are not the live PR state.

  • Replace recurring abort polling with one owned event listener per pending source/write/close operation. Remove the same callback on completion/drop, release retained wakers and preserve cancellation precedence, absolute deadlines, writer-closed wakeup and native accepted-byte accounting. No public API, transport, executor, dependency, capability or acceptance change; demo/templates inherit the private repair.
  • Three new WASM regressions fail before repair. Final verification passes all five root Rust gates: 2,666 workspace tests/33 binaries (one existing ignored), 137 Workers SDK tests, 30 WASM contracts, five native contracts and both strict WASM feature configurations. Docs format/lint/build and legacy/documentation/coverage guards pass. The ownership fixture uses an explicit write-acknowledgement handshake, retaining its four-byte and resource-release assertions without a timer or microtask-settlement workaround.
  • One isolated inventory-exact native invocation on clean 9b31a697 retains 17 passed/two failed constituent scenarios, exit 1, plus failed tail validation. Cold Begin is 8 ms >7 ms; ordinary-close cancellation/cleanup is unproved. These map to 12 passed/two failed lifetime legs. All previous failures, including the preceding 6e5ce765 capture, remain retained; no warmup, tolerance change, reset substitution or retry-for-credit.
  • Assemble 54 exact-head observations: 52 passed/two failed, using the existing evidence checker and unchanged commands/sentinels/build hashes. 163 required-local results and 44 required-external results remain missing. The pre-freeze checker accepts zero qualified legs and rejects release. The 50 conditional and 200 unpromoted follow-up legs retain their previous scope. Passing execution is not qualification.
  • Reply to the cancellation review and the blocking Fastly question. The current provider guide describes successful identical re-registration and failure on conflicting properties; keep actual NameInUse fail-closed rather than adopting unverified settings. Hosted controls and maintainer re-review remain open. Other non-blocking review suggestions are not silently marked resolved.

Local unqualified review packet: /private/tmp/pr275-9b31a697-closeout.tar.gz, SHA-256 878876dda65f8404af5b75215631e4402c5dce80f8b73a11e0080ac5c17faa3f. It retains 52 hashed payloads plus their manifest, actual raw failures/builds, the caught fixture-ordering failure and final verification. Temporary retention is not durable Q5 publication. New-head CI runs separately; no prior-head green check is relabelled as new-head proof. Your eight manual hosted checkboxes are unchanged. No deployment, final freeze, capability promotion, review dismissal or merge is performed.

Earlier Checkpoint: Workers Batch Cutoff

Head: 8f82547b. Repairs a confirmed adapter-owned scheduling gap without changing public APIs, request budgets, capability levels or SDK/runtime pins. Specification, regression and native-evidence boundaries.

  • Fail first through the actual Cloudflare coordinator: one timer notification leaves sibling exchanges parked; an earlier own deadline and a batch-timer failure cannot finish promptly. Retain all three original failures. Separately reproduce expiry crossed during synchronous child cleanup before correcting the recheck.
  • Own one independent paired wait on the original observation cutoff. On expiry, final-poll parked pipelines once through their existing absolute guards, then destroy the collection and batch timer before terminal publication. Recheck expiry after child work and before parking. No container polling resumes after the borrowed terminal sweep.
  • Preserve earlier-own timeout provenance/elapsed values, exact timer errors, published partial results, cancellation, normal completion and all-own-timeout completion. Controls include withheld callbacks and synthetic release work: completion includes cleanup cost instead of reporting the earlier timer-wake instant.
  • Verify 2,666 workspace tests across 33 binaries, 124 Cloudflare SDK tests, 30 contracts, all five root gates, both strict WASM feature configurations, docs format/lint/build and docs/legacy/coverage guards. Eight added WASM regressions execute through the existing phase; no new CI job, runner, transport or compatibility layer.
  • Retain one unchanged native working-tree characterization: pending-head/body cutoff returns at 202/203 ms, within the original 205 ms upper bound. All three native pending-body sources release by return. This capture uses its archived pre-simplification variant, not final-source bytecode, and earns no frozen-candidate acceptance credit.
  • Close the remaining failed native observations: cold Begin 10 ms >7 ms, fast-origin observation 326 ms >325 ms, and ordinary-close cleanup, plus failed final-tail validation. The full suite remains 16 pass / three fail, exit 1. Concurrent workspace verification is disclosed; this is not an isolated performance comparison. The new fast-origin miss and earlier 207 ms body-cutoff failure remain retained, not retried for credit.
  • Complete exact-candidate acceptance/build binding, source freeze, the maintainer's manual hosted checklist and durable Q5 publication before final acceptance/merge.

The local review packet contains 99 hash-verified files: /private/tmp/pr275-cutoff-coordinator-packet.tar.gz, SHA-256 27837dc0fdb5785550ced06f7666e58091508e9ac64bbd11c884d2755195be68. It preserves RED/GREEN and final verification logs, the actual native patch/build and all failures, separately from the final source patch. Temporary local storage is not durable qualification. Demo/templates require no signature migration and inherit the adapter repair. No deployment, freeze, merge or maintainer review-state change is performed; the eight manual hosted checkboxes below are unchanged. New-head CI must execute separately.

Earlier Checkpoint: Exact-Head Evidence Mapping

Tracking commit: 46019b2d. This documentation-only checkpoint maps evidence for reviewed source 562acc1dedbf169315853900f9395e7ff5160076; it does not freeze or qualify either revision. Audit and evidence boundaries.

  • Account for all 255 criteria / 511 legs through the existing inventory and evidence checker: 217 required local, 44 required external, 50 conditional pending and 200 unpromoted follow-up legs. No requirement, acceptance identity, applicability, command, tolerance or capability is changed.
  • Assemble the 14 exact-head lifetime observations: 11 passed / three failed, 19 constituent scenarios. Retain cold Begin at 9 ms (7 ms upper bound), pending-body cutoff at 207 ms (205 ms upper bound), ordinary-close cleanup failure and the separate failed final tail. Preserve both cutoff phases and all five cleanup paths, actual Wasm/raw hashes and original outcomes.
  • Keep supporting evidence out of exact-head results: 17 local-wire mappings use a different output directory; 106 CI matches include every primary/additional sentinel but lack exact-head provenance. Axum/Fastly runtime records report 12287f9f; Cloudflare/Spin do not embed a source SHA. The 30 successful checks associated with reviewed head 562acc1d do not establish exact-head runtime qualification.
  • Retain missing results explicitly: 247 required results absent (203 local / 44 external). These are missing eligible records, not 247 demonstrated implementation failures. All three existing assembly/report/release invocations correctly exit 1; accepted release legs remain zero. Unknown runner image and unobserved version precision stay unknown, not placeholder values satisfying gates.
  • Verify all five root Rust gates, all 131 evidence-checker regressions, docs format/lint/build, documentation/legacy guards and unchanged inventory coverage. Independent provenance and quality reviews retain all qualification boundaries.
  • Complete remaining exact-head execution/build/command bindings and companion/comparison evidence, without relabelling supporting captures or filling missing rows with invented results.
  • Resolve the original timing and ordinary-close cancellation/cleanup qualification failures without weakened deadlines or substituted reset evidence.
  • After source completion, separately freeze the candidate, complete the maintainer's manual hosted checklist and publish durable exact-candidate Q5 evidence before final acceptance/merge.

The local review packet contains 231 hash-verified files: /private/tmp/pr275-562acc1d-closeout-packet.tar.gz. SHA-256: d96a6a50a80b87dad910fe43e2ca71d53ea887f7711d83cc5d26867af56fc4d6. It retains the original metadata draft separately, corrected pre-freeze record, complete derived leg map, raw captures/builds and supporting CI crosswalk. This temporary archive is not durable Q5 publication. Source APIs, adapters, demo/templates, SDK/runtime pins and outbound guarantees are unchanged. No deployment, freeze, merge or change to the maintainer-managed PR review state is performed. The eight manual hosted checkboxes below are preserved unchanged.

Earlier Checkpoint: Workers Timer Setup Repair

Head: 562acc1d. The reviewed repair, regressions, specification, driver plan and changelog are committed and pushed. Source is clean. Investigation and evidence boundaries.

  • Reproduce the adapter-owned timer setup defect with three failing deterministic WASM tests: stale 10 ms registration instead of 3 ms after 7 ms setup, and unnecessary registration after expiry/regression.
  • Retain the original absolute deadline and paired clock through callback construction and both host lookups. Recompute immediately before arming; skip expired registration, fail closed on regression and preserve scheduling errors. Release callback/clock/waker state on terminal success and failure. Keep one primitive and genuine zero-delay fairness turns; remove the relative constructor and extra timer async wrapper.
  • Verify all five root Rust gates, 2,666 native tests across 33 binaries (zero failures; one existing ignored), 113 Cloudflare SDK tests, 30 Cloudflare contracts, strict WASM lint with and without test-utils, documentation formatting/lint and legacy/documentation guards.
  • Run the existing wire command once on this exact clean commit: 68 exchanges / 84 origin requests, zero failures, exit 0.
  • Retain the exact-commit lifetime result: 16 pass / three fail, exit 1, plus cleanup-tail failure. Cold Begin is 9 ms versus 2 ms +/-5 ms; pending-body cutoff is observed at 207 ms versus the original 200 ms +5 ms target; ordinary-close cleanup remains unproved.
  • Isolate notification behavior with the unchanged native launcher and a plain JavaScript TransformStream, without EdgeZero/Wasm in the request path. Ordinary close produces neither request-abort nor writer rejection through the last sample; reset produces request-abort, while write-half-close retains its live read owner without notification. The last ordinary-close sample is 1.999815041 s from client request start, not two seconds after close. Explicit diagnostic cleanup and healthy recovery occur afterward. This is bounded local evidence, not FIN proof, a universal provider claim or acceptance credit.
  • Close the unchanged native timing and ordinary-close qualification gates. Neither the timer setup repair nor reset success marks them passed.
  • Complete all applicable exact-candidate acceptance results, candidate freeze, durable Q5 retention and final review after local and manual hosted gates pass.

Hosted CI at this update: 30 successful / 0 neutral or skipped / 0 running or queued / 0 failed reported checks. Local verification is not a substitute for unfinished hosted checks.

A separate, hash-verified review packet contains 111 files preserving raw controls, temporary instrumentation, RED/GREEN logs, final verification and exact-commit runtime artifacts. Packet SHA-256: 00a5573ac25d504e50e1953bb585ffb043b04a7d10bb6a440268bc0e78e6eb25. Lifetime Wasm SHA-256: e4730e8f068c15ac35b7c7e28e70327fb06097d787eee7eba60fa6ac26c6c6bd. Prior macOS/Linux failures and earlier packets remain retained separately; diagnostic passes are not copied into release evidence. Local temporary storage is not durable Q5 publication.

No public API, SDK/runtime pin, capability, application policy, deadline, observation cutoff or tolerance is changed. Demo/template APIs remain unchanged. No deployment, candidate freeze or merge is performed; this PR remains draft. Hosted verification remains the maintainer's checklist below.

Maintainer Hosted Verification (Manual)

The maintainer will perform and check off these hosted gates manually. No deployment is delegated. Record the exact candidate SHA and link durable evidence; unchecked means unverified, not passed. Keep documented BestEffort/Unsupported limitations unless stronger behavior is actually proved.

  • Record the candidate SHA, staging targets, SDK/runtime revisions, compatibility settings and effective limits used for every hosted run.
  • Cloudflare: verify normal/bodyless delivery, slow or non-reading clients, disconnect/cancellation with pending sources, write expiry and source/conversion/transport failures; verify exactly-once completion, SDK-resource release and healthy capacity recovery where observable.
  • Fastly: verify the same applicable lifecycle cases, original deadlines, bounded B31 backend identity across heterogeneous multi-host batches, strict-TLS/readback and pooling/resource behavior. Distinguish host handoff from confirmed delivery.
  • Fastly Image Optimizer: verify successful transformation and optimized-first-hop redirects, direct native request mapping, blocking horizon and performance parity on an enabled staging service.
  • Spin: verify applicable lifecycle cases, silent-connect/deadline behavior, pending body production, disconnect/cancellation, teardown and capacity recovery on the recorded hosted runtime.
  • Verify applicable config-store allocation/peak-memory bounds and interrupted-read cleanup against the selected provider limits; record exclusions and unobservable provider-owned allocations explicitly.
  • Confirm consuming applications pass the hard-cut integration cases without an alternate transport or weakened outbound requirements.
  • Attach complete applicable Q4 hosted results and the durable exact-candidate Q5 evidence record; retain negative results and approved deferrals, then approve final review/merge.

Local CI, parser replay and native diagnostics do not check off these hosted gates. Acceptance identities and remaining local failures stay unchanged.

Earlier Checkpoint: Completed-Test Host-Diagnostic CI Repair

Head: f82e4025. The prior head has 29 successful checks/one failed Fastly job. Its native library reports 309 passed/zero failed, but a known host diagnostic is concatenated directly after a complete named ok; the sentinel guard incorrectly rejects that line. This checkpoint changes only that guard, its co-located tests and tracking docs.

  • Reproduce the failure from the downloaded fastly-lifecycle-evidence artifact and fail-first regressions: 103 pass/6 expected failures, covering two formats and four exact inventory environments.
  • Extend only the exact completion condition: accept ok immediately followed by the existing anchored timestamped Viceroy diagnostic. Reject arbitrary ok... suffixes, malformed/missing timestamps, unrelated diagnostics, failed/ignored statuses and wrong names. Preserve every sentinel, listing checks, positive zero-failure summaries, and command/stream exit handling. No workflow, invocation or inventory change.
  • All 109 guard tests/284 tooling tests pass, and the exact failed artifact passes parser replay. The original guard-error text remains retained: replay is not fresh native execution or a claim that the old hosted job succeeded.
  • Local verification: all five root gates, 2,666 tests/33 binaries (one existing ignored), Bash syntax, script formatting, docs format/lint/build and docs/legacy contracts pass. Independent reviews are clear. Production APIs/adapters, pins, deadlines, caps, acceptance identities and capability levels are unchanged. Remaining strict timing, ordinary-close cancellation, candidate freeze and hosted qualification stay open.
  • All 30 exact-head checks passed, including the test workflow, the previously failing Fastly job, every other adapter, workspace, CLI/generated-project, demo and CodeQL checks. The PR remains draft and unqualified until the separate required runtime/hosted evidence is complete.

See the repair and verification tracker and original failing job.

Earlier Checkpoint: Absolute Deferred Batch Dispatch

Head: 158b0c0e. This checkpoint corrects the existing lifecycle fixture, not production outbound behavior. APIs/adapters, SDK/runtime pins, original deadlines, caps, acceptance identities and capability levels remain unchanged.

  • Fix a deterministic duration-rebasing defect: the fixture computed a relative delay before returning its response, but the provider starts that delay on first body poll. A fail-first actual-route regression reproduces dispatch at 41 ms after a 7 ms first poll. Replace that wait with the existing native paired timer's original absolute 34 ms target, inside the same body producer. Add no timer mechanism, executor, transport, compatibility path, suite or CI job.
  • Require no dispatch through 33.999 ms for initial polls at 0/7 ms, followed by dispatch at 34 ms. A late first poll at 40 ms dispatches at actual 40 ms without another sleep or backdating; strict timing validation is unchanged. Egress still owns its original one-second deadline before dispatch.
  • The native Workers batch-write case passes: dispatch 34 ms, response DeadlineExceeded 1,001 ms, original per-call deadlines 1,034 ms. The complete local suite still exits 1, with 16 pass/3 fail plus a final-log-tail failure: cold Begin entry 9 ms, pending-body cutoff 206 ms, and ordinary-close cancellation remain unqualified. Pending-head cutoff returns at 204 ms in this run; earlier failed observations are not erased. Retain the initial missing-workerd preflight as unverified exit 2.
  • Historical cold Begin/handoff misses precede API entry; their call durations are 0-1 ms at recorded resolution. Pre-entry latency is not yet attributed. Likewise, head-operation release precedes the outer abort guard and cannot measure actual abort duration. Timer delivery, executor resumption and cleanup cost remain separate unknowns.
  • Local verification passes all five root gates, 2,666 workspace tests/33 binaries (one existing ignored), 85 fixture tests with native witnesses/69 without, 150 owning lifecycle-harness tests, strict fixture Clippy, all three fixture WASM checks, 267 tooling guards, formatting, docs format/lint/build and docs/legacy contracts. Independent implementation/evidence reviews are clear. New-head GitHub checks are running; PR remains draft and unqualified. Candidate freeze and hosted qualification remain open.

See the current tracker and audit.

Earlier Checkpoint: Native Spin Result Before EOF

Head: 8fbc4f83. This checkpoint changes the existing Spin SDK controls, validation and retained evidence only. Production APIs/adapters, SDK/runtime pins, original deadlines, acceptance identities and capability levels are unchanged.

  • Sample the same pinned native response-result future at most twice before EOF, around the timer control's existing 100 ms wait. Retain the same writer, body-result owner, writes and executor. Cache completed results without repolling; Pending keeps its original post-EOF await. This explicitly changes diagnostic polling order; it is not passive instrumentation or a production polling workaround.
  • Native Spin 4.1.0/SDK 7.0.0 observes Pending then Ready(Ok) across 102.1215 ms, while the raw SDK response writer is still owned and the body result remains unsent. The ready control's single Pending sample is inconclusive. This unbounded-response result can complete before its body and cannot then report a later disconnect; it does not prove universal provider impossibility, client delivery or a new production defect. Existing early-Ok retirement remains correct.
  • The complete existing progress suite and four response controls pass, with two exact case/token-correlated canonical component terminals. Validate bounded transcripts and reject missing/malformed/mismatched evidence or completed-future repolls. Retain the post-shutdown runtime panic as a separate negative, not safe-teardown proof. Preserve component/runtime/stderr SHA-256 hashes and the working-tree build identity.
  • Workers source review is consistent with write-side/read-side close differences, not packet-level attribution. Passive BPF capture requires administrator access; no traffic capture or privilege bypass occurred. Original ordinary-close/reset cancellation failures, strict timing, exact-candidate and hosted gates remain open. No upstream issue, deployment, freeze or capability promotion is claimed.
  • Local verification: all five root gates, 2,666 workspace tests/33 binaries (one existing ignored), 586 harness tests (two existing ignored), 84 fixture tests, three feature-enabled/nonzero-guarded Spin race model tests, strict harness/Spin fixture WASM lint, all three workspace format checks, 165 tooling guards, legacy checks and docs format/lint/build pass. Independent reviews are clear. All 30 GitHub checks passed at this exact head, including tests, lint, static/release checks and CodeQL. PR remains draft and unqualified.

See the native record, pinned source analysis and implementation tracker.

Earlier Checkpoint: Explicit Reset Diagnostic

Head: cd214362. This checkpoint changes test tooling and evidence only. Production outbound APIs, adapters, SDK/runtime pins, original deadlines, caps, acceptance identities and capability levels are unchanged.

  • Give the existing Rust HTTP helper an explicit close/reset policy. All original gates retain ordinary close. Only intentional prefix rejection in the reset control requests zero-linger reset; four raw TCP controls distinguish reset, close, success and ordinary read failure while preserving partial-response evidence.
  • After an original cancellation failure, finish that runtime and strict tail, then reuse the existing fixture/runner in one isolated, startup-attested runtime. Separate logs, tokens and cursors prevent cross-run credit. Preserve original-error precedence, all case budgets and counts; add no suite, CI job, transport or executor. Passing runs incur no extra startup.
  • Native Workers reset produces 19 validated cleanup/isolation stages, RequestCancelled, exactly-once SDK/source release, healthy survivor work and a clean diagnostic tail. Ordinary close still fails, with no victim terminal through the original coordination window. Reset success is diagnostic only, not a replacement acceptance gate.
  • Spin has no victim cleanup under either ordinary close or reset. Keep that negative and all timing failures: Workers 15 pass/4 fail, macOS Spin 17 pass/2 fail/1 unverified, both exit 1. No provider-impossibility or hosted-cancellation claim. Retain the earlier invalid first-reset capture and the corrected startup setup separately.
  • Verification: all five root gates (2,666 tests/33 binaries, one existing ignored), 585 harness tests (two existing ignored), strict harness lint, 165 evidence/workflow guard tests, docs format/lint/build and legacy guard pass. Preserve an initial concurrent run's existing process-cancellation timing miss; unchanged isolated and final serialized tests pass, without widening timeouts. Independent reviews are clear.
  • All 30 GitHub checks pass at this exact head, including tests and generated-project checks, static/docs/release checks, and Rust CodeQL. PR stays draft and unqualified: 217/217 required-local bindings means executable coverage, not passing qualification. Ordinary cancellation, remaining strict timing observations, candidate freeze and hosted proof remain open.

See the current diagnostic and runtime tracker.

Earlier Checkpoint: Native Runtime Boundaries And Linux Connect

Head: f036425b. Production adapters, public APIs, provider pins, deadlines, caps and original acceptance identities are unchanged.

  • Preserve the first strict pending-source cancellation failure while collecting separate bounded diagnostic records until the original two-second coordination deadline. The survivor's independent one-second timeout no longer ends observation early. Validated cursors, native credit and successful paths are unchanged; malformed/incomplete/truncated/late diagnostics never pass validation.
  • Select the native application default socket in the existing Workers fixture launcher using the pinned Miniflare API. No entry-forwarding fallback; the SDK entry/control listener remains on a distinct ephemeral loopback port. Guest flags, modules, bindings, persistence and owned shutdown remain intact. Standard serve/deploy paths are unchanged.
  • Verify the existing Linux Spin silent-connect case: 26 exact SYN_SENT ownership samples, typed ConnectTimeout/504 at 202,813 us, original 200 ms connect / 2 s aggregate settings, exactly-once SDK operation release, and same-client 200/ok recovery. Whole Linux suite remains 9 pass/11 fail, exit 1. This is working-tree phase-timeout evidence, not aggregate-expiry or candidate qualification.
  • Verification: all five root gates (2,666 tests/33 binaries, one existing ignored), 583 harness tests (two existing ignored), strict lint and docs/evidence/legacy guards; all 97 Workers paired wire vectors, all 11 ingress-overflow controls, fresh/retained Workers smoke, and 19/19 Axum lifecycle cases pass.
  • Runtime evidence remains deliberately incomplete: Workers lifetimes 15 pass/4 fail, macOS Spin 18 pass/1 fail/1 unverified. Both original pending-source cancellation captures now reach the original deadline with 11 canonical records and no diagnostic-error flags, but still lack victim terminal/source-release proof. Cold/batch-entry timing misses remain failures. Controller source release is not SDK release.
  • The reset diagnostic decision remains separate: no reset control replaces the original failing close/cancellation gate. No deployment, capability promotion, candidate freeze or hosted qualification is authorized or claimed.
  • All reported GitHub checks completed successfully at this exact head, including native/WASM integration, workspace/generated-project checks, lint, docs/static checks, smokes and Rust CodeQL (tests, format/lint, CodeQL). PR remains draft. Coverage stays 217/217 required-local bindings, qualified:false; external, conditional and unpromoted scope is unchanged.

See the current tracker and canonical Linux connect evidence.

Earlier Checkpoint: CI And Local Lifecycle Repairs

Head: 22c6386c. All 217 required-local acceptance legs now have executable bindings (255 criteria/511 legs; 246 ready/265 planned overall). Binding completeness is not passing runtime qualification. External, conditional and unpromoted scope remains unchanged.

  • Repair the previous head's CI failures: observation-cutoff fixture names, the unpublished-plan link, and the single-exchange parity origin's missing connection-close framing. Preserve production pooling/retries and exact SDK-visible header caps, including Spin's hidden Connection field.
  • Observe one cached Workers writer-closed promise during pending response production, preserving deadline/abort precedence and exactly-once cleanup. Guest SDK tests pass, but HTTP pending-source cancellation is not yet proved.
  • Measure the unchanged 300-305 ms quiet interval using the existing client's paired timer, separately from completion of the same auxiliary HTTP Send. Accept either actual completion order; retain causal clock guards and wait for both before collection. No public context getter, second executor, transport or compatibility path.
  • Local verification: all five root gates (2,666 tests/33 binaries, one existing ignored); 576 harness tests (two existing ignored); 84 qualification-fixture tests; 111 Workers SDK tests; strict lint and docs/evidence/legacy guards. All four complete wire suites pass 97 paired Send/Begin vectors each, and Axum's complete CI owner passes.
  • Local lifecycle runs: Axum 19 pass, exit 0; repaired Viceroy 33 pass plus one recorded row, exit 0; Spin 18 pass/1 fail/1 unverified, exit 1; Workers 14 pass/5 fail, exit 1. All four repaired quiet-window cases pass. Workers timing misses and Workers/Spin pending-source cancellation proof remain open; Spin's silent-connect proof requires Linux. Earlier failures are retained, not retried for credit.
  • All reported GitHub checks completed successfully at this exact head: build/test, native/WASM lint, docs/static checks, demo/generated-project checks, release smokes and Rust CodeQL (tests, format/lint, CodeQL). All 30 reported checks are successful. PR remains draft and unqualified. No deployment, capability promotion, frozen-candidate or hosted qualification is inferred.

See the current implementation and evidence tracker for exact observations and remaining gates.

Earlier Checkpoint: Fastly Post-Delivery Batch

Head: 775366fd. Extend the existing D6 lifetime suite with a three-slot batch created after canonical response handoff, using the same retained client and original request clock. Preserve 34 ms batch entry, 534 ms cutoff and one-second per-call timeouts. No production API, SDK/runtime pin, capability or CI invocation changes.

  • Delay collection until 450 ms through the existing paired/provider-backed scope. Require three actual SDK body releases before collection and unchanged single-release counts after callback return.
  • Preserve original arrival elapsed values. Cross-bind release-relative timing to the matching controlled source writes; reject collection-stamped results, coordinated early timestamps and malformed/duplicate protocol tails. Reuse the existing bounded origin and one canonical parser/wait loop.
  • Local repaired-Viceroy inventory: 24/24 pass, including all eight callback-wrapper cases and final log tails. Handoff 0.115 ms; batch entry 35.198 ms; SDK bodies released by 96.181 ms; collection 451.266 ms; elapsed 21.146/40.970/60.992 ms.
  • Verification: all five root gates, 2,650 root tests across 33 binaries (one existing ignored), 454 harness tests (two existing ignored), 61 fixture-core tests, strict Fastly WASM fixture/harness lint, docs checks/build, legacy/inventory guards and frozen identities pass locally. Independent ownership/specification and evidence reviews are clear after the causal timestamp guard.
  • Remains draft and unqualified. Exact-candidate, remaining runtime acceptance and hosted qualification stay open; local SDK body-handle release does not prove universal provider teardown or client delivery. Inventory remains 255 criteria/511 legs, 153 ready/358 planned. Historical failed timing evidence is retained.
  • Previous head had no Actions runs for an unestablished triggering reason. Actions has started on this head; hosted CI is not claimed green while running.

Earlier Checkpoint: Fastly Trickling Slots And Live-Batch Send

Checkpoint f0ce2b38 adds EZ-6 criteria 7/8 to the existing Fastly fixture and lifetime suite. Production outbound APIs, transport, scheduler, deadlines, limits, SDK/runtime pins and CI invocations are unchanged.

  • Join original origin-body consumption with the original bounded Next loop. Record real byte receipts and Next returns immediately in one ordered log: at most 64 completed reads / 64 payload bytes / 4 Next returns. Do not equate SDK reads with source chunk boundaries, backdate data, sort observations or accept absent results as EOF. Require actual intermediate receipts inside the terminal wait, then origin progress after cutoff.
  • Both new local cases pass. The trickling bidder remains unresolved; terminal Next spans 185.055..534.902 ms, origin receipts continue during that wait and at 551.722/601.538 ms afterward, with EOF at 601.573 ms. Send-after-batch preserves A/B elapsed 146.516/286.371 ms and C unresolved under the original cutoff.
  • All five root gates pass: 2,650 tests / 33 binaries, one existing ignored. Final serialized harness 449, two existing ignored; fixture 61 with/without witnesses. Strict harness/fixture/Fastly WASM lint, docs, legacy and frozen-identity guards pass. Both review findings have fail-first regressions and reviewed fixes. The earlier full harness run retains two process/cancellation failures; no timeouts were widened.
  • The expanded inventory remains 21/23, exit 1: slow-origin EOF 1306.260 ms vs 1300 ms +/- 5 ms, and fourth-slot B elapsed 294.511 ms vs 286 ms +/- 5 ms still fail. All eight callback, three cleanup and log-tail cases pass. New-case success does not waive these or historical failures.
  • Inventory stays 153 ready / 358 planned, unqualified. Exact-candidate, hosted and other remaining local acceptance stay open; PR remains draft. Local execution uses the tracked Viceroy repair, not a stock/hosted qualification claim. Demo/templates need no migration for test-only coverage. Prior-head checks were all green; new-head CI is pending.

See the master/D6 tracker, SUITES.md and changelog for exact scope and retained results.

Earlier Checkpoint: Fastly Origin-Body Arrival Coverage

Checkpoint eb8ac5df adds the missing EZ-6 criterion 6 case to the existing Fastly lifetime suite. No new runner, transport, scheduler, SDK probe, production API/policy, pin or CI invocation.

  • Begin the original streamed origin at 2 ms, start the original batch at 34 ms with cutoff 534 ms, and retain A/B/C at 180/320/900 ms. Consume only the origin body at 100/800 ms before checking the batch. Validate original arrival elapsed, unresolved C, source framing, successful Begin and nonoverlapping operations; reject polling during body reads or collection-stamped results.
  • Fail-first fixture/origin/controller tests and negative controls pass. All five root gates pass: 2,650 tests / 33 binaries, one existing ignored. Full harness 445, two existing ignored; fixture 59 with and without witnesses. Strict harness/fixture/Fastly WASM lint, docs and frozen-identity guards pass. Independent spec and quality reviews are clear. The prior head had all 30 checks green; new-head CI is pending.
  • The expanded local inventory is 19/21, exit 1. All eight callback and three cleanup cases pass. The new case recovers A/B at 148.972/288.603 ms after EOF at 802.816 ms, but fails its head/body-start timing at 110.546/110.549 ms vs 100 ms +/- 5 ms. Slow-origin also fails A elapsed at 151.378 ms vs 146 ms +/- 5 ms. Recovered outcomes do not qualify the complete case; all failed evidence is retained.
  • Timing tolerances, normative identities and acceptance bindings are unchanged. Inventory remains 153 ready / 358 planned, unqualified. Missing local scenarios 7/8, other acceptance, exact-candidate and hosted qualification remain open; PR stays draft. Local Fastly uses the tracked Viceroy repair, not a stock/hosted qualification claim. Demo/templates need no migration for this test-only case.

See the current master/D6 tracker, SUITES.md and changelog for exact scope and evidence.

Earlier Checkpoint: Fastly Native Body-Readiness Witness

Checkpoint 5950548d adds the distinctly labelled, request-owned test-only Fastly native readiness witness. Production outbound APIs, deadlines, B31 policy, limits, SDK/runtime pins and CI invocations remain unchanged.

  • Observe one existing native select using the actual submitted SDK body owners. Preserve Ready, OtherReady, TimedOut and Failed outcomes without added SDK probes, reads, timers or wakeups. Share the generic test-only owner with existing stream witnesses; record release only after the actual resource drops.
  • Require head <= wait-entry < 200 ms <= timeout-return <= next-return <= 205 ms, original URI/clock/mode/budget, exactly-once owner release and independent origin hold/release. Reject overlapping/conflicting call records and underreported counts. This is sdk-readiness-wait, never SDK-read Pending or proof of host teardown.
  • All five root Rust gates pass: 2,650 tests / 33 binaries, one existing ignored. Core 1,116, native Fastly 176, repaired-Viceroy WASI 293 + 36, fixture 59, Axum contracts 28, final harness 443, two existing ignored. Strict lint, docs and frozen-identity guards pass; independent spec/ownership and quality reviews are clear. Actual Axum, Spin and workerd ten-case lifetime regressions pass.
  • The latest reviewed Fastly inventory is 17/20, exit 1: native-wait and all eight callback cases pass, but slow/fast origin heads miss 700/120 ms +/- 5 ms at 705.897/125.273 ms, and CPU batch start is 40.513 ms vs 34 ms +/- 5 ms. Earlier two instrumented inventories pass 20/20; they do not override the latest failure. No tolerance was widened or failed evidence suppressed.
  • Exact-candidate timing, remaining acceptance bindings, consumer and hosted qualification stay open. Local Fastly evidence uses the tracked Viceroy repair, not a stock/hosted runtime claim. Inventory stays 153 ready / 358 planned, unqualified; PR remains draft. New-head CI is pending.

See the implementation tracker, SUITES.md and changelog for scope and retained evidence. Demo/templates require no migration for test-only metadata.

Earlier Checkpoint: Spin Native Pending-Body Witness

Checkpoint 92e03fe8 extends the existing request-owned, test-only native witness to Spin. Production outbound APIs, transport, deadlines and resource policies are unchanged.

  • Observe the raw borrowed WASI read inside the unchanged deadline race, retain the first actual SDK poll result, mark a head only after successful construction and record source release after its reader/trailers/completion-writer owner drops. Do not treat cooperative/timer Pending or guest destruction as completed host teardown.
  • Require three identity-bound Spin witnesses in the existing fixture and negative controls, snapshot before collect, and preserve 34/200/205 ms targets. No observer type, scheduler, SDK call, extra poll/wakeup, compatibility path or CI invocation is added.
  • Red proof rejects host-only evidence; the actual pre-hook runtime fails for missing SDK observations. The final complete pending-body case passes: Pending at 38.238-38.321 ms, each source released once at 203.806-203.845 ms, before next returns. The pending-head control remains head/read/owner-free.
  • All five root Rust gates pass: 2,645 tests / 33 binaries, one existing ignored. Spin-native 114, actual SDK 55 on verified Wasmtime 48.0.2, complete serialized harness 442, two existing ignored. Strict native/WASM/harness lint, docs and frozen-identity guards pass; independent spec/ownership and quality reviews are clear. Earlier process-timeout, lint and PATH-ABI setup failures remain recorded, with no suppressed or widened assertions.
  • Final Spin lifetime suite remains 9/10, retaining the cold arrival failure: 163.147 ms vs 146 ms +/- 5 ms. The prior full Cloudflare timing gate, distinctly labelled Fastly native-wait proof, candidate freeze and hosted qualification remain open. No capability, runtime pin, acceptance or normative identity change. Inventory remains 153 ready / 358 planned, unqualified; PR stays draft. New-head CI is pending.

See the implementation tracker, SUITES.md and changelog for the verified scope and retained evidence. Demo/templates need no change for this test-only integration.

Earlier Checkpoint: Cloudflare Native Module Manifests

Checkpoint deb00840 repairs the existing local launcher for the selected Wrangler 4.131.0 / Miniflare 5.20260910.0-alpha / workerd 2026-09-10 combination. Production outbound APIs and adapters are unchanged.

  • Rust supplies a sorted, confined staged-file inventory: at most 128 entries / 64 MiB / eight directory levels. The selected SDK applies module rules, puts the ESModule main first and converts explicit modules into one native constructor. No legacy fallback, extra script, runtime selector or pin change.
  • Preserve exact KV IDs and the local persist/v3 layout, startup attestation, private diagnostics and owned disposal. Both smoke modes pass, including seeded config, declared-KV and outbound-progress checks. A separate bounded local diagnostic proves namespace ...02 survives process restart and is read before another write.
  • 442 harness tests pass, two existing ignored; strict harness Clippy/formatting, 200 tooling tests, consolidated dependency/forwarding/focus compiler checks, docs build/lint/format, legacy/inventory guards and scoped/historical API identities pass. Independent spec/security and quality reviews are clear. Prior root-gate counts below remain historical, not relabelled as fresh execution.
  • Both unchanged 10-case Cloudflare lifetime runs pass nine cases. The cold first case still fails: begin 8 ms vs 2 ms +/- 5 ms, then slot elapsed 153 ms vs 146 ms +/- 5 ms. Retain both failures. Each complete pending-body case passes SDK Pending/release checks; the first observes Pending at 36 ms and release at 202-203 ms after the original 200 ms cutoff. Passing that case does not waive the whole-suite failure.
  • Full Cloudflare lifetime timing, Spin/Fastly native-read witnesses, candidate freeze and hosted qualification remain open. No new CI job, capability promotion or normative identity change. Inventory stays 153 ready / 358 planned, unqualified; PR stays draft. All 30 reported checks passed at that checkpoint.

See the current implementation tracker and SUITES.md for exact scope, checks and retained evidence.

Earlier Checkpoint: Native Pending-Body Witnesses

Checkpoint 3523e121 adds finite, request-owned test-only SDK observations on Axum and Cloudflare. Prepared URI, clock, mode and budget come from shared dispatch; production policies are unchanged.

CI follow-up c6f3aaf0 uses canonical observation_cutoff naming. The unchanged guard, 28 Axum contracts and strict lint pass locally; all 30 reported checks passed at that checkpoint.

  • Observe the real reqwest chunk future / Worker ByteStream after the inner poll, before cooperative wrappers. Retain the first read result; record release after the actual SDK owner drops.
  • Snapshot three original requests after next, before collect. Require full controller-selected target/origin identity, Batch/BatchCutoff, the original deadline, Pending before cutoff and release between cutoff and return. Preserve 34/200/205 ms targets and zero-head/read control.
  • Reject missing/reused, Ready-first, late, backwards, early-released and foreign evidence. Validate optional fixture rootPath against the owned config directory with sanitized errors.
  • All five root gates pass: 2,645 workspace tests / 33 binaries, one existing ignored. Core 1,111; Cloudflare SDK WASM 105; Axum contracts 28; harness 438, two existing ignored; fixture core 59 with/without witnesses; tooling 200. Strict native/WASM fixture lint, docs and audit checks pass. Spec/quality reviews are clear; earlier sandbox/process-timing failures remain recorded without widened assertions.
  • Axum's full 10-case local lifetime suite passes. Cloudflare runtime attempts fail before attestation: Wrangler 4.131.0/Miniflare 5 alpha needs a manifest/module migration; its converter refuses modulesRules. Failed runs get no credit; SDK-reader unit tests do not substitute for actual fetch/runtime proof.
  • Spin/Fastly native witnesses, Cloudflare runtime integration, strict lifecycle qualification, candidate freeze and hosted evidence remain open. No new CI jobs, dependencies, runtime pins, capability promotions or normative identity changes. Inventory stays 153 ready / 358 planned, unqualified; PR stays draft.

See the implementation tracker and SUITES.md for exact checks and retained failures. The prior Fastly dispatch checkpoint retains its SDK-dispatch-only scope and three guarded proofs.

The previous focus compiler checkpoint added numeric and compiler ownership through the existing grouped Rust check; its 434-test harness and other verification remain recorded separately, not relabelled as fresh evidence. Earlier provider-boundary proofs, typed-focus approval/CLI lint, audit formatting and tooling consolidation retain their scopes and open gates.

Earlier Checkpoint: Registration Observations And Connect Capabilities

  • Implement the approved Fastly request-scoped observation handle and immutable numeric snapshots. Count eligible configurations separately from actual registrations, include failed and late returns, retain successful late registrations, and warn at 50 ms without exposing request/provider details. Callback-owned recording freezes without retaining keys, scopes, bodies or SDK work. Preparation and handlers receive the canonical handle.
  • Publish outbound-connect-timeout: Fastly declares independent provider-backed support as Native; Axum, Cloudflare and Spin retain BestEffort acceptance. Fastly's split-free flexible phase budgeting is Native. These declarations do not qualify hosted connect precision, DNS/TLS coverage, blocking-call preemption, deadlines or cancellation.
  • Verification on the reviewed tree: all five root gates pass, with 2,633 tests in 33 binaries, zero failures and one existing ignored test. Strict workspace Clippy, formatting, workspace feature checks and the Spin target check pass. Four native CLI matrix cards execute five passing tests; 85 manifest tests, docs checker/self-tests, 57 relevant tooling tests and legacy/digest guards pass. The preceding registration checkpoint also passes the guarded 291-test Fastly WASI phase and strict WASI Clippy.
  • Bind only the deterministic Fastly registration-observation acceptance leg. Inventory is 85 ready / 426 planned, across unchanged 255 criteria / 511 identities, still unqualified. CLI metadata tests are not credited as adapter SDK/runtime timing evidence. No new CI jobs, runtime selectors, compatibility paths or SDK/runtime pins.
  • Still open: remaining native forwarding/encoding/lifetime acceptance, retained strict local timing and provider-recovery/reference-parity failures, consumer/hosted evidence, Image Optimizer entitlement and its unknown synchronous blocking horizon, remaining resource-policy decisions, candidate freeze and qualification. No deployment or hosted promotion was performed. The PR remains draft; green build checks alone do not close these gates.

Checkpoint: 2cbbc549, following registration observations at 00a9edbd. The implementation tracker records exact scope, evidence and remaining gates. This checkpoint is historical; current readiness and check status are recorded above.

Earlier Checkpoint: Response-Owned Outbound Handoff

  • Add EZ-6 criterion 20 to the existing local lifetime runner for Axum, workerd and Spin. One exact response-body owner retains the original pending begun origin and three-slot batch across dispatch; the body collects those same handles, without replay or another transport.
  • Hold all source heads, payload and EOF until the native client validates a canonical, bounded NDJSON prelude. Require closed callback/live body scopes and Cloudflare/Spin's fixture-only dispatch-return signal. Validate causal source write-start order, strict original arrival cutoffs and valid late collection. This names EdgeZero dispatch/native handoff, not literal exported callback return or hosted delivery.
  • Harden the same HTTP reader against ambiguous framing, short/fragmented prefixes and notification after waiting for a gated tail. Keep exact two-record bounds, malformed-evidence controls, preexisting resource-owner counts and exactly-once terminal family release. Retain primary client failures, interrupted captures, cleanup errors and client/runtime logs.
  • Verification: all five root Rust gates pass (2,568 tests / 32 binaries, one ignored); 263 harness tests (one ignored), 54 fixture tests, strict harness/core/all-four-entrypoint Clippy, 51 Node guard tests, docs checks and dependency audits pass. Independent ownership, evidence, code-quality and verification reviews are clear.
  • Reviewed local handoff passes on Axum, Spin and workerd. Fastly records this criterion as inapplicable and still runs its original nine cases. Complete suites remain red: Axum 5/10, repaired Viceroy 5/9, Spin 6/10, workerd 5/10. Strict source/arrival/cadence misses and earlier failed observations are retained, not waived by the new case.
  • Inventory remains 33 ready / 478 planned, unqualified. D6, native body-entry witnesses, cleanup/callback cases, candidate freeze and authorized hosted qualification remain open. No production API, SDK/runtime pin, capability or frozen requirement changes; no deployment.

Checkpoint: abbc2c11. Detailed local artifacts and failures are recorded in the D6 response-handoff tracker checkpoint.

Earlier Checkpoint: Core Mixed-Observation Acceptance Binding

  • Activate only EZ-6/acceptance/22@unit/core, replacing its placeholder with five executable receipt/accumulator proofs. Run six exact three-slot cases: fresh and mixed check/next/collect under Completed, Cutoff and an original typed driver failure. Stage terminal production after both mixed receipts transfer.
  • Preserve the earlier four-slot partial-failure/cutoff proofs and out-of-order test. Assert precise states before merge, cardinality and termination afterward, original error identity, no premature body consumption, exact payloads and one body-owner release. Invalid insert/merge recovery retains both inputs.
  • Broaden one existing grouped CI execution to --lib outbound::; discover all 56 exact sentinels, including the original 51 redirect proofs. No new job or per-test CI list. The old redirect-only selection fails before execution because it omits the five batch proofs. Use the shared raw Cargo log path without relabelling older evidence.
  • Final verification passes all five root Rust gates: 2,568 tests in 32 binaries (one existing ignored), strict workspace Clippy, formatting and feature/Spin checks. The grouped outbound command passes 296 tests; 37 evidence-checker and 14 nonzero-guard tests pass, as do docs/tooling and dependency audits. The sandboxed full run denied existing TCP binds; unchanged loopback-enabled and final reviewed runs pass. Independent specification, tooling and code-quality reviews are clear.
  • Inventory is now 33 ready / 478 planned across the unchanged 255 criteria / 511 legs, still unqualified. All four adapter legs of criterion 22 remain planned. Prior runtime timing misses, native body-entry witnesses, response-owned/callback lifecycle cases, candidate freeze and authorized hosted qualification remain open. No production API, implementation behavior, SDK/runtime pin, capability, frozen requirement or deployment changes.

Checkpoint: 279c5a93.

Earlier Checkpoint: Buffered Cap During Batch Progress

  • Add the ninth caps-while-waiting lifetime case through the production clients: 128 KiB decoded gzip trickling over 300 ms, independent 1 MiB decoded / 64 KiB buffered / 256 KiB encoded limits, healthy B/C siblings, and the original 34/534 ms batch timeline.
  • Record original slot-error provenance before bounded report-body reads. Prove the written over-cap prefix and both healthy payloads precede guest-response return in the same host clock, and that return precedes the source ending. Retain exact partial-write evidence, bounded encoding/retries and strict source cadence; no guest dependency or extra production path.
  • Add negative controls for report-generated errors, impossible elapsed times, bunched writes, incorrect schedule labels and post-return sibling writes. Independent source, guest, validator and documentation reviews cover the checkpoint.
  • Verification: all five root Rust gates pass, including 2,568 tests in 32 binaries (one ignored); 252 shared-harness tests (one ignored), 51 qualification guest tests, strict harness/all-four fixture Clippy, formatting, docs lint/format, dependency/legacy/docs/inventory guards and 36 evidence-checker regressions pass.
  • Final local cap case passes on Axum, repaired Viceroy 0.21.0 and Spin 4.1.0. Workerd returns the correct original typed cap error early but fails its source-write precondition at 5.055 ms late against the unchanged 5 ms allowance. Preserve that failed run instead of substituting an earlier passing sample. Complete nine-case suites remain nonzero (8/9, 7/9, 7/9 and 6/9 respectively), with the other timing misses recorded in the tracker.
  • No production API, outbound transport, scheduler, SDK/runtime pin or capability changes. D6, native SDK body-entry witnesses, acceptance bindings, candidate freeze and hosted qualification remain open; inventory is still 32 ready / 479 planned and unqualified. No deployment or qualification promotion.

Checkpoint: 8c11307d. Final local artifacts and individual failed timing preconditions are recorded in the D6 buffered-cap tracker checkpoint.

Earlier Checkpoint: Reviewed Spin SDK Integration

  • Consume the reviewed Spin SDK contribution at immutable fork revision 6c59ed5b4fcb377072763aee0b27a3abdd8edb72 (WASIp3 0.10.0 / wit-bindgen 0.63.0). Align the root, demo, reusable/request-fidelity fixtures, generator and all four application lockfiles.
  • Explicitly detach the SDK's new join handle at one private provider launcher. Preserve the existing scheduler, registration-before-spawn and cancellation ownership; do not add a compatibility entrypoint or another executor.
  • Migrate native SDK request construction in its WASM refusal contract to the SDK executor. Preserve the exact status/header/body and handler/middleware assertions. Update the dependency audit, design, implementation tracker and changelog together.
  • Local verification passes all five root gates (2,568 tests, one existing ignored), 35 Spin WASM contracts, 21 SDK-resource cases, the actual eager component, both strict Spin WASM Clippy configurations and all eight feature combinations. Demo tests/builds and the full ignored fresh generated-workspace gate pass, including all three WASM adapters and fallback TCP checks.
  • Spin 4.1.0 passes all 68 wire exchanges with 84 correlated origin receipts. The eight-case D6 lifetime run passes six cases but retains two strict timing failures: slow-origin Begin timing and the CPU B-write window. The complete run still exits 1; no target is relaxed or capability promoted.

Checkpoint: 87b5e1ce. These are local regression results, not a frozen candidate or hosted qualification. D6 closure, native body-entry evidence, consumer acceptance and hosted qualification remain open. The acceptance inventory remains 32 ready / 479 planned and unqualified.

Earlier Checkpoint: Fourth-Slot Per-Call Expiry

Implemented in cd8e77e4, within the approved D6 acceptance work:

  • Add an eighth lifetime case through the same production clients. Preserve the original slow-origin case, original 534 ms cutoff and A/B/C schedules; only fourth slot D has a 300 ms per-call timeout. Late observation returns indices 0/1/3 with typed D timeout and exact lossless collection states. D's source never responds and closes boundedly; this is not SDK-cancellation evidence.
  • Validate C's actual late response or supported late peer close in both slow cases. Reject missing terminal evidence, early payload, extra writes, wrong timeout provenance and collection-stamped latency. Keep timeout-contract diagnostics separate from overall strict source-timing acceptance.
  • Verification: all five root Rust gates pass, including 2,568 workspace tests in 32 binaries (one ignored); 243 shared-harness tests (one ignored), 49 qualification guest tests, strict harness/guest Clippy and formatting, docs lint/format, legacy/docs/inventory guards and 36 evidence-checker regressions pass. Independent specification, cleanup, code-quality and evidence reviews are clear.
  • Reviewed local Axum, repaired Viceroy 0.21.0, Spin 4.1.0 and workerd 2026-09-10 runs all retain D's exact 300 ms PerCallTimeout and completed siblings. Only Spin passes the complete new case; all four eight-case suites exit 1 under unchanged strict timing targets. Source-timing misses remain visible, and no passing earlier sample replaces a failed final run.
  • No production API, transport, scheduler, runtime/SDK pin or capability change. Inventory stays 32 ready / 479 planned, qualified: false. Native body-entry evidence, remaining lifetime/redirect acceptance, candidate freeze and hosted/resource qualification remain open. No deployment or qualification promotion is authorized by this checkpoint.

Earlier Checkpoint: Readiness-Driven Test Origins

Implemented in d3f48564, under the requester's scoped fixture-accept approval:

CI diagnostic follow-up 11b8f2c4 adds underlying errors to two existing request-fidelity version-test assertions. The prior Linux workspace failure occurred before the collector tests; its OS/process cause was not exposed and remains unproven. The fresh run passes both the request-fidelity step and the complete shared-harness tests on Linux. No test conditions, retries, version policy or subprocess behavior changed; other CI checks and provider qualification remain separate.

  • Replace the test origin's 5 ms accept sleep with one private, pinned Mio 1.2.1 readiness owner per listener. Register both listeners/wakers before spawning; drain accepts through WouldBlock and check sticky stop before accounting or worker creation. No alternate collector, raw-handle code, synthetic TCP wakes or production dependency changes.
  • Stop releases existing gates and attempts both wakes outside the accounting mutex. Each collector retains its wake owner until exit; a finite 50 ms poll fallback protects against missed/failed stop wakes. Preserve 128 cumulative/eight-active connections, existing parser/I/O/gate budgets and collector-before-worker joins. This is not a realtime or whole-shutdown guarantee.
  • Add colocated queue, wake-race, idle finish/drop, stop-after-accept, capacity, partial-header and failed-wake regressions. Synchronize both collectors past their last stop check before failure injection. Removing the poll timeout makes both watchdog tests fail cleanly around two seconds, with real-waker rescue and joined cleanup; restore the finite timeout before all passing runs.
  • Verification: all five primary Rust gates, 2,568 workspace tests (one ignored), 238 harness tests (one ignored), 48 qualification-enabled fixture tests, strict harness Clippy/formatting, docs lint/format, legacy/docs/inventory guards and 36 evidence-checker regressions pass. Independent code, test and documentation reviews are clear; forced failure of the second OS thread spawn remains reviewed rather than directly injected.
  • All four ordinary outbound-wire suites pass: 68 exchanges / 84 correlated origin requests each. Final seven-case timing runs remain failed/unqualified: four controls pass per target; fast-origin passes on Axum and repaired Viceroy; only Spin's CPU case passes. Slow-origin misses remain on every target. Retain actual all-offset failures rather than selecting earlier passing runs or relaxing targets.
  • Fixture accept polling is now implemented. Remaining timing/native SDK body-entry evidence, measurement-approval and hosted qualification gates stay open. Requirements, public APIs, production transport/code, SDK/runtime pins and capabilities are unchanged. Inventory remains 32 ready / 479 planned, unqualified; no deployment or candidate freeze is authorized.

Earlier Checkpoint: Same-Client CPU Calibration

Implemented in 0d09236a, under the requester's scoped calibration approval:

  • Add one buffered calibration Send through the same injected client after origin consumption: 64-byte cap, one-second timeout, absolute request-start-relative 520 ms safety deadline, bodyless 204. A/B/C wait boundedly for its causal sample; only a scheduling estimate uses the midpoint.
  • Keep A's entire arrival interval inside that EdgeZero Send. End the calibration hold at 450 ms plus the causal upper offset bound, then use the original timer for next500. Preserve A180/B530/C900, batch34/cutoff534, CPU520..560, all-offset source proof, invocation-epoch bracketing and strict SDK receipt cutoffs.
  • Close an independently reproduced evidence defect: calibration receipt cannot precede the source payload's pre-write sample. This proves necessary host ordering, not peer receipt; a post-flush sample could incorrectly reject legitimate preemption.
  • Verification: all five primary Rust gates, 2,568 workspace tests (one ignored), 229 harness tests (one ignored), 48 qualification-enabled guest tests, strict native/three-target fixture Clippy, formatting/docs lint, legacy/docs/inventory guards and 36 evidence-checker tests pass. Independent guest, host and documentation reviews are clear.
  • Final local CPU characterization: workerd passes unqualified; Axum, repaired Viceroy and Spin fail the full B-write timing proof. All four complete seven-case suites still exit 1 because earlier slow/fast timing misses remain; four controls pass per target. Retain the earlier intermediate runs rather than selecting passing samples.
  • The shared origin's existing 5 ms accept polling adds fixture-side uncertainty. Its bounded readiness/wake replacement is separately proposed, not implemented or approved here. No production API/transport/pin/capability changes, deployment, candidate freeze or hosted qualification. Inventory stays 32 ready / 479 planned, unqualified; native SDK body-entry evidence and remaining acceptance gates stay open.

Earlier Checkpoint: CPU Stretch Characterization

Implemented in 44baaf3b:

  • Extend the existing Rust harness/guest to seven lifetime cases with bounded synchronous CPU work from 520 to 560 ms, the original 34/534 ms batch start/cutoff, B's unchanged 530 +/-5 ms source target, and both permitted B outcomes. No new production API, transport, runtime pin or capability change.
  • Require the complete actual B write to fit every possible causal host/guest clock offset. Bracket core's batch epoch by the full measured invocation before reconstructing receipt arrival; equality at the original cutoff cannot pass. Reject collection-stamped arrivals, early C bodies hidden by late closure, frozen/backwards clocks and finite-work exhaustion.
  • Verification: all five primary Rust gates pass, including 2,568 workspace tests (one ignored); 223 shared-harness tests (one ignored), 47 qualification-enabled fixture tests, native/three-target fixture Clippy, formatting, docs lint, legacy/docs/inventory guards and 36 evidence-checker tests pass. Independent reviews found and closed the batch-epoch proof issue.
  • Local runtime characterization remains failed/unqualified on all four targets: each has four passing controls, the earlier slow/fast timing misses, and a CPU source-timing precondition failure. Cold-connection clock bounds are too wide/late to prove B's required write. These failures do not establish a production batch defect and do not qualify the CPU scenario.
  • Fixture-only clock calibration is proposed, not approved or implemented. Native SDK body-entry evidence, remaining lifetime/redirect acceptance and hosted qualification stay open. Inventory remains 32 ready / 479 planned, unqualified. No deployment or candidate freeze is authorized.

Earlier Checkpoint: Pending Cutoff Lifetime Probes

Implemented in 73ed39ba, following the main integration:

  • Extend the existing Rust harness and production-client guest to six lifetime cases, adding pending heads and host-written heads with withheld bodies. Preserve the original request-start-relative 200 ms observation cutoff, genuinely Pending polling, three unresolved slots and the 205 ms return target.
  • Fix two independently reproduced evidence bugs: record operation return before formatting the report, and require host phases before actual guest-response return rather than merely before a later fixture release. Preserve event-index ordering even when timestamps tie.
  • Verification: all five primary Rust gates, 212 shared-harness tests (one ignored), 43 qualification-enabled fixture tests, strict native/three-target fixture Clippy, formatting and legacy/docs/inventory guards pass. The existing CI groups cover these tests; no per-case workflow is added.
  • Local characterization: the two new cases pass on Axum, repaired Viceroy 0.21.0, Spin 4.1.0 and workerd, returning between 201.179 and 204 ms. Each complete six-case run still exits 1 because its original slow/fast timeline cases miss strict targets. Do not present these as passing six-case inventories or widen tolerances.
  • Pending-body evidence remains explicitly host-head-written-only, not proof that the SDK consumed headers or entered a body read. Full criterion 16, remaining D6 work and hosted qualification stay open. Production outbound code, signed SDK/decoder pins, frozen requirements/interfaces and capability levels are unchanged; inventory remains 32 ready/479 planned and unqualified.
  • Main integration preserves route-aware admission and adapter-owned egress while adding request fidelity and pre-dispatch hooks; the signed SDK pin and existing outbound implementation remain intact. The current checkpoint does not rerun the expensive ignored generated-workspace matrix.

Earlier Checkpoint: Coordinated Spin SDK Executor Upgrade

Implemented in 7b68f737:

  • Create the public Stackpop Spin SDK fork and submit spinframework/spin-rust-sdk#169. Pin the signed SDK revision e65ff90dadf4173b08b14194e878ad3acd555337; its DCO check passes. Upstream Rust 1.94 CI remains separate from local Rust 1.98.1 verification.
  • Upgrade the coordinated chain to WASIp3 0.9.0 / wit-bindgen 0.62.0, retaining WASI 0.3.0 and one SDK executor. The upstream nested-spawn regression fails on the old chain and passes after this change. No observer-driven producers, second executor, vendoring or compatibility alias.
  • Hard-migrate callers to spawn_local. Root, demo, reusable fixture and generator use identical Git URL/revision and package version; guards compare all three. Lock changes stay within the affected SDK/bindings/macro graph. Decoder/map contracts remain unchanged.
  • Verification: all five root Rust gates; both affected Spin WASM strict Clippy combinations; 35 Spin WASM contracts, 21 resource tests and the eager ownership component on exact Wasmtime 48.0.2; 60 demo-core, 42 fixture-core, 33 fresh-scaffold-core tests, demo/scaffold Spin compile checks and scaffold Spin Clippy; docs/build/dependency/legacy/evidence guards pass. The expensive ignored full generated-workspace gate was not rerun locally.
  • Actual Spin 4.1 wire controls pass 68 exchanges/84 receipts. Slow-origin batch starvation is removed. The four fixed lifetime cases remain two passes and two strict timing failures; do not widen tolerances or extend deadlines. A pre-existing host worker diagnostic was reproduced after requested shutdown with identical guest artifacts; retain that limitation, not a clean-hosted-shutdown claim.
  • Keep the frozen requirements/Interfaces, all outbound guarantees and capability values unchanged. Inventory stays 32 ready/479 planned and qualified: false; full acceptance binding, candidate freeze and authorized hosted qualification remain open.

Earlier Checkpoint: Begin/Batch Lifetime Probes and SDK Gap

Implemented in 834a009e:

  • Add four fixed EZ-6 lifetime scenarios to the standalone Rust harness and existing production-client guests. Reuse the bounded origin parser/collector, process supervision and cleanup; no new transport, executor or host-language runner.
  • Preserve actual host receipts/writes, measured guest observations, original cutoffs, arrival elapsed, typed mediator errors and exact mixed collection states. Add conservative quiet-window clock bounds, cold-entry and timestamp-reanchoring regressions, and late-cleanup failure coverage.
  • Verification: 203 harness tests, 42 fixture-core tests, the five root Rust gates, strict harness/fixture lint, docs and legacy/evidence checks pass. The existing Axum redirect regression remains 68 exchanges/84 receipts.
  • Actual local lifetime results: Begin/unobserved-batch cases 4/5 pass on all four adapters. Slow/fast timeline cases 1/2 do not pass their strict timing targets. Spin additionally loses the slow-origin batch before dispatch; tracing confirms newly spawned work can wait until the origin responds in pinned wit-bindgen 0.57.1.
  • Remove all temporary diagnostics. A compatible audited SDK scheduler repair remains an explicit dependency decision, not an observer-polling workaround or another executor. No production API, SDK/runtime pin or capability claim changes in this checkpoint.
  • Keep all new lifetime bindings planned: inventory remains 32 ready/479 planned and qualified: false. CPU-stretch, remaining lifetime/registration/consumer acceptance, candidate freeze and authorized hosted qualification remain open.

This is a verified test/diagnostic checkpoint, not a passing full lifetime suite or completed feature qualification.

Earlier Checkpoint: Standalone Test Harness and Redirect Bindings

Implemented in b5f94c70:

  • Move the shared native Rust runner to crates/edgezero-test-harness, with a non-published standalone workspace and its own lockfile. Remove the old runner package/directory without an alias or duplicate implementation. Keep deployable fixture apps separate and unit/contract tests in their owning crates.
  • Preserve fixture cwd, target directories, staging, selected runtimes and evidence paths. Record the fixture and runner locks as different artifacts. Preserve all 23 runner registry package versions/checksums and all 329 retained fixture-lock entries; the root lockfile and SDK/decoder/runtime pins are unchanged.
  • Fix launcher cwd selection so repository-pinned tools work when invoked from outside the checkout. Verify both an unrelated working directory and paths containing spaces with real Axum exchanges. Cargo metadata regressions prove isolation from both production and fixture dependency graphs.
  • Extend the same wire matrix to Axum and Fastly: all four adapters pass 68 exchanges/84 ordered origin requests. Preserve Spin's bodyless/upload/scope/SDK-terminal controls, Fastly smoke exit 0 and its separate 12/18 plus 23/43 progress inventories with unqualified exit 2.
  • Strengthen existing core redirect tests with the literal signed-date header, first-hop cancellation and two-hop query-redaction controls, preserving the older tests. Bind EZ-9 unit criteria 01-15 to all 51 guarded core tests and runtime criteria 16/17 to six exact environment legs.
  • The inventory is now 32 ready legs and 479 planned, with qualified: false. Readiness records executable assertions, not consumer adoption, applicability approval or exact-candidate qualification.
  • Update active scripts, guides, changelog, tracker and CI fetch/cache/fmt/lint/test commands for separate workspaces. Axum/Fastly/Spin execute the declared runtime paths with strict inventory guards; workerd runtime CI activation remains explicitly pending tool setup, separate from its passing local execution.
  • Verification: 185 standalone harness tests, 35 fixture-core tests, 36 evidence-checker tests, 51 guarded core redirect tests, all five root Rust gates, strict fixture/harness lint, docs checks, dependency/legacy guards and workflow lint pass. The one ignored Rust subprocess helper is explicitly exercised by parent tests. Independent reviews have no remaining concrete findings.

This completes the approved runner relocation and local redirect binding checkpoint, not the whole adoption plan. Lifetime/registration acceptance, successful native optimizer transformation/redirect/performance evidence, consumer integration, candidate freeze and authorized hosted qualification remain open. Optional idle-body policy is not silently adopted. No deployment, capability promotion or production outbound behavior change occurs.

Earlier Checkpoint: Local Workerd Redirect Acceptance

Implemented in 4e64db99:

  • Run Cloudflare through the same Rust outbound-wire matrix and bounded two-origin servers as Spin. A fixture-only route obtains the production client from the canonical request context; no Fetch replacement, parallel outbound transport or new host-language runner is used.
  • Pass all 68 local workerd exchanges and 84 ordered origin receipts. Assert exact bytes, coding and cap parity, manual redirects, unread intermediate-body progress, stopped/limited final bodies, cross-origin hops, streamed final responses and terminal metadata.
  • Preserve exact provider error classification: workerd requires 502 Transport at ResponseBodyRead with the observed sanitized Unknown cause; Spin retains its existing Protocol expectation. Negative controls reject wrong reasons, phases, causes and status.
  • Record the selected workerd executable/version and effective compatibility settings. Local execution uses Wrangler 4.131.0, workerd 2026-09-10 and worker 0.8.5 with the existing compatibility date/flags. Rebuild only the exact cached wasm-bindgen 0.2.129 CLI offline in a temporary directory; no global tools or project pins change.
  • Extend the common cleanup scope before local binding/build/startup work. Preserve primary failures, clean partial state and finish runtime/origins before publishing an inventory. Accept wrapper exit 143 only after confirmed sent TERM; premature 143 and failed/absent signaling remain rejected by regression controls.
  • Independently review the probe and host lifecycle changes. Pass 35 fixture-core tests, 184 harness tests, strict fixture/native/provider-target checks, all five root Rust gates, documentation checks and dependency/evidence/legacy guards. The intentional Rust subprocess helper remains parent-test-driven.
  • Rerun Spin's unchanged 68/84 inventory and bodyless/upload/scope/SDK-terminal controls. Preserve Fastly's 12/18 and 23/43 progress inventories with their unqualified exit 2. Missing workerd tools produce unsupported exit 2 and no success inventory.
  • Update the changelog, fixture guide and master tracker. No production API, SDK/runtime pin, capability promotion, hosted deployment or candidate freeze occurs.

Successful native optimizer redirects, conditional idle-policy coverage, complete criterion-level binding and consumer/hosted acceptance remain open. The inventory remains eleven ready legs, 500 planned and qualified: false.

Earlier Checkpoint: Rust Spin Wire Harness

Implemented in 361da4f1:

  • Replace the Python wire runner with --adapter spin --suite outbound-wire in the existing Rust fixture harness. No Python or Node replacement runner, compatibility wrapper, dependency changes or second process supervisor remains.
  • Reuse bounded raw HTTP parsing/capture, two tracked origins, file-backed runtime logs and process-group ownership. Preserve the unchanged Spin guest and all 68 exchanges/84 correlated origin requests, plus bodyless, upload, canonical-scope and attached Begin/Batch handoff controls.
  • Fail closed on late worker errors, malformed framing, non-text wire payloads, altered raw handoff bytes, premature runtime exits and cleanup failures. Keep unconfirmed cleanup retryable and reap timed-out signal helpers within bounded deadlines.
  • Verify 180 Rust harness tests and 29 qualification-enabled fixture-core tests. The one ignored Rust subprocess helper is explicitly invoked by its parent regression tests. Independent network and process-cleanup reviews report no unresolved findings.
  • Pass all five root Rust gates, strict fixture Clippy/format, documentation lint/format/build, workflow lint and dependency/evidence/legacy guards.
  • Rerun the actual pinned Spin runtime with exit 0 and the complete 68/84 inventory. Preserve Fastly smoke exit 0 and both progress inventories (12 probes/18 receipts plus 23 probes/43 receipts) with their required unqualified exit 2.
  • Keep the same CI job and pinned runtimes. Validate the Spin inventory predicate against complete evidence and missing-case, wrong-count and cleanup-failure controls. Preserve all production outbound contracts, frozen requirements and qualification status.

This checkpoint is test-tooling only. No hosted deployment, capability promotion, candidate freeze or optimizer performance claim is made. The inventory remains eleven ready legs, 500 planned and qualified: false.

Earlier Checkpoint: Provider Redirect Boundaries

Implemented in 699cd49c:

  • Exercise the actual clients through the shared Send/Follow pipeline: Cloudflare guest fetch fixtures, local Spin HTTP wire controls and the explicitly selected repaired Viceroy.
  • Pass the complete Cloudflare suites: 82 guest library tests, 29 contracts and 12 Node fixture tests. New cases cover coding/cap parity, unread intermediate bodies, final stream behavior, original deadlines, drop cancellation and fixture cleanup. Guest abort observations are not workerd cancellation evidence.
  • Pass 68 Spin redirect exchanges with 84 origin-observed requests, preserving the existing bodyless, upload, canonical-scope and attached Begin/Batch handoff controls. Teardown joins request handlers before validating their outcomes and rejects unexpected runtime exits. Both false-green paths reproduce before the fix and are rejected afterward.
  • Preserve Fastly's original 12 progress probes and 18 origin receipts; require an additional 23 redirect probes and 43 receipts. Validate exact typed cap causes, normal-decoding unsupported-token passthrough, two-origin behavior, pending unread heads and native optimizer unavailable boundaries.
  • Close the reviewed evidence-validation gap: correlate overflow Follow metadata or exact hop-failure indices, reject contradictory/missing/foreign data, and preserve the original inventories. The new regression fails before the fix; all 149 harness and 29 qualification-enabled fixture-core tests pass afterward.
  • Run the Spin wire harness in its existing CI job using the checksum-verified pinned CLI. Extend Fastly's existing predicate without adding another job or weakening its required unqualified exit-2 disposition.
  • Pass all five root Rust gates, affected WASM clippy/build checks, docs lint/format/build, workflow lint and dependency/evidence guards. Core remains at 1,044 passing unit tests.
  • Update the changelog, fixture guide and requirement tracker. No production API, SDK/decoder/runtime pin or capability guarantee changes in this checkpoint.

Workerd/origin parity, successful optimized-first-hop redirects, conditional idle-policy coverage, complete criterion-level acceptance binding, candidate freeze and consumer/hosted qualification remain open. Native optimizer dispatch is preserved; unavailable-boundary controls do not establish transformation or performance. No deployment is performed. The inventory remains eleven ready legs, 500 planned and qualified: false.

Earlier Checkpoint: Fastly Cooperative-Wake CI Repair

Implemented in f57b8130:

  • Reproduce the red reusable-HTTP smoke locally: the first real outbound origin request stalls after earlier lifecycle probes pass. The dependency future-incompatibility warning is not the failure.
  • Fix the existing Fastly progress loop to honor cooperative source wakes before host selection. A yielding body must not enter an empty-handle wait until its 30-second deadline.
  • Preserve direct/nested progress, retained older wakers and current-caller notification, expiry publication and normal waiting on later pending polls. Retain only a shared notification flag in progress state, not a parent task waker.
  • Add three colocated deterministic regressions. The direct/nested and retained-waker assertions fail before the fix; all thirty progress tests pass afterward.
  • Verify the exact unchanged Fastly smoke with exit 0 across every existing variant and assertion. All five root gates, four Fastly WASM clippy configurations, 265 guest library tests and 36 contracts pass.
  • Verify all twelve production-client progress probes and eighteen expected origin requests using the unchanged CI assertions. Its expected exit 2 preserves qualification: false.
  • Update changelog and tracker. No new CI job, public API, deadline, read-size policy, runtime pin, capability promotion or hosted deployment.

Still open: provider-runtime redirect acceptance, exact-candidate criterion bindings, resource-policy decisions and consumer/hosted qualification. All previous outbound guarantees and direct native Image Optimizer integration remain intact.

Earlier Checkpoint: Explicit Redirect Following

Implemented in e02c2360:

  • Add the approved HttpClient::follow API with a required synchronous application policy and validated 1..=20 redirect limit. Plain Send, Begin and Batch remain unfollowed.
  • Route every hop through the same supplied client, prepared dispatch and shared head finalizer. Preserve method/replay precedence, original Location evidence, unchanged request bounds, typed hop errors and arbitrary application-policy errors.
  • Replay buffered Bytes by sharing storage; never buffer streamed uploads for replay. Drop followed heads unread, without draining, decoding or applying final-body caps to intermediate payloads.
  • Preserve the original absolute chain deadline through replacement requests and final streamed consumption. Retain per-hop timeouts/defaults, injected time, B31, indexed batches and direct native Image Optimizer dispatch without another fetch.
  • Strip only Authorization, Cookie and authority overrides from automatically prepared cross-origin hops; deliberate policy replacements remain application-owned. Demo and generator enforce initial and per-hop same-origin policy.
  • Close independently reproduced scope-reentrancy and backwards-clock gaps in shared preparation/source binding/finalization, including terminal cleanup. No compatibility transport, second collector or adapter-specific cancellation engine.
  • Verification: all five root gates; 1,044 core unit tests and 14 integration tests, including 50 redirect cases; 261 native TCP library tests plus 27 contracts; 60 demo-core tests; 33 fresh generated-core tests and strict generated-workspace clippy. The separate expensive ignored full-generator gate was not rerun locally.
  • Guest verification on final source: all eight WASM feature clippy configurations; Cloudflare 74 unit/29 contract cases plus nine Node fixture cases; Spin 35 contracts/21 SDK-resource cases and an executed eager-exchange component; Fastly 262 unit/36 contract cases with zero filtering under the approved repaired Viceroy. Fastly uses the system CA bundle with TLS verification intact, not a runtime-pin promotion.
  • Update guides, capability declarations, changelog, tracker and documentation guards. Frozen requirements, approved Interfaces and audited dependency/runtime pins are unchanged.

Still open: other provider-runtime Follow parity, actual Fastly optimizer-hop mapping, consumer composition, exact-candidate criterion bindings, resource-policy decisions and hosted qualification. The inventory remains eleven ready legs, 500 planned and qualified: false. No deployment or capability promotion occurred, and the PR remains draft. Current tracker.

Earlier Checkpoint: Shared Outbound Pipeline and Ownership

Implemented in 2be1dd87:

  • Hard-cut prepared dispatch, paired time/progress, eager Begin and identity-bound batch receipts. Core shares validation, GET/HEAD settlement, normalization, decoding, limits, revocation, task cleanup and Body source binding; adapters retain provider I/O and scheduling.
  • Capture complete unpolled operations at invocation. Accept exact provisional Body owners before egress policy/framing, preserve response completion through fallback terminal, and keep source EOF distinct from terminal transmission.
  • Make nonblocking readiness observation SDK-pump-free. Preserve original deadlines, timeout causes, partial results, cutoff equality, independent caps, pooling and direct native Image Optimizer integration.
  • Retire Fastly's separately approved preparation-slack rejection while retaining B31 and actual expiry/backwards-clock checks. No compatibility aliases, duplicate lifetime registry or alternate transport remains.
  • Migrate all adapters, supplied clients, demo, generator, fixtures, guides, changelog and guards atomically.
  • Local verification: five primary Rust gates; 984 core unit tests; 261 native TCP cases plus 25 contracts; Cloudflare 74 guest unit/29 contract cases; repaired-Viceroy Fastly 262 unit/36 contract cases, zero filtered; Spin 56 guest cases. Refreshed demo default-core 55, rendered scaffold 28 and qualification fixture 29 tests pass. Strict provider lint and documentation/dependency/legacy guards pass.
  • Record the scoped interface authorization separately from historical exact-digest approvals. Normative requirements and audited SDK/decoder pins are unchanged.

Open at that checkpoint: explicit same-client redirect following (now implemented in the checkpoint above), remaining exact-candidate acceptance bindings, resource-policy decisions and hosted qualification. Local runtime evidence does not promote capabilities or prove end-client delivery. The PR remains draft. See the current tracker.

Earlier Checkpoint: Live Paired Time

Implemented in d86d4c0d, with the isolated Cloudflare test-ordering correction in f4d660b1:

  • Hard-migrate all four clients and injectors to with_time / Axum try_with_time, removing clock-only constructors without aliases.
  • Use OutboundTime for live asynchronous deadline handling. Check timers before source polling, preserve operational errors and diagnostics, retain clock high-water observations across chunks/yields, and release terminal/cancelled ownership.
  • Integrate timer-only waits into Fastly's existing progress multiplexer. Bound nested engines by original ancestor deadlines, isolate expired mailboxes, and wake outside state borrows.
  • Preserve B31, cutoffs/partial results, limits, coding/provenance and direct native Image Optimizer dispatch. Demo/templates remain aligned through standard injection.
  • Local verification passes: five required Rust gates, 2,293 workspace tests (one existing ignored test), strict provider WASM Clippy, Cloudflare 53 library/29 contract cases, Fastly 238 library/36 contract cases, and Spin 15 SDK-resource cases on Wasmtime 48.0.2. Demo, fixtures, documentation and dependency/legacy checks pass.
  • Actual-client Fastly receive probes and Spin local component functional cases pass, but both smoke suites correctly remain unqualified (exit 2). No hosted deployment or capability promotion occurred.

This completes only the approved sequencing checkpoint. The public head-first trait, settlement/context issuance, batch receipts, eager Begin, redirect integration and hosted qualification remain open. Requirements and frozen Plan 02 hashes are unchanged. See the verification audit.

Earlier Checkpoint: Bounded Synthesized Headers

Added in d47c8026:

  • Immutable app-level and validated Fastly per-request response-header sets, bounded to 32 values and 4096 serialized field-section bytes. Preserve canonical field precedence, ordered repeated values, typed overflow retry suppression, completion ownership and fallback budgets.
  • Cover normalized ingress errors, admitted source errors, canonical 404/405 and adapter fallbacks; do not decorate matched application responses/errors, admission refusals or failed application assembly.
  • Align the demo, generated application, specifications, implementation plans and changelog.
  • Local verification: all five required Rust gates, 2,193 workspace tests, 99 fixture tests, strict provider-target Clippy and documentation checks pass.
  • Actual local runtime evidence: 55 header wire cases, 40 assembly wire exchanges and one Axum pre-listener startup control pass. HEAD payload absence requires clean EOF; Fastly reports are request-correlated, and failed state release is observed before retry.
  • Evidence: header matrix, assembly and repeated header matrix.

This checkpoint is not completion of the broader adoption plan or hosted qualification. The full head-first/eager/redirect migration, Fastly's failed G2 preserving-read proof, all nine failed Fastly framing SDK parity controls, Cloudflare overflow recovery, consumer integration and exact-candidate hosted qualification remain open. No capability is promoted, no compatibility layer is restored, and the PR remains draft.

Reviewer correctness follow-up

Addressed R1–R4 in 92111839:

  • Axum validates exact-length source EOF before handing Hyper the final frame, validates transmitted zero-length streams before handoff under the original deadline, and preserves detached fallback payloads with HEAD suppression.
  • Batch completion requires clean driver EOF; trailing failures and malformed events are checked even after the last item or with zero slots. Collected slots survive driver failure.
  • Application assembly rejects changed platform metadata after successful configuration while preserving hook-error precedence.
  • Added regression coverage and aligned specs, implementation plans, guides, changelog, and the docs checker. Local workspace tests (720 core unit tests), strict native/WASM clippy, explicit adapter contracts, and demo/docs checks pass. Local WASM results are compilation/lint evidence; hosted lifecycle qualification requires separate deployed runtime artifacts.

Main merge and shared-policy follow-up

Addressed R5 and R6 in 91f4af38, merging main 683202c6 without restoring superseded APIs:

  • Axum, Cloudflare, and Spin share OutboundBatch::from_futures; adapters retain preflight validation, exchange ownership, clocks, and terminal samples. Ready results remain retained, driver EOF/index invariants remain enforced, and cutoff releases pending futures before reporting termination. Fastly retains its distinct provider-handle driver.
  • All four adapters share ResponseBodyPolicy and apply_response_body_policy, preserving declared-length checks, encoded limits, decoding/header rewriting, decoded limits, passthrough, and rechunk order. Transport reads, bodyless handling, deadlines, cancellation, and final buffering remain adapter-owned.
  • Main's opt-in reusable application lifecycles now use fallible assembly, fresh request registries, canonical admission, and lazy owned egress. Fastly custom callbacks return terminal Result<(), E> through a wrapper that prevents SDK-generated second responses.
  • Migrated the reusable fixtures to the current SDK pins and hard-cut APIs; expanded the legacy guard and CI coverage. Updated the outbound specification, lifecycle specification, implementation plans, guides, and changelog.
  • Local verification passes: full workspace tests (733 core unit tests), strict native/provider-WASM Clippy, Cloudflare/Fastly/Spin runtime contract suites, reusable fixture tests (5 core and 26 harness), Fastly fixture finalization and Viceroy 0.21.0 HTTP smoke, demo host/lint/WASM checks, fresh generated-project compilation, and docs/dependency/legacy gates.
  • Physical partial delivery is proved by a client-observed handshake before upstream truncation. Immediate zero-byte source failures are verified through exactly-once correlated completion with no fallback; only no-status EOF/reset is accepted, never timeouts or malformed responses. Local smoke evidence does not promote deployed capability guarantees.

R7's dependency-audit-script deduplication remains deferred; dependency contract checks remain unchanged and pass.

Bounded native ingress and config follow-up

  • Axum owns a bounded HTTP/1 accept loop and parser boundary: finite live connections, joint wire-head bytes, field count, parser buffers, absolute header-read deadlines, and idle keep-alive expiry. Capacity is acquired before parser construction; saturation closes the one accepted overshoot without spawning a refusal task. HTTP/2 is not enabled.
  • Axum loads immutable local config snapshots before listener binding. Validated file, entry, decoded key/value, registry-residency, and sequential startup-staging limits are enforced before owned snapshot allocation. Duplicate keys are rejected; allocation overflow fails startup; unavailable declarations remain typed failed bindings.
  • Hard-cut all four adapters and fixtures to shared UTF-8 ConfigValue returns. Axum constructors require ConfigStoreLimits; from_map is fallible. Reads share payloads, and local config push updates require a server restart.
  • Pin the parser allocation audit to serde_json 1.0.150, hash canonical JSON incrementally, and release extraction's raw handle before secret retrieval. A moving-reallocation malformed-root probe measured 36,428 requested peak bytes under a 100,060-byte staging cap, with zero retained bytes after rejection.
  • Add real local TCP tests for HEAD/204/304 suppression, stalled producers, non-reading clients, disconnects, exactly-once completion/resource release, and connection-capacity recovery. These prove the adapter-owned native boundary, not confirmed client delivery.
  • Align the demo, generator, migration guide, specs, plans, and changelog. Fresh local verification passes: full workspace tests (737 core unit tests), strict native and three-provider WASM Clippy, demo tests/lint, generated all-adapter host/WASM builds, docs checks, and dependency audits.
  • Hosted config preallocation and response-lifecycle qualification remain open. Provider whole-value materialization and complete RSS accounting are not certified; broad config-allocation support remains Unsupported and response egress remains BestEffort. Local emulators and guest contract tests are not deployed runtime evidence.

Bounded provider reads and lifecycle qualification infrastructure

  • Cloudflare bounded config reads now use caller-sized BYOB byte streams, strict UTF-8, independent byte allowances, clock checks and cooperative task yields. Reader termination cancels the stream; an acquisition owner also cancels a stream arriving after the waiter was abandoned. Cancellation rejections cannot publish provider diagnostics.
  • Bounded KV dispatch requires FinalizationRegistry cleanup support. Generated and demo Workers explicitly enable enable_weak_ref, and regression tests inspect their TOML rather than assuming an exact one-flag string.
  • Fastly uses the pinned config-store host ABI with one initialized caller-sized buffer and no oversized-buffer retry. Root/pointer/chunk limits, declared-total preflight, transport integrity and reassembly bounds precede envelope materialization. Duplicate reserved discriminators fail closed, deadline expiry wins metadata errors, and try_open now returns typed ConfigStoreError.
  • Publish conservative requested-payload charges and retain explicit exclusions for opaque provider buffering, JS GC retention, synchronous interruption and complete allocation certification.
  • Add default-off qualification applications through all four standard adapter entrypoints, an allowlisted finite HTTPS probe runner and a fail-closed correlated ownership-evidence validator. CI runs the validator/fixture unit tests and compiles every qualification target without deploying anything.
  • Local verification: workspace tests, strict native and three-provider WASM Clippy, feature checks, Cloudflare WASM tests (28), Fastly bounded-config Viceroy tests (18), fixture tests (7 core/26 harness), validator tests (5), demo-core tests (48), dependency/legacy/documentation checks and docs build. A temporary native Axum fixture exercised nine normal/bodyless/failure/deadline scenarios with one terminal report and one resource release each, then was stopped.
  • Still open: a bounded Spin host/backend interface or independently verified read-allocation ceiling, plus actual hosted qualification with authorized isolated targets and complete revision-pinned telemetry. The validator qualifies only ownership for scheduled probes; it does not authenticate telemetry, prove same-instance recovery, confirm client delivery or promote capabilities.

Downstream acceptance impact

Cloudflare outbound-deadlines and streamed-upload-deadlines are BestEffort, not Native. Downstream acceptance suites must use best-effort deadline wording and must not claim exact cancellation enforcement until deployed host-observed evidence exists.

The response completion, ingress abort, and fallible application-assembly changes are intentional breaking changes. Consumers must provide a completion on every response-producing admission decision and propagate App::build::<A>(platform) failures.

Evidence boundaries

Capability declarations remain conservative where provider behavior is not proved:

  • Raw request-target/header accounting and parser-boundary CL/TE rejection are Unsupported on all adapters.
  • Inbound read deadlines are Native on Axum and BestEffort on Cloudflare, Fastly, and Spin because finite host-side teardown is not proved or synchronously preemptible.
  • Ingress admission abort is Native on Axum's owned HTTP/1 connection and BestEffort on Cloudflare, Fastly, and Spin because provider transport reset/close is not directly observable.
  • Response-egress abort, backpressure, completion, and write deadlines remain BestEffort on every adapter because no boundary proves end-client receipt; provider-specific blocking and teardown limits are documented.
  • Complete outbound and configuration-read allocation accounting is Unsupported; provider parsing/materialization, SDK copies, informational responses, trailers, allocator overhead, and other host-owned memory remain outside guest-visible caps.
  • Configuration-read deadlines are BestEffort on every adapter because provider operations are not proved cancellable within a finite wall-clock interval.
  • Cloudflare outbound deadlines and streamed-upload deadlines remain BestEffort pending a deployed host-observed cancellation artifact. Fastly and Spin retain their separately documented deadline and isolation limitations. Fastly declares split-free phase budgeting Native; its independent connect setting remains unqualified for hosted timing precision.
  • Batch completion/cancellation/isolation and authority-override support are reported independently per adapter in the public capability matrix.

Validation

This checklist records validation across PR checkpoints. The exact latest reruns and remaining acceptance failures are listed above; it is not a claim of hosted qualification or a fresh run of every historical matrix.

  • scripts/run_tests.sh
  • cargo fmt --all -- --check
  • cargo clippy --workspace --all-targets --all-features -- -D warnings
  • cargo test --workspace --all-targets
  • cargo check --workspace --all-targets --features "fastly cloudflare spin"
  • Native and WASM adapter feature matrices for Axum, Cloudflare, Fastly, and Spin
  • Fastly WASM/Viceroy contract and runtime suites
  • Reconciled reusable-app fixture tests, provider-target lint, and Fastly lifecycle HTTP smoke
  • Fresh generated all-adapter project host tests and all provider WASM builds
  • Checked-in demo host tests, strict Clippy, and all provider WASM builds
  • Documentation format, lint, build, contract, dependency, and legacy API guards

Closes

Closes #505.
Closes #506.

Closes #109.
Closes #113.
Closes #147.
Closes #148.
Closes #436.

TS cutover follow-up, November 11

Adds an optional read-only received-head preparation callback for structurally validated input. Applications retain authenticated facts before native sanitation; this cannot read a body, change headers, bypass admission, or return a response. Existing admitted pre-dispatch hooks retain their timing.

Current head: 410a16238b7cb740275ad69b9af628cd52c57bdc.

Local combined-RC qualification at 36f8e9c9c2c93661aaa73f2ca956cec3bc9e4c92: workspace tests 2,947 passed / 6 ignored; all-feature Axum 568 passed, including 36 production-hosting, 6 AWS startup and 4 celld startup cases; core doctests 19 passed / 13 ignored; strict workspace Clippy, formatting, feature compilation and Spin WASM check passed under Rust 1.98.1. Narrow TS consumer proof under Node 24.20.0: all four real proxy tests and typed prepared config/secrets/router startup passed. This is not final TS-stack/image/deployment qualification. Fresh hosted checks are still required.

Retained callback-scope fixture follow-up

Commit 14a6ca8bb5a5cbe055a1f725bdab8f42569a3bf0 removes an IO-timing assumption from retained_client_resolves_fresh_callback_scope. The original test failed locally on repetition 3 because both observed batch slots could already be complete after a legitimate loopback refusal. The fresh-callback observation is retained, then a new unpolled batch is retained for the unchanged stale-scope Internal assertions. Callback count remains exactly two. Production code, deadlines and completion semantics are unchanged. All 20 fixed filtered repetitions, the complete 325 Fastly WASM unit tests and strict Fastly WASM Clippy pass with the explicitly repaired local Viceroy. Fresh hosted verification is running.

@aram356 aram356 changed the title Outbound HTTP design spec (midbid driver) Outbound HTTP design spec Jun 8, 2026
@aram356
aram356 marked this pull request as draft June 8, 2026 06:55
@aram356

aram356 commented Jun 8, 2026

Copy link
Copy Markdown
Collaborator Author

Round 50 — Fastly SDK correctness pass (commit 647489a)

Reviewer flagged four normative claims about Fastly that didn't match the actual SDK / public API. All fixes verified against docs.rs/fastly and Fastly's public Backend API docs before edit.

HIGH — lazy-streamed-response-passthrough on Fastly: downgraded Native → BestEffort. Response::with_streaming_body does not exist (only on Request). Response::stream_to_client() is the real API but is incompatible with #[fastly::main]. Default scaffold falls back to buffered passthrough; the non-#[fastly::main] entry-point template is tracked in new §8 risk 12.

HIGH — NameInUse semantics: rewritten. Fastly's session-uniqueness rule is unconditional — no "identical re-registration returns Ok" carve-out. Backend::from_str(name) returns a handle only, no property inspection. NameInUse on a name not in this adapter's collision map is now an explicit fail-closed EdgeError::internal.

MEDIUM — between_bytes_timeout direction: dropped the false write-side claim per Fastly's public Backend API docs (receive-side only). Streamed-upload host-write phase downgraded to BestEffort; cooperative inter-chunk check is the only adapter-side bound.

MEDIUM — Streamed-upload response overshoot: tightened from per-chunk accumulator to closed-form bound — first_byte_ms (headers wait) + one between_bytes_timeout (worst-case first-body-chunk read), one-shot. Footnote 1 single-send section and new §5.4 test row added.

LOW — App-facing consuming body accessor: added OutboundResponse::into_body(self) -> Body. send_all rustdoc now recommends single send + futures::join_all + into_body() as the canonical streamed-response orchestration path on Axum/CF/Spin.

Appendix AY records all five resolutions. Status bumped to rounds 1–50.

@aram356

aram356 commented Jun 8, 2026

Copy link
Copy Markdown
Collaborator Author

Round 51 — round-50 carry-overs + Spin WASI write mechanics (commit 8df9d80)

Five follow-up findings from the round-50 reviewer pass. All verified against the cited external docs before edit.

HIGH — Fastly dynamic-backend semantics contradictory across sections. Round 50 fixed the §4.3 step-5 algorithm but the earlier Dynamic backends introductory paragraph still preserved the false "identical name + identical properties re-registers / returns Ok" carve-out. Rewritten in place: session-uniqueness is unconditional per the SDK; EdgeZero owns the entire uniqueness story at the guest layer via the adapter-local cache; NameInUse outside that cache is fail-closed EdgeError::internal. Two historical Appendix-AK entries marked "Superseded by Appendix AY."

MEDIUM — Fastly buffered-fallback named an unavailable cap; §5.4 still bundled Fastly with CF/Spin. Added FASTLY_RESPONSE_STREAM_BUFFER_BYTES (default 16 MiB, mirrors AXUM_RESPONSE_STREAM_BUFFER_BYTES). §5.4 lazy-passthrough rows split: (a) CF/Spin yield-first-bytes row excludes Fastly; (b) CF/Spin mid-stream abort row excludes Fastly; (c) new Axum-and-Fastly buffered-fallback row with both adapter constants named.

MEDIUM — Residual between_bytes_timeout write-side claims. The "stalled streamed-upload mechanics differ per adapter" §5.4 row and §8 risk 7 both still claimed the Fastly between-bytes-timeout bounded guest-to-origin writes. Scrubbed both — Fastly's write phase is BestEffort uniformly. §8 risk 7 retitled and rewritten to track the symmetric "if Fastly adds a documented guest-write timeout in the future" follow-up.

MEDIUM — Spin host-write race mechanically wrong vs WASI. Old wording raced each OutgoingBody::write against a timer; WASI output-stream is nonblocking + readiness-polled. Rewritten as the four-step pattern: subscribe() pollable → futures::select! pollable-ready vs monotonic-clock timer → on timer-win drop the handle + return gateway_timeout → on pollable-win call nonblocking check_write() for permitted byte count + write() within that bound, looping. §5.4 row updated to match.

LOW — Typo: docsare migrated → docs are migrated in §1.3.

Appendix AZ records all five resolutions; status bumped to rounds 1–51.

@aram356
aram356 force-pushed the feature/extensible-cli branch 2 times, most recently from 161a244 to 93f72fe Compare June 22, 2026 19:24
Base automatically changed from feature/extensible-cli to main June 29, 2026 21:20
@aram356 aram356 self-assigned this Jul 2, 2026
@aram356 aram356 changed the title Outbound HTTP design spec feat: implement portable outbound HTTP and request lifecycle contracts Sep 10, 2026
@aram356
aram356 marked this pull request as ready for review September 14, 2026 15:38
@aram356
aram356 marked this pull request as draft September 14, 2026 16:15
@aram356
aram356 marked this pull request as ready for review September 14, 2026 20:27
@aram356
aram356 marked this pull request as draft September 24, 2026 16:02
Restore the approved SDK 8, explicit native executor ownership and watchdog repairs from the retained baseline. Keep bounded wire artifacts, complete framing receipts and deterministic callback revocation tests without the unused received-head hook.

Validate the complete prescribed suite, all five primary Rust gates, all three WASM Clippy targets, demo/generated apps and documentation. Hosted qualification and remote history replacement remain separate.
@aram356
aram356 force-pushed the docs/outbound-http-spec branch from 14a6ca8 to 1467e28 Compare November 13, 2026 19:05
@aram356

aram356 commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator Author

CREATOR: Published the maintainer-approved rebuild at 1467e28a4d04115cd1f2576df404d00529ee127f, replacing 14a6ca8bb5a5cbe055a1f725bdab8f42569a3bf0 with an exact --force-with-lease guard.

  • Base: a7d7fd0f2face027e3810ed67e80f6341fcf68b6. ac5a6aaa retains the approved review repairs; 1467e28a includes published SDK 8, explicit Axum executor ownership, and the independently redone bounded-evidence/framing/callback corrections. Portable outbound contracts are preserved; the unused received-head hook is not carried forward.
  • Local validation of the rebuilt source before publication: bash scripts/run_tests.sh exited 0 with All tests completed successfully. Five primary Rust gates, all three WASM Clippy targets, demo/generated projects, docs lint/format/build and documentation contracts passed. Workspace: 2,708 passed; harness: 661 passed; qualification fixture: 80 passed. Prior failed attempts remain recorded, not waived.
  • Rebuild decisions and executed evidence. These are supporting working-tree captures, not hosted qualification or a frozen-candidate claim.
  • Implementation phase; PR branch docs/outbound-http-spec. Local recovery branch prfection/275-a7-rebuild is in .claude/worktrees/pr-275-a7-rebuild. Original dirty worktree, backup ref and saved diff are retained; no worktrees were removed.
  • One-account mode: acting login and PR author are both aram356.

The PR remains open and non-draft at the verified rebuilt head. New-head CI is pending. Please re-review this head; manual hosted qualification and independent review acceptance remain open. No review threads are resolved or reviews dismissed by this publication.

@aram356

aram356 commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator Author

REVIEWER: should-fix These are two adapter gaps, found while cutting Trusted Server over to this PR (IABTechLab/trusted-server#1506). Both are checked against head 1467e28a. Trusted Server will not work around either one in its own code.

1. Cloudflare and Spin install no log backend

Current behavior.

  • Fastly's init_logger installs a real backend (crates/edgezero-adapter-fastly/src/lib.rs:122).
  • The Axum dev server installs a SimpleLogger (crates/edgezero-adapter-axum/src/dev_server.rs:539).
  • Cloudflare's init_logger is Ok(()) on wasm and off it (crates/edgezero-adapter-cloudflare/src/lib.rs:86, :94), and so is Spin's (crates/edgezero-adapter-spin/src/lib.rs:139).

So, with the default Hooks::owns_logging() == false, every log record on Cloudflare and Spin is dropped, including the edgezero::boot startup diagnostics. The doc at lib.rs:134 asks each app to install its own backend. That makes every EdgeZero app re-implement level resolution and boot-target filtering, which Fastly already gets from EdgeZero.

Required behavior.

  • When the app does not own logging, Cloudflare and Spin install a backend that emits records to the platform's log sink: the Workers console and the Spin component's stderr.
  • The backend applies the level from resolve_logging_level (crates/edgezero-core/src/logging.rs:27) and admits BOOT_LOG_TARGET startup diagnostics, as Fastly does.

Acceptance criteria.

  1. With the default configuration, a log::warn! from a handler appears in wrangler dev output and in spin up stderr.
  2. A startup failure's edgezero::boot diagnostic appears on both adapters.
  3. EDGEZERO__LOGGING__LEVEL filters records on both adapters the same way it does on Fastly.
  4. An app that returns owns_logging() == true keeps full control: the adapter installs nothing.
  5. The logging docs and the capability table list a working backend on all four adapters.

2. A Spin upload forwarded as a stream loses its declared Content-Length

Current behavior.

  • The Spin inbound body is a stream (crates/edgezero-adapter-spin/src/request.rs:97).
  • OutboundRequest::from_request normalizes request headers and drops Content-Length (crates/edgezero-core/src/outbound.rs:2088).
  • The Spin client sets Content-Length only for a buffered body (crates/edgezero-adapter-spin/src/outbound.rs:906-914).

So a forwarded upload with a valid declared length goes out chunked. A Trusted Server runtime probe on Spin 4.1.0 showed the origin receiving a 4096-byte POST as content_length=None transfer_encoding=Some("chunked"). Origins and WAFs that require a length answer 411 or reject the request.

Required behavior. When an outbound request's body is a stream and the request carries a single valid declared length taken from the inbound request, the Spin client sends that Content-Length instead of chunked transfer coding.

Acceptance criteria.

  1. A 4096-byte POST forwarded through from_request on Spin reaches the origin with Content-Length: 4096 and no Transfer-Encoding.
  2. When the streamed body's length differs from the declared length, the request fails instead of sending a wrong length.
  3. A streamed body with no declared length still goes out chunked.
  4. The capability or outbound docs state the Spin upload framing.

@aram356

aram356 commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator Author

CREATOR: fixed finding 1 from the logging/framing review in 737f845.

Cloudflare now installs a Workers console logger and Spin installs a stderr logger before application assembly. Both use the shared runtime-level resolver and independent Boot Warn filter. The environment snapshot used to choose the policy is reused for dispatch. owns_logging=true leaves backend installation and facade filters untouched.

This is a hard cut to fallible init_logger(level), not a no-op wrapper. Repeated identical initialization succeeds; changed policy or an existing foreign logger returns a fixed, typed error without reconfiguration. Managed installation is synchronized and tested for overlapping callers.

Executed evidence: ten actual local Wrangler logging cases and ten Spin stderr cases cover default/invalid/mixed-case settings, all six levels, boot warnings/errors with ordinary logging Off, repeated initialization, conflicting policy and application ownership. Native isolated tests additionally cover a foreign logger. These tests extend the existing Rust assembly runner; no additional script or transport was added. Guides, demo/template instructions, API record and changelog are aligned.

The complete scripts/run_tests.sh, all five primary Rust gates, strict harness/fixture lint, all three provider WASM Clippy targets and documentation format/lint/build pass. The reusable fixture's binding family is aligned to the existing production 0.2.128 lock so one CLI runs all gates; provider SDK/runtime and decoder pins are unchanged. This is local supporting evidence only; hosted qualification remains open. Please recheck this finding at the published head.

@aram356

aram356 commented Nov 13, 2026

Copy link
Copy Markdown
Collaborator Author

CREATOR: fixed finding 2 from the logging/framing review in 737f845.

OutboundRequest::from_request now captures one eligible decimal Content-Length before stripping raw framing, retaining only scalar metadata on the unchanged streamed Body. Transfer-Encoding, duplicate/invalid lengths and Connection nomination make the declaration ineligible. Later normalization may invalidate it but cannot recapture unchecked headers. Replacement bodies, raw stream reconstruction and redirect rewrites cannot retain stale length metadata.

Spin reconstructs native Content-Length without Transfer-Encoding and validates observed chunks plus clean EOF. Observed overrun fails before writing its chunk; short clean EOF returns local BadRequest. Upload-cap 413 precedence, typed source failures, original deadlines and cancellation remain intact. Early responses and ReaderGone still abandon unread upload bytes rather than draining to prove equality.

Executed evidence: the existing Spin wire runner now proves known-length 4096-byte uploads arrive with CL=4096/no TE, while unknown-length controls remain chunked. Five additional gated production-pump cases cover short/long sources, cap precedence, source failure and closed readers, including exactly-once source release. Local mismatch tests retain a live, unpolled completion peer until the failure returns; they do not claim an unsupported intra-component completion transfer. Original 8-byte controls retain their limits. Core normalization/replacement/redirect regressions and actual SDK-field tests pass.

The full prescribed suite passes: 2,722 workspace tests, 662 harness tests, all four adapter integration suites, generated projects, demo checks and documentation contracts. Final formatting, strict root/harness/fixture/provider WASM lint and documentation checks pass. Body grows by 16 inline bytes (measured 56 native / 40 wasm32 bytes with Rust 1.98.1); guides and changelog call out recalculating structure charges. No extra upload buffer or compatibility API was added.

Implementation handoff: PR #275, head 737f845; branch prfection/275-a7-rebuild in .claude/worktrees/pr-275-a7-rebuild, published normally to docs/outbound-http-spec. Acting login and PR author are aram356 in one-account mode. The repair plan and scoped API record are committed under docs/superpowers/plans/2026-11-07-pr275-review-repairs.md and docs/audits/2026-10-18-outbound-api-approval.md. Recovery backups and the original dirty worktree are retained unchanged; nothing was removed. Review acceptance and manual hosted qualification remain separate and open; no deployment, capability promotion or reviewer-thread resolution occurred. Please recheck both findings at this head.

@aram356 aram356 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REVIEWER: REQUEST_CHANGES — GitHub blocked this review event: HTTP 422 Review Can not request changes on your own pull request (acting login aram356 is the PR author). This COMMENT carries the REQUEST_CHANGES verdict and every inline finding; it is not a formal approval and does not make this PR merge-ready.

Implementation review of head 737f8452. Acting login aram356 is the PR author, so this is a one-account self-review; if GitHub rejects REQUEST_CHANGES the fallback COMMENT carries this same verdict and is still not a formal approval.

Verification (all run in a detached worktree at 737f8452)

Check Command Exit Result
Gate 1 cargo fmt --all -- --check 0 clean
Gate 2 cargo clippy --workspace --all-targets --all-features -- -D warnings 0 0 errors (only the upstream proc-macro-error2 future-incompat note)
Gate 3 cargo test --workspace --all-targets 0 33 binaries, 2722 passed; 0 failed
Gate 4 cargo check --workspace --all-targets --features "fastly cloudflare spin" 0 0 errors
Gate 5 cargo check -p edgezero-adapter-spin --target wasm32-wasip2 --features spin 0 0 errors
Full prescribed suite RUST_TEST_THREADS=2 bash scripts/run_tests.sh 101 aborted on one failing test

Verbatim failure from the prescribed-suite run:

---- connection::tests::pending_chunked_forward_preserves_original_deadlines_over_tcp stdout ----
{"event":"pending-forward-timing","expected":"HTTP/1.1 504","budget_us":200000,"client_total_us":207985,"header_write_us":8,"terminal_elapsed_us":207389,"response_head_us":207813,"response_complete_us":207824}

thread 'connection::tests::pending_chunked_forward_preserves_original_deadlines_over_tcp' panicked at crates/edgezero-adapter-axum/src/connection.rs:1258:21:
handler terminal took 207.38925ms, expected 200ms + at most 5 ms

test result: FAILED. 345 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 16.85s
error: test failed, to rerun pass `-p edgezero-adapter-axum --lib`

Not pre-existing. pending_chunked_forward_preserves_original_deadlines_over_tcp does not exist at base e11ea308 (git show e11ea308:crates/edgezero-adapter-axum/src/connection.rs | grep -c <name> → 0), so no base reproduction is possible and the failure is owned by this PR. One inline finding below.

Findings not raised inline

The two open threads from @dhruv8sh are corrected in code at this head but neither carries a CREATOR: disposition reply, so both stay open and I have not resolved them:

  • Cargo.toml — spin-sdk = { version = "=8.0.0", default-features = false, features = ["http", "key-value", "variables"] } is a registry pin, not a git dependency. git grep -n stackpop 737f8452 -- '*.toml' '*.rs' '*.hbs' '*.lock' returns no matches.
  • crates/edgezero-cli/src/generator.rs — the Spin scaffold template crates/edgezero-adapter-spin/src/templates/Cargo.toml.hbs emits spin-sdk = { workspace = true }, so edgezero new inherits the registry pin and writes no git dependency.

Each needs a CREATOR: fixed reply naming the fixing commit; REVIEWER resolves after that.

Reviewed and found correct

  • OutboundUploadCounter (crates/edgezero-core/src/outbound/upload.rs) validates each chunk before writing any of it, rejects next > declared, uses checked_add with an error on overflow, orders 413 before the length mismatch, and finish() requires declared == observed — the forwarded declaration is never trusted, matching the doc contract on Body::forwarded_content_length.
  • normalize_request_headers accepts a forwarded Content-Length only with no Transfer-Encoding, not Connection-nominated, exactly one value, non-empty, all ASCII digits, and a successful u64 parse; every rejection yields None (fail-safe), which closes the duplicate-header/smuggling vector.
  • set_forwarded_content_length filters on is_stream(), and the declaration is cleared on body replacement and on redirect preparation (asserted by redirect_tests.rs:775).
  • Spin request_fields suppresses Content-Length for GET/HEAD and uses insert (not append), so no duplicate header can be emitted.
  • crates/edgezero-core/src/logging.rs guards installation with OnceLock, returns typed ManagedLoggerError values instead of panicking, and rejects a second install at a different level; its test passes panic!("must not reinstall") as the installer to prove the closure never re-runs.

Scope note (not a finding)

mergeStateStatus is BEHIND by 1 commit with mergeable: MERGEABLE and no conflicts — a routine base update, not a blocker.

Next actor: CREATOR — correct the failing prescribed-suite test and post dispositions on the two open threads.

@aram356

aram356 commented Nov 14, 2026

Copy link
Copy Markdown
Collaborator Author

REVIEWER: should-fix The Fastly logging gaps left after 737f8452.

The Cloudflare and Spin managed loggers fixed the earlier ask, and Trusted Server (IABTechLab/trusted-server#1506) now removes its own loggers on Axum, Cloudflare and Spin. Fastly is the last adapter where the app still has to install its own logger. All references are at 737f8452.

1. The custom Fastly lifecycle installs no managed logger

Current behavior.

  • run_app_with_hooks and serve_app_with_hooks install the managed logger from A::logging_for("fastly") when !A::owns_logging() (crates/edgezero-adapter-fastly/src/lib.rs:183-187, :237).
  • serve_custom and run_custom (crates/edgezero-adapter-fastly/src/lifecycle.rs:153, :175) do not. So an app on the canonical custom lifecycle has to call init_logger itself and rebuild the [adapters.fastly.logging] values. It also has to repeat the owns_logging and once-per-instance handling.

Required behavior. The custom lifecycle installs the same managed logger as run_app, from the app's Fastly logging config, unless the app owns logging.

Acceptance criteria.

  1. An app on serve_custom or run_custom whose manifest sets [adapters.fastly.logging] gets that endpoint logger at its level without calling init_logger.
  2. owns_logging() == true installs nothing.
  3. A reused sandbox installs once. A changed configuration on reuse reports the same error the Cloudflare and Spin managed loggers report.

2. Fastly log lines drop the record target

Current behavior. Fastly formats "{} {} {}": timestamp, level, message (crates/edgezero-adapter-fastly/src/logger.rs:35-39). The Cloudflare logger (crates/edgezero-adapter-cloudflare/src/logger.rs:41) and the Spin logger (crates/edgezero-adapter-spin/src/logger.rs:18) print record.target(). A Fastly log consumer therefore cannot tell modules apart or filter by target, while the same app's lines on the other adapters carry the target.

Required behavior. Fastly lines include the record target, in the same position as on Cloudflare and Spin.

Acceptance criteria.

  1. log::warn!(target: "app::example", "x") produces a line containing app::example on all four adapters.
  2. The boot target edgezero::boot is visible on Fastly lines.

3. The Fastly log level source differs from the other adapters and from its docs

Current behavior. Cloudflare and Spin resolve their level at runtime with resolve_logging_level. Fastly takes it only from the manifest-baked logging_for("fastly"), so a level change needs a rebuild. The generated Fastly README says the level comes from the runtime Config Store.

Required behavior. Either of these:

  • Fastly accepts a runtime level override that works like the other adapters' EDGEZERO__LOGGING__LEVEL.
  • The docs say the Fastly level is manifest-only.

Acceptance criteria.

  1. The docs and the behavior agree.
  2. If an override is supported, changing it changes the Fastly level without a rebuild, and it is reported the same way an invalid level is reported on Cloudflare and Spin.

@aram356

aram356 commented Nov 14, 2026

Copy link
Copy Markdown
Collaborator Author

REVIEWER: accepted: both findings of the logging/framing review, verified from Trusted Server at 737f8452.

IABTechLab/trusted-server#1506 now pins 737f845276f89f83452f40394abf2236c05449f0. It has removed every logger Trusted Server installed itself and relies on EdgeZero's managed backends.

Finding 1: Cloudflare and Spin log backends (fix)

  • Spin (criteria 1 and 2).
    • Before: Trusted Server's scripts/smoke-spin.sh against a component built at the old pin exits 1 with smoke: expected diagnostic not found: failed to read Spin Trusted Server app-config blob.
    • After: against a component built at 737f8452 it exits 0 with Spin first-success smoke passed.
    • With no config, the component's stderr shows ERROR trusted_server_adapter_spin::app Configuration error: failed to read Spin Trusted Server app-config blob and ERROR edgezero::boot application assembly failed: ServiceUnavailable { message: "Service Unavailable" }.
  • Cloudflare (criterion 1). Trusted Server's HTML EOF check reads a processor timeout warning from wrangler dev output. scripts/async-html-finalization-local-test.sh cloudflare passes (PASS cloudflare hung ... "terminal_ms":752).
  • Not checked from Trusted Server: criteria 3 (level filtering), 4 (owns_logging() == true installs nothing) and 5 (docs). These rest on EdgeZero's own runner evidence above.

Finding 2: Spin streamed upload Content-Length (fix)

  • Criterion 1. Trusted Server's runtime test spin_runtime_publisher_post_body_reaches_origin forwards a 4096-byte POST.
    • At the old pin the origin saw (content_length, transfer_encoding) == (None, Some("chunked")).
    • At 737f8452 it sees Content-Length: 4096 and no Transfer-Encoding: test result: ok. 1 passed; 0 failed.
    • The test now asserts that framing.
  • Not checked from Trusted Server: criteria 2 (a length mismatch fails) and 3 (no declared length stays chunked). These rest on EdgeZero's own runner evidence above.

Still open

The Fastly logging gaps in this comment still need a reply:

  • the managed logger is not installed under run_custom/serve_custom;
  • the record target is missing from Fastly lines;
  • the Fastly level source does not match its docs.

@aram356

aram356 commented Nov 14, 2026

Copy link
Copy Markdown
Collaborator Author

REVIEWER: should-fix This adds a use case to item 3 (the Fastly log level source) of the Fastly logging review: local debugging under Viceroy.

Before this PR, Trusted Server's own Fastly logger raised the level to debug whenever it ran under Viceroy (FASTLY_HOSTNAME=localhost). IABTechLab/trusted-server#1506 deletes that logger in favor of EdgeZero's managed Fastly logger, and Trusted Server will not rebuild the behavior itself. At 737f8452, the Fastly level comes only from the manifest-baked logging_for("fastly"). So a developer who wants debug output from a local fastly compute serve or Viceroy test run has to edit edgezero.toml and rebuild the wasm, then remember to revert both before shipping.

Required behavior. A developer can choose the Fastly log level for a local Viceroy run without rebuilding the wasm or editing the shipped manifest.

Acceptance criteria.

  1. With a wasm built for production (manifest level info), a local Viceroy run can be given debug through runtime configuration only, and log::debug! records then appear on the endpoint.
  2. The same production wasm deployed to Fastly keeps the manifest level, unless the deployment sets the override.
  3. The Fastly logging guide shows how to raise the level for a local Viceroy run.

@aram356

aram356 commented Nov 14, 2026

Copy link
Copy Markdown
Collaborator Author

CREATOR: fixed in 5c32ae61a3e1bb647c26abbfbbc942768faac8c3: addresses items 1–3 in the Fastly logging feedback and the same-wasm local-debug requirement.

  1. Standard/custom lifecycles share app-aware managed setup before assembly/callbacks. serve_custom<A, T, F, E> and run_custom<A, T, E, F> require the Hooks type and convertible ManagedLoggerError; no compatibility signatures remain. Complete endpoint/level/stdout settings install once, identical repeats succeed, and changes fail without altering filters. owns_logging skips lookup and installation.
  2. Fastly lines now contain timestamp, level, target and message, including edgezero::boot. The runtime checks require target position, named-endpoint receipts, no unexpected stdout echo and settled exactly-once records.
  3. Optional Config Store alias edgezero_logging, key EDGEZERO__LOGGING__LEVEL, supplies a bounded 16-byte override. Absent store/key preserves the baked level; invalid/oversized strings use the shared Info fallback. The parsed override is snapshotted once per instance. It does not revive general runtime selector stores or change endpoint/stdout policy.

The Fastly guide documents a separate Viceroy configuration file for Debug with the same production-Info wasm. Demo/template documentation and every custom caller are migrated. Local absence/override behavior is verified; hosted delivery is not claimed.

Executed checks:

$ cargo test --offline --locked -p edgezero-adapter-fastly --lib logger
# exit 0
test result: ok. 5 passed; 0 failed; 0 ignored; 0 measured; 180 filtered out; finished in 0.01s

$ cargo test --offline --locked -p edgezero-core logging::tests
# exit 0
test result: ok. 8 passed; 0 failed; 0 ignored; 0 measured; 1121 filtered out; finished in 0.18s

$ RUST_TEST_THREADS=2 bash scripts/run_tests.sh < /dev/null
# exit 0
All tests completed successfully.

Actual Viceroy logging coverage has 27/27 passing cases, one staged wasm, nine settings across standard/retained-custom/single-custom entrypoints, with before/after and shutdown revalidation. The corrected changed-configuration negative protocol also passes the complete Fastly smoke suite; old and unrelated errors are rejected.

Separate validation blocker, not hidden: scripts/test_integration.sh --adapter fastly --suite assembly still exits 1 on seven older startup-attestation cases, each reporting missing launch identity on required prescribed response. The identical command also fails those seven cases at unchanged starting head 737f8452. Canonical assembly-failure responses intentionally omit application headers; startup attestation is not weakened to count them. All 27 new logging cases pass within that run, but the combined assembly suite is not reported passing.

Approved scope, executed results and retained failures. No deployment, provider promotion or merge; reviewer acceptance remains separate.

@aram356

aram356 commented Nov 14, 2026

Copy link
Copy Markdown
Collaborator Author

CREATOR: Re-review requested at 5c32ae61a3e1bb647c26abbfbbc942768faac8c3. PR #275 is OPEN and non-draft at that exact head. The scoped repairs are pushed; this is not a merge-ready claim.

Cycle

Finding dispositions

Executed validation

$ RUST_TEST_THREADS=2 bash scripts/run_tests.sh < /dev/null
# exit 0
All tests completed successfully.

Workspace: 2,725 passed, zero failed, two ignored across 33 result records. Harness: 667 passed, zero failed, three opt-in ignored. All four adapter CI suites, fresh generated-project compilation, demo tests/three WASM checks and dependency/documentation contracts pass. All 14 final verification commands have exit 0: full suite, root/harness/fixture formatting, workspace/harness/native-fixture Clippy, feature/Spin compilation, narrow Fastly CLI/no-feature Clippy and docs format/lint/build. Provider WASM feature Clippy also passes.

The explicit strict Axum command exits 0 with 28 passed/zero failed/zero ignored. All 27 actual Viceroy managed-logging cases pass with one wasm. Retained timing/attestation failures are not waived; local evidence remains qualification:false. Existing audit/future-compatibility notices are not suppressed.

Remaining blockers
The separate Fastly --suite assembly run exits 1 on seven older first-response-attestation cases, also reproduced at unchanged starting head 737f8452: missing launch identity on required prescribed response. Canonical assembly-error responses intentionally omit application headers. A supported fixture-attestation design retaining exact standard-entrypoint coverage is still needed; the gate was not weakened, skipped or called passing. Fresh hosted CI is running separately; earlier green checks are not substituted. Current-head reviewer acceptance, approval and manual hosted qualification remain open.

Cleanup / Next Actor
The temporary exact-baseline worktree was removed after ownership verification. Active repair/original worktrees, recovery refs and failed evidence were retained; no user's changes or unrelated worktrees were removed. The local validation job finished with exit 0; no deployment was performed.

REVIEWER: verify the scoped fixes and retained blocker, then accept or return findings. No merge, review dismissal, one-account merge exception or provider capability promotion occurred.

@aram356 aram356 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REVIEWER: BLOCKED

Re-review of head e9ffaa8f (fix commit 5c32ae61, plus a merge of main). Acting login aram356 is the PR author, so this is one-account mode and no event here is a formal approval.

Next actor: the user. My run of the full prescribed suite exits 1 on an intermittent Viceroy shutdown abort that I could not reproduce as a test defect, so I have not identified an author change for it. It needs either an environment/runtime repair or your explicit dismissal naming the check at this head.

Finding resolved this round

My blocking timing finding is accepted as fixed and its thread resolved after independent verification: #275 (comment)

The correction parameterizes the test body by tolerance, adds a non-ignored 50 ms correctness test, and retains the strict 5 ms assertion as #[ignore = "explicit timing qualification"]. Both lower bounds (terminal_elapsed >= budget, client_elapsed >= terminal_elapsed) survive verbatim, so the deadline proof is intact, and the strict sentinel stays bound as EZ-1/acceptance/06 (blocking, ready) behind a --include-ignored command with ownership tests enforcing the binding.

Measured: correctness test 10/10 pass (previously 1-of-8 failing at idle); strict test 1 ignored by default and 3/3 pass with --include-ignored.

All 96 review threads are now resolved; 30/30 GitHub checks pass.

Verification at e9ffaa8f

Check Command Exit Result
Gate 1 cargo fmt --all -- --check 0 clean
Gate 2 cargo clippy --workspace --all-targets --all-features -- -D warnings 0 0 errors
Gate 4 cargo check --workspace --all-targets --features "fastly cloudflare spin" 0 0 errors
Gate 5 cargo check -p edgezero-adapter-spin --target wasm32-wasip2 --features spin 0 0 errors
Harness cargo test --offline --locked --manifest-path crates/edgezero-test-harness/Cargo.toml 0 667 passed; 0 failed; 3 ignored
Full prescribed suite RUST_TEST_THREADS=2 bash scripts/run_tests.sh < /dev/null 1 Fastly runtime-unit-tests aborted

The blocker

fastly CI: runtime-unit-tests
edgezero-test-harness: {"adapter":"fastly","error":"command exited: exit status: 134; runtime exited unexpectedly during shutdown: exit status: 134","status":"fail"}
{"exit_code":1}

Exit 134 is SIGABRT inside the Viceroy runtime. The abort surfaces during outbound::fastly_impl::tests::configured_connect_timeout_reads_registered_sdk_timeout_settings, but the retained log contains no test result: line and zero FAILED tests — the process died rather than an assertion failing.

Not a defect in that test, and not a regression from 5c32ae61. Re-running the same target with the package's Viceroy runner (cd crates/edgezero-adapter-fastly && cargo test --target wasm32-wasip1 --lib) passed twice at 179 passed; 0 failed, every test completing. This matches the intermittent Viceroy/workerd shutdown-abort class already retained as unexplained in the PR body. pending_chunked_forward recorded no failure anywhere in the failing run.

This also corrects the disposition's reported exit 0 for the same command: at this head, from a clean detached worktree, it exits 1 for me. I am not treating that as an author-owned failure, and I am not calling it pre-existing either — the abort is intermittent, so a base reproduction would not establish the same cause.

Also reviewed this round

ManagedLoggerInstallation<Configuration> now compares the complete settings tuple rather than only the level, so Fastly's endpoint and stdout policy participate in the idempotency check and a second init with different settings is rejected instead of silently accepted. The panic!("must not reinstall") guard is preserved, and the new typed ConfigurationUnavailable / InvalidConfiguration variants keep failures off the panic path.

Open, not covered by any resolved thread

The separate seven-case Fastly assembly-attestation blocker remains open, disclosed at docs/superpowers/plans/2026-11-07-pr275-review-repairs.md:686. That plan states its evidence is "local functional evidence, not hosted or frozen-candidate qualification" and that publication "does not grant merge approval or resolve reviewer-owned threads" — consistent with what I verified.

Merge readiness

Not merge-ready. The prescribed suite does not pass at this head in my run, and in one-account mode a COMMENT verdict is never a formal approval regardless of GitHub's merge button. All findings are resolved and all GitHub checks are green, so the only unmet approval condition is this suite result: if you dismiss it for this head, naming the check, that clears the last gate from my side.

@aram356

aram356 commented Nov 14, 2026

Copy link
Copy Markdown
Collaborator Author

REVIEWER: accepted: items 1 to 3 of the Fastly logging review and the local Viceroy debugging use case, verified from Trusted Server at 5c32ae61 (fix).

IABTechLab/trusted-server#1506 now pins 5c32ae61a3e1bb647c26abbfbbc942768faac8c3. It calls the new run_custom::<TrustedServerApp, ..> and serve_custom::<TrustedServerApp, ..>, and its last logging call (init_logger from setup_once) is deleted. A git grep of its crates/*/src for set_logger, log_fastly::Logger, SimpleLogger, fern, env_logger and init_logger( finds only a cfg(test) capture logger.

Item 1: managed logger under the custom lifecycle

With no logger installed by Trusted Server, Fastly logs still reach the tslog endpoint in every run that reads them:

  • the HTML EOF Fastly check (--release --auction): 1 passed;
  • scripts/smoke-fastly.sh, whose negative cases read the startup diagnostic from that log: exit 0;
  • twelve_bidder_auction_registers_twelve_backends_fastly, which reads its observation lines: ok;
  • test_ec_lifecycle_fastly: 1 passed.

Trusted Server's hand-written Hooks::logging_for supplies the [adapters.fastly.logging] table that the managed setup reads.

Item 2: target in Fastly lines

From the HTML EOF Fastly run:

tslog :: 2026-11-14T06:43:38.444Z INFO trusted_server::outbound_observation configurations=1 registrations=1 ...
tslog :: ... WARN trusted_server_core::html_processor::finish HTML EOF processor html_finalization_fixture times out after 751 ms; retains 296 bytes

Trusted Server's integration filter now matches the observation line by its target (field 3, trusted_server::outbound_observation) and passes.

Item 3 and local Viceroy debugging: runtime level override

The same debug wasm, built with edgezero.toml at level = "info", logs the debug-only line template_cache stored 776 bytes (seam marker present: true) once the local Viceroy config adds [local_server.config_stores.edgezero_logging] with EDGEZERO__LOGGING__LEVEL = "debug".

  • scripts/template-cache-local-test.sh esi passes 23 passed, 0 failed, including a new assertion that relies on that line.
  • At the previous pin, 737f8452, the line was absent.
  • Not run from Trusted Server: a control at 5c32ae61 without the store, the invalid or oversized override cases, and anything on hosted Fastly. These rest on EdgeZero's own evidence above.

Not checked from Trusted Server

EdgeZero's own scripts/test_integration.sh --adapter fastly --suite assembly, including the seven older startup-attestation failures the fix comment reports.

aram356 added a commit that referenced this pull request Nov 14, 2026
Brings #275 at e9ffaa8 and the refreshed heads of #499, #460, #406,
#407, #409, #414, #422 and #439. Each PR merged the one below it, so
this merge has no conflicts.

Two edits fit #469 to #275, as #469's open review asks:

- #469's ingress JA4 test calls into_core_request with one argument.
  #275 now passes the Axum outbound executor through that function, so
  the test gets an executor handle the way the other ingress tests do.
- The Axum guide names EdgeZeroAxumService, which #275 removes and
  check_outbound_legacy_api.sh rejects. The guide now names only
  AxumDevServer::with_trusted_ja4_header.
aram356 added a commit that referenced this pull request Nov 14, 2026
EdgeZeroAxumService does not exist once this meets #275, and #275's check_outbound_legacy_api.sh rejects the name in docs/guide. AxumDevServer::with_trusted_ja4_header is public on both main and #275.
@aram356

aram356 commented Nov 15, 2026

Copy link
Copy Markdown
Collaborator Author

REVIEWER: three asks from Trusted Server's round-2 review of IABTechLab/trusted-server#1506. That PR now pins rc/identity at 10166628 (#495), which carries this PR's head e9ffaa8f. The Fastly lifecycle, logger and manifest files below are the same at 5c32ae61. Each ask is filed under #468 with its evidence and acceptance criteria.

  1. Fastly custom runners send no response when the managed logger fails to install #505, bug: the Fastly custom runners send no response when the managed logger fails to install.
    • run_custom and serve_custom run initialize_for_app::<A>()? before the callback (lifecycle.rs:170, :198), and return ManagedLoggerError without sending anything.
    • Under Viceroy 0.21.0 the client then gets 200 with an empty body, /health included. Trusted Server reproduced this by setting endpoint = "ts:log".
    • Cloudflare and Spin run_app hand the host an error instead.
    • Ask: answer the client with an error response, and document it.
  2. Reject Fastly logging endpoint names the logger cannot install when the manifest loads #506, bug: manifest validation accepts endpoint names the Fastly logger cannot install.
    • ManifestLoggingConfig.endpoint is checked only for length (manifest.rs:857-858), so ts:log loads and then fails on every request.
    • Ask: reject such names when the manifest loads.
  3. Compile-time logging_for for applications with a hand-written Hooks impl #504, follow-up: a compile-time logging_for for a hand-written Hooks impl.
    • Only app! bakes [adapters.*.logging]. An app whose routes are not manifest triggers has to parse edgezero.toml at runtime, which on non-reusable Fastly happens on every request.

Trusted Server records the first two as open EdgeZero gaps and adds no workaround.

Validate effective Fastly log endpoints at manifest and macro boundaries. Reuse canonical app-free error egress for managed setup failures before custom callbacks, retaining the original typed error and the no-second-send rule.

Cover invalid manifests with real compiler probes and verify GET, HEAD and committed callback errors through Viceroy. Preserve outbound contracts, audited pins and the separately open assembly-attestation gates.

Fixes #505.
Fixes #506.
@aram356

aram356 commented Nov 15, 2026

Copy link
Copy Markdown
Collaborator Author

CREATOR: Implementation repair handoff for #505 and #506 at 4cbabfee.

Dispositions

  • #506: fixed: shared effective-endpoint validation in manifest loading and app!; field-specific value-free errors, unchanged whole-section overrides, valid generator metadata/examples.
  • #505: fixed: one shared pre-callback rejection gate for both custom runners, canonical redacted 500/HEAD suppression, original typed-error precedence, bounded secondary diagnostics, and no automatic send after callback entry.
  • Reviewer-owned threads and all other dispositions are untouched; issue closure waits for default-branch integration.

Executed verification

  • Complete prescribed RUST_TEST_THREADS=2 bash scripts/run_tests.sh < /dev/null: exit 0 on the implementation tree before this commit's final verification-record update. Workspace: 2,737 passed / zero failed / two ignored / 33 binaries. Harness: 670 passed / zero failed / four opt-in ignored. All four local adapter CI suites, generated project, demo tests and three demo WASM checks pass.
  • All 35 additional workflow format/lint/build/docs checks: exit 0. They cover narrow Fastly CLI/no-feature builds, every adapter's supported lint selections, provider/ingress fixtures, harness, demo and docs.
  • Committed-head reruns: manifest 88 passed, macro units 120 passed, lifecycle 13 passed; each exit 0. Strict workspace Clippy also exits 0.
  • Committed-head scripts/run_test_nonzero.sh --ignored fastly_pre_callback_wire_controls cargo test --offline --locked --manifest-path crates/edgezero-test-harness/Cargo.toml fastly_pre_callback_wire_controls: exit 0, one selected test executes six real Viceroy GET/HEAD/setup-error/committed-callback controls. Capture: crates/edgezero-test-harness/fixtures/reusable-app/.runs/fastly-505-18e84c75c93eeea0-cb97-0.
  • Red controls preceded implementation. The actual compiler probe originally accepted the invalid endpoint; the wire red run exposed non-JSON host-generated WASI-error bodies. The nonzero guard and frozen-source reruns replace initial formatting, zero-selection and identity-drift attempts; those attempts receive no qualification credit.

Remaining blocker
The separate scripts/test_integration.sh --adapter fastly --suite assembly exits 1: seven cases retain missing launch identity on required prescribed response. All 27 existing logging controls and six new controls pass. Its capture is crates/edgezero-test-harness/fixtures/reusable-app/.runs/18e84c266f7a83d8-9501-0. No gate was weakened, dismissed or credited. Hosted/manual qualification and independent current-head acceptance remain open; this is not a merge-ready claim.

Approved repair plan and results · logging contract

Two read-only investigation/review lanes checked manifest/macro selection and Fastly lifecycle ownership against the pinned SDK. The selected implementation reuses one schema check and the existing canonical egress path. Outbound contracts, SDK/runtime/decoder pins, body suppression, deadlines, limits and capabilities remain unchanged.

Phase: implementation. Branch prfection/275-a7-rebuild, worktree .claude/worktrees/pr-275-a7-rebuild. Acting login and PR author: aram356 (one-account mode). Cleanup: removed none; kept unmerged worktrees, evidence and build caches; destructive cleanup is not eligible while the cycle remains open. No deployment, thread resolution or merge. Request re-review of this head; reviewers/maintainer own the remaining acceptance and qualification decisions.

@aram356

aram356 commented Nov 15, 2026

Copy link
Copy Markdown
Collaborator Author

REVIEWER: accepted in code, at 4cbabfee: #505 and #506 from Trusted Server's round-2 asks, against their acceptance criteria. I have not run them from Trusted Server yet: the reason is under "Not run from Trusted Server" below. #504 remains open.

#505: a setup failure before the callback now gets an error response

  • One gate for both runners. serve_custom and run_custom both go through with_managed_logging (lifecycle.rs). If initialize_for_app::<A>() fails, reject_logging_failure sends send_assembly_error(request, EdgeError::internal(error), ..), which is the canonical redacted 500. The gate then returns the original ManagedLoggerError, and the callback never runs.
  • Criterion 1 (a 5xx instead of nothing): met.
  • Criterion 2 (the error still returns to the caller): met. A callback error after entry still triggers no extra response.
  • Criterion 3 (the guide): met. docs/guide/adapters/fastly.md now says what the client receives, that HEAD carries no payload, and that serving still ends with the original error.
  • Criterion 4 (a Viceroy test): met.
    • fastly_failure_controls (runners/assembly/managed_logging.rs) runs pre-callback GET and HEAD failures for both serve and run. It requires the canonical internal error, no application launch header, and an empty HEAD body.
    • Two committed-callback controls require the application response to stay as it was.
    • The unit test managed_logging_setup_failure_rejects_once_without_calling_callback covers all five ManagedLoggerError variants.

#506: invalid endpoint names are rejected when the manifest loads

  • validate_manifest_fastly_logging (manifest.rs) checks the effective Fastly endpoint, from either [adapters.fastly.logging] or [logging.fastly]. It rejects:
    • an empty name;
    • a name containing a line feed or :;
    • the reserved names stdout and stderr.
  • The error names the field ([<section>].endpoint) and never echoes the value.
  • Tests cover both sections with "", ts:log, line\nbreak, stdout and stderr. Another test keeps whole-section shadowing and other adapters as they were.
  • app! rejects the same manifest at compile time (edgezero-macros, fastly_logging_endpoint_reports_compile_time_manifest_field).
  • The guide example moved from stdout to app_log.
  • Trusted Server's endpoint, tslog, passes the new check.

Not run from Trusted Server

  • Why not: Trusted Server pins rc/identity at 10166628 ((DO NOT MERGE) Identity integration branch #495), which does not carry 4cbabfee. PR 275's head alone lacks the stacked PRs (Preserve outbound cache tags and borrowed readiness #460, Review received-head and CI fixture follow-ups to PR 275 #499) that Trusted Server's head already uses, so I could not pin to it directly.
  • What remains: the end-to-end replay. Once rc/identity merges this head and Trusted Server repins, I will replay the round-2 probe, endpoint = "ts:log", and check a logger failure on /health.
  • What this rests on: the code read above and the runs this handoff reports: lifecycle::tests 13 passed, manifest::tests 88 passed, macro units 120 passed, and fastly_pre_callback_wire_controls 1 passed with six Viceroy controls.
  • Also not checked: the seven failing assembly-attestation cases this handoff reports.

@aram356 aram356 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REVIEWER: REQUEST_CHANGES — GitHub blocked this review event: HTTP 422 Review Can not request changes on your own pull request (acting login aram356 is the PR author). This COMMENT carries the REQUEST_CHANGES verdict and the inline finding; it is not a formal approval and does not make this PR merge-ready.

Re-review of head 4cbabfee (fix(fastly): validate logging and reject setup failures, +1344/-23, closing #505 and #506). Acting login aram356 is the PR author, so one-account mode applies and no event here is a formal approval.

No finding was pending resolution. All 96 threads were already resolved when this round began, including the timing finding I accepted and resolved at e9ffaa8f. Nothing new was awaiting a disposition, so there was nothing to resolve this round; the inline finding below is new.

The new commit is sound

I reviewed 4cbabfee in full and found no defect in it:

  • validate_manifest_fastly_logging rejects empty endpoints, names containing \n or :, and the reserved stdout/stderr; it resolves per-adapter config before the global fallback, reports the correct field path, and sets skip_on_field_errors = false so it still runs when other fields fail.
  • #[app] rejects an invalid manifest at compile time, before emitting application code, so a bad endpoint cannot reach runtime.
  • with_managed_logging sends the rejection exactly once, writes a payload-free diagnostic (category enums only, no request or provider text), and returns the original typed error via error.into(). The no-second-send rule holds structurally: it returns before handler(...) runs, and lifecycle.rs:391 proves it with panic!("callback must not run after failed setup") as the handler plus panic!("callback results must not send automatic errors") as a second guard.
  • The new managed_logging.rs fixture exercises healthy and failure control paths through Viceroy with witnesses, so this is runtime coverage rather than unit tests alone.

Both issues it closes are real and open, and their titles match the change precisely.

Verification at 4cbabfee

Check Command Exit Result
Gate 1 cargo fmt --all -- --check 0 clean
Gate 2 cargo clippy --workspace --all-targets --all-features -- -D warnings 0 0 errors
Gate 4 cargo check --workspace --all-targets --features "fastly cloudflare spin" 0 0 errors
Gate 5 cargo check -p edgezero-adapter-spin --target wasm32-wasip2 --features spin 0 0 errors
Harness cargo test --offline --locked --manifest-path crates/edgezero-test-harness/Cargo.toml 0 670 passed; 0 failed; 4 ignored
Fastly wasm, default features cargo test --target wasm32-wasip1 --lib (in-crate) 0 183 passed; 0 failed ×3
Full prescribed suite RUST_TEST_THREADS=2 bash scripts/run_tests.sh < /dev/null 1 aborted at fastly CI: runtime-unit-tests, exit status 134

Why this is REQUEST_CHANGES rather than BLOCKED

The one inline finding is an author-owned test defect with a deterministic reproduction, so a code change is identified and the blocker is not external. This replaces the BLOCKED verdict I gave at e9ffaa8f for the same symptom, which I had misclassified as intermittent.

Open, outside this finding

The separate seven-case Fastly assembly-attestation gate remains open, disclosed at docs/superpowers/plans/2026-11-07-pr275-review-repairs.md:686, whose own text states its evidence is "local functional evidence, not hosted or frozen-candidate qualification."

Merge readiness

Not merge-ready. The prescribed suite fails deterministically at this head, this finding is open, and in one-account mode a COMMENT verdict is never a formal approval regardless of GitHub's merge button.

Next actor: CREATOR.

.between_bytes_timeout(Duration::from_millis(913))
.finish()
.expect("native SDK registration control");
assert_eq!(control.get_connect_timeout(), Duration::from_millis(711));

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REVIEWER: blocking This test aborts the whole wasm process under Viceroy, so the full prescribed suite fails deterministically at this head.

Impact. RUST_TEST_THREADS=2 bash scripts/run_tests.sh exits 1 at fastly CI: runtime-unit-tests. The retained log contains no test result: line and zero FAILED tests, because the process is aborted rather than a test failing — so the phase reports exit status: 134 (SIGABRT) and the suite stops.

Root cause, from the wasm backtrace. Backend::get_connect_timeout() is not Viceroy-safe: the SDK itself panics inside it.

11: core::result::unwrap_failed
12: <Result<Duration, fastly_shared::FastlyStatus>>::expect
      at fastly-0.13.1/src/backend.rs:384:14
13: <fastly::backend::Backend>::get_connect_timeout
14: edgezero_adapter_fastly::outbound::fastly_impl::tests::configured_connect_timeout_reads_registered_sdk_timeout_settings
      at crates/edgezero-adapter-fastly/src/outbound.rs:3670:32

The getter .expect()s on a FastlyStatus; Viceroy does not implement that hostcall, so it aborts the process. The test is gated #[cfg(target_env = "p1")], so it is selected exactly in the environment where the getter cannot work.

Evidence that it is deterministic, not flaky. Reproduced with the suite's own flags (from crates/edgezero-test-harness/src/runners/ci/platforms.rs:367-385):

CARGO_TARGET_WASM32_WASIP1_RUNNER="$(command -v viceroy) run" \
  cargo test --offline --locked -p edgezero-adapter-fastly \
  --no-default-features --features fastly,test-utils --target wasm32-wasip1 --lib
  • 3 of 3 runs: exit 134.
  • The single test alone, -- --exact: exit 134.
  • The same suite with -- --skip configured_connect_timeout_reads_registered_sdk_timeout_settings: still exit 134, so at least one other test in this file hits the same getter.
  • Under default features the same --lib target passes 3/3 at 183 passed; 0 failed — the differentiator is --no-default-features --features fastly,test-utils, which is what the suite uses.

This corrects my previous round. At head e9ffaa8f I recorded this as an intermittent Viceroy shutdown abort and classified it as an environment blocker with no author change identified. That was wrong: with the suite's exact feature set it reproduces every time, and it is author-owned test code.

Not a production defect. grep -n get_connect_timeout crates/edgezero-adapter-fastly/src/outbound.rs returns only lines 3670, 3690 and 6250, all inside the #[cfg(test)] region that begins at line 1763. No production path calls these getters, so the shipped adapter is unaffected.

Requested correction. Stop calling Backend::get_*_timeout() in any test that runs under wasm32-wasip1/Viceroy. The readback assertion can use the adapter's own connect_timeout_setting() / registered-identity state, which this crate already exposes and which earlier rounds verified; or gate these three assertions off Viceroy explicitly rather than onto it. Please do not address this by removing the fastly,test-utils phase from the suite or by marking the phase non-blocking — that would hide the suite failure instead of fixing it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

5 participants