Skip to content

feat(computers): an egress proxy in the ubuntu-24.04 computer, so live sites load at any universe date (FD-5478) - #657

Merged
earakely-scale merged 10 commits into
mainfrom
edgararakelyan/fd-5478-hb18-egress-proxy-in-the-computer
Nov 13, 2026
Merged

earakely-scale merged 10 commits into
mainfrom
edgararakelyan/fd-5478-hb18-egress-proxy-in-the-computer

Conversation

@earakely-scale

@earakely-scale earakely-scale commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

What changes

Since #649, a computer's session programs read the universe's time. Chromium, Java and an official Node build check a site's certificate against that time with their own TLS, which the OpenSSL shim (#639) can't reach. So with the clock months back or ahead, they refuse live sites. This adds an egress proxy to the ubuntu-24.04 base, on real time, that every session program goes out through (FD-5478).

What it does with each connection:

  • It serves CONNECT and absolute-form HTTP on 127.0.0.1:3128. It runs as its own user, agentenv-egress, under root's supervisord. The clock library leaves that user on real time. There are no firewall rules and no NET_ADMIN, so it runs on any provider.
  • Pass through: when the clock is unarmed, or when every universe a client may check against is more than 8 days inside the dates every certificate of the site's verified chain shares, it relays the client's bytes untouched. It also passes through when its own probe handshake doesn't finish, as with a site asking for the client's certificate, so the client answers it itself. Each of a site's addresses is decided by its own probe, cached per address: one that doesn't answer is passed over, one already found working is tried first, and a connection goes only to an address a probe judged. The universes a client may check against are now, the universe at the clock's anchor, and, for 30 s after a change, the one it left.
  • Intercept: otherwise it verifies the site against real time and the image's public roots. It then hands the client a certificate from the computer's own CA, valid from 1950 to 9999, the same window feat(computers): routed sites' certificates hold at any date a computer's clock can show (FD-5480) #654 gave routed sites. A site that doesn't verify is handed on under a CA nothing trusts, so the client refuses it as it would the site.
  • The client's own fingerprint: the proxy's handshake with the site uses uTLS, built from the client's own ClientHello. The site sees the client's fingerprint and ALPN offer either way, and h2 runs end to end.
  • Destinations: it refuses every destination that isn't global (RFC 1918, loopback, link-local, CGNAT, the metadata address and IANA's other special blocks). It tries each of a name's addresses in turn, IPv4 first. It allows a routed site's name, which the hosts file maps to the site proxy.
  • Plain HTTP: it forwards the client's own head, rewriting only its request line and the proxy-only headers, then relays both ways. An upgrade, such as a WebSocket, keeps its Connection and Upgrade headers; any other request closes once answered, and an upload waiting on 100 Continue gets it at once.
  • Logging: one JSON line per connection, in root's supervisord log, which the session user can't read. Its probe and leaf caches are bounded (4,096 each).

The CA and the session's settings:

  • Init mints the CA at boot (egress boot), once per computer. Its key is readable by agentenv-egress alone.
  • Boot trusts the CA in the system store, agent-env's trust files (/etc/agentenv/ca.pem, ca-bundle.pem), every certifi bundle, every JDK's store and the session user's NSS database. A step that fails fails init, naming it, except a certifi bundle or a JDK's store an app brings: one that can't take the CA is named among the boot's warnings, and the computer boots.
  • The session's supervisord, and a login shell through /etc/profile.d, set:
    • HTTP(S)_PROXY, ALL_PROXY and NO_PROXY (localhost and host.docker.internal);
    • SSL_CERT_FILE, REQUESTS_CA_BUNDLE and NODE_EXTRA_CA_CERTS, at agent-env's trust files, as the local provider already did;
    • JAVA_TOOL_OPTIONS with Java's proxy properties, and NODE_USE_ENV_PROXY=1 for Node's fetch.

agent-env's own servers go direct. The computer's driver and app servers (computer.serve, computer.serve_app) run on real time, so as they start they call agentenv_protocol.transfers.direct(): their own transfers, recordings, snapshots and files by grant, reach the object store directly, never through the proxy, which would refuse a store or gateway at a private address, still trusting the CAs SSL_CERT_FILE names. The programs they start keep the session's proxy settings, since the environment is untouched. (The driver's sync_time already read the gateway directly.)

How it's built: one Go program, artifacts/ubuntu-24.04/egress/, decided on FD-5478 instead of the prototype's Python proxy plus Go helper. The base's Dockerfile builds it in a stage of its own, which runs go vet and its tests first, so no Go toolchain is left in the image. make egress-test runs gofmt, go vet and its tests under -race in that same Go image, so it needs Docker, not Go, and CI runs it as a new egress job, which pulls the image from mirror.gcr.io first. egress/README.md says what runs where, what the proxy does with each kind of connection, and its log's fields. go.sum pins its modules, Dependabot's new gomod entry keeps them current, and THIRD_PARTY_NOTICES lists them.

Behaviour to know about

  • Chromium on the desktop now goes through the proxy too. It reads the environment's proxy settings under XFCE. FD-5481 still adds its managed proxy policy.
  • Clients that ignore proxy settings still go direct. That includes an older Node's fetch, which predates NODE_USE_ENV_PROXY. They check certificates against universe time unless they use the system's OpenSSL, which the shim covers.
  • A computer with no network now answers 502 through the proxy. The 502 names the cause, "network is unreachable", where a socket error used to come back.
  • The base now runs a third user. The tests' "who runs what" check expects agentenv-egress, running the proxy alone.

Testing

On an EC2 x86_64 host with local sandboxes:

Test What it shows Result
Go, in the image build and the egress job The pages the follower's writer wrote (testdata/) read as the universe they hold; a page reader that rejects a torn page; the universe left held for 30 s; ClientHellos read across records, and their SNI and ALPN, and one whose fields overrun it refused; resumption extensions left out upstream; 22 non-global addresses refused; and, against TLS servers in the process, a site passed through on real time and inside its certificate, intercepted over h2 outside it, an unverified site refused by the client, plain HTTP forwarded, and boot minting the CA once with its key private. After review: every address tried in turn, each by its own probe, one already found working first, one that stops answering giving way to the next, and a new one judged by its own probe; the verified chain's shared dates bounding pass-through; a site asking for a client certificate passed through; an upgrade relayed both ways over plain HTTP; a 100 Continue upload answered at once; the caches bounded; no site held up by another's slow probe; a panicking probe leaving no one waiting; an app's store that can't take the CA warned of while the boot goes on. All under -race 27/27
test_a2a_object_transfers.py (protocol, unit) A process made direct takes no proxy from the environment for its transfers but still its certificates, and both computer servers make themselves direct as they start 2 new
test_computer_egress_base.py (new, unit) The proxy's CA window is routed sites'; root's supervisord runs it as its own user; init boots it first; the session's settings match a login shell's and the local provider's trust variables, and name the address the proxy listens on; the pages the Go tests read are what clockpage.encode writes 5/5
egress_test.py (new) The session user can't read the CA's key, stop the proxy, replace it, or add to its roots or the trust files, and a killed proxy is back within 3 s. All 22 private and local destinations are refused and logged. With the universe 182 days back, Java's HttpClient, an official Node 24 build's fetch and curl reach example.com through it (Java and Node refuse it without the proxy). Fresh handshakes through 12 jumps of +1 week with the universe a month on: none aborted, intercepting along the way 4/4
base_computer_test.py, computer_env_test.py, computer_clock_test.py, clock_desktop_test.py, ca_trust_test.py Regression, with every session program now going through the proxy: who runs what, the probe at full, recordings, a closed network, the clock through jumps, a fresh Chrome on a routed site, the CA trust of Node and both JDKs 23/23

The clock helpers these tests share (UniverseClock, computer_container, session_run) moved into the computer tests' conftest. CI's slow tier ran them at f337d4a: egress_test.py 4/4, clock_desktop_test.py 3/3, computer_clock_test.py 1/1.

Measured on the host, through the clock desktop's curl and Chromium. "Passed through" is the clock armed at now; "intercepted" is the universe 182 days back:

Check Passed through Intercepted
JA4 at tls.peet.ws, curl t13d3112h2_e8f1e7e78f70_b26ce05bbdd6 the same
JA4 at tls.peet.ws, Chromium t13d1518h2_8daaf6152771_4980c97edce0 the same
24 bot-protected sites, page loaded with no challenge (headless Chromium) 11 12 (23 of 24 sites gave the same result both ways)
Debian 13.7.0 netinst ISO, 792,723,456 bytes, sha256 intact, 7.2 s intact, 6.6 s

On Modal (modal_vm), at ab865c9: the bundle registered into throwaway S3 and ECR stores, composed and probed on a Modal VM, and deployed in a multi env on modal_vm:

Check Result
Who runs what root, agentenv-clock, agentenv-egress, the session user
Boot the CA minted and trusted in agent-env's trust files, the driver venv's certifi bundles and NSS, in 0.75 s, no warnings
A live site from the session, the clock armed 182 days back example.com 200, intercepted (outside its certificate)
A file loaded by its S3 read grant, and a snapshot uploaded by its write grant both through, straight to S3: the proxy's log has no S3 row, and no inline fallback was taken
Teardown clean

Docs

The computers page should say that the ubuntu-24.04 base goes out through an egress proxy on real time, so live sites load at any universe date. It should also cover which clients follow the proxy's settings and which don't.

🤖 Generated with Claude Code

RetriggerView in GreptileConfidence Score: 5/5

The changes since the previous review appear safe to merge, and both numbered findings are addressed.

Summary

Adds a real-time egress proxy to the ubuntu-24.04 computer so supported clients can reach live HTTPS sites while using universe time.

  • Live sites load through the computer even when its universe clock shifts.
  • Computer servers send their own object-store transfers directly.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Session client] --> B[Egress proxy on real time]
  B --> C{Destination allowed?}
  C -->|No| D[Refuse connection]
  C -->|Yes| E{TLS needs interception?}
  E -->|No| F[Relay client bytes unchanged]
  E -->|Yes| G[Check site certificate on real time]
  G --> H[Give client a locally signed certificate]
  H --> I[Relay decrypted traffic over TLS]
  J[Agent-env server transfers] --> K[Object store directly]
Loading

Reviews (12) · Last reviewed commit: "fix(computers): the egress job pulls Go ..." · Reviewed by Greptile

earakely-scale and others added 2 commits November 12, 2026 16:27
…e sites load at any universe date (FD-5478)

Every session program goes out through the egress proxy on 127.0.0.1:3128, which runs on real time as its own user
under root's supervisord. A TLS connection is passed through untouched when the clock is unarmed or every universe a
client may check against is inside the site's certificate by more than 8 days; otherwise the site is verified against
real time and the public roots, and the client is handed a certificate from the computer's own CA, minted at boot and
valid from 1950 to 9999. The handshake with the site is uTLS's, built from the client's own ClientHello, so the site
sees the client's fingerprint and ALPN offer either way, and h2 runs end to end. Destinations that aren't global are
refused. The CA is trusted in the system store, agent-env's trust files, every certifi bundle, every JDK's store and
the session's NSS database; the session's environment points HTTP(S)_PROXY, ALL_PROXY, Java's proxy properties and
Node's NODE_USE_ENV_PROXY at the proxy, and the trust variables at agent-env's trust files.

The proxy is one Go program, egress/, built and tested in a stage of the base's Dockerfile.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s egress proxy or through it

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale
earakely-scale requested a review from a team as a code owner November 13, 2026 01:11
@socket-security

socket-security Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Review the following changes in direct dependencies. Learn more about Socket for GitHub.

Diff Package Supply Chain
Security
Vulnerability Quality Maintenance License
Addedgolang/​github.com/​refraction-networking/​utls@​v1.8.288100100100100

View full report

@socket-security

socket-security Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Warning

Review the following alerts detected in dependencies.

According to your organization's Security Policy, it is recommended to resolve "Warn" alerts. Learn more about Socket for GitHub.

Priority Alert  (click "▶" to expand/collapse) Action
High priority
High CVE: compress zstd: Panic on two-block frames in golang github.com/klauspost/compress

CVE: GHSA-fpmh-frq8-7rj8 compress zstd: Panic on two-block frames (HIGH)

Affected versions: < 1.20.2

Patched version: 1.20.2

From: src/agent_env/computers/artifacts/ubuntu-24.04/egress/go.mod → golang/github.com/refraction-networking/utls@v1.8.2 → golang/github.com/klauspost/compress@v1.17.4

ℹ Read more on: This package | This alert | What is a CVE?

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: Remove or replace dependencies that include known high severity CVEs. Consumers can use dependency overrides or npm audit fix --force to remove vulnerable dependencies.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore golang/github.com/klauspost/compress@v1.17.4. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn
Low priority
Potential vulnerability: golang golang.org/x/crypto with risk level "medium"

Location: Package overview

From: src/agent_env/computers/artifacts/ubuntu-24.04/egress/go.mod → golang/github.com/refraction-networking/utls@v1.8.2 → golang/golang.org/x/crypto@v0.36.0

ℹ Read more on: This package | This alert | Navigating potential vulnerabilities

Next steps: Take a moment to review the security alert above. Review the linked package source code to understand the potential risk. Ensure the package is not malicious before proceeding. If you're unsure how to proceed, reach out to your security team or ask the Socket team for help at support@socket.dev.

Suggestion: It is advisable to proceed with caution. Engage in a review of the package's security aspects and consider reaching out to the package maintainer for the latest information or patches.

Mark the package as acceptable risk. To ignore this alert only in this pull request, reply with the comment @SocketSecurity ignore golang/golang.org/x/crypto@v0.36.0. You can also ignore all packages with @SocketSecurity ignore-all. To ignore an alert for all future pull requests, use Socket's Dashboard to change the triage state of this alert.

Warn

View full report

earakely-scale and others added 2 commits November 12, 2026 17:23
…through by the whole chain, and relays plain HTTP raw

From review:
- Every checked address of a destination is dialled in turn, IPv4 first, not the first alone.
- Passing a TLS connection through is decided on the dates every certificate of the verified chain shares, not the
  leaf's alone.
- A probe handshake that doesn't finish, as with a site asking for the client's certificate, passes the connection
  through for the client to answer; it's cached for a minute, not an hour.
- Plain HTTP forwards the client's own head, its request line and proxy-only headers rewritten, and relays both ways,
  so an upgrade keeps its Connection and Upgrade headers and an upload waiting on 100 Continue isn't held for its body.
- The builder's platform check starts the image the shipped base's last stage builds from, now that the base has a
  build stage before it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From review: a site's probes share one of 64 fixed locks rather than one each, the probe cache drops its expired
entries once past 4,096, and the leaf cache drops its oldest leaf past 4,096; a connection keeps the leaf it was handed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…erver its probe judged, and holds no lock while probing

From review: a probe's result is cached under the address it reached, not the name's whole address list, and a
connection it lets through is dialled to that address alone, so another of the name's servers, with other dates, can't
be passed through on its result. A site's cached result is read before anything waits; concurrent probes of one site
wait on the one under way, and no lock is held during a probe, so a slow site holds up no other.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…at address's own probe

From review: a site's addresses are tried in turn, each by its own probe, cached per (address, SNI). An address that
doesn't answer is passed over, its probe cached as unreachable for a minute for that address alone, so a new address
gets a probe of its own; one that stops answering after its probe is forgotten and the next tried. A probe's
bookkeeping is released however it ends, a panic included, and a ClientHello whose session id, cipher suites or
compression methods overrun it is refused rather than sliced past its end.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… app's store that can't take the egress CA doesn't stop the boot, and Dependabot keeps the proxy's Go modules current

- agentenv_protocol.transfers.direct() has this process's transfers take no proxy from the environment; the computer's
  driver and app servers call it as they start, so their own transfers (recordings, snapshots, files by grant) reach
  the object store directly, on real time, while the programs they start keep the settings that send them through
  the egress proxy.
- The egress proxy's boot names a certifi bundle or a JDK's store that can't take the CA among its warnings and boots
  on; the CA, the system store, agent-env's trust files and the session's NSS database still fail it.
- Dependabot's gomod entry covers the egress proxy's modules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
earakely-scale and others added 2 commits November 12, 2026 20:59
…s, and the egress proxy tries an address it found working first

From review:
- transfers.client_options() gives a direct process's clients no proxy from the environment but still its
  certificates (SSL_CERT_FILE, SSL_CERT_DIR), through an SSL context made from them: httpx's trust_env takes both or
  neither, and a local store's CA is trusted through SSL_CERT_FILE.
- The egress proxy tries the addresses whose fresh probe answered before those it would have to probe again, so a
  working address isn't held behind one whose failure has expired.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…job, and both sides read the same clock pages

- `make egress-test` runs gofmt, go vet and go test -race in the Go image the base's Dockerfile builds with, so it
  needs Docker, not Go; CI runs it as the `egress` job, so a failing Go test shows there rather than as a broken base
  image in the slow tier.
- `clockpage.encode` is the one place the follower's writer lays out a page. `egress/testdata/` holds two pages it
  wrote; the Go test reads them as the universe they hold, and a unit test checks they're still what `encode` writes.
  A unit test also checks the session's proxy settings name the address the proxy listens on.
- `egress/README.md` says what runs where, what the proxy does with each kind of connection, and its log's fields.
- The clock helpers the computer tests share (`UniverseClock`, `computer_container`, `session_run`) live in the
  computer tests' conftest instead of one test module.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comment thread .github/workflows/local-backends.yml Resolved
…y's limits have names

- The `egress` job pulls its Go image through mirror_base_images.py first, as the image builds do, since ECR Public's
  anonymous pulls fail on hosted runners for hours at a time.
- The proxy's inline limits are named: HeadTimeout and HandshakeTimeout (both 15 s), HeadLimit, ErrorLimit,
  LeafInset, PagePoll and PageReads in main.go, the TLS record and handshake types in hello.go, and X.509's
  common-name bound in leaves.go.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale
earakely-scale merged commit c63d6cf into main Nov 13, 2026
39 checks passed
@earakely-scale
earakely-scale deleted the edgararakelyan/fd-5478-hb18-egress-proxy-in-the-computer branch November 13, 2026 15:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant