feat(computers): the clock app and its follower put a computer on its gateway's clock (FD-5476)

src/agent_env/computers/README.md

@@ -239,3 +239,8 @@ computer failed for good.
 
 This bundle has no tasks of its own: other bundles pull its computers by alias, and `agent-env run computers` has
 nothing to run.
+
+`envs/clock/` is the `clock` app, which a Linux X11 computer env composes (`apps = ["clock"]`) to follow its multi env's
+gateway clock: agent-env's libfaketime from `clock/`, preloaded into every process, and the clock follower that keeps it
+on the gateway's universe time once `sync_env_clock` arms the multi env. `clock/README.md` says how. Name it from
+another bundle as `clock`, the alias `aliases.toml` gives it, or by its id, `@local/agentenv-framework/computers/clock`.

src/agent_env/computers/aliases.toml

@@ -5,3 +5,7 @@
 "macos-26" = ["macos-26"]
 "windows-11" = ["windows-11"]
 "omarchy-4.5" = ["omarchy-4.5"]
+
+# Ids a computer env's apps can name these by; a store's env of the id wins over an alias.
+[envs]
+clock = ["clock"]

src/agent_env/computers/clock/README.md

@@ -38,11 +38,41 @@ What follows the universe, and what doesn't:
 | `tls_realtime.c` | the TLS shim: OpenSSL's certificate checks on real time, preloaded after the library | Apache-2.0 |
 | `clockpage.py` | the clock page's writer, which the follower uses | Apache-2.0 |
 | `clock.conf` | the library's default settings | Apache-2.0 |
+| `clock-follower` | the follower: keeps the page in step with the gateway's clock (below) | Apache-2.0 |
+| `follow` | root's loop that keeps the follower running as `agentenv-clock` | Apache-2.0 |
 
 Build it on x86_64 or aarch64 Linux with a C toolchain, make, patch and curl:
 
     src/agent_env/computers/clock/build.sh OUT [TARBALL]
 
+## On a computer: the `clock` app and the follower
+
+A computer env follows its multi env's gateway clock by composing the `clock` app (`../envs/clock`, the alias
+`clock`): `apps = ["clock"]` on a Linux X11 computer, `amd64` or `arm64`. Its install builds this library at compose,
+installs `clock.conf` and preloads the library into every process (`/etc/ld.so.preload`), and adds the follower, its
+loop and a user of its own, `agentenv-clock`, whose uid `clock.conf` names as the page's owner, so the library trusts no
+page another user made.
+
+When `sync_env_clock` arms a multi env whose computer composes it, core configures the computer's follower before the
+arm, as root, through the sandbox's root channel (`agent_env.env.envs.computer.utils.clock`), never through the driver,
+which runs as the session user. It writes the follower's config (the gateway's `/clock/watch` URL as the computer's
+sandbox reaches it, the computer's read token, its follower name and a config id) to
+`/etc/agentenv/clock-follower/follower.json`, the follower's alone to read; makes the page, the follower's from its first
+byte; and starts `follow`, which runs `clock-follower` as `agentenv-clock` and runs it again within a second whenever it
+exits. The arm then waits for the computer to show the clock (`wait_followers_s`) and fails the step naming it if it
+doesn't; the app's `clock/v1` `sync_time` reports what the follower showed for that config id.
+
+Each round, the follower long-polls `/clock/watch` with the version it last showed, which acknowledges that version. On a
+new version it writes the page: the universe the gateway reported, at its rate, from the local real time the gateway's
+`sent` stamp corresponds to, the offset between the two clocks estimated from the watch's four stamps as NTP does. It
+records what it showed in `/run/agentenv-clock/status.json`. A failed round is recorded there and retried with backoff,
+and a new config starts it over from no version.
+
+The session user can't write the page or the clock file, read the follower's config, or stop either process: each is
+another user's. Not covered yet: a computer reset (`reset_env`) boots it afresh with no follower until the next arm;
+agent-env's own servers run on universe time too, with no real-time marker; and routed sites' certificates, Chromium's
+time zone and an egress proxy for live sites at other dates are later work.
+
 ## The clock page
 
 An 80-byte page (default `/dev/shm/agentenv-clock`, version 2) that `clockpage.py` rewrites in place: `seq` odd, the

src/agent_env/computers/clock/clock-follower

@@ -0,0 +1,104 @@
+#!/usr/bin/env python3
+"""The computer's clock follower: keeps the clock page agent-env's libfaketime reads in step with the gateway's clock.
+
+It runs as its own user (agentenv-clock), restarted by root's ``follow`` loop, so the session user can't stop it, read
+its config or write the page. Core writes its config through the sandbox's root channel at arm time, never through the
+driver: ``{"watch_url", "token", "follower", "config_id"}`` in CONFIG, readable by this user alone.
+
+Each round long-polls the gateway's ``/clock/watch`` with the version it last showed, which is its acknowledgement of
+that version. The gateway answers at once when the clock is at another version, else at its next change or after its
+hold. On a new version it writes the page (``clockpage.write``): the universe the gateway reported, at its rate, from
+the local real time the gateway's ``sent`` stamp corresponds to, the offset between the two clocks estimated from four
+stamps as NTP does. Then it records what it showed in STATUS, which anyone may read, for the clock app's ``sync_time``.
+A failed round is recorded there too and retried with backoff; a changed config starts over from no version."""
+import json
+import os
+import sys
+import time
+import urllib.error
+import urllib.parse
+import urllib.request
+from datetime import datetime
+
+sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
+import clockpage  # noqa: E402  (beside this file, in /opt/agentenv/clock)
+
+CONFIG = os.environ.get("AGENTENV_CLOCK_FOLLOWER_CONFIG", "/etc/agentenv/clock-follower/follower.json")
+STATUS = os.environ.get("AGENTENV_CLOCK_FOLLOWER_STATUS", "/run/agentenv-clock/status.json")
+#: Beyond the gateway's hold (CLOCK_WATCH_HOLD_S, 25 s), how long a watch may take before it's given up and tried again.
+WATCH_TIMEOUT_S = 40.0
+BACKOFF_S = (0.2, 0.5, 1.0, 2.0, 5.0)
+
+
+def load_config() -> dict | None:
+    try:
+        with open(CONFIG) as f:
+            config = json.load(f)
+    except FileNotFoundError:
+        return None
+    missing = [key for key in ("watch_url", "token", "follower", "config_id") if not config.get(key)]
+    if missing:
+        raise ValueError(f"{CONFIG} has no {', '.join(missing)}")
+    return config
+
+
+def epoch(rfc3339: str) -> float:
+    return datetime.fromisoformat(rfc3339.replace("Z", "+00:00")).timestamp()
+
+
+def watch(config: dict, after: str) -> tuple[dict, float, float]:
+    """One watch: the gateway's answer, and the local real times it was sent and received."""
+    query = urllib.parse.urlencode({"follower": config["follower"], "after": after})
+    request = urllib.request.Request(f"{config['watch_url']}?{query}",
+                                     headers={"Authorization": f"Bearer {config['token']}"})
+    sent = time.time()
+    with urllib.request.urlopen(request, timeout=WATCH_TIMEOUT_S) as response:
+        body = json.load(response)
+    return body, sent, time.time()
+
+
+def show(state: dict, sent: float, received: float) -> dict:
+    """Write the page for the gateway's ``state``, its ``received`` and ``sent`` stamps against ours."""
+    offset = ((state["received"] - sent) + (state["sent"] - received)) / 2  # gateway's real time minus ours
+    if not state.get("armed"):
+        return clockpage.write(0, 0.0, 1.0, time.time())
+    return clockpage.write(1, epoch(state["virtual_time"]), float(state["virtual_seconds_per_real_second"]),
+                           state["sent"] - offset)
+
+
+def record(status: dict) -> None:
+    os.makedirs(os.path.dirname(STATUS), exist_ok=True)
+    with open(STATUS + ".part", "w") as f:
+        json.dump({**status, "at": time.time()}, f)
+    os.chmod(STATUS + ".part", 0o644)
+    os.replace(STATUS + ".part", STATUS)
+
+
+def main() -> None:
+    # The clock file and the status are the follower's to write and anyone's to read, whatever umask it was started
+    # with: one of 0000 (docker exec's) would let the session user rewrite the clock file.
+    os.umask(0o022)
+    after, config_id, failures = "", None, 0
+    while True:
+        try:
+            config = load_config()
+            if config is None:
+                time.sleep(1)
+                continue
+            if config["config_id"] != config_id:
+                after, config_id = "", config["config_id"]
+            state, sent, received = watch(config, after)
+            if state.get("version") != after:
+                written = show(state, sent, received)
+                after = state["version"]
+                record({"config_id": config_id, "version": after, "armed": bool(state.get("armed")),
+                        "virtual_time": state.get("virtual_time"), "page": written.get("spec")})
+            failures = 0
+        except Exception as e:  # any round's failure is the status's to show; the next round tries again
+            record({"config_id": config_id, "version": after or None, "error": f"{type(e).__name__}: {e}"})
+            time.sleep(BACKOFF_S[min(failures, len(BACKOFF_S) - 1)])
+            failures += 1
+
+
+if __name__ == "__main__":
+    main()

src/agent_env/computers/clock/follow

@@ -0,0 +1,16 @@
+#!/bin/sh
+# As root: the computer's clock follower, kept running as agentenv-clock, back within a second whenever it exits. Core
+# starts it through the sandbox's root channel once it has written the follower's config; a second start replaces the
+# first. The session user can't stop either process: each runs as another user.
+set -u
+# Whatever exec started it (docker exec's is 0000), nothing the follower writes is any other user's to change.
+umask 022
+for pid in $(pgrep -f 'opt/agentenv/clock/follow$'); do
+    [ "$pid" != "$$" ] && kill "$pid" 2>/dev/null
+done
+pkill -u agentenv-clock 2>/dev/null
+while :; do
+    setpriv --reuid=agentenv-clock --regid=agentenv-clock --init-groups --no-new-privs --reset-env \
+        python3 /opt/agentenv/clock/clock-follower
+    sleep 1
+done

src/agent_env/computers/envs/clock/app.py

@@ -0,0 +1,57 @@
+"""The computer's clock on universe time: agent-env's libfaketime and its TLS shim, which keeps certificate checks on
+real time, built at compose from the recipe in ./recipe (src/agent_env/computers/clock) and upstream's tarball, which
+the app downloads at the sha256 build.sh pins, preloaded into every process (install.sh), with the clock follower
+beside them.
+
+The follower keeps the clock page in step with the gateway's clock. Core configures and starts it through the sandbox's
+root channel when ``sync_env_clock`` arms the env (``agent_env.env.envs.computer.utils.clock``), never through this app,
+whose server runs as the session user. The app's ``clock/v1`` ``sync_time`` only reports what the follower has shown,
+which is also what makes a multi env's gateway count the computer as one of its clock's followers."""
+import asyncio
+import json
+import re
+import time
+from pathlib import Path
+
+from agentenv_protocol.agent_env_environment import extension
+from agentenv_protocol.computer import (ComputerAppEnvironment, Download, Packages, RunScript, computer_app, linux)
+from agentenv_protocol.computer.errors import InvalidArgument, Timeout
+from agentenv_protocol.website.clock import CLOCK_EXTENSION_URI
+
+_BUILD = (Path(__file__).parent / "recipe" / "build.sh").read_text()
+TARBALL = re.search(r"https://\S+\.tar\.gz", _BUILD).group(0)
+SHA256 = re.search(r"\b[0-9a-f]{64}\b", _BUILD).group(0)
+#: What the follower last showed, written by it (agentenv-clock) and readable by anyone.
+STATUS = Path("/run/agentenv-clock/status.json")
+#: How long ``sync_time`` waits for the follower to show the config core just wrote.
+SYNC_WAIT_S = 20.0
+
+
+@computer_app(supports=[{"os": {"type": "linux", "display": ["x11"]}, "arch": ["amd64", "arm64"]}])
+@linux(packages=Packages(apt=["build-essential", "patch"]),
+       downloads=[Download(url=TARBALL, to="libfaketime.tar.gz", sha256=SHA256)],
+       run=[RunScript(script="install.sh")])
+class Clock(ComputerAppEnvironment):
+
+    @extension(CLOCK_EXTENSION_URI, description=(
+        "The computer's clock follows the gateway's. Core writes the follower's config as root and starts it; this "
+        "waits until the follower has shown the clock for that config and returns what it showed."))
+    async def sync_time(self, env_get_time_url: str, follower_config: str = "") -> dict:
+        """What the follower showed for ``follower_config``, the id core gave the config it wrote, once it has shown
+        any version for it. ``env_get_time_url`` is the gateway-network URL every server is handed; the follower
+        reaches the gateway at the URL in its config instead, as this computer's sandbox reaches it."""
+        if not follower_config:
+            raise InvalidArgument("sync_time needs follower_config: core writes the follower's config through the "
+                                  "sandbox's root channel before it calls this")
+        deadline, status = time.monotonic() + SYNC_WAIT_S, None
+        while time.monotonic() < deadline:
+            try:
+                status = json.loads(STATUS.read_text())
+            except (FileNotFoundError, json.JSONDecodeError):
+                status = None
+            if status and status.get("config_id") == follower_config and status.get("version") and \
+                    not status.get("error"):
+                return {"follower": status, "env_get_time_url": env_get_time_url}
+            await asyncio.sleep(0.2)
+        raise Timeout(f"the clock follower didn't show config {follower_config!r} within {SYNC_WAIT_S:g} s; "
+                      f"its status: {status}")

src/agent_env/computers/envs/clock/env.toml

@@ -0,0 +1,2 @@
+type = "computer_app"
+module = "app.py:Clock"

src/agent_env/computers/envs/clock/install.sh

@@ -0,0 +1,15 @@
+#!/bin/sh
+# As root at compose: agent-env's libfaketime and its TLS shim, built from ./recipe and the downloaded tarball, preloaded
+# into every process the computer starts, the library first, with the clock follower and its root loop beside them. The follower's user, agentenv-clock,
+# owns the clock page, which core creates at arm time; clock.conf says so, so the library trusts no other page.
+set -eu
+sh recipe/build.sh /opt/agentenv/clock libfaketime.tar.gz
+useradd --system --no-create-home --shell /usr/sbin/nologin agentenv-clock
+install -m 0644 recipe/clockpage.py /opt/agentenv/clock/clockpage.py
+install -m 0755 recipe/clock-follower /opt/agentenv/clock/clock-follower
+install -m 0700 recipe/follow /opt/agentenv/clock/follow
+install -d -m 0755 /etc/agentenv
+sed "s/^AGENTENV_CLOCK_PAGE_UID=.*/AGENTENV_CLOCK_PAGE_UID=$(id -u agentenv-clock)/" recipe/clock.conf \
+    > /etc/agentenv/clock.conf
+chmod 0644 /etc/agentenv/clock.conf
+printf '%s\n' /opt/agentenv/clock/libfaketimeMT.so.1 /opt/agentenv/clock/libtls_realtime.so > /etc/ld.so.preload

src/agent_env/computers/envs/clock/recipe/COPYING

@@ -0,0 +1 @@
+../../../clock/COPYING

src/agent_env/computers/envs/clock/recipe/agentenv_clock_page.c

@@ -0,0 +1 @@
+../../../clock/agentenv_clock_page.c

src/agent_env/computers/envs/clock/recipe/agentenv_clock_waits.c

@@ -0,0 +1 @@
+../../../clock/agentenv_clock_waits.c

src/agent_env/computers/envs/clock/recipe/build.sh

@@ -0,0 +1 @@
+../../../clock/build.sh

src/agent_env/computers/envs/clock/recipe/clock-follower

@@ -0,0 +1 @@
+../../../clock/clock-follower

src/agent_env/computers/envs/clock/recipe/clock.conf

@@ -0,0 +1 @@
+../../../clock/clock.conf

src/agent_env/computers/envs/clock/recipe/clockpage.py

@@ -0,0 +1 @@
+../../../clock/clockpage.py

src/agent_env/computers/envs/clock/recipe/follow

@@ -0,0 +1 @@
+../../../clock/follow

src/agent_env/computers/envs/clock/recipe/libfaketime.patch

@@ -0,0 +1 @@
+../../../clock/libfaketime.patch

src/agent_env/computers/envs/clock/recipe/tls_realtime.c

@@ -0,0 +1 @@
+../../../clock/tls_realtime.c

tst/unit/bundle/shipped_bundles_test.py

@@ -65,7 +65,7 @@ def test_the_shipped_env_aliases_name_the_mail_env_the_validator_deploys_the_dri
         "macos-harness": "@local/agentenv-framework/drivers/macos-harness",
         "codex-computer-use": "@local/agentenv-framework/drivers/codex-computer-use",
         "aside": "@local/agentenv-framework/drivers/aside",
-        "chromium": "@local/agentenv-framework/browsers/chromium"}
+        "chromium": "@local/agentenv-framework/browsers/chromium", "clock": "@local/agentenv-framework/computers/clock"}
     assert A2AAgent.VALIDATION_ENV_ID == "mail"
 
 def test_a_task_naming_no_agent_plans_with_the_shipped_default(tmp_path, monkeypatch, local_stores, cli_routing):