Skip to content

fix(computers): two composes of one Linux disk on a machine take turns, so a boot's pinned disk stays put - #668

Closed
earakely-scale wants to merge 2 commits into
mainfrom
edgararakelyan/linux-disk-compose-lock
Closed

earakely-scale wants to merge 2 commits into
mainfrom
edgararakelyan/linux-disk-compose-lock

Conversation

@earakely-scale

@earakely-scale earakely-scale commented Nov 13, 2026 •

Copy link
Copy Markdown
Collaborator

What changes

launcher_test.py's three tests fail now and then in the slow tier with KeyError: 'image': on main at eac7660 and at #580's merge (d1fcdf8), and on #666. The shared desktop's amd64 disk had failed its probe, so the computer had no disk image:

docker run ... --mount type=image,source=localhost:5000/agentenv-computers:832369…@sha256:9ee29… ...
docker: Error response from daemon: No such image: localhost:5000/agentenv-computers:832369…@sha256:9ee29…

Why. A slow shard runs two pytest workers, and both compose the shared desktop. When its disk's key is new, as on the first run after the computer images change, neither finds it stored, so both build it. Each build tags localhost:5000/agentenv-computers:<key> and pushes it, and the two builds' bytes differ. So the second build moves the tag off the image the first had pinned by digest. The first's probe then mounts name:tag@digest with --mount type=image, which Docker doesn't pull for, and the image under that name is gone. A run that copies the disk from the computer image mirror builds nothing, which is why later main runs pass.

I checked Docker's side on a Docker 28 daemon on the containerd image store, the store CI uses. docker image inspect and --mount type=image agree on a pinned name every time. Once the tag moves to a rebuilt image, the first image's pinned name is gone for both, and an explicit pull brings it back.

The fix. The Linux disk is now looked up, made and pushed holding a lock of its key (linux-disk:<key>), as Windows and Omarchy disks already are. A second compose of the same disk on the machine waits, finds the first's push, and reuses it: one build, and no tag moves.

Testing

  • A new unit test, test_a_disk_is_looked_up_made_and_pushed_holding_a_lock_of_its_key_so_composes_of_it_take_turns: each arch's disk is looked up, made and pushed while its key's lock is held. It fails without the change.
  • tst/unit/env/envs/test_computer_disk.py: 43 passed.

🤖 Generated with Claude Code

RetriggerView in GreptileConfidence Score: 5/5

The PR appears safe to merge; no actionable new issue was found.

Summary

The PR keeps Linux disk lookup, build, and push under one lock per disk key.

  • Linux disk composes now reuse the first image built for a key.
  • Integration workers now share locks for the machine resources they use.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Compute disk key] --> B[Acquire shared lock]
  B --> C{Disk already stored?}
  C -->|Yes| D[Reuse pinned image]
  C -->|No| E[Build and push disk]
  E --> F[Pin image digest]
  D --> G[Release lock]
  F --> G
  G --> H[Probe disk]
Loading

Reviews (2) · Last reviewed commit: "test: the integration tier's workers sha..." · Reviewed by Greptile

…s, so a boot's pinned disk stays put

Two composes of the same new disk key on one machine (the slow tier's two pytest workers, each composing the shared
desktop) both found nothing stored and both built it. Each build tags localhost:5000/agentenv-computers:<key> and its
bytes differ, so the second moved the tag off the image the first had pinned by digest. The first's probe then
mounted name:tag@digest with --mount type=image, which Docker doesn't pull for, and failed with "No such image",
failing launcher_test's three tests with KeyError: 'image' on every run that built the disk rather than copied it from
the mirror (eac7660 and d1fcdf8 on main, and PRs that change the computer images).

The Linux disk is now looked up, made and pushed holding a lock of its key, as Windows and Omarchy disks are, so the
second compose waits and reuses the first's.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…machine's Docker and registry

agent-env keeps its locks in the state root, and each integration test has a state root of its own, so the disk lock
alone didn't stop two pytest workers composing one computer at once: on this PR, both still built the desktop's disk
under the same tag and the first's boot failed as before. A session fixture keeps the locks whose ids name this
machine's image work (composes, disks, disk exports, image builds, pulls and saves) in one folder for the run, as one
user's processes share theirs, so such work takes turns across workers and the second reuses what the first pushed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@earakely-scale

Copy link
Copy Markdown
Collaborator Author

Folded into #666 (commits 9f14ba4 and 3a8f4b4 there): #666 changes the disk, so its slow shard 3 hit this race on every run; one PR keeps the release fix and what lets its CI pass together.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant