Repository navigation
Make a reproducible clean build part of done: vendoring, declared tests, no host installs, nothing gitignored - #52
Open
anakanemison wants to merge 2 commits into
Conversation
anakanemison
force-pushed
the
mschaller/u17-done-means-reproducible
branch
from
October 26, 2026 13:30
28fe7fc to
ee839c5
Compare
armandomontanez
approved these changes
Oct 29, 2026
The definition of done was parity of the action graph. Parity is unaffected by whether a TU compiles: the diff compares commands, and the command for a TU that fails compares as well as one that succeeds. So an agent could converge, truthfully report 0 discrepancies, and hand back a workspace where `bazel build //...` fails. Measured in any2bazel-lab: across ten larger projects, the skill-guided migration failed `bazel build //...` on three, and the unguided one on none. On Kodi the agent converged over 1,789 TUs while ten TUs could not compile, reasoning that FFmpeg and two other libraries were missing from the host and that "the parity diff is unaffected". The unguided run in the same environment vendored FFmpeg and SWIG through rules_foreign_cc and built crossguid and libdvdcss from pinned sources. - The "what converged does not cover" list gains the build itself, and the Repeat step says: repeat until converged, then, once, `bazel build //...` from a fresh output base must succeed, and so must `bazel test` over the tests the migration declares, with the files they read declared as data. Any target left out is named and explained. - "Create initial BUILD.bazel files" says a missing host dependency can be vendored (a pinned module or http_archive, or rules_foreign_cc), that a TU left uncompiled for a missing dependency is a gap in the migration, and not to install the dependency into the host instead (make install, /usr/local, /opt), since the build then passes only on the machine that was prepared. - The report includes the clean-build result. In the lab's later runs with this change: - The next Kodi migration built every target from a fresh output base for the first time, by vendoring FFmpeg, SWIG and four smaller libraries. Its ported kodi-test, though, aborted under `bazel test` with its data files undeclared, which is what the test clause addresses. - With the test clause, a later Kodi migration ran kodi-test under `bazel test`, and 1,686 of the 1,689 reference-passing tests passed. That agent had compiled FFmpeg 9 into its container's /usr/local instead of vendoring it, so on a fresh machine with the distribution's FFmpeg 6.1 the tree does not compile. The host-install clause was written after that run and has not yet been tested. Lab-Proposal: P23 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n's files Before reporting, the agent now checks `git status --short --ignored` for files it wrote that the project's .gitignore hides, and renames them or adds an exception. A file git ignores is left out of the migration anyone commits, and nothing in the parity loop or the build notices: both read the working tree. Found in any2bazel-lab: QGIS's `/build*` rule matched an agent's `build_defs/`, dropping from a plain `git add` the ten .bzl and .py files every BUILD file loads. Blender's .gitignore hid an unguided run's `.bazelrc`, YugabyteDB's its four `bazel/thirdparty` files, and GDAL's an agent's `cpl_config.h`. In each case the build passed in the agent's tree and fails from a commit of it. In the lab's six runs with this change, no migration had a file the project's .gitignore hides, and the QGIS agent ran the check and kept its rules out of `/build*`. Lab-Proposal: P27 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anakanemison
force-pushed
the
mschaller/u17-done-means-reproducible
branch
from
November 2, 2026 17:51
ee839c5 to
6113ef8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Parity compares commands, so a migration can converge while translation units fail to compile. Done now also requires, once converged,
bazel build //...from a fresh output base andbazel testover the declared tests with their data files declared, and any target left out must be named. SKILL.md says to vendor a missing host dependency rather than install it into the host, and, before reporting, to checkgit status --short --ignoredfor migration files the project's .gitignore hides.In any2bazel-lab, the skill-guided migration failed
bazel build //...on three of ten larger projects and the unguided one on none; on Kodi the agent converged with ten TUs uncompiled. The next Kodi run built every target by vendoring, and a later one passed 1,686 of 1,689 reference-passing tests underbazel test, though it had installed FFmpeg into its container; the host-install clause, not yet tested, rules that out. A project's .gitignore hid files from four migrations' commits; in six runs with the check, none was hidden.Stacked on #51 (tracking issue: #35).
pytest tests/: 186 passed (132 onmain). Lab proposal: P23, P27.🤖 Generated with Claude Code