Repository navigation
Pin the reference configuration, record its command, and require parity for the default invocation - #47
Open
anakanemison wants to merge 2 commits into
Conversation
anakanemison
force-pushed
the
mschaller/u12-reference-configuration
branch
from
October 26, 2026 13:30
a0b6b20 to
23ea713
Compare
anakanemison
force-pushed
the
mschaller/u12-reference-configuration
branch
from
November 2, 2026 17:51
23ea713 to
4f25e37
Compare
bazel_args records the extra args aquery runs with to mirror the real build. Nothing said where settings the migration itself introduces should live, so parity could be established for a named --config that a plain `bazel build //...` never applies: the claim then holds for a command no user runs. Measured in any2bazel-lab: one pono migration put the reference's Release settings behind --config=release, recorded it in bazel_args, and converged. A plain `bazel build //...` produced an assertion-enabled binary that crashes on an input where the reference prints an error and exits. The four other runs on the same project put the settings in .bazelrc's build lines and were equivalent. Adds one bullet to the "invoke aquery the way the project is built" list: settings the migration adds go in `.bazelrc` build lines, and parity must hold for the project's documented build command, which is a plain `bazel build //...` unless the project documents flags; only those documented flags go in bazel_args. The bazel_args field description now says the same, and that the example config (a project config plus the top-level -fno-exceptions BoringSSL expects) is that kind of entry. In the lab's later pono and lensfun runs with this change, bazel_args was empty and the settings were in .bazelrc build lines. Lab-Proposal: P10 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Nothing in the procedure said which CMake configuration the reference should use, or that choosing one is a decision. An agent can turn options off before extracting the reference and then reach parity with that narrower build; both sides of the comparison move together, so no error count shows it. Measured in any2bazel-lab: a MySQL Server migration configured the reference with Router, NDB, the unit tests and the distribution's package compiler flags all turned off, recorded only in a comment in any2bazel.json. Against the project's default configuration the result showed 4,928 differences; against its own, 13. The reason given for the package flags was defensible; the problem was that the choice was silent. Agents redefined the reference with and without the skill, on small projects as well as large. "Extract the reference model" now says to configure with the project's documented default options, to surface any change as a scope decision, and to record the exact command in any2bazel.json as reference.configure_command (added to the example and the field list as record-only; config.load ignores it, like any unknown key) and never change it after the first diff. The optional test step reconfigures with tests on; that command is recorded separately, as reference.test_configure_command, so it does not count as changing the reference. In the lab's later runs with this change, every migrating agent recorded its configure command. Lab-Proposal: P13 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
anakanemison
force-pushed
the
mschaller/u12-reference-configuration
branch
from
November 9, 2026 18:45
4f25e37 to
c438a83
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A migration could reach parity with a build nobody runs, in two ways: by turning options off in the reference before extracting it, or by putting its own settings behind a named
--configthat a plainbazel build //...never applies. SKILL.md now says to configure the reference with the project's documented defaults, surface any change as a scope decision, record the command asreference.configure_command(record-only) and not change it after the first diff. Settings the migration adds go in.bazelrcbuildlines, and only the project's documented flags go inbazel_args.In any2bazel-lab, a MySQL Server migration narrowed its reference silently (13 differences against its own configuration, 4,928 against the default), and a pono migration converged behind
--config=releasewhile a plain build produced a binary that crashes. In later runs, every migrating agent recorded its configure command, and the pono and lensfun agents kept their settings in.bazelrc.Stacked on #46 (tracking issue: #35).
pytest tests/: 186 passed (132 onmain). Lab proposal: P10, P13.🤖 Generated with Claude Code