Skip to content

Compare queue insertion-search strategies in CI - #1602

Open
Wei Wu (wuw92) wants to merge 5 commits into
wuw92-beam-queue-benchmarkfrom
wuw92-queue-insertion-search
Open

Wei Wu (wuw92) wants to merge 5 commits into
wuw92-beam-queue-benchmarkfrom
wuw92-queue-insertion-search

Conversation

@wuw92

@wuw92 Wei Wu (wuw92) commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor
  • Does this PR have a descriptive title that could go in our release notes?
  • Does this PR add any new dependencies?
  • Does this PR modify any existing APIs?
  • Is the change to the API backwards compatible?
  • Should this result in any changes to our documentation, either updating existing docs or adding new ones?

Reference Issues/PRs

Stacked on #1600 against wuw92-beam-queue-benchmark, fixed base a58bd09808915b3f457008217cb369a201677fd0. Related to #1590. Neither existing PR is modified or replaced.

What does this implement/fix? Briefly explain your changes.

Compare the existing SIMD insertion-point scan with binary search and filled-length hybrids using focused lookups and complete queue updates. The latest narrow outlining attempt did not eliminate the small-L and L=1025 regressions. Global optimization is unfinished; this candidate is not recommended as a general replacement. The checked-in source is retained as an exactly measured experiment, not a successful fix.

Single-item insertion keeps the original SIMD/scalar lookup through filled length 1024, then calls an outlined 16-element linear-prefix/binary lookup. Batch/remove retain their original lookup. No dependencies or public configuration/API are added. Regression tests preserve first-equal ordering, NaN filtering ownership and NaN remove, ±0/±infinity, unique-ID assumptions, visited/cursor state, partial fill, capacity, and resizable queues. ARM large insertion is measured and tested; its small insertion and batch/remove paths retain the original scalar scan.

The lookup benchmark consumes every returned index. Full updates reuse insert and batch_insert, exclude fixture generation/reset from timing, bound repetitions with an untimed work model, and record actual blocks. The original neighbor-queue.json example is unchanged. The read-only-permissions workflow supports this stacked PR base. Its latest run compares the exact original scan, previous guarded hybrid, and outlined candidate. Earlier binary/shared-hybrid/insertion-only variants remain reproducible through the driver and earlier source commits.

Any other comments?

Latest result: round 4 CI passes correctness, build, and evidence checks on all three platforms at 233ce5f63191b6ebf9a9f85d4a8f713e568215dd. Passing CI is not passing performance acceptance. Large-L insertion gains remain repeatable, but the outlining attempt fails its regression criterion. No merge or auto-merge is enabled.

Narrow diagnosis and causal limit

The actual Linux ELF binaries establish a code-shape difference: original insert is 801 bytes, previous guarded insert is 1307 bytes, and outlined insert is 784 bytes plus a separate 459-byte lookup. The previous guarded version embeds another Vec-length guard and SIMD fallback; two hot-loop conditional jumps change from short to near encodings. After excluding linked RIP-relative addresses, both measurement-harness specializations and batch_insert have identical instructions across all three variants.

Outlining also changes register allocation: the target is stored at function entry and reloaded/broadcast from the stack on the small-L path; capacity moves into a callee-saved register. The original uses the live target register for those operations. Therefore reducing function size does not preserve the original insertion machine-code hot path. The smaller function still loses: code-size growth alone is not a sufficient explanation. The precise microarchitectural cause of the earlier 22–24% loss remains unassigned; no hardware-counter attribution was performed. We do not dismiss these regressions as noise or CPU differences.

Source/hash/AST checks confirm that the benchmark, reset, work model, release profiles/flags, toolchain, cases, measurement orchestration, and summary rules are unchanged. Every previous control's queue source and every generated input match round 3 after appropriate newline normalization. The checked-in candidate and helper match the measured snapshots on all three platforms. Local screening used the same 47 cases and both seeds, scoped to 128 fixtures, and did not establish regression elimination before CI.

Targeted complete-insert results. N=8, full queues, uniform/shuffled candidates, 128 fixtures. Speedup is original scan / candidate. Seeds are shown separately. “Inconclusive” does not establish equivalence. The full tables for both 8 and 128 fixtures, all methods/cases, nine raw rounds, and all three process medians are retained in each artifact's summary.md, comparisons.json, process-medians.json, and raw JSON.

Runner L Seed Previous guarded speedup/screen Outlined speedup/screen
Linux x86 / EPYC 9V74 40 20261114 0.958x inconclusive 0.866x loss
Linux x86 / EPYC 9V74 40 9372461 0.919x inconclusive 0.931x inconclusive
Linux x86 / EPYC 9V74 100 20261114 0.981x inconclusive 0.920x loss
Linux x86 / EPYC 9V74 100 9372461 0.980x inconclusive 0.961x inconclusive
Linux x86 / EPYC 9V74 1025 20261114 0.945x loss 0.935x loss
Linux x86 / EPYC 9V74 1025 9372461 0.917x loss 0.907x loss
Windows / EPYC 7763 40 20261114 0.989x inconclusive 1.013x inconclusive
Windows / EPYC 7763 40 9372461 0.990x inconclusive 0.992x inconclusive
Windows / EPYC 7763 100 20261114 0.998x inconclusive 0.969x inconclusive
Windows / EPYC 7763 100 9372461 1.002x inconclusive 1.008x inconclusive
Windows / EPYC 7763 1025 20261114 0.972x loss 0.961x loss
Windows / EPYC 7763 1025 9372461 0.981x inconclusive 0.977x inconclusive
ARM / Neoverse-N2 40 20261114 0.998x inconclusive 1.009x inconclusive
ARM / Neoverse-N2 40 9372461 1.004x inconclusive 1.007x inconclusive
ARM / Neoverse-N2 100 20261114 0.991x loss 0.983x loss
ARM / Neoverse-N2 100 9372461 0.994x inconclusive 0.990x inconclusive
ARM / Neoverse-N2 1025 20261114 2.056x gain 2.058x gain
ARM / Neoverse-N2 1025 9372461 2.075x gain 2.087x gain

Per-process ns/update for the two Linux regression targets, same configuration as above. Each value is the median of nine rounds within one independent process. No favorable repeat is selected.

L Seed Original process medians Previous guarded process medians Outlined process medians
100 20261114 193.19 / 193.12 / 191.10 203.88 / 196.89 / 194.29 209.88 / 220.91 / 209.80
100 9372461 195.95 / 199.08 / 200.92 212.87 / 203.19 / 198.59 207.18 / 214.24 / 202.39
1025 20261114 962.43 / 935.60 / 961.57 1017.74 / 1013.05 / 1018.92 1023.23 / 1028.47 / 1038.41
1025 9372461 954.76 / 926.21 / 900.15 1015.09 / 996.39 / 1010.55 1013.98 / 1021.50 / 1025.26

Large-L context from the same run, N=2000, uniform/shuffled/full queue. Ranges span the four two-seed × 8/128-fixture configurations; they are not confidence intervals or pooled workload scores. All 24 large-L N=2000 insertion configurations pass the gain screen. Some batch configurations have small screen-passing losses; neither batch equivalence nor batch acceleration is claimed.

Runner L Outlined insert speedup Outlined batch_insert speedup
Linux x86 / EPYC 9V74 2000 1.145–1.159x 0.961–1.027x
Linux x86 / EPYC 9V74 8192 1.283–1.290x 0.976–1.000x
Windows / EPYC 7763 2000 1.101–1.107x 1.001–1.025x
Windows / EPYC 7763 8192 1.201–1.209x 0.999–1.017x
ARM / Neoverse-N2 2000 2.746–2.757x 0.992–1.010x
ARM / Neoverse-N2 8192 2.736–2.775x 0.984–1.003x

CI and source history

Round Source Outcome and change Evidence
Setup 87d7341e4d93834933ff7637f3afae1df115796c Failed before timing: missing rustfmt; component installation fixed Run
1 4a3669cb1c53c911c40af73b9571c4bbc41305c1 Original/binary/shared hybrid512; 144 raw processes complete; summary parser failed on input envelope and was fixed; all raw data re-audited Run and raw artifacts
2 65180f2a8f20df683c602c7499771855b895b4eb Insertion-only hybrid512; 180 processes, all jobs pass; main batch regressions largely disappear, but x86 L=513/1000 remains slower Run
3 6cb3b23a6333be818800aa9004eee2bbcfcbc11f Guard above 1024; add only 1023/1024/1025 boundaries; 216 processes, all jobs pass; large-L gains but Linux L100/N8/128 fixtures is 0.760x/0.785x across seeds, L1025 is 0.949x for seed9372461 Run and complete summaries
4 233ce5f63191b6ebf9a9f85d4a8f713e568215dd One outlined-lookup attempt, unchanged threshold/cases; contemporaneous original and previous guarded controls; 144 processes, all jobs pass, performance acceptance fails Latest run and complete summaries

Method and applicability. The final two rounds have the same 47 complete-update cases, L=20/40/100/200/500/1000/2000/8192 plus 511/512/513 and 1023/1024/1025 boundaries, selected N=1/2/4/8/2000, head/middle/tail/reject, sorted input, and partial fills. Seeds are 20261114 and 9372461; populations are 8 and 128 fixtures. Each configuration has three independent processes and nine recorded rounds, with warmup excluded. Executable order rotates between processes; method/kernel order rotates within processes. Compilation finishes before measurement. Linux children are pinned to one allowed logical CPU; Windows inherits runner affinity. Exact commands, actual affinity, OS, CPU, rustc 1.97.1 and release flags are retained in artifacts.

The repeatability screen requires all paired process ratios to agree in direction and the median effect to exceed the larger relative process range. It is not a confidence interval. Unfavorable or noisy processes remain visible. Round 3 Linux used Xeon Platinum 8573C; round 4 used EPYC 9V74. The earlier 22–24% magnitude does not reproduce in round 4's old guarded control, but boundary losses do reproduce. Cross-round changes are not attributed to source changes, and no regression is dismissed on hardware grounds. These are warm-fixture microbenchmarks, not end-to-end ANN workloads. A sorted batch minimum differs from a random single-lookup probe; the batch top-L partition loop is unchanged.

Latest raw artifacts: Linux x86, ARM, Windows. The run page also exposes all artifacts and step summaries. GitHub authentication may be required; server retention is 30 days, and downloaded copies are preserved. Each artifact includes exact sources/executables and SHA-256, configs, verbose build/test logs, inputs, commands/affinity, raw JSON, all per-process medians and comparison tables. Previous-run evidence remains available through the history links above; no recorded timing was rewritten.

Reproduce the latest narrow comparison from the fixed source, with no overlapping builds or benchmarks:

python .github/scripts/benchmark_queue_search.py build --base a58bd09808915b3f457008217cb369a201677fd0 --output benchmark-results --variants simd insert_guarded insert_outlined
python .github/scripts/benchmark_queue_search.py run --output benchmark-results
python .github/scripts/benchmark_queue_search.py summarize --output benchmark-results

Validation:

  • cargo test --locked --release -p diskann --lib neighbor::queue --features experimental_diversity_search: passed, 41 tests for the checked-in candidate; repeated for every measured variant.
  • cargo test --locked --release -p diskann-benchmark neighbor_queue: passed, 9 tests.
  • cargo clippy --locked --release -p diskann -p diskann-benchmark --all-targets -- -D warnings: passed.
  • cargo fmt --all -- --check: passed.
  • python .github/scripts/test_benchmark_queue_search.py: passed, 4 audit/comparison tests.
  • python .github/scripts/check_license_headers.py check: passed after adding the source files; no new source files were added by the narrow attempt.
  • git diff --check: passed.

Latest CI repeats validation on all three platforms. Independent artifact audit passes all 684 formal processes across four rounds: complete inputs/cases/process identities, nine rounds per method, identical actual block/work counts, finite positive timings, and actual binary/source/input SHA-256. Full workspace tests and end-to-end ANN benchmarks were not run. The bounded narrow diagnosis stops without an adoption claim or an asserted root cause that the evidence cannot establish.

Wei Wu (wuw92) and others added 5 commits November 16, 2026 17:45
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Apply the measured insertion-only hybrid above filled length 1024 and retain baseline, binary, and earlier hybrid comparisons in CI.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep small-queue insertion on the original inline lookup path and compare the outlined candidate with both the original scan and prior guarded implementation. Preserve the existing cases, seeds, timing harness, and release flags.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant