Repository navigation
feat(compaction): report when a pinned reader stopped the pass - #52
Merged
farhan-syah merged 1 commit intoNov 5, 2026
Merged
Conversation
mkhairi
force-pushed
the
feat/compaction-declined-readers
branch
from
November 5, 2026 05:44
1effb13 to
26f0374
Compare
mkhairi
marked this pull request as ready for review
November 5, 2026 05:45
farhan-syah
requested changes
Nov 5, 2026
farhan-syah
left a comment
Member
There was a problem hiding this comment.
Changes requested. No blockers: the flag is accurate, because the reader check runs under the admission gate.
Rebase onto current main first:
Dependency audit:mainnow has tokio 1.54.1 (6ec8e0b), andcargo denypasses there. The rebase reruns this job on your branch.CHANGELOG.md: this PR and #51 both add### Addedunder[Unreleased], so they conflict. The PR that merges second needs one more rebase.
Then make these three changes:
Db::compact_stepdoc,src/txn/db/misc.rs:86-87: it says to loop untilprogress.more_work == falseto compact fully. A decline now ends that loop with nothing compacted. Point this doc atdeclined_readers_pinned, as you did forDb::compact_now.compact_stepdoc,src/compaction/step.rs:24: it says compaction runs to completion. Add the decline case here too.src/compaction/full.rs:71-72: see the inline comment.
The two doc lines are outside your diff, so they are listed here. The rest of the Db::compact_step doc is stale too, but it is out of scope for this PR.
Checked:
- Both PRs merged onto
main:cargo clippy --all-targets --all-features -- -D warningsis clean, andcargo nextest run --all-featurespasses. - Your two mutants,
compact_stepdropping the flag and the flag alwaystrue:compact_reports_a_pinned_reader_declining_the_passfails on both, as you state.
The issue-first rule in CONTRIBUTING.md is waived for this PR. You do not need to open an issue.
mkhairi
force-pushed
the
feat/compaction-declined-readers
branch
from
November 5, 2026 08:52
26f0374 to
31f2128
Compare
Member
Author
|
Rebased onto
I left the rest of the |
`compact_now` refuses while any reader pins the page range, and returned `Ok(CompactStats::default())` to say so. That is byte-identical to the result of a pass that ran and found nothing to reclaim: every count zero, no error. A caller deciding whether to retry, or an operator reading "0 pages reclaimed" off a maintenance endpoint, could not tell "the store is already dense" from "compaction never ran". `compact_step` wraps `compact_now` and reported the same zeros with `more_work == false`. `CompactStats` and `CompactProgress` gain `declined_readers_pinned`, and `compact_step` copies it through. `more_work` stays `false` on a decline: another call does nothing until the readers drop, so a caller looping on it would spin. Both structs are `#[non_exhaustive]`, so this is additive for downstream matchers. The `compact_now` doc no longer implies the pass runs and only skips truncation under a reader. The `compact_step` docs now name the decline too, since a `more_work` loop can end on one with nothing compacted. The test builds a store with garbage, then checks three results: under a live read handle `compact_now` and `compact_step` both report the decline with zero counts, and after the reader drops the pass clears the flag and reclaims pages. The reclaim proves the zeros under the reader came from the decline, not from an already-dense store.
mkhairi
force-pushed
the
feat/compaction-declined-readers
branch
from
November 5, 2026 09:41
31f2128 to
3d0ecb6
Compare
farhan-syah
approved these changes
Nov 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
compact_nowdoes not run while a registered reader pins the page range. In that case it returnsOk(CompactStats::default()).That result is identical to a pass that ran and found nothing to reclaim: every count is zero and there is no error. A caller deciding whether to retry cannot tell "the store is already dense" from "compaction never ran". An operator reading "0 pages reclaimed" has the same gap.
compact_stepwrapscompact_now, so it has the same gap: it reports zero counts withmore_work == false.Change
CompactStatsandCompactProgressgaindeclined_readers_pinned: bool.compact_nowsets it on the early return taken when readers are registered. Every count stays at zero, as before.compact_stepcopies it into its progress report.more_workstaysfalseon a decline. Another call does nothing until the readers drop, so a caller looping onmore_workwould spin.Db::compact_nowdoc said the pass "truncatesmain.dbif no reader pins the old high-water range", which implies the repack still runs under a reader. It now says the pass does not run while a reader is registered, and points at the flag.Db::compact_stepandcompaction::compact_stepdocs name the decline too. A loop untilmore_work == falsecan end on a decline with nothing compacted.The reader check is the only path where compaction returns
Okwithout doing work.ensure_usableandrequire_modereturnErr. The empty-free-list and below-5% segment skips are real "nothing to reclaim" results.Compatibility
Both structs are
#[non_exhaustive], so the new field is additive for downstream code. Existing callers see the same counts and the samemore_workas before. No on-disk format change. No durability-ordering change. No new dependency.Testing
compact_reports_a_pinned_reader_declining_the_passintests/compaction_basic.rs, next tocompact_respects_reader_pins, builds a store with garbage (100 keys, half deleted). Then:compact_nowreturnsdeclined_readers_pinned == truewith zeromain_db_pages_reclaimedandbytes_truncatedcompact_stepreturnsdeclined_readers_pinned == true,more_work == falseand zeropages_relocatedcompact_nowreturnsdeclined_readers_pinned == falseand reclaims pagesThe reclaim in the last step proves the earlier zeros came from the decline, not from a dense store. Two mutants were checked: the test fails when
compact_stepdrops the flag, and when the flag is alwaystrue.Gates run locally on stable, one at a time:
cargo fmt --all --check,cargo clippy --all-targets --all-features -- -D warnings,cargo nextest run --all-features,cargo test --doc --all-features,cargo doc --no-deps --all-featureswithRUSTDOCFLAGS="-D warnings", andcargo check -p pagedb --libforwasm32-unknown-unknown --features opfsandwasm32-wasip1.CHANGELOG.mdhas an entry under[Unreleased]→Added.