Repository navigation
Conversation
…er pool The write-path scaling benchmarks use 150-byte values, which stay inline. Inline rows never put dirty pages in the pager cache before commit, so none of them reaches the case where one transaction's dirty set outgrows the buffer pool. `scaling_overflow_into_large_tree` commits 50,000 small rows, so the tree does not fit the default 1024-page pool, then times one transaction that puts 1,000 to 4,000 rows of 8 KiB values and then aborts. Each value writes an overflow chain into the cache as dirty pages, and each `put` reads leaves of the existing tree back as clean pages. Per-row cost should stay flat across the row counts.
farhan-syah
left a comment
There was a problem hiding this comment.
Changes requested. The eviction fix is correct: the quadratic cost is gone, and I found no bug in the list handling.
Make three changes:
visitedreset: move it out of this PR. See the inline comment on line 418.- New
.expect()calls: replace both. See the inline comments on lines 272 and 290. src/pager/core.rs:885: the comment says the eviction scan skips dirty entries. Dirty entries are now off the list, so the scan never reaches them. Update the comment. This line is outside your diff, so it is listed here.
Checked:
scaling_overflow_into_large_tree@4000: 62.20 Ginstructions:uonmain, 2.885 G on this PR, counted over the whole bench binary.scaling_overflow_into_large_tree@1000: 1.9455 G on both.scaling_file_committed@100000: about +0.1%, within your +0.27%.dirty_entries_stay_off_the_eviction_list_until_clean: fails on themainimplementation, as you state.clearing_a_file_drops_unlinked_dirty_entries: fails without thelinkedguard, as you state.- Flush, abort, re-dirty, pin, and
clear_filepaths: traced against the new list handling, with no bug found. cargo clippy --all-targets --all-features -- -D warnings: clean.cargo nextest run --all-features: 864 passed, 10 skipped.
| if let Some(node) = self.slab[idx].as_mut() { | ||
| node.visited = false; | ||
| } |
There was a problem hiding this comment.
Move this reset to its own PR, with hit-rate data. The quadratic fix does not need it: scaling_overflow_into_large_tree@4000 runs 2.885 G instructions:u with or without the reset.
The reset changes eviction behavior, and this PR reports no hit rates. I measured cache misses with a 256-page pool and 20,000 rows of 150 B. Each of the 200 rounds runs 500 skewed reads and one write transaction:
| Write transaction | main |
This PR | This PR without the reset |
|---|---|---|---|
| Commit, 20 rows | 17,857 | 20,562 (+15%) | 17,905 |
| Commit, 1000 rows | 281,995 | 263,377 (-6.6%) | 280,390 |
| Abort, 1000 rows | 39,552 | 28,244 (-29%) | 38,366 |
Small commits get worse, and large commits and aborts get better. This is one synthetic workload. The pool also grew past its budget in these runs, to between 1152 and 1501 pages. That happens on main too and is a separate bug.
A flush lookup that skips visited is no substitute. It gave 20,379 misses on small commits and lost most of the abort gain, with 38,733 misses.
Remove these parts with the reset:
if let Some(node): this guard tolerates a dead slot, andlink_as_headthen panics on the same slot.cleared_entry_rejoins_unvisited: this test covers only the reset.- The PR description bullet about the reset.
| fn link_as_head(&mut self, idx: usize) { | ||
| let old_head = self.head; | ||
| { | ||
| let node = self.slab[idx].as_mut().expect("linked node alive"); |
There was a problem hiding this comment.
Remove this .expect() and the one at line 290. CONTRIBUTING.md:86 bans .expect() in library code, and the existing calls in this file do not make new ones acceptable. Those existing calls, including the one moved to line 279, are out of scope for this PR.
You are right that a let ... else return hides list corruption. Return a typed PagedbError instead, and propagate it through insert_and_index, insert, mark_dirty, and clear_dirty.
Production callers:
src/pager/core.rs:903,:961-962,:1131,:1371: these functions already returnResult, so use?.discard_dirty_main,src/pager/core.rs:923: this function returns nothing. Log the error and keep discarding the remaining pages.
Do not use PagedbError::corruption. Its message tells the operator to quarantine the store, and a dead slot is in-memory state.
| /// Take the node at `idx` off the list, moving the hand off it first. | ||
| fn unlink_from_list(&mut self, idx: usize) { | ||
| let (prev, next) = { | ||
| let node = self.slab[idx].as_mut().expect("unlinked node alive"); |
There was a problem hiding this comment.
Remove this .expect() and return the same typed error as link_as_head. See the comment on line 272.
Dirty pages are never evicted, but they stayed on the SIEVE list, and the hand checked each one against the dirty set as it walked past. `evict_one` already returns early when nothing is evictable, so a transaction that only dirties pages stays linear. The slow case is a large dirty set with a few clean pages arriving: every `put` that reads a leaf back into the cache needs a slot, and the hand walks nearly the whole dirty set to find one. Inserts then cost time quadratic in the pages the transaction wrote. A page now leaves the list when it is marked dirty and rejoins as the newest entry when it is clean again. The hand only visits clean or pinned pages. A debug assertion in `evict_one` checks that an entry is on the list exactly when it is clean. Linking and unlinking an entry return the new `PagedbError::PageCacheInconsistent` when the index points at an empty slot, in place of a panic. It is not a corruption error: nothing on disk is known to be wrong. `insert`, `insert_and_index`, `mark_dirty` and `clear_dirty` propagate it, and the pager's callers return it with `?`. In `mark_dirty` and `clear_dirty` the list update runs before the dirty set and the evictable count change, so an error leaves those two as they were. `discard_dirty_main` (abort) and `clear_file` cannot fail, so they log the error and go on with the remaining pages. The page that failed stays dirty in `discard_dirty_main`. In `clear_file` the failed slot is already empty, so it is not freed a second time. `clear_dirty` now returns early for a key that was not dirty. Before, it incremented `evictable` for any present evictable key, dirty or not. Now the guard also prevents linking a node twice. `discard_dirty_main` (abort) cleans every dirty page through `clear_dirty`, so aborted pages also rejoin as the newest entries. Before, they kept the list position they got when the transaction inserted them, which was also near the newest end. `scaling_overflow_into_large_tree`, median of 3 samples, two interleaved rounds on one machine, measured with an earlier revision of this change that also reset `visited` on rejoin. The reset is not part of this change and does not move the instruction counts below: rows before after 1000 12.3 / 11.8 ms 12.2 / 11.5 ms 2000 760 / 768 ms 27.7 / 28.5 ms 4000 2.83 / 2.80 s 55.1 / 54.5 ms User-space instructions for the whole bench binary (`perf stat -e instructions:u`, `-n 3`), interleaved runs on this change: bench before after scaling_overflow_into_large_tree@4000 62.32 G 3.015 G (2 runs each) scaling_overflow_into_large_tree@1000 2.060 G 2.062 G (2 runs each) scaling_file_committed@100000 3.922 G 3.937 G (median of 6) Paths that never build a large dirty set pay one unlink and one relink per written page. On `scaling_file_committed@100000` that is +0.38%, and the run ranges overlap: 3.921-3.950 G before, 3.930-3.947 G after. Wall time on this shared host varies more than that between runs of the same build.
8608469 to
10ebac0
Compare
|
Made the three changes, amended into the
Instruction counts on the final code ( All local gates pass on stable. |
|
Thanks @mkhairi. The dirty-page list change and |
Problem
Dirty pages are never evicted, but they stay on the SIEVE list. The eviction hand checks each one against the dirty set as it walks past.
evict_onereturns early when nothing is evictable, so a transaction that only dirties pages stays linear. The slow case is a large dirty set with a few clean pages arriving.putwrites overflow chains into the cache as dirty pages and does not flush them before commit. Eachputalso descends the tree, and a leaf that missed the cache comes back as a clean page that needs a slot. To free one, the hand walks nearly the whole dirty set. One transaction's inserts then cost time quadratic in the pages it wrote.Change
Two commits:
test(bench):scaling_overflow_into_large_treeinbenches/write_path.rs. It commits 50,000 small rows, so the tree does not fit the default 1024-page pool. Then it times one transaction that puts 1,000 to 4,000 rows of 8 KiB values and aborts. The existing scaling benchmarks use 150-byte inline values and never reach this path.perf(pager): a page leaves the SIEVE list when it is marked dirty and rejoins as the newest entry when it is clean again. The hand only visits clean or pinned pages.clear_dirtynow returns early for a key that was not dirty. Before, it incrementedevictablefor any present evictable key, dirty or not. The guard also prevents linking a node twice.evict_onechecks that an entry is on the list exactly when it is clean, next to the existingevictablecount check.PagedbError::PageCacheInconsistentwhen the index points at an empty slot, in place of a panic. It is not a corruption error, because nothing on disk is known to be wrong.insert,insert_and_index,mark_dirtyandclear_dirtypropagate it, and the pager's callers return it with?. Inmark_dirtyandclear_dirty, the list update runs before the dirty set and the evictable count change, so an error leaves those two as they were.discard_dirty_main(abort) andclear_filecannot fail, so they log the error and go on with the remaining pages. Indiscard_dirty_mainthe page that failed stays in the dirty set. Inclear_filethe failed slot is already empty, so it is not freed a second time.Results
cargo bench --bench write_path -- --group btree/write-path/scaling. Median of 3 samples, two interleaved rounds (before, after, before, after) on one machine. These wall times come from an earlier revision that also resetvisitedon rejoin. That reset is not part of this PR, and it does not move the instruction counts below.The "before" runs use this PR's benchmark file on
main.User-space instructions for the whole bench binary (
perf stat -e instructions:u,-n 3), interleaved runs on this PR's final code:scaling_overflow_into_large_tree@4000(2 runs each)scaling_overflow_into_large_tree@1000(2 runs each)scaling_file_committed@100000(median of 6)Paths that never build a large dirty set pay one unlink and one relink per written page. On
scaling_file_committed@100000the run ranges overlap: 3.921-3.950 G before, 3.930-3.947 G after.The host is shared, with a load average of 4-6 during the runs. Wall time for one build varies more between runs than the difference between builds. For example,
scaling_uncommitted@20000on the same build ranged from 26 to 45 ms. The instruction counts are the stable comparison.Tradeoff
A page that becomes clean rejoins as the newest entry instead of keeping its old list position.
discard_dirty_maincleans every dirty page throughclear_dirty, so aborted pages rejoin as the newest entries too. They will never be flushed. Before, they kept the position they got when the transaction inserted them, which was also near the newest end.Compatibility
One additive public API change:
PagedbErrorgainsPageCacheInconsistent { operation }. The enum is#[non_exhaustive]. No on-disk format change. No durability-ordering change: flush still writes the same dirty set, andclear_dirtyonly moves the page within the cache. No new dependency.Testing
Four unit tests in
src/pager/cache.rs:dirty_entries_stay_off_the_eviction_list_until_clean: eviction takes the clean entry and not a dirty one. A cleaned entry rejoins as the newest. A key marked dirty before it is cached is inserted off the list. This test fails onmain.clearing_a_file_drops_unlinked_dirty_entries:clear_file(Main)removes a dirty, unlinked entry while a segment entry survives on the list and is evicted in its turn. The test fails ifunlink_from_listdrops itslinkedguard.dead_slot_is_an_error: an index entry whose slot is empty makesmark_dirty(unlink) andclear_dirty(link) returnPageCacheInconsistent. Each call also leaves the dirty set and the evictable count as they were. The test fails if either list operation goes back to.expect().clearing_past_a_freed_slot_does_not_free_it_twice:clear_filepast a freed slot that a live index entry still names, then two inserts get two different slots. The test fails if the clear frees the slot again.Gates run locally on stable, one at a time:
cargo fmt --all --check,cargo clippy --all-targets --all-features -- -D warnings,cargo nextest run --all-features,cargo test --doc --all-features,cargo doc --no-deps --all-featureswithRUSTDOCFLAGS="-D warnings", andcargo check -p pagedb --libforwasm32-unknown-unknown --features opfsandwasm32-wasip1.CHANGELOG.mdhas entries under[Unreleased]:Changedfor the eviction list,Addedfor the error variant.