Skip to content

feat(stats): expose oldest reader commit id and age - #51

Merged
farhan-syah merged 1 commit into
NodeDB-Lab:mainfrom
mkhairi:feat/stats-oldest-reader
Nov 5, 2026
Merged

farhan-syah merged 1 commit into
NodeDB-Lab:mainfrom
mkhairi:feat/stats-oldest-reader

Conversation

@mkhairi

@mkhairi mkhairi commented Nov 5, 2026 •

Copy link
Copy Markdown
Member

Problem

A long-lived read transaction pins the pages freed after its snapshot. Free-page reuse stalls behind it, and the file grows.

Db::stats() reports tracked_readers only. That says a reader exists. It does not say which commit the reader pins or how long it has held it. An operator who sees the file grow cannot tell a harmless short read from the one blocking reuse.

Change

DbStats gains three fields:

  • oldest_reader_commit_id: Option<u64> — the smallest commit id any registered reader pins.
  • oldest_reader_age_ms: Option<u64> — the age of the longest-lived reader.
  • tracked_readers_non_abortable: u32 — the subset of tracked_readers exempt from reader-stall abort.

Both Option fields are None when no reader is registered. Each tracked reader records its start time through a new clock::Instant re-export. It is web_time::Instant on wasm32-unknown-unknown, where std::time::Instant::now() panics, and std::time::Instant everywhere else. register_read reads the clock before it takes the tracked-readers lock. It uses the same cfg as clock::unix_seconds, so wasm32-wasip1 reports the age too.

The longest-lived reader is not always the one pinning the oldest commit: begin_read_at can open a new reader on an older commit. The oldest_reader_age_ms doc comment says so.

Readers are not the only bound on free-page reuse. Retained commit history holds back pages freed after the oldest retained commit. The oldest_reader_commit_id doc says so, so None is not read as "nothing blocks reuse".

stats() reads all reader figures in one pass under a single lock (reader_census). The count, the oldest commit and the age therefore describe the same set of readers.

Compatibility

DbStats is #[non_exhaustive], so the new fields are additive for downstream code. No on-disk format change. No durability-ordering change. No new dependency: web-time is already a dependency of the browser target.

Testing

Two tests in tests/stats_basic.rs.

stats_reports_reader_census covers:

  • the idle case: both Option fields are None
  • a pinned reader plus a non-abortable reader opened at least 10 ms later: the oldest commit and the age are both the older reader's, and the non-abortable count is 1. With .max() in place of .min() in the census, the age assertion fails.
  • age growth across a sleep
  • the return to None after both readers drop

stats_reader_census_fields_can_describe_different_readers opens a reader on the latest commit, waits 10 ms, then opens a reader with begin_read_at on an older commit. oldest_reader_commit_id is the second reader's commit and oldest_reader_age_ms is the first reader's age. Taking the age from the oldest-commit reader (min_by_key(|r| r.commit_id.value())) fails the age assertion.

Gates run locally on stable, one at a time: cargo fmt --all --check, cargo clippy --all-targets --all-features -- -D warnings, cargo nextest run --all-features, cargo test --doc --all-features, cargo doc --no-deps --all-features with RUSTDOCFLAGS="-D warnings", and cargo check -p pagedb --lib for wasm32-unknown-unknown --features opfs and wasm32-wasip1.

CHANGELOG.md has an entry under [Unreleased] → Added.

@mkhairi
mkhairi marked this pull request as ready for review November 5, 2026 05:09
@mkhairi
mkhairi force-pushed the feat/stats-oldest-reader branch from c0bf13f to cddc3c2 Compare November 5, 2026 05:44

@farhan-syah farhan-syah left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes requested. No blockers: the census logic is correct.

Rebase onto current main first:

  • Dependency audit: main now has tokio 1.54.1 (6ec8e0b), and cargo deny passes there. The rebase reruns this job on your branch.
  • CHANGELOG.md: this PR and #52 both add ### Added under [Unreleased], so they conflict. The PR that merges second needs one more rebase.

Then make the four inline changes.

Checked:

  • Both PRs merged onto main: cargo clippy --all-targets --all-features -- -D warnings is clean, and cargo nextest run --all-features passes.
  • .max() mutant in reader_census: stats_reports_reader_census fails, as you state.

The issue-first rule in CONTRIBUTING.md is waived for this PR. You do not need to open an issue.

Comment thread src/observability/mod.rs Resolved
Comment thread src/observability/mod.rs Resolved
Comment thread src/txn/db/reader.rs Resolved
Comment thread tests/stats_basic.rs Resolved
A long-lived read transaction pins the pages freed after its snapshot,
so free-page reuse stalls and the file grows. `Db::stats()` reported
only `tracked_readers`, which says a reader exists but not which commit
it pins or how long it has held it. An operator could not tell a
harmless short read from the one blocking reuse.

`DbStats` gains three fields:

- `oldest_reader_commit_id`: the smallest pinned commit id.
- `oldest_reader_age_ms`: age of the longest-lived reader.
- `tracked_readers_non_abortable`: the subset of `tracked_readers` exempt
  from reader-stall abort.

The first two are `None` when no reader is registered. Each tracked
reader records its start time through `clock::Instant`, which is
`web_time::Instant` on the browser target, where `std::time::Instant::now()`
panics, and `std::time::Instant` everywhere else. The clock is read
before the tracked-readers lock is taken.

The longest-lived reader is not always the one pinning the oldest
commit: `begin_read_at` can open a new reader on an older commit. The
`oldest_reader_commit_id` doc also notes that retained commit history
bounds reuse, so `None` does not mean nothing blocks it.

`stats()` reads all reader figures in one pass under a single lock via
`reader_census`, so the count, oldest commit and age describe the same
set of readers.

The test checks the idle case, a pinned reader with a newer
non-abortable one (the oldest commit and the age must both be the older
reader's), age growth across a sleep, and a return to `None` after both
drop. A second test opens a reader with `begin_read_at` on an older
commit after a longer-lived one, and checks that the oldest commit and
the age then come from different readers.
@mkhairi
mkhairi force-pushed the feat/stats-oldest-reader branch from cddc3c2 to d746a86 Compare November 5, 2026 08:52
@mkhairi

mkhairi commented Nov 5, 2026

Copy link
Copy Markdown
Member Author

Rebased onto main (6ec8e0b) and made the four changes, amended into the one commit:

  • oldest_reader_commit_id doc: says retained commit history (OpenOptions::commit_history_retain) also bounds reuse, so None does not mean nothing blocks it.
  • Renamed reader_count_non_abortable to tracked_readers_non_abortable, and the doc calls it a subset of tracked_readers. The CHANGELOG entry is updated.
  • register_read reads the clock before it takes the tracked-readers lock.
  • New test stats_reader_census_fields_can_describe_different_readers: a reader on the latest commit, then 10 ms later a reader opened with begin_read_at on an older commit. The oldest commit is the second reader's and the age is the first reader's. With your min_by_key(|r| r.commit_id.value()) mutant the age assertion fails.

All local gates pass on stable.

@farhan-syah
farhan-syah merged commit 2823311 into NodeDB-Lab:main Nov 5, 2026
20 checks passed
@mkhairi
mkhairi deleted the feat/stats-oldest-reader branch November 6, 2026 06:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants