Repository navigation
fix: keep one-time setup costs out of the first measured benchmark - #123
not-matthias wants to merge 2 commits into
Conversation
Analysis runs called the `__codspeed_root_frame__` wrapper for the first time after `startBenchmark()`, so V8 compiled it, and the perf map recorded it, inside the measurement. Only the first benchmark of a process paid for it: `switch 1` of examples/with-typescript-simple-esm measured 156,638 Ir under callgrind, against 3,152 Ir for `switch 2`. The warmup now calls the benchmark through the same wrapper, which brings `switch 1` down to 6,964 Ir.
With lazy binding, the first `stopBenchmark()` of a process resolved `instrument_hooks_stop_benchmark` through the dynamic linker while the benchmark was still measured. Link the addon with `-z now` so symbols resolve when it loads. Under callgrind this removes 2,339 Ir from the first benchmark of examples/with-vitest-v5, and brings `switch 1` of examples/with-typescript-simple-esm to 4,625 Ir.
Merging this PR will regress 5 benchmarks4 warnings, 1 note
Warning Please fix the performance issues or acknowledge them on CodSpeed. Performance Changes
Tip Investigate this regression by commenting Comparing 
Footnotes
|
TLDR: In analysis mode, the first benchmark measured in a process included one-time setup costs: V8 compiling the root-frame wrapper (tinybench) and the dynamic linker resolving
instrument_hooks_stop_benchmark(all integrations). Both now happen before measurement starts.tinybench: the warmup calls the benchmark through the__codspeed_root_frame__wrapper, so V8 compiles it beforestartBenchmark().core: link the native addon with-z now, so its symbols resolve at load instead of on the firststopBenchmark().Callgrind Ir, using the runner's simulation args:
with-typescript-simple-esmswitch 1(first in process)with-typescript-simple-esmswitch 2with-vitest-v5fibo 15 recursive(first in process)Review notes