Skip to content

Add write_zipper benchmarks and run the remaining benches in CI - #188

Merged
Adam-Vandervorst merged 3 commits into
Adam-Vandervorst:masterfrom
adamv-symbolica:adam/wz-bench
Nov 17, 2026
Merged

Adam-Vandervorst merged 3 commits into
Adam-Vandervorst:masterfrom
adamv-symbolica:adam/wz-bench

Conversation

@adamv-symbolica

@adamv-symbolica adamv-symbolica commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Adds a write_zipper bench target, and makes the CI bench job run it together with the other benches it was skipping.

The benchmark

A WriteZipper clones every node it descends into that has another owner, even when it never writes. No existing benchmark moves a WriteZipper through shared nodes, so nothing showed it.

  • shared_descend_to_{rz,wz_owned,wz_shared}: the same 16 descents, 15 nodes deep, as a ReadZipper, as a WriteZipper over nodes it owns, and as a WriteZipper over the same trie with every node shared. Nothing writes. The argument is the width of the nodes passed through.
  • shared_set_val_{wz_owned,wz_shared}: the same descents, each followed by set_val. Here the clones are needed. These cases are there so that a change to how a WriteZipper moves over shared nodes also shows what it does to the write that follows.

On an EPYC 9B45 (medians, 3 rounds):

node width rz wz_owned wz_shared set_val owned set_val shared
1 233 ns 407 ns 694 ns 434 ns 750 ns
4 1.69 µs 2.62 µs 12.50 µs 2.65 µs 12.51 µs
16 1.72 µs 2.73 µs 29.45 µs 2.71 µs 29.44 µs
64 1.94 µs 2.80 µs 105.60 µs 2.88 µs 105.83 µs
256 2.31 µs 2.91 µs 360.53 µs 3.03 µs 358.17 µs

Moving over shared nodes costs as much as writing through them. This PR only adds the measurement.

CI

  • The default bench set gains write_zipper, graft_masked_branches, restrict, prune, oeis, byte_mask, serde and pointcloud. That is every divan bench that runs on one core. parallel stays out because its cases spawn up to 256 threads and each bench is pinned to one core; sla and multiplicities are plain mains that print no divan table.
  • bench_ab.py required every bench target to exist at base and at head, so a PR that adds a bench (this one) failed the job. A target that only head has now runs for head alone and is reported without a comparison. One that only base has is skipped. One that neither has is still an error.
  • Each base/head pair now starts with a half-second spin on its core. Several of the added benches finish in under a second, and on an idle core the side that goes first measured its first cases up to 50% slower: prune_path took 141 ns, against 92 ns for the same binary run right after.

Testing

I ran the job's script the way CI does (default set, 3 rounds, against master) on a 192-core EPYC 9B45:

  • It exits 0 in 8.5 minutes. The added benches take under 20 s per side each; binary_keys at 82 s per side still sets the wall time.
  • write_zipper is reported as new in head, with its 25 cases listed.
  • A target named in BENCHES that neither side has fails with a message, and one that only base has is listed as skipped.

No library code changes, so base and head bench binaries are byte-identical for every bench but the new one, which makes that run an A/A. Of 672 compared cases, 22 moved more than 5%, the largest by +102% (superdense_zipper_prev_cursor/800) and −30% (pointcloud run/query). That is the noise floor of three rounds on that machine; 18 of the 22 are in benches that were already in the set.

🤖 Generated with Claude Code

adamv-symbolica and others added 3 commits November 17, 2026 10:18
A WriteZipper clones every node it descends into that has another owner,
even when it never writes.  No existing benchmark exercises this: none of
them moves a WriteZipper through shared nodes.

shared_descend_to_* runs the same 16 descents, 15 nodes deep, as a
ReadZipper, as a WriteZipper over nodes it owns, and as a WriteZipper over
the same trie with every node shared, sweeping the width of the nodes
passed through from 1 to 256.  Nothing writes to the trie.

On a Xeon 6975P-C at width 256 the shared WriteZipper takes 381 us against
4.15 us owned and 3.2 us for the ReadZipper; at width 1 it is 0.73 us
against 0.41 us and 0.28 us.  The owned and read cases stay flat in width.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
shared_set_val_* runs the descents of shared_descend_to_*, each followed by
set_val at the end of the path, as a WriteZipper over nodes it owns and over
the same trie with every node shared.

Here the clones are needed, so this is the cost that movement over shared
nodes should not have.  It also guards the other side of that trade: a
WriteZipper that stops cloning on the way down has to walk the shared part of
its path again when it writes, and these cases show what that costs.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The default set gains write_zipper, graft_masked_branches, restrict, prune,
oeis, byte_mask, serde and pointcloud, which the bench job did not run.
parallel, sla and multiplicities stay out; the comment on DEFAULT_BENCHES says
why.

bench_ab.py required every bench target to exist at base and at head, so a PR
that adds a bench could not have it run.  A target that only head has now runs
for head alone and its numbers are reported without a comparison; one that
only base has is skipped.  A target that neither side has is still an error.

Several of the added benches finish in under a second, which exposes a bias
in how a pair is run: the side that goes first starts on an idle core and
measures its first cases up to 50% slower (prune_path: 141 ns against 92 ns
for the same binary run right after).  Each pair now starts with a half-second
spin on its core.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@imlvts

imlvts commented Nov 17, 2026

Copy link
Copy Markdown
Collaborator

Yep, that's a bench. The code looks sane at a glance.
LGTM.

@Adam-Vandervorst
Adam-Vandervorst merged commit de125bb into Adam-Vandervorst:master Nov 17, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants