Benchmarks, 2026-09-30

Rebuilt copies and column metadata (2026-09-30)#

A defensive test round against the load limit (release build, counting allocator) found four kinds of input whose peak the estimate did not bound, because Automerge copies something the input holds once, per change or per op, or keeps per-entry structures the estimate did not price. Peak and estimate in MB (2^20 bytes), before and after the terms of The estimate that price them:

Where Automerge 0.12 copies:

• Messages: ChangeCollector::collect rebuilds every change of a document as a StoredChange whose bytes hold the message and whose message is a String copy of it (op_set2/change/collector.rs, finish); Document::verify_changes clones every rebuilt change into MismatchedHeads when the heads do not match. 4.0 bytes per byte of the expanded run.
• Actor ids: ActorId is a TinyVec<[u8; 16]>, so a longer id is a heap copy wherever it is cloned; every rebuilt change holds its own actor and the other actors its ops refer to in its bytes and as ActorIds (ChangeCols::actor, other_actors), cloned again into the error. 4.0 bytes per byte.
• Keys and mark names: a document's rebuilt changes hold their ops' keys in their bytes (2.0 bytes per byte with the error's clone); applying a change (import_ops in op_set2/change/batch.rs) makes an owned String of every op's key (Key::map) and mark name, all held until the batch is applied (1.0 and 2.0 bytes per byte), and the document's key column holds a key literally where other keys come between its rows (4.0 with the Strings, 5.9 with normalize's save).
• Column metadata: RawColumns::parse collects the entries with parse::apply_n into a vector that doubles as it grows, collects them again into RawColumns, and the change and document paths copy the list once more (uncompressed(), ChangeOpsColumns, OpSet::validate): up to 120 bytes per entry (document), 168 (change chunk), 174 (compressed), at counts just past a power of two.

The scan computes the expanded sizes from run headers (a repeat run of n strings of len bytes: (n - 1) × len; an actor column's run: its length, at most the changes, times the actor's length), so it stays proportional to the input. It no longer allocates per metadata entry: before, it kept two vectors per entry (48 bytes per two input bytes), so refusing such input cost about a third of its estimate; a block whose extra entries alone exceed the limit now stops the scan before they are read (12,000,000 entries in 24 kB: refused at the default limit with a peak under 72 MB, in under 2 s).

Saves written by Automerge of the same shapes load (2,000 commits sharing one 100 kB message 0.40, one 100 kB key 0.33, by a 100 kB actor 0.40, one 100 kB mark name, whose rows text characters come between, so the document holds it literally, 0.60), and so do they after another save, where they are rebuilt and applied as changes (0.15, 0.18, 0.31, 0.60), and through merge_changes (0.15, 0.18, 0.31, 0.70). The crafted document chunks list no heads, so a load after another save, or merge_changes, skips them as already contained (a peak of 32 bytes); with one made-up head they are rebuilt and applied (Automerge does not check a later chunk's changes against its heads): 0.15 to 0.46 after a save, 0.18 to 0.36 through merge_changes.

These terms raise the estimate of ordinary documents too, where they repeat keys longer than a few bytes or have actor ids longer than 16 bytes (their other inputs moved by at most +1%: map keys repeated by a run, one key overwritten 40,000 times +0.3%, 400 actors × 400 changes of 16 keys +0.6%; worst ratio still 0.80). An overwritten key is a repeat run of the document's key column, and every overwrite a rebuilt copy; a long actor id is counted per change and per successor. Measured against the commit before these terms (save_nocompress, MB; the "apply" estimate is of merge_changes of the save into another document), with keys and mark names first priced like messages (5 per byte), then at 3:

(The "3" column also caps a change's own and other long actors together at every long actor once, where before only the others were capped.) The real cost of the overwritten 64-byte keys is 1.6-1.7 bytes per expanded byte (normalize +4.1 MB for 2.6 MB expanded), under the 3 charged; applied to another document they cost about 1.9 per byte against 11 charged (3 rebuilt + 8 applied), because the applied keys are priced for the worst case, where the document they are applied to has rows between theirs and holds every copy literally (4.0 per byte measured, 5.9 with normalize's save). So a document whose keys of tens of bytes are overwritten many times, or whose actor ids are long, is priced 10-40% higher than before, and one near the limit before (1.4-1.6 GB at the default 2 GB) can now exceed it on a merge, an INSERT or a restore. The battery holds two such documents now (long_keys: the first row, long_actors: the fifth; 0.40 and 0.38 as saves). A 200,000- input release fuzz session found no violation. tests/limits.sh with the four inputs (a 1 kB compressed save of 6,000 changes sharing a 100 kB message, a 100 kB document chunk of 6,000 changes by a 100 kB actor, a 100 kB change chunk of 16,000 puts of one 100 kB key, and a 24 kB compressed change chunk listing 12,000,000 empty columns): each aborts the capped cluster without the limit, and gets 53400 with it through every path, with no restart.

Cost, interleaved (the previous commit and this one alternating, five pairs; median of the per-run medians of five repetitions; release). The scan alone (budget::scan_input, median of seven batches of 2,000): 83 kB list stored 66.4 → 64.2 µs, compressed 293 → 292 µs; 877 kB list stored 673 → 647 µs, compressed 2,476 → 2,470 µs (no per-entry vectors). bench-core normalize, ms:

bench-sql, ms:

All within ±3.1% (at 83 kB, ±0.6 ms either way), about the drift of R1, whose code did not change (−2.2% at 877 kB); the scan, the only code on these paths that changed, got faster, by 2 to 26 µs.

The Docker image against the pgrx Postgres (2026-09-30)#

The numbers above were all measured on the pgrx-managed Postgres 18.6, which pgrx builds with --enable-cassert and RANDOMIZE_ALLOCATED_MEMORY (every palloc filled with noise). The Docker image runs the same extension (a release build, the same profile) on the PGDG Postgres 18.6 without either. mise run bench-sql and mise run docker-bench-sql (a throwaway container, default settings like the pgrx cluster's, over TCP on 127.0.0.1 like the pgrx runs), alternated two rounds each, BENCH_REPS=5, the mean of the two runs' medians, ms:

Everything that loads a document is within 0 to 6% (up to 11% at 83 kB, where a run is 20 ms and the noise ±1 ms): the time is Automerge's, in Rust, and identical code. The paths that avoid the load (heads, no-op merges, merge_agg of one row, containment by heads and change counts) take about half the time on the image: what is left there is Postgres' own work (detoasting, palloc, the executor), which the assertions and the memory randomization slow down. The earlier sections' conclusions hold on a production build; their millisecond-scale absolute numbers are about twice what a production server shows.

Deep blocks (2026-09-30)#

The jsonb walk before and after it skips Automerge's document iterator for documents with blocks (see Deep blocks); release builds of the previous commit (555110e) and this one, interleaved.

The crash: a Docker image built from an earlier commit, before automerge_spans (the one the reported crash came from), segfaults on INSERT INTO t (doc) VALUES (:'deep_block') into a table with a stored generated doc::jsonb column (the regress fixture: 2,000 levels, 4 kB); so did tests/limits.sh step 3 with the previous commit's library, on its 5,000-level input. With this commit both pass.

mise run bench-core (five rounds, each binary's median of 5; the median of the rounds, ms; "walk" is the sweep before and the checked walk, block check included, after):

The load, which this commit does not touch, differs by up to 7% between the two binaries: code layout. The walk after costs the sweep plus the check (items20k: 65.5 ms for the new binary's own sweep, 66.4 ms checked).

mise run bench-sql read paths (R1: doc->>'status', R2: three accessors, R3: automerge_heads; five interleaved rounds of 5, the median of the rounds, ms), and R1 again (six rounds of 9):

Merge results read as jsonb (expanded values without stored bytes, which are saved once for the check; five interleaved rounds, each the median of 7, ms): merge(doc, c1)::jsonb->>'status' items20k 263.5 → 260.9, items2k 26.1 → 26.3, text3mb 2846 → 2896 (+1.8%); two merges into a PL/pgSQL variable and ten reads of it items20k 1074 → 1046, items2k 100.9 → 98.1, text3mb 5196 → 5256 (+1.2%). The added work there is one save_nocompress (7.3 ms on text3mb, kept for when the value is stored) and the check; the rest of the difference on text3mb is of the size and sign of the untouched load's above.