Benchmarks, 2026-10-01

Soak test (2026-10-01)#

mise run soak as described in Soak test: 70 minutes, 8 clients, documents up to 1000 kB (SOAK_DOC_SIZE; the largest a 1.26 MB rich text of 1.04 million operations and a 1.07 MB list of 23,000 maps), 200 rows at the start, a 4 GB container, shared_buffers = 256MB, max_load_memory = 512MB, the Docker image on this development machine (8 cores, all busy throughout). Three runs: before the trim (run 1, image of 453cb9e), with it (run 2), and the final configuration (run 3: the trim, the GIN index with fastupdate = off, the recalibrated checks).

What they show:

• No leak. The Rust heap went back to 16 bytes between statements in every sample of every backend for 70 minutes (about 34,000 loads per backend), no document stayed alive, and the memory contexts stayed at 3.0 to 3.1 MB per backend (the 0.02 MB/h slope is the first samples after the backend started; flat after the first window). The backends' anonymous RSS did not trend: in run 1 its high was 344-354 MB in the first half and 339-356 MB in the second.
• malloc kept what the largest document needed, in every backend (fixed). In run 1 every backend held 200 to 355 MB all the time, for documents that need 342 MB while loaded and 16 bytes afterwards: see Returning freed memory. With the trim (run 2) a backend drops to 5-75 MB after such a transaction, and the container's median anonymous memory fell by about 1 GB (40%); its maximum (2.3 GB) is the documents being loaded at that moment. Throughput was the same (28.9 against 29.2 transactions per second, run to run noise on this machine being about as large).
• GIN with a pending list stops being used under write load (documented). read_gin's median grew with the table, 85 → 499 ms. On run 2's data (2,545 rows, kept with SOAK_KEEP=1), idle, the containment query takes 1 ms through the index (80 ms for the big documents' slot: the recheck detoasts their 425 kB jsonb); under the same load (150 s, 8 clients) the planner chose a sequential scan instead (600 to 900 ms: it detoasts every row's jsonb, 25,000 buffers), because the pending list, which every write of a big document fills with thousands of keys, is part of the index's cost. With ALTER INDEX .. SET (fastupdate = off): the bitmap scan again (1.5 ms), read_gin 11.6 ms against 466 ms on average, writes 74 ms against 68 ms (big ones 3.45 s against 3.60 s), 31.5 against 25.5 transactions per second. Operations recommends fastupdate = off, and the soak creates its index that way (SOAK_GIN_FASTUPDATE=on for Postgres' default). Run 3 confirms it over the whole run: read_gin's median 1.4 → 3.0 ms (its p95, 16 → 83 ms, is the big documents' slot, whose candidates grow with the table and are rechecked against their 425 kB jsonb), and 32.6 instead of 28.9 transactions per second. The cost is WAL: 1,336 kB per write instead of 949 kB (+41%), every write inserting its keys into the index tree instead of appending them to the pending list (and 90 "checkpoints are occurring too frequently" against 6). Operations says both.
• WAL and TOAST are the cost of writing whole documents. 949 kB of WAL per write over the mix: every update writes the new TOAST value in full, twice with the generated jsonb copy (425 kB against 427 kB stored for a big document), plus full-page images after each of the frequent checkpoints (max_wal_size 1 GB: six "checkpoints are occurring too frequently" in run 2). Autovacuum's defaults kept the TOAST table's dead tuples under 27% (about one autovacuum a minute), its free space was reused, and the table grew with the rows, not with the writes (the heap's free space after VACUUM 26-28%). See Operations.
• Latency did not drift, except read_gin above and what the growing table and documents explain: the small writes' median rose from 12 to 16 ms as the rows and their change counts grew (54 → 71 changes per small document on average) and the client time read_gin took grew (throughput per window fell about 20% over the run for that reason); p95 and p99 of every other script were flat. The big documents' scripts (write_big, read_big, upsert) are bimodal (read_big takes about 0.7 s on the list, 3.2 s on the rich text), so their median jumps between the two from window to window.
• Checks recalibrated after run 1 and 2: run 1's "no ERROR" check failed because of the author's own queries against the server during the run (an ERROR from deliberately exceeding max_load_memory, now documented: don't); its RSS growth check (a least-squares slope) failed on noise (−121 to +46 MB/h between backends with no trend), replaced by comparing the high-water marks of the two halves, which run 2 missed at 32 MB (42.7 MB: one sample of 382 MB, a load's Rust peak plus its palloc memory at the same moment), so the limit is 64 MB; the memory contexts got their own check (1 MB). Run 3 passed every check but the log check, on one ERROR: canceling autovacuum task: Postgres cancels an autovacuum that blocks the harness's own manual VACUUM, which is expected and now allowed. Run 4 (65 minutes, the final harness, pgbench from the image): every check passed; 124,514 transactions (15,564 per client), 55,460 notifications, 16 bytes allocated and no live document between statements, memory contexts 3.01-3.02 MB, backend RSS high-water growth 11.1 MB, container anonymous memory 1.48-1.63 GB at the median per window and 2.34 GB at most, read_gin median 1.0 → 2.3 ms, 1,305 kB of WAL per write.

The estimate's neighbourhood (2026-10-01)#

The terms of Rebuilt copies and column metadata were measured on the shapes that found them (one repeat run of a message, key or mark name; a long actor id as the change's actor or the object's). This round measured the shapes next to them, 592 new measurements in tests/memory_bounds.rs (release build, the counting allocator, the battery run under ulimit -v 32 GB; worst peak / estimate per shape, over its plain and deflated or compressed forms, on its own, after another save and through merge_changes; "refused" means Automerge refused the input after allocating for it, as the crafted document chunks without heads are):

One term was too low: mark names repeated by a run in a document chunk cost 3.0 bytes per expanded byte (a rebuilt change holds a mark name about as a message, where a key costs 2.0), and were charged 3, with the keys: 2,000 changes sharing one 100 kB mark name peaked at 603 MB against an estimate of 606 MB (0.996; plain and deflated), and runs of nine interleaved with different names at 0.86. Below the estimate, but with none of the margin every other measured shape keeps (0.80 at most), so the next shape over could have exceeded it. Mark names are now rebuilt bytes, charged 5 like messages: 0.60 and 0.63. The scan is unchanged otherwise (the mark name column's excess was already computed separately per column; it is now added to the other sum). The estimates of all 129 measurements that existed before are byte for byte the same (none of their documents repeats a mark name by a run), so their worst ratio is still 0.80.

Found besides, and not fixed: applying many overlapping marks takes time quadratic in their number (see What the limit cannot do). normalize, release, the limit off:

The memory of every one of them stayed within the estimate (0.77 at most). The battery's inputs with many marks are exempt from its time bound, which every other input still meets.

Also: the scan holds the distinct actor ids of change chunks (by name, to count them), so refusing a change listing 1,000 actor ids of 1 kB peaks at 1 MB, the input's size. The rejection check of the battery allowed only the inflated bytes; it allows the input's size now (estimate / 10 at most, as each byte costs 10).

Cost: the scan adds one comparison per string column. bench-core, the commit before and this one alternating, five pairs, median of the per-run medians (ms; release):

All within −1.0% to +1.4%, the noise: Automerge::load, whose code did not change, moved −0.4% to +2.8%. A 200,000-input release fuzz session (mise run fuzz, with the new seeds) found no violation. The battery takes 156 s in a release build (37 s before) and 240 s in the dev build of mise run test.