pg_automergev0.1.0

Configuration

SettingDefaultWho can change itEffect
pg_automerge.max_load_memory2GBsuperusers, or roles granted SET on it; also ALTER ROLE/ALTER DATABASE .. SET by a superuserRefuse client input and merge results whose estimated load exceeds this (kB, like work_mem; -1: no limit)
pg_automerge.verify_writesonsuperusers, or roles granted SET on it (GRANT SET ON PARAMETER pg_automerge.verify_writes TO writer); also ALTER ROLE/ALTER DATABASE .. SET by a superuserLoad back the normalized save of every value built from client bytes (text input, binary receive, the bytea cast, merge(automerge, bytea) results) before it is stored or sent
pg_automerge.trim_threshold64MBsuperusers, or roles granted SET on it; also ALTER ROLE/ALTER DATABASE .. SET by a superuserAt the end of a transaction, return the memory malloc keeps for reuse to the operating system (malloc_trim) when pg_automerge's allocation fell by at least this much since the last time (kB; 0: after every transaction in which it fell at all, never after one that freed nothing; -1: never)

The pg_automerge prefix is reserved: once the library is loaded, a misspelled name (SET pg_automerge.max_load_memroy = ...) is an error, and one set earlier (in postgresql.conf, ALTER SYSTEM, ALTER ROLE/DATABASE .. SET) is removed with a WARNING naming it, rather than silently leaving the real setting at its default. Check the server log for such warnings after changing these settings.

pg_automerge.max_load_memory protects the server from input that describes far more than it holds (see Limitations). Every value built from client bytes (text input, COPY, binary parameters, the bytea cast, merge(doc, $bytes), automerge_contains(doc, $bytes)) is priced from its headers before it is loaded, and so is every merge result (merge, ||, merge_agg, PL/pgSQL chains); over the limit it fails with SQLSTATE 53400:

ERROR:  estimated memory to load automerge input exceeds "pg_automerge.max_load_memory" (2048 MB)
DETAIL:  Loading it could take up to 9537 MB (10000001 operations, 1 change, 1 actor, 123 bytes uncompressed).
HINT:  A superuser can raise "pg_automerge.max_load_memory".

The estimate is deliberately pessimistic (plain text is priced at about five times its real cost): the 2 GB default admits about 4.4 million characters of text, whose load really takes about 0.4 GB and up to 16 s. Each session can use up to the limit at once, so size it like work_mem. Reading stored values is never limited, so lowering the limit never makes data unreadable, and writes that add nothing (re-sent saves or changes) are not refused. But merging two documents that each fit can fail when the result would not, and restoring a dump into a server with a lower limit refuses the larger rows: restore with PGOPTIONS='-c pg_automerge.max_load_memory=-1' pg_restore ... as a superuser. On a logical replication subscriber, set it for the subscription's owner at least as high as on the publisher. Details in Resource limits.

With pg_automerge.verify_writes = off, incremental writes (merge(doc, $changes)) cost one load instead of two, and input that is not already a canonical or compressed save skips its second load. Parsing, checksums, missing-dependency checks and panic guards still run. The risk: malformed input that loads, but whose re-save does not (so far seen only in fuzzing), is stored instead of rejected with 22P02, and every later read of that row fails with XX000 until it is overwritten. It is superuser-only because one writer turning it off could store a value that breaks reads for everyone. Turn it off only for trusted writers, e.g. ALTER ROLE app_backend SET pg_automerge.verify_writes = off for the role of a backend that sends what its own Automerge produced. It never changes a result's bytes, so functions stay IMMUTABLE and dumps are unaffected. Details in The pg_automerge.verify_writes setting.

pg_automerge.trim_threshold: a loaded document lives in the backend's malloc heap, which keeps freed memory for reuse, so without it every connection would hold on to the most it ever needed: in the soak test, eight connections reading and writing documents of up to 1 MB each held 200 to 355 MB for the whole run (342 MB is what loading the largest one takes), even between statements. With the default, a transaction that freed more than 64 MB hands it back when it ends; that cost the next load of the same 1.26 MB document 15% (1.7 → 2.0 s), and nothing for documents whose loads stay below the threshold. Set it to -1 (or higher) for a connection that keeps reloading the same large documents and has the memory to keep them. glibc only (the Docker image and the usual Linux distributions); elsewhere it does nothing. Details in Returning freed memory (pg_automerge.trim_threshold).