How OpenChronology was built

#138 · Share saved history between timelines and their forks

2026-10-13T21:19:42Z CE

• timeline-app
• pull request
• sharing

Stage 2 of moving saved history to rows (after #134): forks and duplicates share the saved history of the timeline they copy, instead of copying it. ## What changes A copy records the revision it was copied from as its **base** (`oc_history_bases`), and reads through to it for every entity it hasn't saved itself (`server/history.mjs`). **Overrides.** When a copy first saves an entity that differs from what it reads through to, it records an override (`oc_history_{moment,duration,relationship}_overrides`). The override is the revision from which the copy keeps its own versions of that entity. An override without a version is an entity the copy removed. There's at most one override per entity, and it never changes. **Reading** revision *n* of a copy takes each entity from the first level that saved it: the copy itself at *n*, then its base, then the base's base. An entity is skipped if that level removed it. **Saves on a copy** compare each touched entity with what the copy reads through to: - Unchanged entities write nothing. - Changed entities get an override and their own version. - Whole-document saves (syncs, merges, large patches) do this a page of keys at a time, as before. **Levels.** A copy of a copy reads through each level. Past four levels, a new copy starts its own history with one copy made in the database, so reads never go through more than four. **Retention and deletion:** - A base revision is kept, with the versions it sees, while a copy reads through to it. - Deleting a timeline keeps what its copies read through to. - When the last copy is deleted, the deleted source's remaining history goes too, down the chain. - Bases and overrides deliberately don't reference `oc_timelines`, so a deleted source's kept revisions still read through. **Integrity:** both new tables are immutable, like revisions and versions. ## Measured (200,000 benchmark moments, same VM as #134) | | `main` (copies history) | This PR (shares it) | |---|--:|--:| | History charged to a new fork | 37.4 MB | 164 bytes | | History rows a new fork stores | 200,000 | 0 | | Saving 5,000 moments on the fork | 5.5 s | 2.4 s | | Saving 10 moments on the fork | 0.79 s | 0.83 s | | Reading the fork's revision | 4.4 s | 6.2 s | Reading a copy's revision takes about a third longer, because it reads through to its base. **Plan fix.** The first measurement found that, right after a fork is written, PostgreSQL's statistics count its live rows as none. The per-key lookups were then flattened into joins that read every live row for every key: one 10-moment save hit the 30 s statement timeout. Each per-key lookup in `history.mjs` is now fenced (`OFFSET 0`), so it stays an index lookup whatever the statistics say. That brought 5,000 overrides to 0.3 s. **Not in this PR:** - Some lookups in `incremental.mjs` have the same pattern. They make the first save after a fork about 0.8 s on `main` too. That belongs with #135. - Forking still builds the copy's live index in Node from a whole document: about 2 minutes at 200k, on `main` as well. Copying that index in SQL is a separate change. ## Tests - New `test/shared-history-postgres.mjs`, added to CI. It runs 199 random saves, some sparse and some whole-document, over a chain of copies: - a fork that syncs; - a duplicate of the fork; - a fork of that duplicate; - a fifth copy that must start its own history. It then checks: - every saved revision of every copy (206) reads back as the document it saved; - each new copy is charged only its settings; - pruning the source and the copies keeps what they read through to; - deleting the whole chain leaves nothing behind. I broke the code on purpose three times, to check the test catches it: - not recording removals: fails; - pruning overrides too eagerly: fails; - not protecting base revisions from retention: fails. - `test/random-edits.mjs`: the random edit generator from `incremental-postgres.mjs`, moved so both tests share it. The incremental test still makes the same 120 s…

Open #138 · Share saved history between timelines and their forks in the timeline