How OpenChronology was built

#152 · Forks past four levels copy only what their source changed

2026-10-14T20:52:35Z CE

• timeline-app
• pull request

Phase 4 of #145: the background copy now applies only to forks past four levels, and even then copies only what the source changed. ## What changed Since phase 2, every fork reads through to its source's index. The exception was a fork of a timeline already four levels deep (`INDEX_LEVELS`). That fork copied the source's **whole** index: in the request when small, otherwise in the background while marked as "preparing", then brought back to the forked revision (or rebuilt from it). Now such a fork: - reads through to the same levels its source reads, at the same clocks; - copies only the source's own rows as they were at the fork (`copyLevel` in `server/index-versions.mjs`), including hidden rows, which hide what deeper levels have. So it costs what the source changed, not the size of the timeline. **Pending level.** At fork time it reads through to the source as well, like any fork (five levels while pending), so the source is pinned at the forked clock. `takeLevels` then copies that level and drops it from the lineage, releasing the pin (`oc_release_index`). It runs in the request when the source's own rows are under 20,000 moments and durations. Otherwise it runs in the background (`prepare`); the fork is marked as preparing meanwhile and answers 409 for its index, as before. **No revert or rebuild.** The pin keeps the forked state while a background copy waits, whatever the source does: saves, a whole rewrite, compaction, or deletion. The copy is therefore exactly the forked revision. - Removed: `copyIndex`, `copyIndexLater` and `revertCopy`. - A copy an earlier release left to prepare (no lineage, empty index) is rebuilt from its revision. **Statistics.** After copying 1,000 or more moments and durations, statistics are refreshed before answering, as after a save of that size to a fork. Only the live tables are refreshed, since the copy retires nothing. Comparisons are rebuilt as before. ## Measurements 200,000 moments, `bench/fork-save.mjs`, now extended with a chain of four forks each moving 1,000 moments: | | Answered | Ready | Save 10 after | |---|--:|--:|--:| | Fork (reads through) | 0.59 s | 1.59 s | 0.36 s | | Fork past 4 levels, source changed 1,000 moments | 5.5 s | 5.5 s | 0.57 s | | Fork past 4 levels, source changed 26,000 moments | 0.36 s | 9.3 s (background) | 0.51 s | | Before: a whole copy past 4 levels | ~0.5 s | ~30 s | | In the 5.5 s case: - About 3.8 s is the statistics refresh, which costs the same as after a large fork save. Without it, lookups on the fork were 13–34× slower in phase 2. - About 0.3 s is the copy itself. - About 0.4 s is the timeline-row update every fork pays. ## Tests - `test/fork-preparing-postgres.mjs` now copies through a level limit of 0 instead of disabling sharing. It checks the fork holds exactly the forked revision without a rebuild when the source: - saved, - was rewritten whole, - or was deleted. A deleted source's rows are kept until the copy is made, then released. Every case also checks the copy reads through to nothing and nothing stays pinned for it. New cases: a copy of a fork with hidden rows drops them once it reads through to nothing, and a copy left by an earlier version is rebuilt. - `test/fork-read-through-postgres.mjs`: the copy past four levels must read through to its source's levels at its source's clocks, hold exactly its source's own rows, and leave its source unpinned. It also keeps the deepest levels' rows after the deepest fork is deleted, until the copy goes too. - `test/statistics-postgres.mjs` uses the level limit instead of disabling sharing. - Fault checks, each caught: skipping retired versions, skipping hidden rows, not releasing the pending level, and keeping hidden rows once reading through to nothing. All database suites pass locally (32). `sharing-postgres` and `syslog-live-postgres` couldn't start a browser here; Playwright's browser isn't installed on this VM. Unit tests pass (257). No schema change. Closes #145.

Open #152 · Forks past four levels copy only what their source changed in the timeline