How OpenChronology was built

#147 · Version index rows so forks can later share them

2026-10-14T11:39:07Z CE

• timeline-app
• pull request
• sharing

Phase 1 of option A in #145 (design: https://github.com/openchronology/timeline-app/issues/145#issuecomment-6160959475): versioned index rows and pin-aware writes. Nothing pins an index yet, so behavior is unchanged. Phase 2 makes forks read through instead of copying. ## What changes I changed one thing from the posted design: **live tables keep only current rows.** Old versions go to `_retired` twins instead of living beside current rows behind `until IS NULL`. Reading the current index is therefore exactly as before: same tables, same keys, same plans. **Index clocks** (`server/index-versions.mjs`): - `oc_timelines.index_clock` is advanced by every transaction that writes a timeline's index (`beginIndexWrite`). - Each live row of the six index tables records the clock that wrote it, in `since`. Rows written before this keep a null, which counts as 0. - `index_pinned` will hold the highest clock a fork reads the timeline at. Phase 2 sets it; here it's always null. **Retiring pinned versions.** Triggers generated per table, with a static insert of its columns, copy a row a pin can see into its `_retired` twin, with the clock that ended it (`until`), before it's updated or deleted. - The delete trigger's `WHEN` clause skips rows nothing pins, so unpinned deletes cost nothing extra. - The update trigger sets `since`. - The triggers are regenerated on every schema run, so columns added later are retired too. **Reading an earlier index.** `indexAtSql` reads an index at a clock: its live rows written by then, plus its retired rows that were live then. **No unversioned writes.** `since` defaults to the transaction's clock, and that default fails when no clock is set, so an index write outside `beginIndexWrite` errors instead of recording no version. It found one path I'd missed, the search-row backfill run by `migrate.mjs`, which now takes a clock. **Clocks are set by** the three operations that write the index: `replace`, `applyIncrementally` and `copyIndex`. `copyIndex` doesn't copy `since`, so a copy's rows get the copy's own clock. **Upgrading** adds nullable columns, without rewriting existing rows, plus empty twin tables and triggers. A timeline saved before upgrading versions only the rows it changes afterwards. ## Measured | 200,000 benchmark moments, nothing pinned | `main` | This PR | |---|--:|--:| | Create | 135 s | 137 s | | Saving 10 moved moments | 0.20–0.32 s | 0.14–0.19 s | | Saving 5,000 moved moments | 23.7 s | 25.5 s | | Whole-document save | 149 s | 136 s | That's within this VM's noise. | 50,000 moments | Unpinned | Pinned (every row retired as it changes) | |---|--:|--:| | Saving 10 moments | 0.12 s | 0.17 s | | Saving 5,000 moments | 17.8 s | 19.3 s | | Whole-document save (retires 170,000 rows) | 35 s | 37.5 s | A first, generic trigger that read the catalog per row cost 130 µs per retired row: a whole-document save took 57 s when pinned. Static per-table triggers brought that to about 15 µs. ## Tests - New `test/index-versions-postgres.mjs`, added to CI. It's the oracle phase 2 rests on: - 120 random saves, sparse and whole-document, to a timeline pinned at three points along the way; - after them, the index at each pinned clock reads back exactly as it was then, across all six tables; - a second timeline saved in turn retires nothing; - an index write without a clock fails; - deleting a timeline deletes its retired rows. Breaking retirement on update, or on delete, makes it fail. I checked both by breaking each trigger on purpose. - Tests that fake older index data, or call the migration repairs directly, now do so in a transaction with a clock, as `migrate.mjs` does: - durations, search and summaries; - the versioning test's legacy timeline; - the SQL oracle; - two unit-test mocks. - I upgraded a database saved by `main`'s code with this branch's `migrate.mjs`, then saved: the old rows kept a null `since`, and only the changed row got a clock. Locally, all of these pass: - unit tests (257) and typecheck;…

Open #147 · Version index rows so forks can later share them in the timeline