How OpenChronology was built

#141 · Keep pull requests as revisions instead of whole documents

2026-10-14T00:09:25Z CE

• timeline-app
• pull request

Stage 3a of moving saved history to rows (after #134 and #138): pull requests stop storing whole timelines. Stage 3b (merges, rebases and syncs per entity, stacked on this PR) comes next. ## What changes **A pull request stores no documents.** `oc_proposals` keeps its base and source revision IDs, and `base_document` and `document` are no longer written. - A timeline submitted for review (and the result of rebasing one) becomes a revision outside any timeline's history. It has its own one-revision history, with its ID as `timeline_id` and number 1. It reads through to its base the way a fork does (#138), keeping only what it changes. It's charged for those changes, and no snapshot is made. - Fork pull requests already pointed at saved fork revisions, so they now store nothing extra. - Upgrading clears the documents stored by earlier versions, since their revisions hold the same ones, and storage charges drop accordingly. Revisions saved before history was kept as rows still read back as before. **Reviews get a summary computed from history.** `GET …/proposals/:id` returns `changes` instead of two whole documents: - the settings that changed; - how many moments, durations and relationships are added, removed and changed; - the first 30 moments of each. The summary comes from a new `differences(from, to)` in `server/history.mjs`, which compares two revisions kept as rows. It only looks at entities either history saved since the two last shared a state: - everything saved by the levels above the first timeline both read through; - whatever that timeline saved between the two revisions. New indexes on `since` and `until` find those. Its cost follows the changes, not the timeline's size. For revisions saved before history was kept as rows, the summary falls back to comparing documents. **The proposed timeline is read only when it's opened.** The new `GET …/proposals/:id/document` is used by View, Edit and Export, in the editor and on the platform's review page. The client still accepts documents in the proposal from servers that send them. **Deleting a timeline** also deletes the revisions its pull requests kept, and every row they hold. ## Measured (200,000 benchmark moments; a pull request changing 100 moments) | | `main` | This PR | |---|--:|--:| | Storage charged for the pull request | 112 MB | 18 KB | | Opening the review | 3.05 s, a 71 MB response | 0.03 s, an 11 KB response | | Creating it (a whole document uploaded) | 27 s | 17 s | | Reading the proposed timeline (view, edit, export) | sent with every review | 6.2 s | **Creating it.** Most of the remaining time goes to PostgreSQL parsing the submitted document and comparing each entity with its base. Staging happens in 5,000-entity batches, and the staged tables' keys are analyzed. **Still whole documents (stage 3b):** - Merge, rebase and sync read whole documents into memory. - Submitting a document pull request uploads the whole timeline, which is still bounded by the 16 MiB request limit. ## Tests - New `test/pull-history-postgres.mjs`, added to CI. It covers: - a submitted timeline kept as rows, outside the timeline, with no snapshot and no stored documents, charged a fraction of the timeline; - the summary; - the document endpoint; - update, rebase and merge; - deleting the timeline leaving nothing behind; - migrating a pull request saved with documents. - `test/shared-history-postgres.mjs` now also checks `differences` on 80 random pairs of saved revisions across the fork chain, against a comparison of their whole documents. It fails if either the "ended between" branch or the copies' overrides are left out of the candidates; both were checked by breaking them on purpose. - Updated tests: - `versioning-postgres` and `postgres`: summaries and the document endpoint instead of documents; - the merge unit test; - the browser test of the review dialog, which now mocks the summary and loads the proposed timeline through the endpoint. Locally, all of these pass: - unit test…

Open #141 · Keep pull requests as revisions instead of whole documents in the timeline