timescale.info#142 · Merge, rebase and sync entity by entity
- timeline-app
- pull request
Stage 3b of moving saved history to rows: merges, rebases and syncs work entity by entity, instead of merging three whole documents in memory and rebuilding the timeline's index. **This PR is stacked on #141** (pull requests keep no documents). It targets that branch, so its diff shows only this change. Once #141 merges, GitHub retargets it to `main`. ## What changes `mergePatch` in `server/merge.mjs` merges what one revision (`other`) changed since a base into a timeline whose head is its current state: 1. **Read what changed.** `differences(base, other)` (#141) gives only the entities `other` changed since the base. 2. **Merge each entity.** Each is merged with the timeline's current version using the same field-level three-way merge as `rebaseDocument`, now shared as `mergeValue`. Conflicts produce the same HTTP 409 with a `conflicts` list. Settings merge the same way. 3. **Complete the result** as a whole-timeline merge would leave it: - links to removed moments and durations go; - links added to entities that no longer exist aren't added; - durations anchored to removed moments keep the moment's last time. 4. **Apply it.** The result is a patch. - **Merge** (`merge` in `collaboration.mjs`) and **sync** (`sync` in `versioning.mjs`) apply it like a save, with the merge's kind and parents. Small patches update the index in place; patches changing more than an eighth of the timeline rebuild it, as before. - **Rebase** keeps it as a revision that reads through to upstream's head (`recordDetachedPatch`), charged what it changes. Revisions saved before history was kept as rows still merge as whole documents. ## Measured (200,000 benchmark moments, same VM) | | `main` | This PR | |---|--:|--:| | Merging a fork's pull request (200 fork changes, 100 upstream changes) | 150 s | 0.35 s | | Syncing 100 upstream changes into a fork with 100 changes of its own | 161 s | 0.92 s | **Catalogue text.** Because merges now go through the incremental path, moment text reaches the catalogue just after the merge commits, as it does after any sparse save. The HTTP test now waits for it, and settings such as titles and tags are searchable immediately as before. ## Tests New `test/merge-postgres.mjs`, added to CI: an oracle comparing against `rebaseDocument` on the same three documents. - It runs 45 rounds of random edits on both sides of forks: 15 syncs, 15 fork pull request merges, and 15 rebases of submitted pull requests onto a changed upstream. - The results must match exactly, including conflicting exactly when the oracle does (9 rounds here). - Spies check that each round took the per-entity path: no index rebuild, and a rebase kept as a patch. - Crafted rounds cover the cases random edits rarely reach: - both sides change different fields of one moment (they merge); - both make the same change; - one changes a moment the other deleted (a conflict); - one side links and anchors a duration to a moment the other deletes, in both directions; - upstream links and anchors to a moment a rebased proposal deletes. - I broke each completion step on purpose, one at a time; each break makes the test fail: - not pinning the head's durations anchored to removed moments; - not removing the head's links to removed entities; - not dropping links added to missing entities; - not pinning merged durations' anchors. Locally, all of these pass: - unit tests (257) and typecheck; - every PostgreSQL suite, including the platform's (`test:platform:postgres`).
Sources
Open #142 · Merge, rebase and sync entity by entity in the timeline