Benchmarks

The benchmark suite measures how operation latency grows with timeline size on each storage backend:

• PostgreSQL: the server's store (server/store.mjs), as the HTTP API calls it: viewport and page queries, and sparse saves.
• SQLite (desktop): the native store (native-store), through the same API the desktop app calls. Tauri IPC is not included.
• In-memory (browser): the editor's TimelineIndex, which serves offline HTML files and opened .ochx documents.

The report draws one chart per operation, with timeline size on the x axis, latency on a shared y axis, and one line per backend. Interaction benchmarks time real input in the editor against the responsiveness budgets these measurements define.

Running

npm run bench:seed     # build the editor, generate seeds (once per size)
npm run bench          # measure, then write bench/results/report.html
npm run bench:capacity # only the in-browser capacity measurements
npm run bench:ui       # only the interaction benchmarks
npm run bench:report   # rebuild the report from existing results

Use a dedicated database. Seeding creates a bench user (with quota bypass) and replaces that user's timelines; nothing else is touched. Opening 200,000-moment timelines in a throttled browser takes minutes per sample, so a run at the default sizes takes about an hour. BENCH_SIZES=1000,10000 finishes in a few minutes. The capacity measurements need Chromium (npx playwright install chromium); without it they are skipped.

The Benchmarks workflow runs the suite on demand (Actions → Benchmarks → Run workflow) at 1,000, 10,000, 50,000, 100,000 and 200,000 moments by default, against a fresh PostgreSQL 16 with native pgmp, and uploads bench/results/ as the benchmark-report artifact. Hosted runners share hardware, so compare backends and sizes within one run rather than absolute numbers across runs.

Seeds

bench/generate.mjs generates deterministic timelines: the same size always produces the same document. Coordinates span 0 to 10⁹ with exact 1/8 fractions. Two thirds of moments fall in dense clusters, with coincident times, and the rest are spread uniformly. Each moment has a title and notes, and about half of the entities carry tags. There is one duration per ten moments (a third anchored to moments; most short, some long) and one relationship per ten moments.

bench/seed.mjs writes each timeline as timeline-<size>.ochx, converts it to timeline-<size>.och with och-seed (the native store's own save), and, with BENCH_DATABASE_URL, creates it in PostgreSQL through the server's store. It also writes workload-<size>.json: the viewports, page window and target moment (the middle of the document's moments) the operations use, with summary thresholds computed for a 1,000-pixel-wide view. Seeds are reused unless --force is passed.

Operations

Every backend performs the same operations with the same inputs. Writes are sparse patches, as the editor sends them. The cleanup between samples is excluded from the measurement. SQLite writes save to a working copy of the seed file with save_patch, as the desktop app saves the open file; each sample is undone by another save, unmeasured.

Opening differs by platform:

• Browser: parses and validates the whole file, builds the in-memory index and draws the first frame.
• Desktop: copies and checks the file into a private baseline, then queries the first view.
• Platform: reads the timeline's metadata and the first view.

Capacity

bench/capacity.mjs measures the in-browser editor itself, in Chromium through Playwright. Each measurement runs at full speed and with the CPU throttled 4× through the DevTools protocol; the throttled run roughly approximates a mid-range phone. For each size it records:

• Open: from importing the file until the editor has drawn the fitted view (the median of three fresh pages, two at 100,000 moments and more).
• Redraw: from applying new bounds until the view is drawn (the median of five).
• Pause after an edit: the longest gap between animation frames in the three seconds after renaming the timeline. It is measured while signed in with a timeline kept only in the browser, so it includes the redraw and the draft saved 0.7 s later. Guests keep no draft, so for them only the redraw remains.
• Draft save: the time the editor spends on the main thread storing each change to the draft, from its openchronology:draft-changes performance measures.
• Memory: the JavaScript heap held after opening, after garbage collection.

It also records the SQLite file size and the PostgreSQL rows a timeline occupies (its moments, indexes, search and links, excluding B-tree indexes and history). The report's capacity charts place these beside the equivalent desktop and platform measurements, with reference lines at 100 ms (responses feel instant) and 1 s (users keep their attention). Where a line crosses them shows where that platform stops feeling responsive.

Realtime updates

npm run bench:realtime (or test/syslog-live-postgres.mjs with SOAK_RATE and SOAK_SECONDS) streams syslog messages at a steady rate through the syslog receiver, the API, PostgreSQL and the live event stream to an editor open in Chromium. For each message it measures the time from sending it until the open editor's moment count includes it. Afterwards it checks that every message is stored exactly once. Messages alternate between UDP and TCP; the receiver saves every 200 ms (the default SYSLOG_FLUSH_MS of 1 s adds up to a second).

Measured on an AMD Ryzen 7 7730U laptop (16 threads), with PostgreSQL in Docker and the production Next.js server:

Latency stays near the save interval up to 500 messages a second; at 1,000 a second the slowest messages wait longer (p95 482 ms, max 801 ms), with each save carrying about 200 moments. Nothing was lost or duplicated at any rate. Each save is one API request, so a single timeline needs about 300 requests a minute at a 200 ms interval and 60 at the default. API_KEY_REQUESTS_PER_MINUTE (default 120) must allow this.

Save time against timeline size

node bench/save-size.mjs [sizes] [runs] (with BENCH_DATABASE_URL; sizes comma-separated, default 100000,1000000) creates a timeline of each size and times appending 10, 1,000 and 5,000 moments after its last (as log ingestion does) and changing 1,000 moments spread through it, each the median of three saves. The server stops any statement after 30 s, so the largest saves must stay well under it as timelines grow (#135).

Measured on an 8-core virtual machine with PostgreSQL 17:

Before #135, a 200,000-moment timeline took 283 s to create and 4.0 s to append 5,000 moments to (1.7 s for 1,000), and a million-moment one about 96 minutes to create and 21–25 s to append 5,000: a save walked the tree once per key, and starting a new timeline's saved history looked each duration up again by an expression the planner did not yet know to use.

Saves after large writes

node bench/fork-save.mjs [moments] (with BENCH_DATABASE_URL) creates a timeline, forks it, and times a 10-moment save to the fork straight away and again after an ANALYZE. A timeline written in one go (created, imported, forked, or rebuilt by a large save or merge) leaves PostgreSQL's statistics counting its rows as none until they are refreshed, and the planner can then read a key through any index that starts with the timeline, filtering every row. The server refreshes the index tables' statistics after such a write of 20,000 moments and durations or more, before answering, and the save path's lookups by key are written to stay one lookup per key whatever the statistics say (#139).

Measured at 200,000 moments on an 8-core virtual machine with PostgreSQL 17:

Before, the catalogue text refresh after a large save to a new fork could also exceed the 30 s statement limit; it now takes about 5 s at this size.

Interactions

bench/ui.mjs drives the editor with real input and times each interaction. In Chromium the input comes from Playwright's mouse, keyboard and touch. In the desktop app it comes from W3C WebDriver actions, through tauri-driver and WebKitWebDriver. Timing runs from the input event that triggers the result (a click on release, a key on its press, each wheel step) to the frame in which the result is painted. For zooming and panning, that is the frame showing the moments at the new view, which the editor records on #markers as it draws them; the axis and the bounds fields can move a frame earlier. The editor's capacity advice times navigation the same way. This is how Event Timing measures interactions, so the time a button is held down is not counted. For continuous gestures it records the longest frame while the gesture runs. An injected probe (bench/ui/probe.js) does the timing in the page, with no browser-specific APIs and no eval, so it works the same in both engines and under the editor's content security policy.

It checks the results against two rules, both from src/capacity.ts:

1. Within a platform's comfortable size (COMFORTABLE, from the capacity data above), every interaction meets its budget.
2. Wherever an interaction misses its budget, the editor is already advising another platform. A miss without that advice is reported as an advice gap.

The budgets follow the advice thresholds (response is THRESHOLDS.edit, view and frame are THRESHOLDS.navigation, open is THRESHOLDS.open), and results is the 1 s line in the capacity charts.

The runs cover:

• the in-memory browser editor, at full speed and with the CPU throttled 4×, whose comfortable sizes are 10,000 and 1,000 moments;
• the platform: the production Next.js server with PostgreSQL, signed in as a bench_ui user, whose comfortable size is the 200,000-moment limit;
• the desktop app (WebKitGTK on Linux), whose comfortable size is also the limit. Timelines over 2,048 moments move into the native SQLite store after opening, as for desktop users; smaller ones stay in memory. Pinch gestures are measured in Chromium only, through its DevTools touch input; the desktop driver sends mouse, keyboard and wheel input.

Each interaction is measured UI_SAMPLES times (default 5) and judged by its median.

node bench/ui.mjs --gate exits with an error on any budget missed within a comfortable size, and on any advice gap, apart from the known misses. In CI, the gate allows 1.5× the budgets (UI_TOLERANCE) for slower shared runners:

• the browser and the platform at 1,000 and 10,000 moments;
• the browser throttled 4× at 1,000 moments, its comfortable size there;
• the browser past its comfortable sizes, at 20,000 moments and at 2,000 throttled 4×: wherever an interaction misses its budget, the advice must be showing;
• the desktop app at 1,000 moments (in memory) and 10,000 moments (SQLite), under Xvfb, allowing 3×: software rendering on shared runners is slow and varies widely (opening search took 96 ms on a laptop and 140–173 ms in CI). This gate guards against breakage, such as an interaction that fails or never finishes, and gross regressions. Navigation and new durations are reported there, not gated: in software they vary severalfold between runners, and the WebDriver path now and then loses a wheel or key input.

The Benchmarks workflow runs the browser and platform at the full set of sizes and adds an Interactions table to the report.

Desktop

The desktop runs need tauri-driver, WebKitWebDriver (Ubuntu's webkit2gtk-driver) and a display. On systems that don't package WebKitWebDriver, such as Arch or macOS, use the container in bench/ui/desktop.Dockerfile:

docker build -t och-desktop-ui -f bench/ui/desktop.Dockerfile bench/ui
docker run --rm -v "$PWD":/repo -v och-desktop-target:/home/ubuntu/target och-desktop-ui \
  bash -c 'node scripts/build.mjs && npx tauri build --debug --no-bundle &&
           UI_PLATFORMS=desktop UI_SIZES=1000,10000 xvfb-run -a node bench/ui.mjs'

Xvfb renders in software, so drawing is slower than on a desktop with a GPU; compare desktop results with each other rather than with the Chromium runs.

Results

Medians from one run on an AMD Ryzen 7 7730U laptop (16 threads): headless Chromium, PostgreSQL in Docker, and the desktop app under Xvfb in Ubuntu 24.04. "×4" throttles the CPU 4×. Budgets: open 1.5 s, view and frame 150 ms, response 100 ms, results 1 s. Bold marks a miss.

What this shows:

• Navigation was the bottleneck on every platform. Discrete actions (opening a moment, typing, saving, deleting, searching) stayed within budget almost everywhere. Zooming, panning and pinching missed it once a few thousand moments were in view, and opening a new duration missed at 10,000 moments. Drawing has since become several times faster (#68, below).
• The comfortable sizes were optimistic for gestures. Navigation missed its budget in the browser at 4,000 moments, then the documented comfortable size, and on a throttled CPU already at 1,000. The platform and the desktop app, documented as comfortable up to the 200,000 limit, miss it from 10,000.
• The advice showed too late. Beyond the comfortable sizes, the browser missed its budgets without the advice showing ("advice gaps"): the advice compared the editor's own drawing time with 150 ms, while users wait longer, from input to paint. The advice now times those waits (#69), and the comfortable sizes come from the runs below.

This run predates #68's faster drawing, and timed zooming until the bounds fields changed, a frame before the moments were drawn.

Comfortable sizes

The comfortable sizes in src/capacity.ts come from the browser runs on GitHub's hosted runners (ubuntu-24.04), about 1.3× slower than the laptop above. Medians in ms; bold misses its budget (150 ms for views and frames, 100 ms for Cancel and Delete, 1.5 s for opening):

At full speed every interaction meets its budget up to 10,000 moments; at 20,000 opening, edits and undo miss it, and the advice shows from the timeline's size. Throttled 4×, every interaction meets it up to 1,000 moments, and zooming misses it from 2,000.

Before #68, navigation was the bottleneck: on the same runners wheel zoom took 136 ms at 1,000 moments and 270 ms at 4,000, and throttled 4× it missed even at 250 moments (257 ms), so the comfortable sizes were 1,000 and 500. Drawing each frame read the stage's width once per marker, and asked for every label's animations and style; each read after drawing changed the page made the browser lay it out or restyle it again. The editor now keeps the stage's size from a ResizeObserver and tracks the animations it starts; duration bands and relationship arcs reshape rather than rebuild, and an edit rebuilds the duration and arc trees only when a moment they name moves.

CI checks the browser and the platform at 1,000 and 10,000 moments, the browser throttled 4× at 1,000, the advice just past the comfortable sizes (20,000, and 2,000 throttled), and the desktop app at 1,000 and 10,000 moments. Every interaction meets its budget there (on the desktop app, apart from navigation and new durations, which are only reported), and past the comfortable sizes the advice shows wherever one misses.

Method

• In-memory and PostgreSQL use tinybench in Node. Each task runs at least 10 iterations and for at least BENCH_TIME, after 2 warm-up iterations. The report shows the mean and standard deviation of individual calls, plus the median, p99, sample count and relative margin of error.
• SQLite uses criterion in-process, with 10 samples after a 500 ms warm-up. Criterion times batches of calls and reports the mean per call and the standard deviation across samples. Its spread is therefore narrower than tinybench's per-call spread, and the p99 is the slowest sample's mean per call.
• Both axes are logarithmic, so equal slopes mean equal growth rates: a line with slope 1 grows linearly with size. Whiskers show ±1 standard deviation. When the deviation exceeds the mean, typically because of garbage-collection pauses, the lower whisker stops at a tenth of the mean and is dashed. Hover or focus a point to see the median and p99; a table under the charts lists every measurement.

Findings

A local run (16 cores, Node 26, PostgreSQL 16 in Docker, BENCH_TIME=1000) measured these means:

• Opening is where the in-memory editor runs out. It parses and indexes the whole file, which takes 0.7 s at 10,000 moments and 34 s at 200,000 in Node (21 s in Chromium). The desktop app copies the file into a private baseline (1.1 s at 200,000), and the platform reads only metadata and the first view. The capacity charts and Timeline size and platforms turn this into guidance per platform.
• Reads scale well. Pages and single moments stay flat from 1,000 to 100,000 moments on every backend. PostgreSQL's few milliseconds are mostly round trips.
• PostgreSQL saves are flat. A sparse save updates only the changed rows and the tree paths they touch (incremental saves), so a one-moment save costs the same at 1,000 and 100,000 moments. Before that change, every save rebuilt the timeline's derived data: 180 ms at 1,000 moments, 1.6 s at 10,000 and 18.5 s at 100,000. History is kept as rows, so a save also writes only the versions it changes (saved history); until that change, every 1,000th save wrote a full snapshot, about 2 s at 100,000 moments.
• SQLite saves are flat too. They apply in place to the open file and its baseline (2.1 ms at 1,000 moments, 2.6 ms at 100,000). Previously, every save copied the whole file twice and rebuilt the duration and arc indexes: 13 ms at 1,000 moments, 105 ms at 10,000 and 1.2 s at 100,000. The benchmark saves to the opened file, as the desktop app does, and undoes each sample unmeasured.
• Viewports grow slowly. Summaries of collapsed durations used to dominate. The tree walk visited nearly every duration, because long durations blocked its shortcuts and pruning used durations' ends. SQLite and PostgreSQL now compute summaries set-based over an index on starts, and the browser prunes by start. At 100,000 moments this took SQLite from 945 ms to 31 ms for the whole timeline and from 430 ms to 14 ms for a 1% window. It took PostgreSQL from 99 ms to 27 ms and from 49 ms to 11 ms. The in-memory 1% window went from 34 ms to 8.4 ms. SQLite also returns bands and arcs in start order and stops at the 257th, instead of finding every match first. With few durations, PostgreSQL's per-cluster queries cost a little more than the old walk (10 ms against 7 ms at 1,000 moments). The in-memory index has large outliers at 100,000 moments, likely garbage collection (mean 292 ms, median 50 ms).