timescale.info#160 · Export large server timelines from the server
- timeline-app
- pull request
Part of #135: the editor's `.ochx` export of a large server timeline no longer builds the document in the page. ## What changed **The problem.** **Export .ochx** in the editor used to read a server timeline's whole document from the server, apply unsaved edits, embed linked images and serialize it all in the page. That's about 1 GB of page memory at a million moments. **When the server exports instead.** The editor follows a link to the server's streamed export when all of these hold: - the user is signed in; - the timeline has over 100,000 moments; - it has no unsaved edits, or was opened read-only; - it isn't a comparison; - the editor runs on the web. The desktop app talks to servers through its native bridge, which a plain link doesn't use, so it's left out. **The export address.** It's a new route: `GET /api/timelines/:id/export/ .ochx` (signed in). - It streams the document like `/document?download=1` (#156), as an attachment named ` .ochx`. - The editor names it from the timeline's title, as its own exports are named. - Putting the name in the address matters: Chromium and WebKit name a download from an unnamed link after its address, while Firefox follows `Content-Disposition`. With the name in both, every browser saves ` .ochx`. - `/document?download=1` now also names its file from the title. **What's different:** images the timeline links to by address aren't embedded in a server-made file. Embedding them needs the browser, and the server shouldn't fetch arbitrary URLs. Images it stores in `assets` are included. **Unchanged:** - Smaller timelines, timelines with unsaved edits, and guests export in the page as before. - The guest path stays bounded by the browser-fork limits. ## Tests - **`test/community-browser.mjs`:** a signed-in editor opens a mocked server timeline reporting 150,000 moments and clicks **Export**. - **Every browser:** the editor must follow an unnamed download link to `/api/timelines/ /export/featured-timeline.ochx`. - **Chromium and Firefox:** the download itself must also arrive as `featured-timeline.ochx` from that address. - **WebKit:** the download check is skipped. In these tests nothing but the page's routing serves the address, and in CI Playwright's WebKit reported no download for the link, through a link, a frame or a navigation alike. The link is checked there instead. - **`test/next-postgres.mjs`:** `/export/fork-title.ochx` streams the document with that name, guests get 401, and `/document?download=1` is named from the title. - **Other checks:** all 60 CI checks pass; typecheck, build and unit tests pass locally. ## Also looked at **SQLite `.och` export:** it converts a whole document through the `och-convert` binary, with explicit limits (32 MB, 20 s), and fails cleanly past them. It's a bounded exchange format for the desktop app, so I left it as is.
Sources
Open #160 · Export large server timelines from the server in the timeline