timescale.info#360 · Index moments and durations individually: entity sitemaps and structured data (#335)
- timeline-app
- pull request
Closes #335 Moment and duration pages already had their own addresses (`/ / /moments/ `, `…/durations/ `), server-rendered text and their own title, description, canonical address, Open Graph tags and preview image. Search engines had no way to find most of them, though: the sitemap listed only timelines and profiles, and the timeline page linked just its first 50 moments. This PR makes each one individually indexable (the investigation is in [this comment](https://github.com/openchronology/timeline-app/issues/335#issuecomment-6434539437)). ## What changed **Sitemap** (`platform/lib/sitemap.js`, `server/store.mjs`) - `/sitemap.xml` now also names `/sitemap/entities- .xml` pages, which list the moment and duration pages of a few timelines each. - Which timelines are listed: public ones (moderator-restricted timelines are made private) with no sensitive-content labels, that aren't comparisons or copies still being prepared, and that have at most `SITEMAP_TIMELINE_ENTITIES` moments (default 5,000, settable from 1 to 45,000). Each listed timeline contributes at most that many entities, durations first. - Larger timelines, such as log imports, are left out entirely. Their own page is still listed, along with its links to the first 50 moments. - Each page holds at most max(page size, limit) ≤ 45,000 addresses, which stays within the protocol's 50,000-URL / 50 MB limits. - Pages are found by ID range with the same index-range technique the timeline and profile pages use. Eligibility uses `event_count` from the timeline row, so no entities are read to decide it. - Entities are read by the primary key of `oc_entity_search` through the new `PostgresStore.entityKeys`. It reads as a guest (the store's own access rules), reads through forks, and uses `LIMIT`. A timeline that became private after the index was read lists nothing. **Entity pages** (`moments/[moment]/page.jsx`, `durations/[duration]/page.jsx`, new `components/entity-summary.jsx`, `lib/sharing.js`) - Server-rendered text, inside the existing timeline summary that is hidden on screen but read by screen readers: - the title; - the time or extent as the timeline displays it, with ` ` in ISO form on Gregorian timelines; - tags and notes; - the enabled plugins' text fields, and their link fields such as Sources (https only, `rel="ugc nofollow noopener noreferrer"`); - up to 50 related entities, each linked to its own page; - a link into the timeline focused on the entity. - schema.org JSON-LD: - `Event` with `startDate`, plus `endDate` for a duration, on Gregorian timelines in years 0000–9999; - otherwise `CreativeWork` with `temporalCoverage` set to the displayed time; - either way: description, keywords from the tags, and `isPartOf` naming the timeline and its owner; - every `<` is escaped, so text in the data can't close the script. - Pages of a labelled timeline are now `noindex`, and their description gives the labels instead of the notes, matching how the preview image already treats these timelines. They also get no JSON-LD. - Related entities are fetched with an empty cursor. That skips counting the whole connected group, which can be expensive. **Docs:** `docs/sharing.md` (link previews, search engines), `docs/search.md` and `docs/docker.md` (`SITEMAP_TIMELINE_ENTITIES`), plus the folder READMEs. I didn't touch the dashboard lists (#337/#338) or the timeline image (#339). ## Testing - `test/entity-pages.test.mjs` (new, runs in `npm test`): ISO dates (time zone offsets, fractions, years before 0, other displays), the `Event` and `CreativeWork` forms, no data for private or labelled timelines, escaping, metadata (labelled → labels and `noindex`; private → `noindex`), and the rendered text (tags, notes, https-only sources, disabled plugins' fields hidden, related links, link into the timeline). - `test/sharing-postgres.mjs` (extended, live browser): the main test timeline is now Gregorian, with tags, Sources and dating plugins and a relationship. It checks: - moment and dura…