Incident 2026-03-14: orders API outage

Three hours of syslog from web-1, web-2, db-1 and backup-1, with the investigation on top: the cause, the outage and what the on-call engineer did. A made-up incident for the logs video.

2,297 moments · 2026-03-14T01:30:00Z CE – 2026-03-14T04:29:57.914461Z CE · by @athan · @athan

GET /api/orders 200 72ms

2026-03-14T01:31:45.014748Z CE