HyperTrack experienced a major incident on July 9, 2026, lasting —. The incident has been resolved; the full update timeline is below.
Update timeline
- resolved Jul 15, 2026, 04:10 AM UTC
Summary On July 9, 2026, beginning at 10:00 UTC, devices attempting to establish or re-establish live connections to the HyperTrack platform experienced failures and delays. Devices that were already connected and tracking were largely unaffected, but newly starting devices — including workers beginning shifts during the window — experienced delayed or missing location data until service was fully restored at 19:25 UTC. What happened An unusual pattern in a production customer's orders API usage caused a large number of devices to stop tracking simultaneously, which triggered an internal data-reprocessing burst far larger than our systems had previously encountered. Although the database recovered quickly, the brief outage disconnected a large portion of the device fleet at once. The resulting wave of simultaneous reconnection attempts overwhelmed a connection-handling limitation in our real-time gateway service, creating a self-sustaining backlog that prevented new devices from connecting reliably for the remainder of the window. At 19:14 UTC we deployed a fix that bounds request handling in the gateway service; connectivity recovered within minutes and was fully restored by 19:25 UTC. Customer impact * Devices starting sessions during the window experienced delayed first location updates — in the worst cases by several hours — and a portion never delivered location data for orders created during the window. * Some orders completed during the window are missing arrival or on-site time data. * Location data for devices that remained connected was captured normally. What we are doing to prevent recurrence * Fixed the database connection-handling configuration that allowed the burst to exhaust capacity. * Fixed the gateway connection-handling limitation: bounded request handling, timeouts, and load-shedding so mass reconnections can no longer create a sustained backlog. * Capping and pacing the internal reprocessing behavior that generated the burst. * New monitoring and alerting across every layer involved, tuned to detect this failure pattern well before customer impact.