VergeSense incident

Sparesly populated Space Type and Space Group metrics

Major Resolved View vendor source →

VergeSense experienced a major incident on January 18, 2024 affecting AWS and API, lasting 2d 16h. The incident has been resolved; the full update timeline is below.

Started
Jan 18, 2024, 01:31 AM UTC
Resolved
Jan 20, 2024, 05:56 PM UTC
Duration
2d 16h
Detected by Pingoru
Jan 18, 2024, 01:31 AM UTC

Affected components

AWSAPI

Update timeline

  1. investigating Jan 17, 2024, 04:39 PM UTC

    We've observed that our Space Type and Space Group metrics have sparse data populated from the previous outage we've experienced, we're currently investigating and working on a solution.

  2. identified Jan 17, 2024, 06:49 PM UTC

    The issue has been identified and a fix is being implemented.

  3. monitoring Jan 17, 2024, 11:33 PM UTC

    We've deployed a fix and are currently backfilling Space Group and Space Type metrics for the missing hours and dates. We are working backwards so expect the most recent metrics to fill in before older ones.

  4. monitoring Jan 18, 2024, 01:31 AM UTC

    We are continuing to work on fully restoring historical Space Type and Space Group metrics data and will be fully caught up later today

  5. monitoring Jan 19, 2024, 04:38 PM UTC

    We are continuing to monitor for any further issues.

  6. resolved Jan 20, 2024, 05:56 PM UTC

    Space Group and Space Type metrics have been fully backfilled are operational once again