Telerivet incident

Datacenter connectivity issue

Minor Resolved View vendor source →

Telerivet experienced a minor incident on April 12, 2016, lasting 1d 2h. The incident has been resolved; the full update timeline is below.

Started
Apr 12, 2016, 02:25 AM UTC
Resolved
Apr 13, 2016, 05:10 AM UTC
Duration
1d 2h
Detected by Pingoru
Apr 12, 2016, 02:25 AM UTC

Update timeline

  1. identified Apr 12, 2016, 02:25 AM UTC

    Telerivet's primary datacenter in Google Compute Engine is currently experiencing network connectivity issues, so we have activated Telerivet's secondary datacenter in Amazon Web Services. Most services should now be functional, although search may be unavailable.

  2. monitoring Apr 12, 2016, 06:32 PM UTC

    At 02:09 UTC, Google Compute Engine began experiencing severe network connectivity issues worldwide ( https://status.cloud.google.com/incident/compute/16007 ). In preparation for this type of outage, Telerivet has a standby datacenter from another hosting provider (Amazon Web Services), which we activated at 02:19 UTC. Most Telerivet services were operational at that time, except for search and some SMPP shortcode connections. SMPP connections were reactivated over the next half hour. Google resolved the issue with Google Compute Engine at 02:27 UTC, and subsequently we reactivated our primary datacenter in Google Compute Engine. Search indexes for each project have gradually been coming back online. Search is now available for most active projects, and should become available in all projects over the next several hours.

  3. resolved Apr 13, 2016, 05:10 AM UTC

    This incident has been resolved.