Sila incident

AWS email shutdown impacting services

Minor Resolved View vendor source →

Sila experienced a minor incident on August 1, 2023 affecting Production API and Developer Console, lasting 22h 26m. The incident has been resolved; the full update timeline is below.

Started
Aug 01, 2023, 06:29 PM UTC
Resolved
Aug 02, 2023, 04:56 PM UTC
Duration
22h 26m
Detected by Pingoru
Aug 01, 2023, 06:29 PM UTC

Affected components

Production APIDeveloper Console

Update timeline

  1. identified Aug 01, 2023, 06:29 PM UTC

    Because we lost the ability to send Email temporarily, there will be degraded performance in team invitations, transactions for accounts with non-functional transaction webhooks, webhook email alerts (webhooks themselves are OK), sending statements, password reset, creating console logins. Working with AWS to get the ability to send emails back.

  2. monitoring Aug 02, 2023, 04:15 AM UTC

    The ability to send email from our API servers and AWS infrastructure (Lambdas, etc.) has been restored as of 19:30 PDT. We are currently monitoring all functionality that was affected by this service interruption, but are expecting no further impact.

  3. resolved Aug 02, 2023, 04:56 PM UTC

    This incident has been resolved and all operations performed normally.