Database outage in Serverless Cloud affecting customers databases

Update timeline

1. investigating Apr 04, 2024, 06:48 AM UTC

   We are currently investigating this issue.

2. monitoring Apr 04, 2024, 07:31 AM UTC

   A fix has been implemented and we are monitoring the results.

3. monitoring Apr 04, 2024, 07:31 AM UTC

   We are continuing to monitor for any further issues.

4. monitoring Apr 04, 2024, 10:50 AM UTC

   Currently not possible to create new databases. Note that this doesn't affect live sites.

5. monitoring Apr 04, 2024, 01:00 PM UTC

   Update - We are still experiencing issues with database creation. And working on a solution to fix this. This has caused some downtime for live databases for some customers - we are prioritizing to get this up and running as soon as possible. We are sorry for any inconvenience this may cause.

6. monitoring Apr 04, 2024, 01:11 PM UTC

   We are continuing to monitor for any further issues.

7. investigating Apr 04, 2024, 02:02 PM UTC

   We are actively investigating an issue affecting databases in our Serverless Cloud and are working to resolve it as quickly as possible. This is our top priority. While we do not yet have an estimated resolution time, we commit to providing updates every hour until the issue is resolved. We appreciate your patience and understanding during this time and apologize for any inconvenience this may cause.

8. investigating Apr 04, 2024, 03:03 PM UTC

   Our team is still diligently working to identify and resolve the problem as swiftly as possible. Although we do not have new information to share at this moment, please rest assured that addressing this issue continues to be our highest priority. We understand the impact this may have on your operations and truly appreciate your continued patience and support during this time. We will provide another update by 18:00, or sooner if there are any developments. Thank you for your understanding.

9. investigating Apr 04, 2024, 04:03 PM UTC

   As we continue to address the database issues in our Serverless Cloud, we want to assure you that our team is working relentlessly to find a resolution. Although we do not have new developments to report at this moment, please know that this remains our utmost priority. We are fully committed to restoring full service as quickly and safely as possible. Your patience and understanding during this time are incredibly valuable to us. We will provide our next update at 19:00. Thank you once again for your continued patience.

10. identified Apr 04, 2024, 05:03 PM UTC

    We are pleased to inform you that our team has identified a solution to the database issues in our Serverless Cloud and is currently rolling out the fix. While we are optimistic about this resolution, we are closely monitoring the situation to ensure stability and effectiveness across all affected systems. It may take some time for the fix to be fully implemented and for services to return to normal for everyone. We will continue to provide updates as the situation progresses and more information becomes available. Our next update will be at 20:00, or sooner should there be significant developments. Thank you for your patience and understanding.

11. monitoring Apr 04, 2024, 05:49 PM UTC

    We are delighted to announce that the fix for the database issues in our Serverless Cloud has been fully implemented, and all services are now fully operational. Our team worked tirelessly to ensure a swift and effective resolution, and we will continue to closely monitor the situation to prevent any further disruptions. We want to extend our deepest gratitude for your patience and understanding. Rest assured, we are taking steps to enhance our systems to ensure the reliability and stability of our services moving forward.

12. resolved Apr 05, 2024, 05:38 AM UTC

    This incident has been resolved.