Sharetribe incident

Markertplace APIs and Integration APIs are unresponsive

Critical Resolved View vendor source →

Sharetribe experienced a critical incident on December 29, 2023 affecting Marketplace API and Integration API and 1 more component, lasting 3h 32m. The incident has been resolved; the full update timeline is below.

Started
Dec 29, 2023, 04:39 AM UTC
Resolved
Dec 29, 2023, 08:11 AM UTC
Duration
3h 32m
Detected by Pingoru
Dec 29, 2023, 04:39 AM UTC

Affected components

Marketplace APIIntegration APIAuthentication API

Update timeline

  1. investigating Dec 29, 2023, 04:39 AM UTC

    We are currently investigating an issue with Marketplace API and Integration APIs being unresponsive.

  2. monitoring Dec 29, 2023, 05:29 AM UTC

    The APIs are up and running again; we will continue investigating the root cause and monitoring the system.

  3. monitoring Dec 29, 2023, 05:30 AM UTC

    We are continuing to monitor for any further issues.

  4. monitoring Dec 29, 2023, 06:05 AM UTC

    We are continuing to monitor for any further issues.

  5. resolved Dec 29, 2023, 08:11 AM UTC

    This incident has been resolved.

  6. postmortem Dec 29, 2023, 08:11 AM UTC

    Today, Friday 29th Dec at 4:20 UTC - 5:22 UTC we had a major system wide outage, affecting all the APIs, Console and CLI. At 5:22 UTC the system became partially operational. At 5:34 the system because fully operational. At 4:21 UTC our team was alerted and started immediate investigation. We identified a memory issue in our main database and after identifying it, the issue was resolved. To prevent this happening in the future, we took actions and allocated more resources to our database instances. In addition, we will continue investigating the root cause and we will implement additional alert system to identify the issue quicker in the future.