Mariana Tek incident
Intermittent issues with scheduled actions
Mariana Tek experienced a notice incident on June 26, 2024 affecting Admin and Webhooks, lasting 3d 10h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jun 26, 2024, 03:07 PM UTC
The Mariana Tek system is experiencing some intermittent issues with the following scheduled actions: memberships did not auto renew, livestream emails not sending, spot holds not releasing, and delays in actions in the Admin system. Engineering is investigating and will report back as soon as possible.
- identified Jun 28, 2024, 02:43 PM UTC
We are currently experiencing delays with auto renewals, spot holds not releasing, livestream emails not sending, and automatic refresh of the roster. Our engineering team has identified the root cause and is actively working on a solution. We will provide updates as soon as we have more information.
- identified Jun 28, 2024, 11:24 PM UTC
Update: Membership auto renewals have caught up and will be processed at a regular cadence through the weekend. We are still experiencing intermittent issues for the following scheduled actions: livestream emails not sending, spot holds not releasing, and automatic refresh of the roster. Our engineering team is continuing to work toward a permanent solution for these. We will provide updates as soon as we have them.
- resolved Jun 30, 2024, 01:29 AM UTC
This incident has been resolved as of 12pm ET and the team has been monitoring since. The engineering team will continue to monitor through the weekend. We will reach out to any impacted customer or users should delays in renewals occur.