Ometria experienced a major incident on March 31, 2023 affecting Transactional messages and Broadcast messages and 1 more component, lasting 5h 20m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Mar 31, 2023, 11:23 AM UTC
We are currently investigating reported issues with degraded UI performance.
- monitoring Mar 31, 2023, 11:45 AM UTC
A fix has been implemented and we are monitoring the results.
- identified Mar 31, 2023, 12:15 PM UTC
We are aware of reoccurrences of this behaviour and are continuing to investigate as these happen and monitor until these are resolved.
- identified Mar 31, 2023, 12:57 PM UTC
We are seeing an increase in API calls not being accepted which is linked to the wider issue. If you do not have a retry mechanism, these will need to be resent once the issue is resolved. We are actively investigating solutions to return back to normal service as quickly as possible - thank you for your patience whilst we achieve this.
- monitoring Mar 31, 2023, 02:05 PM UTC
Our engineers have applied a fix and we are continuing to monitor. The application is currently processing the backlog of data that has been created.
- monitoring Mar 31, 2023, 03:56 PM UTC
All impacted areas have been resolved. The API restored, emails that have been delayed are sent and the Application/UI has resumed to near prior operational speed. We will continue to monitor in case of any further issues.
- resolved Mar 31, 2023, 04:43 PM UTC
We continued to monitor any impact from our last update and have not seen any further issues. As such this incident is now resolved. Thank you for you patience and understanding earlier.