MX Merchant experienced a major incident on March 19, 2024 affecting MX Merchant and Checkout API and 1 more component, lasting 10h 36m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Mar 19, 2024, 03:19 PM UTC
We are investigating reports of latency with authentication for MX Merchant (https://mxmerchant.com/). We are actively researching and will provide more information shortly.
- investigating Mar 19, 2024, 03:46 PM UTC
We are continuing to investigate this issue.
- investigating Mar 19, 2024, 04:33 PM UTC
We are continuing to investigate this issue.
- identified Mar 19, 2024, 04:57 PM UTC
The issue has been identified and a fix is being implemented.
- identified Mar 19, 2024, 05:33 PM UTC
The issue has been identified and a fix is being implemented.
- identified Mar 19, 2024, 06:17 PM UTC
We are continuing to implement the fix for this issue.
- identified Mar 19, 2024, 06:58 PM UTC
We are continuing to implement the fix for this issue.
- identified Mar 19, 2024, 07:34 PM UTC
We are continuing to implement the fix for this issue.
- identified Mar 19, 2024, 08:11 PM UTC
We are continuing to implement the fix for this issue.
- identified Mar 19, 2024, 08:42 PM UTC
We are continuing to work on a fix for this issue.
- identified Mar 19, 2024, 09:24 PM UTC
team is Awaiting notification amounts in queue to decrease
- identified Mar 19, 2024, 09:45 PM UTC
We will re-check in 1 hour to confirm: kafka lag is dropping (grafana), messages available for queueing in MSSQL are dropping, and errors to not pick up on consumers (currently low or zero).
- resolved Mar 20, 2024, 01:55 AM UTC
The authentication latency issues we were experiencing with MX Merchant (https://mxmerchant.com/) appear to be resolved. We apologize for the inconvenience and thank you for your patience and continued support. Please rest assured that system reliability is a top priority at Priority Payment Systems, and we are making continuous improvements to make our systems better.