Jenzabar experienced a major incident on October 4, 2023 affecting North America Central, lasting 4d 16h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Oct 04, 2023, 08:20 PM UTC
We are currently investigating an issue with connectivity to our Dallas location.
- identified Oct 04, 2023, 08:24 PM UTC
Engineers have been able to restore services at this time, and customer workloads should be restoring to normal operation at this time. Jenzabar engineers are still working to identify the cause of the issue at this time.
- identified Oct 04, 2023, 11:40 PM UTC
There was a second outage which our engineers have recovered from. We are continuing to work with IBM Cloud and the appliance vendor to identify the cause of these failures.
- identified Oct 05, 2023, 02:43 AM UTC
We continue to experience intermittent issues with connectivity related to our Dallas location as we work to resolve the underlying issue. Just as earlier in the evening we are continuing to work with the IBM Cloud team and the appliance vendor to stabilize connectivity. It should be noted that this just relates to connectivity and all servers and maintenance plans continue to run unimpeded.
- monitoring Oct 05, 2023, 10:49 AM UTC
Jenzabar and IBM engineers identified a software bug and patched systems in our Dallas location accordingly. We continue to monitor the results of these changes. If you have any issues at this time, please submit a ticket via the Jenzabar support portal.
- resolved Oct 09, 2023, 01:09 PM UTC
Systems have remained stable since patching border firewall appliances - resolving this incident.