Fat Zebra incident

Changes for retirement of early TLS and addition of Perfect Forward Secrecy (PFS)

Resolved after 62 days 23 hours, on February 16, 2016.

completed Apr 19, 2016, 04:21 AM UTC

This change has been completed.

Started
Feb 16, 2016, 04:39 AM UTC
Resolved
Apr 19, 2016, 04:21 AM UTC
Duration
62d 23h
Detected by Pingoru
Feb 16, 2016, 04:39 AM UTC

Update timeline

  1. scheduled Feb 16, 2016, 04:39 AM UTC

    In order to meet annual security and audit requirements TLSv1 is due to be disabled on all HTTPS endpoints, including the Merchant Dashboard, Hosted Payment Pages and the Payment Gateway. In addition to this PFS will be implemented with new 2048 bit Diffe-Hellman group parameters and updated cipher suites used on these endpoints to include all strong Diffe-Hellman and Elliptical Curve Diffe-Helman key exchange cipher suites. For most merchants this change will not have any impact, however as there are some older systems which may experience issues with this change we are planning a change window of 1 month which will involve the following: * On the 20th of February the changes to our border load balancers will take place for all Sandbox services. From this date all merchants are encouraged to test their systems against the sandbox environment to ensure their systems continue to work properly. * On the 20th of March these changes will be carried out for all Production services. Following the changes in the Sandbox environment should there be any issues with systems talking to our Gateway API please contact [email protected] and we will assist where possible. These changes are not expected to result in any downtime, however we will be providing further scheduling details for the Production change following successful sign off in the Sandbox environment. If any further information regarding this change is required please contact [email protected] for assistance.

  2. scheduled Feb 19, 2016, 11:59 AM UTC

    Maintenance will begin as scheduled in 60 minutes.

  3. in progress Feb 19, 2016, 01:00 PM UTC

    Scheduled maintenance is currently in progress. We will provide updates as necessary.

  4. monitoring Feb 19, 2016, 02:05 PM UTC

    Changes have now been applied to the Sandbox environment and confirmed to be operating as expected. Please test your websites and integrations against the gateway and contact [email protected] if there are any issues which cannot be resolved following this test.

  5. monitoring Mar 20, 2016, 01:41 PM UTC

    Just a quick reminder for our merchants - this change will be taking place during business hours today (Monday 21 March 2015) so that we can closely monitor and respond to issues with merchant integrations where required. Should there be any concerns please contact [email protected]

  6. monitoring Mar 21, 2016, 05:11 AM UTC

    TLS v1.0 has been disabled on https://gateway.pmnts.io and https://gateway.cloudpayments.com.au. This change will also be applied to https://gateway.fatzebra.com.au in the next 30 minute once we have verified there has been no impact to merchants websites. If any merchants are having issues processing transactions please contact [email protected] for assistance.

  7. in progress Mar 21, 2016, 05:12 AM UTC

    Scheduled maintenance is currently in progress. We will provide updates as necessary.

  8. monitoring Mar 21, 2016, 11:16 PM UTC

    All services have been successfully switched over to supporting TLS v1.1 and TLS v1.2 only. If any merchants are experiencing difficulties processing transactions please contact [email protected] for assistance.

  9. scheduled Mar 22, 2016, 10:21 AM UTC

    Due to issues experienced with merchants integrations following the disabling of TLS v1.0 we have rolled back this change for https://gateway.fatzebra.com.au. This change was rolled back at 10:30am AEDT. If any merchants experienced issues between 9am and 11am today please contact Fat Zebra Support ([email protected]) as soon as possible to discuss issue remediation. Further updates regarding our scheduling will follow tomorrow.

  10. in progress Mar 22, 2016, 10:22 AM UTC

    Scheduled maintenance is currently in progress. We will provide updates as necessary.

  11. scheduled Mar 22, 2016, 10:23 AM UTC

    We will be undergoing scheduled maintenance during this time.

  12. in progress Mar 31, 2016, 01:00 PM UTC

    Scheduled maintenance is currently in progress. We will provide updates as necessary.

  13. in progress Apr 08, 2016, 04:03 AM UTC

    This change has been postponed until Monday 11th April due to timing difficulties. The TLS v1.0 support will be switched off at 11am AEST.

  14. in progress Apr 11, 2016, 01:12 AM UTC

    TLS v1.0 support will be disabled at 11:30 AM AEST today (Monday 11th April). If any merchants experience issues processing transactions after this time please contact support for assistance.

  15. scheduled Apr 11, 2016, 02:09 AM UTC

    We will be undergoing scheduled maintenance during this time.

  16. in progress Apr 11, 2016, 02:09 AM UTC

    Scheduled maintenance is currently in progress. We will provide updates as necessary.

  17. scheduled Apr 11, 2016, 06:34 AM UTC

    Due to issues with merchant systems updating to the newer TLS versions we have postponed this change to Friday 15th of April, 11:30AM AEST.

  18. scheduled Apr 17, 2016, 03:26 PM UTC

    Reminder: This change will be taking place at 12:30pm on Monday 18th April.

  19. scheduled Apr 18, 2016, 01:50 AM UTC

    This change has been postponed until 1:30pm due to scheduling issues.

  20. scheduled Apr 18, 2016, 03:31 AM UTC

    This change has been postponed until 3pm to finalise the transfer of merchants off of old-TLS. We apologise for the inconvenience this rescheduling has caused.

  21. monitoring Apr 18, 2016, 05:17 AM UTC

    The change to disable TLS v1.0 has been completed and we are monitoring for the next hour. If any merchants are experiencing issues performing transactions please contact support for assistance. We apologise for the number of reschedules and delays involved with this change.

  22. completed Apr 19, 2016, 04:21 AM UTC

    This change has been completed.

Your product runs on more than Fat Zebra

Add the vendors you depend on and get one message when any of them reports trouble. Free for 3 vendors, no card.

Add your stack →