Open Exchange Rates experienced a minor incident on August 11, 2016, lasting 13h 24m. The incident has been resolved; the full update timeline is below.
Update timeline
- investigating Aug 11, 2016, 08:47 AM UTC
We have recently completed our migration to new infrastructure, and we are investigating an SSL certificate issue when attempting to access our API via curl. We will update this page as soon as we have more information.
- investigating Aug 11, 2016, 09:11 AM UTC
We have redirected all API traffic to our previous architecture while we investigate an issue that prevented some clients from connecting to our API via curl. Please let us know at [email protected] if you continue to experience any issues.
- monitoring Aug 11, 2016, 09:35 AM UTC
We have deployed a solution and continued with our infrastructure migration as planned. We'll publish a post-mortem shortly, detailing the issue and steps we took to resolve it. In the meantime, please email us at [email protected] if you experience any further issues connecting.
- resolved Aug 11, 2016, 10:11 PM UTC
This incident has been resolved.
- postmortem Aug 02, 2018, 04:38 PM UTC
We deployed our new platform infrastructure this morning, bringing a load-balanced, high-availability backend to our industry-leading API. Despite several weeks' extensive testing to prevent issues to our clients, this change resulted in an unforeseen SSL certificate issue, which meant that a subset of clients were unable to access our API via curl. Affected clients received errors like: *"curl: (60) SSL certificate problem: unable to get local issuer certificate"*, and could not access the API without changing their integration to non-verified SSL or plain HTTP. We posted a status page update at 08:47 UTC, shortly after first discovering the issue. As soon as we realised it would take longer than a few minutes to address, we redirected all traffic back to our previous infrastructure at 09:04 UTC, to allow us time to identify and deploy a solution. On investigation, we discovered that the issue was caused by a missing certificate-chain that was accidentally left out when our server certificate was created on AWS. To resolve this, we added the certificate bundle to our SSL certificate, and then directed traffic back to our new infrastructure at 09:30 UTC. We anticipate that fewer than 5% of clients were unable to connect for up to 45 minutes. Regrettably this issue didn't manifest during testing, because only clients that didn't have the latest certificates on their system were affected. In addition to the above, we have received several edge-case issue reports involving non-standard API integrations (in particular, where our legacy API was more lenient towards malformed parameters) and are working with affected clients to resolve any issues caused. Other than these, we're happy to report our new API infrastructure is fully operational. Please don't hesitate to contact us at [email protected] if you experience any unexpected behaviour when connecting to our API. Open Exchange Rates