Noko incident

CDN issues preventing Freckle from being displayed correctly

Major Resolved View vendor source →

Noko experienced a major incident on November 27, 2014, lasting 1h 56m. The incident has been resolved; the full update timeline is below.

Started
Nov 27, 2014, 12:41 AM UTC
Resolved
Nov 27, 2014, 02:37 AM UTC
Duration
1h 56m
Detected by Pingoru
Nov 27, 2014, 12:41 AM UTC

Update timeline

  1. investigating Nov 27, 2014, 12:41 AM UTC

    Our CDN (Amazon CloudFront) appears to be offline, unfortunately affecting serving of stylesheets and graphics for Freckle. We're looking into this problem right now.

  2. investigating Nov 27, 2014, 12:45 AM UTC

    Note: Freckle is down, because JavaScript can't be served as well. We're monitoring and have a fallback strategy in case Amazon CloudFront is not coming back up soon.

  3. monitoring Nov 27, 2014, 12:54 AM UTC

    For some users, Freckle has returned to loading normally. The root cause of this outage was that Amazon Cloudfront's DNS entries couldn't be resolved. It appears that DNS is coming back online for it. There's no word from Amazon, so we're watching the situation. It might take a while to work for you as DNS changes propagate.

  4. monitoring Nov 27, 2014, 12:59 AM UTC

    CloudFront DNS seems to be down again. We're preparing a workaround should this situation not be resolved by Amazon in 10-15 minutes.

  5. monitoring Nov 27, 2014, 01:23 AM UTC

    Freckle is back—we've temporarily disabled Amazon CloudFront. Loading times will be a bit slower and we're seeing some Avatars not being loading, but everything else should work. We'll restore CDN functionality once Amazon CloudFront gets fixed.

  6. resolved Nov 27, 2014, 02:37 AM UTC

    Issues with Amazon's CloudFlare DNS not resolving have been fixed by Amazon. Happy Thanksgiving!