Noko experienced a minor incident on December 1, 2014, lasting 4h 13m. The incident has been resolved; the full update timeline is below.
Update timeline
- investigating Dec 01, 2014, 07:13 PM UTC
We're again seeing 500 errors & are looking into it.
- monitoring Dec 01, 2014, 08:00 PM UTC
We've rolled out a fix for the main cause of the 500 errors. Certain types of reports could cause our Unicorn server processes to become saturated with new request being denied. We'll monitor load and error occurrences and will update as necessary, though we don't expect any more issues.
- monitoring Dec 01, 2014, 08:06 PM UTC
We still see occasional errors and are working on fixing these.
- monitoring Dec 01, 2014, 08:35 PM UTC
Load is back to normal levels. We'll continue to monitor.
- resolved Dec 01, 2014, 11:27 PM UTC
We've tuned some background processing to lighten the load and haven't seen 500 errors in several hours. We're marking this issue as solved, but will continue to keep an eye on it.