Flying Sphinx experienced a notice incident on March 21, 2016, lasting —. The incident has been resolved; the full update timeline is below.
Update timeline
- resolved Mar 21, 2016, 10:44 PM UTC
The API was having database related issues for three hours early this morning (Melbourne time), which were related to the previous incident on March 1st. To follow up on the planned changes from then: the new alerting worked, wasn't persistent enough to wake me up from an exhausted sleep. The database changes had improved the situation, but not stopped it completely (hence this repeat). The proxy issue didn't return from what I can see (thus, only API requests, not search requests, were impacted). I've got next steps planned for further database improvements, and will look into making the alerting a little more insistent. The delay in resolving this issue was certainly not acceptable.