Kickserv experienced a major incident on May 30, 2023 affecting Web Application (app.kickserv.com), lasting 33m. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating May 30, 2023, 09:04 PM UTC
We've received reports that the New Opportunities tab is not showing opportunities. We're looking into the issue now.
- investigating May 30, 2023, 09:05 PM UTC
We are continuing to investigate this issue.
- monitoring May 30, 2023, 09:17 PM UTC
We identified the issue as a bug introduced with some recent changes to our opportunities filter. We rolled back those changes and you should once again see your new opportunities back where they belong. We're sorry for the inconvenience.
- resolved May 30, 2023, 09:38 PM UTC
This incident has been resolved.
- postmortem May 31, 2023, 03:32 PM UTC
On Monday afternoon, we deployed a new option to the filters you see at the top of the Jobs lists \(Opportunities, Jobs, and Invoices\). The new option allows you to choose a custom date range \(from any date, to any date\) rather than being limited to a specific timeframe \(this month, last year, etc\). Unfortunately, there was a bug in the code for the new option that went undiscovered during QA testing. Our product and success team members test every fix and new feature before approving it for deployment. This means our software is tested by the people who are most familiar with the ways that real-world users use it. However, it also means our QA process is less formalized and there is room for improvement. We’re sorry this happened and we know you depend on our software to run your business. Reliability and trust are of prime importance to us. We will guard against this in the future by adding specific steps and hints to testers before releasing code changes to QA, and making sure that the code review process includes verifying that QA documentation is in place.