Spiceworks incident
Cloud Help Desk - Reports of email notifications not being sent for new tickets
Spiceworks experienced a minor incident on June 28, 2024 affecting Cloud Help Desk, lasting 3d 6h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Jun 28, 2024, 06:38 PM UTC
We've received reports that certain emails are creating tickets successfully in the cloud helpdesk, but notifications are not being sent out to admins or the end user. Our development team is investigating the issue.
- identified Jun 28, 2024, 10:03 PM UTC
The issue has been identified and a fix is being implemented.
- identified Jun 28, 2024, 11:48 PM UTC
We are making changes to address the underlying issue preventing outbound ticket notification emails from firing in some cases. To address this we have temporarily stopped all outbound ticket notification email flow. Inbound email flow and ticket creation should not be impacted. As of now, the issue appears to be scoped to generating download URLs for ticket attachments, which in turn impacts the generation of ticket notification emails.
- identified Jun 29, 2024, 06:40 AM UTC
We are continuing to work on a fix for the issue. ETA for the fix is multiple hours out, pending a long running process and validation.
- monitoring Jun 29, 2024, 02:40 PM UTC
At approx. 2024-06-29 0830UTC (~6 hours ago) the long-running fix process completed, and approx. 1030UTC outbound email jobs were restarted. We are evaluating expected behavior related to ticket attachments and email notifications for tickets created during the timeframe between 2024-06-28 1430UTC and 2024-06-29 0830UTC.
- resolved Jul 02, 2024, 01:24 AM UTC
We have been monitoring the health of the application through normal peak traffic today and do not have indication of further issues or new issues associated with email notifications for tickets & ticket comments. Please refer to our post on the Community for more specific details (https://community.spiceworks.com/t/some-new-tickets-not-creating-agent-email-notifications/1091315/20).