Proofpoint Security Awareness Training incident
Intermittent Phish Simulation Issues
Proofpoint Security Awareness Training experienced a minor incident on December 9, 2024 affecting Simulated Phishing - U.S., lasting 6d 18h. The incident has been resolved; the full update timeline is below.
Affected components
Update timeline
- investigating Dec 09, 2024, 10:30 PM UTC
We are experiencing intermittent degraded service with our Phishing Simulation tool in the US region. Customers may see delays sending initial Campaign Emails, or inability to download certain reports. The following "Export to CSV" reports are impacted: - Export Campaign Data To CSV - Export Campaign Events To CSV - Export Campaign Click Exclusions To CSV
- identified Dec 12, 2024, 04:53 PM UTC
We are experiencing intermittent degraded service with our Phishing Simulation tool in the US region. Over the last several days, Proofpoint has made incremental performance improvements to the tool. These have begun to take affect, reducing the severity of symptoms. The phishing data export will consistently generate and contain accurate up to date events. Phishing campaigns will now consistently meet our expectations of the first email being sent within an hour of campaign start. Customers may see "Users are still being synced. 0% Complete." However, the export is safe to generate. The following types of data may be stale or missing: - User Properties (Department, Manager Name, etc) The following "Export to CSV" reports are now available but impacted: - Export Campaign Data To CSV - Export Campaign Events To CSV - Export Campaign Click Exclusions To CSV
- monitoring Dec 14, 2024, 12:31 AM UTC
Proofpoint deployed an upgrade in the afternoon of December 13th to reduce admin wait time when generating reports and increase the systems consistency when updating user records. This upgrade will alter the behavior of the report generation process. Upon first accessing the portal, Admins will continue to see a loading indicator stating that users are being synced. Once completed, reports will have accurate user data and exports can be generated. For susequent report generations, the admin may see a message stating "User Sync has run in the last hour." In this case, your user data is up to date and the report can be generated immediately. These changes are expected greatly improve the consistency of report generation and accuracy of included user properties
- resolved Dec 16, 2024, 05:00 PM UTC
Proofpoint has observed the consistent generation of phishing reports through peak traffic this morning. To reflect these improvements status is being returned to operational. The Proofpoint team will continue to closely monitor the situation and ensure report generation remains reliable. We apologize for any inconvenience caused by these challenges.