Datadog US1 experienced a notice incident on October 27, 2015, lasting 4d 18h. The incident has been resolved; the full update timeline is below.
Update timeline
- scheduled Oct 27, 2015, 04:30 PM UTC
If your servers' clock are set to UTC or to a timezone without DST, you can ignore this message. We recently discovered an issue that affects Datadog Agents only on servers with clocks set to Daylight Saving Time (DST). If your servers' clock are set to DST, you should upgrade the Agent to v5.5.2 before DST ends or restart the Agent shortly after DST ends. DST ends November 1st 2015, 2AM local time. Impact of the DST bug If the server's clock is set to DST (instead of UTC), the Agent will stop sending metrics for up to one hour after DST ends, or until it is is restarted. In the United States for instance, this is happening this Sunday (November 1st 2015, 2AM local time). In practice this means that the agent will collect metrics but won't report for up to one hour after DST ends. If the agent is restarted in this hour, it will resume reporting normally. If you have monitors on metrics with timeframes shorter than 1 hour, you may receive spurious alerts coming from the affected Agents. How to tell whether your servers are affected If you are using Linux/Unix, simply run "date" on your servers. If they read "Sat Oct 31 22:00:56 UTC 2015", your servers is not affected. If they read something like "Sat Oct 31 18:09:09 EDT 2015" they are affected. On Windows, simply look at the time in the task bar. What you can do to avoid the DST bug You have 2 options: * Schedule an agent restart shortly after DST ends (e.g. use cron or similar to restart the agent at 01:01 local time) * Upgrade the Agents on the affected servers to 5.5.2, directions may be found here: https://app.datadoghq.com/account/settings#agent Changelog for Agent 5.5.2: https://github.com/DataDog/dd-agent/blob/master/CHANGELOG.md#552--10-26-2015 If you have any questions or concerns please don’t hesitate to reach out to us at [email protected] - sorry for the inconvenience.
- scheduled Nov 01, 2015, 06:00 AM UTC
Maintenance will begin as scheduled in 60 minutes.
- in progress Nov 01, 2015, 07:00 AM UTC
Scheduled maintenance is currently in progress. We will provide updates as necessary.
- completed Nov 01, 2015, 11:00 AM UTC
The scheduled maintenance has been completed.