Happo Outage History

Happo is up right now

Happo had 7 outages in the last 2 years totaling 4h 7m of downtime — averaging 0.3 incidents per month.

There were 7 Happo outages since April 9, 2025 totaling 4h 7m of downtime. Each is summarised below — incident details, duration, and resolution information.

Source: https://happo.statuspage.io

Major August 17, 2026

API failures

Detected by Pingoru
Aug 17, 2026, 02:54 PM UTC
Resolved
Aug 17, 2026, 05:33 PM UTC
Duration
2h 38m
Affected: API
Timeline · 2 updates
  1. investigating Aug 17, 2026, 02:54 PM UTC

    We’re seeing errors coming from our own API calls to GitHub causing most jobs to fail on our end as well. We’re investigating this right now.

  2. resolved Aug 17, 2026, 06:33 PM UTC

    A GitHub outage caused some of our API endpoints started to fail. To reduce the impact we briefly turned off some communication with GitHub which made things stabilize on our side. At this time, no Happo statuses could be posted to GitHub PRs and commits. At 18.11 UTC we saw things starting to stabilize and briefly after we were back in a good state again.

Read the full incident report →

Major April 22, 2026

Timeouts and failed API requests

Detected by Pingoru
Apr 22, 2026, 05:01 PM UTC
Resolved
Apr 22, 2026, 04:30 PM UTC
Duration
—
Timeline · 1 update
  1. resolved Apr 22, 2026, 05:01 PM UTC

    During a window of about 7 minutes, our API and web application served at happo.io was unresponsive. This was caused by a configuration mistake in our backend systems.

Read the full incident report →

Major April 21, 2026

GitHub statuses not working

Detected by Pingoru
Apr 21, 2026, 11:53 AM UTC
Resolved
Apr 21, 2026, 12:10 PM UTC
Duration
16m
Affected: APIWeb UI
Timeline · 2 updates
  1. identified Apr 21, 2026, 11:53 AM UTC

    We are investigating an issue with Happo not being able to post build statuses to PRs at the moment.

  2. resolved Apr 21, 2026, 12:10 PM UTC

    This issue has now been fixed.

Read the full incident report →

Critical April 18, 2026

Happo.io temporarily down

Detected by Pingoru
Apr 18, 2026, 11:26 AM UTC
Resolved
Apr 18, 2026, 11:22 AM UTC
Duration
—
Timeline · 1 update
  1. resolved Apr 18, 2026, 11:26 AM UTC

    While applying an infrastructure change to our cluster, our load balancer was misconfigured and ended up in a state where it sent HTTP traffic to the wrong nodes. The issue was discovered right away but it took a while to figure out the root cause (load balancer config). External traffic was briefly affected with API requests to happo.io timing out. The issue has since been resolved and all services are back up and running.

Read the full incident report →

Major June 23, 2025

General downtime

Detected by Pingoru
Jun 23, 2025, 03:59 PM UTC
Resolved
Jun 23, 2025, 03:59 PM UTC
Duration
—
Affected: APIChromeWeb UIFirefoxEdgeSafariiOS SafariiOS Safari (iPad)
Timeline · 2 updates
  1. resolved Jun 23, 2025, 03:59 PM UTC

    We are continuing to monitor an issue which caused most Happo jobs and API calls to fail. We are seeing things recover right now and will keep an eye on this to make sure we're not regressing again. Issue started at 15:00 UTC and was ongoing until 15:43 UTC.

  2. postmortem Jun 23, 2025, 04:27 PM UTC

    The root cause of the downtime and slowness from the API was a new database query that we deployed a few days ago. Once it had some traffic we started noticing slow queries coming from the new query. It took us a about an hour to track everything down, and we quickly reverted the code that added the new query. We will continue to monitor things but as of right now the system is stable.

Read the full incident report →

Major June 2, 2025

Increased error rates for snap-request jobs

Detected by Pingoru
Jun 02, 2025, 07:05 PM UTC
Resolved
Jun 02, 2025, 08:17 PM UTC
Duration
1h 12m
Affected: API
Timeline · 2 updates
  1. investigating Jun 02, 2025, 07:05 PM UTC

    We are investigating an issue with snap-requests randomly failing with a `HTTP error 404: http://happo-view-service/api/snap-requests/:id/payload -- Not Found` error.

  2. resolved Jun 02, 2025, 08:17 PM UTC

    We have found the root cause of this issue now. A temporary fix is currently active and we are soon to deploy a proper fix. The issue was caused by a stale cache that caused snap-request jobs to not save properly.

Read the full incident report →

Notice April 9, 2025

Service outage

Detected by Pingoru
Apr 09, 2025, 03:10 PM UTC
Resolved
Apr 09, 2025, 03:10 PM UTC
Duration
—
Timeline · 1 update
  1. resolved Apr 09, 2025, 03:10 PM UTC

    During approximately 20 minutes, happo.io and the Happo API was broken. A majority of API calls would fail with a 502 Bad Gateway error, and the website was down. We are still investigating the root cause, but for now we have all systems running again. Expect some delay while our workers pick up work from our backlog of pending jobs.

Read the full incident report →