Factorial HR Outage History

Factorial HR is up right now

Factorial HR had 32 outages in the last 2 years totaling 3h 3m of downtime — averaging 1.3 incidents per month.

There were 32 Factorial HR outages since June 5, 2024 totaling 3h 3m of downtime. Each is summarised below — incident details, duration, and resolution information.

Source: https://status.factorialhr.com

Minor September 30, 2024

Degraded performance on Factorial app

Detected by Pingoru
Sep 30, 2024, 07:41 AM UTC
Resolved
Sep 30, 2024, 08:51 AM UTC
Duration
1h 9m
Affected: Factorial website
Timeline · 4 updates
  1. investigating Sep 30, 2024, 07:41 AM UTC

    The performance of the application is heavily degraded since 9:00 CEST. We are investigating the source of the issue to restore the service as soon as possible.

  2. investigating Sep 30, 2024, 08:23 AM UTC

    We have decided to fail over the database to the secondary instance in another availability zone. This should resolve the issue in a matter of minutes.

  3. monitoring Sep 30, 2024, 08:37 AM UTC

    The fail-over to the secondary database has improved the situation as expected. We are monitoring the recovery and looking at side-effects of the situation before marking the incident fully resolved.

  4. resolved Sep 30, 2024, 08:51 AM UTC

    We are pleased to inform you that the performance degradation issue has been successfully resolved. Our team has conducted a thorough investigation and identified enhancements to our monitoring systems. These improvements will enable us to detect and address similar situations more effectively in the future. We appreciate your patience and understanding during this incident. Thank you for your continued support.

Read the full incident report →

Major August 28, 2024

Delay in time tracking calculations

Detected by Pingoru
Aug 28, 2024, 12:00 PM UTC
Resolved
Aug 28, 2024, 01:19 PM UTC
Duration
1h 19m
Affected: Factorial website
Timeline · 2 updates
  1. monitoring Aug 28, 2024, 12:40 PM UTC

    The system in charge of computing the time tracking totals for display on the application interface is experiencing unusual delay since 12:00 UTC. While the shifts were being registered, today's totals may not include the most recent ones. Our team has submitted a fix and we are hoping to see a recovery in a matter of minutes.

  2. resolved Aug 28, 2024, 01:19 PM UTC

    The issue has been resolved. There may be some inconsistencies in the calculation that will eventually be reconciled with the shifts in the database.

Read the full incident report →

Major August 26, 2024

Core component failure made Factorial application unavailable

Detected by Pingoru
Aug 26, 2024, 02:34 PM UTC
Resolved
Aug 26, 2024, 12:00 PM UTC
Duration
Timeline · 1 update
  1. resolved Aug 26, 2024, 02:34 PM UTC

    An incident occurred today, between 14:22 and 14:49 CEST, which affected the performance of our application and website. During this time, a failure in a core component of our infrastructure resulted in slower response times, increased error rates, and, ultimately, service unavailability. Our incident response team acted swiftly to identify the issue and successfully replaced the failing component, restoring full service shortly thereafter. Following the incident, our infrastructure team conducted an investigation to understand the root cause of the failure. We have since implemented improvements to our configuration to prevent similar issues from occurring in the future. We sincerely apologize for any inconvenience this may have caused and appreciate your understanding as we continue to enhance the reliability of our services. Thank you for your continued support.

Read the full incident report →

Critical July 15, 2024

Factorial application unavailable

Detected by Pingoru
Jul 15, 2024, 11:50 AM UTC
Resolved
Jul 15, 2024, 12:13 PM UTC
Duration
22m
Affected: Factorial website
Timeline · 3 updates
  1. investigating Jul 15, 2024, 11:50 AM UTC

    Due to an error introduced in our latest release, the Factorial application is currently unavailable or partially loading. Our teams have identified the source of the problem and are investigating a fix to be deployed as soon as possible.

  2. identified Jul 15, 2024, 11:58 AM UTC

    A fix is underway - we expect the service to be restored in the next hour.

  3. resolved Jul 15, 2024, 12:13 PM UTC

    The service has been restored; app.factorialhr.com is fully operational again. We apologize for the inconvenience caused and will perform an investigation to ensure such errors don't happen again in the future.

Read the full incident report →

Minor July 9, 2024

Elevated error rates

Detected by Pingoru
Jul 09, 2024, 11:56 AM UTC
Resolved
Jul 09, 2024, 12:09 PM UTC
Duration
12m
Affected: API & backendFactorial website
Timeline · 2 updates
  1. monitoring Jul 09, 2024, 11:56 AM UTC

    Our monitoring systems have detected higher error rates than usual. In most cases these are timeouts caused by a malfunctioning system. Our Engineers have applied remediation and we are confirming the recovery of the service levels back to normal.

  2. resolved Jul 09, 2024, 12:09 PM UTC

    The affected system has been replaced. This incident is now resolved.

Read the full incident report →

Notice July 1, 2024

Brief service interruption during Database migration

Detected by Pingoru
Jul 01, 2024, 02:58 PM UTC
Resolved
Jul 01, 2024, 02:58 PM UTC
Duration
Affected: API & backend
Timeline · 1 update
  1. resolved Jul 01, 2024, 02:58 PM UTC

    As part of our continuous efforts to improve the application and its performance, an unanticipated short service interruption has been noticed while upgrading our database services. We apologize for the inconvenience this event may have caused our customers and will improve our protocols to ensure this kind of interruption does not reoccur.

Read the full incident report →

Major June 5, 2024

Full outage after routing misconfiguration

Detected by Pingoru
Jun 05, 2024, 07:00 AM UTC
Resolved
Jun 05, 2024, 07:00 AM UTC
Duration
Timeline · 1 update
  1. resolved Jun 05, 2024, 10:31 AM UTC

    Our team have introduced a misconfiguration at 09:20 CEST with an automated deployment, immediate action have been taken and the sevice were restored at 09:28. Despite our validation processes this introduced an unwanted change that triggered a second downtime at 10:00 CEST. Our emergency procedure have been launched and we restored our services at 10:27 CEST. We are committed to delivering exceptional services and we are constantly reviewing all processes to avoid similar inconveniences in the future.

Read the full incident report →