Sansform - Upgrade Failures – Incident details

Experiencing partially degraded performance

Upgrade Failures

Resolved
Degraded performance
Started about 20 hours agoLasted about 5 hours

Affected

Supabase

Operational from 2:20 PM to 2:20 PM, Degraded performance from 2:20 PM to 6:51 PM

Supabase → Database

Operational from 2:20 PM to 2:20 PM, Degraded performance from 2:20 PM to 6:51 PM

Updates
  • Resolved
    Resolved

    Through our monitoring, we have determined that project upgrade behavior has returned to normal across the fleet. This incident is resolved.

  • Monitoring
    Monitoring

    We have implemented all necessary mitigations and the project upgrade rates have returned to normal across the fleet. We will continue to monitor these flows for any irregularities or recurring degradation. If you are still experiencing delayed or blocked upgrades on your Project, please contact Support.

  • Update
    Update

    We are starting to notice improvements as a result of our mitigation efforts, and expect that project upgrades should start to become less delayed. We’ll continue monitoring and will update as we observe improvements across the fleet.

  • Update
    Update

    We are continuing our operational and investigative work to improve upgrade flow performance and mitigate further problems.

  • Update
    Update

    We are implementing operational mitigations to try to improve upgrade performance.

  • Update
    Update

    We are continuing to work through a variety of options for improving upgrade performance. This includes working with our upstream provider and adjusting operational parameters.

  • Update
    Update

    We are aware that upgrades continue to be delayed. We are exploring other factors that are contributing to the slow upgrades.

  • Identified
    Identified

    The upgrade delays have been caused by resource capacity limits. We are taking actions to increase the appropriate resources.

  • Update
    Update

    Upgrades are taking longer than usual, but generally will complete if allowed to proceed. We are investigating the cause of the delays.

  • Investigating
    Investigating

    We are investigating potential upgrade issues in multiple regions. We are working to confirm impact and determine next steps.