Sansform - Some Projects Unreachable following Project Restart – Incident details

Experiencing partially degraded performance

Some Projects Unreachable following Project Restart

Resolved
Partial outage
Started about 2 months agoLasted about 9 hours

Affected

Supabase

Operational from 10:54 AM to 10:54 AM, Partial outage from 10:54 AM to 12:39 PM, Operational from 12:39 PM to 1:12 PM, Partial outage from 1:12 PM to 1:31 PM, Operational from 1:31 PM to 2:21 PM, Partial outage from 2:21 PM to 3:00 PM, Operational from 3:00 PM to 4:05 PM, Partial outage from 4:05 PM to 4:22 PM, Operational from 4:22 PM to 8:05 PM

Supabase → Dashboard

Operational from 10:54 AM to 10:54 AM, Partial outage from 10:54 AM to 12:39 PM, Operational from 12:39 PM to 1:12 PM, Partial outage from 1:12 PM to 1:31 PM, Operational from 1:31 PM to 2:21 PM, Partial outage from 2:21 PM to 3:00 PM, Operational from 3:00 PM to 4:05 PM, Partial outage from 4:05 PM to 4:22 PM, Operational from 4:22 PM to 8:05 PM

Updates
  • Resolved
    Resolved

    All projects are patched and this is now resolved. Thank you for your patience while we ensured all projects were updated.

  • Monitoring
    Monitoring

    The fix has been rolled out, and we are monitoring for continued stability. Any remaining impacted projects are being manually patched.

  • Update
    Update

    The permanent fix has now been rolled out across the fleet. We are continuing to manually patch any remaining impacted projects to restore service.

  • Identified
    Identified

    We’ve identified additional impacted projects and are currently working to manually patch and restore them.
    We are also continuing to roll out a fix across the fleet to prevent further impact.

  • Monitoring
    Monitoring

    All impacted projects with open support tickets have now been manually patched and restored.
    We are currently rolling out a fix across the fleet to prevent further impact.
    We will continue to monitor progress and provide updates.

  • Update
    Update

    Our team is actively working on a fix and is manually patching impacted projects to restore service as quickly as possible.

    If you are running a version of Postgres older than 15.1.1.57, please avoid restarting your project until this issue is resolved.

  • Update
    Update

    We’ve identified that a recent change to the Postgres systemd service has a dependency on a prestart script.
    This script is not present on some older projects, and when those projects restart, Postgres may fail to start successfully.
    This can result in projects remaining offline.
    We are working on a fix to restore service.

    If you are running a version of Postgres older than 15.1.1.57, please avoid restarting your project until this issue is resolved.

  • Identified
    Identified

    We’ve identified that a recent change to the Postgres systemd service has a dependency on a prestart script.
    This script is not present on some older projects, and when those projects restart, Postgres may fail to start successfully.
    This can result in projects remaining offline. We are working on a fix to restore service.