Sansform - Notice history

A fix for this issue has been implemented and our data shows previous impacted users have recovered.

A fix for this issue has been implemented and our data shows recovery for users on projects that have previously been affected. We expect this situation to be resolved, but will keep monitoring for a while until fully resolving this incident.

While we have observed significant improvement in rates of high latency, we note that some users are still experiencing issues during peak Eastern US business hours. We continue to work with our networking partners to isolate and resolve any lingering sources of high latency. We recognize that this is a longstanding issue and appreciate the patience of affected customers. We will continue to share additional updates here as soon as they are available.

Our networking partners have implemented a mitigation to the latency issue at the problematic PoP in the Eastern US, and we have since observed significant improvement in rates of high latency across the region. We are continuing to work with those partners to isolate any additional root causes of latency and improve the reliability of our networking infrastructure across the region.

We are seeing signs of improvement and are continuing to work alongside our networking partners towards a resolution.
We will provide another update as more information becomes available.

We're continuing to coordinate with our networking partners to deliver a permanent resolution.
We will provide updates as soon as they're available.

We are working with our networking partners on a permanent fix.

While we do not have an estimated resolution time, we will share updates here as they become available.

A mitigation for this issue was applied earlier today, but we do not have sufficient evidence to conclude that the mitigation resulted in improvement to this eastern US latency issue. We have tied the issue to a particular PoP, and we are continuing to work with our networking partners to bring this issue to a final resolution. We will provide another update within 4 hours, if not before. We appreciate your patience.

We are continuing our work to fix this latency issue, which has been observed to impact clients connecting to Supabase Projects from the eastern United States. These clients may include servers and serverless functions located in that geographical region, regardless of the region in which the Project is hosted.

This incident has been resolved.

We are continuing to monitor to ensure ongoing stability.

The remaining projects have now been brought back up, and we have no projects remaining on the unhealthy underlying hardware in the upstream infrastructure. We are monitoring to ensure ongoing stability.

Most projects have been brought back up, and the team is working on the remaining projects, which should be completed soon.

This issue was related to specific underlying hardware in the region, so this was isolated to only projects on that hardware. Project lifecycle operations (restarts, upgrades, etc) broadly across the region remain available.

All projects affected by the upstream infrastructure issues, and mitigation efforts are underway internally. These affected projects will be brought back up on new hardware as quickly as possible.

We have isolated the scope of the problem to a specific upstream incident. We are working to address the upstream incident and determine mitigations.

This incident has been resolved.
Administrative operations in the Supabase Dashboard are completing as expected, and we have observed no further permission related errors following the fix.

We have applied a fix, and administrative operations in the Supabase Dashboard are now completing as expected.
We are monitoring the results to ensure the issue is fully resolved.

We have identified the issue causing some users to encounter permission-related errors, such as “You need additional permissions,” when performing administrative actions in the Supabase Dashboard.
We are working on a fix and will provide another update once it has been implemented.

Storage is now operating normally. We apologise for the disruption.

Hardening measures are being put in place to prevent recurrence and will be communicated upon completion.

A fix has been implemented for the affected tenants and we are monitoring the results. We will continue to monitor the situation and will provide further updates as more information becomes available.

This incident has been resolved.

A fix has been implemented and we are monitoring the results.

We've identified the issue affecting project creation and are implementing a fix. Users may experience delays when creating new projects while we work to resolve this.

Project creation has returned to baseline consistently.

Project creation is reporting as recovered across the platform. We will monitor for stability in the near term.

We are seeing the mitigation steps having the desired effect. Project creation is recovering.

While the first set of actions are taking effect, we are implementing a second set of likely mitigations to attempt to facilitate improvement in project creation.

We continue to work this issue. The steps we have taken are addressing the hypothesized cause of the issue. We are implementing further related mitigations.

We have a working hypothesis on the cause of the issue, and are implementing steps to try to improve project creation performance.

The fix has remained stable across all regions, and the issue has been resolved.

A fix has been rolled out across all regions, and results are positive.
We will continue monitoring to ensure the service remains stable.
Moving this incident to monitoring.

An enhanced automated solution is currently being tested in staging.
It is expected to be more effective and efficient than the approach used previously.
Once testing has completed successfully, we will use this approach to roll out the fix in production.
We will provide further updates as this work progresses.

We are continuing to rollout the fix to further regions.

This issue is fixed in eu-central-2 and ap-south-1. We are continuing to rollout the fix to all regions.

Some users on Project versions 15.1 and older are experiencing a pgBouncer configuration issue causing connection issues for the Dedicated Pooler and Storage services. We are continuing our investigation of this issue.

The issue affecting automatic disk resizing has been resolved.
We will continue to follow up directly with affected users who opened a support ticket.

A fix has been applied, and the issue is now resolved. We are moving the incident to monitoring.
We will continue to follow up directly with affected users who opened a support ticket.

Project upgrades have remained stable, we are resolving this incident.

We have deployed a tested fix for the upgrade issue impacting some projects. The fix has been effective and we are monitoring for continued stability.

We are continuing to test the fix to project upgrades. We will update again when the fix is applied.

The team has identified a likely fix and is consolidating the details in order to move to thorough testing. We will update again when the fix is applied.

We are continuing to fix project upgrade issues and are investigating further remediation. We will provide our next update within the next 14 hours.

We are developing a fix for the upgrade issues.

We have identified the cause of the upgrade delays and are working on a fix.

We are continuing to investigate project upgrade issues.