Sansform - Notice history

Through our monitoring, we have determined that project upgrade behavior has returned to normal across the fleet. This incident is resolved.

We have implemented all necessary mitigations and the project upgrade rates have returned to normal across the fleet. We will continue to monitor these flows for any irregularities or recurring degradation. If you are still experiencing delayed or blocked upgrades on your Project, please contact Support.

We are starting to notice improvements as a result of our mitigation efforts, and expect that project upgrades should start to become less delayed. We’ll continue monitoring and will update as we observe improvements across the fleet.

We are continuing our operational and investigative work to improve upgrade flow performance and mitigate further problems.

We are implementing operational mitigations to try to improve upgrade performance.

We are continuing to work through a variety of options for improving upgrade performance. This includes working with our upstream provider and adjusting operational parameters.

We are aware that upgrades continue to be delayed. We are exploring other factors that are contributing to the slow upgrades.

The upgrade delays have been caused by resource capacity limits. We are taking actions to increase the appropriate resources.

Upgrades are taking longer than usual, but generally will complete if allowed to proceed. We are investigating the cause of the delays.

Project create and unpause operations have returned to normal and remained stable for the past 30 minutes.

We have resolved the degraded performance affecting our Management API in the impacted regions. All systems are now fully operational.

All payment features have been re-enabled. Customers should no longer experience failed payments or failed project creations.

We are continuing to monitor the system closely and will provide a final update once we have confirmed full stability.

We are currently experiencing degraded performance with our Management API in the following regions: ca-central-1, sa-east-1, us-east-1, and us-east-2.

Customers may experience failed payments, failed project creations, and degraded performance in Supabase Studio.

As a precautionary measure, we have temporarily disabled the following features while we investigate:

• Top up Credits
• Update Billing Information
• Redeem Credits

To reduce impact, we have redirected traffic to healthy regions. Our team continues to actively investigate and work toward a full resolution.

All residual jobs have drained. Storage performance and job activity is normal.

Job queues continue to drain. We will actively monitor until they are complete.

Backlog is being effectively processed and operations have returned to normal. We continue to monitor the performance in us-west-2.

We are seeing improvement in the performance of us-west-2 after implementation of our mitigation strategies. Storage operation is returning to normal. We will continue to monitor performance in this region.

We have implemented further mitigations and are tracking their progress. We continue to actively work this issue.

The team is still working on a solution, and mitigation work is underway.

We are implementing load mitigation strategies to improve the performance of storage requests in US-West-2.

The issue affecting SSO logins has now been resolved.
Thank you for your patience while we worked through this issue.

We have applied a fix and are seeing login services return to normal.
We will continue to monitor authentication success rates over the next 60 minutes to ensure stability.

A fix has been implemented and is currently being rolled out. We will continue to provide updates as the deployment progresses.

We have identified the issue as being related to a recent key rotation and are currently working on a fix.

We are currently investigating an issue causing login failures for some users attempting to authenticate to the Supabase dashboard via SSO or email/password login flows.

Affected users may see an error similar to:
“captcha protection: request disallowed (sitekey-secret-mismatch)”

Currently logged-in users are not affected, and GitHub authentication continues to operate normally.

We will continue to provide updates as more information becomes available.

Communication between read replicas and their primaries for impacted projects has been restored. This incident is now resolved.

We have merged the fix and are monitoring for stability.

We have developed a fix and it is currently being validated. We will provide more updates soon.

Our team continues to work on implementing a fix for the issue. We will provide additional updates as progress is made.

Our team continues to work on implementing a fix for the issue. We are continuing to monitor the impact and will provide additional updates as progress is made.

We have identified the cause of the issue impacting read replica communication for projects. Our team is actively working on a fix and continuing to assess the impact to affected projects.

We are reviewing the impact of this issue, which is limited to projects with read replicas and network restrictions. We continue to both investigate mitigation options for the issue and identify projects that may be impacted.

The UI message incident has been resolved.

This change has been reverted. We are monitoring to ensure things remain stable.

The revert of the change is in process.

We’ve isolated the cause of the issue to a recent change and are working on reverting it now.

All changes hav been applied.

The update has been implemented. Service changes can be made. We will monitor the effect for the next 30 minutes to ensure stability.

We've determined that the connectivity issues are isolated to users accessing Supabase through Mullvad VPN and potentially other VPN providers. The root cause appears to be network disruptions involving a specific colocation/transit provider used by Mullvad, rather than an issue with Supabase or Cloudflare infrastructure.

We've reached out to Mullvad support, but as this is dependent on external providers, there are no further mitigation steps we can take internally. Users experiencing issues are advised to avoid affected VPN networks or contact their VPN provider directly for support.

The incident is now resolved

Our investigation has identified traffic loss along a specific network path associated with a VPN exit point located in Western Europe.

Some customers are experiencing connectivity issues that impact a subset of users connecting through certain VPN providers, including Mullvad.

Customers impacted while using Mullvad can contact Mullvad support directly to help raise visibility into the issue from their side.

We are continuing to actively investigate the connectivity issues impacting access to Supabase APIs for some users.

Our team remains engaged in identifying the underlying cause and validating the scope of impact. We will continue to provide updates as our investigation progresses.

We are continuing to investigate connectivity issues impacting access to Supabase APIs, which may result in connection timeout errors or rejected connections for some users.
At this time, we have narrowed the issue down to traffic originating from at least one VPN provider (Mullvad VPN). However, we cannot yet confirm whether additional VPN providers may also be impacted.
We are actively working to identify the root cause and will continue to provide updates as we learn more.

This is now resolved. Thank you for your patience while we worked to resolve this issue.

The exceptions fix has been rolled out, and we are monitoring for continued stability.

We are continuing to work on a fix. We’ll share further updates as progress is made.

We are continuing to work on a fix for this issue.

The issue has been identified and we are working on a fix.

All projects are patched and this is now resolved. Thank you for your patience while we ensured all projects were updated.

The fix has been rolled out, and we are monitoring for continued stability. Any remaining impacted projects are being manually patched.

The permanent fix has now been rolled out across the fleet. We are continuing to manually patch any remaining impacted projects to restore service.

We’ve identified additional impacted projects and are currently working to manually patch and restore them.
We are also continuing to roll out a fix across the fleet to prevent further impact.

All impacted projects with open support tickets have now been manually patched and restored.
We are currently rolling out a fix across the fleet to prevent further impact.
We will continue to monitor progress and provide updates.

Our team is actively working on a fix and is manually patching impacted projects to restore service as quickly as possible.

If you are running a version of Postgres older than 15.1.1.57, please avoid restarting your project until this issue is resolved.

We’ve identified that a recent change to the Postgres systemd service has a dependency on a prestart script.
This script is not present on some older projects, and when those projects restart, Postgres may fail to start successfully.
This can result in projects remaining offline.
We are working on a fix to restore service.

If you are running a version of Postgres older than 15.1.1.57, please avoid restarting your project until this issue is resolved.

The issue has been resolved and DNS resolution is now operating normally.

A fix has been implemented, and we are seeing improvements in DNS resolution.
We’re closely monitoring to ensure the issue is fully resolved and services remain stable.

We are currently experiencing an issue propagating new DNS records for our zone. We are working closely with our upstream network providers technical team to implement a fix and will share updates as we learn more.

All metrics have been nominal and stable for several hours now, and our upstream provider has resolved their incident.

We've received confirmation from our networking provider that there is recovery across all regions. We are able to corroborate this with our own metrics collections. We will continue to monitor for a bit, but we believe this to be resolved at this time.

We apologize for the trouble, and we appreciate your patience.

We've seen a marked improvement in error rates across all regions in the last few minutes. We are still waiting on confirmation from our provider on status, but we believe things to be improved at the moment. We'll keep working on this until we have for sure reached a resolution.

We recognize that some of our users, especially in parts of North and South America, are continuing to experience DNS and 5XX errors when trying to connect to their projects. We have escalated this issue with our network provider and will work with them continuously until the problems are resolved.

Some users continue to experience errors. We are raising the issue with our provider.

Our upstream provider has reported their incident resolved. We will continue to monitor for any errors our customers may be experiencing for another 30 minutes.

The fix that our upstream provider implemented has taken effect. Service has significantly improved across all previously impacted regions. There are still residual elevated error rates in a handful of regions and we will continue to work with our provider to eliminate them.

Our upstream provider has rolled the fix out to most regions. While we are seeing improvement, we are still seeing increased error rates in Mexico and Brazil and are continuing to work with them until this is fully resolved.

Upstream provider mitigations are still under way, and we are seeing improvements incrementally as routing regions have the mitigations applied. We are still seeing particular impact in South America, but some users in North America may still be affected as well. We anticipate continued incremental improvements and will update as we get more information.

Our upstream networking provider has identified a fix and is currently implementing a fix. We are seeing preliminary improvements to error rates for users in North America, but are still following this issue closely with our provider.

Our upstream networking partner is continuing their investigation. We will continue to post regular updates as we have more information.

Our upstream networking provider has discovered an issue and has declared an incident on their side. They are currently working toward resolution. We are actively working with them and will provide updates on progress as more information is available.

We continue to have active lines of investigation going with our upstream network provider partners, and are still working toward a resolution.

This issue affects the network-level access to projects. The projects themselves and the data in them are safe and unaffected.

Users are seeing two separate symptoms of these networking issues:

1. DNS Lookup failures for supabase project URLs
2. 530 responses to HTTP requests to supabase project API Endpoints

We continue to work with partners and investigate.

We have further clarified the scope and impact of the issues currently affecting users. A subset of users across North and South America are experiencing DNS resolution failures and HTTP 530 errors.
These symptoms indicate a networking-related issue impacting the availability of Supabase projects. We are actively working with both internal teams and external network providers to isolate the root cause and determine the most effective path to resolution.
We will continue to provide updates as more information becomes available.

We are actively investigating this issue and working with relevant partner organizations. At this time, we do not believe this issue is specific to any particular network provider. A subset of users continue to be impacted in both South and North America.

We are currently investigating reports of users experiencing login issues when using Supabase Auth.

At this time, the impact appears to be limited to a subset of users, primarily in North and South America.
The exact cause and scope are still being determined.

Our team is actively working to identify the root of the issue and will provide updates as more information becomes available.

We appreciate your patience while we investigate.