apps/docs/content/guides/database/replication/pipelines-faq.mdx
<$Partial path="pipelines-public-alpha.mdx" />
Pipelines currently supports BigQuery as the managed destination. See the BigQuery destination guide for configuration details. You can request early access to ClickHouse, Snowflake, and DuckLake. Supported destinations can be Supabase-managed or third-party systems as support expands.
Yes. The further apart your source database, the pipeline, and your destination are, the more network latency is added to replication, which increases replication lag and reduces throughput.
Managed Pipelines run in AWS eu-central-1 (Frankfurt). For the best performance, place both your source project and destination as close to Frankfurt as possible. When configuring a destination provider such as BigQuery, choose its closest available region.
If you can only optimize one side, prioritize placing your destination close to the pipeline's region. Replicated data continuously streams out to the destination, so latency on that leg has a bigger impact on overall replication lag than latency between the pipeline and the source.
Pipelines requires a Pro, Team, or Enterprise plan. During public alpha, access is being rolled out gradually, so an eligible plan does not guarantee that Pipelines is enabled for every organization yet. If it is not available, request access from the Database > Replication page or contact your account manager.
We are currently working on a new Supabase Warehouse product designed to address the limitations of the previous Analytics Buckets. Our goal is to build a solution we can confidently stand behind, rather than continuing to support an approach that does not meet the quality and flexibility we want for our users.
As a result, replication into Analytics Buckets via Pipelines is no longer supported. Right now, BigQuery is the only supported managed destination, and we are actively working on expanding capabilities.
When you enable Pipelines, Supabase installs database objects that help track replication state and support schema changes:
ALTER TABLE statement. Supabase uses this to support schema change handling.etl schema. These tables track replication state for your pipelines.The replication state tables are not updated very often, especially after the initial sync is complete.
Schema change support is currently in beta and limited to the BigQuery destination.
Supported schema changes:
NULLABLE in BigQuery)NOT NULL constraintInitial BigQuery table creation preserves whether each scalar, non-array Postgres column allows NULL. Arrays use BigQuery's REPEATED mode. For later changes, DROP NOT NULL relaxes a BigQuery REQUIRED column to NULLABLE, while SET NOT NULL leaves an existing BigQuery column nullable and logs a warning. Newly added scalar, top-level columns are always nullable in BigQuery. Supported defaults are applied separately as destination metadata and do not populate existing destination rows. Pipelines does not currently support changing column data types. See BigQuery schema change support for details.
Disabling Pipelines removes the database objects that were installed in your source database, including the replication state tables in the etl schema and the DDL event trigger.
To remove Pipelines from a project, delete all Pipelines destinations first. After all destinations are deleted, the disable action becomes available. For the Dashboard steps, see Disabling Pipelines.
Disabling Pipelines stops Supabase from managing replication for the project. It does not delete tables or data that were already written to your destination.
Common reasons:
Check your publication settings and verify your table meets the requirements.
Custom data types replicate as strings. Check that your destination can interpret those string values correctly.
Postgres controls this with the publication's publish_via_partition_root setting. If the setting is false, or if you created the publication manually with SQL and did not set it, Postgres publishes changes from the leaf partitions. Pipelines then creates destination tables for those leaf partitions. If publish_via_partition_root = true, Postgres publishes changes as the partition root, so the partition hierarchy is treated as the published partition root.
Publications created from the Dashboard replication flow use publish_via_partition_root = true.
See Partitioned tables for examples and the full behavior.
If inserts replicate but updates or deletes fail, the source table might not be sending enough old-row identity through Postgres logical replication.
Every table sent to BigQuery must have a primary key, and every primary-key column must be included in the publication. For updates and deletes, use the primary key replica identity or REPLICA IDENTITY FULL. Full replica identity is also recommended for tables with large text, jsonb, bytea, or other values that Postgres can store out of line.
alter table public.your_table replica identity full;
Full replica identity increases WAL volume and only affects new WAL records. Fix the setting before generating more changes. If the failing update is already retained in WAL, restarting may fail again; recreate the pipeline or restart the affected table's initial sync. See BigQuery source table requirements for details.
After modifying your Postgres publication, you must restart the replication pipeline for changes to take effect. See Adding or removing tables for instructions.
A pipeline enters Failed when it encounters a non-retryable pipeline-level error during startup or ongoing replication. The pipeline stops instead of silently skipping the failure. To recover:
See Handling errors for more details.
A pipeline is stopped when you select Stop pipeline, when its project becomes inactive, or while an operation requires it to restart. A non-retryable configuration, schema, or data error can instead put the pipeline in Failed state. Transient connection and destination errors retry automatically when possible.
To recover:
If the same error continues after restart, the pipeline may stop again. Review Monitor pipeline status for troubleshooting steps.
Lag increases when Postgres produces WAL faster than the pipeline can confirm it has processed. Common causes include a slow or rate-limited destination, a pipeline issue, heavy source database activity, long transactions, network latency between the pipeline and source database, or a stopped/disconnected pipeline.
Open Database > Replication, click View pipeline, and check Waiting to sync, WAL retention remaining, Last check-in, Connected, and Slot status. See Dealing with replication lag for the full investigation and response flow.
Lost slot status mean?Lost means Postgres has already removed WAL files that the pipeline's replication slot needed. The pipeline cannot continue from that slot.
You can recreate the pipeline, or open the pipeline's Advanced settings, set Invalidated slot behavior to Recreate, and start the pipeline again. The pipeline resets its saved table-sync state, creates a new replication slot, and replaces each destination table through a new initial sync. This destructive restart is required for consistency because the old slot can no longer provide every change the pipeline missed, and the data processed during the new initial sync is billed again.
See Slot statuses for all slot states and what to do next.
Table errors occur during the initial sync. To recover, click View pipeline, find the affected table, and click its restart action. This restarts that table's initial sync from the beginning, deletes its existing destination data, and bills the successfully processed row data again.
Check the Database > Replication section of the Dashboard:
See Monitor pipeline status for comprehensive monitoring instructions.
You can manage your pipeline using the actions menu in the destinations list. See Managing your pipeline for details on available actions.
<Admonition type="note">Stopping replication causes changes to queue up in the WAL.
Stopping requests a graceful shutdown, so the pipeline can remain Stopping for up to five minutes while in-flight work finishes. Configured pipeline-hour billing continues while the pipeline is stopped. Delete the destination to end that charge.
</Admonition>If your project becomes inactive, Pipelines stops any running pipelines and does not automatically resume them after the project is restarted.
After restarting the project, restart each replication pipeline manually from the Database > Replication section of the Dashboard.
When a project is downgraded to the Free Plan, all replication pipelines created with Pipelines for that project are deleted.
Don't delete or modify tables or views managed by Pipelines. For BigQuery, deleting a managed object can stop replication and may require a new, billable initial sync. Pipelines does not guarantee that it will automatically repair or fully resynchronize a destination object that was removed manually.
To permanently remove a table from your destination you have two options:
Option 1: End replication permanently
Option 2: Remove from publication first
ALTER PUBLICATION ... DROP TABLERemoving a table from the publication and restarting the pipeline does not delete the table downstream, it only stops replicating new changes to it.
</Admonition>Yes. Pipelines uses at-least-once processing. Failed destination write attempts that Pipelines retries are not counted. Data is counted only after the destination acknowledges successful processing.
In rare cases, Pipelines can count an acknowledged batch but crash or be interrupted before its replication checkpoint is persisted. Recovery can then process and count the same data again.
BigQuery uses the replicated source primary key and CDC ordering metadata to converge on the current table state. Pipelines does not provide a history of each delivery attempt that you can query or guarantee exactly-once event processing.
Navigate to the Logs > Replication section of the Dashboard to see all pipeline logs. Logs contain diagnostic information. If you're experiencing issues, contact support with your error details.
If you need assistance: