Data mirroring for Snowflake Postgres is now generally available
Replicate data continuously from Snowflake Postgres into Snowflake with sub-minute lag, removing third-party CDC tools and ETL pipelines, with native or Iceberg support.
O que mudou
Data mirroring for Snowflake Postgres is now generally available (GA). The feature continuously replicates tables from a Snowflake Postgres instance into a Snowflake database with minimal lag, eliminating the need for third-party CDC connectors or batch ETL pipelines.
Use cases
Imagine an OLTP application recording transactions in Snowflake Postgres where analytics teams need immediate access to this data in Snowflake OLAP without impacting the transactional workload.
- Replicate individual tables or entire schemas without operating separate CDC infrastructure.
- Automatically propagate DDL operations (such as column additions or renames) without breaking replication.
- Query pending changes with approximately 30-second lag through the
$livecompanion view. - Audit or consume the last 7 days of row-level changes (inserts, updates, deletes) via the
$changestable. - Target either standard Snowflake tables or Snowflake-managed Apache Iceberg tables.
How to use
Data mirroring uses Snowflake serverless compute to apply changes, recording change logs and metadata into Iceberg via the pg_lake extension. To monitor the compute consumption of mirror apply tasks:
-- Inspect serverless task execution history for mirroring
SELECT *
FROM SNOWFLAKE.ACCOUNT_USAGE.SERVERLESS_TASK_HISTORY
WHERE TASK_NAME LIKE 'APPLY_MIRROR_%';
What happens in this code:
- Accesses the
SERVERLESS_TASK_HISTORYview inSNOWFLAKE.ACCOUNT_USAGE. - Filters tasks by the
APPLY_MIRROR_prefix assigned to the serverless task running the mirror apply procedure. - Outputs execution metrics and credit consumption for that specific mirror.
To inspect storage usage incurred by the change logs and metalog on the Postgres side:
-- Query storage usage history for Snowflake Postgres change logs
SELECT *
FROM SNOWFLAKE.ACCOUNT_USAGE.POSTGRES_STORAGE_USAGE_HISTORY
ORDER BY START_TIME DESC;
What happens in this code:
- Queries the Postgres storage account usage view.
- Returns storage consumption incurred by
$changesand metalog tables written viapg_lake.
Limitations and requirements
- Maturity: Generally Available (GA).
- Cloud support: Supported on AWS and Microsoft Azure across all regions where Snowflake Postgres is offered.
- Boundaries: Mirrors cannot be set up across different accounts, cloud providers, or regions.
- Membership changes: Adding or removing replicated objects requires
alter_mirrorwithadd_tables/remove_tablesoradd_schemas/remove_schemas; replacing the full object set in a single command is unsupported.
Using it in a pipeline
Data mirroring serves as the zero-ETL ingestion layer landing data into Snowflake Bronze/Raw tables. Downstream pipelines can query the $live view for sub-minute operational reports or read the $changes log table to drive incremental transformations toward Silver and Gold layers.
- snowflake postgres
- data mirroring
- cdc
- iceberg
- pg_lake
Data mirroring for Snowflake Postgres (General availability)
Resumo e explicação gerados por IA a partir da release note oficial. Em caso de dúvida, a documentação oficial da plataforma prevalece.
Relacionadas
- SnowflakeGA
Geração de documentos em PDF, Word e PowerPoint no Snowflake CoWork
O Snowflake CoWork agora gera arquivos PDF, Word, PowerPoint e HTML diretamente de suas análises usando a ferramenta de execução de código do Cortex Agent.
snowflake cowork · cortex agent - SnowflakePublic Preview
Cortex Agent SDK para Python e TypeScript no Snowflake
O Cortex Agent SDK permite criar aplicações que utilizam Cortex Agents e Cortex Code em Python e TypeScript.
cortex agents · python - SnowflakeAtualização
Mensagens de erro mais detalhadas na classificação de dados sensíveis
A view DATA_CLASSIFICATION_LATEST agora traz mensagens de erro específicas e códigos do Snowflake para falhas na classificação.
data governance · trust center