Shopify connector in Lakeflow Connect for data engineering
The new Shopify connector in Lakeflow Connect brings e-commerce data directly into Databricks with support for incremental ingestion.
O que mudou
Use cases
Imagine you need to centralize sales, customer, and product data from an online store into Databricks. With this managed connector, you can automate your e-commerce data ingestion.
- Ingest products, orders, customers, inventory, fulfillments, and discounts.
- Capture deletions of items like products, variants, and collections.
- Integrate seamlessly with Unity Catalog and Databricks Workflows.
How to use
Configuration is done in three main steps:
- Configure your Shopify app to use OAuth M2M (Client Credentials) authentication.
- Create a connection in Catalog Explorer using the Databricks UI.
- Create and run your ingestion pipeline using the UI, API, or Databricks Asset Bundles.
Limitations and requirements
- This feature is in Beta and must be enabled by a workspace admin from the Previews page.
- Supported authentication is OAuth M2M (Client Credentials); basic auth and OAuth U2M are not supported.
- Data type changes and column renames require a full refresh.
Using it in a pipeline
Use this connector in your Bronze layer to ingest raw e-commerce data from Shopify before transforming it into clean Silver and Gold tables.
- lakeflow connect
- shopify
- unity catalog
- ingestion
- pipelines
Shopify connector (Beta)
Resumo e explicação gerados por IA a partir da release note oficial. Em caso de dúvida, a documentação oficial da plataforma prevalece.
Relacionadas
- DatabricksGA
Repositórios de pacotes em nível de workspace para Databricks Apps
Administradores de workspace podem configurar repositórios padrão de pacotes Python e npm para Databricks Apps.
databricks apps · workspace settings - DatabricksGA
Redimensione partições de stateful streaming sem perder o checkpoint no Databricks
Agora é possível alterar o número de partições de shuffle em queries Structured Streaming e pipelines Lakeflow sem descartar o estado do checkpoint.
structured streaming · lakeflow - DatabricksAtualização
Google Gemini 4 Argon disponível no Unity Gateway do Databricks
O modelo multimodal Google Gemini 4 Argon já pode ser acessado via Foundation Model APIs e Unity Gateway.
unity catalog · foundation model apis