Lakeflow Designer updates: Guardrails, multi-input joins and Genie Code

Use cases

Imagine you need to build visual data preparation workflows and now have new features to validate data quality, join multiple sources, and automate tasks. Lakeflow Designer expands the capabilities of the visual canvas.

• Validate data against rules for nulls, unique values, and accepted values using the Guardrails operator.
• Combine three or more tables in a single Join operator.
• Create custom user-defined operators using natural language descriptions with Genie Code.
• Import multiple files at once as source operators.

How to use

To create a new visual data prep workflow in Databricks with Unity Catalog enabled:

1. Click New in the sidebar and select Visual data prep.
2. Add source operators by dragging files or connecting tables.
3. Configure new operators like Guardrails or Join directly in the side pane.
4. Use Genie Code in the sidebar to generate Python functions and configurations from text descriptions.

Limitations and requirements

• Requires a Databricks workspace with Unity Catalog enabled.
• Requires CAN USE permission on at least one general-purpose compute resource (serverless or all-purpose).