Cross-engine attribute-based access controls (ABAC) now generally available

Cross-engine attribute-based access controls (ABAC) are now generally available. External engines can read Unity Catalog tables with fine-grained access policies, row filters, and column masks enforced dynamically via a Databricks serverless compute layer.

Use cases

Imagine you need to query Unity Catalog data from an external engine like Apache Spark or Iceberg while keeping strict security and privacy rules enforced. With this feature, you can:

• Query Unity Catalog tables from external engines.
• Ensure ABAC policies, row filters, and column masks are consistently applied.
• Monitor serverless compute costs associated with the cross-engine processing.

How to use

To create a new managed Delta table supporting catalog commits, use the following SQL command:

CREATE TABLE meu_catalogo.meu_schema.minha_tabela (id INT, name STRING)
USING delta
TBLPROPERTIES ('delta.feature.catalogManaged' = 'supported');

What happens in this code:

1. Creates a new managed Delta table.
2. Enables the catalog managed feature property required for ABAC support.

To upgrade an existing table, run:

ALTER TABLE meu_catalogo.meu_schema.minha_tabela
SET TBLPROPERTIES ('delta.feature.catalogManaged' = 'supported');

What happens in this code:

1. Modifies an existing managed Delta table.
2. Adds support for catalog managed features.

To configure Apache Spark with Delta for reading the table, set the catalog options:

"spark.sql.extensions": "io.delta.sql.DeltaSparkSessionExtension",
"spark.sql.catalog.spark_catalog": "org.apache.spark.sql.delta.catalog.DeltaCatalog",
"spark.sql.catalog.meu_catalogo": "io.unitycatalog.spark.UCSingleCatalog",
"spark.sql.catalog.meu_catalogo.uri": "https://workspace-url.cloud.databricks.com",
"spark.sql.catalog.meu_catalogo.ServerSidePlanning.enabled": "true",
"spark.sql.defaultCatalog": "meu_catalogo"

What happens in this code:

1. Configures Spark extensions and the Unity Catalog connector.
2. Enables server-side planning to enforce fine-grained access control from external engines.

Limitations and requirements

• Generally available (GA).
• Requires external data access enabled on the Unity Catalog metastore and the EXTERNAL_USE_SCHEMA privilege granted.
• Only read operations are supported when fine-grained access controls are enforced.
• Dynamic views, projecting VARIANT columns, filtering on BINARY columns, and column-masking functions with return types differing from the original column type are not supported.