Lakeflow Connect adds managed connector for Box audit logs

Databricks has released the managed Box Audit Logs connector in Lakeflow Connect in Beta. This feature enables teams to ingest enterprise audit events directly from Box into governed tables in Databricks.

Use cases

Imagine your security and compliance teams need to centralize file access logs, user activities, and administrative changes from Box alongside other enterprise logs. With this connector, you can:

• Ingest Box enterprise audit events incrementally and automatically.
• Govern ingested security logs directly under Unity Catalog.
• Schedule and orchestrate ingestion runs through Databricks Workflows or Declarative Automation Bundles (DABs).

How to use

Setup is handled through Unity Catalog connections and Lakeflow ingestion pipelines. The workflow consists of:

1. Configure Box: Set up Box Server Authentication using the OAuth Machine-to-Machine flow with Client Credentials Grant (CCG).
2. Create a Unity Catalog connection: A workspace admin configures the connection in Catalog Explorer or via API, enabling pipeline creation.
3. Create the ingestion pipeline: Admins or data engineers create the ingestion pipeline using the UI, API, or Declarative Automation Bundles linked to the existing connection.

Limitations and requirements

• Stage: Available in Beta. A workspace administrator must activate Lakeflow Connect for Box Audit Logs on the Previews page.
• Authentication: Supports only OAuth M2M (Box Server Authentication with CCG). Basic authentication and OAuth U2M are not supported.
• Schema evolution: Supports new and deleted columns. Column renames are processed as an addition of a new column and deletion of the previous column. Changes to data types are not supported.
• Unsupported features: API-based row filtering and native SCD Type 2 are not supported.

Using it in a pipeline

Raw audit logs are ingested incrementally into a Bronze Delta table managed by Unity Catalog. Downstream pipelines can clean and parse activity types in the Silver layer for anomaly detection and aggregate metrics in the Gold layer for audit reporting.