Share serverless compute among notebooks in Asset Bundles
Allow notebooks and files in the same Asset Bundle to share serverless compute and an environment defined in the pyproject.toml file.
What changed
The Shared Serverless for Asset Bundles feature allows multiple notebooks and files within the same bundle to use the same serverless compute and a centralized environment. This simplifies project execution and dependency management.### Use casesImagine you have a project with multiple notebooks that require the same libraries and dependencies. With this feature, you centralize the configuration and avoid recreating environments for each file. What you can do:- Share serverless compute among files in the same bundle.- Define dependencies and environment uniformly in the pyproject.toml file.- Optimize compute resource usage in projects based on Databricks Asset Bundles.### How to useTo use this feature, workspace administrators must first enable Project-Scoped Shared Compute on the Databricks Previews page. Then, configure your project environment in the pyproject.toml file at the root of the bundle.### Limitations and requirements- The feature is in Beta.- It is not intended for production use.- Available for workspace admins to enable on the Previews page.
- asset bundles
- serverless
- compute
- notebooks
- pyproject.toml
Shared Serverless for Asset Bundles (Beta)
Summary and explanation generated by AI from the official release note. When in doubt, the platform's official documentation prevails.
Related
- DatabricksPublic Preview
DataFrame caching arrives on Databricks serverless compute
Now in Public Preview: call df.cache() and df.persist() on Databricks serverless compute to materialize expensive results into snapshots and speed up session reuses.
serverless compute · pyspark - DatabricksBeta
Lakeflow Connect adds managed connector for Box audit logs
Databricks released the Box Audit Logs managed connector in Beta for Lakeflow Connect, enabling direct ingestion of enterprise audit events into Unity Catalog.
lakeflow connect · ingestion - DatabricksBeta
Configure Serverless Job Environments Using pyproject.toml
Serverless notebook and Python script tasks can now use a pyproject.toml file to define their job environment and manage project dependencies.
serverless · jobs