Run Python, Spark, and ML workloads directly on Snowflake with Code Bundles

Code Bundles (formerly Notebook Projects) let you package non-SQL code, such as Python scripts, and run it directly on Snowflake compute. Snowflake automatically injects a Snowpark session at runtime, giving your code direct access to data without manual credential management.

Use cases

Imagine you have Python ETL scripts or ML applications and want to run them without building complex containers or wrapping logic in stored procedures.

• Run Python code on virtual warehouses or compute pools.
• Submit Spark jobs using Snowpark Connect.
• Run Machine Learning workloads using the Snowflake Container Runtime.

How to use

Create a configuration file named code_bundle.yml at the root of your project specifying the bundle type and compute target.

bundle:
  type: custom
  compute_type: warehouse
  language: python

  compute_options:
    runtime_version: '3.11'

  properties:
    requirements_file: requirements.txt

What happens in this code:

1. Sets the bundle type to custom for Python scripts.
2. Configures execution to run on a virtual warehouse.
3. Specifies the Python 3.11 runtime version.
4. References a requirements file for automatic dependency installation.

Next, create and execute the bundle using the Snowflake CLI:

snow bundle create MY_BUNDLE --source ./my_project --exclude "venv/**"
snow bundle execute MY_BUNDLE --entrypoint main.py

What happens in this code:

1. Creates a Code Bundle named MY_BUNDLE from a local directory while excluding virtual environments.
2. Executes the entrypoint script main.py directly on Snowflake compute.

Limitations and requirements

• Scheduling notebooks on compute pools is generally available (GA).
• Warehouse execution, Spark job submission, ML jobs, inline specification overrides, and the REST API, Python API, and Snowflake CLI clients are in Public Preview.
• Requires CREATE CODE BUNDLE permission on the schema or account, plus OWNERSHIP or USAGE on the Code Bundle to execute it.