Custom runtime images now generally available for ML Jobs

Custom runtime images for ML Jobs are now generally available, allowing you to use specific libraries, meet security requirements, and integrate with CI/CD pipelines. Imagine you need to run machine learning training workloads with exact dependencies not included in the default Snowflake environment; with CREs, you package everything into a container image derived from the official base and register it. ### Use cases - Comply with strict enterprise security and compliance policies - Integrate image building into existing CI/CD pipelines - Ensure exact package and version reproducibility across environments ### How to use To register and use a custom image, follow these steps using the Snowflake CLI and SQL commands. 1. Log in to the image registry: bash snow spcs image-registry login What happens in this code: - Authenticates the local Docker client with the Snowflake image registry using the Snowflake CLI. 2. Create the Custom Runtime Environment via SQL: sql CREATE CUSTOM RUNTIME ENVIRONMENT my_cre IMAGE_PATH = '/mydb/myschema/my_repo/sklearn_custom:v1' BASE_IMAGE_TYPE = CPU; What happens in this code: - Registers a new CRE object in Snowflake pointing to the image path in the repository. - Automatically performs a server-side validation check during execution. 3. Reference the environment in a machine learning job: python from snowflake.ml import jobs @jobs.remote( "MY_COMPUTE_POOL", stage_name="payload_stage", runtime_environment="cre@my_cre" ) def test_function(): ... job = test_function() What happens in this code: - Submits the ML Job using the @jobs.remote decorator. - Configures the runtime_environment parameter to use the registered my_cre environment. ### What it returns The DESCRIBE CUSTOM RUNTIME ENVIRONMENT command returns the following columns: Field | Description --- | --- name | Custom runtime environment name. image_name | Image name. tag | Image tag. digest | SHA256 digest of the image. created_on | Date and time when the custom runtime environment was created. updated_on | Date and time when the environment was last updated. owner | Role that owns the custom runtime environment. cre_status | State of the environment (RESOLVED or DEPRECATED). base_image_type | Hardware type (CPU or GPU). image | Image path in the image repository. ### Limitations and requirements - Custom runtime images for ML Jobs are Generally Available (GA), while support for Snowflake Notebooks remains in preview. - Requires snowflake-ml-python version 1.37.0 or higher. - Custom images must derive from an official Snowflake ML base image and include required packages and entrypoint. - The active role must hold the USAGE privilege on the target CRE and access to the image repository. ### Using it in a pipeline This fits into the MLOps training and model scoring stages, allowing data engineering teams to package complex dependencies during the build phase (CI/CD) to guarantee reliable executions in production workflows.