Databricks AI Runtime SDK replaces serverless_gpu

Use cases

Imagine you need to train machine learning models using multiple GPUs in a notebook without managing Spark clusters. With the Databricks AI Runtime SDK, you can:

• Use the @distributed decorator to run distributed training across all GPUs on the node.
• Integrate with PyTorch DDP, FSDP, and DeepSpeed.
• Automatically track metrics in MLflow.

How to use

Import the decorator and apply it to your training function:

from databricks.air import distributed

@distributed()
def train():
    import os
    import torch
    import torch.distributed as dist

    local_rank = int(os.environ["LOCAL_RANK"])
    torch.cuda.set_device(local_rank)
    device = torch.device(f"cuda:{local_rank}")
    dist.init_process_group("nccl")
    # ... your training code here ...
    dist.destroy_process_group()

train.distributed()

What happens in this code:

1. The function is decorated for distributed execution.
2. The process is bound to its respective GPU using the LOCAL_RANK environment variable.
3. The NCCL process group is initialized for communication between GPUs.
4. The .distributed() method is called to trigger parallel execution.

Limitations and requirements

• This feature is in Beta.
• Available in GPU environment version 6 and above.
• Supports only a single node per decorator execution.
• The default execution timeout is 3 hours.