Redesigned MLflow Logs tab adds filtering and live tailing for AI Runtime

The Logs tab on the MLflow run page has been redesigned for workloads running on AI Runtime. This update replaces the log files previously kept on the Artifacts tab and adds interactive debugging tools directly in the UI.

Use cases

Imagine you launched a distributed training job across multiple nodes on AI Runtime and need to diagnose why a single worker failed, without manually downloading large text logs from the Artifacts tab.

With the redesigned tab, you can:

• Isolate driver and worker output from a specific node in a multi-node workload.
• Filter log lines by severity level (info, warning, or error) and time range.
• Stream output in real time (live tailing) as the workload executes.
• Search for specific keywords across output and download only the filtered lines.

How to use

In the workspace interface, navigate to the MLflow run page and click the Logs tab. While a workload is running, the view follows new output as it arrives; scrolling up pauses the auto-follow behavior.

If you prefer the command line, you can inspect and download the same workload logs using the Databricks CLI:

# Stream logs for the entire workload
databricks air logs <job-run-id>

# Filter logs from node 2 only
databricks air logs <job-run-id> --node 2

# Download log files to a local directory
databricks air logs <job-run-id> --download-to ./logs/

What happens in this code:

1. The command retrieves output streams directly from the job run backing the AI Runtime workload.
2. The --node flag isolates output to inspect worker-specific behavior in distributed jobs.
3. The --download-to flag stores the log files on your local machine for offline analysis.

Limitations and requirements

• Retention: logs remain available in the Logs tab for 60 days.
• Permissions: access to the MLflow experiment alone does not grant log access. Users must have at least Can View permissions on the underlying Databricks job run.
• Legacy runs: runs created prior to this update retain their log files on the Artifacts tab. For new runs, storing logs on the Artifacts tab is deprecated.
• Version: MLflow 3.7 or above is recommended when using these tracking and observability features.