ai_parse_document now extracts chart data into JSON

The ai_parse_document function now extracts layout information and structured numerical data from charts embedded in unstructured documents. When a chart is detected, the function provides detailed descriptions and outputs its underlying data as structured JSON.

Use cases

Imagine you process market analysis PDFs containing bar and line charts showing quarterly revenue. Previously, you had to inspect images manually or rely on textual summaries without structured metrics. Now you can query numeric series data directly in SQL or Spark jobs to:

• Ingest time-series and categorical metrics displayed visually inside charts.
• Feed multimodal RAG and analytics pipelines with precise figures instead of approximations.
• Eliminate external OCR tools customized for data charts.

How to use

Call ai_parse_document passing the binary content of the document. Optionally pin the schema version to 2.0 and enable figure descriptions by setting 'descriptionElementTypes', '*'. Chart data is populated in the content property for figure element types.

SELECT
  path,
  ai_parse_document(
    content,
    map(
      'version', '2.0',
      'descriptionElementTypes', '*'
    )
  ) AS parsed_doc
FROM READ_FILES('/Volumes/my_catalog/my_schema/my_volume/documents/', format => 'binaryFile');

What happens in this code:

1. READ_FILES reads the documents from the Unity Catalog volume using the binaryFile format.
2. ai_parse_document processes the binary payload under the version 2.0 schema.
3. The 'descriptionElementTypes', '*' parameter activates AI-generated descriptions and structured parsing for figures.

To isolate extracted chart series data directly:

WITH parsed AS (
  SELECT
    path,
    ai_parse_document(content, map('version', '2.0', 'descriptionElementTypes', '*')) AS doc
  FROM READ_FILES('/Volumes/my_catalog/my_schema/my_volume/reports/', format => 'binaryFile')
)
SELECT
  path,
  elem.description AS figure_description,
  elem.content AS chart_json_data
FROM parsed,
LATERAL EXPLODE(doc:document:elements) AS t(elem)
WHERE elem.type = 'figure'
  AND elem.content IS NOT NULL;

What happens in this code:

1. The CTE parses source files from the designated volume path.
2. LATERAL EXPLODE flattens the elements array from the VARIANT document output.
3. The WHERE clause filters for figure elements with non-null content, returning the chart JSON with series, labels, and axis values.

What it returns

When the identified element is a readable chart, the content string inside elements returns structured JSON (illustrative values):

Field	Description
title	Title identified within the chart (e.g., "Quarterly revenue")
values	Nested key-value dictionary containing series and data points (e.g., {"Revenue": {"Q1": 12, "Q2": 15}})
additional_info	Contextual details or axis labels parsed from the figure

Limitations and requirements

• Requires Databricks Runtime 17.3 or above.
• For serverless compute, environment version 3 or above is required for VARIANT support.
• Supports binary inputs from PDF, JPG/JPEG, PNG, TIFF/TIF, DOC/DOCX, and PPT/PPTX files.
• 500-page limit per document (documents exceeding this limit fail unless constrained with pageRange).
• Costs are tracked under the AI_FUNCTIONS SKU; subject to regional Foundation Model API availability.

Using it in a pipeline

Store raw PDFs as binary files in a Bronze table. In Silver, invoke ai_parse_document to extract all elements. Filter type = 'figure' and parse the JSON content column into typed tabular rows for ingestion into downstream Gold fact tables.