Write support for nested namespaces in catalog-linked databases

Catalog-linked databases now allow creating and managing Iceberg tables within nested namespaces in remote catalogs. Snowflake has also added the is_nested column to the SHOW SCHEMAS output to identify hierarchical structures.

Use cases

Imagine you organize your Iceberg tables into parent and child namespaces in an external catalog and need to write data directly to these structures from Snowflake.

• Create schemas pointing to nested namespaces.
• Write and query data in Iceberg tables located in hierarchical namespaces.
• Quickly identify nested namespaces using schema metadata.

How to use

Configure the NAMESPACE_MODE parameter to FLATTEN_NESTED_NAMESPACE and define a valid delimiter in NAMESPACE_FLATTEN_DELIMITER. Then, create the child schema using the flattened name.

CREATE SCHEMA namespace1_namespace1a;

What happens in this code:

1. Snowflake interprets the composite name using the configured delimiter.
2. A new schema corresponding to the nested namespace is created in the external catalog.

To check if a schema belongs to a nested namespace, run:

SHOW SCHEMAS IN DATABASE my_db;

What happens in this code:

1. The command lists the schemas in the catalog-linked database.
2. The is_nested column returns 'Y' for nested namespaces and 'N' for top-level namespaces.

Limitations and requirements

• The parent namespace must already exist in the remote catalog; Snowflake does not automatically create parent namespaces.
• The NAMESPACE_FLATTEN_DELIMITER parameter only supports punctuation, symbols, or digits (letters and whitespace are not allowed).
• Do not use a period (.) in the schema name unless a period is explicitly set as the delimiter.

Using it in a pipeline

Useful in the Bronze or Silver layer when integrating Snowflake with external cloud catalogs that rely on hierarchical namespaces to organize Iceberg data.