Managed SurveyMonkey connector arrives in Databricks Lakeflow Connect

The managed SurveyMonkey connector is now available in Beta within Lakeflow Connect. It enables direct ingestion of surveys, responses, and contact lists into Databricks using incremental synchronization.

Use cases

Imagine you run NPS or customer satisfaction surveys on SurveyMonkey and need to combine that feedback with transactional customer data in your Lakehouse. Instead of maintaining custom Python scripts or third-party integration pipelines, the connector automates the entire ingestion lifecycle.

It supports ingesting:

• Surveys and survey structures
• Collectors and recipient metadata
• Form responses
• Contacts and contact lists

How to use

Setup does not require custom extraction code and follows three main steps:

1. Configure SurveyMonkey: set up permissions in SurveyMonkey to allow authentication with Databricks.
2. Create a Unity Catalog connection: define a connection in Catalog Explorer using OAuth user-to-machine (U2M) credentials.
3. Create the ingestion pipeline: author the pipeline using the Databricks web UI, REST APIs, or Declarative Automation Bundles (DABs), with orchestration handled by Lakeflow Jobs.

Limitations and requirements

• Stage: Beta. A workspace admin must enable Lakeflow Connect for SurveyMonkey from the Previews page.
• Authentication: only OAuth user-to-machine (U2M) is supported. OAuth machine-to-machine (M2M) and basic authentication (API key or username/password) are unsupported.
• SCD Type 2: tables containing VARIANT columns (surveys, collectors, survey_responses, contact_list_contacts, and collector_recipients) do not support SCD Type 2.
• Schema evolution: automatically handles new and deleted columns. Data type alterations and column renames are not supported and require a full refresh.
• Filtering: API-based column selection is supported, but API-based row filtering is not supported.

Using it in a pipeline

In a standard medallion architecture, the connector lands raw SurveyMonkey tables into a Bronze layer within Unity Catalog. Downstream pipelines can then flatten semi-structured fields stored in VARIANT columns to populate curated Silver tables for business reporting.