Lakeflow Connect Managed Connectors

Databricks · Data Ingestion

Theory

Lakeflow Connect is the ingestion part of Lakeflow. Its managed connectors pull data from SaaS apps and databases (for example Salesforce, Workday, ServiceNow, Google Analytics, SharePoint, SQL Server, MySQL and PostgreSQL) into Unity Catalog streaming tables with a guided UI and no code. Under the hood each connector creates a Lakeflow pipeline that loads incrementally, using change data capture or cursor columns where the source supports it, on serverless compute. Credentials are stored as a Unity Catalog connection.

Example: the sales ops team wants Salesforce Accounts and Opportunities in Databricks. Click Data Ingestion → Salesforce, choose the connection, select the objects, set destination main.salesforce, add a daily schedule and click Save and run pipeline. Tables refresh incrementally from then on.

Why we use it

Building and maintaining API or CDC code for every business system is slow and fragile. Managed connectors handle authentication, incremental loading, schema changes and retries, and land the data governed in Unity Catalog.

When to use it

Use Lakeflow Connect when a managed connector exists for your source. For files in storage use Auto Loader or COPY INTO. For sources without a connector, use Lakehouse Federation to query in place, or write custom JDBC or API code in a job. Check the docs for which connectors are GA.

How it works

  1. Sidebar → Data Ingestion
  2. Pick the source under Databricks connectors (e.g. Salesforce)
  3. Does a connection already exist? Yes: Select it. No: Create connection (admin privilege needed).
  4. Name the pipeline, select objects and the destination catalog/schema
  5. Add a schedule and failure notifications
  6. Save and run pipeline: streaming tables appear in main.salesforce

Syntax

UI: Data Ingestion → Add data → Databricks connectors → <source>
  Connection → Ingestion setup → Source → Destination → Schedules and notifications → Save and run pipeline
Result: a managed ingestion pipeline writing streaming tables in <catalog>.<schema>

Example

-- After the Salesforce connector pipeline has run
SELECT Name, Industry, BillingCity
FROM main.salesforce.account
WHERE BillingCity IN ('Jaipur', 'Pune', 'Delhi');

-- Join SaaS data with lakehouse data
SELECT c.name, c.city, COUNT(o.order_id) AS orders
FROM main.sales.customers c
JOIN main.salesforce.account a ON a.Name = c.name
LEFT JOIN main.sales.orders o ON o.customer_id = c.customer_id
GROUP BY c.name, c.city;

Key points

Databricks topics

Databricks Fundamentals

Compute

Notebooks & Development

Apache Spark on Databricks

Delta Lake

Unity Catalog & Governance

Data Ingestion

Declarative Pipelines (Lakeflow)

Lakeflow Jobs

Databricks SQL & BI

Architecture, Performance & Cost

Machine Learning & AI

Certification & Interview

T
Tech Town
Log InSign Up Free
Home SQL Python DSA Power BI Snowflake Databricks Spark Gen AI Practice Problems Contest Blog

Master Databricks

The lakehouse platform from the ground up: Apache Spark, Delta Lake, Unity Catalog, Lakeflow pipelines and jobs, Databricks SQL and MLflow. Click any topic in the sidebar to open it right here.

{{ sectionCount }}
Sections
{{ topicCount }}
Topics
Free
Forever
Hands-on practice Practice SQL / Spark SQL Sharpen the SQL you use in Databricks SQL and Spark SQL: 150 query problems in a live editor with instant checking. Start practising →
Section {{ sec.num }}

{{ sec.label }}

Databricks  /  {{ current.sectionLabel }}  /  {{ current.name }}

{{ current.name }}

Theory

{{ current.def }}

Why we use it

{{ rich.why }}

When to use it

{{ rich.when }}

{{ sec.h }}

{{ sec.body }}

{{ sec.body }}

Diagram

{{ rich.dg.caption }}

{{ tb.title }}
{{ c.v }}
{{ rich.dg.op }}
➜
{{ rich.dg.output.title }}
{{ c.v }}
{{ nd.label }}
{{ nd.sub }}
{{ nd.label }}
{{ nd.sub }}
{{ rich.dg.left.title }}
{{ it }}
{{ rich.dg.right.title }}
{{ it }}
On screen — where to click

{{ rich.ui.title }}

{{ u.text }}
  1. {{ st.n }}{{ st.text }}
{{ rich.ui.title }}
{{ u.text }}
Flowchart — how it works
{{ nd.text }}
{{ nd.text }}
◇ {{ nd.text }}
Yes → {{ nd.yes }}
No → {{ nd.no }}
Syntax
{{ current.syntax }}
Example
{{ current.example }}
Want to run this yourself? Databricks needs a real workspace — there's no in-browser engine for it. Click Copy above, then paste it into a notebook or the SQL editor in Databricks Free Edition (free, no credit card) and run it on serverless compute.
Key Points — click to check off as you learn
  • {{ pt.mark }} {{ pt.text }}

Share this topic