Model Serving

Databricks · Machine Learning & AI

Theory

Model Serving (often called Mosaic AI Model Serving in older material) deploys models as REST API endpoints on serverless compute that scales up with traffic and, optionally, down to zero when idle. You can serve your own custom models from Unity Catalog (any MLflow model, such as the churn classifier), foundation models hosted by Databricks through the Foundation Model APIs, and external models such as OpenAI or Anthropic through governed endpoints. One endpoint can serve several model versions with a traffic split for A/B tests. Requests can be logged to Unity Catalog inference tables for monitoring, and you can call an endpoint from apps over REST or from SQL with ai_query.

Example: Ravi creates endpoint churn-scorer serving main.sales.churn_model version 3. The CRM app posts a customer's recent order count and spend and gets a churn probability back in milliseconds.

Why we use it

Batch scoring is too slow when an app needs a prediction during a request. Model Serving turns a registered model into a secure, autoscaling API without you building containers, load balancers or GPU clusters.

When to use it

Use Model Serving for real-time or low-latency predictions and for LLM endpoints used by apps and agents. Use batch inference (spark_udf or ai_query over a table in a job) for scoring millions of rows on a schedule, which is cheaper than calling an endpoint row by row.

How it works

  1. Model registered in Unity Catalog
  2. Serving → Create serving endpoint, choose model and version
  3. Pick compute size and whether to scale to zero
  4. Endpoint becomes Ready
  5. Called by an app in real time? Yes: POST to /serving-endpoints/<name>/invocations. No: Use ai_query or batch scoring in a job.
  6. Monitor latency, errors and inference tables

Syntax

UI: Serving → Create serving endpoint → Name → Served entities: Entity (My models- Unity Catalog), Version, traffic %, compute size, Scale to zero → Create
REST: POST https://<workspace>/serving-endpoints/<name>/invocations  {"dataframe_records": [...]}
SQL: ai_query('<endpoint>', <request>)

Example

# Query the endpoint from Python with the Databricks SDK
from databricks.sdk import WorkspaceClient
w = WorkspaceClient()
resp = w.serving_endpoints.query(
    name="churn-scorer",
    dataframe_records=[{"orders_90d": 2, "spend_90d": 1800.0, "days_since_last": 41}])
print(resp.predictions)

# Or with plain REST
# curl -X POST https://<workspace>/serving-endpoints/churn-scorer/invocations \
#   -H "Authorization: Bearer $TOKEN" -H "Content-Type: application/json" \
#   -d '{"dataframe_records": [{"orders_90d": 2, "spend_90d": 1800.0, "days_since_last": 41}]}'

-- Or from SQL over a table
SELECT customer_id,
       ai_query('churn-scorer', named_struct('orders_90d', orders_90d,
                'spend_90d', spend_90d, 'days_since_last', days_since_last)) AS churn_score
FROM main.sales.customer_features;

Key points

Databricks topics

Databricks Fundamentals

Compute

Notebooks & Development

Apache Spark on Databricks

Delta Lake

Unity Catalog & Governance

Data Ingestion

Declarative Pipelines (Lakeflow)

Lakeflow Jobs

Databricks SQL & BI

Architecture, Performance & Cost

Machine Learning & AI

Certification & Interview

T
Tech Town
Log InSign Up Free
Home SQL Python DSA Power BI Snowflake Databricks Spark Gen AI Practice Problems Contest Blog

Master Databricks

The lakehouse platform from the ground up: Apache Spark, Delta Lake, Unity Catalog, Lakeflow pipelines and jobs, Databricks SQL and MLflow. Click any topic in the sidebar to open it right here.

{{ sectionCount }}
Sections
{{ topicCount }}
Topics
Free
Forever
Hands-on practice Practice SQL / Spark SQL Sharpen the SQL you use in Databricks SQL and Spark SQL: 150 query problems in a live editor with instant checking. Start practising →
Section {{ sec.num }}

{{ sec.label }}

Databricks  /  {{ current.sectionLabel }}  /  {{ current.name }}

{{ current.name }}

Theory

{{ current.def }}

Why we use it

{{ rich.why }}

When to use it

{{ rich.when }}

{{ sec.h }}

{{ sec.body }}

{{ sec.body }}

Diagram

{{ rich.dg.caption }}

{{ tb.title }}
{{ c.v }}
{{ rich.dg.op }}
➜
{{ rich.dg.output.title }}
{{ c.v }}
{{ nd.label }}
{{ nd.sub }}
{{ nd.label }}
{{ nd.sub }}
{{ rich.dg.left.title }}
{{ it }}
{{ rich.dg.right.title }}
{{ it }}
On screen — where to click

{{ rich.ui.title }}

{{ u.text }}
  1. {{ st.n }}{{ st.text }}
{{ rich.ui.title }}
{{ u.text }}
Flowchart — how it works
{{ nd.text }}
{{ nd.text }}
◇ {{ nd.text }}
Yes → {{ nd.yes }}
No → {{ nd.no }}
Syntax
{{ current.syntax }}
Example
{{ current.example }}
Want to run this yourself? Databricks needs a real workspace — there's no in-browser engine for it. Click Copy above, then paste it into a notebook or the SQL editor in Databricks Free Edition (free, no credit card) and run it on serverless compute.
Key Points — click to check off as you learn
  • {{ pt.mark }} {{ pt.text }}

Share this topic