Medallion Architecture (Bronze, Silver, Gold)

Databricks · Architecture, Performance & Cost

Theory

The medallion architecture organises lakehouse data into layers of increasing quality. Bronze holds raw data as it arrived, usually append-only with load metadata such as file name and ingest time, so you can always replay history. Silver holds cleaned, typed, de-duplicated and conformed data: bad rows filtered or quarantined, keys standardised, and related sources joined, for example orders matched to customers. Gold holds business-ready tables shaped for a use case, such as daily sales by city, customer lifetime value or a star schema for BI. In Unity Catalog, layers are often separate schemas (main.bronze, main.silver, main.gold) or separate catalogs per environment.

Example: CSV order files land in bronze.orders_raw, are cleaned into silver.orders, and summarised into gold.daily_city_sales for the dashboard.

Why we use it

Mixing raw and cleaned data in one place makes errors hard to trace and fix. Layers give each step a clear job, let you rebuild silver and gold from bronze when logic changes, and give consumers stable, trusted gold tables.

When to use it

Use it as the default pattern for lakehouse ETL, implemented with Lakeflow pipelines or jobs. It is a guideline, not a rule: small projects may merge layers, and some teams add a platinum or serving layer. Avoid copying data through layers with no transformation.

How it works

  1. Raw files and SaaS data arrive
  2. Ingest as-is into bronze with load metadata
  3. Clean, type, de-duplicate and join into silver
  4. Do rows fail quality rules? Yes: Drop or send them to a quarantine table. No: Keep them in silver.
  5. Aggregate and model into gold for each use case
  6. Dashboards, Genie and ML read gold

Syntax

Typical layout:
  main.bronze.<source>_raw   -- raw, append-only (Auto Loader / Lakeflow Connect)
  main.silver.<entity>       -- cleaned, typed, de-duplicated, joined
  main.gold.<business_mart>  -- aggregates, star schema, ML features
Grants: engineers on bronze/silver, analysts SELECT on gold

Example

-- Bronze: raw files plus metadata
CREATE OR REFRESH STREAMING TABLE main.bronze.orders_raw AS
SELECT *, _metadata.file_name AS src_file, current_timestamp() AS ingested_at
FROM STREAM read_files('/Volumes/main/sales/raw/orders/', format => 'csv');

-- Silver: typed, filtered, de-duplicated
CREATE OR REFRESH MATERIALIZED VIEW main.silver.orders AS
SELECT DISTINCT CAST(order_id AS BIGINT) AS order_id, customer_id, region,
       CAST(amount AS DECIMAL(10,2)) AS amount, CAST(order_date AS DATE) AS order_date
FROM main.bronze.orders_raw
WHERE order_id IS NOT NULL;

-- Gold: business-ready aggregate
CREATE OR REFRESH MATERIALIZED VIEW main.gold.daily_city_sales AS
SELECT o.order_date, c.city, SUM(o.amount) AS sales, COUNT(*) AS orders
FROM main.silver.orders o JOIN main.sales.customers c USING (customer_id)
GROUP BY o.order_date, c.city;

Key points

Databricks topics

Databricks Fundamentals

Compute

Notebooks & Development

Apache Spark on Databricks

Delta Lake

Unity Catalog & Governance

Data Ingestion

Declarative Pipelines (Lakeflow)

Lakeflow Jobs

Databricks SQL & BI

Architecture, Performance & Cost

Machine Learning & AI

Certification & Interview

T
Tech Town
Log InSign Up Free
Home SQL Python DSA Power BI Snowflake Databricks Spark Gen AI Practice Problems Contest Blog

Master Databricks

The lakehouse platform from the ground up: Apache Spark, Delta Lake, Unity Catalog, Lakeflow pipelines and jobs, Databricks SQL and MLflow. Click any topic in the sidebar to open it right here.

{{ sectionCount }}
Sections
{{ topicCount }}
Topics
Free
Forever
Hands-on practice Practice SQL / Spark SQL Sharpen the SQL you use in Databricks SQL and Spark SQL: 150 query problems in a live editor with instant checking. Start practising →
Section {{ sec.num }}

{{ sec.label }}

Databricks  /  {{ current.sectionLabel }}  /  {{ current.name }}

{{ current.name }}

Theory

{{ current.def }}

Why we use it

{{ rich.why }}

When to use it

{{ rich.when }}

{{ sec.h }}

{{ sec.body }}

{{ sec.body }}

Diagram

{{ rich.dg.caption }}

{{ tb.title }}
{{ c.v }}
{{ rich.dg.op }}
➜
{{ rich.dg.output.title }}
{{ c.v }}
{{ nd.label }}
{{ nd.sub }}
{{ nd.label }}
{{ nd.sub }}
{{ rich.dg.left.title }}
{{ it }}
{{ rich.dg.right.title }}
{{ it }}
On screen — where to click

{{ rich.ui.title }}

{{ u.text }}
  1. {{ st.n }}{{ st.text }}
{{ rich.ui.title }}
{{ u.text }}
Flowchart — how it works
{{ nd.text }}
{{ nd.text }}
◇ {{ nd.text }}
Yes → {{ nd.yes }}
No → {{ nd.no }}
Syntax
{{ current.syntax }}
Example
{{ current.example }}
Want to run this yourself? Databricks needs a real workspace — there's no in-browser engine for it. Click Copy above, then paste it into a notebook or the SQL editor in Databricks Free Edition (free, no credit card) and run it on serverless compute.
Key Points — click to check off as you learn
  • {{ pt.mark }} {{ pt.text }}

Share this topic