Metastore & three-level namespace

Databricks · Unity Catalog & Governance

Theory

Unity Catalog is Databricks' governance layer for data and AI. At the top is the metastore: one per cloud region, owned at the account level and attached to every workspace in that region, so permissions and objects are shared across those workspaces. Inside the metastore, objects are organized in a three-level namespace: catalog.schema.object. A catalog is the top grouping (often per environment or business unit, such as dev, prod or main); a schema (also called a database) groups related objects; and objects are tables, views, volumes, functions and models.

You refer to a table by its full name, main.sales.orders, or set defaults with USE CATALOG main and USE SCHEMA sales. Built-in catalogs include system (system tables) and samples, plus hive_metastore for legacy tables. For example, the same orders table can be read from the dev and prod workspaces in one region because both are attached to the same metastore.

Why we use it

Before Unity Catalog, each workspace had its own Hive metastore with separate permissions, so the same data was governed differently in each place. One metastore and a clear namespace give one set of permissions, lineage and audit for all workspaces.

When to use it

Always use three-level names in production code so it does not depend on the current catalog. Design catalogs around environments or domains (dev, prod, finance) and schemas around subject areas. Treat hive_metastore as legacy and migrate tables out of it.

How it works

  1. Query names main.sales.orders
  2. Resolve catalog main in the region's metastore
  3. Resolve schema sales inside main
  4. Resolve table orders inside sales
  5. Does the user have USE CATALOG, USE SCHEMA and SELECT? Yes: Read the table's files. No: Fail with a permission error.
  6. Return the rows

Syntax

<catalog>.<schema>.<table | view | volume | function | model>
CREATE CATALOG [IF NOT EXISTS] name
CREATE SCHEMA [IF NOT EXISTS] catalog.schema
USE CATALOG main;  USE SCHEMA sales;
SHOW CATALOGS | SHOW SCHEMAS IN main | SHOW TABLES IN main.sales

Example

CREATE CATALOG IF NOT EXISTS main;
CREATE SCHEMA IF NOT EXISTS main.sales
  COMMENT 'Sales orders and customers';

USE CATALOG main;
USE SCHEMA sales;

SHOW TABLES IN main.sales;

-- Always-safe full name
SELECT c.city, SUM(o.amount) AS revenue
FROM main.sales.orders o
JOIN main.sales.customers c ON o.customer_id = c.customer_id
GROUP BY c.city;

Key points

Databricks topics

Databricks Fundamentals

Compute

Notebooks & Development

Apache Spark on Databricks

Delta Lake

Unity Catalog & Governance

Data Ingestion

Declarative Pipelines (Lakeflow)

Lakeflow Jobs

Databricks SQL & BI

Architecture, Performance & Cost

Machine Learning & AI

Certification & Interview

T
Tech Town
Log InSign Up Free
Home SQL Python DSA Power BI Snowflake Databricks Spark Gen AI Practice Problems Contest Blog

Master Databricks

The lakehouse platform from the ground up: Apache Spark, Delta Lake, Unity Catalog, Lakeflow pipelines and jobs, Databricks SQL and MLflow. Click any topic in the sidebar to open it right here.

{{ sectionCount }}
Sections
{{ topicCount }}
Topics
Free
Forever
Hands-on practice Practice SQL / Spark SQL Sharpen the SQL you use in Databricks SQL and Spark SQL: 150 query problems in a live editor with instant checking. Start practising →
Section {{ sec.num }}

{{ sec.label }}

Databricks  /  {{ current.sectionLabel }}  /  {{ current.name }}

{{ current.name }}

Theory

{{ current.def }}

Why we use it

{{ rich.why }}

When to use it

{{ rich.when }}

{{ sec.h }}

{{ sec.body }}

{{ sec.body }}

Diagram

{{ rich.dg.caption }}

{{ tb.title }}
{{ c.v }}
{{ rich.dg.op }}
➜
{{ rich.dg.output.title }}
{{ c.v }}
{{ nd.label }}
{{ nd.sub }}
{{ nd.label }}
{{ nd.sub }}
{{ rich.dg.left.title }}
{{ it }}
{{ rich.dg.right.title }}
{{ it }}
On screen — where to click

{{ rich.ui.title }}

{{ u.text }}
  1. {{ st.n }}{{ st.text }}
{{ rich.ui.title }}
{{ u.text }}
Flowchart — how it works
{{ nd.text }}
{{ nd.text }}
◇ {{ nd.text }}
Yes → {{ nd.yes }}
No → {{ nd.no }}
Syntax
{{ current.syntax }}
Example
{{ current.example }}
Want to run this yourself? Databricks needs a real workspace — there's no in-browser engine for it. Click Copy above, then paste it into a notebook or the SQL editor in Databricks Free Edition (free, no credit card) and run it on serverless compute.
Key Points — click to check off as you learn
  • {{ pt.mark }} {{ pt.text }}

Share this topic