Storage credentials & external locations

Databricks · Unity Catalog & Governance

Theory

Unity Catalog needs permission to read and write your cloud storage. A storage credential wraps a cloud identity: an IAM role on AWS, an Azure managed identity (access connector) or a service principal, or a Google service account. An external location combines a storage credential with a cloud path, such as s3://techtown-landing/, and is the object you grant privileges on. Both are created by admins or users with the right metastore privileges.

Users never see the cloud keys. Instead they get privileges on the external location, such as READ FILES, WRITE FILES, CREATE EXTERNAL TABLE and CREATE EXTERNAL VOLUME, and Unity Catalog hands out short-lived credentials when a query touches that path. For example, the admin creates the credential landing_cred for an IAM role, then the external location landing_loc on s3://techtown-landing/ and grants READ FILES to the data-engineers group.

Why we use it

Before Unity Catalog, teams pasted access keys into notebooks or used workspace mounts that anyone could read. Storage credentials and external locations keep secrets out of code and control access to cloud paths with normal GRANT statements.

When to use it

Create an external location for every bucket or container path Databricks must reach: landing zones, external tables, external volumes and managed storage roots. Grant on external locations, not on storage credentials, for everyday users.

How it works

  1. Cloud admin creates an IAM role or managed identity
  2. Databricks admin creates a storage credential for it
  3. Create an external location: credential + cloud path
  4. Grant READ FILES or CREATE EXTERNAL TABLE to a group
  5. Does the user's query touch a path under that location? Yes: Unity Catalog checks privileges and vends a temporary token. No: Access is denied.
  6. Data is read or written securely

Syntax

CREATE STORAGE CREDENTIAL (usually via UI: Catalog → gear icon → Credentials)
CREATE EXTERNAL LOCATION name URL 's3://bucket/path' WITH (STORAGE CREDENTIAL cred)
GRANT READ FILES, WRITE FILES ON EXTERNAL LOCATION name TO `group`
GRANT CREATE EXTERNAL TABLE ON EXTERNAL LOCATION name TO `group`

Example

-- Admin: bind a path to an existing storage credential
CREATE EXTERNAL LOCATION IF NOT EXISTS landing_loc
  URL 's3://techtown-landing/'
  WITH (STORAGE CREDENTIAL landing_cred)
  COMMENT 'Raw landing zone';

GRANT READ FILES, CREATE EXTERNAL TABLE
  ON EXTERNAL LOCATION landing_loc TO `data-engineers`;

-- Engineer: read files directly from the path
SELECT * FROM read_files('s3://techtown-landing/orders/', format => 'json');

SHOW EXTERNAL LOCATIONS;

Key points

Databricks topics

Databricks Fundamentals

Compute

Notebooks & Development

Apache Spark on Databricks

Delta Lake

Unity Catalog & Governance

Data Ingestion

Declarative Pipelines (Lakeflow)

Lakeflow Jobs

Databricks SQL & BI

Architecture, Performance & Cost

Machine Learning & AI

Certification & Interview

T
Tech Town
Log InSign Up Free
Home SQL Python DSA Power BI Snowflake Databricks Spark Gen AI Practice Problems Contest Blog

Master Databricks

The lakehouse platform from the ground up: Apache Spark, Delta Lake, Unity Catalog, Lakeflow pipelines and jobs, Databricks SQL and MLflow. Click any topic in the sidebar to open it right here.

{{ sectionCount }}
Sections
{{ topicCount }}
Topics
Free
Forever
Hands-on practice Practice SQL / Spark SQL Sharpen the SQL you use in Databricks SQL and Spark SQL: 150 query problems in a live editor with instant checking. Start practising →
Section {{ sec.num }}

{{ sec.label }}

Databricks  /  {{ current.sectionLabel }}  /  {{ current.name }}

{{ current.name }}

Theory

{{ current.def }}

Why we use it

{{ rich.why }}

When to use it

{{ rich.when }}

{{ sec.h }}

{{ sec.body }}

{{ sec.body }}

Diagram

{{ rich.dg.caption }}

{{ tb.title }}
{{ c.v }}
{{ rich.dg.op }}
➜
{{ rich.dg.output.title }}
{{ c.v }}
{{ nd.label }}
{{ nd.sub }}
{{ nd.label }}
{{ nd.sub }}
{{ rich.dg.left.title }}
{{ it }}
{{ rich.dg.right.title }}
{{ it }}
On screen — where to click

{{ rich.ui.title }}

{{ u.text }}
  1. {{ st.n }}{{ st.text }}
{{ rich.ui.title }}
{{ u.text }}
Flowchart — how it works
{{ nd.text }}
{{ nd.text }}
◇ {{ nd.text }}
Yes → {{ nd.yes }}
No → {{ nd.no }}
Syntax
{{ current.syntax }}
Example
{{ current.example }}
Want to run this yourself? Databricks needs a real workspace — there's no in-browser engine for it. Click Copy above, then paste it into a notebook or the SQL editor in Databricks Free Edition (free, no credit card) and run it on serverless compute.
Key Points — click to check off as you learn
  • {{ pt.mark }} {{ pt.text }}

Share this topic