Task DAGs & Orchestration

Snowflake · Views, Pipelines & Automation

Theory

A task graph, often called a DAG (directed acyclic graph), links tasks so that finishing one starts the next. Only the root task has a SCHEDULE; every child task declares AFTER one or more predecessor tasks and runs when all of them finish successfully. For example, dag_extract runs at 2 AM, then dag_transform runs AFTER dag_extract, and dag_load runs AFTER dag_transform.

Each task in the graph must be resumed for it to run. SYSTEM$TASK_DEPENDENTS_ENABLE('root_task') resumes the root and all its children in one call. To change a child task you suspend the root first. A finalizer task can be added to run after the whole graph, for example for cleanup or notifications.

Why we use it

Real pipelines have several steps that must run in order. A task graph expresses those dependencies inside Snowflake, so a step never runs on half-finished data from the step before.

When to use it

Use task graphs for multi-step SQL pipelines inside Snowflake. Use an external orchestrator when steps span other systems or need complex retries and alerts across tools, and dynamic tables when dependencies can be declared as queries.

Building a DAG

Only the root task carries a SCHEDULE; every child task instead declares AFTER <parent_task>, so it runs the moment its parent finishes successfully rather than on its own clock.

The Extract → Transform → Load → Validate Flow

A common pattern chains four steps end to end: task_extract (SCHEDULE) │ AFTER ▼ task_transform │ AFTER ▼ task_validate Each step only starts once the previous one has actually completed.

Resuming a DAG

Tasks are created SUSPENDED. Resume leaf tasks first and work back up to the root — resuming the root before its children exist as active tasks means they'd never get triggered; suspending the root task pauses the entire DAG downstream of it.

How it works

  1. Root task schedule fires
  2. Root task runs (extract)
  3. Did all predecessors of the next task succeed? Yes: Run the child task. No: Stop this branch; run is marked failed.
  4. Children run in dependency order (transform, load, validate)
  5. Finalizer task, if defined, runs at the end

Example

CREATE OR REPLACE TASK dag_extract
  WAREHOUSE = etl_wh SCHEDULE = 'USING CRON 0 2 * * * UTC'
AS INSERT INTO staging_customers (raw_data, loaded_at) SELECT event_data, CURRENT_TIMESTAMP() FROM raw_events;

CREATE OR REPLACE TASK dag_transform
  WAREHOUSE = etl_wh AFTER dag_extract
AS INSERT INTO customers (first_name, email)
  SELECT raw_data:first_name::STRING, raw_data:email::STRING FROM staging_customers;

CREATE OR REPLACE TASK dag_validate
  WAREHOUSE = etl_wh AFTER dag_transform
AS INSERT INTO temp_analysis (metric, value) SELECT 'new_customers', COUNT(*) FROM customers;

ALTER TASK dag_validate RESUME;
ALTER TASK dag_transform RESUME;
ALTER TASK dag_extract RESUME;

Key points

Snowflake topics

Snowflake Fundamentals

Warehouses & Performance

Staging & SnowSQL

Tables & Data Handling

Cloud Integrations

Storage & Optimization

Security & Governance

Views, Pipelines & Automation

Data Types & SQL Fundamentals

Semi-Structured Data Deep Dive

Advanced Features

Data Sharing & Collaboration

Cost Management & Monitoring

Databases, Schemas & Objects

Performance Deep Dive

Advanced Security & Governance

Stored Procedures & UDFs

Ecosystem & Integrations

SnowPro Certification Prep

T
Tech Town
Log InSign Up Free
Home SQL Python DSA Power BI Snowflake Databricks Spark Gen AI Practice Problems Contest Blog

Mastering Snowflake

The modern data cloud, from architecture to SnowPro certification. Click any topic in the sidebar to open it right here.

{{ sectionCount }}
Sections
{{ topicCount }}
Topics
Free
Forever
Hands-on practice Practice Snowflake SQL Sharpen the SQL you use in Snowflake: 150 query problems in a live editor with instant checking. Start practising →
Section {{ sec.num }}

{{ sec.label }}

Snowflake  /  {{ current.sectionLabel }}  /  {{ current.name }}

{{ current.name }}

Theory

{{ current.def }}

Why we use it

{{ rich.why }}

When to use it

{{ rich.when }}

{{ sec.h }}

{{ sec.body }}

{{ sec.body }}

Diagram

{{ rich.dg.caption }}

{{ tb.title }}
{{ c.v }}
{{ rich.dg.op }}
➜
{{ rich.dg.output.title }}
{{ c.v }}
{{ nd.label }}
{{ nd.sub }}
{{ nd.label }}
{{ nd.sub }}
{{ rich.dg.left.title }}
{{ it }}
{{ rich.dg.right.title }}
{{ it }}
On screen — where to click

{{ rich.ui.title }}

{{ u.text }}
  1. {{ st.n }}{{ st.text }}
{{ rich.ui.title }}
{{ u.text }}
Flowchart — how it works
{{ nd.text }}
{{ nd.text }}
◇ {{ nd.text }}
Yes → {{ nd.yes }}
No → {{ nd.no }}
Syntax
{{ current.syntax }}
Example
{{ current.example }}
Want to run this yourself? Unlike the SQL and Python courses, Snowflake needs a real account — there's no in-browser engine for it. Click Copy above, then paste it into a free Snowflake trial (30 days, no card required) and run it in Snowsight.
Key Points — click to check off as you learn
  • {{ pt.mark }} {{ pt.text }}

Share this topic