Snowpark for Python

Snowflake · Advanced Features

Theory

Snowpark is a set of libraries for Python, Java and Scala that let you work with Snowflake data through a DataFrame API. When you write session.table("orders").filter(col("amount") > 500).group_by("region").agg(sum_("amount")), nothing runs at first; Snowpark builds a plan and, when you call an action such as collect() or save_as_table(), turns it into SQL that runs on a Snowflake warehouse. Data stays in Snowflake instead of being pulled to your laptop.

For custom logic, you can package Python code as UDFs or stored procedures that run in a secure sandbox on the warehouse, using third-party packages that Snowflake makes available. Snowpark-optimized warehouses offer more memory per node for heavy workloads like model training.

Why we use it

Moving large data out of the warehouse to Spark or pandas is slow, costly and creates security copies. Snowpark brings the code to the data, so developers keep Python while Snowflake does the heavy lifting and governance stays in one place.

When to use it

Use Snowpark when transformations are easier in Python than SQL, for ML feature engineering, or to reuse Python libraries. Use plain SQL for straightforward transformations, since Snowpark DataFrames become SQL anyway.

Why Run Code Inside Snowflake

Data never leaves Snowflake's governed environment, so the same RBAC, masking policies, and audit logging that protect SQL queries also protect Snowpark code — a common requirement in regulated industries.

DataFrame API

Snowpark's DataFrame API mirrors pandas/PySpark syntax (.filter(), .group_by(), .select()) but Snowflake translates those operations into SQL under the hood and runs them on a virtual warehouse.

Snowflake ML

Snowflake ML (the snowflake-ml-python library, formerly called Snowpark ML) adds feature engineering, model training, a Feature Store and a Model Registry, so the whole ML lifecycle can run without exporting data to an external platform.

How it works

  1. Write DataFrame code in Python
  2. Snowpark builds a lazy plan
  3. Is an action like collect() or save_as_table() called? Yes: Generate SQL and send it to Snowflake. No: Keep building the plan.
  4. Warehouse executes the SQL and any Python UDFs
  5. Results are returned or written to a table

Example

CREATE OR REPLACE PROCEDURE analyze_orders()
RETURNS TABLE(region VARCHAR, total_revenue FLOAT)
LANGUAGE PYTHON RUNTIME_VERSION = '3.11'
PACKAGES = ('snowflake-snowpark-python') HANDLER = 'run'
AS
$$
def run(session):
    df = session.table("orders")
    return df.group_by("region").agg({"amount": "sum"}).to_df("region", "total_revenue")
$$;

CALL analyze_orders();

Key points

Snowflake topics

Snowflake Fundamentals

Warehouses & Performance

Staging & SnowSQL

Tables & Data Handling

Cloud Integrations

Storage & Optimization

Security & Governance

Views, Pipelines & Automation

Data Types & SQL Fundamentals

Semi-Structured Data Deep Dive

Advanced Features

Data Sharing & Collaboration

Cost Management & Monitoring

Databases, Schemas & Objects

Performance Deep Dive

Advanced Security & Governance

Stored Procedures & UDFs

Ecosystem & Integrations

SnowPro Certification Prep

T
Tech Town
Log InSign Up Free
Home SQL Python DSA Power BI Snowflake Databricks Spark Gen AI Practice Problems Contest Blog

Mastering Snowflake

The modern data cloud, from architecture to SnowPro certification. Click any topic in the sidebar to open it right here.

{{ sectionCount }}
Sections
{{ topicCount }}
Topics
Free
Forever
Hands-on practice Practice Snowflake SQL Sharpen the SQL you use in Snowflake: 150 query problems in a live editor with instant checking. Start practising →
Section {{ sec.num }}

{{ sec.label }}

Snowflake  /  {{ current.sectionLabel }}  /  {{ current.name }}

{{ current.name }}

Theory

{{ current.def }}

Why we use it

{{ rich.why }}

When to use it

{{ rich.when }}

{{ sec.h }}

{{ sec.body }}

{{ sec.body }}

Diagram

{{ rich.dg.caption }}

{{ tb.title }}
{{ c.v }}
{{ rich.dg.op }}
➜
{{ rich.dg.output.title }}
{{ c.v }}
{{ nd.label }}
{{ nd.sub }}
{{ nd.label }}
{{ nd.sub }}
{{ rich.dg.left.title }}
{{ it }}
{{ rich.dg.right.title }}
{{ it }}
On screen — where to click

{{ rich.ui.title }}

{{ u.text }}
  1. {{ st.n }}{{ st.text }}
{{ rich.ui.title }}
{{ u.text }}
Flowchart — how it works
{{ nd.text }}
{{ nd.text }}
◇ {{ nd.text }}
Yes → {{ nd.yes }}
No → {{ nd.no }}
Syntax
{{ current.syntax }}
Example
{{ current.example }}
Want to run this yourself? Unlike the SQL and Python courses, Snowflake needs a real account — there's no in-browser engine for it. Click Copy above, then paste it into a free Snowflake trial (30 days, no card required) and run it in Snowsight.
Key Points — click to check off as you learn
  • {{ pt.mark }} {{ pt.text }}

Share this topic