Loading Data from Azure ADLS
Snowflake · Cloud Integrations
Theory
Once an external stage points at an Azure container, loading works exactly as with S3 or GCS. COPY INTO orders FROM @my_azure_stage/orders/ FILE_FORMAT = my_csv_format reads the files, and the same options apply: PATTERN, FILES, ON_ERROR, VALIDATION_MODE and load metadata that skips files already loaded.
Only the stage definition is cloud-specific: the azure:// URL and the integration. Everything after that, including file formats, COPY options and transformations in the COPY SELECT, is the same. For continuous loading, Snowpipe on Azure uses Event Grid notifications through a notification integration, the counterpart of S3 event notifications.
Why we use it
Teams often have data in more than one cloud. Because COPY and file formats are cloud-neutral, you learn the load workflow once and only change the stage when the source moves.
When to use it
Use scheduled COPY from Azure stages for batch loads, and Snowpipe with Event Grid for continuous loads. Organise containers by date or source folder so PATTERN or path prefixes can load just the new data.
Same Workflow as S3
Once an external stage points at an ADLS container, COPY INTO loads files from it with the exact same syntax used for S3 or GCS — same file format objects, same pattern matching.
Switching Clouds Without Relearning
Because the load syntax doesn't change between providers, a pipeline written for S3 can point at ADLS instead with just a new stage definition — no rewritten COPY INTO logic.
Incremental Loading by Path
COPY INTO tracks already-loaded files by path just as it does for S3, so scheduled reruns against a growing ADLS container only pick up newly arrived files.
How it works
- Files arrive in the Azure container
- LIST @my_azure_stage/orders/ to check them
- COPY INTO orders with a file format and ON_ERROR rule
- Were any rows rejected? Yes: Inspect with VALIDATE() and fix the source. No: Continue.
- Schedule the COPY or move to Snowpipe with Event Grid
Example
COPY INTO orders FROM @my_azure_stage/orders/
FILE_FORMAT = my_csv_format
ON_ERROR = 'SKIP_FILE';Key points
- File format objects handle CSV, JSON and Parquet the same way on every cloud
- Same COPY INTO syntax works across S3, Azure and GCS
- Supports incremental loads by file pattern or path
- Load metadata prevents reloading the same file
- Continuous Azure loading needs a notification integration for Event Grid