SnowDDL
Search
⌃K
🐍

Architecture overview

SnowDDL can be used directly in your code as Python package to build custom automation.
import snowddl

Components

SnowDDL consists of the following components:
  • ​Blueprint - dataclass representing the desired state of an object in Snowflake account;
  • ​Config - collection of blueprints;
  • ​Parsers - used to parse YAML config files into blueprints, one parser class per object type;
  • ​Resolvers - used to compare blueprints with existing metadata in Snowflake account and generate DDL commands to apply changes, one resolver class per object type;
  • ​Engine - initialized with Snowflake connection and SnowDDL config, used to build, format and execute commands by resolvers.

Workflow

These components are normally combined into "applications" with the following workflow:
  1. 1.
    Initialize an empty config.
  2. 2.
    Generate individual blueprints using parsers or custom Python code, add blueprints to config.
  3. 3.
    Validate config. If encountered any config errors, display error messages and stop execution.
  4. 4.
    Open connection to Snowflake using Python connector.
  5. 5.
    Initialize engine using Snowflake connection and config.
  6. 6.
    Execute resolvers to generate DDL commands and suggest or apply changes.
  7. 7.
    Display statistics and close the application.
Default application used by CLI interface is available on GitHub: base.py​

Extensions

You may start with default application, overload it and adjust according to your specific needs.
For example:
  • generate blueprints for some tables and views dynamically, based on external data source;
  • generate blueprints for users dynamically, based on information from HR systems;
  • add a new object type, with its own parser and resolver;
  • add more fine-grained DDL command classification to engine, on top of existing "safe" and "unsafe" categories;
  • send error logs to centralized monitoring system of your organization;
  • store execution results of each query to OLTP DMBS (MySQL, PostgreSQL, etc.) for audit and future analysis;
When you start writing a custom code using SnowDDL, make sure to pin the current "snowddl" version in dependencies and keep an eye on CHANGELOG.

​

Single DB - Previous
Overview
Next
Blueprints
Last modified 11mo ago