SnowDDL
Search
⌃K

STAGE

Config path: /<database>/<schema>/stage/<name>.yaml
Example for internal stage:
file_format: test_csv_format
copy_options:
on_error: ABORT_STATEMENT
enforce_length: true
Example of external stage:
url: "gcs://test-bucket"
storage_integration: test_storage_integration
file_format: test_avro_format

Schema

  • url (str) - bucket URL for external stage
  • storage_integration (ident) - name of storage integration for external stage
  • encryption (dict)
    • {key} (ident) - name of encryption parameter
    • {value} (bool, float, int, str) - value of encryption parameter
  • directory (dict)
    • {key} (ident) - name of directory parameter
    • {value} (bool, float, int, str) - value of directory parameter
  • file_format (ident) - file format for files in stage
  • copy_options (dict)
    • {key} (ident) - COPY option name
    • {value} (bool, float, int, list, str) - COPY option value
  • comment (str)

Usage notes

  1. 1.
    Stages with url and storage_integration are EXTERNAL. Stages without these parameters are INTERNAL.
  2. 2.
    When INTERNAL stage is being dropped, all files will be lost. When EXTERNAL stage is being dropped, nothing happens. EXTERNAL stage is just a metadata pointing to location in bucket.
  3. 3.
    Business users should only access buckets through stages. Business users should never use STORAGE INTEGRATION objects directly for security reasons.
  4. 4.
    SnowDDL is able to maintain not only stages in general, but also specific files in stages.
  5. 5.
    file_format can only be specified by name referencing to FILE_FORMAT object.

Encryption updates

Encryption on existing EXTERNAL stages cannot be updated by SnowDDL. Snowflake does not provide meta-data, so it is not possible to compare config with existing object.
In order to update encryption information manually, you should use an additional CLI argument:
--refresh-stage-encryption
If you want to remove an encryption from EXTERNAL stage, you should set encryption type to NONE explicitly. It is not enough to remove encryption section from config, since default encryption settings are different for each cloud provider and may change in future.

Safe & unsafe operations

All operations on stages are considered as "safe", EXCEPT:
  • Replace existing stage due to change of stage type (INTERNAL to EXTERNAL or vice versa) or due to change of INTERNAL stage encryption method.
  • Drop existing stage, both INTERNAL and EXTERNAL.
"Unsafe" operations on stages lead to loss of data or meta-data. Also, it may affect other objects depending on stages, like PIPES or EXTERNAL TABLES.
Previous
SHARE (outbound)
Next
STAGE FILE
Last modified 8mo ago