Snowflake

Snowflake is one of many delivery destinations that Bobsled supports. When delivering data from cloud object storage, Bobsled will turn the selected folders into tables in Snowflake. Utilizing Snowflake Secure Data Sharing, Bobsled creates a Snowflake data share in which the configured Snowflake account(s) are granted access to create a read-only database from the share to query within their account.

Bobsled-managed Snowflake

To learn how to Configure a Snowflake destination in Bobsled, please visit Bobsled-managed Snowflake setup guide.

Authorization

Bobsled requires Snowflake organization name and Snowflake account name to grant your account access to the Snowflake share. An Account Admin (or another user with import share privilege) on the Snowflake account that will consume the data must accept the data share to make it available within the account.

To learn more about the Snowflake access identifier used within Bobsled please visit: Account Access Identifiers in Snowflake.

Bobsled supports various advanced settings to further control how tables are delivered in Snowflake.

Clustering keys

Bobsled supports the setting of cluster keys ↗ in Snowflake, resulting in optimized tables for expected query patterns.

• To set up clustering, access the advanced settings icon on the right side of the table configuration screen.

• Each table can have one cluster key configuration.

• The order of the selected keys during setup is important, and Bobsled will respect that order.

To lower cardinality and support more efficient clustering maintenance, transformations are applied to certain column types following Snowflake best practices. The column types and their corresponding transformations are as follows:

• TIMESTAMP: Converted to date using TO_DATE()

• GEOGRAPHY: Converted to a string using ST_GEOHASH()

Clustering can also be set using the tableSettings property on a share using the API ↗

TIP:
If you are interested in using clustering to deliver optimized tables to your consumers but need assistance with the setup, please reach out to your account team.

Datatype override

Bobsled offers the capability to override a column's data type in your source schema with a different data type in the destination table. This functionality is primarily used for certain geospatial data types, which are available in Snowflake but not specifiable in Parquet.

TIP:
If you wish to leverage data type overriding, please reach out to your account team.

Search optimization

Bobsled supports the setting of search optimization ↗ in Snowflake. This service aims to significantly improve the performance of certain types of queries on tables for your customers.

TIP:
If are interested in using search optimization, please reach out to your account team.

Schema migration support

When new columns are added to tables or files, Bobsled efficiently handles schema migrations by adding new columns to existing tables without disrupting deliveries.

• When new columns are introduced, they're seamlessly integrated, and any missing data in these columns is defaulted to null values.

  • This approach ensures that data loading continues smoothly, even with schema changes, preventing load failures and maintaining data integrity.

• Our schema migration strategy is designed for flexibility and reliability during data structure evolution.

• When columns aren't present in new files, the value for missing columns is set to null.

Consuming a data transfer

Once you’ve configured your destination in a share, granted access to a consumer, and transferred data, learn how to consume a data transfer in Snowflake