Configure an automated data transfer to Snowflake

This article describes the steps for a Provider to create an automated transfer to the Bobsled-managed Snowflake destination. 

More information:

• Data Transfers: Sharing Data with Bobsled
• Snowflake destination: Snowflake Destination

Prerequisites

To create an automated transfer, you must:

• Have a share created. To learn how to create a share, visit: Create and manage a share.
• Use a pre-configured source. For more information about sources and how to configure them in Bobsled, please visit Sources. 
• Configure the Snowflake destination. To learn how to configure the Snowflake destination, visit: Configure a Snowflake Destination. 

Creating the automated transfer 

1. From the main menu, select Shares. Click on the share that you would like to create a transfer for or create a new share. 

2. Select on Pick Source. Choose from the list of your pre-configured sources. 

3. Select on Pick Destination. Choose Snowflake and region. At this stage, you do not need to configure access.

4. After choosing your source and destination, the Automate transfer button will be enabled. 

5. Select the folders you would like to transfer from the file picker pictured above.

    5.1 Select the checkbox to the left of the folder to add the data to the transfer

    5.2 Select the folder name to navigate into folders

    5.3 Select the folder path to navigate to the root and parent folders

6. Click Continue to configure the tables that will be created in the destination

    6.1 Choose the file format of the files in each folder. This will be used to load the data into the destination.

    6.2 Choose the loading pattern of the files in each folder. Choose an option that describes how the data should be loaded into the destination table. For more information on loading table transfers, visit Loading tables.

    6.3 (Optional) Rename the table created for each folder. Table names in Snowflake can contain only upper case letters, numbers or underscore, and must not begin with a number.

7. Select Start automation. A new transfer will be created and a new row will appear under the “transfers” section. 

    7.1 By default, the automation will automatically sync and transfer the data every 5 minutes; to run the automation at any time, select Run now.   

8. When the transfer is complete, the button Access Data will appear. To learn how to consume a transfer, visit Consuming a data transfer in Snowflake.

Viewing the transfer details

1. Click the ellipses in the transfer row. Select View.  

    1.2 Click drop down on the destination section to view the full table configuration.  

Pausing and resuming an automated transfer

1. To pause the automation, select Pause automation. No new data will be synced while an automation is paused.

2. After pausing an automation, you may restart it by clicking Resume automation. At that time, the data will be transferred and the automation will proceed as scheduled. 

Editing an automated transfer

1. Click the ellipses in the transfer row. Select Edit. The file picker will open. 

2. Edit the configuration of the automated transfer to the desired state. Click resume automated transfer.

If an automated transfer has not yet started, editing and resuming a transfer will immediately start a new transfer.

If editing an in progress transfer, the existing transfer will continue and the changes will take effect the next time the transfer happens.