Sharing Data with Bobsled

This article describes how data sharing works in Bobsled, the secure data sharing platform that lets you share data to any supported platform without the need to manage the destination platform.

Key Concepts

This section introduces the set of fundamental concepts used in Bobsled.

Share

A share comprises configuration settings for a source dataset, destination platform and region, data access permissions, loading semantics, and data transfer frequency, along with an event log for activity tracking. It serves as a centralized management hub where users can perform actions such as pausing transfers, revoking access, and deleting shared data.

Learn how to Create and manage a share.

Provider

The organization that is using Bobsled to transfer data to a consumer organization.

Consumer organizations

The organization that a Provider is transferring data to.  

Learn how to Create and Manage Consumers and how consumer organizations affect billing.

Source

In Bobsled, a source is a connection to the data provider's data location, a bucket containing files, or a data warehouse with tables. You can set up sources in Bobsled to access the data you want to send to a consumer via a Share. Once you have a source set up, you can easily configure a share to use it so a single source can be used across multiple shares and consumers.

See currently supported sources.

Destination

In Bobsled, a destination is a fully managed instance of the chosen platform that provides a secure and isolated environment for your data. When you create a destination, Bobsled automatically sets up a dedicated account and the required infrastructure for the chosen platform, which is only used by your organization. This means that you don't need any pre-existing knowledge, accounts, or relationships with the target platform to share data in your consumer's chosen platform.

See currently supported Destinations.

Transfers

A transfer is the process of moving data from the provider's storage to a Bobsled-managed destination, which is executed as an ongoing automated transfer.

See transferring data as files and transferring data as tables to learn more about the behavior of transfers.

Scheduling transfers

Bobsled enables scheduling your transfer to run at a cadence of your choice. Bobsled supports scheduling transfers to run at the following intervals:

• 5 minutes
• 30 minutes
• 1 hour
• 6 hours
• 12 hours
• 1 day
• 1 week
• 2 weeks
• 1 month
  

Transfers can also be set to start immediately or at a date and time in the future. This enables you to set up an transfer and then make sure it doesn't start until a contracted date or when the data will be ready. 

High-level overview

At a high-level sharing data with consumer organizations is done via the following steps:

1. Connect Sources: Providers give Bobsled read-only access to the data they wish to transfer to their consumer organizations and set up a data source in Bobsled to specify the location of the data.
2. Create Share: Providers create a data share in Bobsled, selecting a destination, authorizing access, and defining the following configurations:
   1. Data loading patterns (e.g. append or overwrite)
   2. Query optimization settings (e.g. clustering)
3. Transfer Data: Bobsled transfers the data to the target platform and region specified in the share configuration.  Consumers can securely access the shared data from the designated destination.

Sharing data as files

Automated transfers

When the source is stored in cloud object storage, providers may select an entire bucket or specific folders (prefix), to be transferred to a specified cloud object storage technology and a region.

Once the “Start automated transfer” action is taken all of the files under the selected folders are transferred to the destination under the prefix format <share-identifier>/latest/

Sharing data as tables

When the source data is stored in cloud object storage, providers may select an entire bucket or a folder (prefix) to be transferred to a supported storage layer that uses table formats.  For some destinations, a cloud platform and specified region can be selected where necessary.

Once the “Transfer now” or “Create automated transfer” action is taken files under the selected folder path are loaded into the table name specified in the transfer configuration according to the loading pattern that was specified. When an automated transfer is configured Bobsled will check every five minutes for changes in the source and load any new data to the table.

Learn more about loading destination tables .