BigQuery Source

BigQuery, a cloud data warehouse, is one of many source locations that Bobsled supports. A Provider can set up a BigQuery source by granting Bobsled sufficient access to your BigQuery project and datasets.

Set up a BigQuery Source

Configure BigQuery Source in Bobsled - Part 1

1. Log in to your Bobsled Environment.

2. Select Data Sources in the Left Hand Pane and then Add source in the top right of the environment.

3. Enter a human readable name to describe the source.

4. Select BigQuery, the cloud and the region of your BigQuery account. Click Continue to move to the next step.

5. Copy the Bobsled Service Account email for use in configuring access to Bobsled in your Google Cloud Project. Leave this tab open and log in to your Google Cloud Console in a new tab.

Configure BigQuery Source in your Google Cloud Console

Bobsled needs to assume two different roles to connect your specified datasets to the Bobsled platform - one at the project level and another at the dataset level. You will use the service account email you copied from your Bobsled environment for each.

Granting access to your BigQuery project(s)

1. Log in to the Google Cloud Platform Console as a project editor.

2. From the Home dashboard, Select IAM & Admin. 

3. Select IAM from the Left Hand Pane of the console. Make sure you have the correct Project selected from the dropdown in the top left of the console.

4. Click 'GRANT ACCESS' in the 'VIEW BY PRINCIPALS' section.

5. Add the Bobsled Principal from the Source Creation dialog to the 'New principals *' dialog.

6. Search for and select the 'BigQuery Job User' role and 'SAVE'

You may repeat the above steps for as many other BigQuery projects you would like to connect to Bobsled.

Granting access to your BigQuery dataset(s)

1. Log in to the Google Cloud Platform Console as a project editor.

2. From the Home dashboard, Select BigQuery. 

3. Navigate the BigQuery Explorer panel to find the dataset(s) you would like to connect to Bobsled. For each dataset, select the vertical ellipsis -> Share -> Manage Permissions.

4. Select ADD PRINCIPAL in the top right.

4. Add the same Bobsled Principal as above in the 'New principals *' section.  Add the Big Query Data Viewer role and hit SAVE.

Configure BigQuery Source in Bobsled - Part 2

1. Grab the BigQuery Project ID where you want Bobsled to run jobs (Billing Project ID) and the Project ID(s) from your Google Cloud console where your data lives and return to the source setup page in the Bobsled UI.

2. Enter the relevant Billing Project ID and Project ID(s) and hit Save.

Congratulations! Your data is now ready to be shipped to all clouds, platforms and regions! Click here to learn more on configuring data delivery to the full suite of Bobsled supported destinations here.