Granting access to your Amazon S3 Source

This article will describe the steps for a Provider to grant Bobsled access to an Amazon S3 bucket to be used as a source in Bobsled. For more information on the Amazon S3 Source, visit:

• Amazon S3 Source: Amazon S3 Source

High Level Overview:

• Quick Start with Cloud Formation Setup

• Create an IAM Policy

• Create and IAM Role for Bobsled

• For KMS Encrypted Buckets Only:

  • Grant the Bobsled role access to encryption keys 

Prerequisites 

To configure Bobsled access to the bucket, your account must have the sufficient permissions to create policies and assign roles in Amazon Web Services (AWS). 

Quick start with AWS Cloud Formation Template

Bobsled has created a cloud formation template that can be used to quickly create the role and trust policy required below. This saves time from having to create the policies manually and makes sure the policies are correctly configured. 

The path parameter is optional. If you want to filter Bobsled's access to a specific path, fill out the path parameter in the stack without leading or trailing slashes. If you want Bobsled to access the full bucket, leave this parameter empty.

Click on the link to "Quick create stack" in the AWS Console. This link allows you to quickly configure and create the required permission for Bobsled.

Alternatively, you can run the stack creation in the AWS CLI using the command below.

ActionScriptActionScriptaws cloudformation create-stack --stack-name BobsledS3BucketSourceAccessSetup --template-url https://bobsled-cloud-formation-templates.s3.us-east-2.amazonaws.com/S3BucketSourceSetup.yaml --parameters ParameterKey=BucketName,ParameterValue=your-bucket-name ParameterKey=BobsledReadAccessRoleArn,ParameterValue=bobsled-role-from-app ParameterKey=BobsledReadExternalId,ParameterValue=bobsled-external-id-from-app ParameterKey=RoleName,ParameterValue=BobsledAccessRole ParameterKey=TrustPolicyName,ParameterValue=BobsledTrustPolicy ParameterKey=Path,ParameterValue=optional-path --region us-east-2 --capabilities CAPABILITY_NAMED_IAM

To complete the setup of the Source in the next step Configure an Amazon S3 Source, you will need to obtain the ARN of the role created by the stack.

You can access this information using the following CLI command. Ensure that the stack name matches the name you used. 

ActionScriptActionScriptaws cloudformation describe-stacks --stack-name BobsledS3BucketSourceAccessSetup

Alternatively, you can find the output role ARN by locating the stack in the CloudFormation console.

You can also see the stack template which includes the details on the policy and role that will be created by accessing the template here.

Granting access to your bucket

(1) Create an IAM Policy

1. Login to AWS Management Console

2. From the Services dropdown, select IAM under Security, Identify & Compliance section

3. Click Account Settings on the left-hand panel

4. Expand the Security Token Service (STS) list, find the AWS region corresponding to the region where your bucket is located, and choose Activate if the status is Inactive.

5. Choose Policies from the left-hand navigation pane

6. Click Create Policy

7. Click the JSON tab

8. Add JSON policy that allows Bobsled to read from the S3 bucket. The following policies provide Bobsled with required permissions to read data from a specified list of entire buckets or subfolders. Please replace placeholders with your bucket name(s), and pay special attention to the trailing /*. It should be present on the first statement, but not on the second one. 

    8a. Allow Bobsled to read from an entire bucket. 

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:GetObjectVersion"
      ],
      "Resource": [
        "arn:aws:s3:::<bucket-name>/*",
        "arn:aws:s3:::<bucket-name>/*"
      ]
    },
    {
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket",
        "s3:GetBucketLocation"
      ],
      "Resource": [
        "arn:aws:s3:::<bucket-name>",
        "arn:aws:s3:::<bucket-name>"
      ]
    }
  ]
}

    8b. Allow Bobsled to read from subfolders within a bucket. 

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:GetObjectVersion"
      ],
      "Resource": [
        "arn:aws:s3:::<bucket-name>/*",
        "arn:aws:s3:::<bucket-name>/*"
      ]
    },
    {
      "Sid": "AllowListingOfSubFolder",
      "Action": [
        "s3:ListBucket",
        "s3:GetBucketLocation"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws:s3:::<bucket-name>"
      ],
      "Condition": {
        "StringLike": {
          "s3:prefix": [
            "subfolder/*"
          ]
        }
      }
    }
  ]
}

9. Click Next: Tags

10. (Optional) Add tags to the policy to help identify, organize, or search for AWS resources 

11. Create a policy name (e.g. bobsled_access) and an optional description. Click Create policy. 

(2) Create an IAM Role

1. Login to AWS Management Console 

2. From the Services dropdown, select IAM under the Security, Identity & Compliance header 

3. Click Roles on the left-hand panel

4. Click the Create role button. 

5. Under Trusted entity type, select Custom trust policy

6. Set the trust policy using the following json:

    6a. Replace the <awsBobsledReadArn> & <awsBobsledReadExternalId> in the json below with the values found in the Bobsled application. These values can be found by visiting Data Sources > Add Source > select Amazon S3.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "<awsBobsledReadArn>"
      },
      "Action": "sts:AssumeRole",
      "Condition": {
        "StringEquals": {
          "sts:ExternalId": "<awsBobsledReadExternalId>"
        }
      }
    }
  ]
}

7. Click the Next button 

8. Find the policy you created in the previous section, and select this policy 

9. Click the Next button 

10. Enter a name and description for the role, and click the Create role button 

11. Record the Role ARN value located on the role summary page. You will use the Role ARN to configure your source in Bobsled.

For KMS Encrypted Buckets Only

(3) Grant Bobsled Role Access to Encryption Keys

1. Navigate to the S3 bucket, click on Properties > "Default encryption" section. 

2. Click the link for the Encryption key ARN

3. Scroll down to the "Key users" section. Click Add. 

4. Search for the role created for Bobsled in the previous section. Select the checkbox to the left of the role. Click Add.