DynamoDB

Step 1 of 8: Name it

What to call this connection, and the deployment it belongs to.

Step 2 of 8: Name the region

The AWS region the tables are in.

DynamoDB is a regional service, and this connector doesn't loop through all regions on your behalf: aws_region must be set explicitly, and only tables in that region are read.

To cover multiple regions, add one connection per region.

Step 3 of 8: Create the read-only user

A read-only IAM user in your account, and its key.

Make one IAM user for Convalesce in your own account with only the policy below, and issue it an access key. The commands do all three.

The policy below reads only tables in the region you named. AWS's managed AmazonDynamoDBReadOnlyAccess works too, but reaches every table in every region.

dynamodb:ListTables and dynamodb:DescribeTable find tables and their key schema.

dynamodb:Scan is required because DynamoDB has no fixed schema and DescribeTable doesn't return one: the connector samples items and infers a schema from what it reads. Without Scan, tables show up with no fields.

There is no network rule to add: Convalesce calls AWS's own API, which is public. The exception is an IAM condition that limits source addresses (aws:SourceIp): it has to allow 34.66.85.47, the address Convalesce connects from.

cat > convalesce-read.json <<'EOF'
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "ConvalesceList",
      "Effect": "Allow",
      "Action": ["dynamodb:ListTables"],
      "Resource": "*"
    },
    {
      "Sid": "ConvalesceRead",
      "Effect": "Allow",
      "Action": ["dynamodb:DescribeTable", "dynamodb:Scan"],
      "Resource": "arn:aws:dynamodb:eu-west-1:*:table/*"
    }
  ]
}
EOF
aws iam get-user --user-name convalesce-reader >/dev/null 2>&1 || aws iam create-user --user-name convalesce-reader
aws iam put-user-policy --user-name convalesce-reader --policy-name ConvalesceDynamoDBRead \
  --policy-document file://convalesce-read.json
aws iam create-access-key --user-name convalesce-reader

Run it as an IAM admin in your own account, where the AWS CLI is signed in (set AWS_PROFILE first if you use a named profile). It is safe to run again: the user is made once, and each tool's policy goes on under its own name. The last command prints AccessKeyId and SecretAccessKey once: paste them straight into the next step, nowhere else. AWS allows two keys per user, so delete an old one before issuing a third. In the IAM console instead, paste the JSON between the EOF lines as an inline policy.

Step 4 of 8: Add the access key

An access key for the reading identity.

An access key for an IAM user in your account that has only read access to these tables. Convalesce has no AWS identity of its own, so it reads only what that user is granted.

Step 5 of 8: Choose what is read (optional)

Optional: narrow it to some tables.

Everything the credential can see is read unless you narrow it here. List the tables you want, the ones to leave out, or both.

Step 6 of 8: Test the connection

Check Convalesce can reach it with what you entered.

The test runs on the same worker a real run would, with the recipe exactly as it will be saved, so it fails the way a run would.

Step 7 of 8: Choose how often

How often Convalesce reads it.

Step 8 of 8: Review and connect

Check everything, then save the connection.