Mike Slinn
Mike Slinn

Command-Line AWS Utilities

Published 2021-03-22.
Time to read: 1 minutes.

This page is part of the posts collection, categorized under AWS.

Here are some command-line utilities I have written for AWS. They are dependent on aws cli. You can download all of these utilities in tar format. Extract them into the current directory like this:

Shell
$ tar xf mslinn_aws.tar

awsCfInvalidate

Given a CloudFront distribution ID, invalidate the distribution.

awsCfInvalidate 
#!/bin/bash

function help {
  printf "$1$(basename $0) - Invalidate the CloudFront distribution for the given ID.
If no distribution with the given ID exists, the empty string is returned and the return code is 2.
A message is printed asynchronously to the console when the invalidation completes.

Syntax: $(basename $0) distId

Syntax: awsCfS3Dist www.mslinn.com | $(basename $0)
"
  exit 1
}

function waitForInvalidation {
  echo "Waiting for invalidation $2 to complete."
  aws cloudfront wait invalidation-completed \
    --distribution-id "$1" \
    --id "$2"
  echo "Invalidation $2 has completed."
}


if [ "$1" == -h ]; then help; fi

if [ "$1" ]; then
  DIST_ID="$1"
  shift
elif [ ! -t 0 ]; then
  read -r DIST_ID
fi
if [ -z "$DIST_ID" ]; then help 'Error: No CloudFront distribution ID was specified.\n\n'; fi

if [ "$1" ]; then help 'Error: Too many arguments provided.\n\n'; fi

JSON="$( aws cloudfront create-invalidation \
  --distribution-id "$DIST_ID" \
  --paths "/*"
)"

INVALIDATION_ID="$( jq -r .Invalidation.Id <<< "$JSON" )"
waitForInvalidation "$DIST_ID" "$INVALIDATION_ID" &

Example usages:

Shell
$ awsCfInvalidate E2P5S6OYKQNB6B
Waiting for invalidation IFOPKECU4YYHD to complete. 

... do other things ... 

$ Invalidation IFOPKECU4YYHD has completed.
Shell
$ awsCfS3Dist www.mslinn.com | awsCfInvalidate
Waiting for invalidation IFOPKECU4YYHD to complete. 

... do other things ... 

$ Invalidation IFOPKECU4YYHD has completed.

awsCfS3Dist

Given an S3 bucket name, return the CloudFront distribution JSON.

awsCfS3Dist 
#!/bin/bash

function help {
  printf "$1$(basename $0) - Obtain the CloudFront distribution JSON for an S3 bucket.
If no S3 bucket with the given name exists, the empty string is returned and the return code is 2.

Syntax: $(basename $0) bucketName

Syntax: echo bucketName | $(basename $0)
"
  exit 1
}

if [ "$1" == -h ]; then help; fi

if [ "$1" ]; then
  BUCKET_NAME="$1"
  shift
elif [ ! -t 0 ]; then
  read -r BUCKET_NAME
fi
if [ -z "$BUCKET_NAME" ]; then help 'Error: No S3 bucket name was specified.\n\n'; fi

if [ "$1" ]; then help 'Error: Too many arguments provided.\n\n'; fi

if [ "$( aws s3api head-bucket --bucket $BUCKET_NAME 2> >(grep -i 'Not Found') )" ]; then
  >&2 echo "Error: Bucket $BUCKET_NAME does not exist."
  exit 2
fi

DIST_ID="$( awsCfS3DistId "$BUCKET_NAME" )"

if [ -z "$DIST_ID" ]; then exit 2; fi

aws cloudfront get-distribution-config --id "$DIST_ID"

Example usages:

Shell
$ awsCfS3Dist www.mslinn.com
{
  "ETag": "E1DIZUSLMOLXKP",
  "DistributionConfig": {
      "CallerReference": "1454487160038",
      "Aliases": {
          "Quantity": 2,
          "Items": [
              "www.mslinn.com",
              "mslinn.com"
          ]
      },
      "DefaultRootObject": "index.html",
      "Origins": {
          "Quantity": 1,
          "Items": [
              {
                  "Id": "S3-www.mslinn.com",
                  "DomainName": "www.mslinn.com.s3-website-us-east-1.amazonaws.com",
                  "OriginPath": "",
                  "CustomHeaders": {
                      "Quantity": 0
                  },
                  "CustomOriginConfig": {
                      "HTTPPort": 80,
                      "HTTPSPort": 443,
                      "OriginProtocolPolicy": "http-only",
                      "OriginSslProtocols": {
                          "Quantity": 3,
                          "Items": [
                              "TLSv1",
                              "TLSv1.1",
                              "TLSv1.2"
                          ]
                      },
                      "OriginReadTimeout": 30,
                      "OriginKeepaliveTimeout": 5
                  },
                  "ConnectionAttempts": 3,
                  "ConnectionTimeout": 10
              }
          ]
      },
      "OriginGroups": {
          "Quantity": 0
      },
      "DefaultCacheBehavior": {
          "TargetOriginId": "S3-www.mslinn.com",
          "TrustedSigners": {
              "Enabled": false,
              "Quantity": 0
          },
          "ViewerProtocolPolicy": "redirect-to-https",
          "AllowedMethods": {
              "Quantity": 2,
              "Items": [
                  "HEAD",
                  "GET"
              ],
              "CachedMethods": {
                  "Quantity": 2,
                  "Items": [
                      "HEAD",
                      "GET"
                  ]
              }
          },
          "SmoothStreaming": false,
          "Compress": true,
          "LambdaFunctionAssociations": {
              "Quantity": 0
          },
          "FieldLevelEncryptionId": "",
          "CachePolicyId": "658327ea-f89d-4fab-a63d-7e88639e58f6"
      },
      "CacheBehaviors": {
          "Quantity": 0
      },
      "CustomErrorResponses": {
          "Quantity": 2,
          "Items": [
              {
                  "ErrorCode": 403,
                  "ResponsePagePath": "",
                  "ResponseCode": "",
                  "ErrorCachingMinTTL": 60
              },
              {
                  "ErrorCode": 404,
                  "ResponsePagePath": "",
                  "ResponseCode": "",
                  "ErrorCachingMinTTL": 60
              }
          ]
      },
      "Comment": "",
      "Logging": {
          "Enabled": false,
          "IncludeCookies": false,
          "Bucket": "",
          "Prefix": ""
      },
      "PriceClass": "PriceClass_All",
      "Enabled": true,
      "ViewerCertificate": {
          "ACMCertificateArn": "arn:aws:acm:us-east-1:031372724784:certificate/2be42926-829c-4db9-be7d-a72e951256d4",
          "SSLSupportMethod": "sni-only",
          "MinimumProtocolVersion": "TLSv1",
          "Certificate": "arn:aws:acm:us-east-1:031372724784:certificate/2be42926-829c-4db9-be7d-a72e951256d4",
          "CertificateSource": "acm"
      },
      "Restrictions": {
          "GeoRestriction": {
              "RestrictionType": "none",
              "Quantity": 0
          }
      },
      "WebACLId": "",
      "HttpVersion": "http1.1",
      "IsIPV6Enabled": false
  }
}
Shell
$ echo www.mslinn.com | awsCfS3Dist
{
  "ETag": "E1DIZUSLMOLXKP",
  "DistributionConfig": {
      "CallerReference": "1454487160038",
      "Aliases": {
          "Quantity": 2,
          "Items": [
              "www.mslinn.com",
              "mslinn.com"
          ]
      },
      "DefaultRootObject": "index.html",
      "Origins": {
          "Quantity": 1,
          "Items": [
              {
                  "Id": "S3-www.mslinn.com",
                  "DomainName": "www.mslinn.com.s3-website-us-east-1.amazonaws.com",
                  "OriginPath": "",
                  "CustomHeaders": {
                      "Quantity": 0
                  },
                  "CustomOriginConfig": {
                      "HTTPPort": 80,
                      "HTTPSPort": 443,
                      "OriginProtocolPolicy": "http-only",
                      "OriginSslProtocols": {
                          "Quantity": 3,
                          "Items": [
                              "TLSv1",
                              "TLSv1.1",
                              "TLSv1.2"
                          ]
                      },
                      "OriginReadTimeout": 30,
                      "OriginKeepaliveTimeout": 5
                  },
                  "ConnectionAttempts": 3,
                  "ConnectionTimeout": 10
              }
          ]
      },
      "OriginGroups": {
          "Quantity": 0
      },
      "DefaultCacheBehavior": {
          "TargetOriginId": "S3-www.mslinn.com",
          "TrustedSigners": {
              "Enabled": false,
              "Quantity": 0
          },
          "ViewerProtocolPolicy": "redirect-to-https",
          "AllowedMethods": {
              "Quantity": 2,
              "Items": [
                  "HEAD",
                  "GET"
              ],
              "CachedMethods": {
                  "Quantity": 2,
                  "Items": [
                      "HEAD",
                      "GET"
                  ]
              }
          },
          "SmoothStreaming": false,
          "Compress": true,
          "LambdaFunctionAssociations": {
              "Quantity": 0
          },
          "FieldLevelEncryptionId": "",
          "CachePolicyId": "658327ea-f89d-4fab-a63d-7e88639e58f6"
      },
      "CacheBehaviors": {
          "Quantity": 0
      },
      "CustomErrorResponses": {
          "Quantity": 2,
          "Items": [
              {
                  "ErrorCode": 403,
                  "ResponsePagePath": "",
                  "ResponseCode": "",
                  "ErrorCachingMinTTL": 60
              },
              {
                  "ErrorCode": 404,
                  "ResponsePagePath": "",
                  "ResponseCode": "",
                  "ErrorCachingMinTTL": 60
              }
          ]
      },
      "Comment": "",
      "Logging": {
          "Enabled": false,
          "IncludeCookies": false,
          "Bucket": "",
          "Prefix": ""
      },
      "PriceClass": "PriceClass_All",
      "Enabled": true,
      "ViewerCertificate": {
          "ACMCertificateArn": "arn:aws:acm:us-east-1:031372724784:certificate/2be42926-829c-4db9-be7d-a72e951256d4",
          "SSLSupportMethod": "sni-only",
          "MinimumProtocolVersion": "TLSv1",
          "Certificate": "arn:aws:acm:us-east-1:031372724784:certificate/2be42926-829c-4db9-be7d-a72e951256d4",
          "CertificateSource": "acm"
      },
      "Restrictions": {
          "GeoRestriction": {
              "RestrictionType": "none",
              "Quantity": 0
          }
      },
      "WebACLId": "",
      "HttpVersion": "http1.1",
      "IsIPV6Enabled": false
  }
}

awsCfS3DistId

Given an S3 bucket name, return the CloudFront distribution ID.

awsCfS3DistId 
#!/bin/bash

function help {
  printf "$1$(basename $0) - Obtain the CloudFront distribution ID for an S3 bucket.
If no S3 bucket with the given name exists, the empty string is returned and the return code is 2.

Syntax: $(basename $0) bucketName

Syntax: echo bucketName | $(basename $0)
"
  exit 1
}

if [ "$1" == -h ]; then help; fi

if [ "$1" ]; then
  BUCKET_NAME="$1"
  shift
elif [ ! -t 0 ]; then
  read -r BUCKET_NAME
fi
if [ -z "$BUCKET_NAME" ]; then help 'Error: No S3 bucket name was specified.\n\n'; fi

if [ "$1" ]; then help 'Error: Too many arguments provided.\n\n'; fi

if [ "$( aws s3api head-bucket --bucket $BUCKET_NAME 2> >(grep -i 'Not Found') )" ]; then
  >&2 echo "Error: Bucket $BUCKET_NAME does not exist."
  exit 2
fi

DIST_ID="$(
  aws cloudfront list-distributions \
  --query "DistributionList.Items[*].{id:Id,origin:Origins.Items[0].Id}[?origin=='S3-$BUCKET_NAME'].id" \
  --output text
)"

if [ -z "$DIST_ID" ]; then exit 2; fi
echo "$DIST_ID"

Example usages:

Shell
$ awsCfS3DistId www.mslinn.com
E2P5S6OYKQNB6B
Shell
$ echo www.mslinn.com | awsCfS3DistId
E2P5S6OYKQNB6B

awsCfS3MakeDist

Creates a CloudFront distribution for the given bucket name. Returns the new distribution's ID.

awsCfS3MakeDist 
#!/bin/bash

function help {
  printf "$1$(basename $0) - Make a new CloudFront distribution for the given S3 bucket name.

Returns the new distribution's ID.

Syntax: $(basename $0) bucketName

Syntax: echo bucketName | $(basename $0)
"
  exit 1
}

function doesDistributionExist {
  DIST_ID="$( awsCfS3Dist "$BUCKET_NAME" )"
  if [ "$DIST_ID" ]; then echo true; fi
}

function createDist {
  read -r -d '' NEW_DIST_JSON <<EOF
{
  "CallerReference": "$BUCKET_NAME",
  "Aliases": {
    "Quantity": 0
  },
  "DefaultRootObject": "index.html",
  "Origins": {
    "Quantity": 1,
    "Items": [
      {
        "Id": "$BUCKET_NAME",
        "DomainName": "$BUCKET_NAME.s3.amazonaws.com",
        "S3OriginConfig": {
          "OriginAccessIdentity": ""
        }
      }
    ]
  },
  "DefaultCacheBehavior": {
    "TargetOriginId": "$BUCKET_NAME",
    "ForwardedValues": {
      "QueryString": true,
      "Cookies": {
        "Forward": "none"
      }
    },
    "TrustedSigners": {
      "Enabled": false,
      "Quantity": 0
    },
    "ViewerProtocolPolicy": "redirect-to-https",
    "MinTTL": 3600
  },
  "CacheBehaviors": {
    "Quantity": 0
  },
  "Comment": "",
  "Logging": {
    "Enabled": false,
    "IncludeCookies": true,
    "Bucket": "",
    "Prefix": ""
  },
  "PriceClass": "PriceClass_All",
  "Enabled": true
}
EOF

  NEW_DIST_RESULT_JSON = "$(
    aws cloudfront create-distribution --distribution-config "$NEW_DIST_JSON"
  )"

  DISTRIBUTION_ID="$( jq -r '.Distribution.Id' <<< "$NEW_DIST_RESULT_JSON" )"

  echo "$DISTRIBUTION_ID"
}


if [ "$1" == -h ]; then help; fi

if [ -t 0 ]; then
  if [ -z "$1" ]; then help 'Error: No S3 bucket name was specified.\n\n'; fi
  BUCKET_NAME="$1"
  shift
else
  read -r BUCKET_NAME
fi
if [ -z "$BUCKET_NAME" ]; then help 'Error: No S3 bucket name was specified.\n\n'; fi

if [ "$1" ]; then help 'Error: Too many arguments provided.\n\n'; fi

if [ "$( aws s3api head-bucket --bucket $BUCKET_NAME 2> >(grep -i 'Not Found') )" ]; then
  >&2 echo "Error: Bucket $BUCKET_NAME does not exist."
  exit 2
fi

if [ "$(doesDistributionExist)" ]; then
  >&2 echo "Error: a CloudFront distibution already exists for S3 bucket $BUCKET_NAME"
  exit 3
fi

createDist

Example usages:

Shell
$ awsCfS3MakeDist my_bucket
E2P5S6OYKQNB6B
Shell
$ echo my_bucket | awsCfS3MakeDist
E2P5S6OYKQNB6B

awsS3Mb

Make a new S3 bucket with the given name in the default AWS region. If the --public-read option is provided, set the ACL to public-read

awsS3Mb 
#!/bin/bash

function help {
  printf "$1$(basename $0) - Make a new S3 bucket with the given name in the default AWS region.

Syntax: $(basename $0) bucketName [OPTIONS]

Syntax: echo bucketName | $(basename $0) [OPTIONS]

Options are:
  --public-read  Set bucket ACL to public-read
"
  exit 1
}

if [ "$1" == -h ]; then help; fi

if [ "$1" == "--public-read" ]; then
  ACL="public-read"
  shift
fi

if [ "$1" ]; then
  BUCKET_NAME="$1"
  shift
elif [ ! -t 0 ]; then
  read -r BUCKET_NAME
fi
if [ -z "$BUCKET_NAME" ]; then help 'Error: No S3 bucket name was specified.\n\n'; fi

if [ "$1" ]; then help 'Error: Too many arguments provided.\n\n'; fi

aws s3 mb s3://$BUCKET_NAME

if [ "$ACL" ]; then
  aws s3api put-bucket-acl --bucket $BUCKET_NAME --acl $ACL
fi

Example usages:

Shell
$ awsS3Mb my_bucket
Shell
$ awsS3Mb my_bucket --public-read
Shell
$ echo my_bucket | awsS3Mb --public-read

awsS3Website

Enable an S3 bucket to be a website.

awsS3Website 
#!/bin/bash

function help {
  printf "$1$(basename $0) - Enable an S3 bucket to be a website.

Syntax: $(basename $0) bucketName

Syntax: echo bucketName | $(basename $0)
"
  exit 1
}

if [ "$1" == -h ]; then help; fi

if [ "$1" ]; then
  BUCKET_NAME="$1"
  shift
elif [ ! -t 0 ]; then
  read -r BUCKET_NAME
fi
if [ -z "$BUCKET_NAME" ]; then help 'Error: No S3 bucket name was specified.\n\n'; fi

if [ "$1" ]; then help 'Error: Too many arguments provided.\n\n'; fi

aws s3 website s3://$BUCKET_NAME \
  --index-document index.html \
  --error-document 404.html

Example usages:

Shell
$ awsS3Website my_bucket
Shell
$ echo my_bucket | awsS3Website


* indicates a required field.

Please select the following to receive Mike Slinn’s newsletter:

You can unsubscribe at any time by clicking the link in the footer of emails.

Mike Slinn uses Mailchimp as his marketing platform. By clicking below to subscribe, you acknowledge that your information will be transferred to Mailchimp for processing. Learn more about Mailchimp’s privacy practices.