Published 2021-04-29.
Last modified 2021-05-03.
Time to read: 3 minutes.
posts collection, categorized under AWS Lambda, Django, Docker, Linux, Python.
This article is a work in progress. Some of it may be incorrect, and some thoughts might lead nowhere. I am publicly posting it in this state so I can discuss it with others. This article will be improved as information becomes available.
Goal
As previously discussed,
Buildah is a drop-in replacement for using docker build and a Dockerfile.
Buildah’s build-using-dockerfile, or bud argument makes it behave just like docker build does.
The goal of this article is to use Buildah / podman to create an Open Container Initiative (OCI) container image with a Django app,
including the Python 3.8 runtime installed.
The Django app will start when the container is created.
The code for the Django app will be stored on the local machine where its source code can be edited,
and it will be mapped into the container from the host system.
Changes made to the code from the host system will be immediately visible inside the container.
TODO
Background: AWS publishes Deploying Python with an AWS base image,
but that does not discuss running or testing.
Create a Lambda function with the console
is a more complete article, but is focused on using the web browser console, using Docker, and Node.js.
So many differences from the desired goal make the articles difficult to translate to AWS CLI, Buildah / podman and Python.
Talk about the AWS Lambda Runtime Interface Emulator, compare and contrast with the AWS Lambda Python Runtime Interface Client.
Compare these AWS Lambda Runtimes with other, equivalant runtimes.
OCI images are swapped in when AWS Lambda is invoked. Do larger images cost more to use? If so, discuss.
Deploy Python Lambda function with Container Image
Consider this Dockerfile, which launches a Python 3.8 command-line application in a manner compatible with AWS Lambda:
FROM public.ecr.aws/lambda/python:3.8 COPY app.py ./ CMD ["app.handler"]
Following is a small Python app called app.py, which will be launched by the Dockerfile.
The Python app can be run as an AWS Lambda program because it implements the handler entry point.
import sys
def handler(event, context):
return f"Hello from AWS Lambda using Python {sys.version}!"
Build Image
Buildah builds the image, just the same way that Docker would:
$ buildah bud -t hello . STEP 1: FROM public.ecr.aws/lambda/python:3.8 Getting image source signatures Copying blob 03ac043af787 skipped: already exists Copying blob 420e64b38334 done Copying blob ff259f25b075 done Copying blob 3ff716981d54 done Copying blob 6b6e623a48a8 done Copying blob 9aa8f1e66d54 done Copying config 67dc3a2a54 done Writing manifest to image destination Storing signatures STEP 2: COPY app.py ./ STEP 3: CMD ["app.handler"] STEP 4: COMMIT hello Getting image source signatures Copying blob 683073d39306 skipped: already exists Copying blob 658871a69e1f skipped: already exists Copying blob 6fa16f35d11e skipped: already exists Copying blob d6fa53d6caa6 skipped: already exists Copying blob 61c062506436 skipped: already exists Copying blob 1c1d66a5fd95 skipped: already exists Copying blob 33af9dc6463a done Copying config 98862dfd20 done Writing manifest to image destination Storing signatures --> 98862dfd208 98862dfd2087152ee821553d6cb1c033e735af06e5f11c814bcc9300fb65584e
Test Lambda function Locally
Before calling the Lambda API from a local container, first run the container.
Containers default to running in the foreground, but the -d option causes a container to be run as a background process.
This container is given the name hello,
the external HTTP endpoint at 9000 is mapped to internal port 8080,
and the latest version of the hello lambda function is run in the container.
$ podman run \ -d \ --name hello \ -p 9000:8080 \ hello:latest d4d296e4c91d01c98d312e3f79599dca53990d95218e94bbdfbbac6a43cde9e8
Call the local version of the Lambda API:
$ curl \ -XPOST "http://localhost:9000/2015-03-31/functions/function/invocations" \ -d '{}' "Hello from AWS Lambda using Python 3.8.9 (default, Apr 20 2021, 13:58:54) \n[GCC 7.3.1 20180712 (Red Hat 7.3.1-12)]!"
Stop the container called hello.
$ podman stop hello 96cc1b1ed92368a1165d6a6ad0b1e5544d4ac751b64e94df33bf2322e6d7b30c
Create AWS ECR Repository
AWS provides a registry for OCI-compatible image repositories called the AWS Elastic Container Registry (ECR).
$ aws ecr create-repository help CREATE-REPOSITORY() CREATE-REPOSITORY()
NAME create-repository -
DESCRIPTION Creates a repository. For more information, see Amazon ECR Repositories in the Amazon Elastic Container Registry User Guide .
See also: AWS API Documentation
See 'aws help' for descriptions of global parameters.
SYNOPSIS create-repository --repository-name <value> [--tags <value>] [--image-tag-mutability <value>] [--image-scanning-configuration <value>] [--cli-input-json <value>] [--generate-cli-skeleton <value>]
OPTIONS --repository-name (string) The name to use for the repository. The repository name may be spec- ified on its own (such as nginx-web-app ) or it can be prepended with a namespace to group the repository into a category (such as project-a/nginx-web-app ).
--tags (list) The metadata that you apply to the repository to help you categorize and organize them. Each tag consists of a key and an optional value, both of which you define. Tag keys can have a maximum character length of 128 characters, and tag values can have a maximum length of 256 characters.
(structure) The metadata that you apply to a resource to help you categorize and organize them. Each tag consists of a key and an optional value, both of which you define. Tag keys can have a maximum character length of 128 characters, and tag values can have a maximum length of 256 characters.
Key -> (string) One part of a key-value pair that make up a tag. A key is a general label that acts like a category for more specific tag values.
Value -> (string) The optional part of a key-value pair that make up a tag. A value acts as a descriptor within a tag category (key).
Shorthand Syntax:
Key=string,Value=string ...
JSON Syntax:
[ { "Key": "string", "Value": "string" } ... ]
--image-tag-mutability (string) The tag mutability setting for the repository. If this parameter is omitted, the default setting of MUTABLE will be used which will al- low image tags to be overwritten. If IMMUTABLE is specified, all im- age tags within the repository will be immutable which will prevent them from being overwritten.
Possible values:
o MUTABLE
o IMMUTABLE
--image-scanning-configuration (structure) The image scanning configuration for the repository. This setting determines whether images are scanned for known vulnerabilities af- ter being pushed to the repository.
scanOnPush -> (boolean) The setting that determines whether images are scanned after be- ing pushed to a repository. If set to true , images will be scanned after being pushed. If this parameter is not specified, it will default to false and images will not be scanned unless a scan is manually started with the StartImageScan API.
Shorthand Syntax:
scanOnPush=boolean
JSON Syntax:
{ "scanOnPush": true|false }
--cli-input-json (string) Performs service operation based on the JSON string provided. The JSON string follows the format provided by --gen- erate-cli-skeleton. If other arguments are provided on the command line, the CLI values will override the JSON-provided values. It is not possible to pass arbitrary binary values using a JSON-provided value as the string will be taken literally.
--generate-cli-skeleton (string) Prints a JSON skeleton to standard output without sending an API request. If provided with no value or the value input, prints a sample input JSON that can be used as an argument for --cli-input-json. If provided with the value output, it validates the command inputs and returns a sample output JSON for that command.
See 'aws help' for descriptions of global parameters.
EXAMPLES Example 1: To create a repository
The following create-repository example creates a repository inside the specified namespace in the default registry for an account.
aws ecr create-repository \ --repository-name project-a/nginx-web-app
Output:
{ "repository": { "registryId": "123456789012", "repositoryName": "sample-repo", "repositoryArn": "arn:aws:ecr:us-west-2:123456789012:repository/project-a/nginx-web-app" } }
For more information, see Creating a Repository in the Amazon ECR User Guide.
Example 2: To create a repository configured with image tag immutabil- ity
The following create-repository example creates a repository configured for tag immutability in the default registry for an account.
aws ecr create-repository \ --repository-name sample-repo \ --image-tag-mutability IMMUTABLE
Output:
{ "repository": { "registryId": "123456789012", "repositoryName": "sample-repo", "repositoryArn": "arn:aws:ecr:us-west-2:123456789012:repository/sample-repo", "imageTagMutability": "IMMUTABLE" } }
For more information, see Image Tag Mutability in the Amazon ECR User Guide.
Example 3: To create a repository configured with a scanning configura- tion
The following create-repository example creates a repository configured to perform a vulnerability scan on image push in the default registry for an account.
aws ecr create-repository \ --repository-name sample-repo \ --image-scanning-configuration scanOnPush=true
Output:
{ "repository": { "registryId": "123456789012", "repositoryName": "sample-repo", "repositoryArn": "arn:aws:ecr:us-west-2:123456789012:repository/sample-repo", "imageScanningConfiguration": { "scanOnPush": true } } }
For more information, see Image Scanning in the Amazon ECR User Guide.
OUTPUT repository -> (structure) The repository that was created.
repositoryArn -> (string) The Amazon Resource Name (ARN) that identifies the repository. The ARN contains the arn:aws:ecr namespace, followed by the re- gion of the repository, AWS account ID of the repository owner, repository namespace, and repository name. For example, arn:aws:ecr:region:012345678910:repository/test .
registryId -> (string) The AWS account ID associated with the registry that contains the repository.
repositoryName -> (string) The name of the repository.
repositoryUri -> (string) The URI for the repository. You can use this URI for Docker push or pull operations.
createdAt -> (timestamp) The date and time, in JavaScript date format, when the reposi- tory was created.
imageTagMutability -> (string) The tag mutability setting for the repository.
imageScanningConfiguration -> (structure) The image scanning configuration for a repository.
scanOnPush -> (boolean) The setting that determines whether images are scanned after being pushed to a repository. If set to true , images will be scanned after being pushed. If this parameter is not speci- fied, it will default to false and images will not be scanned unless a scan is manually started with the StartImageScan API.
CREATE-REPOSITORY()
The following creates an AWS ECR image repository in
called hello within the test namespace.
Images are scanned
for known vulnerabilities after they are pushed to the repository.
$ aws ecr create-repository \ --repository-name test/hello \ --image-scanning-configuration scanOnPush=true { "repository": { "repositoryArn": "arn:aws:ecr:us-east-1:031372724784:repository/test/hello", "registryId": "031372724784", "repositoryName": "test/hello", "repositoryUri": "031372724784.dkr.ecr.us-east-1.amazonaws.com/test/hello", "createdAt": 1620232146.0, "imageTagMutability": "MUTABLE", "imageScanningConfiguration": { "scanOnPush": true } } }
Tag Image
podman tag –
Assigns a new image name to an existing image.
A full name refers to the entire image name, including the optional tag after the :.
If there is no tag provided, then podman will default to latest for both the image and the target-name.
– From man podman-tag.
$ IMAGE_NAME=hello $ IMAGE_VERSION=0.1 $ podman tag $IMAGE_NAME:$IMAGE_VERSION \ $REGISTRY/$IMAGE_NAME:$IMAGE_VERSION $ podman images REPOSITORY TAG IMAGE ID CREATED SIZE localhost/hello 0.1 98862dfd2087 39 minutes ago 622 MB 752246127823.dkr.ecr.us-east-1.amazonaws.com/hello latest 98862dfd2087 39 minutes ago 622 MB public.ecr.aws/lambda/python 3.8 67dc3a2a54fb 25 hours ago 622 MB 752246127823.dkr.ecr.us-east-1.amazonaws.com/ancientwarmth latest 5d18ea34fc30 28 hours ago 2.03 GB localhost/ancientwarmth latest 5d18ea34fc30 28 hours ago 2.03 GB <none> <none> 40ef32b39cf4 5 days ago 622 MB docker.io/library/amazonlinux latest 53ef897d731f 5 days ago 170 MB docker.io/amazon/aws-lambda-python 3.8 e12ea62c5582 9 days ago 622 MB docker.io/library/alpine latest 6dbb9cc54074 2 weeks ago 5.88 MB docker.io/lambci/lambda build-python3.8 714c659c9f6f 3 months ago 2.03 GB
Push Image to ECR
Podman will use the IAM credentials for the dev
profile
in ~/.aws/credentials to log into that AWS account:
[default] aws_access_key_id = ******************** aws_secret_access_key = **************************************** region = us-east-1
[dev] aws_access_key_id = ******************** aws_secret_access_key = **************************************** region = us-east-1
$ export AWS_PROFILE=dev
$ AWS_ACCOUNT="$( aws sts get-caller-identity \ --query Account \ --output text )"
$ AWS_REGION="$( aws configure get region )"
$ REGISTRY="$AWS_ACCOUNT.dkr.ecr.$AWS_REGION.amazonaws.com"
$ aws ecr get-login-password \ --region "$AWS_REGION" | \ podman login \ --password-stdin \ --username AWS \ "$REGISTRY" Login Succeeded!
Now that podman is logged into AWS, use podman push the image to AWS ECR:
$ podman push test/$IMAGE_NAME \ $REGISTRY/$IMAGE_NAME:$IMAGE_VERSION Getting image source signatures Copying blob 692590faf2d1 [--------------------------------------] 8.0b / 8.2MiB Copying blob 397718cff58d [--------------------------------------] 8.0b / 206.2MiB Copying blob 9ca787b1c91c [--------------------------------------] 8.0b / 93.1MiB Copying blob ef26f5221b79 [--------------------------------------] 8.0b / 196.7MiB Copying blob 0a3f69c27a89 [--------------------------------------] 8.0b / 316.4MiB Copying blob 5b3cbb76df75 [--------------------------------------] 8.0b / 1.1GiB Copying blob e9cad39831b0 [--------------------------------------] 8.0b / 3.5KiB Error: Error copying image to the remote destination: Error writing blob: Error initiating layer upload to /v2/ancientwarmth/blobs/uploads/ in 752246127823.dkr.ecr.us-east-1.amazonaws.com: name unknown: The repository with name 'hello' does not exist in the registry with id '752246127823'
The results of an image scan for the new repository can be retrieved as follows:
$ aws ecr describe-image-scan-findings \
--repository-name test/hello \
--image-id imageTag=tag_name
Deploy Python Lambda function with Container Image
Podman can invoke the app using an OCI container with Amazon Linux 2 and Python 3.8:
$ podman container run -ti \ public.ecr.aws/lambda/python:3.8 \ blog/docker/podman/app.py Trying to pull public.ecr.aws/lambda/python:3.8... Getting image source signatures Copying blob 1de4740de1c2 done Copying blob 03ac043af787 done Copying blob 2e2bb77ae2dc done Copying blob 842c9dce67e8 done Copying blob df513d38f4d9 done Copying blob 031c6369fb2b done Copying config e12ea62c55 done Writing manifest to image destination Storing signatures time="2021-05-02T23:38:30.971" level=info msg="exec '/var/runtime/bootstrap' (cwd=/var/task, handler=)"