Truly Serverless Container

First, my definition of "truly serverless" is as follows:

1. Significantly managed service. I.e. I don’t want to have to setup servers, operating systems, etc.
2. Secure, no compromises or ugly hacks just to make it serverless
3. Pay only for usage, not paying for idle time or underutilized resources

I generally prefer sticking with Lambdas for my serverless backend, but there are cases where Lambda limits get in the way. One such example is working with video where Lambda’s 512MB storage space and the 15-minute time-limit can be a problem. While you can also keep data in memory - which can be configured up to 3GB, this can get expensive and it’s still limited to 3GB. Another limitation is that a lambda function can be max 50MB zipped / 250MB unzipped so if you need to bundle a lot of tools and libraries you can quickly run into issues with this limit. Layers can offer some additional space for libraries but it still has the same size limits per layer and a Lambda function can have max 5 layers attached to it. In any of these cases, a possible solution can be containers.

AWS Fargate promises a serverless approach to containers whereby you don’t need to manage the EC2 instances that the container runs on - this is true, that serverless requirement is fully addressed. However, the "pay only for usage" requirement is a bit more challenging to meet.

For this example, my goal is to have an FFMPEG container that I can pass a job to which it then executes. For those looking to run a service that is externally accessible, I have included some direction on how to achieve that in this article too. I use CDK to write all of the fixed infrastructure and a Lambda function to load the on-demand services that I need. Before copying my CDK instructions please do checkout the CDK git page, especially the change log as there are FREQUENT changes. I created the below in version 1.38. https://github.com/aws/aws-cdk

For the container to be truly serverless, I should only be paying for storage, requests, and container time spent executing the job. To avoid having to give the container full access to my S3 buckets, I will be generating signed GET and POST S3 URLs in the Lambda and passing these to the container via an SQS message.

The container will retrieve the video file(s) from the signed GET URL(s) and store the result on the signed POST URL. Once the job completes, the container will report out via another SQS message which will trigger a Lambda that can confirm the job and terminate the container. The SQS queues keep the containers - which sit in a VPC, completely decoupled from my Lambdas - that I do not want to put into a VPC.

The components that go into getting a container up and running:

• Create, configure and build your Docker container
• Put the container in a registry
• Enable long ARN names in ECS (if you want to use tags on resources)
• Create support services for your use case
• Create the policies and roles
• Create a task definition
• Add the container to the task definition
• Create the cluster (and VPC) that the task will run in
• Launch the container

Let’s look at each of these with the associated options and pricing. (Note that all pricing here is for the Singapore region, the USA regions tend to be cheaper, and I am also ignoring the free tier).

1. Create, configure and build your Docker container

This is fairly straightforward and you can follow any guide for building and testing your Docker container locally. For this example, I was using a modified version of this container: https://hub.docker.com/r/jrottenberg/ffmpeg/ and adding bash scripts to handle the interactions with SQS and execute my video jobs.

AWS Cost: $0, this can be done locally

2. Put the container in a registry

I used AWS’ registry service called ECR as it makes it easier to automate with CDK and integrate with other AWS services. I put my container config and files into a "src" folder in my CDK stack and use ecrassets.DockerImageAsset() to build and deploy the container to ECR.

After including @aws-cdk/aws-ecr-assets you just need 1 line of code:

const ffmpegContainer = ecrassets.DockerImageAsset(this, "ffmpeg", { directory: __dirname +"/src" });

Note that this will add your container to a CDK-managed repo in your AWS account, it does not create a unique repo for your container. If you want your own unique repo, then this is also possible with the appropriate functions in the ECS module for CDK but it takes a few more lines of code.

AWS Cost: $0.10 / GB / month storage and $0.12 / GB data transfer.
The total cost depends on the size of your container, you will also be billed for data transfer out (i.e. each time you launch the container). I consider this serverless because we are paying only for usage.

3. Enable long ARN names in ECS

I didn't realize this was needed until my Lambda threw an error later on. ECS has a default and a long format for ARN’s and if you want to use tags (which you should), you must enable the long format. You can do this with 3 commands in the AWS CLI:

aws ecs put-account-setting-default --name serviceLongArnFormat --value enabled
aws ecs put-account-setting-default --name taskLongArnFormat --value enabled
aws ecs put-account-setting-default --name containerInstanceLongArnFormat --value enabled

(remember to add --profile if needed)

AWS Cost: $0

4. Create any support services

For this example, I need two SQS queues. One for new jobs - for the container to execute, and one for completed jobs - for the container to report out. Creating SQS queues is just another 1-liner with CDK:

// new task messages
const sqsNew = new sqs.Queue(this, "ffmpegJobNew", { visibilityTimeout: cdk.Duration.seconds(7200) });

// completed task messages
const sqsDone = new sqs.Queue(this, "ffmpegJobDone", { visibilityTimeout: cdk.Duration.seconds(300) });

The container will be getting and putting video files from S3, I didn't include that here as it’s specific to my use case compared to the SQS which is a common pattern.

AWS Cost: $0.0000004 per message. This is again a truly serverless service where you are only paying for what you use.

5. Create the policies and roles

Using CDK, we can create the necessary policies and attach them to the roles. We need 2 roles:

1. an execution role that will be used to load the container from the CDK-managed ECR repository
2. and a task role that will be used by the container to access roles.

// execution policy
const policyExecution = new iam.PolicyStatement({
  effect: iam.Effect.ALLOW,
  resources:[
    ffmpegContainer.repository.repositoryArn // from step 2
  ],
  actions: [
    "ecr:BatchCheckLayerAvailability",
    "ecr:GetDownloadUrlForLayer",
    "ecr:BatchGetImage"
  ]
});

// execution role
const erole = new iam.Role(this, "roleEcsExecution", {
  assumedBy: new iam.ServicePrincipal("ecs-tasks.amazonaws.com")
});

// attach policy to role
erole.addToPolicy(policyExecution);

For my use case, the container will need access to the two SQS queues.

// sqs access policy
const policyNew = new iam.PolicyStatement({
  effect: iam.Effect.ALLOW,
  resources:[sqsNew.queueArn],
  actions: [
    "sqs:GetQueueAttributes",
    "sqs:ReceiveMessage",
    "sqs:DeleteMessage"
  ]
});

const policyDone = new iam.PolicyStatement({
  effect: iam.Effect.ALLOW,
  resources:[sqsDone.queueArn],
  actions: [
    "sqs:SendMessage",
  ]
});

// task role
const role = new iam.Role(this, "roleEcsTask", {
  assumedBy: new iam.ServicePrincipal("ecs-tasks.amazonaws.com")
});

// attach policies to task role
role.addToPolicy(policyNew); // new sqs policy
role.addToPolicy(policyDone); // done sqs policy

AWS Cost: $0. AWS does not bill for policies and roles.

6. Create a task definition

An ECS task definition defines the resources (CPU and memory) that the container will be given. As there is no cost attached to the definition itself, this is another step that we can safely do in CDK.

const ecstask = new ecs.FargateTaskDefinition(this, "ecstask", {
  cpu: 1024, // 1 cpu... yes, this is confusing
  memoryLimitMiB: 2048, // 2gb memory
  executionRole: erole, // the above execution role
  taskRole: role // the above task role
});

AWS Cost: $0

7. Add the container to the task definition

This step simply adds a specific container to use to the task definition. Again, no cost attached to it as we are not launching anything yet, just providing the configuration. The CDK function "addContainer" can be used on the created ECS task. This also assumes that your container is in ECR as per step 2.

// get the container tag, ecrassets doesn’t provide this so you have to get it from imageUri
let imageTag = ffmpegContainer.imageUri.split(':').pop();

ecstask.addContainer("ffmpeg",{
  image: ecs.ContainerImage.fromEcrRepository(ffmpegContainer.repository, imageTag), 
  environment:{
     INPUT_QUEUE_URL: sqsNew.queueUrl, // queue url for the new task messages
     OUTPUT_QUEUE_URL: sqsDone.queueUrl // queue url for the completed task messages
  },
  workingDirectory: "/app"
})

I don’t need these for my FFMPEG use case, but if you are creating a container service that you need to access externally, then this is also where you would add port mappings. You can add them to the end of the addContainer function.

// continued from above
.addPortMappings({
    containerPort: 1234,
    // host port will be automatically assigned
    protocol: ecs.Protocol.TCP
});

AWS Cost: $0, AWS does not charge for this definition

8. Create the cluster (and VPC) that the task will run in

This is where things stop being so straightforward.

If you want to run a container that is running 24/7 then it's fairly simple and you can follow the documentation for launching a Fargate service. I would not consider this serverless, however, because while you would not need to manage any servers or OS, you would likely be paying for idle time.

In my case, I want to run a task on-demand and not pay anything when no task is running.

If I go ahead and create a cluster in CDK, this will create a default VPC with 2 private subnets and NAT gateways. Those gateways will be costing $80+ / month for doing nothing most of the time - NOT serverless.

There are three approaches to consider:

1. Use an existing VPC

If you already have a VPC with a public subnet or private subnet with a NAT, then you could use that. There would be no additional cost in using the existing VPC for the container so this new feature could still be considered serverless (though the overall solution would not be).

In CDK create an ECS cluster and refer to your existing VPC

const myCluster = new ecs.Cluster(this, "ecscluster", {
    clusterName:"myCluster",
    vpc: YOUR_EXISTING_VPC // this needs to be an ivpc object in CDK, not an ARN
});

AWS Cost: $0 (on top of what you are already paying for the VPC)

2. Use a public subnet

If you don’t mind your containers being on a public network (or if you need them on a public network as would be the case if you need to access your container externally), then you could create a VPC with a public subnet and internet gateway. This would not include the billable NAT gateway. Note that access to the container can be controlled through a security group and the presence or lack of mapped ports.

In CDK create a VPC with a public subnet, internet gateway and no NAT Gateway. (You will need to include the EC2 CDK module for this function).

const myVPC = new stack.services.ec2.Vpc(this, "myvpc", {
  cidr: "10.0.0.0/16",
  natGateways: 0,
  subnetConfiguration: [{
    cidrMask: 24,
    name: 'mySubnet1',
    subnetType: ec2.SubnetType.PUBLIC,
  }]
});

Then create the ECS cluster and refer to the VPC you created

const myCluster = new ecs.Cluster(this, "ecscluster", {
    clusterName:"myCluster",
    vpc: myVPC
});

AWS Cost: $0, if done right, there should be no billable components in this setup

3. Dynamically create a VPC and Cluster in Lambda

This is probably the trickiest option and it has a minute or so of latency to launch the task as it takes some time to launch the NAT. There are also several steps, any of which could go wrong and your per-minute cost of running a task will be higher than other options. The primary reason that I include this option is that in some regulated scenarios you may not have a choice and have to use a private subnet. Make sure to have monitoring in place to avoid orphan VPCs from lingering around and driving up costs for unutilized time.

No further steps are needed in CDK for this case, the rest will be done in the Lambda function.

AWS Cost: $0.059 per hour or partial hour that each NAT is running (1 NAT should be sufficient)

Note 1. Security group

While all options above will create a default security group to go with the VPC, I do recommend creating one specifically for the container just to have some measure of control over it and in case you need to open ports or provide access. If you are launching a container that needs external access (and you configured portMapping when creating the task definition), then you will need to add the ports you need to this security group. Note that if you picked option 3, then you will need to create this in your Lambda instead of in CDK.

Create a security group in CDK:

const secGroup = new ec2.SecurityGroup(this, "secgroup", {
    vpc: myVPC,
    allowAllOutbound: false // enable ports you needs below
});

Add ports to the security group

secGroup.addIngressRule(ec2.Peer.ipv4('0.0.0.0/0'), ec2.Port.tcp(1234), "Example of opening an ingress port for the container");

secGroup.addEgressRule(ec2.Peer.ipv4('0.0.0.0/0'), ec2.Port.tcp(443), "Example of opening an egress port for the container");

Note 2. VPC Gateways

If you are working with S3 and/or DynamoDB, you can use VPC Endpoint Gateway to connect a VPC to those services for free and avoid sending data over the public internet. This free option only supports those 2 services, however. The alternative is via PrivateLink which has an hourly fee so this would only be considered true serverless if you added the PrivateLink from Lambda for the duration of the task. As Fargate requires the internet to load the container image and reach cloudwatch, if you are using a private subnet then you can enable PrivateLink to avoid using the public internet for those connections. If you want to avoid using a NAT then you MUST use PrivateLink for these services for Fargate to work.

Note 3. CloudWatch Logs

You can enable console output to CloudWatch logs on your fargate container. Like Lambda functions, you can then write out statuses, errors and any other kind of information from your container. This is useful for debugging, proactive monitoring, and cost or activity tracking.

First, create a new LogGroup using CDK (this will need the CDK module "Logs"):

const logGroup = new logs.LogGroup(stack.scope,"ecslogs"+cn, {
  logGroupName: "/aws/ecs/groupname",
  removalPolicy: cdk.RemovalPolicy.DESTROY,
  retention: logs.RetentionDays.THREE_DAYS // destroy logs after 3 days
});

Add this "logging" parameter in the addContainer function params (step 7 above):

logging: new ecs.AwsLogDriver({
  streamPrefix: "MYSTREAM",
  logGroup: logGroup, // created above
  retention: logs.RetentionDays.THREE_DAYS // destroy logs after 3 days
})

Note that you can opt to use the same LogGroup for multiple containers and just indicate the container name in the streamPrefix, or you can create a unique LogGroup for each container.

You will also need the following permissions in your execution task role:

"logs:CreateLogStream",
"logs:PutLogEvents"

And this resource:

"arn:aws:logs:"+region+":"+account+":log-group:/aws/ecs/groupname/*" // bold part as per the name given to your log group above

Note 4. Load Balancer

AWS recommends putting a Load Balancer in front of Fargate containers that need to be publicly accessible. I did not need it for my use case but it makes sense for public containers. Keep in mind that Load Balancers have a per-time cost, so to keep it truly serverless, the Loadbalancer would need to be created and attached in your Lambda function. This will add latency to the container being available and require your Lambda function to run a bit longer.

9. Launch the container

This will be done in the Lambda function as you will be billed for the time that the container is running. As my use case can be interrupted and it’s not time-sensitive, I will be using Fargate Spot pricing which is quite a bit cheaper.

AWS Cost of running the container: $0.015168 / CPU hour + $0.001659 / GB memory hour

For the FFMPEG container, we have 1 CPU and 2 GB, so the total cost is $0.018486 / hour.

The grand total of all of the above will be:

• ECR
  $0.01 / month storage for the container (my ffmpeg container is just under 100mb)
  $0.012 data transfer per FFMPEG job
• LAMBDA
  $0.0000004 per job (there will be 2 requests for each ffmpeg job)
  
  Options 1-2 (if the vpc and cluster is created in CDK)
  $0.000031245 per job (1.5 seconds at 128MB)
  
  OR
  
  Option 3 (if the vpc and cluster is created in lambda)
  $0.000007291 per job (35 seconds at 128MB per ffmpeg job)
• SQS
  $0.0000008 for 2 SQS messages per FFMPEG job
• VPC (for option 3: the vpc and cluster is created in lambda with a NAT)
  $0.059 / hour (depends how long the FFMPEG job needs)
• ECS
  $0.018486 / hour (depends how long the FFMPEG job needs)
• S3
  Data transfer between S3 and EC2/Fargate in the same region should be free. Certainly if you are using an Endpoint Gateway for S3 as it remains on the internal network.

For example, if you would run five FFMPEG jobs per day, with videos of average 1GB, and each job takes 1 hour to complete, the total average cost per month would be:

For option 1-2 (VPC created in CDK without a NAT):

$ 4.59 / month

For option 3 (VPC created in Lambda with a NAT):

$ 13.43 / month

Lambda Microservice

This is not my full Lambda function, but I have put the key parts in basic Javascript or pseudocode below to make it easier for readers to understand. Generally speaking, the flow would be to create the services - some in parallel, some in a specific order. For some services you may need to have some wait time. I tend to use promises followed by a Promise.all for the items that can happen in parallel, and loop setTimeOut when I need to wait for some previous step to complete.

I also suggest having a cleanup function in your Lambda microservice. Essentially a function that can undo every step that you need to go through to launch the container. You can trigger this function if something goes wrong at any point to avoid leaving orphan services running and costing you money.

The goal is to minimize idle time in lambda, so do as much as possible in parallel and only wait when you really have to. Luckily, a NAT launches in around 30 seconds. I had situations with other services in the past that needed 15 minutes or more and in those cases I would suggest using a decoupled approach with a delayed SQS queue or CloudWatch events.

First, the access and environment variables will you need for your Lambda microservice:

Resources:

• The task and execution role ARN’s that you created in step 5
• "arn:aws:ecs:"+region+":"+account+":service/CLUSTER_NAME/*"

Actions:

• "ecs:CreateService",
• "ecs:DeleteService",
• "ecs:TagResource",
• "iam:PassRole"

For Option 3 you will also need these actions (with appropriate resources)

• "ecs:CreateCluster",
• "ec2:TagResources",
• "ec2:CreateVpc",
• "ec2:CreateInternetGateway",
• "ec2:AttachInternetGateway",
• "ec2:AllocateAddress",
• "ec2:CreateSubnet",
• "ec2:CreateRouteTable",
• "ec2:CreateNatGateway",
• "ec2:CreateRoute",
• "ec2:CreateSecurityGroup",
• "ec2:AuthorizeSecurityGroupIngress",
• "ec2:AuthorizeSecurityGroupEgress",
• "ec2:AssociateRouteTable",
• "ec2:DisassociateRouteTable",
• "ec2:DeleteSecurityGroup",
• "ec2:DeleteRouteTable",
• "ec2:DeleteNatGateway",
• "ec2:ReleaseAddress",
• "ec2:DetachInternetGateway",
• "ec2:DeleteInternetGateway",
• "ec2:DeleteSubnet",
• "ec2:DeleteVpc"

Environment Variables:

• "ecs_taskdefinition": ecstask.taskDefinitionArn,
• "ecs_cluster": myCluster.clusterArn,
• "ecs_subnet": myVPC.publicSubnets[0].subnetId,
• "ecs_secgroup": secGroup.securityGroupId

(Option 3 would only have the "ecs_taskdefinition" variable).

Launching a VPC and Cluster (if you went with option 3 above)

I have noted below the function names that you need from the AWS SDK and some of the params. Please see the documentation for more details and other options.

Start with these in parallel:

• Get an elastic IP address
  ec2.allocateAddress
  Domain: "vpc"
  • → Create NAT
• Create IGW
  ec2.createInternetGateway
  no params
  • → Attach IGW
• Create VPC
  ec2.createVPC
  CidrBlock: "10.0.0.0/16"
  • → Attach IGW
  • Create a Subnet
    ec2.createSubnet(
    CidrBlock: "10.0.1.0/24", VpcId: vpcresult.Vpc.VpcId
    • → Create NAT
  • Create Route Table
    ec2.createRouteTable
    VpcId: vpcresult.Vpc.VpcId
    • → Create Route

These steps are dependent on some of the above steps

• Attach IGW
  Dependent on: VPC and IGW
  ec2.attachInternetGateway
  InternetGatewayId: igwresult.InternetGateway.InternetGatewayId, VpcId: vpcresult.Vpc.VpcId
  • → Create NAT
• Create NAT
  Dependent on: VPC, Elastic IP, Subnet, IGW
  ec2.createNatGateway
  AllocationId: eidresult.AllocationId, SubnetId: subnetresult.Subnet.SubnetId
  • → Create Route
• Create Security Group
  Dependent on: VPC
  ec2.createSecurityGroup
  GroupName: "my-security-group", VpcId: vpcresult.Vpc.VpcId
  • → Add rules (If needed)
    ec2.AuthorizeSecurityGroupIngress and ec2.AuthorizeSecurityGroupEgress
    See documentation for details

The "Create NAT" step will need around 30 seconds. The result will come back right away but the NAT will not have launched yet so you need to wait for it to do so. I set a timeout for 30 seconds then I try to run Create Route. If it fails with an err.code "InvalidNatGatewayID.NotFound" I wait for another 5 seconds then try again. This keeps looping until it succeeds (or Lambda times out if something went wrong).

• Create Route
  Dependent on: Route Table and NAT
  ec2.createRoute
  DestinationCidrBlock: "0.0.0.0/0", NatGatewayId: natresult.NatGateway.NatGatewayId, RouteTableId: routetableresult.RouteTable.RouteTableId
• Associate the Route Table with the Subnet
  Dependent on Route Table and Subnet
  ec2.associateRouteTable
  RouteTableId: routetableresult.RouteTable.RouteTableId, SubnetId: subnetresult.Subnet.SubnetId
• For public containers, it is recommended that you create a Load Balancer in a public subnet at this point too.

Once all of the above has completed, you have a running VPC that we can use for our cluster and Fargate service. The ecs.createCluster function does not require a VPC so you can actually run it in parallel with VPC creation.

Create Cluster:

ecs.createCluster({
  capacityProviders: ["FARGATE_SPOT"], // or FARGATE (see documentation)
  clusterName: "ffmpeg-cluster-"+uniqueID,
  defaultCapacityProviderStrategy: [{capacityProvider: "FARGATE_SPOT", base: 1, weight: 1}],
  tags: [{
    key: "expiry",
    value: expire, // can be used to cleanup a cluster and avoid orphans
    key: "user",
    value: userid // for linking the cost to a specific user
  }]
});

Launching a Fargate Service

This service manages the task and ensures that the container is running and available. If you went with option 1-2, then you only need this step in Lambda and it will start costing money once it’s running.

Create Fargate Service:

ecs.createService({
  serviceName: "ffmpegjob_"+uniqueID,
  taskDefinition: process.env.ecs_taskdefinition, // created earlier in CDK
  launchType: "FARGATE",
  desiredCount: 1,
  cluster: clusterresult.cluster.clusterName, // or the variable reference if you   picked option 3
  deploymentController: {
    type: "ECS"
  },
  networkConfiguration:{
    awsvpcConfiguration:{
      subnets: [subnetresult.Subnet.SubnetId],  // or the variable reference if you picked option 3
      assignPublicIp: "ENABLED", // counter-intuitive for a private container, but needed to load the container image from ECR
      securityGroups:[process.env.ecs_secgroup]  // or the variable reference if you picked option 3
    }
  },
  propagateTags: "SERVICE",
  tags: [{
    key: "expiry",
    value: expire, // can be used to cleanup a cluster and avoid orphans
    key: "user",
    value: userid // for linking the cost to a specific user
  }]
});

Retrieve the Container IP

There is one more step you will need to do in Lambda after the above if you have a container that needs to be externally accessible: get the public IP address.

The steps are:

Finished

And that’s it, you now have the ability to launch truly serverless containers for one-off tasks or for longer running services. Terminating the service when you are done is a matter of calling ecs.deleteService at the right time. For my FFMPEG case, the container will send a "job done" message (which includes the task ARN) to the output SQS queue which will trigger my Lambda so that it can terminate that particular service. Another solution might be to have a CloudWatch event CRON that calls the Lambda to terminate it. You can also have a recurring CRON triggering a Lambda function to list all of the services and terminate the ones that have expired (checking the expire tag we used above).

One other thing I bumped into that may be relevant to some use cases: Fargate does not permit the "privileged" param on containers. So existing containers that need to run something as a service may not work without some changes.