Elastic Container Service (ECS)

ECS

  • AWS managed container orchestration platform

  • Integrates with ALB for load balancing ECS tasks

  • Launch Docker containers on AWS = Launch ECS Tasks on ECS Clusters

  • EFS is used as persistent multi-AZ shared storage for ECS tasks

Launch Types

EC2 Launch Type

  • Not Serverless

  • Containers run on underlying EC2 instances

  • ECS takes care of launching & stopping containers (ECS tasks)

  • You must provision & maintain EC2 instances (use ASG)

  • EC2 instances have ECS agent running on them as a docker container

  • Inside a VPC spanning multiple AZ, there is an ECS cluster spanning multiple AZ. Inside the ECS cluster, there will be an ASG responsible for launching container instances (EC2). On every EC2 instance, ECS agent will be running (happens automatically if you choose the AMI for ECS when launching the instance) which registers these instances to the ECS cluster. This will allow the ECS cluster to run Docker containers (ECS tasks) on these instances.

    attachments/Pasted image 20220509230845.jpg

Fargate Launch Type

  • Serverless

  • No need to provision infrastructure (no EC2 instances to manage)

  • No need to worry about infrastructure scaling

  • Just create task definitions

  • AWS just runs ECS Tasks for you based on the CPU / RAM you need (we won’t know where these containers are running)

  • To scale, just increase the number of tasks - no more EC2 instances

  • VPC and ECS cluster are setup the same way as in EC2 launch type, but instead of using ASG with EC2 instances, we have a Fargate cluster spanning multiple AZ. The Fargate cluster will run ECS tasks anywhere within the cluster and attach an ENI (private IP) to each task. So, if we have a lot of ECS tasks, we need sufficient free private IPs.

    attachments/Pasted image 20220509231345.jpg

Launch Type – Auto Scaling EC2 Instances

  • Accommodate ECS Service Scaling by adding underlying EC2 Instances •

  • Auto Scaling Group Scaling

    • Scale your ASG based on CPU Utilization

    • Add EC2 instances over time

  • ECS Cluster Capacity Provider

    • Used to automatically provision and scale the infrastructure for your ECS Tasks

    • Capacity Provider paired with an Auto Scaling Group

    • Add EC2 Instances when you’re missing capacity (CPU, RAM…)

IAM Roles for ECS Tasks

EC2 Instance Profile (IAM role for the EC2 instance)

  • Used by the ECS agent to:

    • Make API calls to ECS service

    • Send container logs to CloudWatch Logs

    • Pull Docker image from ECR

    • Reference sensitive data in Secrets Manager or SSM Parameter Store

ECS Task Execution Role

  • Allows ECS tasks to access AWS resources

  • Each task can have a separate role

  • Use different roles for the different ECS Services

  • Task Role is defined in the task definition

  • Use taskRoleArn parameter to assign IAM policies to ECS Task Execution Role

  • Ex. Reference sensitive data in Secrets Manager or SSM Parameter Store

ECS – Load Balancer Integrations

  • Application Load Balancer supported and works for most use cases

  • Network Load Balancer recommended only for high throughput / high performance use cases, or to pair it with AWS Private Link

  • Classic Load Balancer supported but not recommended (no advanced features – no Fargate)

ECS – Data Volumes (EFS)

  • Mount EFS file systems onto ECS tasks

  • Works for both EC2 and Fargate launch types

  • Tasks running in any AZ will share the same data in the EFS file system

  • Fargate + EFS = Serverless

  • Use cases: persistent multi-AZ shared storage for your containers

  • Amazon S3 cannot be mounted as a file system

ECS Service Auto Scaling

  • Automatically increase/decrease the desired number of ECS tasks

  • Amazon ECS Auto Scaling uses AWS Application Auto Scaling

    • ECS Service Average CPU Utilization

    • ECS Service Average Memory Utilization - Scale on RAM

    • ALB Request Count Per Target – metric coming from the ALB

  • Target Tracking – scale based on target value for a specific CloudWatch metric

  • Step Scaling – scale based on a specified CloudWatch Alarm

  • Scheduled Scaling – scale based on a specified date/time (predictable changes)

  • ECS Service Auto Scaling (task level) ≠ EC2 Auto Scaling (EC2 instance level)

  • Fargate Auto Scaling is much easier to setup (because Serverless)

Rolling Updates

  • Minimum healthy percentage - determines how many tasks, running the current version, we can terminate while staying above the threshold

  • Maximum percentage - determines how many new tasks, running the new version, we can launch while staying below the threshold

  • Min: 50% and Max: 100% and starting number of tasks 4

    attachments/Pasted image 20220509234001.jpg

  • Min: 100% and Max: 150% and starting number of tasks 4

    attachments/Pasted image 20220509234016.jpg

Secrets in ECS tasks

  • Store the secrets in Parameter Store and encrypt them using KMS

  • Reference the secrets in container definition with the name of the environment variable

  • Create an ECS task execution role and reference it with your task definition, which allows access to both KMS and the Parameter Store/Secrets Manager.

  • Supported for both EC2 and Fargate launch types

Scaling ECS Tasks using EventBridge

  • You can use EventBridge (CloudWatch Events) to run Amazon ECS tasks when certain AWS events occur.

  • Ex: set up a CloudWatch Events rule that runs an Amazon ECS task whenever a file is uploaded to an S3 bucket. You can also declare a reduced number of ECS tasks whenever a file is deleted from the S3 bucket.

Troubleshooting Steps

  • Verify that the Docker daemon is running on the container instance.

  • Verify that the Docker Container daemon is running on the container instance.

  • Verify that the container agent is running on the container instance.

  • Verify that the IAM instance profile has the necessary permissions.