When Docker offered convenient way to build, ship, and run applications with their unique runtimes independently inside containers the next was to strip systems leaving the only possibility to run containers. LinuxKit is a tool designed to build such minimal Linux dystros. It is like Lego — every part of the system is a container, so you can change and customize everything as you wish using containers and base images you want. LinuxKit dystros are immutable and stateless, but you are able to attach and mount persistent storage. It also supports imposing list of hypervisors and cloud platforms and can be built on any system with docker from yaml file.

LinuxKit yaml for ECS instance

After we’ve played enough with examples we decided to build our own system which is able to connect to the specified ECS-cluster and run our test services and tasks on AWS. The core of the ECS is the ecs-agent which registers/deregisters instances in cluster, manages resources on each instance, runs tasks and services, grants permissions and roles etc.

Image.

How to build ecs-optimized images

Amazon builds it’s own ecs-optimized images but also it is possible to do this by yourself. There are several requirements to run agent properly:

  • mount /data volume and docker.sock to the agent’s container
  • set net.ipv4.conf.all.route_localnet=1 and execute iptables rules to use task role arns
  • also container should run in privileged mode with net: host

To execute sysctl commands on LinuxKit we should run sysctl container in rw mode and mount sysctl.conf with prefered content to the /etc/sysctl.d/:

onboot:
  - name: sysctl
    image: linuxkit/sysctl:4c1ef93bb5eb1a877318db4b2daa6768ed002e21
    binds:
     - /etc/sysctl.d/01-ecs.conf:/etc/sysctl.d/01-ecs.conf
    readonly: false

where /etc/sysctl.d/01-ecs.conf:

files:
  - path: etc/sysctl.d/01-ecs.conf
    contents: |
      net.ipv4.conf.all.route_localnet = 1
      net.ipv4.ip_forward = 1

To execute iptables rules we built and added custom image to the onboot part:

- name: iptables
    image: maddevsio/iptables-ecs:alpine

with Dockerfile:

FROM alpine:3.7
RUN apk add --no-cache iptables
COPY iptables.sh /
CMD ["/iptables.sh"]
LABEL org.mobyproject.config='{"net": "host", "capabilities": ["CAP_NET_ADMIN", "CAP_NET_RAW"]}'

and simple iptables.sh:

#!/bin/shiptables -t nat -A PREROUTING -p tcp -d 169.254.170.2 --dport 80 -j DNAT --to-destination 127.0.0.1:51679
iptables -t nat -A OUTPUT -d 169.254.170.2 -p tcp -m tcp --dport 80 -j REDIRECT --to-ports 51679

Then we configured docker to use /var/run/docker.sock and mounted /var/run to the docker and ecs-agent services:

services:
  - name: docker
    binds:
      - /var/run:/var/run
      - /etc/docker/daemon.json:/etc/docker/daemon.jsonfiles:
  - path: etc/docker/daemon.json
    contents: '{"debug": true, "hosts": ["unix:///var/run/docker.sock"]}'

To configure agent we use json config file which we get from the userdata during onboot by metadata task. This task gets all aws metadata and places it to/var/configso we just need to mount /var/config/userdata to /ecs.config.json inside agent container.

# cat /var/config/userdata  
{
  "Cluster":  "linuxkit-ecs", 
  "TaskIAMRoleEnabled": true, 
  "TaskIAMRoleEnabledForNetworkHost": true 
}
Image.

You can see the whole example here.

I used this post as a base for terraform code which creates s3 bucket and grants policy for AMI import, to build and import image just run:

git clone git@github.com:maddevsio/terraform.git
cd terraform/examples/linuxkit-ecs/linuxkit-ami/
terraform init
terraform apply

Terraform will create all necessary resources, build and upload LinuxKit image.

Free Boilerplate to Run Kubernetes on AWS.
Run Kubernetes on AWS.

A True Easy Way to Run Kubernetes on AWS

A True Easy Way to Run Kubernetes on...

A True Easy Way to Run Kubernetes on AWS

Let me be honest with you — I often feel being a Luddite. Not that I am against progress or trying to stop the hype train, no. But I often think that...

What Is FinOps and How It Changes Approach to Cloud Financial Management?

What Is FinOps and How It Changes Approach to Cloud...

What Is FinOps and How It Changes...

What Is FinOps and How It Changes Approach to Cloud Financial Management?

When a company adopts a hybrid, multi-cloud approach, it may struggle to optimize value and control cloud spend. One of the solutions in this...

Observavility: use cases, benefits and Tools

Observability: Use Cases, Benefits, and Tools

Observability: Use Cases, Benefits,...

Observability: Use Cases, Benefits, and Tools

Imagine launching a new feature only to discover hours later that it's causing widespread outages. Or, picture spending days debugging a critical...