Git Product home page Git Product logo

containerd-stargz-snapshotter's Introduction

Image Pull Optimization with stargz snapshotter on k3s

This repo will outline the steps to use stargz snapshotter with k3s on a rasberrry pi cluster and the benefits of using the same

Stargz snapshotter is a snapshotter implementation for containerd, for more details please refer the below link

https://github.com/containerd/stargz-snapshotter

Set Up

This set up will be on arm64 cluster with Ubuntu (version: Ubuntu 20.04.1 LTS)

stargz-snapshotter

This daemon needs to be running in every node of the cluster. Choose between nightly and release

Release

export tag=v0.4.1

pushd $(pwd) && \
sudo mkdir -p stargz-snapshotter-k3s/files && cd stargz-snapshotter-k3s && \
sudo curl -LO https://github.com/containerd/stargz-snapshotter/releases/download/$tag/stargz-snapshotter-$tag-linux-arm64.tar.gz && \
cd files && \
sudo curl -LO https://raw.githubusercontent.com/itsmurugappan/containerd-stargz-snapshotter/main/files/config.toml.tmpl && \
sudo curl -LO https://raw.githubusercontent.com/itsmurugappan/containerd-stargz-snapshotter/main/files/stargz-snapshotter.service && 
sudo curl -LO https://raw.githubusercontent.com/itsmurugappan/containerd-stargz-snapshotter/main/files/config/etc/containerd-stargz-grpc/config.toml && \
sudo mkdir -p /etc/containerd-stargz-grpc && \
sudo mv config.toml /etc/containerd-stargz-grpc/ && \
sudo mv stargz-snapshotter.service /etc/systemd/system/ && \
cd .. && \
sudo gunzip stargz-snapshotter-$tag-linux-arm64.tar.gz || true && \
sudo tar -xvf stargz-snapshotter-$tag-linux-arm64.tar && \
sudo chmod +x containerd-stargz-grpc ctr-remote && \
sudo mv ctr-remote containerd-stargz-grpc /usr/local/bin/ && \
sudo systemctl enable stargz-snapshotter && \
sudo systemctl restart stargz-snapshotter

Nightly

For nightly arm 64 image , download from this repo.

pushd $(pwd) && \
git clone https://github.com/itsmurugappan/stargz-snapshotter-k3s.git && \
cd stargz-snapshotter-k3s/files && \
sudo mkdir -p /etc/containerd-stargz-grpc && \
sudo mv config/etc/containerd-stargz-grpc/config.toml /etc/containerd-stargz-grpc/ && \
sudo mv stargz-snapshotter.service /etc/systemd/system/ && \
cd ./.. && \
sudo gunzip releases/stargz-snapshotter-nightly-linux-arm64.tar.gz || true && \
sudo tar -xvf releases/stargz-snapshotter-nightly-linux-arm64.tar && \
sudo chmod +x -R out/ && \
sudo mv out/* /usr/local/bin/ && \
sudo systemctl enable stargz-snapshotter && \
sudo systemctl restart stargz-snapshotter && \
cd ..

k3s

Please fetch the latest k3s (1.19.x onwards) to get the containerd newer than 1.4.1 which is a pre req for using this.

once you start the k3s like below

curl -sfL https://get.k3s.io | INSTALL_K3S_EXEC="--disable=traefik" sh -s -

move the containerd config files

sudo mv files/config.toml.tmpl /var/lib/rancher/k3s/agent/etc/containerd/
# master
sudo systemctl restart k3s
# agent
sudo systemctl restart k3s-agent

Set up is complete on master, please do the same on all worker nodes

cleanup

popd && \
sudo rm -rf stargz-snapshotter-k3s

Testing

Create the below pods to check the start up times.

# stargz image
apiVersion: v1
kind: Pod
metadata:
  name: gohw-estgz
spec:
  containers:
  - name: gohw-estgz
    image: ghcr.io/itsmurugappan/hw:v3
    ports:
    - containerPort: 8080 

# plain image
apiVersion: v1
kind: Pod
metadata:
  name: gohw
spec:
  containers:
  - name: gohw
    image: ghcr.io/itsmurugappan/go-hw:latest
    ports:
    - containerPort: 8080 
  • Plain image should start in 2 minutes in a rpi cluster
  • Stargz image should start in 20 seconds in a rpi cluster
  • Both images are around 700 mb size

Further steps

You can take it forward by installing knative and see how it improves the cold starts.

https://itsmurugappan.medium.com/knative-on-raspberry-pi-1106984de5b8

Building estargz image

For building the estargz image you need the ctr-remote client which would be installed as part of step number 2

# docker login
docker login ghcr.io

# below is the way to optimize a java non estgz image
sudo ctr-remote image optimize  --oci --reuse  --entrypoint='["java", "-Djava.security.egd=file:/dev/./urandom", "-jar", "/helloworld.jar"]' \
           ghcr.io/itsmurugappan/java-hw:latest \
           ghcr.io/itsmurugappan/java-hw-estgz

containerd-stargz-snapshotter's People

Contributors

itsmurugappan avatar

Watchers

 avatar  avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.