Comments (4)
Thanks for suggesting this feature and for providing the examples too.
It's on our list of things to review / prioritise.
from faas-cli.
@alexellis , this is something I would be willing to work on and support, so if there is a preferred design or an alternative, just let me know.
from faas-cli.
For the sake of a specific implementation, we use a command that looks like this:
docker build
...
--cache-from=type=registry,ref=$CI_REGISTRY_IMAGE:cache-$TARGET_PLATFORM
--cache-to=type=registry,ref=$CI_REGISTRY_IMAGE:cache-$TARGET_PLATFORM,mode=max
--output=type=image,push=true
from faas-cli.
@alexellis After discussing the proposal more in the community calls, we decided it would be good to try and simplify the yaml/flags a little bit so that the most common usecases do not require extensive configuration.
I present two options below.
Initial simplification
First, to simplify the configuration, if build-cache
is configured or if any cache flags are sent to the CLI, then we will
- set the
DOCKER_BUILDKIT=1
env variable, and - automatically add
BUILDKIT_INLINE_CACHE=1
as a build arg.
Second, the yaml and cli will support two options to control the cache from
and to
.
In general, if the value is a string with no ,
or =
, then we assume it is the cache image reference and construct the correct parameterized argument for the docker cache flags.
Also, simplify the configuration, we assume that users of the cache to
will wan to use the max
mode to cache the multistage steps.
This presents several configuation combinations. Below i outline each configuration, what it means, and the equivalent docker build
command for comparison. He main simplification in this proposal is removing the specification of build args and some of the cache parameters for the simple cases.
- A simple inline build cache
version: 1.0
provider:
name: openfaas
gateway: http://127.0.0.1:8080
functions:
my-function:
lang: python3-flask-debian
handler: ./my-function
image: ghcr.io/lucasroesler/my-function:latest
build_cache:
from:
- ghcr.io/lucasroesler/my-function:latest
This is the simplest cache option, the cache metadata is stored with the output image. This is equivalent to
DOCKER_BUILDKIT=1 docker build \
--cache-from ghcr.io/lucasroesler/my-function:latest \
--build-arg BUILDKIT_INLINE_CACHE=1 \
-t ghcr.io/lucasroesler/my-function:latest \
.
- External cache destination,
version: 1.0
provider:
name: openfaas
gateway: http://127.0.0.1:8080
functions:
my-function:
lang: python3-flask-debian
handler: ./my-function
image: ghcr.io/lucasroesler/my-function:latest
build_cache:
from:
- ghcr.io/lucasroesler/my-function:buildcache
to:
- ghcr.io/lucasroesler/my-function:buildcache
In this configuration we assume that the registry
cache is desired and this becomes the equivalent of
DOCKER_BUILDKIT=1 docker build \
--cache-from type=registry,ref=ghcr.io/lucasroesler/my-function:buildcache \
--cache-to type=registry,ref=ghcr.io/lucasroesler/my-function:buildcache,mode=max \
--build-arg BUILDKIT_INLINE_CACHE=1 \
-t ghcr.io/lucasroesler/my-function:latest \
.
- External cache with parameters, we will always try to parse the parameters, if they exist, so this is also valid
version: 1.0
provider:
name: openfaas
gateway: http://127.0.0.1:8080
functions:
my-function:
lang: python3-flask-debian
handler: ./my-function
image: ghcr.io/lucasroesler/my-function:latest
build_cache:
from:
- ghcr.io/lucasroesler/my-function:buildcache
to:
- ref=ghcr.io/lucasroesler/my-function:buildcache,mode=max
In this configuration we assume that the registry
cache is desired and this becomes the equivalent of
DOCKER_BUILDKIT=1 docker build \
--cache-from type=registry,ref=ghcr.io/lucasroesler/my-function:buildcache \
--cache-to type=registry,ref=ghcr.io/lucasroesler/my-function:buildcache,mode=max \
--build-arg BUILDKIT_INLINE_CACHE=1 \
-t ghcr.io/lucasroesler/my-function:latest \
.
- Using alternative cache destination, if the user specifies a
type
parameter, then we use those parameters as is without any modification. This is the most advanced usage and allows the user to simply follow the documentation from buildkit (if they wish) https://github.com/moby/buildkit#export-cache
version: 1.0
provider:
name: openfaas
gateway: http://127.0.0.1:8080
functions:
my-function:
lang: python3-flask-debian
handler: ./my-function
image: ghcr.io/lucasroesler/my-function:latest
build_cache:
from:
- type=gha
to:
- type=gha,mode=max
In this configuration we simply pass the parameters as is
DOCKER_BUILDKIT=1 docker build \
--cache-from type=gha \
--cache-to type=gha,mode=max \
--build-arg BUILDKIT_INLINE_CACHE=1 \
-t ghcr.io/lucasroesler/my-function:latest \
.
Further simplification
I think that example (1) and (2) could potentiallly further simplified. I believe the two most common configurations will be (1) inline cache to the current image or (b) external max cache to a separate tag.
To simplify these two cases we can accept a single string value for the build_cache
or an object for advanced configuration. This creates the three configuration examples:
- For inline mode,
version: 1.0
provider:
name: openfaas
gateway: http://127.0.0.1:8080
functions:
my-function:
lang: python3-flask-debian
handler: ./my-function
image: ghcr.io/lucasroesler/my-function:latest
build_cache: inline
This is equivalent to
DOCKER_BUILDKIT=1 docker build \
--cache-from ghcr.io/lucasroesler/my-function:latest \
--build-arg BUILDKIT_INLINE_CACHE=1 \
-t ghcr.io/lucasroesler/my-function:latest \
.
- External cache destination,
version: 1.0
provider:
name: openfaas
gateway: http://127.0.0.1:8080
functions:
my-function:
lang: python3-flask-debian
handler: ./my-function
image: ghcr.io/lucasroesler/my-function:latest
build_cache: max
In this configuration we assume that the registry
cache is desired and this cache from/to the buildcache
tag of the function image.
DOCKER_BUILDKIT=1 docker build \
--cache-from type=registry,ref=ghcr.io/lucasroesler/my-function:buildcache \
--cache-to type=registry,ref=ghcr.io/lucasroesler/my-function:buildcache,mode=max \
--build-arg BUILDKIT_INLINE_CACHE=1 \
-t ghcr.io/lucasroesler/my-function:latest \
.
- Advanced mode, the user specifies the
from/to
values and matching the configuration options from buildkit https://github.com/moby/buildkit#export-cache
version: 1.0
provider:
name: openfaas
gateway: http://127.0.0.1:8080
functions:
my-function:
lang: python3-flask-debian
handler: ./my-function
image: ghcr.io/lucasroesler/my-function:latest
build_cache:
from:
- type=gha
to:
- type=gha,mode=max
In this configuration we simply pass the parameters as is
DOCKER_BUILDKIT=1 docker build \
--cache-from type=gha \
--cache-to type=gha,mode=max \
--build-arg BUILDKIT_INLINE_CACHE=1 \
-t ghcr.io/lucasroesler/my-function:latest \
.
User experience
There are a couple of errors that can occur and should probably be handled directly in the CLI.
Docker buildx
The max and other advanced modes require buildx
, typically the recommendation would be to use docker buildx install
. This caused docker build
to become equivalent to docker buildx build
.
When cache is enabled, we can either
- add a check and add a warning that
docker buildx install
is required - change the build from
docker build
todocker buildx build
Registry errors
Not all registries will support the cache images, in my experience, this is typically seen as a 400
error during the cache export stage.
When cache mode is enabled and the builder returns an error (doesn't exit cleanly) then we should print a warning and a link to the docs about known supported/unsupported registries. For example GCR is does not support cache but Google Artifact Registry does.
from faas-cli.
Related Issues (20)
- Have the ability to override not only the gateway URL but also the image name (repository, tag) in faas CLI commands HOT 6
- Add ARM64 build for MacOS HOT 1
- Capability to select container driver
- Override template Dockerfile or any file in the template. HOT 6
- feature: add watch mode to faas-cli up workflow HOT 3
- Enhancement: faas-cli local-run should run a build first HOT 6
- Question: How do I publish images when buildkit requires a self-signed certificate? HOT 6
- Question: generating image tags from Git HOT 5
- faas-cli might be incompatible for docker versions 23.0.0 and above HOT 2
- Issue with new feature faas-cli up --watch HOT 13
- After logging in to Faasd local, deploying function return error 401 HOT 6
- Make it possible for image version to have fixed length HOT 4
- I hope to have a more flexible configuration. HOT 1
- provide namespace mangement capabilities in faas-cli
- Failing test: Test_PullTemplates
- bump go-sdk version to v2.7.0
- Specify a local file on disk for the function store HOT 6
- FunctionResources print is not correctly formatted in describe command
- Namespace list command fails with 401 status code on OpenFaaS Enterprise cluster with IAM enabled.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from faas-cli.