Git Product home page Git Product logo

docker-orb's Introduction

Docker Orb CircleCI Build Status CircleCI Orb Version MIT license CircleCI Community

Quickly and easily install Docker, dockerize, and docker-compose in any CircleCI job. Install/use various other Docker-related tools on CircleCI. Build Docker images and deploy them to any registry.

Besides providing commands to easily install Docker, docker-compose, and dockerize, this orb contains the commands/jobs/executors/examples previously published to the docker-publish orb.

Usage

For full usage guidelines, see the orb registry listing.

Contributing

We welcome issues to and pull requests against this repository!

For further questions/comments about this or other orbs, visit CircleCI's orbs discussion forum.

docker-orb's People

Contributors

asdwsda avatar brivu avatar bytesguy avatar danielflookovo avatar emyller avatar ericribeiro avatar felicianotech avatar fruit avatar gep13 avatar guillaumelecerf avatar iwaiktos avatar iynere avatar jaryt avatar johnboyes avatar joshhsoj1902 avatar kelvintaywl avatar ksylvest avatar kyletryon avatar lokst avatar micahvdk avatar mikkopiu avatar n-029894 avatar otremblay avatar slievr avatar timjones avatar varditn avatar vsoch avatar wkovacs64 avatar ymtdzzz avatar zhuangya avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

docker-orb's Issues

hadolint job uploads all of the project sources as artifacts

Orb version

0.5.13

What happened

Given a config.yml like:

version: 2.1

orbs:
  docker: circleci/[email protected]

jobs:
  test:
    steps:
      - checkout
      # ... and so on ...

workflows:
  version: 2
  test-and-lint:
    jobs:
      - test
      - docker/hadolint

Since the last step of the hadolint job is store_artifacts the job will start uploading the whole source tree as artifacts. In my specific case there are quite a lot of files in the checked out tree and the upload takes rather a long time.

It is possible to work around this by adding something like artifacts-path: "/nonexistent" to the hadolint job settings but I don’t quite understand why storing artifacts is a step in a lint job in the first place?

Expected behavior

Linting a Dockerfile shouldn’t upload the whole source tree as artifacts.

I cannot omit $docker_tag_args when execute docker/build step

Orb version

1.0.1

What happened

I experienced this error in Circle CI.

#!/bin/bash -eo pipefail
docker_tag_args=""

IFS="," read -ra DOCKER_TAGS <<< "$CIRCLE_SHA1"
for tag in "${DOCKER_TAGS[@]}"; do
  docker_tag_args="$docker_tag_args -t customer.registy.io/my-image-name:${tag}"
done

docker build  \
  -f ./Dockerfile \
  $docker_tag_args \
  .
Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?

Exited with code exit status 1
CircleCI received exit code 1

Expected behavior

The issue is that I always have to use registry and tag. I would like to separate build and push functionality. In other words, in the current setup, I cannot run docker build -f Dockerfile . command to build an image.

build command ignores step-name parameter

Orb version

0.5.20

What happened

I set step-name in a docker/build step and the step still showed up as the default step name ("Docker build").

jobs:
  test:
    steps:
      - docker/build:
          step-name: My custom step name
          image: whatever

image

Expected behavior

The step should be called "My custom step name" instead of the default step name.

Documentation for the `path` parameter of the `publish` job is misleading

The documentation for the path parameter of the publish job says:

Path to the directory containing your Dockerfile and build context, defaults to . (working directory)

However, the build context is still searched in . (the working directory) regardless of what is specified for path. docker-context must also be set in order to set the location of the build context.

Unable to install version 2.10.0 of docker-compose

Orb version

2.1.2

What happened

Fetching the latest version of docker-compose fails due to the changes in the release files generated by the compose project - version 2.10.0 no longer generate the sha256 files.

Expected behavior

Installing docker-compose without specifying a version should install the latest version.

`install-docker-compose` doesn't clean up after itself

Orb version

2.1.1

What happened

Ran install-docker-compose and it left docker-compose-linux-x86_64.sha256 behind.

Expected behavior

install-docker-compose (and others) should delete any files downloaded to the working directory.

`extra_build_args` wraps everything in single quotes

Orb version

2.1.1

What happened

am trying to use extra_build_args to pass in env vars, but they keep getting wrapped in single quotes, preventing the env var from being expanded
example:

# .circleci/config.yml
# have tried every quoting combination I can think of
extra_build_args: '--build-arg NPM_TOKEN=$NPM_TOKEN'

the command being run is

docker build --build-arg 'NPM_TOKEN=$NPM_TOKEN' ...

bash does not evaluate in single quotes, so the literal string $NPM_TOKEN is being passed to docker instead of the actual value

Expected behavior

the command should be

docker build --build-arg NPM_TOKEN=$NPM_TOKEN

OR wrap in double quotes so we can use env vars

docker build --build-arg "NPM_TOKEN=$NPM_TOKEN"

docker/install-docker-tools tries to move files from projects "docker" folder

Orb version 0.5.13

What happened

I've a folder named "docker" in my projects root. In it I keep dockerfiles for custom images that are used by the project. To organize those dockerfiles I place them in subfolders eg. "docker" -> "php" -> Dockerfile.

As part of my workflow I'm calling "- docker/install-docker-tools" to install docker tools and these lines here are trying to move my folder "php" into /usr/local/bin/ causing an error:

mv: cannot overwrite non-directory '/usr/local/bin/php' with directory 'docker/php'

# install Docker binaries
BINARIES=$(ls docker)
$SUDO mv docker/* /usr/local/bin
$SUDO rm -rf docker

for binary in $BINARIES
do
  $SUDO chmod +x "/usr/local/bin/$binary"
done

Failing workflow:
https://circleci.com/gh/CreativeGS/api-monacofit/2123

Expected behavior

Shouldn't docker binnaries download somewhere out of my projects folder to avoid such clashes?

extra_build_args are no longer working on 2.1.0

Orb version

docker: circleci/[email protected]

What happened

I have just updated a pipeline to the latest orb with the following config and it is failing, it is working fine with docker: circleci/[email protected]

      extra_build_args:
        type: string
        default: "--no-cache --pull"

The error:

+ docker build --file=/home/circleci/project/Dockerfile --tag=********************************************/eve:d1e8b16882301d059d09bedfee0075ea26a3c44e '--no-cache --pull' /home/circleci/project
unknown flag: --no-cache --pull
See 'docker build --help'.

Expected behavior

To accept extra_build_args.

This was introduced in #115.

Realistic example - please?

Hello! 👋

I've started experimenting with CircleCI's Docker ORBs - specifically with circleci/[email protected] and I was wondering if there is a way to show audience and your customers some realistic examples on how to mix this orb with other orbs and how to use this orb after the docker image was built? I.e. docker image is built, now how to properly run it? (not with after_build: hook - obviously. )

Thanks for your feedback and thanks for pushing this!

Regards,

  • Oto

Update Docker Hub description using README.md

When publishing an image to Docker Hub, the description of the image doesn't update. There exists an (unofficial) way of updating the image description by sending a PATCH request to the page of the repository on Docker Hub.

I would be willing to create a PR to set up this functionality but would like feedback on whether or not this would be a useful addition. My tentative idea is to provide an additional command that updates the description from a README file and that command is then optionally called in the publish job.

hadolint Job's dockerfiles parameter does not work correctly

Orb version

0.5.20

What happened

When we specify multiple Dockerfile paths (relative to ~/project) to the dockerfiles parameter, CircleCI says the second Dockerfile does not exist:

  • In .circleci/config.yml:
workflows:
  version: 2
  some_job_name:
    jobs:
      - docker/hadolint:
          dockerfiles: ~/project/frontend/Dockerfile,~/project/worker/Dockerfile
  • Result:
Running hadolint with the following options...


Success! /root/project/frontend/Dockerfile linted; no issues found
hadolint: ~/project/worker/Dockerfile: openBinaryFile: does not exist (No such file or directory)

Expected behavior

Both Dockerfiles should be linted.

Running hadolint with the following options...


Success! /root/project/frontend/Dockerfile linted; no issues found
Success! /root/project/worker/Dockerfile linted; no issues found

docker/push started failing in 2.1.2

Orb version

2.1.2

What happened

I started seeing an error claiming I've got malformed docker repository name, reverting to 2.1.1 fixed the issue - invalid reference format: repository name must be lowercase

The broken job:

https://app.circleci.com/pipelines/github/zmarkan/cicd-workshop/827/workflows/ccae056b-9125-44dc-9027-7a45d4d96220/jobs/1679

After downgrade to 2.1.1:

https://app.circleci.com/pipelines/github/zmarkan/cicd-workshop/828/workflows/52b12662-beee-4f64-b75c-df261e7d7918/jobs/1683

I think it's something to do with how params are being processed. I was able to run docker push command manually with the tag that was created locally & it worked.

Expected behavior

docker/push should pass in 2.1.2

Support for building/pushing images with multiple tags

Orb version

0.5.8

What happened

The orb commands for build/push don't support multiple tags. For example, to use the docker/build command to create an image with 3 tags, you need to use three different steps:

version: 2.1
orbs: 
    docker: circleci/[email protected]
jobs:
    my-job:
        steps:
            - docker/build:
                image: myrepo
                registry: myregistry
                tag: tag1
            - docker/build:
                image: myrepo
                registry: myregistry
                tag: tag2
            - docker/build:
                image: myrepo
                registry: myregistry
                tag: tag3
...

Expected behavior

It would be great if we could build/push images with multiple tags in one step. Something like this:

version: 2.1
orbs: 
    docker: circleci/[email protected]
jobs:
    my-job:
        steps:
            - docker/build:
                image: myrepo
                registry: myregistry
                tag: tag1,tag2,tag3
...

See, for example, the way this has been implemented for AWS: https://github.com/CircleCI-Public/aws-ecr-orb/blob/master/src/commands/build-image.yml#L51

publish job fails with 3rd party registries

Orb version

0.1.0

What happened

If using a 3rd party repo (e.g. quay.io) the publish job will always fail.
The build command does not use the registry parameter so build an image the later push step cannot use.

Expected behavior

publish job should be able to build & push to any 3rd party registry.

Resolving error with custom Docker registry

Orb version

2.1.1

What happened

Resolving error when trying to login to custom Docker registry:

Error response from daemon: Get https://$DOCKER_REGISTRY/v2/: dial tcp: lookup $DOCKER_REGISTRY: no such host

When connected by SSH, the command docker login $DOCKER_REGISTRY works fine.

Expected behavior

A connection to the custom registry should be established, as it is in the version 2.0.5

How to set a dynamic image version?

Orb version

0.5.3

What happened

We would like to be able to use dynamic versioning for our docker images, but we seem to only be able to set static tag values for the docker image.

Is it possible to set a dynamic version akin to something like this?

      - docker/publish:
          tag: $(./version.sh)

dockerlint command bug (missing `-f` GNU option)

Orb version

1.0.5

What happened

using the docker/dockerlint Orb command causes the error cannot open Dockerfile , although the Dockerfile path is properly specified

Expected behavior

That the Orb command uses properly the -f dockerlint option.

Bugfix suggestion

Well look at the source code of the docker/dockerlint Orb Command at https://circleci.com/developer/orbs/orb/circleci/docker?version=1.5.0#commands-dockerlint : yu willimmediately see that it does not make use of the -f option

Unable to use docker/publish in my orb

Orb version

1.0.1

What happened

When importing the docker-orb job 'publish' inside an orb I created I get an error:.

╰─ circleci orb validate src/orb.yml

Error: Error calling job: 'docker/publish'
Missing required argument(s): image

Since I am importing the job for use in my orb I don't have a way to specify the parameter so I'm unable to validate with the cli or use the orb

Expected behavior

I would like to be able to specify the docker-orb/publish job in my orb like so:

orbs:
docker: circleci/[email protected]
jobs:
docker-publish: docker/publish

then in a workflow specify it like this

orbs:
cicd-orb: namespace/[email protected]

workflow:
jobs:
- cicd-orb/docker-publish

without errors related to the required image argument which can't be specified inside my orb (I believe...). If anyone knows of a workaround I'd be most appreciative!

Release 1.0.1 is broken

Orb version

1.0.1

What happened

version: 2.1
orbs:
  docker: circleci/[email protected]

produces error

Error: ERROR IN CONFIG FILE:
[#/jobs/publish] 0 subschemas matched instead of one
1. [#/jobs/publish] only 1 subschema matches out of 2
|   1. [#/jobs/publish/steps] 12 schema violations found
|   |   1. [#/jobs/publish/steps/1] 0 subschemas matched instead of one
|   |   |   1. [#/jobs/publish/steps/1] 0 subschemas matched instead of one
|   |   |   |   1. [#/jobs/publish/steps/1] expected type: String, found: Mapping
|   |   |   |   |   Shorthand commands, like `checkout`
|   |   |   |   |   SCHEMA:
|   |   |   |   |     type: string
...

Login using --password-stdin

docker login \

Docker says:
WARNING! Using --password via the CLI is insecure. Use --password-stdin.

This should be replace by:

        echo "$<<parameters.docker-password>>" | docker login \
          -u "$<<parameters.docker-username>>" --password-stdin \
          <<parameters.registry>>

dockerlint is non-functional as described

Orb version

0.5.17

What happened

PR #12 removed the documentation about requiring NPM, but the step of dockerlint command still attempts to install dockerlint as an npm package, and NOT hadolint as described/documented

Expected behavior

I expected the dockerlint command to work using hadolint as described.

Additional

It looks to me like nobody even approved/reviewed PR #12. Am I right? If so, this isn't the kind of development quality we should be expecting from "certified" orbs.

Job Publish is failing with "invalid reference format: repository name must be lowercase"

Orb version

2.1.2

What happened

build stage passes, push stage fails with "invalid reference format: repository name must be lowercase", apparently unable to parse the parameters or environment variables properly.

default:
  jobs:
    - docker/publish:
          executor: docker/docker
          use-remote-docker: true
          image: $CIRCLE_PROJECT_USERNAME/pycqa-tools
          path: '3.7'
          docker-context: '3.7/.'
          update-description: true
          tag: 3.7,3.7-$CIRCLE_SHA1,3.7-$CIRCLE_BUILD_NUM
          requires:
            - docker/hadolint
          context: dockerhub
          filters:
            branches:
              only:
                - master

Expected behavior

containers build and push. SSHing in and setting the environment variables by hand and running the script commands by hand works.

circleci@fb09f749d290:~/project$ PARAM_TAG=3.7,3.7-$CIRCLE_SHA1,3.7-$CIRCLE_BUILD_NUM
circleci@fb09f749d290:~/project$ echo $PARAM_TAG
3.7,3.7-6db1361431e5eff56a35842cc48f6a6754600c7f,3.7-462
circleci@fb09f749d290:~/project$ IFS="," read -ra DOCKER_TAGS <<< "$PARAM_TAG"
circleci@fb09f749d290:~/project$ PARAM_REGISTRY=docker.io
circleci@fb09f749d290:~/project$ PARAM_IMAGE=$CIRCLE_PROJECT_USERNAME/pycqa-tools
circleci@fb09f749d290:~/project$ for docker_tag in "${DOCKER_TAGS[@]}"; do
>   tag=$(eval echo "$docker_tag")
>   docker push "$PARAM_REGISTRY"/"$PARAM_IMAGE":"$tag"
> done
The push refers to a repository [docker.io/ssplatt/pycqa-tools]
87602ec4b676: Pushed
d9fe01159fea: Pushed
91ab80281724: Layer already exists
dfb2aa1ea2ab: Layer already exists
b456e73bba2a: Layer already exists
12a7221db5c8: Layer already exists
c44b6ed055fe: Layer already exists
a88ee98d25a2: Layer already exists
2484ba8523f6: Layer already exists
37dc959d0ca6: Layer already exists
cae87053165d: Layer already exists
9f54eef41275: Layer already exists
3.7: digest: sha256:28d43a860d06adfdcfec186ac7ce7574b9992cbf2bdaeffe85d76282587843e4 size: 2859
The push refers to a repository [docker.io/ssplatt/pycqa-tools]
87602ec4b676: Layer already exists
d9fe01159fea: Layer already exists
91ab80281724: Layer already exists
dfb2aa1ea2ab: Layer already exists
b456e73bba2a: Layer already exists
12a7221db5c8: Layer already exists
c44b6ed055fe: Layer already exists
a88ee98d25a2: Layer already exists
2484ba8523f6: Layer already exists
37dc959d0ca6: Layer already exists
cae87053165d: Layer already exists
9f54eef41275: Layer already exists
3.7-6db1361431e5eff56a35842cc48f6a6754600c7f: digest: sha256:28d43a860d06adfdcfec186ac7ce7574b9992cbf2bdaeffe85d76282587843e4 size: 2859
The push refers to a repository [docker.io/ssplatt/pycqa-tools]
87602ec4b676: Layer already exists
d9fe01159fea: Layer already exists
91ab80281724: Layer already exists
dfb2aa1ea2ab: Layer already exists
b456e73bba2a: Layer already exists
12a7221db5c8: Layer already exists
c44b6ed055fe: Layer already exists
a88ee98d25a2: Layer already exists
2484ba8523f6: Layer already exists
37dc959d0ca6: Layer already exists
cae87053165d: Layer already exists
9f54eef41275: Layer already exists
3.7-462: digest: sha256:28d43a860d06adfdcfec186ac7ce7574b9992cbf2bdaeffe85d76282587843e4 size: 2859

How to use the executor parameters

Orb version

0.5.0

What happened

When trying to use a custom image for the docker executor (following https://circleci.com/orbs/registry/orb/circleci/docker#executors-docker & https://circleci.com/docs/2.0/reusing-config/#example-build-configuration-using-a-parameterized-executor) and needing to run a npm command before building the image I created the following job:

    build-publish-docker-develop:
      - docker/publish:
          image: TimJones/myimage
          executor:
            docker/docker:
              image: circleci/node
              tag: 10.15
          use-remote-docker: true
          after_checkout:
            - restore_cache:
                key: npm-cache-{{ checksum "package.json" }}
            - run:
                name: Install dependencies
                command: yarn install
            - save_cache:
                key: npm-cache-{{ checksum "package.json" }}
                paths:
                  - ./node_modules

It fails with the following

#!/bin/sh -eo pipefail
# Error calling workflow: 'build-publish-docker-develop'
# Error calling job: 'docker/publish'
# Unexpected executor invocation format
# The value of `name` must be a string
# Examples of correct executor invocation:
#   executor: string-executor-name
#   executor: {name: string-executor-name}
#   executor: << parameters.executor-type-parameter >>
# 
# -------
# Warning: This configuration was auto-generated to show you the message above.
# Don't rerun this job. Rerunning will have no effect.
false
Exited with code 1

Obviously it is expecting something more like executor: my-personal-executor.
Is there a way to use the docker executor from the orb with the parameters as the documentation suggests, or do I have to create an executor separately?

Expected behavior

To use the executor from the orb but with parameters supplied.

Support --cache-from in publish job

The docker build --cache-from option is a very common method for increasing CI build speeds.
For this to work, it's usually preceded by pulling the same images you want to use here.

This need for first pulling the images, I think warrants a separate option for the publish job.
As using extra_build_args would not include the pull step.

Image name used for executor image flake

Orb version

circleci/[email protected]

What happened

On one of our builds: https://circleci.com/gh/sourcecred/sourcecred/2225#tests/containers/0
The executor image was incorrectly set to

  docker/publish-1:
    machine:
      docker_layer_caching: false
      image: sourcecred/sourcecred

Which is the name of the image to publish, not the VM image.
It hasn't done this in previous builds, such as https://circleci.com/gh/sourcecred/sourcecred/2227#config/containers/0
Where it uses the default (as intended)

  docker/publish-1:
    machine:
      docker_layer_caching: false
      image: ubuntu-1604:201903-01

Expected behavior

The image: parameter shouldn't be passed on to the executor.

Feature request: Changelog

I see that the orb was upgraded to v2, which as per semver indicates possibly breaking changes, but I'm not sure what the changes are. It would be great if there was a changelog so I could easily see the changes being made.

Secret Build Args

I'd like to request that there's support for passing in build args that should remain secret, such as passwords that are using in prerequisite build stages.

Unable to specify `hadolint` image with `tag` parameter

Orb version

1.5.0

What happened

We document how to specify an image for hadolint in the orb's documentation:

https://circleci.com/developer/orbs/orb/circleci/docker#executors-hadolint

It should be possible to to override the default latest-debian by specifying the image in the tag parameter. However, this does not appear to be functional.

Slack thread for reference:
https://circleci.slack.com/archives/CEX928FM3/p1620738198453300

Expected behavior

The user should be able to override latest-debian by specifying an orb using the tag parameter.

new compose release breaks scripts

Orb version

1.7.0 (latest)

What happened

New version of docker compose (v2.0.0) have the files with "amd64" instead of "x86_64" in the name, so the curl returns 404 and compose fails to install.

curl: (22) The requested URL returned error: 404

Expected behavior

`docker/check` incorrectly handle registry variable

Orb version

2.1.0

What happened

- docker/check:
          registry: ${CACHE_REGISTRY}
          docker-username: CACHE_USERNAME
          docker-password: CACHE_PASSWORD

The ${CACHE_REGISTRY} used to evaluate to the value. Now it remains as the literal and cannot use 2.1.0 orb.

Orb source 2.1.0

echo "${!PARAM_DOCKER_PASSWORD}" | docker login -u
        "${!PARAM_DOCKER_USERNAME}" --password-stdin "$PARAM_REGISTRY"

Orb source 2.0.5 (notice the << parameters.registry >> vs converting to an env variable.

echo "$<<parameters.docker-password>>" \
          | docker login -u "$<<parameters.docker-username>>" \
              --password-stdin <<parameters.registry>>

Expected behavior

The env variables value should be used and not the literal variable name.

Can't build images on top of images from private registries unless the 'deploy' parameter is set to true

Orb version

2.0

What happened

Docker login takes place only when 'deploy' parameter is set to true, so when I build an image FROM private.registry/repo:tag I can't only build it without pushing to the private registry. Also it's impossible to build FROM private.registry1 and push to private.registry2

Expected behavior

I expect that minimally there's another parameter like 'login-before-build' so I can login to the registry, but without pushing it afterwards.
Ideally I would like to have an option to have one registry to build images on top of images from it and the second registry to push to.

My regards,
Andrew

Support Experimental Features

Orb version

circleci/[email protected]

What happened

Trying to build my image with the --squash command:

      - docker/build:
          extra_build_args: '--squash'
          image: Dockerfile
          tag: 'mytag'

results in:

#!/bin/bash -eo pipefail
docker_tag_args=""

IFS="," read -ra DOCKER_TAGS <<< "${HOST}/node"
for tag in "${DOCKER_TAGS[@]}"; do
  docker_tag_args="$docker_tag_args -t docker.io/Dockerfile:${tag}"
done

docker build --squash \
  -f ./Dockerfile \
  $docker_tag_args \
  .
"--squash" is only supported on a Docker daemon with experimental features enabled

Exited with code exit status 1
CircleCI received exit code 1

Expected behavior

Maybe include a flag to enable experimental features:

      - docker/build:
          experimental: true
          extra_build_args: '--squash'
          image: Dockerfile
          tag: 'mytag'

Add a context parameter to build command

Orb version

0.5.13

What happened

The docker build uses the path as a build context. This makes it impossible to build a Dockerfile that is nested inside a directory in the repository and that uses the files outside of that directory.

Simplest solution is to add a docker-context param that, if not provided, uses the path as a value.

Expected behavior

Be able to use a different directory as context from which the Dockerfile lives.

Problem in "docker/install-docker-compose" script, problem with hash check

Orb version

circleci/[email protected]

What happened

I intalled docker-compose with these steps:

steps:
            - checkout
            - docker/install-docker
            - docker/install-docker-compose

but in the script of install-docker-compose
the script failed to compare hash:

Latest stable version of docker-compose is 1.25.3
A different version of docker-compose is installed (docker-compose version 1.14.0, build c7bdf9e); removing it
sha256sum: standard input: no properly formatted SHA256 checksum lines found

Exited with code exit status 1

Expected behavior

I researched and I think found the problem. Maybe the missing asterisk "*" at the start of the file name is causing the problem.

old file:

5df53b57bba0ad83af69f0f39c15f0316626302aba876954ea842892c875dd6a docker-compose-Linux-x86_64

new file:

786f9390aa8d72bcf34a1757df71577b7d8b7cb915205f72f12b9ee3ca0e0cc6 *docker-compose-Linux-x86_64

[FEATURE] Support env var substitution for extra_build_args or document limitation

This is related to #135 which also deals with support (or lack thereof) for variable expansion.

Description

The parameter for extra_build_args (for example) doesn't support environment variable expansion and are expanded as string literals. (Furthermore, documentation doesn't call out this limitation).

build_args+=("$PARAM_EXTRA_BUILD_ARGS")

In contrast, environment variables ARE expanded by the eval function when used with the tag parameter of docker/build:

docker_arg="--tag=\"$PARAM_REGISTRY/$PARAM_IMAGE:$tag\""
else
docker_arg="$docker_arg --tag=\"$PARAM_REGISTRY/$PARAM_IMAGE:$tag\""
fi
done
DOCKER_TAGS_ARG="$(eval echo $docker_arg)"

Orb version

2.1.1

What happened

Config used

      - docker/build:
          step-name: Build Image
          image: image-name
          path: ./test
          # ${CIRCLE_PULL_REQUEST//*\//} extracts the PR number from the end of the URL.
          tag: pr${CIRCLE_PULL_REQUEST//*\//}-<<pipeline.number>>
          registry: ecr.repo
          docker-context: ./test
          use-buildkit: true
          extra_build_args: "--build-arg tag=pr${CIRCLE_PULL_REQUEST//*\//}-<<pipeline.number>>"

NOTE: In this example, the Dockerfile for image-name uses a base image of FROM ecr.repo/base-image:$tag where tag is configurable via a build argument.

Result

+ docker build --file=./test/Dockerfile --tag=ecr.repo/image-name:pr21287-66010 --build-arg 'tag=pr${CIRCLE_PULL_REQUEST//*\//}-66010' --progress=plain ./test
#1 [internal] load build definition from Dockerfile
#1 sha256:eb014c755523a3e17edaa5ba45823475c0d79e84080bf471329e433bbba40d23
#1 transferring dockerfile: 799B done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 sha256:9a26e70af2b9fa49abfba33982469cde5b1291f497ea89d4136ec92ad02a8531
#2 transferring context: 2B done
#2 DONE 0.0s
failed to solve with frontend dockerfile.v0: failed to create LLB definition: failed to parse stage name "ecr.repo/base-image:pr${CIRCLE_PULL_REQUEST//*\\//}-66010": invalid reference format

NOTE: Notice how the value of --tag=... in the docker build command is expanded correctly, but the value of --build-arg tag=... was not expanded. This is despite the fact that the value provided to both string parameters was identical.

Expected behavior

I'd like to request that extra_build_args get similar treatment to tag to support variable expansion within the docker/build command. This would allow uses can take advantage of built-in variables like $CIRCLE_PULL_REQUEST (as well as custom ones) in their build args for images.

While my immediate need is limited to the extra_build_args parameter of the docker/build command, it's worth considering consistent handling for all parameters in all commands.

However, if there are significant concerns with supporting env var expansion similar to tag for certain parameters, then I think it's worth calling out that limitation because it is not inherently obvious (nor expected) from the parameters type or description as they are currently written.

403 rate limit exceeded error on docker/install-docker-compose

Orb version

0.5.0 / 1.0.0

What happened

Occasionally our circle.ci jobs fail on the following step

      - docker/install-docker-compose:
          version: "1.21.0"

we are getting a 403 rate limit exceeded error (assuming this is triggered when attempting to download docker-compose from github)

A different version of docker-compose is installed (docker-compose version 1.14.0, build c7bdf9e); removing it
curl: (22) The requested URL returned error: 403 rate limit exceeded

Expected behavior

Should install correctly and not hit a rate limit.

This has happened just a couple of times and a build retry has always worked, but was wondering is there is a way to use our organisation token/credentials to bypass this or if there is a different way to install a newer version of docker compose.

BuildKit support

I don't see BuildKit support documented anywhere, but it should be supported in docker 18.09+.

`cache_from` parameter not being used when `use-buildkit: true`

Orb version

2.1.1

What happened

When use-buildkit: true, and cache_from: parameter is populated (in order to pull an image to use as a cache during the docker/build command), the image is pulled but not used as a cache during Docker build (example)

When use-buildkit: false, or not present at all, the pulled image is used as a cache successfully (example)

When cache_from parameter is populated with an envvar, the image is not used as a cache (example)

Expected behavior

When use-buildkit: true, an image should be able to be used as a cache to build another image e.g. the below config should allow for the use of the pulled image as a cache:

      - docker/build:
          cache_from: $AWS_ECR_ACCOUNT_URL/my-app:latest
          image: $AWS_ECR_ACCOUNT_URL/my-app
          tag: latest
          use-buildkit: true

with this in the Docker build output:

Step 2/3 : RUN apt update && apt install -y nginx
 ---> Using cache
 ---> 12345layer

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.