Hey,

Currently, after every img build that builder-task executes, an img save
follows it.

While that's usually the desired case, for leveraging Dockerfiles as a way of
running tests with all of the dependencies, it doesn't make much sense - just
knowing if the build succeeds or fails is enough.

For instance, consider the following job:

jobs:
  - name: build
    public: true
    plan:
      - {get: repository, trigger: true}
      - in_parallel:
          fail_fast: true
          steps:
            - task: build-regular
              file: repository/ci/tasks/build.yml
              params: {TARGET: "release"}
              privileged: true
            - task: test
              file: repository/ci/tasks/build.yml
              output_mapping: { image: discard }  # <<<
              params: {TARGET: "test"}
              privileged: true
      - put: container-image
        inputs: [image]
        get_params: {format: oci}
        params:
          image: image/image.tar

My proposal is:

• have a NO_SAVE (or something with a better name lol) parameter that allows
  build to not be followed by a save.

Wdyt?

thx!

When a file containing key/value pairs is supplied to BUILD_ARGS_FILE, it must not contain empty lines - doing so will cause an empty --build-arg parameter to be appended to the command. The build will then fail.

To reproduce, supply the parameter BUILD_ARGS_FILE, with a file containing the following:

FOO=bar
BAR=eep

(note the extra whitespace)

Our build process requires us to provide a generic tag + unique tag. For instance

 foo:development
 foo:4824be76-0335-4f00-aef8-59c129db289d

which was possible with the docker-image put using additional_tags. I can't replicate this using the concourse/builder-task and I'm wondering if there is an approach that allows for this?

The only way I can think of is to have two concourse/builder-task task steps which rely on caching to avoid rebuilding the image (But with different TAG variables)

We are doing some fast loop CI with an insecure local registry. This is already supported in the old docker image resource with insecure_registries, it would be great to have here too

Could we provide similar function like tag_file in docker-image-resource? Only support params.TAG is not very helpful for the dynamically generated tag.

Thanks :)

Hi,

I am a little unsure how this would work with Concourse v5+ (managed to get it to work with previous versions). When trying to put the output, the concourse complains since it is expecting a resource for put. As no real resource is created (this is just a task), I am not sure how to share the image across multiple jobs.

Thanks

The example.yml is confusing and unreadable. Is it a task? How should I integrate it? What is it exactly building? Where does it get the resources from?

It would also be helpful to have an example how to integrate with registry-image-resource (something similar as in concourse/registry-image-resource#40), as most of us will be integrating with that.

If I wasn't "obliged" to use this for my project, I would have thrown it away immediately because of the lack of clear examples. Yet it costed me way too much time to get this thing to work.

It appears that img reaches out to Docker Hub during the build task:

failed to solve: failed to do request: Head https://registry-1.docker.io/v2/tonistiigi/copy/manifests/sha256:d9d49bedbbe2b27df88115e6aff7b9cd11ed2fbd8d9013f02d3da735c08c92e5: dial tcp 34.200.28.105:443: connect: no route to host

Related:

• genuinetools/img#165
• moby/buildkit#706

Hello! At @wireapp we're trying to cut down our docker build times in concourse and think this builder may help; I have it rigged up in my pipeline; but I need to pass additional arguments at build time such as:

• --env blah=val
• --build-args blah=val

Taking a quick look at the build script looks as though there isn't a way to do this currently;

Am I missing a way to do this? If not; do you have plans to implement this, or a suggestion for how you'd like it to be done so I can submit a Pull request? Thanks!

Cheers 🍻

while there is support for build caches already, this doesn't solve a use case where you want to start with layers from a previous image build without having a local build cache

ideally, I would be able to provide an input image.tar file as an input cache

this image would then be loaded before the build starts, so any cached layers (with no changes) are not rebuilt

similar to a "docker load", except since this task library doesn't really deal with repositories, that could be handled by an upstream resource that saves the image.tar file and passes it to this task

not sure if this is possible based on a cursory glance at img itself, but I wanted to toss out this use case either way

Hey,

It's stated that builder has the stretch goal of running without the need of privileged: true, and looking at img#upstream-patches, it feels like the base functionality is already there at runc level, with only the Docker CLI missing in terms of catching up with the functionality.

What does that mean for us, having the garden interface between ourselves and runc?

Are there specific issues that we could track to know how far we are from having this not running with privileged: true?

Thanks!

Using this task together with registry-image-resource, I'm getting weird errors from artifactory docker registry when trying to upload an image:

INFO[0000] pushing sha256:9231c75dfa48563b354f4d40126d077a3ef5029718c352a53120a16e53704560 to foo-docker-local.jfrog.io/foo/my-image:latest
2019/04/08 19:38:55 Unsolicited response received on idle HTTP channel starting with "0\r\n\r\n"; err=<nil>
2019/04/08 19:38:55 existing blob: sha256:7a817918d4d52ca5e7c3c57d904971a321843ba919e35bab135452b2f4f23b66
...
2019/04/08 19:38:56 pushed blob: sha256:505298b4cecec8ac9200ea514da5f7bdcbb8e5f0950cdc461a51ee697338b8d1
...
ERRO[0003] failed to upload image: MANIFEST_INVALID: "manifest invalid" 

Initially I suspected there was a problem between registry-image-resource and artifactory, but then I narrowed it down to img by building the image with docker daemon and pushing it using registry-image-resource.
Building the image using this task and then pushing it using the old docker-image resource (via load_file) works, too.
GoogleContainerTools/jib#534 may or may not be related.
The issue is still reproducible with img 0.5.6.

I'm posting this here to make it easier for other concourse users to find.

With current version (0.5.1), login to private registry fail with dns queries Oo upgrading to 0.5.6 fixes this issue.

Could we allow user-defined output for saved image tarball? (Like params.OUTPUT)

Hey,

I noticed that for some builds that take a considerable amount of time, the output of img will produce a pretty huge output (given that it keeps replacing the text in the screen to provide the updates).

That seems to make Concourse's web UI take a considerable amount to load.

I'm not very into how much we can configure in terms of img's output, but I think it'd be nice if we could make that output not use such features.

Do you think that's a feasible thing?

Thanks!

Trying to build an image which references an image on AWS ECR.

Logs:

 => [internal] load Dockerfile                                             0.1s
 => => transferring dockerfile: 161B                                       0.0s
 => [internal] load .dockerignore                                          0.1s
 => => transferring context: 56B                                           0.0s
 => ERROR [internal] load metadata for 266174327389.dkr.ecr.eu-west-1.am  10.0s
 => ERROR [1/2] FROM 266174327389.dkr.ecr.eu-west-1.amazonaws.com/shared  10.0s
 => => resolve 266174327389.dkr.ecr.eu-west-1.amazonaws.com/shared-mysql  10.0s
 => [internal] load build context                                          0.2s
 => => transferring context: 6.88MB                                        0.1s
 => CANCELED [internal] helper image for file operations                  10.0s
 => => resolve docker.io/tonistiigi/copy:v0.1.4@sha256:d9d49bedbbe2b27df  10.0s
25h------
 > [internal] load metadata for 266174327389.dkr.ecr.eu-west-1.amazonaws.com/shared-mysql:latest:
------
------
 > [1/2] FROM 266174327389.dkr.ecr.eu-west-1.amazonaws.com/shared-mysql:latest:
------
failed to solve: failed to do request: Head https://266174327389.dkr.ecr.eu-west-1.amazonaws.com/v2/shared-mysql/manifests/latest: dial tcp: lookup 266174327389.dkr.ecr.eu-west-1.amazonaws.com on 172.17.0.2:53: read udp 10.254.0.38:41079->172.17.0.2:53: i/o timeout

Wouldn't it be nice if images build with this task could be run without uploading?
It seems to me this is currently not possible to do something like this:

    - task: build
      privileged: true
      file: input.git/ci/build.yaml
    - task: test
      image: image/
      file: input.git/ci/run.yaml

According to https://concourse-ci.org/tasks.html#task-image-resource

You can use any resource that returns a filesystem in the correct format (a /rootfs directory and a metadata.json file in the top level)

Currently this task outputs a tarball of the image which does not work with the image: key I suppose.

Is this correct or I'm I missing something? has this been done by somebody already?
If its missing I guess the tricky part would be get the content of metadata.json correct.

Is that possible to provide a resource-type for image building? What's the pros and cons of using builder-task? Thanks :)

Hi,

when using the build-task i am getting the following error in the last step:

#12 exporting to image

#12       digest:
sha256:0be3e8ffcff73e71831a5c7e63ba0124fc0fc779b08a6f8a49a1af9b94eea7af

#12         name: "exporting to image"

#12      started: 2019-08-05 12:14:57.909369453 +0000 UTC m=+63.845516601	

#12 exporting layers

#12    completed: 2019-08-05 12:15:01.400674152 +0000 UTC m=+67.336821294

#12     duration: 3.491304693s

#12        error: "failed calculating diff pairs for exported snapshot: mount callback failed on /run/user/1000/containerd-mount895903867: mount callback failed on /run/user/1000/containerd-mount431519333: failed to write compressed diff: failed to create diff tar stream: stat /run/user/1000/containerd-mount431519333/opt/app/node_modules/ansi: no such file or directory"

any ideas? is it an upstream error in https://github.com/genuinetools/img ?

Is there any chance of tagging known good builds please, kind folk?

P.S.

Although it's sadly slower than the docker-image-resource, I do prefer this method as the build output is much more granular. So thankyou!

In UI I can see spinning circle on step to build image. It takes around 2 minutes. Building image after this 2 minutes is + 5-20 seconds depend on cache.

I have super simple Dockerfile for tests:
FROM nginx

I have installed concourse by helm in kubernetes.

Summary up: It takes around 2 minutes freeze each time to start.

• I was trying it about 10 times with cache and without
• each time UI spinning circle is showing and there is no output in task
• after this this 2 minutes delay it start to build and success

How I run it in jobs:

      - task: build-image
        privileged: true
        config:
          platform: linux
          image_resource:
            type: registry-image
            source: {repository: concourse/builder-task}
          params:
            REPOSITORY: kwladyka/ory-hydra-auth-provider
            TAG: latest
            CONTEXT: auth-provider:master
          inputs:
          - name: auth-provider:master
          outputs:
          - name: image
          caches:
          - path: cache
          run: {path: build}

I saw the build script uses rsync to cp the state back and forth between local drive and cache drive. Curious what is the particular reason for doing that?

I have the following definitions in the gist:
https://gist.github.com/dhonig/566354a1b23974819b1eb0c456b6a70e

What I am attempting to do is to store the GIT commit SHA in combined-sources and then push the image by the git commit sha.

So early on in my build I write the sha to combined-sources/tmp/commit.txt
The image should be built by this tag with
params:
REPOSITORY: dhonig/my-api
TAG_FILE: combined-sources/tmp/commit.txt
CONTEXT: combined-sources

But I only see "latest" in the logs:

building
18:25:54 Building docker.io/dhonig/my-api:latest
18:25:54 Setting up the rootfs... this may take a bit.
18:25:55 25l[+] Building 228.6s (30/30) FINISHED

What I would expect is that the image is being built with the contents of combined-sources/tmp/commit.txt

How can I read the commit sha in from a file and then push that tag back out using the registry-image-resource?