karapace/version.py missing when building with pip about karapace HOT 5 CLOSED

@krisek thanks for reporting this.
I'd wait for @ivanyu or @hackaugusto to suggest a better alternative, but in the meantime you can try something like this.

( Next step here would be to pin the dependencies of external libs, but for now installing requirements-dev.txt + setup.py )

FROM fedora:33

RUN dnf update -y \
    && dnf install git make python3 python3-devel gcc -y  \
    && git clone https://github.com/aiven/karapace.git \
    && cd karapace \
    && make \
    && cd karapace \
    && pip install -r requirements-dev.txt \
    && python setup.py install

podman build -t karapace:fedora-33 -f Dockerfile .

❯ podman run --rm -it localhost/karapace:fedora-33 /bin/bash
[root@ced8e7524167 /]# python
Python 3.9.1 (default, Jan 20 2021, 00:00:00) 
[GCC 10.2.1 20201125 (Red Hat 10.2.1-9)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import karapace
>>> 

from karapace.

Yes I think this is a valid option, though I was thinking about something more lightweight, that can very easily cover all sorts of versions and stages (and has the perspective of creating production grade containers)

This is what I managed to come up with:

FROM alpine as karapace
RUN apk add py3-multidict py3-yarl py3-pip
COPY karapace-2.0.1.dev126-py3-none-any.whl /tmp
RUN  pip install https://github.com/aiven/kafka-python/archive/aiven-20212201.zip && pip install /tmp/karapace-2.0.1.dev126-py3-none-any.whl
CMD karapace 

Level 1 target is:

FROM alpine as  karapace
RUN apk add py3-multidict py3-yarl py3-pip
RUN pip install https://github.com/aiven/kafka-python/archive/aiven-20212201.zip && pip install https://github.com/aiven/karapace/archive/master.zip
CMD karapace

Level 2 target (if there are plans to publish karapace as pip package):

FROM alpine as karapace
RUN apk add py3-multidict py3-yarl py3-pip
RUN pip install karapace
CMD karapace

And Level 3 would be the same in multistage manner, so that build dependencies (gcc, musl-dev) can be deleted.

Technically very small difference from a Fedora based build, but I think in general it is a good approach to aim for the smallest possible footprint.

Obviously this whole thing is rather a cosmetic thing, no need to waste too much time on it, if complicated to fix. (Maybe file_ver could be set in version.py to something dummy, if built from a git archive.)

from karapace.

I'd suggest going with an mixed approach here. I went over a bit more of detail on this comment #136 (comment) why we use the repo to define the version. With that in mind, if the goal is to have the smallest end image I would have a builder image with the build dependencies, gcc/git/karapace repo/etc., and then only copy the generate .whl from the builder image to the production image.

Edit: One thing that I forgot to mention, I don't think it is a good idea to have a URL to a dependency in the docker file. It is very easy to have this URL out-of-sync with the requirements.txt, so it would not be the version that we run our CI with, it can easily lead to unusable images.

from karapace.

a builder image with the build dependencies, gcc/git/karapace repo/etc., and then only copy the generate .whl from the builder image to the production image

This needs that yarn and multidict would be installed from OS package, as they contain non python bindings which need to be compiled.

Now I realized that in my current and level 1 Dockerfile git is not required at all. gcc and musel-dev could be avoided if installing py3-multidict and py3-yarl would be installed. I updated accordingly the Dockerfile's above.

URLs in docker file was meant only for dev/test builds, pip would be the gratest, but I think for that we need to first wait until Confluent accepts your PR for python-kafka.

from karapace.

Hi @krisek , we started publishing images to docker hub. https://hub.docker.com/r/aivenoy/karapace this resolve the issue.

from karapace.