Comments (5)
@krisek thanks for reporting this.
I'd wait for @ivanyu or @hackaugusto to suggest a better alternative, but in the meantime you can try something like this.
( Next step here would be to pin the dependencies of external libs, but for now installing requirements-dev.txt
+ setup.py
)
FROM fedora:33
RUN dnf update -y \
&& dnf install git make python3 python3-devel gcc -y \
&& git clone https://github.com/aiven/karapace.git \
&& cd karapace \
&& make \
&& cd karapace \
&& pip install -r requirements-dev.txt \
&& python setup.py install
podman build -t karapace:fedora-33 -f Dockerfile .
❯ podman run --rm -it localhost/karapace:fedora-33 /bin/bash
[root@ced8e7524167 /]# python
Python 3.9.1 (default, Jan 20 2021, 00:00:00)
[GCC 10.2.1 20201125 (Red Hat 10.2.1-9)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import karapace
>>>
from karapace.
Yes I think this is a valid option, though I was thinking about something more lightweight, that can very easily cover all sorts of versions and stages (and has the perspective of creating production grade containers)
This is what I managed to come up with:
FROM alpine as karapace
RUN apk add py3-multidict py3-yarl py3-pip
COPY karapace-2.0.1.dev126-py3-none-any.whl /tmp
RUN pip install https://github.com/aiven/kafka-python/archive/aiven-20212201.zip && pip install /tmp/karapace-2.0.1.dev126-py3-none-any.whl
CMD karapace
Level 1 target is:
FROM alpine as karapace
RUN apk add py3-multidict py3-yarl py3-pip
RUN pip install https://github.com/aiven/kafka-python/archive/aiven-20212201.zip && pip install https://github.com/aiven/karapace/archive/master.zip
CMD karapace
Level 2 target (if there are plans to publish karapace as pip package):
FROM alpine as karapace
RUN apk add py3-multidict py3-yarl py3-pip
RUN pip install karapace
CMD karapace
And Level 3 would be the same in multistage manner, so that build dependencies (gcc, musl-dev) can be deleted.
Technically very small difference from a Fedora based build, but I think in general it is a good approach to aim for the smallest possible footprint.
Obviously this whole thing is rather a cosmetic thing, no need to waste too much time on it, if complicated to fix. (Maybe file_ver could be set in version.py to something dummy, if built from a git archive.)
from karapace.
I'd suggest going with an mixed approach here. I went over a bit more of detail on this comment #136 (comment) why we use the repo to define the version. With that in mind, if the goal is to have the smallest end image I would have a builder image with the build dependencies, gcc/git/karapace repo/etc., and then only copy the generate .whl
from the builder image to the production image.
Edit: One thing that I forgot to mention, I don't think it is a good idea to have a URL to a dependency in the docker file. It is very easy to have this URL out-of-sync with the requirements.txt
, so it would not be the version that we run our CI with, it can easily lead to unusable images.
from karapace.
a builder image with the build dependencies, gcc/git/karapace repo/etc., and then only copy the generate .whl from the builder image to the production image
This needs that yarn and multidict would be installed from OS package, as they contain non python bindings which need to be compiled.
Now I realized that in my current and level 1 Dockerfile git is not required at all. gcc
and musel-dev
could be avoided if installing py3-multidict
and py3-yarl
would be installed. I updated accordingly the Dockerfile's above.
URLs in docker file was meant only for dev/test builds, pip would be the gratest, but I think for that we need to first wait until Confluent accepts your PR for python-kafka.
from karapace.
Hi @krisek , we started publishing images to docker hub. https://hub.docker.com/r/aivenoy/karapace this resolve the issue.
from karapace.
Related Issues (20)
- Protobuf partial paths aren't expanded during
- Schema registry doesnt start doe to snappy compression error HOT 4
- Git clone of project ends with error HOT 1
- KafkaConfigurationError: Unrecognized configs: {'kafka_client'} HOT 5
- Handle timestamp types in backups
- Updating referenced schema version does not create new version of target schema
- Error and connection reset by peer when trying to run against RedPanda HOT 1
- Querying schema with a first schema version with references returns latest schema version
- Schema not getting register if the order of types are changed.
- schema reader can not replay schema topic HOT 18
- Deprecation warnings during installation
- GHSA-rhrv-645h-fjfh / CVE-2023-39410
- GHSA-r9hx-vwmv-q579 / CVE-2022-40897
- Multiple Topics with same schema HOT 1
- Internal Server Error During Schema Registration
- Corrupt schema, different schemas with the same id found
- Support for JSON log format
- Schema reader crashes when replaying protobuf HOT 7
- Ensure Karapace compatibility with Confluent API
- Karapace Schema Registry: separate permissions for WriteCompatibility operation for subjects
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from karapace.