yagajs / docker-mapproxy Goto Github PK
View Code? Open in Web Editor NEWDocker files for mapproxy container
License: ISC License
Docker files for mapproxy container
License: ISC License
Currently only a few uwsgi
startup-options can be specified by the Docker Container user: mainly MAPPROXY_PROCESSES
and MAPPROXY_THREADS
via env vars. Also not in all contexts is stats
(--stats
) publishing required.
I came to this issue when trying to optimize uwsgi
for optimal MapProxy performance. For example this article suggests to disable logging (via --disable-logging
) and as MapProxy is stateless set Max Worker Lifetime (e.g. --max-worker-lifetime 30
). This could all-in-all amount for over 30% performance improvement.
So when using env-vars ( uwsgi.ini
is another option, see below) my suggestion is to:
--stats
optionUWSGI_EXTRA_OPTIONS
optional env-var, that is appended to the existing startup optionsA user could specify e.g. in a docker-compose
file: UWSGI_EXTRA_OPTIONS="--disable-logging --max-worker-lifetime 30"
. This would be forward compatible (except for the stats option).
Another option would be to allow a UWSGI_OPTIONS
where a user specifies a complete startup-options string.
The most flexible would be to start uwsgi
with an uwsgi.ini
file with a default that can be overridden e.g. by Docker-Volume mounting or extending the Docker Image. But at the same time this may create inflexibilities in environments where Docker Volume mounting is less preferred over env-vars like Kubernetes.
apk complains that .build-deps blocks removal of build-base
+ apk del build-base
WARNING: Ignoring APKINDEX.b89edf6e.tar.gz: No such file or directory
WARNING: Ignoring APKINDEX.737f7e01.tar.gz: No such file or directory
World updated, but the following packages are not removed due to:
build-base: .build-deps
OK: 260 MiB in 77 packages
MP 1.13.2 has been out for some time. Fixes some relevant stuff.
Already tested.
Will fix in https://github.com/justb4/docker-mapproxy and related DockerHub Images from that repo.
Mapproxy has released 1.12.0. Could you please upgrade to the latest version?
There seems to be some incompatibilities with the actual python version. Let’s switch back to the python 3.5 image as base for this image
Forked/cloned this repo and tried to build 1.11.0-alpine:
git checkout remotes/origin/version/1.11.0-alpine
docker build -t map5/mapproxy .
Sending build context to Docker daemon 96.77kB
Step 1/14 : FROM python:3-alpine
---> 1a8edcb29ce4
Step 2/14 : MAINTAINER Arne Schubert<[email protected]>
---> Using cache
---> 2bba68471117
Step 3/14 : ARG MAPPROXY_VERSION=1.11.0
---> Using cache
---> b35e1deec43f
Step 4/14 : ENV MAPPROXY_VERSION $MAPPROXY_VERSION
---> Using cache
---> d707738277ce
Step 5/14 : ENV MAPPROXY_PROCESSES 4
---> Using cache
---> 9e79c91e6fbe
Step 6/14 : ENV MAPPROXY_THREADS 2
---> Using cache
---> 03efc934a030
Step 7/14 : RUN set -x && apk add --no-cache --virtual .build-deps build-base linux-headers curl zlib-dev jpeg-dev && apk add --no-cache --repository http://dl-cdn.alpinelinux.org/alpine/edge/testing gdal-dev proj4-dev geos-dev && adduser -h /mapproxy -s /bin/sh -D mapproxy && pip install Shapely Pillow requests geojson uwsgi pyproj boto3 MapProxy==$MAPPROXY_VERSION && apk del build-base && rm -Rf /root/* /root/.cache && mkdir -p /docker-entrypoint-initmapproxy.d
---> Running in 657033671cf0
+ apk add --no-cache --virtual .build-deps build-base linux-headers curl zlib-dev jpeg-dev
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gz
(1/29) Installing binutils (2.30-r5)
(2/29) Installing libmagic (5.32-r0)
(3/29) Installing file (5.32-r0)
(4/29) Installing gmp (6.1.2-r1)
(5/29) Installing isl (0.18-r0)
(6/29) Installing libgomp (6.4.0-r9)
(7/29) Installing libatomic (6.4.0-r9)
(8/29) Installing pkgconf (1.5.3-r0)
(9/29) Installing libgcc (6.4.0-r9)
(10/29) Installing mpfr3 (3.1.5-r1)
(11/29) Installing mpc1 (1.0.3-r1)
(12/29) Installing libstdc++ (6.4.0-r9)
(13/29) Installing gcc (6.4.0-r9)
(14/29) Installing musl-dev (1.1.19-r10)
(15/29) Installing libc-dev (0.7.1-r0)
(16/29) Installing g++ (6.4.0-r9)
(17/29) Installing make (4.2.1-r2)
(18/29) Installing fortify-headers (0.9-r0)
(19/29) Installing build-base (0.5-r1)
(20/29) Installing linux-headers (4.4.6-r2)
(21/29) Installing nghttp2-libs (1.32.0-r0)
(22/29) Installing libssh2 (1.8.0-r3)
(23/29) Installing libcurl (7.61.1-r1)
(24/29) Installing curl (7.61.1-r1)
(25/29) Installing zlib-dev (1.2.11-r1)
(26/29) Installing libjpeg-turbo (1.5.3-r3)
(27/29) Installing libjpeg-turbo-dev (1.5.3-r3)
(28/29) Installing jpeg-dev (8-r6)
(29/29) Installing .build-deps (0)
Executing busybox-1.28.4-r2.trigger
OK: 186 MiB in 63 packages
+ apk add --no-cache --repository http://dl-cdn.alpinelinux.org/alpine/edge/testing gdal-dev proj4-dev geos-dev
fetch http://dl-cdn.alpinelinux.org/alpine/edge/testing/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.8/community/x86_64/APKINDEX.tar.gz
ERROR: unsatisfiable constraints:
so:libcrypto.so.1.1 (missing):
required by:
gdal-2.3.2-r1[so:libcrypto.so.1.1]
gdal-2.3.2-r1[so:libcrypto.so.1.1]
gdal-2.3.2-r1[so:libcrypto.so.1.1]
gdal-2.3.2-r1[so:libcrypto.so.1.1]
gdal-2.3.2-r1[so:libcrypto.so.1.1]
gdal-2.3.2-r1[so:libcrypto.so.1.1]
gdal-2.3.2-r1[so:libcrypto.so.1.1]
gdal-2.3.2-r1[so:libcrypto.so.1.1]
gdal-2.3.2-r1[so:libcrypto.so.1.1]
gdal-2.3.2-r1[so:libcrypto.so.1.1]
gdal-2.3.2-r1[so:libcrypto.so.1.1]
gdal-2.3.2-r1[so:libcrypto.so.1.1]
gdal-2.3.2-r1[so:libcrypto.so.1.1]
gdal-2.3.2-r1[so:libcrypto.so.1.1]
gdal-2.3.2-r1[so:libcrypto.so.1.1]
gdal-2.3.2-r1[so:libcrypto.so.1.1]
gdal-2.3.2-r1[so:libcrypto.so.1.1]
gdal-2.3.2-r1[so:libcrypto.so.1.1]
gdal-2.3.2-r1[so:libcrypto.so.1.1]
gdal-2.3.2-r1[so:libcrypto.so.1.1]
gdal-2.3.2-r1[so:libcrypto.so.1.1]
gdal-2.3.2-r1[so:libcrypto.so.1.1]
gdal-2.3.2-r1[so:libcrypto.so.1.1]
gdal-2.3.2-r1[so:libcrypto.so.1.1]
gdal-2.3.2-r1[so:libcrypto.so.1.1]
gdal-2.3.2-r1[so:libcrypto.so.1.1]
The command '/bin/sh -c set -x && apk add --no-cache --virtual .build-deps build-base linux-headers curl zlib-dev jpeg-dev && apk add --no-cache --repository http://dl-cdn.alpinelinux.org/alpine/edge/testing gdal-dev proj4-dev geos-dev && adduser -h /mapproxy -s /bin/sh -D mapproxy && pip install Shapely Pillow requests geojson uwsgi pyproj boto3 MapProxy==$MAPPROXY_VERSION && apk del build-base && rm -Rf /root/* /root/.cache && mkdir -p /docker-entrypoint-initmapproxy.d' returned a non-zero code: 3
Now this is a known issue also found e.g. here:
postgis/docker-postgis#96, saw several fixes but adding http://dl-cdn.alpinelinux.org/alpine/edge/main
worked for me:
&& apk add --no-cache \
--repository http://dl-cdn.alpinelinux.org/alpine/edge/main \
--repository http://dl-cdn.alpinelinux.org/alpine/edge/testing \
gdal-dev proj4-dev geos-dev \
After build ok, next problem is caused by Python 3.7 incompatibility with MapProxy:
docker run -p 8080:8080 map5/mapproxy mapproxy http
Running additional provisioning
Traceback (most recent call last):
File "/usr/local/bin/mapproxy-util", line 7, in <module>
from mapproxy.script.util import main
File "/usr/local/lib/python3.7/site-packages/mapproxy/script/util.py", line 29, in <module>
from mapproxy.script.defrag import defrag_command
File "/usr/local/lib/python3.7/site-packages/mapproxy/script/defrag.py", line 26, in <module>
from mapproxy.cache.tile import Tile
File "/usr/local/lib/python3.7/site-packages/mapproxy/cache/tile.py", line 45
from mapproxy.util import async
^
SyntaxError: invalid syntax
This can be solved by pinning the Python (and Alpine) version, resulting also in a more reproducable build, i.s.o.:
FROM python:3-alpine
using
FROM python:3.6.8-alpine3.8
Now build and run are ok. Can issue PR if needed.
Create a Dockerfile for the latest version and for all versions available, until version 1.0.0.
This concerns https://github.com/justb4/docker-mapproxy/blob/master/Dockerfile (my fork). The Dockerfile installs both python3-proj
and the PyPi package pyproj
. Think I first added python3-proj
and then pyproj
later as for some build errors. My question arises, since I think one gets two full and even different versions installed at different locations /usr/
(python3-proj
) and /usr/local
(pyproj
).
I was triggered by this warning:
/usr/lib/python3/dist-packages/pyproj/__init__.py:91: UserWarning: Valid PROJ data directory not found. Either set the path us
ing the environmental variable PROJ_DATA (PROJ 9.1+) | PROJ_LIB (PROJ<9.1) or with `pyproj.datadir.set_data_dir`.
warnings.warn(str(err))
"Bashing" into that running MapProxy image I see:
$ /usr/bin/pyproj -v
pyproj info:
pyproj: 3.4.1
PROJ: 9.1.1
data dir: /usr/share/proj
user_data_dir: /tmp/proj
PROJ DATA (recommended version): 1.12
PROJ Database: 1.2
EPSG Database: v10.076 [2022-08-31]
ESRI Database: ArcGIS Pro 3.0 [2022-07-09]
IGNF Database: 3.1.0 [2019-05-24]
System:
python: 3.11.2 (main, Mar 13 2023, 12:18:29) [GCC 12.2.0]
executable: /usr/bin/python3
machine: Linux-5.4.0-182-generic-x86_64-with-glibc2.36
Python deps:
certifi: 2022.9.24
Cython: None
setuptools: None
pip: None
And (the default)
$ /usr/local/bin/pyproj -v
pyproj info:
pyproj: 3.6.1
PROJ: 9.3.0
data dir: /usr/local/lib/python3.11/site-packages/pyproj/proj_dir/share/proj
user_data_dir: /tmp/proj
PROJ DATA (recommended version): 1.15
PROJ Database: 1.2
EPSG Database: v10.094 [2023-08-08]
ESRI Database: ArcGIS Pro 3.1 [2023-19-01]
IGNF Database: 3.1.0 [2019-05-24]
System:
python: 3.11.9 (main, May 14 2024, 08:15:15) [GCC 12.2.0]
executable: /usr/local/bin/python
machine: Linux-5.4.0-182-generic-x86_64-with-glibc2.36
Python deps:
certifi: 2024.2.2
Cython: None
setuptools: 65.5.1
pip: 24.0
The latter is the default:
$ which pyproj
/usr/local/bin/pyproj
Setting an Env var for the Docker Container PROJ_DATA= /usr/local/lib/python3.11/site-packages/pyproj/proj_dir/share/proj
clears the error message (see above), but it all feels brittle.
The PyPi version pyproj
also is more recent. Can I assume that we could leave out python3-proj
? Or do we have to install a libproj-dev
?
According to this issue, we should add a ISC license to the project...
Hi,
Since version 1.10.0, MapProxy supports S3 backend. However, it requires the boto3 package to be installed (https://mapproxy.org/docs/nightly/caches.html#cache-s3).
It could be worth to intergrate it. What are your thoughts on this subject?
Thanks.
Mapproxy has a new release 1.11.0. Add the newest version
We should upgrade all mapproxy images to python 3, like @The-Alchemist showed in his pull request #2.
In addition we should fix the problem with adding configurations into the container like fixed in #1!
How can I secure the Demo Page with SSL/TLS certificate? Are there any environment variables I can use to assign and utilize certificates issued from our CA?
With the help of this, I was able to fix the lib c bug of Python's util, which will get fixed in v3.8 and causes trouble with Shapely. Sure, there might be a better patch than the Alpine one, but this one is working fine in this case.
My edit of this dockerfile isn't elegant, but it drops all the building dependencies (f.e. the huge libc & gcc stuff) and results in a small image of less than 30 MB:
FROM python:3.5-alpine
MAINTAINER Arne Schubert<[email protected]>
ARG MAPPROXY_VERSION=1.11.0
ENV MAPPROXY_VERSION $MAPPROXY_VERSION
ENV MAPPROXY_PROCESSES 4
ENV MAPPROXY_THREADS 2
RUN set -x \
&& printf "%s\n%s\n" '#!/bin/sh' 'if [ "$1" = "-p" ]; then echo " libc.musl-x86_64.so.1 (libc6,x86-64) => /lib/libc.musl-x86_64.so.1"; exit 0; fi' > /tmp/ldconfig \
&& tail -n+2 /sbin/ldconfig >> /tmp/ldconfig \
&& mv /tmp/ldconfig /sbin/ldconfig \
&& chmod +x /sbin/ldconfig \
&& apk add --update --no-cache --virtual .build-deps-stable build-base linux-headers \
&& apk add --update --no-cache --virtual .build-deps-edge \
--repository http://dl-cdn.alpinelinux.org/alpine/edge/main \
--repository http://dl-cdn.alpinelinux.org/alpine/edge/testing \
gdal-dev proj4-dev geos-dev zlib-dev jpeg-dev \
&& adduser -h /mapproxy -s /bin/sh -D mapproxy \
&& pip install --upgrade pip \
&& pip install Shapely Pillow requests geojson uwsgi pyproj==1.9.6 boto3 MapProxy==$MAPPROXY_VERSION \
&& apk del .build-deps-stable .build-deps-edge \
&& runDeps="geos giflib libcrypto1.1 libcurl libgcc libjpeg-turbo libpng libpq libstdc++ musl sqlite-libs tiff zlib" \
&& apk add --no-cache --virtual .run-deps $runDeps \
--repository http://dl-cdn.alpinelinux.org/alpine/edge/main \
--repository http://dl-cdn.alpinelinux.org/alpine/edge/testing \
&& rm -Rf /root/* /root/.cache \
&& mkdir -p /docker-entrypoint-initmapproxy.d
COPY docker-entrypoint.sh /
ENTRYPOINT ["/docker-entrypoint.sh"]
CMD ["mapproxy"]
USER mapproxy
VOLUME ["/mapproxy"]
EXPOSE 8080
# Stats
EXPOSE 9191
Maybe would could just set the automatically installed packages on "hold", like the DPKG-way, to protect them from automatic removal?
MAPPROXY_VERSION is set to 1.10.1 (https://github.com/yagajs/docker-mapproxy/blob/version/1.10.4/Dockerfile#L4). It should be 1.10.4
After some tests I created a mapproxy based on python:alpine, but I had never make a commit in git.
Let's add the alpine and a rpi (respberry-pi) versions of each version of the image...
Currently, building fails, since pyproj >= v2, needs proj >= v6
ERROR: Complete output from command /usr/local/bin/python /usr/local/lib/python3.5/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmprpmfab0y:
ERROR: ERROR: Minimum supported proj version is 6.0.0, installed version is 5.2.0.
----------------------------------------
ERROR: Command "/usr/local/bin/python /usr/local/lib/python3.5/site-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmprpmfab0y" failed with error code 1 in /tmp/pip-install-vlxcmfol/pyproj
Freeze pyproj until proj gets updated: pip install [...] pyproj==1.9.6
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.