renovatebot / docker-buildpack Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
License: MIT License
Let's deprecate renovate/ubuntu
and have everything here
Hi, any chance we might see docker image being upgraded to the latest LTS release, i.e. 22.04 ?
Would be nice to have, due to variety of deps which are now old... such as curl (using it in postpostUpgradeTasks
script)
/usr/local/etc/env
file is sourced two times. we should do this only one time
renovate/buildpack:ruby
or renovate/buildpack:5-ruby
, they won't be updated.As per discussion in renovatebot/renovate#6610
This issue lists Renovate updates and detected dependencies. Read the Dependency Dashboard docs to learn more.
These updates await pending status checks. To force their creation now, click the checkbox below.
These are blocked by an existing closed PR and will not be recreated unless you click a checkbox below.
Dockerfile
containerbase/buildpack 4.16.8@sha256:7fbb21fe2441a497b12e1fb3076e6799fcf9a37189906f4500dfff559591eb99
ubuntu focal@sha256:450e066588f42ebe1551f3b1a535034b6aa46cd936fe7f2c6b0d72997ec61dbd
test/dotnet/Dockerfile
test/erlang/Dockerfile
test/golang/Dockerfile
test/helm/Dockerfile
test/java/Dockerfile
test/latest/Dockerfile
test/nix/Dockerfile
test/node/Dockerfile
test/php/Dockerfile
test/powershell/Dockerfile
test/python/Dockerfile
test/ruby/Dockerfile
test/rust/Dockerfile
test/swift/Dockerfile
.github/workflows/build.yaml
actions/checkout v3.1.0@93ea575cb5d8a053eaa0ac8fa3b40d7e05a33cc8
sigstore/cosign-installer v2.8.1@9becc617647dfa20ae7b1151972e9b3a2c338a2b
renovatebot/internal-tools v1.14.6@8b03c470dacbf6be4199308b06ef467c5d9f5cfc
test/golang/test/a/go.mod
test/golang/test/b/go.mod
test/golang/test/c/go.mod
test/node/test/a/package.json
test/dotnet/test/test.csproj
test/python/test/f/requirements.txt
test/python/test/a/Pipfile
test/python/test/c-poetry/pyproject.toml
test/python/test/d-poetry/pyproject.toml
test/latest/Dockerfile
python 3.10.8
Dockerfile
git v2.38.1
test/dotnet/Dockerfile
dotnet 3.1.424
dotnet 6.0.402
test/erlang/Dockerfile
erlang 24.3.4.6
elixir 1.14.1
test/golang/Dockerfile
golang 1.19.3
test/helm/Dockerfile
helm v3.10.1
test/java/Dockerfile
java 18.0.2+9
gradle 7.5.1
test/latest/Dockerfile
node v16.18.0
powershell v7.2.7
test/nix/Dockerfile
nix 2.11.1
test/node/Dockerfile
node v16.18.0
yarn 1.22.19
pnpm 7.14.0
node v16.18.0
yarn 1.22.19
test/php/Dockerfile
php 8.1.12
test/powershell/Dockerfile
powershell v7.2.7
test/python/Dockerfile
python 3.10.8
python 3.10.8
pipenv 2022.10.25
poetry 1.2.2
python 3.10.8
poetry 1.2.2
pipenv 2022.10.25
poetry 1.2.2
hashin 0.17.0
test/ruby/Dockerfile
ruby 3.1.2
bundler 2.3.24
cocoapods 1.11.3
test/rust/Dockerfile
rust 1.64.0
rust 1.64.0
test/swift/Dockerfile
swift 5.7.0
swift 5.7.0
test/swift/test/a/Package.swift
test/swift/test/b/Package.swift
test/swift/test/c/Package.swift
needed for new scoop package manager
We should test the scripts with some sample docker images
What Renovate type are you using?
Not self-hosted, GitHub App
Describe the bug
Renovate is no longer updating my poetry.lock
when updating pyproject.toml
The issue seems similar to renovatebot/renovate#4843 as both files are in a subfolder (app
)
I am also getting "Artifact update problem" which I think my be why the lockfile isn't created
It says _bz2
is not found - I think this is a dependency that is just included with python so I don't know why it's not found
Did you see anything helpful in debug logs?
Collecting poetry==0.12.17
Downloading poetry-0.12.17-py2.py3-none-any.whl (195 kB)
Collecting shellingham<2.0,>=1.1
Downloading shellingham-1.3.2-py2.py3-none-any.whl (11 kB)
Collecting requests-toolbelt<0.9.0,>=0.8.0
Downloading requests_toolbelt-0.8.0-py2.py3-none-any.whl (54 kB)
Collecting tomlkit<0.6.0,>=0.5.1
Downloading tomlkit-0.5.11-py2.py3-none-any.whl (31 kB)
Collecting pkginfo<2.0,>=1.4
Downloading pkginfo-1.5.0.1-py2.py3-none-any.whl (25 kB)
Collecting jsonschema<4.0,>=3.0a3
Downloading jsonschema-3.2.0-py2.py3-none-any.whl (56 kB)
Collecting requests<3.0,>=2.18
Downloading requests-2.23.0-py2.py3-none-any.whl (58 kB)
Collecting pyparsing<3.0,>=2.2
Downloading pyparsing-2.4.7-py2.py3-none-any.whl (67 kB)
Collecting cachy<0.3,>=0.2
Downloading cachy-0.2.0-py2.py3-none-any.whl (59 kB)
Collecting html5lib<2.0,>=1.0
Downloading html5lib-1.0.1-py2.py3-none-any.whl (117 kB)
Collecting pyrsistent<0.15.0,>=0.14.2
Downloading pyrsistent-0.14.11.tar.gz (104 kB)
Collecting cleo<0.7.0,>=0.6.7
Downloading cleo-0.6.8-py2.py3-none-any.whl (264 kB)
Collecting cachecontrol[filecache]<0.13.0,>=0.12.4
Downloading CacheControl-0.12.6-py2.py3-none-any.whl (19 kB)
Requirement already satisfied: setuptools in /usr/local/python/3.8.2/lib/python3.8/site-packages (from jsonschema<4.0,>=3.0a3->poetry==0.12.17) (41.2.0)
Collecting attrs>=17.4.0
Downloading attrs-19.3.0-py2.py3-none-any.whl (39 kB)
Collecting six>=1.11.0
Downloading six-1.14.0-py2.py3-none-any.whl (10 kB)
Collecting chardet<4,>=3.0.2
Downloading chardet-3.0.4-py2.py3-none-any.whl (133 kB)
Collecting idna<3,>=2.5
Downloading idna-2.9-py2.py3-none-any.whl (58 kB)
Collecting certifi>=2017.4.17
Downloading certifi-2020.4.5.1-py2.py3-none-any.whl (157 kB)
Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1
Downloading urllib3-1.25.9-py2.py3-none-any.whl (126 kB)
Collecting webencodings
Downloading webencodings-0.5.1-py2.py3-none-any.whl (11 kB)
Collecting pylev<2.0,>=1.3
Downloading pylev-1.3.0-py2.py3-none-any.whl (4.9 kB)
Collecting pastel<0.2.0,>=0.1.0
Downloading pastel-0.1.1-py2.py3-none-any.whl (5.8 kB)
Collecting msgpack>=0.5.2
Downloading msgpack-1.0.0-cp38-cp38-manylinux1_x86_64.whl (303 kB)
Collecting lockfile>=0.9; extra == "filecache"
Downloading lockfile-0.12.2-py2.py3-none-any.whl (13 kB)
Could not build wheels for pyrsistent, since package 'wheel' is not installed.
Could not build wheels for setuptools, since package 'wheel' is not installed.
Installing collected packages: shellingham, chardet, idna, certifi, urllib3, requests, requests-toolbelt, tomlkit, pkginfo, attrs, six, pyrsistent, jsonschema, pyparsing, cachy, webencodings, html5lib, pylev, pastel, cleo, msgpack, lockfile, cachecontrol, poetry
Running setup.py install for pyrsistent: started
Running setup.py install for pyrsistent: finished with status 'done'
Successfully installed attrs-19.3.0 cachecontrol-0.12.6 cachy-0.2.0 certifi-2020.4.5.1 chardet-3.0.4 cleo-0.6.8 html5lib-1.0.1 idna-2.9 jsonschema-3.2.0 lockfile-0.12.2 msgpack-1.0.0 pastel-0.1.1 pkginfo-1.5.0.1 poetry-0.12.17 pylev-1.3.0 pyparsing-2.4.7 pyrsistent-0.14.11 requests-2.23.0 requests-toolbelt-0.8.0 shellingham-1.3.2 six-1.14.0 tomlkit-0.5.11 urllib3-1.25.9 webencodings-0.5.1
[ModuleNotFoundError]
No module named '_bz2'
update [--no-dev] [--dry-run] [--lock] [--] [<packages>]...
There is an error with this repository's Renovate configuration that needs to be fixed. As a precaution, Renovate will stop PRs until it is resolved.
Location: renovate.json
Error type: The renovate configuration file contains some invalid settings
Message: Invalid configuration option: packageRules[5].additionalBranchPrefi
We should consider using AdoptOpenJDK for building all java version
api: https://api.adoptopenjdk.net/swagger-ui/
https://api.adoptopenjdk.net/v3/info/available_releases
{
"available_lts_releases": [
8,
11
],
"available_releases": [
8,
9,
10,
11,
12,
13,
14
],
"most_recent_feature_release": 14,
"most_recent_lts": 11
}
{
"adopt_build_number": 1,
"build": 9,
"major": 13,
"minor": 0,
"openjdk_version": "13.0.1+9",
"security": 1,
"semver": "13.0.1+9.1"
},
{
"adopt_build_number": 1,
"build": 33,
"major": 13,
"minor": 0,
"openjdk_version": "13+33",
"security": 0,
"semver": "13.0.0+33.1"
},
What would you like Renovate to be able to do?
Currently, Renovate can seemingly only operate on the version of Python that it has installed globally (currently 3.9 in the 1.2.0 image). While we like to keep Renovate up to date, we have certain use cases where we need to stick with a specific Python version for one reason or another.
Since pipenv
(a supported provider) can use pyenv
to get a Python version, I noticed this in the "artifact update" error:
File name: Pipfile.lock
Command failed: pipenv lock
Warning: Python 3.8 was not found on your system...
Neither 'pyenv' nor 'asdf' could be found to install Python.
You can specify specific versions of Python with:
$ pipenv --python path/to/python
Thus, I think that if pyenv were made available in the Docker image, it might be possible to have Renovate work for non-default Python versions. Happy to hear suggestions if you have an alternate way of solving this.
Did you already have any implementation ideas?
steps
bionic
and focal
bionic
to focal
bionic
Prepare helm v3 builder
Running a command like npm install --global yarn
fails in the current images. Using npx
may not always make sense, e.g. if we need to run npm x
twice and npm
currently takes up to 30 seconds to install.
We could make the install home part of /home/ubuntu
or we could mkdir -p /usr/lib/local/node_modules
and chown it to ubuntu
.
https://docs.npmjs.com/resolving-eacces-permissions-errors-when-installing-packages-globally
No longer used as base, see other open PR / issues
Updating Pipfile.lock fails with
Command failed: pipenv lock
/bin/sh: 1: pipenv: not found
This seems due to the fact that the pipenv is installed in
/root/.local/bin/pipenv
but renovate is run as the ubuntu user.
So I think pipenv need to be installed globally or as user ubuntu
It looks like #13 broke this image pretty badly. Prior to that being merged:
docker run --rm renovate/buildpack:1@sha256:e2e416bf17d2e58a8a6042a15dcb559048b11bc5e3143b7c7a33d38e01d8236a bash -c 'echo hello'
hello
And after that being merged:
docker run --rm renovate/buildpack:1@sha256:10e98460170e2ed4ea5150f5baefe0589c60bd8c63bf1f06fac5b8a61b61040d bash -c 'echo hello'
In the second example, bash runs but there is no output. All arguments are silently dropped. This breaks things like the renovate/go image.
When using binarySource=docker:
Command failed: docker run --rm --name=renovate_rust --label=renovate_child -v "/tmp/renovate/repos/github/renovate-tests/tikv":"/tmp/renovate/repos/github/renovate-tests/tikv" -v "/tmp/renovate/cache":"/tmp/renovate/cache" -w "/tmp/renovate/repos/github/renovate-tests/tikv" renovate/rust bash -l -c "cargo update --manifest-path Cargo.toml --package fail"
info: syncing channel updates for 'nightly-2020-04-23-x86_64-unknown-linux-gnu'
error: could not create temp file: /usr/local/rust/tmp/wq376y4cw_nr6daw_file
error: backtrace:
error: stack backtrace:
0: error_chain::backtrace::imp::InternalBacktrace::new
1: rustup::dist::download::DownloadCfg::download_and_check
2: rustup::dist::dist::dl_v2_manifest
3: rustup::dist::dist::try_update_from_dist_
4: rustup::toolchain::Toolchain::install
5: rustup::toolchain::Toolchain::install_from_dist
6: rustup::config::Cfg::find_override
7: rustup::config::Cfg::find_override_toolchain_or_default
8: rustup::config::Cfg::toolchain_for_dir
9: rustup_init::run_rustup_inner
10: rustup_init::main
11: std::rt::lang_start::{{closure}}
12: main
13: __libc_start_main
14: <unknown>
Seems it wants to write to /usr/local/rust
Maybe can be resolved with an env setting, but I'm still not sure why we can't grant ubuntu
user access to /usr/local/
?
❯ docker run --rm -it renovate/renovate:latest bash -c poetry
bash: /usr/local/poetry/1.1.4/bin/poetry: Permission denied
❯ docker run --rm -it renovate/renovate:latest which poetry
❯ docker run --rm -it renovate/renovate:latest ls -l /usr/local/poetry/1.1.4/bin/poetry
-rwxr--r-- 1 root root 456 Feb 10 08:04 /usr/local/poetry/1.1.4/bin/poetry
❯
vs
❯ docker run --rm -it renovate/renovate:24.38.6 bash -c poetry
Poetry version 1.1.4
... # snipped
❯ docker run --rm -it renovate/renovate:24.38.6 which poetry
/usr/local/python/3.9.1/bin/poetry
❯ docker run --rm -it renovate/renovate:24.38.6 ls -la /usr/local/python/3.9.1/bin/poetry
-rwxr-xr-x 1 root root 234 Feb 6 09:49 /usr/local/python/3.9.1/bin/poetry
❯
/usr/local/bin
(alternate is using the included install.sh
)git lfs install
as ubuntu
userblocks: renovatebot/renovate#6842
We should use other install method, as its faster (install) that the apt version. We should also use this because the available version on ubuntu is old.
Hi folks,
We are using renovate with some Bazel repositories and it would be really helpful to install https://github.com/bazelbuild/bazelisk onto this default docker image.
I want to gauge your thought before putting our a PR
The installation will looks roughly like this but im not quite sure how install-tool
works
RUN curl -fLo /usr/local/bin/bazel https://github.com/bazelbuild/bazelisk/releases/download/v1.6.1/bazelisk-linux-amd64 && \
chown root:root /usr/local/bin/bazel && \
chmod 0755 /usr/local/bin/bazel
What Renovate type are you using?
whitesource/renovate
Describe the bug
Renovate startes an gradle daemon for each repository with gradle inside (same gradle version is reused, different version stays running)
gradle can be started without daemon (the daemon will be ended after run)
https://docs.gradle.org/current/userguide/gradle_daemon.html
default@renovate-renovate-pro-7cbcbd6dd-zqgjq:/usr/src/app$ ps aux |grep gradle
default 228 0.5 1.1 3477980 790420 ? Ssl 06:56 1:53 /usr/lib/jvm/java-11-openjdk-amd64/bin/java --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.lang.invoke=ALL-UNNAMED --add-opens java.prefs/java.util.prefs=ALL-UNNAMED -XX:MaxMetaspaceSize=256m -XX:+HeapDumpOnOutOfMemoryError -Xmx512m -Dfile.encoding=UTF-8 -Duser.country -Duser.language=en -Duser.variant -cp /home/ubuntu/.gradle/wrapper/dists/gradle-5.2-bin/9yrnc2inuu4p0ylanusklg9pv/gradle-5.2/lib/gradle-launcher-5.2.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 5.2
default 14383 0.1 0.3 3292308 261596 ? Ssl 07:45 0:29 /usr/lib/jvm/java-11-openjdk-amd64/bin/java --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.lang.invoke=ALL-UNNAMED --add-opens java.prefs/java.util.prefs=ALL-UNNAMED -XX:MaxMetaspaceSize=256m -XX:+HeapDumpOnOutOfMemoryError -Xms256m -Xmx512m -Dfile.encoding=UTF-8 -Duser.country -Duser.language=en -Duser.variant -cp /home/ubuntu/.gradle/wrapper/dists/gradle-5.6.2-bin/8xw8k66iolvgcacwrbk2lxgq5/gradle-5.6.2/lib/gradle-launcher-5.6.2.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 5.6.2
default 15010 0.7 1.8 5046932 1216656 ? Ssl 07:48 2:18 /usr/lib/jvm/java-11-openjdk-amd64/bin/java --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.lang.invoke=ALL-UNNAMED --add-opens java.prefs/java.util.prefs=ALL-UNNAMED -XX:MaxMetaspaceSize=256m -XX:+HeapDumpOnOutOfMemoryError -Xms256m -Xmx512m -Dfile.encoding=UTF-8 -Duser.country -Duser.language=en -Duser.variant -cp /home/ubuntu/.gradle/wrapper/dists/gradle-5.6.4-bin/bxirm19lnfz6nurbatndyydux/gradle-5.6.4/lib/gradle-launcher-5.6.4.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 5.6.4
default 17292 0.4 1.1 8160660 787952 ? Ssl 08:01 1:15 /usr/lib/jvm/java-11-openjdk-amd64/bin/java --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.lang.invoke=ALL-UNNAMED --add-opens java.prefs/java.util.prefs=ALL-UNNAMED -Xmx4096M -Dfile.encoding=UTF-8 -Duser.country -Duser.language=en -Duser.variant -cp /home/ubuntu/.gradle/wrapper/dists/gradle-5.6.4-all/etoh6efx14qlyb91dp8z5hxth/gradle-5.6.4/lib/gradle-launcher-5.6.4.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 5.6.4
default 28538 0.1 0.7 4563332 477996 ? Ssl 08:48 0:26 /usr/lib/jvm/java-11-openjdk-amd64/bin/java --add-opens java.base/java.util=ALL-UNNAMED -XX:+HeapDumpOnOutOfMemoryError -Xmx1024m -Dfile.encoding=UTF-8 -Duser.country -Duser.language=en -Duser.variant -cp /home/ubuntu/.gradle/wrapper/dists/gradle-4.10-bin/egqyt8ra72bfhiofdk8k5qjqz/gradle-4.10/lib/gradle-launcher-4.10.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 4.10
default 31750 0.2 0.7 8121892 522984 ? Ssl 09:06 0:31 /usr/lib/jvm/java-11-openjdk-amd64/bin/java --add-opens java.base/java.util=ALL-UNNAMED --add-opens java.base/java.lang=ALL-UNNAMED --add-opens java.base/java.lang.invoke=ALL-UNNAMED --add-opens java.prefs/java.util.prefs=ALL-UNNAMED -Xmx4096m -Dfile.encoding=UTF-8 -Duser.country -Duser.language=en -Duser.variant -cp /home/ubuntu/.gradle/wrapper/dists/gradle-5.6.4-all/36ors43i4dolb4mr4ix6qbiqo/gradle-5.6.4/lib/gradle-launcher-5.6.4.jar org.gradle.launcher.daemon.bootstrap.GradleDaemon 5.6.4
From https://github.com/renovatebot/docker-go/runs/842721285
What to do
To have better compability with other workflows we should add global wrapper for installed tools.
Those wrapper should source BASH_ENV
and then call normal tool, so we have our prepared env loaded.
Sample Jenkins
docker run ... -d --rm renovatebot/node cat
/path/to/run.sh
shell script with commands at shared dir.docker exec <container-id> sh /path/to/run.sh
The exec is calling default sh
command in env, which will be linked to dash
on our ubuntu image.
Overwriting it to bash
won't help, as it won't source BASH_ENV
.
exec
is also not calling entrypoint, so again no BASH_ENV
sourcing.
Solution
creating shell wrapper like we did for node < 15
docker-buildpack/src/node/buildpack/tools/node.sh
Lines 39 to 42 in 7fe5151
docker-buildpack/src/base/buildpack/util.sh
Lines 56 to 76 in 97680db
needed for new scoop package manager
We should allow passing versions like v1.2.3
and automatically trim the v
. so we don't need extra renovate package rules to trim.
Can we perhaps have a lint test also to verify that all applicable build/* scripts appear in the docs?
We like to implement a new binarySource=install
where renovate install tools dynamically at runtime.
Use-case is to have smaller docker image used in kubernets, which can download missing tools on demand.
ref #96
Some users have custom SSL certificates, which means they need to manually apply changes to most/all tools which are installed. Something like the following would be great:
CUSTOM_SSL_CERTS
NODE_EXTRA_CA_CERTS
)Also including setting them up in OpenSSL itself, which hopefully many tools support.
Environment
SSL_CERT_FILE=/test/ca.pem
works for openssl based tools
NODE_EXTRA_CA_CERTS=/test/ca.pem
for nodejsSSL_CERT_FILE
)References
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.