heroku / base-images Goto Github PK

As found in the recent stack image updates, the base Ubuntu docker image can sometimes change in a way that causes breakage unrelated to commits in this repo (eg fewer installed-by-default packages). Whilst this repo now has a Travis run for validating the build and installed package list, it is only triggered by pushes to master, so the most recent job can be showing as green even if a rebuild now might not be.

I was thinking it might be good to enable the Travis cron feature to trigger extra builds on master once a {day, week, month} so problems can be identified prior to having to do an urgent release when new package CVEs are announced. This could be combined with the Travis notification system (email, IRC, slack, ...) to avoid having to remember to check the Travis status periodically.

For more details, see:
https://docs.travis-ci.com/user/cron-jobs/
https://docs.travis-ci.com/user/notifications/

I encountered a couple error messages when building, not sure if they are expected or not.

The output of sudo /vagrant/bin/capture-stack foo looked good, but then at the end:

-----> Cleaning up. Logs at /tmp/log/capture-stack.log
       umount: /tmp/cedar64-foo: not mounted

The output of sudo /vagrant/bin/install-stack /tmp/cedar64-foo.img.gz looked good, but then at the end:

-----> Mounting image /mnt/stacks/cedar64-foo
/mnt/stacks/cedar64-foo.img: No such file or directory

During the stack-images CI run, we check that the packages installed in the image matches a list saved in the repository. As well as running on commit, TravisCI has been configured to run regardless on a weekly basis.

The weekly started failing as of 13th Sept, and continues to do so even after retriggers:
https://travis-ci.org/heroku/stack-images/jobs/584488562

The failure is due to the lsb-release and distro-info-data packages no longer being found within the Heroku-16 image.

Checking the last released heroku/heroku:16 Docker image (using apt-cache rdepends --installed <package-name>), it can be seen that the distro-info-data package was previously installed due to being a dependency of lsb-release, and lsb-release was installed due to being a dependency of postgresql-client-common.

Looking into postgresql-client-common further (using apt-cache show postgresql-client-common), and comparing the last released Heroku-16 Docker image with a newly built one, the following changes can be seen:

• Version: 201.pgdg16.04+1 --> 207.pgdg16.04+1
• Recommends: lsb-release, libreadline6 --> libreadline6

Checking the changelog for recent releases, the removal of lsb-release from Recommends is mentioned here:
https://www.postgresql.org/message-id/[email protected]

The reason this didn't affect Cedar-14 is that there was no new release for Trusty (presumably since it's EOL):
https://apt.postgresql.org/pub/repos/apt/dists/

...and Heroku-18 was unaffected since it uses --no-install-recommends, so already had to specify the lsb-release dependency explicitly.

To fix this issue, we should add lsb-release to the Heroku-16 apt-get install invocation explicitly.

It's also great to see that (a) our packages CI check has helped prevent yet another regression, (b) switching to using --no-install-recommends looks to have been a good call (to reduce number of implicit but replied upon dependencies).

The Heroku changelog says that OpenJDK is now included in cedar-14, but I don't see a corresponding commit in this repo.

Ruby 1.9.3 reached End of Life in 2015, consider updating.

Where is it documented how to get the latest heroku:20-build docker image to that happy place for local Dockerized Rails development where it has the default/recommended ruby, bundler versions (at a minimum, and probably also nodejs, npm, and yarn versions)?

I think I understand that "buildpacks" are the magic sauce that lets Heroku support such a wide range of frameworks.

And I think I understand that these stack images are kind of a foundation that those buildpacks rest upon.

Yet, the Heroku online service does have a 'default' ruby version (currently 2.7.3?). And when we do a git push, Heroku only uses its 'curated' Bundler version (2.2.16?). And I expect (especially for Rails 6 apps) there are particular recommended versions of nodejs, npm, and Yarn, too.

After a couple of years of "trial and error" on Heroku-18, trying to iteratively zero in on reasonable dev/prod parity in our local Docker Rails development setup, I figured I'd just ask... **how/where do we 'get' the current-default ruby version out of the correct ruby buildpack? **

I assume that in a Dockerfile, right below FROM heroku/heroku:20-build developers need add a series of apt-get X Y Z commands, and probably purging and reinstalling the default version of Ruby (from where?) and probably manual installation of the "current correct" Bundler version (trivially easy except - which version?)

Is that "default ruby version + correct bundler version" Dockerfile documented anywhere? (This git repo seems the logical place.)

If not, oughtn't it be clearly documented given Heroku (correctly) stresses the importance of dev/prod parity?

A few notes:

Running your latest heroku/heroku:20-build via docker run --rm -v ${PWD}:/usr/src -w /usr/src -ti heroku/heroku:20-build /bin/bash I note that:

• "ruby -v" shows ruby 2.7.0p0 (but I think the the current default is 2.7.3)

This is how we (successfully) could update the obsolete stack image Ruby version back on heroku-18:

When attempting to update Ruby from a heroku buildpack, what USED to work (in heroku:18-build) is:

# Dockerfile   (just seeing if ruby can be updated to a later Heroku-supplied version)
RUN apt-get update -qq && apt-get install -y nodejs

RUN apt remove -y --purge ruby && curl -s --retry 3 -L https://heroku-buildpack-ruby.s3.amazonaws.com/heroku-18/ruby-2.5.8.tgz | tar -xz

RUN ruby -v
RUN gem list

That initial snippet successfully replaced the default image Ruby version with a later version, obtained right from one of your buildpacks. The output of running docker build --progress=plain --no-cache -t hiapptmp . is:

#2 [internal] load build definition from Dockerfile
#2 transferring dockerfile: 1.40kB done
#2 DONE 0.0s

#1 [internal] load .dockerignore
#1 transferring context: 2B done
#1 DONE 0.0s

#3 [internal] load metadata for docker.io/heroku/heroku:18-build
#3 DONE 0.0s

#4 [1/5] FROM docker.io/heroku/heroku:18-build
#4 resolve docker.io/heroku/heroku:18-build done
#4 DONE 0.0s

#5 [2/5] RUN apt-get update -qq && apt-get install -y nodejs
#5 7.257 Reading package lists...
.... bunch of messages
#5 DONE 13.1s

#6 [3/5] RUN apt remove -y --purge ruby && curl -s --retry 3 -L https://her...
#6 0.210 
#6 0.210 WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
#6 0.210 
#6 0.233 Reading package lists...
#6 1.394 Building dependency tree...
#6 1.586 Reading state information...
#6 1.735 The following packages were automatically installed and are no longer required:
#6 1.735   ruby-did-you-mean ruby-minitest ruby-net-telnet ruby-power-assert
#6 1.735   rubygems-integration
#6 1.736 Use 'apt autoremove' to remove them.
#6 1.792 The following packages will be REMOVED:
#6 1.793   libruby2.5* rake* ruby* ruby-dev* ruby-test-unit* ruby2.5* ruby2.5-dev*
#6 2.039 0 upgraded, 0 newly installed, 7 to remove and 6 not upgraded.
#6 2.039 After this operation, 15.2 MB disk space will be freed.
(Reading database ... 36188 files and directories currently installed.)
#6 2.085 Removing ruby-dev:amd64 (1:2.5.1) ...
#6 2.104 Removing ruby2.5-dev:amd64 (2.5.1-1ubuntu1.9) ...
#6 2.143 Removing ruby (1:2.5.1) ...
#6 2.161 Removing ruby2.5 (2.5.1-1ubuntu1.9) ...
#6 2.182 Removing libruby2.5:amd64 (2.5.1-1ubuntu1.9) ...
#6 2.270 Removing ruby-test-unit (3.2.5-1) ...
#6 2.293 Removing rake (12.3.1-1ubuntu0.1) ...
#6 2.335 Processing triggers for libc-bin (2.27-3ubuntu1.4) ...
#6 DONE 5.5s

#7 [4/5] RUN ruby -v
#7 0.211 ruby 2.5.8p224 (2020-03-31 revision 67882) [x86_64-linux]
#7 DONE 0.2s

#8 [5/5] RUN gem list
#8 0.438 bigdecimal (default: 1.3.4)
#8 0.438 cmath (default: 1.0.0)
#8 0.438 csv (default: 1.0.0)
#8 0.438 date (default: 1.0.0)
#8 0.438 dbm (default: 1.0.0)
#8 0.438 did_you_mean (1.2.0)
#8 0.438 etc (default: 1.0.0)
#8 0.438 fcntl (default: 1.0.0)
#8 0.438 fiddle (default: 1.0.0)
#8 0.438 fileutils (default: 1.0.2)
#8 0.438 gdbm (default: 2.0.0)
#8 0.438 io-console (default: 0.4.6)
#8 0.438 ipaddr (default: 1.2.0)
#8 0.438 json (default: 2.1.0)
#8 0.438 minitest (5.10.3)
#8 0.438 net-telnet (0.1.1)
#8 0.438 openssl (default: 2.1.2)
#8 0.438 power_assert (1.1.1)
#8 0.438 psych (default: 3.0.2)
#8 0.438 rake (12.3.3)
#8 0.438 rdoc (default: 6.0.1.1)
#8 0.438 scanf (default: 1.0.0)
#8 0.438 sdbm (default: 1.0.0)
#8 0.438 stringio (default: 0.0.1)
#8 0.438 strscan (default: 1.0.0)
#8 0.438 test-unit (3.2.7)
#8 0.438 webrick (default: 1.4.2)
#8 0.438 xmlrpc (0.3.0)
#8 0.438 zlib (default: 1.0.0)
#8 DONE 0.5s

#9 exporting to image
#9 exporting layers
#9 exporting layers 0.4s done
#9 writing image sha256:734b82c8ef7738c933b92115133747f57183279fd4c19e8435322266b3b8888e done
#9 naming to docker.io/library/hiapptmp done
#9 DONE 0.4s

That method does not work on 20-build

Trying the same thing, using 20-build, and a ruby 2.7.3 does install the new ruby, but the installation is broken:

# Dockerfile     (just seeing if ruby can be updated to a later Heroku-supplied version)
FROM heroku/heroku:20-build

RUN apt-get update -qq && apt-get install -y nodejs

RUN apt remove -y --purge ruby && curl -s --retry 3 -L https://heroku-buildpack-ruby.s3.amazonaws.com/heroku-20/ruby-2.7.3.tgz | tar -xz

RUN ruby -v
RUN gem list

$ docker build --progress=plain --no-cache -t hiapptmp .
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 1.66kB done
#1 DONE 0.0s

#2 [internal] load .dockerignore
#2 transferring context: 2B done
#2 DONE 0.0s

#3 [internal] load metadata for docker.io/heroku/heroku:20-build
#3 DONE 0.0s

#4 [1/5] FROM docker.io/heroku/heroku:20-build
#4 CACHED

#5 [2/5] RUN apt-get update -qq && apt-get install -y nodejs
#5 6.419 Reading package lists...
... bunch of messages
#5 DONE 12.1s

#6 [3/5] RUN apt remove -y --purge ruby && curl -s --retry 3 -L https://her...
#6 0.300 
#6 0.300 WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
#6 0.300 
#6 0.322 Reading package lists...
#6 1.144 Building dependency tree...
#6 1.309 Reading state information...
#6 1.453 The following packages were automatically installed and are no longer required:
#6 1.453   ruby-minitest ruby-net-telnet ruby-power-assert ruby-test-unit ruby-xmlrpc
#6 1.454   rubygems-integration
#6 1.454 Use 'apt autoremove' to remove them.
#6 1.501 The following packages will be REMOVED:
#6 1.502   libruby2.7* rake* ruby* ruby-dev* ruby2.7* ruby2.7-dev*
#6 1.750 0 upgraded, 0 newly installed, 6 to remove and 6 not upgraded.
#6 1.750 After this operation, 19.2 MB disk space will be freed.
(Reading database ... 38839 files and directories currently installed.)
#6 1.800 Removing ruby-dev:amd64 (1:2.7+1) ...
#6 1.824 Removing ruby2.7-dev:amd64 (2.7.0-5ubuntu1.4) ...
#6 1.868 Removing ruby2.7 (2.7.0-5ubuntu1.4) ...
#6 1.892 Removing libruby2.7:amd64 (2.7.0-5ubuntu1.4) ...
#6 2.033 Removing rake (13.0.1-4) ...
#6 2.061 Removing ruby (1:2.7+1) ...
#6 2.102 Processing triggers for libc-bin (2.31-0ubuntu9.2) ...
#6 DONE 12.5s

#7 [4/5] RUN ruby -v
#7 0.253 container_linux.go:349: starting container process caused "exec: \"/bin/sh\": stat /bin/sh: no such file or directory"
#7 ERROR: executor failed running [/bin/sh -c ruby -v]: runc did not terminate sucessfully
------
 > [4/5] RUN ruby -v:
------
failed to solve with frontend dockerfile.v0: failed to build LLB: executor failed running [/bin/sh -c ruby -v]: runc did not terminate sucessfully

setting WORKDIR to /usr gets "closer" (ruby -v works) but Gem systems gets bolluxed

# Dockerfile
FROM heroku/heroku:20-build

WORKDIR /usr
RUN apt-get update -qq && apt-get install -y nodejs

RUN apt remove -y --purge ruby && curl -s --retry 3 -L https://heroku-buildpack-ruby.s3.amazonaws.com/heroku-20/ruby-2.7.3.tgz | tar -xz

RUN ruby -v
RUN gem list

docker build --progress=plain --no-cache -t hiapptmp .

... first part is same
.... but now the ruby -v works:

#8 [5/6] RUN ruby -v
#8 0.276 ruby 2.7.3p183 (2021-04-05 revision 6847ee089d) [x86_64-linux]
#8 DONE 0.3s

... however Gem system broken....

#9 [6/6] RUN gem list
#9 0.228 /usr/lib/ruby/vendor_ruby/rubygems/defaults/operating_system.rb:50:in `<class:Specification>': undefined method `rubyforge_project=' for class `Gem::Specification' (NameError)
#9 0.229 	from /usr/lib/ruby/vendor_ruby/rubygems/defaults/operating_system.rb:49:in `<top (required)>'
#9 0.229 	from /usr/lib/ruby/2.7.0/rubygems.rb:1427:in `require'
#9 0.229 	from /usr/lib/ruby/2.7.0/rubygems.rb:1427:in `<top (required)>'
#9 0.229 	from <internal:gem_prelude>:1:in `require'
#9 0.229 	from <internal:gem_prelude>:1:in `<internal:gem_prelude>'
#9 ERROR: executor failed running [/bin/sh -c gem list]: runc did not terminate sucessfully
------
 > [6/6] RUN gem list:
------
failed to solve with frontend dockerfile.v0: failed to build LLB: executor failed running [/bin/sh -c gem list]: runc did not terminate sucessfully

another possible issue for build-20 related to Rubygems?

Also disconcerting (with the -20:build) is some possible issue related to rubygems out of the box... I assume a simple gem update --system ought to work out-of-the-box on the stack image?

docker run --rm -v ${PWD}:/usr/src -w /usr/src -ti heroku/heroku:20-build /bin/bash

# gem update --system --no-document
Updating rubygems-update
Fetching rubygems-update-3.2.17.gem
Successfully installed rubygems-update-3.2.17
Installing RubyGems 3.2.17
Traceback (most recent call last):
	5: from setup.rb:23:in `<main>'
	4: from setup.rb:23:in `require'
	3: from /var/lib/gems/2.7.0/gems/rubygems-update-3.2.17/lib/rubygems.rb:1348:in `<top (required)>'
	2: from /var/lib/gems/2.7.0/gems/rubygems-update-3.2.17/lib/rubygems.rb:1348:in `require'
	1: from /usr/lib/ruby/vendor_ruby/rubygems/defaults/operating_system.rb:49:in `<top (required)>'
/usr/lib/ruby/vendor_ruby/rubygems/defaults/operating_system.rb:50:in `<class:Specification>': undefined method `rubyforge_project=' for class `Gem::Specification' (NameError)

(although updating to a specific version less than 3.1.4 does seem to work)

Hey there!
Thanks a lot for your work on all of these images!
I wanted to report in case this flew under your radar. I recently upgraded my Heroku stack to 22, but unfortunately needed to revert to 20, as it broke my azure integration.

Version Info:

• Heroku stack 22
• Python 3.10.6
• azure_cognitiveservices_speech==1.23.0 (pip requirement)

upon running:

self.synthesizer = speechsdk.SpeechSynthesizer(speech_config=self.speech_config, audio_config=None)

I received an error:

RuntimeError:
Runtime error: Failed to initialize platform (azure-c-shared). Error: 2153
[CALL STACK BEGIN]

/app/.heroku/python/lib/python3.10/site-packages/azure/cognitiveservices/speech/libMicrosoft.CognitiveServices.Speech.core.so(+0x1b75e1) [0x7f33a29b75e1]
/app/.heroku/python/lib/python3.10/site-packages/azure/cognitiveservices/speech/libMicrosoft.CognitiveServices.Speech.core.so(+0x1dbcfb) [0x7f33a29dbcfb]
/lib/x86_64-linux-gnu/libc.so.6(+0x99f68) [0x7f33c7a6df68]
/app/.heroku/python/lib/python3.10/site-packages/azure/cognitiveservices/speech/libMicrosoft.CognitiveServices.Speech.core.so(+0x1dceb2) [0x7f33a29dceb2]
/app/.heroku/python/lib/python3.10/site-packages/azure/cognitiveservices/speech/libMicrosoft.CognitiveServices.Speech.core.so(+0x1a0a49) [0x7f33a29a0a49]
/app/.heroku/python/lib/python3.10/site-packages/azure/cognitiveservices/speech/libMicrosoft.CognitiveServices.Speech.core.so(+0x1a3197) [0x7f33a29a3197]
/app/.heroku/python/lib/python3.10/site-packages/azure/cognitiveservices/speech/libMicrosoft.CognitiveServices.Speech.core.so(+0x1bedaa) [0x7f33a29bedaa]
/app/.heroku/python/lib/python3.10/site-packages/azure/cognitiveservices/speech/libMicrosoft.CognitiveServices.Speech.core.so(+0x1d667c) [0x7f33a29d667c]
/app/.heroku/python/lib/python3.10/site-packages/azure/cognitiveservices/speech/libMicrosoft.CognitiveServices.Speech.core.so(+0x189f82) [0x7f33a2989f82]
/app/.heroku/python/lib/python3.10/site-packages/azure/cognitiveservices/speech/libMicrosoft.CognitiveServices.Speech.core.so(+0xdf613) [0x7f33a28df613]
/app/.heroku/python/lib/python3.10/site-packages/azure/cognitiveservices/speech/libMicrosoft.CognitiveServices.Speech.core.so(+0x1bedaa) [0x7f33a29bedaa]
/app/.heroku/python/lib/python3.10/site-packages/azure/cognitiveservices/speech/libMicrosoft.CognitiveServices.Speech.core.so(+0x181eb1) [0x7f33a2981eb1]
/app/.heroku/python/lib/python3.10/site-packages/azure/cognitiveservices/speech/libMicrosoft.CognitiveServices.Speech.core.so(+0x1e9d31) [0x7f33a29e9d31]
/app/.heroku/python/lib/python3.10/site-packages/azure/cognitiveservices/speech/libMicrosoft.CognitiveServices.Speech.core.so(synthesizer_create_speech_synthesizer_from_config+0xf3) [0x7f33a28c4be0]
/app/.heroku/python/lib/python3.10/site-packages/azure/cognitiveservices/speech/_speech_py_impl.so(+0xd38cc) [0x7f33a30d38cc]
/app/.heroku/python/lib/python3.10/site-packages/azure/cognitiveservices/speech/_speech_py_impl.so(+0x6bdb3) [0x7f33a306bdb3]
/app/.heroku/python/lib/libpython3.10.so.1.0(+0x156238) [0x7f33c7d57238]
[CALL STACK END]

Not sure if this is your problem to fix or Azure's but wanted to make sure you guys are aware of this.

Hey there!

Heroku's cedar stack images are vulnerable to USN-3438-1. https://devcenter.heroku.com/articles/stack-packages notes that dpkg --list git shows 1:1.9.1-1ubuntu0.6. 1:1.9.1-1ubuntu0.7 contains the fix for USN-3438-1. This affects users of the heroku/cedar:14 image.

https://github.com/search?q=FROM+heroku%2Fcedar%3A14&type=Code&utf8=%E2%9C%93

Hi Heroku!

We've been seeing the occasional width or height exceeds limit exceptions in our Rails application coming from the convert ImageMagick utility (some PDFs with long pages being converted to PNGs), and while investigating, I came across Heroku's default resource limits:

https://github.com/heroku/stack-images/blob/64af8ea068b2a4e60daf7cedf9e422b4e650771b/heroku-18/setup.sh#L204-L218

The resource limits (width/height in particular) surprised me, and I'm curious if there's any specific reasoning for them. The original PR that introduced this default policy.xml was #119, and unfortunately there isn't a ton of context in there.

Any insight would be appreciated! I didn't want to dramatically increase these without understanding some of the reasoning behind these Heroku/system defaults.

As part of ensuring inclusive language is used throughout all Heroku repositories, we should switch from the current default branch of master to main.

This will involve:

• Creating a new branch main, based off current master
• Changing the default branch in GitHub to main
• Updating the branch protection settings
• Updating the target branch of any open PRs
• Updating any in-repo code/docs references (such as the Travis badge) -> #172
• Checking that there are no branch-specific Travis environment variables set
• Updating the weekly Travis cron to build from the new branch
• Checking that Travis still works -> https://travis-ci.com/github/heroku/stack-images/builds/178267673
• Checking for any other deep links to this repository in other repos that include the branch name
• Deleting the master branch

@W-7837260@

Sorry in advance, as I suspect that this is not the best place for this question (but that I also wouldn't find the answer on the Heroku StackExchange and don't have access to Heroku support).

I see that there's an updated heroku-20 image available but that it's not yet the default image for new apps, per https://devcenter.heroku.com/articles/stack. Do you know when it will become the default, or whether it's in a production-ready state? (The article mentions April of each year for EOLs, but I don't know when these images are typically "released".)

We're in the process of preparing our application for Q4 / Holiday scale and have debated upgrading to the new stack before load-testing. If it's safe (enough) now or will be released generally soon, then we'll start down this path, but otherwise we'll stick with Heroku-18.

Thanks in advance!

Now that Postgres 13 is generally available on Heroku, do the postgres client binaries need to be updated?

We are having an issue with Heroku CI builds failing due to a version mismatch when restoring from a Rails structure.sql file:

pg_dump: error: server version: 13.1 (Ubuntu 13.1-1.pgdg20.04+1); pg_dump version: 12.5 (Ubuntu 12.5-1.pgdg20.04+1)
       pg_dump: error: aborting because of server version mismatch
       rake aborted!
       failed to execute:
       pg_dump --schema-only --no-privileges --no-owner --file /app/db/structure.sql

This pull request updated the client binaries to Postgres 12: #151

Is something similar needed to support Postgres 13?

Currently the Heroku-16 images are using version 1:2.7.4-0ubuntu1 of the Ubuntu git package:

$ heroku run -- dpkg-query --show --showformat=\'\${Version}\' git
Running dpkg-query --show --showformat='${Version}' git on heroku-16-test... up, run.8077 (Free)
1:2.7.4-0ubuntu1

This is missing:

git (1:2.7.4-0ubuntu1.2) xenial-security; urgency=medium

  * SECURITY UPDATE: Arbitrary code execution on clients through
    malicious ssh URLs.
    - debian/patches/CVE-2017-1000117.patch: filter out hostnames that
      would interpreted as cli arguments to ssh
    - debian/diff/0002-transport-expose-git_tcp_connect-and-friends-in-new-t.diff:
      update to adjust for changes from CVE-2017-1000117.patch.
    - CVE-2017-1000117

 -- Steve Beattie <SNIP>  Thu, 10 Aug 2017 14:15:28 -0700

git (1:2.7.4-0ubuntu1.1) xenial-security; urgency=medium

  * SECURITY UPDATE: git shell restriction bypass
    - debian/patches/CVE-2017-8386.patch: disallow repo names beginning
      with dash in shell.c.
    - CVE-2017-8386

 -- Marc Deslauriers <SNIP>  Fri, 12 May 2017 09:29:55 -0400

(From http://changelogs.ubuntu.com/changelogs/pool/main/g/git/git_2.7.4-0ubuntu1.2/changelog)

With the release of bundler version 2, it started to fail.

% docker run -it heroku/heroku:18
root@xxxxxxxxxxxx:/# gem install bundler
Fetching: bundler-2.0.0.gem (100%)
ERROR:  Error installing bundler:
        The last version of bundler (>= 0) to support your Ruby & RubyGems was 1.17.3. Try installing it with `gem install bundler -v 1.17.3`
        bundler requires RubyGems version >= 3.0.0. The current RubyGems version is 2.7.6. Try 'gem update --system' to update RubyGems itself.

I can just do as the error message says, but just let you folks know.

bin/capture-stack should only package things, but it also runs some provisioning steps.

If those steps are moved to the provisioning scripts (eg.: bin/cedar.sh), heroku compatible images could be easily built by just running a provisioning script, eg.: inside vagrant VMs to create vagrant boxes or docker containers to create docker images.

Right now, creating vagrant boxes and/or docker images requires additional steps other than just running bin/<stack-name>.sh inside the image being built.

Using Kitematic, the repo must include a "latest" tag for it to be successfully downloaded. Are you able to add?

Hi there!

We have been trying to test the new 22 stack ir our Dockerfiles. However, whenever we try to build using the new stack we are faced with the following:

Step 12/13 : RUN /tmp/buildpack/php/bin/compile /app /tmp/build_cache /tmp/env
 ---> Running in 638f1811c565

 !     ERROR: Failed to download standard library for bootstrapping!
 !     
 !     This is most likely a temporary internal error. If the problem
 !     persists, make sure that you are not running a custom or forked
 !     version of the Heroku PHP buildpack which may need updating.

For reference our Dockerfile is in these lines:

FROM heroku/heroku:22 as base

ENV STACK heroku-22
ENV PHP_BUILDPACK_FILE php.tgz
ENV APP /app
ENV HOME $APP

ADD . $APP
WORKDIR $APP

RUN mkdir -p /tmp/buildpack/php /tmp/build_cache /tmp/env
ADD https://codon-buildpacks.s3.amazonaws.com/buildpacks/heroku/$PHP_BUILDPACK_FILE ./
RUN tar -xzvf $PHP_BUILDPACK_FILE -C /tmp/buildpack/php --strip-components 1 && rm $PHP_BUILDPACK_FILE

# Bootstrap the buildpack!
RUN /tmp/buildpack/php/bin/compile /app /tmp/build_cache /tmp/env

The same Dockerfile works for 18 and 20 (just replacing their versions in the ENV and the image name).

Did anything change in the process for 22 that we need to deal with?

The stack images currently install the PostrgreSQL client, library and headers from the upstream PostgreSQL APT repository (apt.postgresql.org) rather than the less frequently updated packages in Ubuntu's APT repository.

This means that as new major Postgres versions are released, the next stack image release will include updated major versions of libpq5 and libpq-dev (the clients themselves use versioned package names such as postgresql-client-11 so have to be explicitly updated).

New major versions of libpq are generally backwards compatible, however libpq v12 includes the following breaking change that makes the validation of connection parameters more strict:
postgres/postgres@e7a2217

Unfortunately this breaking change wasn't listed in the PostgreSQL 12 release notes, so we weren't able to factor it into the stack release review process.

This meant the latest stack image update caused database connection errors at runtime for a couple of customers, since their Rails apps were using invalid values for connection parameters (such as connect_timeout having trailing non-numeric characters). (Previously with libpq v11 these invalid values were being silently ignored.)

As such we:

• rolled back that stack image release
• reported the missing release notes breaking change entry to the Postgres project
• filed ged/ruby-pg/issues/302 to have the pg gem report the more helpful validation related error message instead of PG::UnableToSend: no connection to the server

Determining how to best roll out the PG 12 client libraries will be more involved, so in the short term we should pin to v11 so that stack image releases are unblocked. After that someone will need to investigate further as to how widespread invalid app configurations are and work on a rollout plan.

CC @heroku/dod-infra since they will need to be aware of this if wanting to upgrade to PG12 in the stack image in the future. For more info see the investigation notes here.

Hi, I would like to request the addition of python3-dev and libperl-dev to the build images. The python3 and perl packages are already in the runtime images so this would only affect the build images. Thanks

Happy to make a pull request if this makes it easier for you

We are using Heroku-20-build Docker image to simulate the Heroku platform locally in our development environments. We are using the build image as some gems are C extensions and require building. However, the Heroku-20-build has Ruby 2.7.0 and when we do a git push to deploy to the Heroku platform, the platform is expecting Ruby 2.7.2 and rejects the push.

I would have expected the Heroku-20-build Docker image to match the Heroku-20 platform.

Hello!

cedar-14 & heroku-16 images are affected by OpenSSL vulnerability too.

Here are fixed versions of libssl1.0.0 package:

Ubuntu 16.04 LTS: libssl1.0.0 1.0.2g-1ubuntu4.9
Ubuntu 14.04 LTS: libssl1.0.0 1.0.1f-1ubuntu2.23

Thank you!

Ubuntu apparently doesn't backport updates to the tzdata package to LTS releases:

http://changelogs.ubuntu.com/changelogs/pool/main/t/tzdata/tzdata_2010i-1/changelog

We will have to pull in an updated version and pin it.

Are there any plans on using a standard ubuntu image instead of the deprecated ubuntu-debootstrap ?

This image is officially deprecated in favor of the standard ubuntu image, and will receive no further updates.

The stack image's libruby isn't used much, but apps without a Ruby buildpack could still be affected.

See: heroku/prodsec#205

Having no license equals to copyrighted.

Looking at the Docker hub build details page, the heroku/heroku:16 image was built 11 days ago, however no corresponding heroku/heroku:16-build run was triggered.

I believe this is because Docker Hub's automated builds try and be helpful, and only watch for changes to the specific directory that the automated build points at (and in this case, only the heroku-16/ directory changed).

To fix this, the Docker hub repository links feature can be used, such that any changes in the heroku/heroku:16 will trigger a heroku/heroku:16-build build too.

If it's not already set up, it's also worth similarly linking ubuntu:16.04 to heroku/heroku:16, so that any upstream Ubuntu image changes (eg for security fixes) get automatically picked up in the Heroku-16 Docker images. (And same for ubuntu-debootstrap:14.04 -> cedar-14.)

@tt, I don't suppose you could take a look? :-)

After upgrading stack to heroku-22 (from heroku-20) many people experience random problems with SSL.
Please see the original thread where this is discussed sidekiq/sidekiq#5402 and the conclusion that problems comes from the stack upgrade sidekiq/sidekiq#5402 (comment)

We would like to have a publicly visible bug report regarding this issue, so everyone can share their findings and have an easy to follow updates on it.

If this is not the right place, then can you please point me where it would be best to open it?

it is currently defined in bytes
https://github.com/heroku/stack-images/blob/e65a994d5540840ebf173528875db4bad5b29467/heroku-18/setup.sh#L213

the documentation is not really clear https://imagemagick.org/script/resources.php

but it looks like that memory/disk resources expect "bytes" and height/width/area expect "pixels"

Define arguments for the MAGICK_MEMORY_LIMIT, MAGICK_DISK_LIMIT, and MAGICK_MEMORY_LIMIT environment variables with SI prefixes (.e.g 100MB). MAGICK_WIDTH_LIMIT, MAGICK_HEIGHT_LIMIT and MAGICK_AREA_LIMIT accepts pixel suffixes such as MP for mega-pixels (e.g. 100MP)

which would make more sense

Hi!

Both Cedar nad Heroku stack images are vurnable to usn-3464-1. wget package should be upgraded to:

Ubuntu 16.04 LTS - heroku/heroku:16
  wget 1.17.1-1ubuntu1.3
Ubuntu 14.04 LTS - heroku/cedar:14
  wget 1.15-1ubuntu1.14.04.3

Is there any way to use this repository to build a local dev setup for Heroku apps? I can see there's a Vagrantfile and some scripts which seem to bootstrap the VM. However, this repo seems to be geared towards another purpose. How can I just end up with a local VM that's running my project?

Hello folks,

Thank you for making the images available to everybody!

I noticed that recent merges and changes to the stack configuration are failing:

https://hub.docker.com/r/heroku/heroku/builds/

Building in Docker Cloud's infrastructure...
Cloning into '.'...

warning: Could not find remote branch heroku-16 to clone.

fatal: Remote branch heroku-16 not found in upstream origin

Unexpected end of command stream

please ensure the remote branch exists (128)

I'm not sure if that branch ever existed, but will be great if the automated builds can be updated to pick up the changes from master instead, now that it contains both 16 and 16-build Dockerfiles.

Once again, thank you for your work! ❤️ ❤️ ❤️

The last set of builds on https://hub.docker.com/r/heroku/heroku/builds/ failed, with error:

fatal: Remote branch heroku-16 not found in upstream origin

The "Branch" and "Dockerfilefile Location" settings on the Docker Hub Build Settings page need updating (docs) after #53.

Whilst on Docker Hub, I don't suppose the Cedar-14 image (here) could be fixed to point to this repo rather than the tt/stack-images fork, so it picks up updates too?

Many thanks :-)

Following the cedar-14 steps in INSTALL.md on master, I get:

...
$ sudo /vagrant/bin/install-stack /tmp/cedar64-14.4.0.img.gz

-----> Starting install at Mon Jul 3 13:55:15 UTC 2017

-----> Gunzipping image /tmp/cedar64-14.4.0.img.gz
/vagrant/bin/install-stack: line 16: /mnt/stacks/cedar64-14.4.0.img: No such file or directory

This is because /mnt/stacks doesn't exist in a clean Vagrant instance.

I guess install-stack should just do a mkdir -p /mnt/stacks before the gzip, just in case it's being run from somewhere that doesn't already have existing images installed.

Hi,

This looks really useful, but I'm not 100% sure what it is. The heroku stacks have hundreds of packages of specific versions, but in these recipes there are only a few dozen with no version listed. Also, the heroku stacks provide very up-to-date versions of things like ruby, as they are released, and that is not reflected here.

What exactly does this provide? Am I thinking about it wrong and it is in fact the buildpacks which provide the more specific and extended packages?

I'll put in a PR to enhance the readme a bit with whatever knowledge comes out of this thread. Thanks!

John

Hello folks,

Thanks for making the stack images build process publicly available. Currently using this in combination with Buildpacks (pack).

When looking at heroku-18 stack and the list of included packages, noticed that libsqlite3-0 (the shared library) is included.

However, libsqlite3-dev package, necessary to compile or link against the shared object is missing.

While there is an article covering why is a bad idea use SQLite3 in Heroku, the shipping of the shared library makes me wonder about the missing dev counterpart.

Thank you in advance for your response!

Cheers.
❤️ ❤️ ❤️

Hello I'm trying to use this image on an M1 mac but I'm facing the same issue reported here. The solution for this issue, which worked for me with other images is to use arm64 compatible images.

After seeing the supported arch/os for the images I noticed only amd64 is supported. Would it be possible to add support for arm64?

Thanks

Are you planning to create Stack images on top of Alpine linux sometime in the future?
I think it would radically reduce the size of the Stack images.

Last lines of output:

+ cd /
+ rm -rf '/var/cache/apt/archives/*.deb'
+ rm -rf '/root/*'
+ rm -rf /tmp/cedar-14.sh /tmp/hsperfdata_root
+ xargs -r chmod u-s
+ pruned_find -perm /u+s
+ find / -type d '(' -name dev -o -name proc ')' -prune -o -perm /u+s -print
+ xargs -r chmod g-s
+ pruned_find -perm /g+s
+ find / -type d '(' -name dev -o -name proc ')' -prune -o -perm /g+s -print
chmod: changing permissions of '/usr/bin/mail-lock': Input/output error
chmod: changing permissions of '/usr/bin/mail-touchlock': Input/output error
The command '/bin/sh -c /bin/bash /tmp/cedar-14.sh' returned a non-zero code: 123
make: *** [cedar-14-stack] Error 1

While using heroku-16 I attempted to install a rails 4.0 project, that project required bcrypt and upon getting to that gem it errored out, complaining that I didn't have the native extensions.

Hello! I was looking into the OpenSSL version on Heroku 16 and was surprised by the version number. It looked to me like the version was 1.0.2g and hadn't been updated in two years, which would be surprising. So I started looking at the OpenSSL changelog and saw that this was lagging the latest branch (1.1.0h), but even within 1.0.2 it looked like it was over two years behind on patches (latest is 1.0.2o). This is what I see with the Ruby buildpack set to 2.5.1, run on a heroku-16 dyno this morning:

$ openssl version -a
OpenSSL 1.0.2g  1 Mar 2016
built on: reproducible build, date unspecified
platform: debian-amd64

I can see in the Heroku docs on stacks that heroku-16 includes openssl 1.0.2g-1​ubuntu4.12 (and that heroku-18 has 1.1.0g-2​ubuntu4). When I asked about this over in the Ruby buildpack repo, @schneems helpfully pointed me over here and suggested that this isn't actually 1.0.2g but contains additional security patches. And then I found the Heroku changelog that there have been several security patch notices for Heroku 16 (eg this recent one), although they don't reveal what they are for. While I suspect you all are on top of this, it'd be helpful to learn more about this aspect of security for Heroku deployments.

Can you help me understand more about how this works, and where I can see what code exactly is running for OpenSSL? This might be as simple as helping understand what 1.0.2g-1​ubuntu4.12 points to, or how heroku-16.sh installs the package (perhaps it's pulled in from libgnutls-openssl27?)
Thanks in advance for the help in learning more about the process you all use for updating security-related packages. Past explanations like #69 (comment) from @tt and #87 (comment) from @cji have been awesome and helpful for understanding how you all work, and gaining confidence in Heroku 👍

Your Dockerfile depends on ubuntu-debootstrap image which is now deprecated, see:
https://hub.docker.com/_/ubuntu-debootstrap/

As of right now, the docker image heroku/cedar:14 does not contain the openssl packages that fix http://www.ubuntu.com/usn/usn-3087-1/:

$ docker pull heroku/cedar:14
14: Pulling from heroku/cedar

Digest: sha256:43c7680c05ac0d68f141d530c55abd3746d2eebcbbcf64fbd36b16649615b19d
Status: Image is up to date for heroku/cedar:14
$ docker run heroku/cedar:14 openssl version
OpenSSL 1.0.1f 6 Jan 2014
$ docker run heroku/cedar:14 apt-cache policy libssl1.0.0
libssl1.0.0:
  Installed: 1.0.1f-1ubuntu2.19
  Candidate: 1.0.1f-1ubuntu2.19
  Version table:
 *** 1.0.1f-1ubuntu2.19 0
        100 /var/lib/dpkg/status

The fix is in the Xenial apt repositories, so heroku/cedar:14 just needs to be rebuilt in order to update and download these security fixes.

ld is required by Python's ctypes.util.find_library

ld is part of binutils-common and currently only available in build images not runtime.

The recent Heroku-16 blog post and the Heroku-16 stack / Container Registry and Runtime devcenter pages just mention the heroku/heroku:16 docker image.

However the docker hub tags page lists two variants, 16 and 16-build.

From reading #53 (not yet merged), 16-build is the image that contains all of the development headers, so is more likely the image that people will need when developing locally, rather than the base 16 image.

Could the docs be updated to clarify this? (The two deventer pages, this repo's readme, and the docker hub description.)

Many thanks :-)

I manage a Django app. I frequently deploy new environments for testing purposes.

Although the Heroku stack seems to have included gettext for a while now, it has never worked for me; I always get CommandError: Can't find msgfmt. Make sure you have GNU gettext tools 0.15 or newer installed., which is mentioned in several GitHub Issues/Stack Overflow questions. So I use heroku-buildpack-gettext.

I haven't had any deploy problems until fairly recently. New versions of my app won't start, although all deploys appear to be successful. It seems this corresponds to when heroku-20 became the default stack.

In the logs, this error is repeated many times:

2021-01-16T15:22:48.065513+00:00 app[web.1]: processing file django.po in /app/.heroku/python/lib/python3.9/site-packages/django/contrib/gis/locale/en/LC_MESSAGES
2021-01-16T15:22:48.094738+00:00 app[web.1]: Execution of msgfmt failed: msgfmt: error while loading shared libraries: libncurses.so.5: cannot open shared object file: No such file or directory

The output from the deployment includes this:

remote: -----> Gettext library app detected
remote: -----> Installing gettext...
remote: /tmp/build_b06e1515/.heroku/gettext/bin/      gettext: error while loading shared libraries: libncurses.so.5: cannot open shared object file: No such file or directory

If I remove heroku-buildpack-gettext in heroku-20 or heroku-18, then I get this error, which is the one that led me to start using the buildpack in the first place:

2021-01-16T15:28:45.462654+00:00 app[web.1]: CommandError: Can't find msgfmt. Make sure you have GNU gettext tools 0.15 or newer installed.

If I set my stack back to heroku-18 (heroku create $app_name -s heroku-18) and include heroku-buildpack-gettext, then the problem goes away.

ubuntu 18.04, container env PYTHONHOME trigger error

apt-get install xvfb

unset PYTHONHOME , normal

The cedar-14 build on master currently fails with:

+ apt-get install -y --force-yes autoconf bind9-host bison build-essential coreutils curl daemontools dnsutils ed git imagemagick iputils-tracepath language-pack-en libbz2-dev libcurl4-openssl-dev libev-dev libevent-dev libglib2.0-dev libjpeg-dev libmagickwand-dev libmysqlclient-dev libncurses5-dev libpq-dev libpq5 librdkafka-dev libreadline6-dev libssl-dev libuv-dev libxml2-dev libxslt-dev netcat-openbsd openjdk-7-jdk openjdk-7-jre-headless openssh-client openssh-server postgresql-client-9.6 postgresql-server-dev-9.6 python python-dev ruby ruby-dev socat stunnel syslinux tar telnet zip zlib1g-dev
...
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
libmagickwand-dev : Depends: libmagickcore-dev (= 8:6.7.7.10-6ubuntu3.7) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

(See: https://travis-ci.org/edmorley/stack-images/jobs/248906956)

The libmagickwand-dev package's dependencies should be getting installed by default, since they're listed as depends rather than say suggests:
https://packages.ubuntu.com/trusty-updates/libmagickwand-dev

Trying to fix this by manually adding libmagickcore-dev to the list of packages being installed doesn't work, since it results in several more unmet dependencies, and after adding those, finally fails with:
libgraphviz-dev : Depends: libgvc6-plugins-gtk (= 2.36.0-0ubuntu3.2) but 2.36.0-0ubuntu3 is to be installed

These errors are all due to the fact that cedar-14 only lists the universe component for the main trusty repository and not also trusty-security or trusty-updates, leading to inconsistent latest package versions:
https://github.com/heroku/stack-images/blob/9443e8875b7448c2cdfb5534d6099e3f9c46ae9a/bin/cedar-14.sh#L7-L14

...since the version of libgvc6-plugins-gtk on which libgraphviz-dev depends is in trusty-updates/universe rather than trusty/universe:

# apt-cache policy libgvc6-plugins-gtk
libgvc6-plugins-gtk:
  Installed: 2.36.0-0ubuntu3.2
  Candidate: 2.36.0-0ubuntu3.2
  Version table:
 *** 2.36.0-0ubuntu3.2 0
        500 http://archive.ubuntu.com/ubuntu/ trusty-updates/universe amd64 Packages
        100 /var/lib/dpkg/status
     2.36.0-0ubuntu3.1 0
        500 http://archive.ubuntu.com/ubuntu/ trusty-security/universe amd64 Packages
     2.36.0-0ubuntu3 0
        500 http://archive.ubuntu.com/ubuntu/ trusty/universe amd64 Packages

Adding universe to the other repositories fixes the issue, and also makes the APT sources list more consistent with that in both the stock Ubuntu 14.04 and the Heroku-16 image.

I'll open a PR to fix this after first adding a Travis configuration to make it easier to test.

Opening this as a placeholder. It would be simple enough to publish Cedar and future stack images on Vagrant Cloud.

Hi,

I'd like to run heroku locally to ease development, i.e.: so as to have the same environment for dev as heroku provides for staging and prod.

I'd like to use the heroku Docker images for that. This way, I wouldn't have to install every requirement manually in OS X, and would be able to juggle between different versions with the same ease as on heroku.

It's not clear yet how I can use those images.

I was able to pull the image with docker pull heroku/heroku:16, then start it with docker run -i -t heroku/heroku:16 /bin/bash.

However, I'm a bit confused as to what to do next. Could you provide some guidelines? e.g.:

How do I (Can I?) configure heroku to deploy locally to the local Docker heroku container, instead of the staging or production environments?