On Heroku-20 the build output showed the errors seen in #12 however the compile of this buildpack did not exit non-zero, so the build was marked as a success.

I believe this would be fixed by using the bash pipefail option, which would stop the |& indent here (and elsewhere) from swallowing the failure:

See also:
http://redsymbol.net/articles/unofficial-bash-strict-mode/

Building an app using this buildpack on Heroku-20 resulted in this output (the build itself didn't fail; for which I'll file a separate issue):

    -----> Installing AWS CLI
           Running cmd: /usr/bin/python -m venv /app/.awscli
           Traceback (most recent call last):
             File "/tmp/build_f2bbe920_/.awscli/awscli-bundle/install", line 223, in <module>
               main()
             File "/tmp/build_f2bbe920_/.awscli/awscli-bundle/install", line 199, in main
               create_install_structure(working_dir, opts.install_dir)
             File "/tmp/build_f2bbe920_/.awscli/awscli-bundle/install", line 77, in create_install_structure
               create_virtualenv(location=install_dir, working_dir=working_dir)
             File "/tmp/build_f2bbe920_/.awscli/awscli-bundle/install", line 84, in _create_virtualenv_internal
               run('%s -m venv %s' % (sys.executable, location))
             File "/tmp/build_f2bbe920_/.awscli/awscli-bundle/install", line 56, in run
               raise BadRCError("Bad rc (%s) for cmd '%s': %s" % (
           __main__.BadRCError: Bad rc (1) for cmd '/usr/bin/python -m venv /app/.awscli': The virtual environment was not created successfully because ensurepip is not
           available.  On Debian/Ubuntu systems, you need to install the python3-venv
           package using the following command.
           
               apt-get install python3-venv
     
           You may need to use sudo with that command.  After installing the python3-venv
           package, recreate your virtual environment.
     
           Failing command: ['/app/.awscli/bin/python', '-Im', 'ensurepip', '--upgrade', '--default-pip']

This is because on Heroku-20, python is now Python 3, not Python 2, and because under Python 3.7+ AWS CLI v1 tries to use venv rather than the virtualenv package:

# From the install script in https://s3.amazonaws.com/aws-cli/awscli-bundle.zip
if GTE_PY37:
    create_virtualenv = _create_virtualenv_internal
else:
    create_virtualenv = _create_virtualenv_external

Looking at the latest AWS CLI docs, they now tell users to install AWS CLI v2 instead:
https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-linux.html

Although that version only supports Python 3.7+, so would only work on Heroku-18+, plus has other breaking changes:
https://docs.aws.amazon.com/cli/latest/userguide/cliv2-migration.html

It seems like our options are to:

1. Adjust the heroku-20 stack image to include the python3-venv package (which isn't a package we've previously made available)
2. Switch this buildpack to using AWS CLI v2 for Heroku-20 only
3. Switch this buildpack to using AWS CLI v2 for Heroku-18+

@dzuelke @dmathieu Thoughts? (Context: This blocks W-8287887).

I'm using this build pack in the hopes that it will allow the package.json "preinstall" hook to fire our aws codeartifact login .. command. This works in other CI environments using secrets/variables.

The build pack does successfully get installed, and it appears as though the login works: "Login expires in 12 hours at .." — however, when the install goes to fetch the package it fails with "401 Unauthorized".

Question 1 —> Can this build pack enable the functionality I am looking for? (e.g. run the aws command within a "preinstall" hook)

Question 2 —> If yes to the above, what am I missing? I've got the config vars set as documented in the readme.

Thanks!

Howdy!

For my application, I need to download an s3 bucket to a local folder whenever the dyno restarts (which resets my ephemeral file system). This currently takes ~ 4 min.

I'd like to set some configuration parameters to try to speed this up. How should I go about configuring this?

aws configure set default.s3.max_concurrent_requests 1000
aws configure set default.s3.max_queue_size 100000

Should I run these during the Heroku build phase? Will they be added to the slug? Or should I set them every time before I run aws s3 sync?

This build pack is no longer compiling on Heroku stack-22.

Started happening within the past 1-2 hours.

Running commands on a one-off dyno with this buildback works nicely:

~ $ aws --version
aws-cli/1.16.208 Python/2.7.15rc1 Linux/4.4.0-1048-aws botocore/1.12.198

But if I try to run aws help I get the following error:

~ $ aws help     

Could not find executable named "groff"

Should groff/less be added as a dependency for this buildback or should this be fixed by aws?
Related aws-cli issue: aws/aws-cli#1957

First: Thanks for providing this buildpack; it's very useful!

We were successfully using this via heroku-community/awscli (which resolves to here), but the recent changes on master caused the following error:

/bin/sh: 1: aws: not found

The only other buildpacks we use are heroku/nodejs and heroku/ruby.

It looks like the AWS CLI binary is now placed in a different location by this buildpack, and that our fairly simple setup chokes on that.

Should we be doing something differently?

We have this as a workaround for now:

1. In the application settings, remove the existing heroku-community/awscli buildpack.
2. Add a new buildpack referencing the specific commit for the last working version, using https://github.com/heroku/heroku-buildpack-awscli.git#01f21a8ac4a3964b0a0475444e0716ec37109256 as the buildpack name.

This seems to successfully overload the "You can optionally specify a tag or a branch ..." functionality described here, but is obviously not a long-term fix.

Hi there!

I tried using the buildpack in the Heroku CI and it failed installing with the following messages:

-----> Fetching heroku-community/awscli buildpack...
       buildpack downloaded
-----> AWS CLI app detected
-----> Installing AWS CLI
       /tmp/buildpacks/737ef5e961c7064fa7fd8cc69b95612443dcbec3b4f9832c1b4849dd69f3156dc19a15b3b06af89a6aff0ccee7d9df74d241d01980c5ca6d440c8dbec62b779b/bin/compile: line 34: /app/.awscli/awscli-bundle/install: No such file or directory

It appears to me that the build, app and tmp directories have a slightly different setup in the Heroku CI than in the slug itself which leads to the deletion of the freshly unzipped folder.

Where is the aws binary installed to? Having trouble getting the command to work.

I have the buildpack in index=1, with ruby and node as well. Have tried aws and /app/.awscli/bin/aws