droplr / aws-env Goto Github PK

We're running into a bit of a problem with the -format dotenv flag.

We've noticed that when reading values out of SSM and using -format dotenv, all of the values are wrapped in double quotes. This ends up breaking most of the other services and tools we use that parse these quotes as literal double quotes.

For instance, if the .env file was:

FOO="bar"

Then something that read that environment variable would read it as "bar" instead of the expected bar.

What's the expect way to deal with the double quotes?

Hi,

We're using aws-env in ECS and therefore get our credentials through an AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variable that points to a relative URI where the credentials can be downloaded from. This doesn't seem to work with the current version of aws-env, as it's using the default config with anonymous credentials. However, if you supply the defaults.Get().Config (defaults.go) when creating new session the SDK will look at the above env-var among other things.

I have made the below change in a fork and it works well. Though, I wanted to check with you before submitting a PR if there are consequences to this that I'm not aware of.

From

func CreateClient() *ssm.SSM {
	sess := session.Must(session.NewSession())
	return ssm.New(sess)
}

To

func CreateClient() *ssm.SSM {
	cfg := defaults.Get().Config
	sess := session.Must(session.NewSession(cfg))
	return ssm.New(sess)
}

AWS_REGION is widely used across multiple aws tools, yet due to parameter-store nature we need to use different region quite often.
Instead of var cycling or sub-shelling (reducing env+stack traceability) its better to use dedicated variable (AWS_ENV_REGION)

proposal (logical solution):
AWS_ENV_REGION || AWS_REGION || current_region(user-data) || null

I switched some services in ECS to use Fargate instead of EC2 and it seems to cause aws-env to blow up.

2018/05/10 03:06:04 NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
panic: NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors
goroutine 1 [running]:
panic(0x6adf20, 0xc42000bfc0)
/usr/local/Cellar/go/1.7.4/libexec/src/runtime/panic.go:500 +0x1a1
log.Panic(0xc4200c5eb0, 0x1, 0x1)
/usr/local/Cellar/go/1.7.4/libexec/src/log/log.go:320 +0xc9
main.ExportVariables(0xc42001200d, 0x19, 0x0, 0x0)
/Users/aorfin/go/aws-env/aws-env.go:42 +0x254
main.main()
/Users/aorfin/go/aws-env/aws-env.go:19 +0x113

Binary is being pulled from latest commit.

This looks like it could be a bug in the aws go sdk or something wonky in Fargate; I am pretty green to AWS and have zero go experience so I wasn't sure where to start on debugging this.

Any suggestions?

Can we set up a proper release of the binaries with hash keys?

Parameters with spaces/new lines are not exported correctly.

Detailed discussion is on SO: https://stackoverflow.com/questions/45098315/exporting-environment-variables-from-script-with-whitespaces

Reference: https://github.com/GoogleContainerTools/distroless

Guide me for integrating in distroless docker.

Hi,

What would you think about adding a customizable region that could be passed with a flag instead of using the AWS_REGION environment variable ?

Here is my use case, I am currently using eu-west-3 (Paris) for most of my services. Unfortunately, SSM Parameter Store is not yet available in this region, thus I am currently putting my SSM Parameter Store data in eu-west-1 region.
But I don't want to put eu-west-1 as AWS_REGION env variable as it could cause some side effect with other services.

Instead, I would like to be able to define my image entrypoint with something like this :

ENTRYPOINT ["/bin/bash", "-c", "eval $(aws-env --region eu-west-1)"]

What do you think about that ? Does it seem legit ?

I am getting the following error when my container executes:

2018-12-05 00:26:51panic: MissingRegion: could not find region configuration
2018-12-05 00:26:51goroutine 1 [running]:
2018-12-05 00:26:51log.Panic(0xc420107ee0, 0x1, 0x1)
2018-12-05 00:26:51 /usr/local/Cellar/go/1.10.3/libexec/src/log/log.go:326 +0xc0
2018-12-05 00:26:51main.ExportVariables(0xc42000c0a0, 0xc42001600d, 0x21, 0x0, 0x75f5f8, 0x7, 0x0, 0x0)
2018-12-05 00:26:51 /Users/aorfin/go/aws-env/aws-env.go:62 +0x17d
2018-12-05 00:26:51main.main()
2018-12-05 00:26:51 /Users/aorfin/go/aws-env/aws-env.go:37 +0x193
2018-12-05 00:26:512018/12/05 06:26:51 MissingRegion: could not find region configuration

This seems to be some sort of AWS configuration issue, but my IAM role has permissions to for ssm:Get* on all resources.

Here is my Dockerfile:

FROM php:7.2.11-fpm

RUN apt-get update

RUN apt-get install wget

# Install opcache
RUN docker-php-ext-install opcache

# Install APCu
RUN pecl install apcu
RUN docker-php-ext-enable apcu

ADD webroot /webroot
ADD etc/zz-docker.conf /usr/local/etc/php-fpm.d/zz-docker.conf

RUN wget https://github.com/Droplr/aws-env/raw/9605a216913c18cf64338509204ca21379832b90/bin/aws-env-linux-amd64 -O /bin/aws-env && \
  chmod +x /bin/aws-env

CMD ["/bin/bash", "-c", "eval $(aws-env) && php-fpm"]

It is my understanding, and my debug shows, that the AWS_REGION env variable is set in my codebuild project that builds the image. I did try changing the last line to the following with the same result:

CMD ["/bin/bash", "-c", "eval $(AWS_REGION=$AWS_REGION aws-env) && php-fpm"]

panic: AccessDeniedException: The ciphertext refers to a customer master key that does not exist, does not exist in this region, or you are not allowed to access. (Service: AWSKMS; Status Code: 400; Error Code: AccessDeniedException; Request ID: xxx)

Seeing the above message on launch, I've confirmed the region, and AWS_ENV_PATH are both set and consistent with what is in the parameter store.

Any ideas?

hi,

this tool is simple as awesome,
I've tested it against two dotenv libraries for node and for ruby,
but this is the result:

• ruby with dotenv-rails

>> ENV['DB_PASSWORD']
=> "$'SecretPassword'"

• node with dotenv

App is starting...
key loaded: DB_USERNAME $'Username
key loaded: DB_PASSWORD $'SecretPassword

Can you provide two options for

• remove export
• remove or change quoting (eg. without $ prefix)

I tried to supply AWS_ENV_PATH=/test and it produced no output. Once I realized I need at least two levels: AWS_ENV_PATH=/test/two, it worked great!

"In [ecs-agent] v1.17.0, a rate a limiter was added to the credentials endpoint, which allows clients to invoke the endpoint at a steady state rate of 10 per second and a burst of 15 per second" (aws/amazon-ecs-agent#1231)

aws-env authenticates every time the ssm api call executes, which hits the rate limit when a lot of variables are declared in the parameter store.

You can see the credential requests on ECS nodes using:

docker logs --tail 50 -f $(docker ps |grep ecs-agent |cut -d ' ' -f1 ) |grep credentials

I'll submit a PR to resolve this

Thanks for setting up this package. I've followed the Docker instructions and successfully have the script running on my container startup. My app is receiving the variables, but with a $ appended to the variable. For example, the app is receiving VAR_NAME=$value instead of VAR_NAME=value. Let me know if this is an issue or if I'm just using the package incorrectly.

Hi Droplr folks,

Thank you for publishing your project to the internet! I appreciate the functionality you've created here - I'd really like to use it in development of my project at work. We're deploying our system to AWS and the injection of encrypted Parameters as env vars would be awesome for us. One tiny bump is holding us back at the moment.

Would you be ok adding a license file to your repository? GitHub has a super easy set of license templates to use.

I'd also kindly suggest/hope for Apache 2.0, MIT, or BSD. Again, many thanks for the OSS contribution!

We just ran into an issue where we had some HTML in an environment variable which was quoted, so for example:
aws ssm put-parameter --name /my/path/HTML_BODY --value "'Lorem Ipsum<br><br>'" --type String

Now eval $(aws-env) breaks:
AWS_ENV_PATH=/my/path aws-env-linux-amd64 export HTML_BODY=$''Lorem Ipsum<br><br>''

eval $(AWS_ENV_PATH=/my/path aws-env-linux-amd64)
-bash: syntax error near unexpected token '<'

If aws-env would escape these we would not have a problem:

$ export HTML_BODY=$'\'Lorem Ipsum<br><br>\''
$ echo $HTML_BODY 
'Lorem Ipsum<br><br>'