bkeepers / dotenv Goto Github PK

Further to #71, I agree with not checking in the .env file and the philosophy of separating deployment configuration from the code.

But there's a capistrano step to link the .env file in production. Surely the dotenv gem is intended to be used in production to load the linked .env file?

I enjoy using dotenv in production to load environment variables specific to a deployed app, so want to discuss why it is that it is recommended to only be in the :development and :test groups in the Gemfile?

Shouldn't it also be for production?

https://github.com/laserlemon/figaro/blob/master/lib/figaro/tasks.rb

Probably a small set of tasks like this:

• dotenv:heroku:set (default if dotenv:heroku)
• dotenv:heroku:unset

Is there some reason this can't be allowed?

ABC       = 123
FOO_BAR   = " multiline\nstring"

It's a simple tweak to the regex and allows the values to be lined up better for readability.

Why is not the .env file automatically loaded during Rails startup/boot? The Figaro gem seems to do this successfully.

If not automatically loaded, can you add to the documentation the best place to do it in a generic Rails app?

Thanks,
Martin

I was having some problems where things in .env.development were being over-ridden by things in .env, when I noticed in https://github.com/bkeepers/dotenv/blob/master/lib/dotenv-rails.rb

if File.exists?(env_file) && !defined?(Dotenv::Deployment)
  warn "Auto-loading of `#{env_file}` will be removed in 1.0. See " +
    "https://github.com/bkeepers/dotenv-deployment if you would like to " +
    "continue using this feature."
  Dotenv.load ".env.#{Rails.env}"
end

A really useful pattern I've found in conjecture with boxen is to have boxen build a .env, and then allow developer to have .env.development for their own things. This would you to have .env point to a default working development environment, with options to override it in .env.development.

Maybe could change that warning to be for non-development or test environments?

3.0.0 version has lots of upgrades and improvements and people are slowly migrating to this new version. Although many of those changes are not backwards compatible with old 2+ capistrano versions.

If you try 3.0.0 now with latest bkeepers/dotenv version it would fail with:

cap staging deploy
cap aborted!
undefined method `instance' for Capistrano::Configuration:Class
/Users/nfedyashev/.gem/ruby/2.0.0/gems/dotenv-0.9.0/lib/dotenv/capistrano/recipes.rb:1:in `<top (required)>'
/Users/nfedyashev/.gem/ruby/2.0.0/gems/dotenv-0.9.0/lib/dotenv/capistrano.rb:1:in `<top (required)>'
config/deploy.rb:1:in `<top (required)>'
/Users/nfedyashev/.gem/ruby/2.0.0/gems/capistrano-3.0.0/lib/capistrano/setup.rb:12:in `load'
/Users/nfedyashev/.gem/ruby/2.0.0/gems/capistrano-3.0.0/lib/capistrano/setup.rb:12:in `block (2 levels) in <top (required)>'
/Users/nfedyashev/.gem/ruby/2.0.0/gems/capistrano-3.0.0/lib/capistrano/application.rb:12:in `run'
/Users/nfedyashev/.gem/ruby/2.0.0/gems/capistrano-3.0.0/bin/cap:3:in `<top (required)>'
/Users/nfedyashev/.gem/ruby/2.0.0/bin/cap:23:in `load'
/Users/nfedyashev/.gem/ruby/2.0.0/bin/cap:23:in `<main>'
Tasks: TOP => staging
(See full trace by running task with --trace)

There are several ways to fix it, but seems like the simplest way to symlink .env in 3.0.0 is to add it to linked_files variables, like:

-set :linked_files, %w{config/database.yml}
+set :linked_files, %w{config/database.yml .env}

I guess that is more obvious and simple than checking capistrano version in our dotenv, and changing linked_files variable.

Two approaches: "We did everything so you don't have to, but please make sure .env is present in shared dir vs "ok, I need to make dotenv work with 3.0+ capistrano, README tells me to add .env to linked_files - now I understand that it has to be present in shared dir too"

I vote for the latter(no updates to lib/dotenv/capistrano.rb and lib/dotenv/capistrano.rb but README instructions has to be update for 3.0+ versions).

Should we allow comments? Eg.

FOO=foo # 'foo', 'bar' or 'baz'

To be admittedly nitpicky, the 12f doc at your pointer raises the following (valid) point, emphasis mine:

Another approach to config is the use of config files which are not checked into revision control, such as config/database.yml in Rails. This is a huge improvement over using constants which are checked into the code repo, but still has weaknesses: it’s easy to mistakenly check in a config file to the repo;

Later it reiterates:

Env vars are easy to change between deploys without changing any code; unlike config files, there is little chance of them being checked into the code repo accidentally;

I have to say, I agree here. As long as you're storing sensitive configuration such as secret keys or database passwords in this file, it shouldn't be under version control. However, you obviously have a good point that not providing sensible defaults gives developers an unnecessary roadblock to getting up and running.

I'm suggesting that the best practice I've seen to deal with this is, like database.yml in some projects, to provide a env.example or env.development or env.README (note the absence of the hiding . at the beginning) with some sensible development defaults and comments directing the user to copy it to .env. That, along with directions to add .env to .gitignore seems like the most responsible method of planning for the inevitable mistakes that occur. Many a time have I accidentally committed sensitive information in a file kept under version control when that file really shouldn't be under vc.

just for the completeness :)

https://travis-ci.org/

When deploying my application using Capistrano 2, none of the environment variables I've specified in my .env.* files are available. It seems that it's because there's nothing like the railtie for Capistrano.

From trawling through the issues it appears that Dotenv will load a .env file and then any environment specific files: env.staging etc.

However this isn't clear from the README. Please add clarification for the behaviour:

1. Does Dotenv always load a .env file even if there is an environmental specific file aswell? In what order?
2. If so, and a variable is defined in the .env file, what happens if it is defined again in the environmental specific file? Is it overridden?

It would also be worth mentioning that Dotenv is used by Foreman as I think that is a good selling point.

Hello,

I've been using Dotenv locally with our application and noticed that neither Dotenv or foreman support multiline environment variables.

We use environment variables to store some RSA private keys that are specific for each environment.

However, Heroku does support multiline keys and that works perfectly.

Having Dotenv support those will help us avoid committing a private key into the repository.

If you agree, I can start looking into adding this support.

Thank you.

❤️ ❤️ ❤️

As the days have passed so has the size of dotenv grown. To allow the main gem focus on what dotenv does well there is a proposal from #66 to break out some features into other gems.

Some of these might be:

• deployment (capistrano or other)
• dotenv executable
• extensions not accepted into core (like the .ensure method)
• rails support

In #66, @bkeepers mentions that he'd like to get dotenv back to being a small shim in development. What we need is a line drawn in the sand over which dotenv shall not step and anything that we might want to add can go in the other gems or have new gems created.

The problem is the fact that DotEnv doesn't overwrite keys in ENV that already exist. Since Unicorns zero downtime feature works by having the "old" master process execute a new master process the new master process inherits the old ENV and DotEnv dutifully skips setting any of the old DotEnv keys which means you can't do zero downtime deploys using Unicorn + DotEnv and have configuration update itself.

I would suggest at least adding a setting to DotEnv so that you can have it overwrite existing keys... and honestly, i.m.o, I don't think NOT overwriting keys in production is a bad default. I can understand it for development but not production.

Hi there,
gem install dotenv returns the following error

~ » gem instal dotenv                                                                                                                                                                                                                                                                                                                                     adiguer@AlbansMac
ERROR:  Error installing dotenv:
        dotenv requires dotenv-deployment (~> 0.0.2)

However when I build locally that version and install it, it works.
It might be worth on your end trying to upload again the gem to rubygems.org @bkeepers.

Ruby 1.8.7-p352
Rails 3.0.3
Linux Mint 13

Ran rake db:create or rake db:migrate. I get error:

rake aborted!
undefined method `desc' for Dotenv::Railtie:Class
/home/me/Projects/tmyf/vendor/bundle/ruby/1.8/gems/dotenv-0.6.0/lib/dotenv/railtie.rb:8

Removing dotenv-rails from gemfile allows me to run rake.

Trying to use dotenv in a Rails 2 app but I get uninitialized constant Rails::Railtie. Is there a specific version of the gem I should be using?

I've been using dotenv in more places recently, but one place that I'm having problems with right now is in config/database.yml on a staging server (ie non dev environment):

staging:
  adapter: postgresql
  database: <%= ENV['POSTGRESQL_DATABASE'] %>
  host: <%= ENV['POSTGRESQL_HOST'] %>
  username: <%= ENV['POSTGRESQL_USERNAME'] %>
  password: <%= ENV['POSTGRESQL_PASSWORD'] %>
  pool: <%= ENV['POSTGRESQL_POOL'] || 5 %>
  timeout: <%= ENV['POSTGRESQL_TIMEOUT'] || 5000 %>

I was having a heck of a time, because these ended up being empty, which means it tries loading on the localhost rather than a different server. I added debugging raises like this:

  database: <%= ENV['POSTGRESQL_DATABASE'] || raise no "ENV['POSTGRESQL_DATABASE'] " %>

And sure enough, it's raised. I also threw in a <%= raise ENV.inspect %>, and none of the variables in .env or .env.staging are loaded. I thought that was weird, because I have used something like this without problem in development when using boxen:

development:
  adapter: mysql2
  database: app_development
  username: root
  password:
  port: <%= ENV['BOXEN_MYSQL_PORT'] || 3306 %>

My current hunch is that ActiveRecord::Railtie is being much earlier in the initialization process than dotenv is. I'll report back if I'm able to debug this anymore.

I have a .env file and a .env.development file. In my .env file I have a setting defined as:

ENABLE_WEB_UI=false

And in my .env.development file I have this value overridden as well as an additional value:

ENABLE_WEB_UI=true
MY_OTHER_SETTING=http://www.test.com

In the rails console, if I can write out this value:

"#{Rails.env}: #{ENV['ENABLE_WEB_UI']} #{ENV['MY_OTHER_SETTING']}"

I get:

"development: true http://www.test.com"

But if I write out the same string in my rake task, I get:

"development: false http://www.test.com"

It appears that it is successfully finding my .env.development file, and the Rails environment is correctly set, but the value from .env.development is not overriding the value in .env when running in the rake task. Any ideas?

If I want a different ENV for development vs. test, should I use dotenv-deployment?

Hi,

I use heroku-config to pull production env variables from Heroku to .env locally, so having Dotenv loading both .env and .env.development|test can potentially result of having production variable in test env. Not good :)

Wouldn't be safer to skip loading .env file if a .env.* file is present?

Thoughts?

If I have this in my .env file

FEATURE_A_ENABLED=true
FEATURE_B_ENABLED=

The first line works as expected, but the second also behaves as true.

if ENV['FEATURE_B_ENABLED']
  do_some_stuff
end

In this case, do_some_stuff will be called. My natural expectation is that it wouldn't be.

dotenv sets ENV['FEATURE_B_ENABLED'] to "", which is truthy.

I would expect that ENV['FEATURE_B_ENABLED'] would be nil.

The recommendation is to commit the .env file with the development only values, so development machines can get running faster, and I like the idea.

But, if Foreman is used to run in production (like heroku does) then the environment is set there by this file (as it uses dotenv internally) and it seems to override the current environment.

To solve this, Heroku recommends not to commit the .env file, but I don't like that, as I like that feature, is there a way to solve it?

Out of the box, Spring will not watch .env for changes. This can cause problems with rails console or test suites.

Current fix is to create a Spring config file and include this:

# config/spring.rb

Spring.watch ".env"

Maybe we can bootstrap rails apps with the new config file and option?

Hi,

We're using a lot of variables with dots in their names, e.g. "toggle.feature_x=true", which are ignored by dotenv. Could dotenv support those, or is it limited to "\w" characters for a reason?

I have been using a .env file in my project to track required environment variables. While I don't want to commit those values it is helpful to list their names to help other developers identify what they might need to set or choose to override in order to support certain features of the app. As a result my .env contained a number of empty assignment statements:

FOO=
BAR=
BAZ=some_default_value

This worked fine for me in 0.7.0 After upgrading to 0.8.0 this triggers a Dotenv::FormatError. I can work around it by commenting out each of these lines but that seems like it shouldn't be necessary.

Is considering empty assignment statements invalid a deliberate choice or an unintended side-effect?

In .env:

S3_BUCKET=my_app

In config/application.rb:

module MyApp
  class Application < Rails::Application
    config.fog_directory = ENV['S3_BUCKET']
  end
end

Now we start an application console:

bundle exec rails console
irb(main):001:0> MyApp::Application.config.fog_directory
=> nil

I'd love to be able to have this in my .env file:

ADMIN_EMAIL=$(git config --get user.email)
ADMIN_NAME=$(git config --get user.name)

This is so that the .env.example file we commit with the repo can work out of the box on other developers' machines.

Could this feature be accepted if I implemented it properly?

How would one go about using this for a GEMFURY_KEY in a Gemfile

Ala...

source 'http://rubygems.org'
source "https://#{ENV['GEMFURY_KEY']}@gem.fury.io/me/"

gem 'rails'

group :development, :test do
  gem 'dotenv-rails'
end

Seems to be a little bit of a chicken-before-egg problem.

Thanks
-- Arron

Right now

FOO=true

results in

>> ENV['FOO']
=> "true"

Wouldn't it be nicer if this would be a boolean?

I'd be happy to prepare a PR, just wanted to get some feedback on the idea first.

For some reason in my setup, the ENV variables are not set before .erb files are interpreted.

I've checked to see if they are being set in controllers and models -- they are. Same with console. Everything looks normal there.

Any guess what could be going on?

Rails 4, ruby 2.1, guard, puma

I just noticed this unexpected behavior and briefly read the source code and couldn't figure out why + sign gets stripped out of my ENV variable

For example:

[email protected]

The + is stripped out and ENV["BLAHDOTCOM_USER"] == dylan [email protected]

I'm attempting to use dotenv to control my RAILS_ENV variable on our staging environment but when I run any command ActiveRecord complains about no adapter being specified. My database.yml contains an entry for our staging environment and when I change it to development rails boots up juts fine. Can dotenv be used to set RAILS_ENV?

I recently ran into a problem because in my .env file I was setting a password, and that password happened to have a $ in it. eg: PASSWORD='pass$word'. Although I single quoted the string, the parser is still expanding the variable, when it shouldn't be.

The correct behaviour would be to expand the variable when the string is double quoted, and to not expand the variable when the string is single quoted.

I wrote some tests that reflect the proper behaviour but I'm having some difficulty writing a solution that passes all the other tests. Any help or pointers in the right direction would be appreciated. I'm happy to keep working on a solution myself but thought I would throw it out there in case this issue is known.

Here are the tests I wrote:

it 'expands variables in double quoted strings' do
  expect(env("FOO=test\nBAR=\"quote $FOO\"")).to eql('FOO' => 'test', 'BAR' => 'quote test')
end

it 'does not expand variables in single quoted strings' do
  expect(env("BAR='quote $FOO'")).to eql('BAR' => 'quote $FOO')
end

The first line of Rakefile in any rails app is:

require File.expand_path('../config/application', __FILE__)

This causes dotenv to always be initialized in "development" (loading .env and .env.development) unless RAILS_ENV is explicitly set in the environment before running rake, which leads to unexpected results when running rake tasks that affect other environments. For example, put a variable in .env.test and run rake test to see that the variable is not properly loaded.

Dotenv currently initializes as soon as the Railtie is loaded so ENV variables can be used in config/application.rb (see #10). Dotenv also avoids overriding existing variables so it doesn't clobber values set in the shell environment. So once dotenv has been initialized, changing the RAILS_ENV and re-initializing doesn't change anything.

If the Rails boot process created a clear distinction between bootstrap, configuration, and initialization in the startup process, then dotenv could hook in before the configuration step. But unfortunately config/application.rb requires code and provides application configuration.

Here are a few options to fix this issue. I would love other suggestions.

1. Remove implicit environment support. Instead of automatically loading .env.test, we could add a :force option and the README could encourage users to add Dotenv.load '.env.test', :force => true to their spec or test helper.
2. Add Dotenv.reload which gets called before initialization. This would track which variables were initially set by dotenv, and reread the configs. This adds a lot of flexibility, but at the cost of some complexity and could lead to unexpected behavior where config/application.rb and initializes are loaded with different configurations.
3. Discourage use of ENV variables in config/application.rb and wait until initialization to load dotenv. This was the original behavior, but is annoyingly inconsistent with dotenv's goal of being an unobtrusive bridge for environment variables.

Personally, I'm leaning toward option 1. It replaces complexity with explicit action. Edit: See comment below.

I'm running afoul of #46 when using dotenv as part of foreman. The gemspec over there specifies >= 0.7, so I can work around the issue by manually installing the older dotenv version. If there were a new release of dotenv I wouldn't need to do that. :-)

I think there is a strong use-case for supporting Dotenv use in production:

Obvoiusly storing a .env file on a production server is a no-no, however there are cases when Rails needs to be run with RAILS_ENV=production (or staging) on your development machine.

A good example of this is when precompiling assets for production. When doing so it is necessary to do:

$ RAILS_ENV=production rake assets:precompile

At this point, a number of Gems, of which AssetSync is the most popular, jump in and handle upload of assets to S3 etc. They need access to envs in order to b e configured. In this situation it would be very useful for Dotenv to support loading of .env files when the production environment is initialized.

I'm currently having to resort to adding the following to my environment specific config:

Dotenv.load ".env.staging", '.env'

First off, thanks for dotenv -- a simple and effective solution to a universal problem.

I've got the following:

• a fresh Rails 4.0 app
• dotenv
• a .env file in repo root
• postgres
• a database.yml that uses ERB to supply username/password from environment variables, which are loaded by dotenv from .env

Everything works fine:

• rails server brings up the server, which successfully connects to the db
• rake db:* commands work correctly

The only exception is "rails dbconsole". It doesn't seem to inherit the env vars, and instead tries to connect via my local username. When I prepend the value manually, it does pick it up:

$ POSTGRESQL_USERNAME=whatever rails dbconsole

I don't know much about Rails internals. I would assume that "rails dbconsole" would have the same bootstrapping/gem-invoking process as "rails server", but either it's failing to load dotenv, or it doesn't consult database.yml for creds.

I believe the command merely invokes the 'psql' command-line client on my behalf, but it should still be supplying values from database.yml, shouldn't it?

Is there any reason for the duplicated spec for Dotenv::Environment#apply in environment_spec.rb lines 33-36 within the describe 'apply! do block? @ecbypi

describe 'apply!' do
   it 'sets variables in the ENV' do
      subject.apply
      expect(ENV['OPTION_A']).to eq('1')
    end
end

Server should start even when .env does not exist

# Load this ASAP to have .env loaded into ENV
begin
  require 'dotenv'
  Dotenv.load
rescue LoadError
end

Consider an existing environment variable FRUIT=apple and a .env file like:

FRUIT=banana
FRUIT2=${FRUIT}

If I evaluate the file with Dotenv.load, FRUIT will not be overridden and will stay set as apple (this is the desired behavior), but FRUIT2 will be set as banana (not desired, IMO).

I believe this happens because parsing w/ substitution happens independent of the decision to load or overload the value ... yet substitution prefers values set in the file over the current environment in either case.

/cc: @wfarr

I'm unable to get dotenv capistrano to work.

I have gem 'dotenv-rails' in my gemfile (with no groups, since I'm using it production)

When I add

require 'dotenv/capistrano'

to config/deploy.rb I get this error:

gems/capistrano-2.15.4/lib/capistrano/configuration/loading.rb:152:in `require': cannot load such file -- dotenv/capistrano (LoadError)
from /path/to/gems/capistrano-2.15.4/lib/capistrano/configuration/loading.rb:152:in `require'
from ./config/deploy.rb:3:in `load'

Help?

RubyGems.org doesn't report a license for your gem. This is because it is not specified in the gemspec of your last release.

via e.g.

spec.license = 'MIT'
# or
spec.licenses = ['MIT', 'GPL-2']

Including a license in your gemspec is an easy way for rubygems.org and other tools to check how your gem is licensed. As you can imagine, scanning your repository for a LICENSE file or parsing the README, and then attempting to identify the license or licenses is much more difficult and more error prone. So, even for projects that already specify a license, including a license in your gemspec is a good practice. See, for example, how rubygems.org uses the gemspec to display the rails gem license.

There is even a License Finder gem to help companies/individuals ensure all gems they use meet their licensing needs. This tool depends on license information being available in the gemspec. This is an important enough issue that even Bundler now generates gems with a default 'MIT' license.

I hope you'll consider specifying a license in your gemspec. If not, please just close the issue with a nice message. In either case, I'll follow up. Thanks for your time!

Appendix:

If you need help choosing a license (sorry, I haven't checked your readme or looked for a license file), GitHub has created a license picker tool. Code without a license specified defaults to 'All rights reserved'-- denying others all rights to use of the code.
Here's a list of the license names I've found and their frequencies

p.s. In case you're wondering how I found you and why I made this issue, it's because I'm collecting stats on gems (I was originally looking for download data) and decided to collect license metadata,too, and make issues for gemspecs not specifying a license as a public service :). See the previous link or my blog post about this project for more information.

Environment variables tend to fall under one of the following categories:

• Global All environments
• Environment-specific (development-only, etc.)
• User-specific

I'm proposing that this library support files in the following formats:

• .env - loaded in all environments (checked into the project)
• .env.<environment> - loaded in that specific environment (checked in)
• .env.local - user-specific (not checked in)

and that they override one another in that order. I'm very open to feedback about the filename format, but I think this would be hugely helpful for keeping variables organized.

Happy to take this on, if there's ❤️ for the idea.

/cc @dblock @fancyremarker @joeyAghion

Rails seems to be loading .env fine but Sidekiq in production doesn't seem to be. However, if I create an initializer with

Dotenv.load Rails.root.join('.env')

it picks it up fine. Note that in production, the sidekiq executable is contained within bin/ so it could be related to #82

I have two different rails environments. In the first, I followed the instructions to set up dotenv and to access variables stored in the .env file, and it worked well.

I then tried the same thing on the other environment and it did not work, in particular, if I defined SOME_KEY in the .env file, then ENV['SOME_KEY'] does not retrieve the key.

I did notice that in the second environment there was already a .env file, which I think had been created for use with foreman, so I am wondering if there is a conflict there?

When using Rails and scanning through the Readme, I subconsciously skip the Sinatra or Plain ol' Ruby paragraph. But it contains the important info that applies to any environment:

As early as possible in your application bootstrap process, load .env:

require 'dotenv'
Dotenv.load

If you don't do this, Rails will load the ENV-vars too late and some settings may get the wrong values. So I think that paragraph should be given its own title.
UPD. Another way I found to make dotenv load first is to place it first in the Gemfile. Maybe this should be in the Readme.

We've noticed that files running in app_root/bin look in the bin/ directory for .env files instead of the app_root.

=== steps to reproduce ===
I use RVM

rvm gemset create testdotenv
rvm gemset use testdotenv
gem install rails -v 4.0.2
rails _4.0.2_ new testdotenv
cd testdotenv

Setup rvm stuff

echo "ruby-2.0.0-p353" .ruby-version
echo "testdotenv" > .ruby-gemset
rvm use ruby-2.0.0-p353@testdotenv

add missing gem not sure why this isn't included by rails.

echo "gem 'i18n'">>Gemfile

install dotenv-rails

echo "gem 'dotenv-rails'">>Gemfile

install gems

bundle

setup a env test

echo "TEST_VAR=12345" > .env
echo "TEST_VAR=12345" > .env.development

Create a file in bin called test.rb and chmod u+x it .

contents of test.rb

#!/usr/bin/env ruby

# You might want to change this
ENV["RAILS_ENV"] ||= "development"

require File.dirname(__FILE__) + "/../config/application"
Rails.application.require_environment!

some_value = ENV['TEST_VAR']
puts "some_value is #{some_value}"

running this with no .env file in app_root/bin will output "some_value is"

if you do "ln -s ../.env" in the bin directory and run it again it will output "some value is 12345"

The .env values don't get loaded into Pry with the dotenv gem.

The same code does work when using irb and when running rspec.

I'm using pry 0.10.0, dotenv 0.11.1 and ruby 2.1.2.