Describe the Enhancement:

Allow appbundler to define a custom GEM_PATH and GEM_HOME instead of nil.

Describe the Need:

Habitat packages have their environment made by the supervisor, when installing a hab package with -b it creates a symlink in /bin. You can expect that binary to behave as if within a supervisor, but that's actually not the case as appbundler reset GEM_PATH to nil here

ENV["GEM_HOME"] = ENV["GEM_PATH"] = nil unless ENV["APPBUNDLER_ALLOW_RVM"] == "true")

This prevent using any vendored gem in an habitat packaged binstub as there's no way to tell appbundler to set the GEM_PATH to something specific instead of nil and the binstub can't find its dependend gems because they're neither in core/rubyXX nor the user .gem directory.

Current Alternative

You can manually add the package vendor directory to the GEM_PATH environment variable and set APPBUNDLE_ALLOW_RVM=true to circumvent the issue.

Example for chef-infra_client:

export GEM_PATH=$(hab pkg path chef/chef-infra-client)/vendor:$GEM_PATH
export APPBUNDLER_ALLOW_RVM=true

# Validate chef-client can find its gems
/bin/chef-client -v

Can We Help You Implement This?:

I assume yes, my actual idea would be using another environment variable at build time as such:

custom_gem_path = ENV['ALLOW_CUSTOM_GEM_PATH'] ? ENV['GEM_PATH'] : nil
ENV["GEM_HOME"] = ENV["GEM_PATH"] = custom_gem_path unless ENV["APPBUNDLER_ALLOW_RVM"] == "true")`

This would allow to construct a proper GEM_PATH at build time in habitat packages and let appbundled binstubs works outside of the supervisor control while still using the same GEM_PATH as within the supervisor.

This change shouldn't change actual behavior by still setting nill in the resulting file if the variable ALLOW_CUSTOM_GEM_PATH isn't existing.

Would that be something ok ?

This causes chef-boneyard/chef-dk#228

To fix it, we can update the executable to read symlinks, e.g.,

bin_dir = File.dirname(__FILE__)
if File.symlink?(__FILE__)
  bin_dir = File.dirname(File.readlink(__FILE__))
end

$:.unshift(File.expand_path("../embedded/apps/chef-dk/lib", bin_dir))
Kernel.load(File.expand_path('../embedded/apps/chef-dk/bin/chef', bin_dir))

When bundler creates the lock file it omits itself from the resolved specs. This means that when appbundler tries to process the lock file it can't find the spec for bundler.

I'm not sure what the best resolution would be but the following strike me:

• Skip bundler (potentially leaves it floating in the bundled app)
• Default bundler to the current bundler version

Anything beyond that would require using or replicating the bundler constraint resolution I think.

Here are steps to reproduce:

Gemfile:

source 'https://rubygems.org'
gemspec

testing.gemspec:

Gem::Specification.new do |spec|
  spec.name          = "testing"
  spec.version       = '0'
  spec.summary       = 'testing'
  spec.executables   = 'testing'
  spec.authors       = [ 'mike' ]
  spec.add_dependency "bundler", ">= 1.5.2"
end

mkdir bin && touch bin/testing

bundle install

appbundler . .

The resulting Gemfile.lock looks like this:

PATH
  remote: .
  specs:
    testing (0)
      bundler (>= 1.5.2)

GEM
  remote: https://rubygems.org/
  specs:

PLATFORMS
  ruby

DEPENDENCIES
  testing!

Trying to launch kitchen off a bundle using chef-dk:

$ bundle exec kitchen list
/opt/chefdk/embedded/lib/ruby/gems/2.4.0/gems/bundler-1.15.1/lib/bundler/rubygems_integration.rb:377:in `block (2 levels) in replace_gem': activesupport is not part of the bundle. Add it to your Gemfile. (Gem::LoadError)
	from /opt/chefdk/bin/kitchen:8:in `<main>'

That is caused by the appbundle -- chef-dk 2.x makes this worse since it pins the world, but chef-dk 1.x had the same behavior.

To run out of the bundle correctly:

$ bundle exec /opt/chefdk/embedded/bin/kitchen list

We could probably detect via BUNDLE_* env vars that we're being launched under a bundle and bypass appbundling since appbundling within a bundle is crossing the streams and will never work.

Even with the refactoring that I want to do of what goes into /opt/chefdk/embedded/bin and what goes into /opt/chefdk/bin the appbundle'd binstubs should always come first in the PATH, so this will
always be a problem.

The ChefDK 0.3.3 package doesn't have executables for berks, chef-zero, strain, or strainer. My hunch is that there is a silent error when evaluating the gemspec; Berks has code like this:

  s.files                     = `git ls-files`.split($\)
  s.executables               = s.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
  s.test_files                = s.files.grep(%r{^(test|spec|features)/})

I guess that the git command is failing, so s.files is an empty array, and then s.executables is an empty array, too.

We should release appbundler so we can build products against the gem again and not the branch

Dependabot couldn't parse the Gemfile found at /Gemfile.

The error Dependabot encountered was:

Dependabot only supports uninterpolated string arguments to eval_gemfile. Got `__FILE__ + ".local"`

View the update logs.

appbundler manages apps in /opt/{chef,chefdk}/embedded/apps. It would be nice to have a tool that can seamlessly upgrade these applications, so end users can optionally test out new versions without the need to download and install the omnibus package.

Note it will still be recommended that users upgrade the whole package once available. This functionality would allow running test/development versions easier.

Per a conversation with @jdmundrawala and @fnichol at ChefConf:

In order to distribute tools to go with ChefDK to non ruby developers, it would be useful if there was a command to easily create an appbundled version of the tools.

For example

chef app gem specific install

As I'm writing this I'm wondering if we could allow auto updates/pulls for added gems . .