Calling chefdk chef command from a bundle

Chef Infra (archive)
1

Hi,

I’m writing a couple of thor jobs to configure a workstation for Chef
development. One of the things I want to do is to install a few gems into
the chefdk installed on the system if they don’t exist. I thought it would
be simple enough to just shell out and run the chef command to install
these gems (i.e. chef gem install kitchen-ec2 but calling the chef
command, or indeed the ChefDK ruby from within my bundle results in an
error stating that a gem can’t be found.

So if I run bin/thor myorg:check (where bin/thor is a binstub in my
bundle) and in the thor jobs there’s any command that calls the chef
executable, e.g.
*chef -v *then I get the following:

/Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/spec_set.rb:92:in
block in materialize': Could not find rake-10.4.2 in any of the sources (Bundler::GemNotFound) from /Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/spec_set.rb:85:inmap!'
from
/Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/spec_set.rb:85:in
materialize' from /Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/definition.rb:139:inspecs’
from
/Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/definition.rb:184:in
specs_for' from /Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/definition.rb:173:inrequested_specs’
from
/Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/environment.rb:18:in
requested_specs' from /Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/runtime.rb:13:insetup’
from /Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler.rb:127:in
setup' from /Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/setup.rb:18:in<top (required)>‘
from
/opt/chefdk/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:54:in
require' from /opt/chefdk/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:54:inrequire’

or *system(‘chef -v’) *then I get the following error:
Could not find rake-10.4.2 in any of the sources
Run bundle install to install missing gems.

My bundle has that rake gem but for ruby v2.0.x instead of v2.1.x which the
chefdk uses. I’m guessing that running from within a bundle is messing up
the bundle environment that the chef executable uses, but I’ve not the
knowlege of how this works to get it working.
Is there any way around this (other than use thor from the system ruby,
which would be the fallback, but not ideal)? Will dropping certain
environment variables related to bundler when shelling out fix the problem?

–
Yoshi Spendiff
Ops Engineer
Indochino
Mobile: +1 778 952 2025
Email: yoshi.spendiff@indochino.com

2

I think you might be missing the environment stuff that you get from chef
shell-init
On Tue, Jun 30, 2015 at 3:40 PM Yoshi Spendiff yoshi.spendiff@indochino.com
wrote:

Hi,

I'm writing a couple of thor jobs to configure a workstation for Chef
development. One of the things I want to do is to install a few gems into
the chefdk installed on the system if they don't exist. I thought it would
be simple enough to just shell out and run the chef command to install
these gems (i.e. chef gem install kitchen-ec2 but calling the chef
command, or indeed the ChefDK ruby from within my bundle results in an
error stating that a gem can't be found.

So if I run bin/thor myorg:check (where bin/thor is a binstub in my
bundle) and in the thor jobs there's any command that calls the chef
executable, e.g.
*chef -v *then I get the following:

/Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/spec_set.rb:92:in
block in materialize': Could not find rake-10.4.2 in any of the sources (Bundler::GemNotFound) from /Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/spec_set.rb:85:in map!'
from
/Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/spec_set.rb:85:in
materialize' from /Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/definition.rb:139:in specs'
from
/Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/definition.rb:184:in
specs_for' from /Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/definition.rb:173:in requested_specs'
from
/Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/environment.rb:18:in
requested_specs' from /Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/runtime.rb:13:in setup'
from
/Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler.rb:127:in setup' from /Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/setup.rb:18:in <top (required)>'
from
/opt/chefdk/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:54:in
require' from /opt/chefdk/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:54:in require'

or *system('chef -v') *then I get the following error:
Could not find rake-10.4.2 in any of the sources
Run bundle install to install missing gems.

My bundle has that rake gem but for ruby v2.0.x instead of v2.1.x which
the chefdk uses. I'm guessing that running from within a bundle is messing
up the bundle environment that the chef executable uses, but I've not the
knowlege of how this works to get it working.
Is there any way around this (other than use thor from the system ruby,
which would be the fallback, but not ideal)? Will dropping certain
environment variables related to bundler when shelling out fix the problem?

--
Yoshi Spendiff
Ops Engineer
Indochino
Mobile: +1 778 952 2025
Email: yoshi.spendiff@indochino.com

3

On Tuesday, June 30, 2015 at 3:40 PM, Yoshi Spendiff wrote:

Hi,

I'm writing a couple of thor jobs to configure a workstation for Chef development. One of the things I want to do is to install a few gems into the chefdk installed on the system if they don't exist. I thought it would be simple enough to just shell out and run the chef command to install these gems (i.e. chef gem install kitchen-ec2 but calling the chef command, or indeed the ChefDK ruby from within my bundle results in an error stating that a gem can't be found.

So if I run bin/thor myorg:check (where bin/thor is a binstub in my bundle) and in the thor jobs there's any command that calls the chef executable, e.g.
chef -v then I get the following:

I’ve noticed recently that when running any gem commands inside of a bundler’d ruby process, you need to wrap everything in

Bundler.with_clean_env do; your_gem_related_code; end;

This used to work fine without the clean env thing, so I think there is a recent change to either bundler or rubygems that caused this.

/Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/spec_set.rb:92:in block in materialize': Could not find rake-10.4.2 in any of the sources (Bundler::GemNotFound) from /Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/spec_set.rb:85:in map!'
from /Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/spec_set.rb:85:in materialize' from /Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/definition.rb:139:in specs'
from /Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/definition.rb:184:in specs_for' from /Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/definition.rb:173:in requested_specs'
from /Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/environment.rb:18:in requested_specs' from /Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/runtime.rb:13:in setup'
from /Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler.rb:127:in setup' from /Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/setup.rb:18:in <top (required)>'
from /opt/chefdk/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:54:in require' from /opt/chefdk/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:54:in require'

You’re definitely crossing the streams here. Bundler is installed to the system ruby, and you seem to have your environment variables set in a way that makes ruby load stuff from there. As Brandon mentions, chef shell-init should fix this.

or system('chef -v') then I get the following error:
Could not find rake-10.4.2 in any of the sources
Run bundle install to install missing gems.

My bundle has that rake gem but for ruby v2.0.x instead of v2.1.x which the chefdk uses. I'm guessing that running from within a bundle is messing up the bundle environment that the chef executable uses, but I've not the knowlege of how this works to get it working.
Is there any way around this (other than use thor from the system ruby, which would be the fallback, but not ideal)? Will dropping certain environment variables related to bundler when shelling out fix the problem?

Aside from the above suggestions, run env and look for any GEM_* variables. rvm in particular works by setting these, if you’re an rvm user, there is a plugin for it to use ChefDK (but I can’t recall the specifics, sorry).

--
Yoshi Spendiff
Ops Engineer
Indochino
Mobile: +1 778 952 2025
Email: yoshi.spendiff@indochino.com (mailto:yoshi.spendiff@indochino.com)

--
Daniel DeLeo

4

Hi,

This worked for me:

Bundler.with_clean_environment do
chef -v
end

It was definitely environment variables breaking chef here. Running env
inside and outside the block showed the following variables removed:
BUNDLE_GEMFILE
GEM_PATH
GEM_HOME
BUNDLE_BIN
RUBYOPT
RUBYLIB

and a different PATH.

I didn't know the Bundler class has useful methods like that!

Cheers for the help

Regards,
Yoshi

On Wed, Jul 1, 2015 at 7:52 AM, Daniel DeLeo dan@kallistec.com wrote:

On Tuesday, June 30, 2015 at 3:40 PM, Yoshi Spendiff wrote:

Hi,

I'm writing a couple of thor jobs to configure a workstation for Chef
development. One of the things I want to do is to install a few gems into
the chefdk installed on the system if they don't exist. I thought it would
be simple enough to just shell out and run the chef command to install
these gems (i.e. chef gem install kitchen-ec2 but calling the chef
command, or indeed the ChefDK ruby from within my bundle results in an
error stating that a gem can't be found.

So if I run bin/thor myorg:check (where bin/thor is a binstub in my
bundle) and in the thor jobs there's any command that calls the chef
executable, e.g.
chef -v then I get the following:

I’ve noticed recently that when running any gem commands inside of a
bundler’d ruby process, you need to wrap everything in

Bundler.with_clean_env do; your_gem_related_code; end;

This used to work fine without the clean env thing, so I think there is a
recent change to either bundler or rubygems that caused this.

/Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/spec_set.rb:92:in
`block in materialize': Could not find rake-10.4.2 in any of the sources
(Bundler::GemNotFound)

from
/Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/spec_set.rb:85:in
map!' from /Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/spec_set.rb:85:in materialize'
from
/Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/definition.rb:139:in
specs' from /Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/definition.rb:184:in specs_for'
from
/Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/definition.rb:173:in
requested_specs' from /Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/environment.rb:18:in requested_specs'
from
/Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/runtime.rb:13:in
setup' from /Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler.rb:127:in setup'
from
/Library/Ruby/Gems/2.0.0/gems/bundler-1.10.3/lib/bundler/setup.rb:18:in
<top (required)>' from /opt/chefdk/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:54:in require'
from
/opt/chefdk/embedded/lib/ruby/site_ruby/2.1.0/rubygems/core_ext/kernel_require.rb:54:in
`require'

You’re definitely crossing the streams here. Bundler is installed to the
system ruby, and you seem to have your environment variables set in a way
that makes ruby load stuff from there. As Brandon mentions, chef shell-init should fix this.

or system('chef -v') then I get the following error:
Could not find rake-10.4.2 in any of the sources
Run bundle install to install missing gems.

My bundle has that rake gem but for ruby v2.0.x instead of v2.1.x which
the chefdk uses. I'm guessing that running from within a bundle is messing
up the bundle environment that the chef executable uses, but I've not the
knowlege of how this works to get it working.
Is there any way around this (other than use thor from the system ruby,
which would be the fallback, but not ideal)? Will dropping certain
environment variables related to bundler when shelling out fix the problem?

Aside from the above suggestions, run env and look for any GEM_*
variables. rvm in particular works by setting these, if you’re an rvm user,
there is a plugin for it to use ChefDK (but I can’t recall the specifics,
sorry).

--
Yoshi Spendiff
Ops Engineer
Indochino
Mobile: +1 778 952 2025
Email: yoshi.spendiff@indochino.com (mailto:yoshi.spendiff@indochino.com
)

--
Daniel DeLeo

--
Yoshi Spendiff
Ops Engineer
Indochino
Mobile: +1 778 952 2025
Email: yoshi.spendiff@indochino.com