Why does chef-solo load all cookbooks?

Chef Infra (archive)
1

I am spinning up a virtual machine for testing recipies, and
chef-solo failed on a couple of cookbooks that required gems not
installed on the host yet. debug output shows that chef-solo is
loading all the cookbooks. Why? The run list is empty as
indicated.

[Tue, 13 Sep 2011 16:33:09 -0400] INFO: *** Chef 0.10.4 ***
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Building node object for localhost.localdomain
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Extracting run list from JSON attributes provided on command line
[Tue, 13 Sep 2011 16:33:10 -0400] INFO: Setting the run_list to [“role[default]”] from JSON
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Applying attributes from json file
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Platform is centos version 5.6
[Tue, 13 Sep 2011 16:33:10 -0400] INFO: Run List is [role[default]]
[Tue, 13 Sep 2011 16:33:10 -0400] INFO: Run List expands to []
[Tue, 13 Sep 2011 16:33:10 -0400] INFO: Starting Chef Run for localhost.localdomain
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: No chefignore file found at /opt/chef/current/cookbooks/chefignore no files will be ignored
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Loading cookbook libraries’s library file: /opt/chef/current/cookbooks/libraries/libraries/find_first.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Loading cookbook httpd’s library file: /opt/chef/current/cookbooks/httpd/libraries/domains.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Loading cookbook cpan’s providers from /opt/chef/current/cookbooks/cpan/providers/package.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Loaded contents of /opt/chef/current/cookbooks/cpan/providers/package.rb into a provider named cpan_package defined in Chef::Provider::CpanPackage
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook sudo’s attribute file /opt/chef/current/cookbooks/sudo/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Loading Attribute users
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook mysql’s attribute file /opt/chef/current/cookbooks/mysql/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook munin’s attribute file /opt/chef/current/cookbooks/munin/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook nagios’s attribute file /opt/chef/current/cookbooks/nagios/attributes/client.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook hosts’s attribute file /opt/chef/current/cookbooks/hosts/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook timezone’s attribute file /opt/chef/current/cookbooks/timezone/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook yum’s attribute file /opt/chef/current/cookbooks/yum/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook users’s attribute file /opt/chef/current/cookbooks/users/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook ssh’s attribute file /opt/chef/current/cookbooks/ssh/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: I am not loading attribute file users, because I have already seen it.
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook postfix’s attribute file /opt/chef/current/cookbooks/postfix/attributes/aliases.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: I am not loading attribute file users, because I have already seen it.
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook postfix’s attribute file /opt/chef/current/cookbooks/postfix/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook httpd’s attribute file /opt/chef/current/cookbooks/httpd/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Loading from cookbook_path: /opt/chef/current/cookbooks
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Converging node localhost.localdomain
[Tue, 13 Sep 2011 16:33:10 -0400] INFO: Chef Run complete in 0.048032 seconds
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Cleaning the checksum cache
[Tue, 13 Sep 2011 16:33:10 -0400] INFO: Running report handlers
[Tue, 13 Sep 2011 16:33:10 -0400] INFO: Report handlers complete
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Exiting

/Allan

Allan Wind
Life Integrity, LLC
http://lifeintegrity.com

2

It's been a while since I used solo, but if I recall correctly, it's a side-affect of not having a server that pushes only the cookbooks you need because solo using your entire cookbook repo that is checked out locally.

On Sep 13, 2011, at 1:42 PM, Allan Wind wrote:

I am spinning up a virtual machine for testing recipies, and
chef-solo failed on a couple of cookbooks that required gems not
installed on the host yet. debug output shows that chef-solo is
loading all the cookbooks. Why? The run list is empty as
indicated.

[Tue, 13 Sep 2011 16:33:09 -0400] INFO: *** Chef 0.10.4 ***
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Building node object for localhost.localdomain
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Extracting run list from JSON attributes provided on command line
[Tue, 13 Sep 2011 16:33:10 -0400] INFO: Setting the run_list to ["role[default]"] from JSON
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Applying attributes from json file
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Platform is centos version 5.6
[Tue, 13 Sep 2011 16:33:10 -0400] INFO: Run List is [role[default]]
[Tue, 13 Sep 2011 16:33:10 -0400] INFO: Run List expands to
[Tue, 13 Sep 2011 16:33:10 -0400] INFO: Starting Chef Run for localhost.localdomain
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: No chefignore file found at /opt/chef/current/cookbooks/chefignore no files will be ignored
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Loading cookbook libraries's library file: /opt/chef/current/cookbooks/libraries/libraries/find_first.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Loading cookbook httpd's library file: /opt/chef/current/cookbooks/httpd/libraries/domains.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Loading cookbook cpan's providers from /opt/chef/current/cookbooks/cpan/providers/package.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Loaded contents of /opt/chef/current/cookbooks/cpan/providers/package.rb into a provider named cpan_package defined in Chef::Provider::CpanPackage
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook sudo's attribute file /opt/chef/current/cookbooks/sudo/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Loading Attribute users
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook mysql's attribute file /opt/chef/current/cookbooks/mysql/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook munin's attribute file /opt/chef/current/cookbooks/munin/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook nagios's attribute file /opt/chef/current/cookbooks/nagios/attributes/client.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook hosts's attribute file /opt/chef/current/cookbooks/hosts/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook timezone's attribute file /opt/chef/current/cookbooks/timezone/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook yum's attribute file /opt/chef/current/cookbooks/yum/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook users's attribute file /opt/chef/current/cookbooks/users/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook ssh's attribute file /opt/chef/current/cookbooks/ssh/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: I am not loading attribute file users, because I have already seen it.
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook postfix's attribute file /opt/chef/current/cookbooks/postfix/attributes/aliases.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: I am not loading attribute file users, because I have already seen it.
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook postfix's attribute file /opt/chef/current/cookbooks/postfix/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook httpd's attribute file /opt/chef/current/cookbooks/httpd/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Loading from cookbook_path: /opt/chef/current/cookbooks
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Converging node localhost.localdomain
[Tue, 13 Sep 2011 16:33:10 -0400] INFO: Chef Run complete in 0.048032 seconds
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Cleaning the checksum cache
[Tue, 13 Sep 2011 16:33:10 -0400] INFO: Running report handlers
[Tue, 13 Sep 2011 16:33:10 -0400] INFO: Report handlers complete
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Exiting

/Allan

Allan Wind
Life Integrity, LLC
http://lifeintegrity.com

3

This is normal. The way Chef runs work is described here:

http://wiki.opscode.com/display/chef/Anatomy+of+a+Chef+Run

Skip the "Synchronize Cookbooks" step for Solo.

Chef will always load the Libraries, Resources, Providers and
Attributes from every cookbook in the cookbook_path (in solo, that
means every cookbook you distribute.) The reason is that you can:

A) Manipulate the run list from within an attribute file
B) Use libraries to make all sorts of early stage decisions
C) Have cookbooks that do nothing but load libraries, resources, or
providers that get consumed by other cookbooks

With a Chef Server, the dependency resolver will handle getting the
set of cookbooks needed to each client - with solo, this is on you.

Best,
Adam

On Tue, Sep 13, 2011 at 1:42 PM, Allan Wind
allan_wind@lifeintegrity.com wrote:

I am spinning up a virtual machine for testing recipies, and
chef-solo failed on a couple of cookbooks that required gems not
installed on the host yet. debug output shows that chef-solo is
loading all the cookbooks. Why? The run list is empty as
indicated.

[Tue, 13 Sep 2011 16:33:09 -0400] INFO: *** Chef 0.10.4 ***
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Building node object for localhost.localdomain
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Extracting run list from JSON attributes provided on command line
[Tue, 13 Sep 2011 16:33:10 -0400] INFO: Setting the run_list to ["role[default]"] from JSON
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Applying attributes from json file
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Platform is centos version 5.6
[Tue, 13 Sep 2011 16:33:10 -0400] INFO: Run List is [role[default]]
[Tue, 13 Sep 2011 16:33:10 -0400] INFO: Run List expands to
[Tue, 13 Sep 2011 16:33:10 -0400] INFO: Starting Chef Run for localhost.localdomain
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: No chefignore file found at /opt/chef/current/cookbooks/chefignore no files will be ignored
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Loading cookbook libraries's library file: /opt/chef/current/cookbooks/libraries/libraries/find_first.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Loading cookbook httpd's library file: /opt/chef/current/cookbooks/httpd/libraries/domains.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Loading cookbook cpan's providers from /opt/chef/current/cookbooks/cpan/providers/package.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Loaded contents of /opt/chef/current/cookbooks/cpan/providers/package.rb into a provider named cpan_package defined in Chef::Provider::CpanPackage
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook sudo's attribute file /opt/chef/current/cookbooks/sudo/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Loading Attribute users
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook mysql's attribute file /opt/chef/current/cookbooks/mysql/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook munin's attribute file /opt/chef/current/cookbooks/munin/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook nagios's attribute file /opt/chef/current/cookbooks/nagios/attributes/client.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook hosts's attribute file /opt/chef/current/cookbooks/hosts/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook timezone's attribute file /opt/chef/current/cookbooks/timezone/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook yum's attribute file /opt/chef/current/cookbooks/yum/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook users's attribute file /opt/chef/current/cookbooks/users/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook ssh's attribute file /opt/chef/current/cookbooks/ssh/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: I am not loading attribute file users, because I have already seen it.
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook postfix's attribute file /opt/chef/current/cookbooks/postfix/attributes/aliases.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: I am not loading attribute file users, because I have already seen it.
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook postfix's attribute file /opt/chef/current/cookbooks/postfix/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Node localhost.localdomain loading cookbook httpd's attribute file /opt/chef/current/cookbooks/httpd/attributes/default.rb
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Loading from cookbook_path: /opt/chef/current/cookbooks
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Converging node localhost.localdomain
[Tue, 13 Sep 2011 16:33:10 -0400] INFO: Chef Run complete in 0.048032 seconds
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Cleaning the checksum cache
[Tue, 13 Sep 2011 16:33:10 -0400] INFO: Running report handlers
[Tue, 13 Sep 2011 16:33:10 -0400] INFO: Report handlers complete
[Tue, 13 Sep 2011 16:33:10 -0400] DEBUG: Exiting

/Allan

Allan Wind
Life Integrity, LLC
http://lifeintegrity.com

--
Opscode, Inc.
Adam Jacob, Chief Product Officer
T: (206) 619-7151 E: adam@opscode.com

4

On 2011-09-13 13:54:31, Adam Jacob wrote:

This is normal. The way Chef runs work is described here:

Thanks for the quick answer. And I can stuff my require in a
ruby_block to only have it execute at run.

/Allan

Allan Wind
Life Integrity, LLC
http://lifeintegrity.com