Wrapper cookbooks


#1

I’m having issues with wrapper cookbooks. The setup is pretty complex, so
I’m not even quite sure how to describe it here in a way that makes sense.

Generally speaking though, I have a wrapper cookbook that wraps an
application cookbook. That application cookbook includes other cookbooks.
Something along the way, a location cookbook is included that calculates
and sets some attributes.

Everything up to and including the application cookbook tests fine with
vagrant. However, when I try and wrap that, it fails to find attributes
previously set in the location cookbook. All I should need in the
application wrapper is a single line for the application cookbook in the
metadata, the Berksfile and and include_recipe, right?

The application wrapper cookbook should not need to reference anything
except the cookbook it is wrapping. If I put an include_recipe "location"
in there, it works. I don’t understand this. I’m also very confused about
the fact that if I don’t mention the location cookbook in the metadata or
the Berksfile, a vagrant up still shows it as grabbing it from git. Not
sure if that’s from a dependancy, or directly.

Help appreciated.

Doug.


#2

a side note, maybe you know this already, but include_recipe runs a recipe and it should not be confused with dependencies. they are related, but quite different in behavior.

regarding the original question, I guess I’m not clear on what you mean by ‘wrapping’ a cookbook. I am assuming you mean ‘resource’ when you say ‘cookbook’ as I can’t think of a reason to adjust everything within a cookbook. I apologize if this is erroneous.

Our squad implements layers of abstraction all the time on top of 3rd party cookbook resources as well as core chef resources. It allows us to adjust the behavior of the resources for specific, custom requirements without having to change the base resource that is likely acceptable 99% of the time.

However, what we do would be more accurately described as a proxy or composition pattern vs inheritance as the term ‘wrapping’ could suggest. Assuming adherence to standard OOP best practices, extending objects within chef land usually seems to lead to increased maintenance overhead and less flexibility over the long term, whereas defining accessor type resources that use the ‘wrapped’ resources and introduce controlled changes to behavior or more commonly, to abstract additional conventions established by your project and outside the scope of the original objects consideration, leads to higher agility and flexibility when those conventions change and/or a new underlying resource becomes desirable.

We strictly avoid the use of include_recipe unless exhaustively justified. Cookbook dependencies are necessary, but should only be established via a cookbook’s metadata file. If we proxy (wrap) a resource we try to wrap the entirety of the resource’s interface, even if it just passes along the call to the proxied resource. this alleviates any need to use both, and allows us to implement additional log feedback and/or adjusted behavior.

The application cookbook is likely the hardest one to deal with as it was intended as what one might know as an ‘abstract’ class/object/resource; intended to require extending to be of any actual use. Normally one doesn’t see this resource being used directly but rather used via a more specific application derivative such as application_python or application_java. So some quick research on how these objects are written would likely be the best source for how to write a new wrapper for the application, assuming you can’t find an extension that more closely suits your requirements.

I hope this helps you sort it out some
Greg

On Mar 26, 2014, at 6:10 PM, Douglas Garstang doug.garstang@gmail.com wrote:

I’m having issues with wrapper cookbooks. The setup is pretty complex, so I’m not even quite sure how to describe it here in a way that makes sense.

Generally speaking though, I have a wrapper cookbook that wraps an application cookbook. That application cookbook includes other cookbooks. Something along the way, a location cookbook is included that calculates and sets some attributes.

Everything up to and including the application cookbook tests fine with vagrant. However, when I try and wrap that, it fails to find attributes previously set in the location cookbook. All I should need in the application wrapper is a single line for the application cookbook in the metadata, the Berksfile and and include_recipe, right?

The application wrapper cookbook should not need to reference anything except the cookbook it is wrapping. If I put an include_recipe “location” in there, it works. I don’t understand this. I’m also very confused about the fact that if I don’t mention the location cookbook in the metadata or the Berksfile, a vagrant up still shows it as grabbing it from git. Not sure if that’s from a dependancy, or directly.

Help appreciated.

Doug.


#3

Gregory,

I suppose the only way to make this clearer is to paste the relevant
portions. See below. For each cookbook I have showed the relevant portions
of metadata, Berksfile and recipes/default.rb from the top of the 'stack’
down. Starting from the bottom of the stack, each cookbook independently
tests ok with vagrant. It’s not until reaching the top of the stack
(bc-role-downloader) that a failure occurs.

The bottom line is that the top level cookbook, bc-role-downloader should
not need to know anything about the internal details of the cookbooks it
sits on top of. All bc-role-downloader should have to do is reference
bc-app-downloader (which independently tests ok), and bc-role-base (which
also independently tests ok), and that’s it. However, it fails. In the
bc-apt cookbook, i check to see if the bc-apt attribute has been set, and
if not, abort. The bc-apt attribute is set in the bc-location cookbook.
However, as I said, this all tests fine in all these cookbooks right up to
before the top level bc-role-downloader one.

It should also be noted that if I add an include_recipe ‘bc-location’ to
the recipes/default.rb file in the bc-role-downloader cookbook, the error
does not occur. How that can happen without putting a corresponding entry
in metadata.rb and/or the Berksfile, I cannot fathom.

bc-role-downloader:

metadata:
depends "bc-role-base"
depends “bc-foocom-downloader”

Berksfile:
cookbook ‘bc-role-base’, git: "git@git.co.foocom.com:chef/bc-role-base.git"
cookbook ‘bc-foocom-downloader’, git: "git@git.co.foocom.com:
chef/bc-foocom-downloader.git"

recipes/default.rb
include_recipe “bc-foocom-downloader”

bc-foocom-downloader:

metadata:
depends 'bc-location’
depends ‘bc-apt’

Berksfile:
cookbook ‘bc-location’, git: "git@git.co.foocom.com:chef/bc-location.git"
cookbook ‘bc-apt’, git: "git@git.co.foocom.com:chef/bc-apt.git"

recipes/default.rb
include_recipe “bc-apt”

bc-role-base:

metadata:
depends "bc-location"
depends "bc-apt"
depends "bc-ohai"
depends "bc-chef-statuslog"
depends "bc-yali-client"
depends "bc-users"
depends "bc-rsyslog"
depends "chef-solo-search"
depends "bc-resolv"
depends "postfix"
depends "sysctl"
depends "bc-motd"
depends "sudo"
depends "ntp"
depends "bc-nrpe"
depends "bc-statsd"
depends “bc-chef-client”

Berksfile:

kbook “bc-location”, git: "git@git.co.foocom.com:chef/bc-location.git"
cookbook “bc-apt”, git: "git@git.co.foocom.com:chef/bc-apt.git"
cookbook “bc-ohai”, git: "git@git.co.foocom.com:chef/bc-ohai.git"
cookbook “bc-chef-statuslog”, git: "git@git.co.foocom.com:
chef/bc-chef-statuslog.git"
cookbook “bc-yali-client”, git: "git@git.co.foocom.com:
chef/bc-yali-client.git"
cookbook “bc-users”, git: "git@git.co.foocom.com:chef/bc-users.git"
cookbook “bc-rsyslog”, git: "git@git.co.foocom.com:chef/bc-rsyslog.git"
cookbook “bc-resolv”, git: "git@git.co.foocom.com:chef/bc-resolv.git"
cookbook “bc-motd”, git: "git@git.co.foocom.com:chef/bc-motd.git"
cookbook “bc-nrpe”, git: "git@git.co.foocom.com:chef/bc-nrpe.git"
cookbook “bc-statsd”, git: "git@git.co.foocom.com:chef/bc-statsd.git"
cookbook “bc-chef-client”, git: "git@git.co.foocom.com:
chef/bc-chef-client.git"

recipes/default.rb:
include_recipe "bc-location"
include_recipe "bc-apt"
include_recipe "bc-ohai"
include_recipe "bc-chef-statuslog"
include_recipe "bc-yali-client"
include_recipe "bc-users"
include_recipe "bc-rsyslog"
include_recipe "chef-solo-search"
include_recipe "bc-resolv"
include_recipe "postfix"
include_recipe "sysctl"
include_recipe "bc-motd"
include_recipe "sudo"
include_recipe "ntp"
include_recipe "bc-nrpe"
include_recipe "bc-statsd"
include_recipe “bc-chef-client”

bc-location:

metadata:
depends ‘bc-dnsmasq’

Berksfile:
(nothing)

recipes/default.rb
(no includes)

bc-apt:

metadata:
depends "bc-location"
depends “apt”, “2.1.1”

Berksfile:
cookbook ‘bc-location’, git: ‘git@git.co.foocom.com:chef/bc-location.git’

recipes/default.rb
(no includes)

ERROR:

[2014-03-27T16:38:18+00:00] INFO: Forking chef instance to converge…
[2014-03-27T16:38:18+00:00] INFO: *** Chef 11.6.0 ***
[2014-03-27T16:38:18+00:00] INFO: Setting the run_list to
[“recipe[bc-role-downloader]”] from JSON
[2014-03-27T16:38:18+00:00] INFO: Run List is [recipe[bc-role-downloader]]
[2014-03-27T16:38:18+00:00] INFO: Run List expands to [bc-role-downloader]
[2014-03-27T16:38:18+00:00] INFO: Starting Chef Run for test-co.foocom.com
[2014-03-27T16:38:18+00:00] INFO: Running start handlers
[2014-03-27T16:38:18+00:00] INFO: Start handlers complete.
+++++ HERE I AM

================================================================================
Recipe Compile Error in
/tmp/vagrant-chef-1/chef-solo-1/cookbooks/bc-role-downloader/recipes/default.rb

RuntimeError

The bc-apt attribute is missing. Aborting.

Cookbook Trace:

/tmp/vagrant-chef-1/chef-solo-1/cookbooks/bc-apt/recipes/default.rb:4:in
`from_file’

/tmp/vagrant-chef-1/chef-solo-1/cookbooks/bc-foocom-downloader/recipes/default.rb:9:in
`from_file’

/tmp/vagrant-chef-1/chef-solo-1/cookbooks/bc-role-downloader/recipes/default.rb:15:in
`from_file’

Relevant File Content:

/tmp/vagrant-chef-1/chef-solo-1/cookbooks/bc-apt/recipes/default.rb:

1: log "@@@ This is cookbook ‘#{cookbook_name}’, version
’#{run_context.cookbook_collection[cookbook_name].version}’, environment
’#{node.chef_environment}’"
2:
3: if not node.has_key? ‘bc-apt’ then
4>> raise "The bc-apt attribute is missing. Aborting.
#{node[‘bc-location’]}"
5: #raise "The bc-apt attribute is missing for location
#{node[‘bc-location’][‘location’]}. Aborting."
6: end
7:
8: template “/etc/apt/apt.conf.d/02allow_unsigned_pkgs” do
9: source "etc/apt/apt.conf.d/02allow_unsigned_pkgs.erb"
10: owner "root"
11: group "root"
12: end
13:

[2014-03-27T16:38:23+00:00] ERROR: Running exception handlers
[2014-03-27T16:38:23+00:00] ERROR: Exception handlers complete
[2014-03-27T16:38:23+00:00] FATAL: Stacktrace dumped to
/var/chef/cache/chef-stacktrace.out
[2014-03-27T16:38:23+00:00] FATAL: Chef::Exceptions::ChildConvergeError:
Chef run process exited unsuccessfully (exit code 1)
Chef never successfully completed! Any errors should be visible in the
output above. Please fix your recipes so that they properly complete.

On Thu, Mar 27, 2014 at 8:18 AM, Gregory Patmore
gregorypatmore@gmail.comwrote:

a side note, maybe you know this already, but include_recipe runs a recipe
and it should not be confused with dependencies. they are related, but
quite different in behavior.

regarding the original question, I guess I’m not clear on what you mean by
’wrapping’ a cookbook. I am assuming you mean ‘resource’ when you say
’cookbook’ as I can’t think of a reason to adjust everything within a
cookbook. I apologize if this is erroneous.

Our squad implements layers of abstraction all the time on top of 3rd
party cookbook resources as well as core chef resources. It allows us to
adjust the behavior of the resources for specific, custom requirements
without having to change the base resource that is likely acceptable 99% of
the time.

However, what we do would be more accurately described as a proxy or
composition pattern vs inheritance as the term ‘wrapping’ could suggest.
Assuming adherence to standard OOP best practices, extending objects within
chef land usually seems to lead to increased maintenance overhead and less
flexibility over the long term, whereas defining accessor type resources
that use the ‘wrapped’ resources and introduce controlled changes to
behavior or more commonly, to abstract additional conventions established
by your project and outside the scope of the original objects
consideration, leads to higher agility and flexibility when those
conventions change and/or a new underlying resource becomes desirable.

We strictly avoid the use of include_recipe unless exhaustively justified.
Cookbook dependencies are necessary, but should only be established via a
cookbook’s metadata file. If we proxy (wrap) a resource we try to wrap the
entirety of the resource’s interface, even if it just passes along the call
to the proxied resource. this alleviates any need to use both, and allows
us to implement additional log feedback and/or adjusted behavior.

The application cookbook is likely the hardest one to deal with as it was
intended as what one might know as an ‘abstract’ class/object/resource;
intended to require extending to be of any actual use. Normally one doesn’t
see this resource being used directly but rather used via a more specific
application derivative such as application_python or application_java. So
some quick research on how these objects are written would likely be the
best source for how to write a new wrapper for the application, assuming
you can’t find an extension that more closely suits your requirements.

I hope this helps you sort it out some
Greg

On Mar 26, 2014, at 6:10 PM, Douglas Garstang doug.garstang@gmail.com
wrote:

I’m having issues with wrapper cookbooks. The setup is pretty complex,
so I’m not even quite sure how to describe it here in a way that makes
sense.

Generally speaking though, I have a wrapper cookbook that wraps an
application cookbook. That application cookbook includes other cookbooks.
Something along the way, a location cookbook is included that calculates
and sets some attributes.

Everything up to and including the application cookbook tests fine with
vagrant. However, when I try and wrap that, it fails to find attributes
previously set in the location cookbook. All I should need in the
application wrapper is a single line for the application cookbook in the
metadata, the Berksfile and and include_recipe, right?

The application wrapper cookbook should not need to reference anything
except the cookbook it is wrapping. If I put an include_recipe "location"
in there, it works. I don’t understand this. I’m also very confused about
the fact that if I don’t mention the location cookbook in the metadata or
the Berksfile, a vagrant up still shows it as grabbing it from git. Not
sure if that’s from a dependancy, or directly.

Help appreciated.

Doug.


Regards,

Douglas Garstang
http://www.linkedin.com/in/garstang
Email: doug.garstang@gmail.com
Cell: +1-805-340-5627