# Cookbook reusability

**URL:** <https://discourse.chef.io/t/cookbook-reusability/2627>\
**Category:** Chef Infra (archive)\
**Created:** [March 5, 2012, 2:54pm UTC](https://discourse.chef.io/t/cookbook-reusability/2627 "2012-03-05T14:54:58Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Opscode](https://avatars.discourse-cdn.com/v4/letter/o/839c29/32.png) [@Opscode](https://discourse.chef.io/u/Opscode)\
**Post date:** [March 5, 2012, 2:54pm UTC](https://discourse.chef.io/t/cookbook-reusability/2627/1 "2012-03-05T14:54:58Z")

</div>

Hi,

after adapting more than enough community cookbooks to our local setup,  
I published a blog entry yesterday to describe my idea of improving the  
status quo wrt/ reusable cookbooks:

[http://blog.nistu.de/2012/03/04/reusability-in-configuration-management-systems/](http://blog.nistu.de/2012/03/04/reusability-in-configuration-management-systems/)

I would love to discuss these ideas and see what people think about it.

cheers,

Nikolay

–  
“It’s all part of my Can’t-Do approach to life.” Wally

---

<div class="post-metadata">

**Author:** ![Wes\_Morgan](https://avatars.discourse-cdn.com/v4/letter/w/3be4f8/32.png) [@Wes\_Morgan](https://discourse.chef.io/u/Wes_Morgan)\
**Post date:** [March 5, 2012, 3:25pm UTC](https://discourse.chef.io/t/cookbook-reusability/2627/2 "2012-03-05T15:25:25Z")

</div>

On Mar 5, 2012, at 9:54 AM, Nikolay Sturm wrote:

> Hi,
> 
> after adapting more than enough community cookbooks to our local setup,  
> I published a blog entry yesterday to describe my idea of improving the  
> status quo wrt/ reusable cookbooks:
> 
> [http://blog.nistu.de/2012/03/04/reusability-in-configuration-management-systems/](http://blog.nistu.de/2012/03/04/reusability-in-configuration-management-systems/)
> 
> I would love to discuss these ideas and see what people think about it.
> 
> cheers,
> 
> Nikolay
> 
> --  
> "It's all part of my Can't-Do approach to life." Wally

Here here! I couldn't agree more with your suggestions.

In addition, I think we need better tools to manage, share, and collaborate on our cookbooks. So, for example, if I write a cookbook that's tightly-coupled to my infrastructure and you come along and improve it along the lines you describe in your post, there should be a way for you to put your version forward to the community and then for everyone to collectively communicate that yours is the new best version and submit their own improvements, etc.

So this could be a layer on top of GitHub (and we'd need to make the knife-GitHub integration more official), or it could be a vastly-improved [community.opscode.com](http://community.opscode.com), or something else entirely. But right now there is just too much friction between "I've made this cookbook more general" and "this is now the primary version of this cookbook."

---

<div class="post-metadata">

**Author:** ![John\_E\_Vincent\_lusis](https://avatars.discourse-cdn.com/v4/letter/j/c89c15/32.png) [@John\_E\_Vincent\_lusis](https://discourse.chef.io/u/John_E_Vincent_lusis)\
**Post date:** [March 5, 2012, 3:40pm UTC](https://discourse.chef.io/t/cookbook-reusability/2627/3 "2012-03-05T15:40:46Z")

</div>

I made this comment on the blog post and haven't had time to respond  
to its reponse.

"cookbooks by default have to be opinionated"

I stand by that for this reason (with a few caveats):

- You must have defaults. You can't lay bits out on disk without some  
opinion of where they need to go
- Some things don't make sense as configurable (I'm not going to  
deploy a php app to rails and I'm I might HAVE to use a specific  
version of php)
- In cases where cookbooks have interdependencies, you must be able to  
express that (take the opscode users cookbook and nagios cookbook -  
both want an explcitly named admin group)

Having said that, yes, folks who publish cookbooks should consider  
making as much as possible as attributes. The reason here is obvious.  
It's much easier to override attributes than to override an entire  
recipe. As a counter though, if the attitude is that folks have to  
have a fully configurable cross-platform cookbook before they can  
release it, then I don't want that. I'd rather have an unnecessarily  
opinionated cookbook than no cookbook at all. In fact, I'm working  
with someone at the new company who has very little chef experience  
and my advice was to hard code everything. We can refactor into  
attributes later.

Here's an example of a cookbook we used at my last gig where I made  
some things overly configurable even though we weren't ready for it  
yet:

default[:portal][:container] = "jetty"  
default[:portal][:base\_dir] = "/usr/local/portal/"  
default[:portal][:user] = "portal"  
default[:portal][:group] = "portal"  
default[:portal][:context\_path] = "/portal"  
default[:portal][:warfile] = "portal.war"

We were not even remotely ready to run on anything other than jetty  
and the context root HAD to be portal, however I went ahead and made  
them attributes just in case we ever got to the point where we weren't  
tied so heavily to jetty.

I think cookbook authors could do at least the following as attributes  
since these areas are probably the most common:

user  
group  
path  
(maybe) init script  
use 'conf.d' setups where available instead of owning a file wholesale.

The main reason for not doing initscript is that it litters the  
codebase with ugly-ass conditionals which actually HURT readability.  
I'm considering a proposal to make init scripts more intelligent like  
the runit LWRP. Yes, you can sort of do it with file specificity in  
the templates directory but upstart files don't go in the same places  
as SYSV init scripts so you've still got some conditionals.

Also, the post is awesome and very much appreciated. I just think that  
in a few areas it goes to far to the extreme. With proper attributes,  
there's no reason you need a distinct recipe JUST for group creation  
(and at worst case you've created an ordering issue that might not be  
readily apparent.

On Mon, Mar 5, 2012 at 10:25 AM, Wes Morgan [cap10morgan@gmail.com](mailto:cap10morgan@gmail.com) wrote:

> On Mar 5, 2012, at 9:54 AM, Nikolay Sturm wrote:
> 
> > Hi,
> > 
> > after adapting more than enough community cookbooks to our local setup,  
> > I published a blog entry yesterday to describe my idea of improving the  
> > status quo wrt/ reusable cookbooks:
> > 
> > [http://blog.nistu.de/2012/03/04/reusability-in-configuration-management-systems/](http://blog.nistu.de/2012/03/04/reusability-in-configuration-management-systems/)
> > 
> > I would love to discuss these ideas and see what people think about it.
> > 
> > cheers,
> > 
> > Nikolay
> > 
> > --  
> > "It's all part of my Can't-Do approach to life." Wally
> 
> Here here! I couldn't agree more with your suggestions.
> 
> In addition, I think we need better tools to manage, share, and collaborate on our cookbooks. So, for example, if I write a cookbook that's tightly-coupled to my infrastructure and you come along and improve it along the lines you describe in your post, there should be a way for you to put your version forward to the community and then for everyone to collectively communicate that yours is the new best version and submit their own improvements, etc.
> 
> So this could be a layer on top of GitHub (and we'd need to make the knife-GitHub integration more official), or it could be a vastly-improved [community.opscode.com](http://community.opscode.com), or something else entirely. But right now there is just too much friction between "I've made this cookbook more general" and "this is now the primary version of this cookbook."

---

<div class="post-metadata">

**Author:** ![Brad\_Knowles](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/brad_knowles/32/716_2.png) [@Brad\_Knowles](https://discourse.chef.io/u/Brad_Knowles)\
**Post date:** [March 5, 2012, 3:44pm UTC](https://discourse.chef.io/t/cookbook-reusability/2627/4 "2012-03-05T15:44:49Z")

</div>

On Mar 5, 2012, at 9:25 AM, Wes Morgan wrote:

> So this could be a layer on top of GitHub (and we'd need to make the knife-GitHub integration more official), or it could be a vastly-improved [community.opscode.com](http://community.opscode.com), or something else entirely. But right now there is just too much friction between "I've made this cookbook more general" and "this is now the primary version of this cookbook."

As Opscode splits their cookbooks into individual repos, they will be able to more easily do this. So long as everyone does a proper fork of the project on github, the branches should all remain visible and accessible, and then it's up to the folks who run the repo as to whose changes they actually choose to incorporate.

I think the Cookbooks project on github is effectively already doing this kind of thing for the various repos they track.

IMO, the real trick is getting everyone to properly fork and track their projects, and then getting people who are willing and able to apply the generalizations to the site-specific cookbooks that are released.

I like Nikolay's idea, but I'm not a real programmer and I don't apply the methods he describes. It would be up to someone else to come along and take whatever I've rigged together with spit, bailing wire, chewing gum, and a whole lot of prayer, and then turn that into something more general and useful.

For example, when people hear that I've written a cookbook for the Zerigo dynamic DNS service, they might be interested to see what I've got -- that is, until they actually do see it, and realize just how horrible it really is. But, it actually works for us, which is something that I can't say for the other Zerigo cookbook we found. And what I really care about is that something works, not how pretty it is.

--  
Brad Knowles [bknowles@ihiji.com](mailto:bknowles@ihiji.com)  
SAGE Level IV, Chef Level 0.0.1

---

<div class="post-metadata">

**Author:** ![Bryan\_Berry](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/bryan_berry/32/213_2.png) [@Bryan\_Berry](https://discourse.chef.io/u/Bryan_Berry)\
**Post date:** [March 6, 2012, 9:59am UTC](https://discourse.chef.io/t/cookbook-reusability/2627/5 "2012-03-06T09:59:09Z")

</div>

I have to disagree with you john, I do think that there can be value into  
moving components as granular as group creation into separate recipes.

There certainly are downsides as you point out:

- less readable
- can hide or create ordering problems

But the upsides are huge

I don't think anyone thinks that your cookbook should start out looking  
like Nikolay's example but it can be the ultimate refactored version.

In my work I often have to model an existing configuration because that is  
what the customer wants, there are lots of arbitrary config details that  
are not relevant to other parts of the cookbook, further i have many  
similar servers w/ minor variations.

Splitting parts of recipes into lots of small individual recipes could make  
cookbooks much more testable.

We're churning out tons of cookbooks w/ tons of overlap. If we don't figure  
out to better reuse code, well, we won't be automating as much as we should  
be 😉

congrats to Nikolay for the great article. It is important data point in a  
larger discussion.

On Mon, Mar 5, 2012 at 4:40 PM, John E. Vincent (lusis) \<lusis.org+  
[chef-list@gmail.com](mailto:chef-list@gmail.com)\> wrote:

> I made this comment on the blog post and haven't had time to respond  
> to its reponse.
> 
> "cookbooks by default have to be opinionated"
> 
> I stand by that for this reason (with a few caveats):
> 
> - You must have defaults. You can't lay bits out on disk without some  
> opinion of where they need to go
> - Some things don't make sense as configurable (I'm not going to  
> deploy a php app to rails and I'm I might HAVE to use a specific  
> version of php)
> - In cases where cookbooks have interdependencies, you must be able to  
> express that (take the opscode users cookbook and nagios cookbook -  
> both want an explcitly named admin group)
> 
> Having said that, yes, folks who publish cookbooks should consider  
> making as much as possible as attributes. The reason here is obvious.  
> It's much easier to override attributes than to override an entire  
> recipe. As a counter though, if the attitude is that folks have to  
> have a fully configurable cross-platform cookbook before they can  
> release it, then I don't want that. I'd rather have an unnecessarily  
> opinionated cookbook than no cookbook at all. In fact, I'm working  
> with someone at the new company who has very little chef experience  
> and my advice was to hard code everything. We can refactor into  
> attributes later.
> 
> Here's an example of a cookbook we used at my last gig where I made  
> some things overly configurable even though we weren't ready for it  
> yet:
> 
> default[:portal][:container] = "jetty"  
> default[:portal][:base\_dir] = "/usr/local/portal/"  
> default[:portal][:user] = "portal"  
> default[:portal][:group] = "portal"  
> default[:portal][:context\_path] = "/portal"  
> default[:portal][:warfile] = "portal.war"
> 
> We were not even remotely ready to run on anything other than jetty  
> and the context root HAD to be portal, however I went ahead and made  
> them attributes just in case we ever got to the point where we weren't  
> tied so heavily to jetty.
> 
> I think cookbook authors could do at least the following as attributes  
> since these areas are probably the most common:
> 
> user  
> group  
> path  
> (maybe) init script  
> use 'conf.d' setups where available instead of owning a file wholesale.
> 
> The main reason for not doing initscript is that it litters the  
> codebase with ugly-ass conditionals which actually HURT readability.  
> I'm considering a proposal to make init scripts more intelligent like  
> the runit LWRP. Yes, you can sort of do it with file specificity in  
> the templates directory but upstart files don't go in the same places  
> as SYSV init scripts so you've still got some conditionals.
> 
> Also, the post is awesome and very much appreciated. I just think that  
> in a few areas it goes to far to the extreme. With proper attributes,  
> there's no reason you need a distinct recipe JUST for group creation  
> (and at worst case you've created an ordering issue that might not be  
> readily apparent.
> 
> On Mon, Mar 5, 2012 at 10:25 AM, Wes Morgan [cap10morgan@gmail.com](mailto:cap10morgan@gmail.com) wrote:
> 
> > On Mar 5, 2012, at 9:54 AM, Nikolay Sturm wrote:
> > 
> > > Hi,
> > > 
> > > after adapting more than enough community cookbooks to our local setup,  
> > > I published a blog entry yesterday to describe my idea of improving the  
> > > status quo wrt/ reusable cookbooks:
> 
> [http://blog.nistu.de/2012/03/04/reusability-in-configuration-management-systems/](http://blog.nistu.de/2012/03/04/reusability-in-configuration-management-systems/)
> 
> > > I would love to discuss these ideas and see what people think about it.
> > > 
> > > cheers,
> > > 
> > > Nikolay
> > > 
> > > --  
> > > "It's all part of my Can't-Do approach to life." Wally
> > 
> > Here here! I couldn't agree more with your suggestions.
> > 
> > In addition, I think we need better tools to manage, share, and  
> > collaborate on our cookbooks. So, for example, if I write a cookbook that's  
> > tightly-coupled to my infrastructure and you come along and improve it  
> > along the lines you describe in your post, there should be a way for you to  
> > put your version forward to the community and then for everyone to  
> > collectively communicate that yours is the new best version and submit  
> > their own improvements, etc.
> > 
> > So this could be a layer on top of GitHub (and we'd need to make the  
> > knife-GitHub integration more official), or it could be a vastly-improved  
> > [community.opscode.com](http://community.opscode.com), or something else entirely. But right now there is  
> > just too much friction between "I've made this cookbook more general" and  
> > "this is now the primary version of this cookbook."

---

<div class="post-metadata">

**Author:** ![John\_E\_Vincent\_lusis](https://avatars.discourse-cdn.com/v4/letter/j/c89c15/32.png) [@John\_E\_Vincent\_lusis](https://discourse.chef.io/u/John_E_Vincent_lusis)\
**Post date:** [March 6, 2012, 4:59pm UTC](https://discourse.chef.io/t/cookbook-reusability/2627/6 "2012-03-06T16:59:01Z")

</div>

On Tue, Mar 6, 2012 at 4:59 AM, Bryan Berry [bryan.berry@gmail.com](mailto:bryan.berry@gmail.com) wrote:

> I have to disagree with you john, I do think that there can be value into  
> moving components as granular as group creation into separate recipes.

Disagreement is not allowed. We need group think consensus

> There certainly are downsides as you point out:
> 
> - less readable
> - can hide or create ordering problems
> 
> But the upsides are huge
> 
> I don't think anyone thinks that your cookbook should start out looking like  
> Nikolay's example but it can be the ultimate refactored version.
> 
> In my work I often have to model an existing configuration because that is  
> what the customer wants, there are lots of arbitrary config details that are  
> not relevant to other parts of the cookbook, further i have many similar  
> servers w/ minor variations.
> 
> Splitting parts of recipes into lots of small individual recipes could make  
> cookbooks much more testable.
> 
> We're churning out tons of cookbooks w/ tons of overlap. If we don't figure  
> out to better reuse code, well, we won't be automating as much as we should  
> be 😉

The problem here is that a code reuse system already exists in Chef -  
attributes. This is the mechanism by which we define  
variable...variables 😉 It is much easier to me to simply be able to  
say "production environment uses X for a variable while development  
uses Y" via environment level attributes than to have to modify a  
recipe (and in this case hunt down which one it the value is being  
used).

The problem here is that going down the route of millions of recipes  
that do one thing is just as bad as overdoing attributes. The only  
REAL variation that chef does not yet address is abstracting out  
certain things that are hard problems (init scripts, package naming  
differences).

> congrats to Nikolay for the great article. It is important data point in a  
> larger discussion.

Indeed. Great post.

---

<div class="post-metadata">

**Author:** ![Opscode](https://avatars.discourse-cdn.com/v4/letter/o/839c29/32.png) [@Opscode](https://discourse.chef.io/u/Opscode)\
**Post date:** [March 6, 2012, 7:53pm UTC](https://discourse.chef.io/t/cookbook-reusability/2627/7 "2012-03-06T19:53:41Z")

</div>

- John E. Vincent (lusis) [2012-03-05]:

> Also, the post is awesome and very much appreciated. I just think that  
> in a few areas it goes to far to the extreme. With proper attributes,  
> there's no reason you need a distinct recipe JUST for group creation  
> (and at worst case you've created an ordering issue that might not be  
> readily apparent.

John, I agree with most of your comments in this thread, I would argue,  
however, that my approach to code reuse (composable recipes) solves a  
different problem than using attributes does.

IMHO attributes are primarily used to vary specific data points: which  
username to use, which IP to bind to, which directory to install to...  
While you can vary these things, you still have to agree to the  
policies(?) set in place by the cookbook author: setup a local user and  
group with the same name, linking executables to /usr/local/bin, ...  
Composable recipes allow you to adapt these policy decisions without  
much hassle.

After reading the responses to my post, I see other problems as possibly  
more urgent for the community at large (comming up with clean cookbooks,  
improving existing cookbooks instead of reinventing the wheel). I myself  
will further explore composable recipes and see if the policies I talk  
about are a real problem, or just a perceived one. ATM I honestly don't  
know.

I would like to thank everyone for their feedback, it was really helpful  
to me.

cheers,

Nikolay