Organizing cookbooks, and individual projects

I use chef using chef-solo to bootstrap my ec2 instances etc.

Now currently I have a project where I have all my recipes in a folder:

/myapp/infra/chef/cookbooks

I want to keep doing this for other projects, but obviously don’t want to
be duplicating my work.

Since recipes are suppose to work by themeselves (ignoring dependent
recipes), how would you recommend I organize things?

I was thinking of this:

  1. Have a git repo that has all my cookbook and recipes
  2. for my individual projects, I somehow link to this master git repo, and
    then have whatever is specific to this project like:

chef.json
/config/solo.rb
**/cookbooks <-- git submodule of my main cookbooks
/roles/base.rb etc.

Does this make sense?

I think the key point to make sure things don’t break is for me to link to
the master /cookbooks git repo using a specific changeset/branch to ensure
things work even when I update the master cookbooks repo.

I’m guessing this is an obvious setup, but I just haven’t though things
through to set things up for multiple projects.

I’m not a big company, so I don’t need anything more eleborate than this
assuming it makes sense.

Comments?
Is a git sub-module the way to go, I haven’t done this before so not sure
how to set this up, do you guys use rake?

We moved to cookbook per repo and use berkshelf to describe the cookbooks and versions a project requires. Initially we used sub modules but that ended up being not exactly the right solution when cookbooks are being updated often across a larger team.

On Dec 3, 2012, at 3:49 AM, S Ahmed sahmed1020@gmail.com wrote:

I use chef using chef-solo to bootstrap my ec2 instances etc.

Now currently I have a project where I have all my recipes in a folder:

/myapp/infra/chef/cookbooks

I want to keep doing this for other projects, but obviously don't want to be duplicating my work.

Since recipes are suppose to work by themeselves (ignoring dependent recipes), how would you recommend I organize things?

I was thinking of this:

  1. Have a git repo that has all my cookbook and recipes
  2. for my individual projects, I somehow link to this master git repo, and then have whatever is specific to this project like:

chef.json
/config/solo.rb
**/cookbooks <-- git submodule of my main cookbooks
/roles/base.rb etc.

Does this make sense?

I think the key point to make sure things don't break is for me to link to the master /cookbooks git repo using a specific changeset/branch to ensure things work even when I update the master cookbooks repo.

I'm guessing this is an obvious setup, but I just haven't though things through to set things up for multiple projects.

I'm not a big company, so I don't need anything more eleborate than this assuming it makes sense.

Comments?
Is a git sub-module the way to go, I haven't done this before so not sure how to set this up, do you guys use rake?

On 12/02/2012 07:10 PM, Jesse Nelson wrote:

We moved to cookbook per repo and use berkshelf to describe the cookbooks and versions a project requires. Initially we used sub modules but that ended up being not exactly the right solution when cookbooks are being updated often across a larger team.

Yeah, we currently use git submodules for all cookbooks that we need to
make changes to and put each of the cookbooks in a separate git
repository, but I agree that it can be cumbersome to manage SHA1s of
individual cookbook in the git submodules (and remembering to update the
.gitmodules stuff when you make changes!).

That said, though, I prefer to use Git for source-control and
version-related things, and not "yet another dependency-tracking library
tool". It's not that I don't see value in things like Spiceweasel,
Librarian or Berkshelf (the last of which I have not yet tried -- need
to find time to research and test it out!).

I just think that using Git submodules is a good way of separating the
source-control/versioning aspects of the configuration management from
the configuration itself. As for dependencies between things, packages
for OS distributions were developed for this exact thing, and I don't
think it's useful to have another tool tracking dependencies when OS
packages exist to provide (and have a long history of supporting)
dependency tracking.

Best,
-jay