Workflow of Making, Testing and Using a Cookbook?

I’m writing a cookbook that deploys a web app. The cookbook, "my_cookbook,"
does the following:

(1) install Java and Tomcat
(2) copies a remote WAR into Tomcat’s webapps directory.

I’m using test-kitchen to test out this cookbook.

Once this cookbook is properly tested, here’s my understanding as to how to
deploy it using chef-client & chef-server:

(1) set up Chef Server to host “my_cookbook”
–> involves forking opscode/chef-repo
(2) install and set up chef-client on my “node”, i.e. machine to be
provisioned from Chef-Server

Please let me know if I have the right idea as far as deployment of
my_cookbook to the Chef Server & provisioning on the node.

Thanks,
Kevin

On Sun, May 18, 2014 at 11:08 AM, Kevin Meredith
kevin.m.meredith@gmail.com wrote:

I'm writing a cookbook that deploys a web app. The cookbook, "my_cookbook,"
does the following:

(1) install Java and Tomcat
(2) copies a remote WAR into Tomcat's webapps directory.

I'm using test-kitchen to test out this cookbook.

Once this cookbook is properly tested, here's my understanding as to how to
deploy it using chef-client & chef-server:

(1) set up Chef Server to host "my_cookbook"
--> involves forking opscode/chef-repo

Actually it doesn't involve that at all, though it's a good idea to
store your cookbook in source control (in its OWN repository, not as a
subdirectory of a 'chef-repo' repository or using git submodules or
any of that funny business). However, 'knife' doesn't care about that;
you merely need to upload your cookbook to the server.

(2) install and set up chef-client on my "node", i.e. machine to be
provisioned from Chef-Server

Right.

  • Julian

--
[ Julian C. Dunn jdunn@aquezada.com * Sorry, I'm ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]

Julian,

Not to go off on a tangent, but I'm curious about a statement in here. You
mention that the cookbook should not be "using git submodules or any of
that funny business".

What would be the negative reason for putting chef-repo in source control,
and then all cookbooks in their own repos, but putting a submodule of each
cookbook under the cookbooks directory of the chef-repo? I've been working
on a few projects where we intended to do this eventually, and just haven't
done so yet. What issues/problems might I not be realizing? :slight_smile:

On Mon, May 19, 2014 at 9:49 AM, Julian C. Dunn jdunn@aquezada.com wrote:

On Sun, May 18, 2014 at 11:08 AM, Kevin Meredith
kevin.m.meredith@gmail.com wrote:

I'm writing a cookbook that deploys a web app. The cookbook,
"my_cookbook,"
does the following:

(1) install Java and Tomcat
(2) copies a remote WAR into Tomcat's webapps directory.

I'm using test-kitchen to test out this cookbook.

Once this cookbook is properly tested, here's my understanding as to how
to
deploy it using chef-client & chef-server:

(1) set up Chef Server to host "my_cookbook"
--> involves forking opscode/chef-repo

Actually it doesn't involve that at all, though it's a good idea to
store your cookbook in source control (in its OWN repository, not as a
subdirectory of a 'chef-repo' repository or using git submodules or
any of that funny business). However, 'knife' doesn't care about that;
you merely need to upload your cookbook to the server.

(2) install and set up chef-client on my "node", i.e. machine to be
provisioned from Chef-Server

Right.

  • Julian

--
[ Julian C. Dunn jdunn@aquezada.com * Sorry, I'm ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]

On Mon, May 19, 2014 at 10:53 AM, Matt Stratton matt.stratton@gmail.com wrote:

Julian,

Not to go off on a tangent, but I'm curious about a statement in here. You
mention that the cookbook should not be "using git submodules or any of that
funny business".

What would be the negative reason for putting chef-repo in source control,
and then all cookbooks in their own repos, but putting a submodule of each
cookbook under the cookbooks directory of the chef-repo? I've been working
on a few projects where we intended to do this eventually, and just haven't
done so yet. What issues/problems might I not be realizing? :slight_smile:

A submodule is pinned to a particular Git hash, so you wind up in a
whole world of maintenance hell when you want to get a consistent
state for your top level. example: if I 'git submodule update' to get
to a new hash, I'm immediately impacting everyone.

I usually set things up like this for customers:

  • chef-repo [its own Git module, with a .gitignore containing cookbooks/*]
    • roles
    • environments
    • data_bags
    • cookbooks (empty directory, maybe with a README.md in it
      explaining why it's empty)

and then a Git module per cookbook.

The workflow for someone starting with Chef at that company would be:

  • Clone chef-repo
  • cd into cookbooks, clone individual cookbooks as needed

Yeah, there's Berkshelf, and upstream dependencies, and all that jazz
I haven't accounted for. But when people are first starting out, the
most important thing they are interested in doing is to start writing
some Chef code immediately without having to learn a pile of tools.

  • Julian

--
[ Julian C. Dunn jdunn@aquezada.com * Sorry, I'm ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]

I knew there was a reason, other than "submodules can confuse the heck out
of git newbies, and most of our customers are new to git". Your recommended
structure is similar to the one I've been using (we actually put the
chef-repo is a specific subdir named after the chef org, and then we put
the cookbooks into individual git modules in a cookbooks dir at the same
level as the chef-repo, as we have rake tasks we put in the chef-repo for
our customers to do things like create new cookbooks (but with chefspec,
berks init, etc).

On Mon, May 19, 2014 at 10:06 AM, Julian C. Dunn jdunn@aquezada.com wrote:

On Mon, May 19, 2014 at 10:53 AM, Matt Stratton matt.stratton@gmail.com
wrote:

Julian,

Not to go off on a tangent, but I'm curious about a statement in here.
You
mention that the cookbook should not be "using git submodules or any of
that
funny business".

What would be the negative reason for putting chef-repo in source
control,
and then all cookbooks in their own repos, but putting a submodule of
each
cookbook under the cookbooks directory of the chef-repo? I've been
working
on a few projects where we intended to do this eventually, and just
haven't
done so yet. What issues/problems might I not be realizing? :slight_smile:

A submodule is pinned to a particular Git hash, so you wind up in a
whole world of maintenance hell when you want to get a consistent
state for your top level. example: if I 'git submodule update' to get
to a new hash, I'm immediately impacting everyone.

I usually set things up like this for customers:

  • chef-repo [its own Git module, with a .gitignore containing cookbooks/*]
    • roles
    • environments
    • data_bags
    • cookbooks (empty directory, maybe with a README.md in it
      explaining why it's empty)

and then a Git module per cookbook.

The workflow for someone starting with Chef at that company would be:

  • Clone chef-repo
  • cd into cookbooks, clone individual cookbooks as needed

Yeah, there's Berkshelf, and upstream dependencies, and all that jazz
I haven't accounted for. But when people are first starting out, the
most important thing they are interested in doing is to start writing
some Chef code immediately without having to learn a pile of tools.

  • Julian

--
[ Julian C. Dunn jdunn@aquezada.com * Sorry, I'm ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]

On Mon, May 19, 2014 at 8:06 AM, Julian C. Dunn jdunn@aquezada.com
wrote:

[...]

I usually set things up like this for customers:

  • chef-repo [its own Git module, with a .gitignore containing cookbooks/*]
    • roles
    • environments
    • data_bags
    • cookbooks (empty directory, maybe with a README.md in it
      explaining why it's empty)

Following is something that we use and has worked really well for us and
our customers.

/vagrant/chef/

  • cookbooks/
  • cookbooks-for-chef-solo-run/
  • data-bags/
  • environments/
  • file-cache-path/
  • handlers/

Below are some notable differences and an overview of the approach that
we take.

  • This directory structure maps directly to what we consider are
    fundamental concepts in Chef. Its lot easier for us to explain Chef
    Server once these concepts are well understood by beginners.

    We also want to be able introduce concepts in a layered fashion, and
    this directory structure allows us to do just that.

  • Any changes to this directory structure is managed through

    • Vagrantfile, solo.rb and client.rb.
  • During development we use Berksfile and Berksfile.lock in
    cookbooks/ directory to copy (or download) cookbooks and their
    dependencies to cookbooks-for-chef-solo-run/ directory. This is
    done using the following commands,

    • cd /vagrant/chef/cookbooks
    • berks apply
    • berks install --path /vagrant/chef/cookbooks-for-chef-solo-run/

    For beginners, this de-mystifies all the magic around cookbook
    dependencies, versioning etc., (We use Berkshelf 2.0.16 and ignore
    rest of Berkshelf features at this stage, including Berkshelf 3.x).

    Chef::Config[:cookbook_path] points to cookbooks-for-chef-solo-run/
    directory.

    Later when we are ready, we can nuke
    /vagrant/chef/cookbooks-for-chef-solo-run/ directory and introduce
    Chef Server.

  • Next we introduce Chef::Node object along with Chef Attribute
    precedence - Automatic, Override, Normal (set) and Default. We don't
    cover (or encourage the use of) Force Default and Force Override.

    This reduces the attribute precedence to 4 levels - Automatic,
    Override, Normal and Default. However, effectively you have only
    3 because automatic attributes are read-only.

  • With Chef::Node object explained along with attribute precedence
    we've the required foundation to explain cookbook patterns.

  • Yes, we explain Cookbook Patterns to beginners. They are going to
    encounter it anyway, and we might as well teach it to them. We cover
    library, application, wrapper, base cookbook patterns.

  • We don't talk about environment cookbook pattern because to
    a beginner it can be very confusing with Chef environment. Instead we
    introduce the idea in a different way.

  • Our cookbooks/ directory consists of,

    1. Library Cookbooks, Application Cookbooks (when they are under
      development)

    2. Wrapper Cookbooks and Base Cookbooks (These are checked in along
      with cookbook/Berksfile and cookbooks/Berksfile.lock)

  • We don't cover roles. That is the reason why you see it missing in
    the directory structure. We expect roles not to feature in the book
    "Chef: The Good Parts" :slight_smile:

Hopefully we can make this material (slides, exercises etc.,) available
to the community soon.

Best,
Rajiv