Organizing multiple "clients" and cookbooks

Chef Infra (archive)
1

Greetings,
Perhaps this has been covered before, but a quick search turned up empty.

We have many large “clients.” Each of these clients runs a multitude of
apps (tomcat, apache, nginx, mysql, etc). As we use chef more and more, we
really need to decide how we are going to organize them. How are other
people doing it? Should we have a cookbook per client and put the recipes
there, including other recipes as needed (this is how we currently do it in
puppet) or should we put everyone’s apache stuff in the apache cookbook,
etc?

I think a cookbook per client makes sense, however, I fear we may repeat
ourselves allot and possibly have multiple implementations of common tasks
if we are not careful.

Thoughts?

–Brian

2

The DRYest approach would be common cookbooks with individual roles per
client. The roles could override unique client attributes, things like
apache, tomcat and mysql tuning parameters...or ports that apache runs on.
Since each role also has it's own run_list it would allow you to account
for the different groupings of software each client may also have (ie some
clients use apache only, no tomcat).

One way to help visualize the different components of Chef (and what goes
where) is to think of it as an MVC for your infrastructure. The model is
represented by data bags and attributes, the recipes (and resource actions)
are controllers and the rendered templates and final resource states are
views. And just like in a web application you wouldn't want to "hard code"
a lot of client specific logic into a controller...you would try to
externalize it as model data. Trying to approach things as data-driven as
possible will also mean less Chef cookbook code to maintain and an
infrastructure that will scale as you grow.

Seth

--
Opscode, Inc.
Seth Chisamore, Technical Evangelist
T: (404) 348-0505 E: schisamo@opscode.com
Twitter, IRC, Github: schisamo

On Tue, Oct 19, 2010 at 8:18 AM, Brian Akins brian@akins.org wrote:

Greetings,
Perhaps this has been covered before, but a quick search turned up empty.

We have many large "clients." Each of these clients runs a multitude of
apps (tomcat, apache, nginx, mysql, etc). As we use chef more and more, we
really need to decide how we are going to organize them. How are other
people doing it? Should we have a cookbook per client and put the recipes
there, including other recipes as needed (this is how we currently do it in
puppet) or should we put everyone's apache stuff in the apache cookbook,
etc?

I think a cookbook per client makes sense, however, I fear we may repeat
ourselves allot and possibly have multiple implementations of common tasks
if we are not careful.

Thoughts?

--Brian

3

On Tue, Oct 19, 2010 at 9:10 AM, Seth Chisamore schisamo@opscode.comwrote:

The DRYest approach would be common cookbooks with individual roles per
client. The roles could override unique client attributes, things like
apache, tomcat and mysql tuning parameters...or ports that apache runs on.
Since each role also has it's own run_list it would allow you to account
for the different groupings of software each client may also have (ie some
clients use apache only, no tomcat).

Unfortunately, I'm not sure that will work. Each of our "clients" is
probably larger than the average chef user's complete infrastructure in size
and complexity. Imagine several large web companies with a single
operations group...

For example, client "A" may have 4 different application "stacks":

  • apache -> tomcat -> mysql
  • varnish -> apache -> NFS mounts
  • apache + wsgi python app -> memcache + mysql
  • varnish -> proprietary app server -> who knows what

and also have development, reference, and productions environments for each
of those.

Clients B and C may have some of the same components, but in a completely
different layout.

Granted, we have been standardizing more, but this is the unfortunate place
we find ourselves in now.

Our first deployments will be for some of the more "standardized stacks" we
have, so we have some time to figure it out, I hope.

--Brian

4

I have two instances of Chef server running, one for development, one for production. But you can probably collapse them into one if you’d like.

I second the idea of using roles.

I have the following model:
Role [prod_env] - sets up DNS, NTP, etc. Pretty minimal stuff
Role [web] - which has run_list:
Role[prod_env]
Recipe[apache]
Recipe[daemontools]
Recipe[ganglia::gmond]
Recipe[log::client]
Recipe[nagios::client]

And each node in “web” cluster has in its run_list:
Role[web]

So, to reiterate (since the above example may be a bit confusing), I have a role for each type of host (web cluster, api cluster, database cluster, etc.)
And each role’s run_list is a list of roles or recipes. This also allows me to modify “default_attributes” and “override_attributes” for each type of hosts as necessary.

-Paul

From: Brian Akins [mailto:brian@akins.org]
Sent: Tuesday, October 19, 2010 8:04 AM
To: chef@lists.opscode.com
Subject: [chef] Re: Re: Organizing multiple “clients” and cookbooks

On Tue, Oct 19, 2010 at 9:10 AM, Seth Chisamore <schisamo@opscode.commailto:schisamo@opscode.com> wrote:
The DRYest approach would be common cookbooks with individual roles per client. The roles could override unique client attributes, things like apache, tomcat and mysql tuning parameters…or ports that apache runs on. Since each role also has it’s own run_list it would allow you to account for the different groupings of software each client may also have (ie some clients use apache only, no tomcat).

Unfortunately, I’m not sure that will work. Each of our “clients” is probably larger than the average chef user’s complete infrastructure in size and complexity. Imagine several large web companies with a single operations group…

For example, client “A” may have 4 different application “stacks”:

  • apache -> tomcat -> mysql
  • varnish -> apache -> NFS mounts
  • apache + wsgi python app -> memcache + mysql
  • varnish -> proprietary app server -> who knows what

and also have development, reference, and productions environments for each of those.

Clients B and C may have some of the same components, but in a completely different layout.

Granted, we have been standardizing more, but this is the unfortunate place we find ourselves in now.

Our first deployments will be for some of the more “standardized stacks” we have, so we have some time to figure it out, I hope.

–Brian

5

If I were you I would maintain separate chef repo's/servers/platform orgs per Client.

On Oct 19, 2010, at 8:04 AM, Brian Akins wrote:

On Tue, Oct 19, 2010 at 9:10 AM, Seth Chisamore schisamo@opscode.com wrote:
The DRYest approach would be common cookbooks with individual roles per client. The roles could override unique client attributes, things like apache, tomcat and mysql tuning parameters...or ports that apache runs on. Since each role also has it's own run_list it would allow you to account for the different groupings of software each client may also have (ie some clients use apache only, no tomcat).

Unfortunately, I'm not sure that will work. Each of our "clients" is probably larger than the average chef user's complete infrastructure in size and complexity. Imagine several large web companies with a single operations group...

For example, client "A" may have 4 different application "stacks":

  • apache -> tomcat -> mysql
  • varnish -> apache -> NFS mounts
  • apache + wsgi python app -> memcache + mysql
  • varnish -> proprietary app server -> who knows what

and also have development, reference, and productions environments for each of those.

Clients B and C may have some of the same components, but in a completely different layout.

Granted, we have been standardizing more, but this is the unfortunate place we find ourselves in now.

Our first deployments will be for some of the more "standardized stacks" we have, so we have some time to figure it out, I hope.

--Brian

6

I should’ve read your question more carefully… You want to actually host Chef like Opscode Platform for various outside companies (as in “clients”).
Yeah, if that’s the case, the simplest option would be to maintain separate repos/servers, etc for each client. You can’t really use the same chef server for different clients, because you might run into namespace collisions, where different clients might have “apache” cookbooks that do different things.

Unless Opscode open-sources the code to Opscode Platform, you’ll need to maintain separate servers, or somehow figure out a way to extend Chef on your own to support multiple clients.

-Paul

From: Alex Soto [mailto:apsoto@gmail.com]
Sent: Tuesday, October 19, 2010 11:12 AM
To: chef@lists.opscode.com
Subject: [chef] Re: Re: Re: Organizing multiple “clients” and cookbooks

If I were you I would maintain separate chef repo’s/servers/platform orgs per Client.

On Oct 19, 2010, at 8:04 AM, Brian Akins wrote:

On Tue, Oct 19, 2010 at 9:10 AM, Seth Chisamore <schisamo@opscode.commailto:schisamo@opscode.com> wrote:
The DRYest approach would be common cookbooks with individual roles per client. The roles could override unique client attributes, things like apache, tomcat and mysql tuning parameters…or ports that apache runs on. Since each role also has it’s own run_list it would allow you to account for the different groupings of software each client may also have (ie some clients use apache only, no tomcat).

Unfortunately, I’m not sure that will work. Each of our “clients” is probably larger than the average chef user’s complete infrastructure in size and complexity. Imagine several large web companies with a single operations group…

For example, client “A” may have 4 different application “stacks”:

  • apache -> tomcat -> mysql
  • varnish -> apache -> NFS mounts
  • apache + wsgi python app -> memcache + mysql
  • varnish -> proprietary app server -> who knows what

and also have development, reference, and productions environments for each of those.

Clients B and C may have some of the same components, but in a completely different layout.

Granted, we have been standardizing more, but this is the unfortunate place we find ourselves in now.

Our first deployments will be for some of the more “standardized stacks” we have, so we have some time to figure it out, I hope.

–Brian

7

On Tue, Oct 19, 2010 at 2:20 PM, Paul Choi paulchoi@plaxo.com wrote:

I should’ve read your question more carefully… You want to actually host
Chef like Opscode Platform for various outside companies (as in “clients”).

No, we are a "large" company with many branches.

8

On Tue, Oct 19, 2010 at 5:19 PM, Brian Akins brian@akins.org wrote:

On Tue, Oct 19, 2010 at 2:20 PM, Paul Choi paulchoi@plaxo.com wrote:

I should’ve read your question more carefully… You want to actually host
Chef like Opscode Platform for various outside companies (as in “clients”).

No, we are a "large" company with many branches.

Right. So while it still might make sense to have a separate platform
account for each business unit, there'd be a lot of redundancy across
those accounts, stuff like at least a subset of the user data that
we'd have to keep in synch.

--
Mark J. Reed markjreed@gmail.com