Environment Cookbook Patterns & Questions

Chef Infra (archive)
1

Hi all,

I’ve been playing with the Berkshelf Way for quite some time now: application, library, base, and wrapper cookbooks all make sense to me. However, I’ve been stuck on two issues and reading the available documentation online hasn’t really solved them. I’m looking for some insight on how folks seem to accomplish what I’m trying to achieve.

From what I’ve read, there seems to be two ways to look at “environment cookbooks”: one that is application centric, and one that is code environment centric.

For instance, Berksflow seems to encourage the creation of environment cookbooks that actually have an application name in them (such as myface), and then environment files live in Chef (prod, qa, dev, etc.), and the two get concatenated when managing the environments with a tool like “blo” (IE: myface-prod). However, this seems to get confusing when code and infrastructure environments do not go hand-in-hand. For example, our infrastructure lives at EC2, and there’s often different configuration for nodes living in US East vs US West (usually dealing with networking). However, instances living in both US East and US West are technically in the “prod” code environment, so I’m hesitant to create environment files in Chef called “prod-east” and “prod-west” when in actuality, these should be treated identically from a Chef standpoint (we’d want to upgrade cookbooks at the same time, etc.). How does the environment cookbook pattern handle situations like this where “prod” from a code environment (IE: what your customers are accessing) doesn’t necessarily match “prod” for an infrastructure standpoint (IE: multiple data centers)? And is it safe to say that environment specific configurations can live in environment files, as those aren’t versioned so I didn’t think their use was encouraged. Additionally, in this blog post (http://blog.vialstudios.com/the-environment-cookbook-pattern/) by Jamie Windsor, he states that environment cookbooks often look like application cookbooks. Does that mean the environment cookbook replaces the application cookbook in most cases? Would the myface::database_server recipe include everything it needs to setup a myface database server, or simply include the myface::database_server recipe in an application cookbook living elsewhere? (IE: https://github.com/reset/myface-cookbook). I’m kind of confused as to where the “logic” of the application goes when using the “environment cookbook pattern": what does the environment cookbook’s recipe look like?

As for solving the main issue, there seems to be another route that is entirely 100% cookbook based (with zero use of environment files or even roles): making environment cookbooks start with the environment name (IE: prod, dev) and then contain the application configuration within. For example, creating a prod cookbook and then having recipes like “prod::myface_dbserver” that installs a production my face database server and can be applied to any run-list anywhere. In this case, for my example I’d have to do something like prod-east and prod-west to accomplish what I’m looking for, with even more cut/pasting between the cookbooks, and/or come up with something perhaps I’m missing (which is why I’m emailing the list!).

Thank you for any insight!

-Matt

2

This is a great question. I’m looking forward to other people’s answers.

Here’s my thoughts to get the ball rolling.

It makes sense to me to create a base ‘myface’ cookbook that knows how to
set up the logical components of the application like database servers,
application servers, the actual application code etc. I’d consider this to
be an application cookbook, one that you could then reuse in multiple
contexts (e.g. the different production data centers). Perhaps it might
even be reused in different lifecycle phases (prod, staging, systest).

The above might become an environment cookbook if you use it to lock down
the cookbook versioning for actual deployments. In your case it sounds like
intend the east and west data centers to be part of one overall production
environment and want to keep them in sync from a cookbook (and application
software) version perspective. If so, the environment cookbook could be
’myface-prod’ and consist of little more than a berksfile.lock plus
attributes to customize the application cookbook specifically for prod
environment.

The key question is how you can then add in the ‘east’ and ‘west’ data
center variances. Environment files for myface-prod-east and
myface-prod-west would seem a decent option, if the variances are fairly
simple attribute differences, and particularly if the variances were
relatively independent/orthogonal to the application (e.g. updating
networking information for one data center is relatively independent of
changes to the myface application). You’d likely want to version the
environment definitions externally in source control. You may need to
coordinate cookbook changes with environment changes, in the case where an
application change requires networking changes in the two data centers.

However, if application changes routinely need different updates in the two
data centers, you may want to take a different approach - e.g. creating
myface-prod-east and myface-prod-west cookbooks that customize the myface
cookbook for those environments. You gain the flexibility of two
separately versionable/testable cookbooks at the expense of extra
coordination.

Regards,
Christine Draper

On Sat, Jan 3, 2015 at 2:04 PM, Matt Juszczak matt@atopia.net wrote:

Hi all,

I’ve been playing with the Berkshelf Way for quite some time now:
application, library, base, and wrapper cookbooks all make sense to me.
However, I’ve been stuck on two issues and reading the available
documentation online hasn’t really solved them. I’m looking for some
insight on how folks seem to accomplish what I’m trying to achieve.

From what I’ve read, there seems to be two ways to look at “environment
cookbooks”: one that is application centric, and one that is code
environment centric.

For instance, Berksflow seems to encourage the creation of environment
cookbooks that actually have an application name in them (such as myface),
and then environment files live in Chef (prod, qa, dev, etc.), and the two
get concatenated when managing the environments with a tool like “blo” (IE:
myface-prod). However, this seems to get confusing when code and
infrastructure environments do not go hand-in-hand. For example, our
infrastructure lives at EC2, and there’s often different configuration for
nodes living in US East vs US West (usually dealing with networking).
However, instances living in both US East and US West are technically in
the “prod” code environment, so I’m hesitant to create environment files in
Chef called “prod-east” and “prod-west” when in actuality, these should be
treated identically from a Chef standpoint (we’d want to upgrade cookbooks
at the same time, etc.). How does the environment cookbook pattern handle
situations like this where “prod” from a code environment (IE: what your
customers are accessing) doesn’t necessarily match “prod” for an
infrastructure standpoint (IE: multiple data centers)? And is it safe to
say that environment specific configurations can live in environment files,
as those aren’t versioned so I didn’t think their use was encouraged.
Additionally, in this blog post (
http://blog.vialstudios.com/the-environment-cookbook-pattern/) by Jamie
Windsor, he states that environment cookbooks often look like application
cookbooks. Does that mean the environment cookbook replaces the application
cookbook in most cases? Would the myface::database_server recipe include
everything it needs to setup a myface database server, or simply include
the myface::database_server recipe in an application cookbook living
elsewhere? (IE: https://github.com/reset/myface-cookbook). I’m kind of
confused as to where the “logic” of the application goes when using the
“environment cookbook pattern": what does the environment cookbook’s recipe
look like?

As for solving the main issue, there seems to be another route that is
entirely 100% cookbook based (with zero use of environment files or even
roles): making environment cookbooks start with the environment name (IE:
prod, dev) and then contain the application configuration within. For
example, creating a prod cookbook and then having recipes like
“prod::myface_dbserver” that installs a production my face database server
and can be applied to any run-list anywhere. In this case, for my example
I’d have to do something like prod-east and prod-west to accomplish what
I’m looking for, with even more cut/pasting between the cookbooks, and/or
come up with something perhaps I’m missing (which is why I’m emailing the
list!).

Thank you for any insight!

-Matt

3

Christine,

It makes sense to me to create a base ‘myface’ cookbook that knows how to set up the logical components of the application like database servers, application servers, the actual application code etc. I’d consider this to be an application cookbook, one that you could then reuse in multiple contexts (e.g. the different production data centers). Perhaps it might even be reused in different lifecycle phases (prod, staging, systest).

The above might become an environment cookbook if you use it to lock down the cookbook versioning for actual deployments.

Gotcha. So in this case, the application cookbook and environment cookbook are literally the same thing and you don’t have BOTH. An application cookbook simply becomes an environment cookbook once a Berksfile.lock is checked in. Jamie mentions that application cookbooks and environment cookbooks don’t differ much, except to me it’s been unclear whether having an environment cookbook for an app replaces the functionality of the application cookbook all-together (simply by now having a Berksfile.lock in revision control). I think I got it now.

In your case it sounds like intend the east and west data centers to be part of one overall production environment and want to keep them in sync from a cookbook (and application software) version perspective. If so, the environment cookbook could be ‘myface-prod’ and consist of little more than a berksfile.lock plus attributes to customize the application cookbook specifically for prod environment.

I’m thinking we would make our environment cookbooks code environments only. In other words, while a prod environment might have static attributes that differ from a staging environment and will always differ, it seems the environment cookbook pattern that Jamie describes is about managing an SDLC. You apply version 1.0.1 of a cookbook to your staging environment, and eventually that makes its way to prod. Therefore, to me, it makes sense to have a “myface” environment cookbook that can be versioned, and utilize the berkflow tool to promote different versions of that cookbook through chef server environments.

But that still doesn’t solve the problem of infrastructure environments (IE: configuration that applies to servers in the staging environment but never the prod environment and vice versa). For example, if you deploy version 1.0.0 of myface to staging, you’re eventually going to deploy that to production. However, if you deploy changes to “staging environment configuration” (perhaps network addresses?), you likely aren’t going to copy those configuration parameters to production any time soon. That configuration is strictly for the staging infrastructure environment!

So…

The key question is how you can then add in the ‘east’ and ‘west’ data center variances. Environment files for myface-prod-east and myface-prod-west would seem a decent option, if the variances are fairly simple attribute differences, and particularly if the variances were relatively independent/orthogonal to the application (e.g. updating networking information for one data center is relatively independent of changes to the myface application). You’d likely want to version the environment definitions externally in source control. You may need to coordinate cookbook changes with environment changes, in the case where an application change requires networking changes in the two data centers.

Based on your feedback, and the more I think about this, the more I think it does make sense to create more environment files in chef server and maintain them in revision control. These environment files would contain both application (managed by berkflow) and infrastructure environment configuration options (such as network configuration). Therefore, we would likely create something like:

myface-prod-us-east
myface-prod-us-west
myface-staging-us-east
myface-staging-us-west

and use berkflow to manage version pinning:

blo upgrade myface-prod-useast myface
blo upgrade myface-prod-uswest myface

…and simply build a wrapper tool to upgrade all “prod” environments at once that calls blo upgrade numerous times. Meanwhile, the environments above (myface-prod-useast) have specific configuration for servers in that environment, such as network configuration.

Thanks for your insight. I’m still curious to hear some other opinions but it’s good to know others have been confused on this and I’m not missing something simple.

-Matt

4

Matt,

Your analysis makes a lot of sense to me. We need a way to separate the
inherent variances between infrastructure environments as one dimension;
and the through-time release (“code”) variances as another.

Christine

On Sat, Jan 3, 2015 at 10:09 PM, Matt Juszczak matt@atopia.net wrote:

Christine,

It makes sense to me to create a base ‘myface’ cookbook that knows how
to set up the logical components of the application like database servers,
application servers, the actual application code etc. I’d consider this to
be an application cookbook, one that you could then reuse in multiple
contexts (e.g. the different production data centers). Perhaps it might
even be reused in different lifecycle phases (prod, staging, systest).

The above might become an environment cookbook if you use it to lock
down the cookbook versioning for actual deployments.

Gotcha. So in this case, the application cookbook and environment cookbook
are literally the same thing and you don’t have BOTH. An application
cookbook simply becomes an environment cookbook once a Berksfile.lock is
checked in. Jamie mentions that application cookbooks and environment
cookbooks don’t differ much, except to me it’s been unclear whether having
an environment cookbook for an app replaces the functionality of the
application cookbook all-together (simply by now having a Berksfile.lock in
revision control). I think I got it now.

In your case it sounds like intend the east and west data centers to be
part of one overall production environment and want to keep them in sync
from a cookbook (and application software) version perspective. If so, the
environment cookbook could be ‘myface-prod’ and consist of little more than
a berksfile.lock plus attributes to customize the application cookbook
specifically for prod environment.

I’m thinking we would make our environment cookbooks code environments
only. In other words, while a prod environment might have static attributes
that differ from a staging environment and will always differ, it seems
the environment cookbook pattern that Jamie describes is about managing an
SDLC. You apply version 1.0.1 of a cookbook to your staging environment,
and eventually that makes its way to prod. Therefore, to me, it makes sense
to have a “myface” environment cookbook that can be versioned, and utilize
the berkflow tool to promote different versions of that cookbook through
chef server environments.

But that still doesn’t solve the problem of infrastructure environments
(IE: configuration that applies to servers in the staging environment but
never the prod environment and vice versa). For example, if you deploy
version 1.0.0 of myface to staging, you’re eventually going to deploy that
to production. However, if you deploy changes to “staging environment
configuration” (perhaps network addresses?), you likely aren’t going to
copy those configuration parameters to production any time soon. That
configuration is strictly for the staging infrastructure environment!

So…

The key question is how you can then add in the ‘east’ and ‘west’ data
center variances. Environment files for myface-prod-east and
myface-prod-west would seem a decent option, if the variances are fairly
simple attribute differences, and particularly if the variances were
relatively independent/orthogonal to the application (e.g. updating
networking information for one data center is relatively independent of
changes to the myface application). You’d likely want to version the
environment definitions externally in source control. You may need to
coordinate cookbook changes with environment changes, in the case where an
application change requires networking changes in the two data centers.

Based on your feedback, and the more I think about this, the more I think
it does make sense to create more environment files in chef server and
maintain them in revision control. These environment files would contain
both application (managed by berkflow) and infrastructure environment
configuration options (such as network configuration). Therefore, we would
likely create something like:

myface-prod-us-east
myface-prod-us-west
myface-staging-us-east
myface-staging-us-west

and use berkflow to manage version pinning:

blo upgrade myface-prod-useast myface
blo upgrade myface-prod-uswest myface

…and simply build a wrapper tool to upgrade all “prod” environments at
once that calls blo upgrade numerous times. Meanwhile, the environments
above (myface-prod-useast) have specific configuration for servers in that
environment, such as network configuration.

Thanks for your insight. I’m still curious to hear some other opinions but
it’s good to know others have been confused on this and I’m not missing
something simple.

-Matt

5

Agreed!! Thanks for the help! It’s worth mentioning that “poise-appenv” seems to try to solve this problem.

Matt

On Jan 4, 2015, at 7:08 AM, Christine Draper christine_draper@thirdwaveinsights.com wrote:

Matt,

Your analysis makes a lot of sense to me. We need a way to separate the inherent variances between infrastructure environments as one dimension; and the through-time release (“code”) variances as another.

Christine

On Sat, Jan 3, 2015 at 10:09 PM, Matt Juszczak matt@atopia.net wrote:
Christine,

It makes sense to me to create a base ‘myface’ cookbook that knows how to set up the logical components of the application like database servers, application servers, the actual application code etc. I’d consider this to be an application cookbook, one that you could then reuse in multiple contexts (e.g. the different production data centers). Perhaps it might even be reused in different lifecycle phases (prod, staging, systest).

The above might become an environment cookbook if you use it to lock down the cookbook versioning for actual deployments.

Gotcha. So in this case, the application cookbook and environment cookbook are literally the same thing and you don’t have BOTH. An application cookbook simply becomes an environment cookbook once a Berksfile.lock is checked in. Jamie mentions that application cookbooks and environment cookbooks don’t differ much, except to me it’s been unclear whether having an environment cookbook for an app replaces the functionality of the application cookbook all-together (simply by now having a Berksfile.lock in revision control). I think I got it now.

In your case it sounds like intend the east and west data centers to be part of one overall production environment and want to keep them in sync from a cookbook (and application software) version perspective. If so, the environment cookbook could be ‘myface-prod’ and consist of little more than a berksfile.lock plus attributes to customize the application cookbook specifically for prod environment.

I’m thinking we would make our environment cookbooks code environments only. In other words, while a prod environment might have static attributes that differ from a staging environment and will always differ, it seems the environment cookbook pattern that Jamie describes is about managing an SDLC. You apply version 1.0.1 of a cookbook to your staging environment, and eventually that makes its way to prod. Therefore, to me, it makes sense to have a “myface” environment cookbook that can be versioned, and utilize the berkflow tool to promote different versions of that cookbook through chef server environments.

But that still doesn’t solve the problem of infrastructure environments (IE: configuration that applies to servers in the staging environment but never the prod environment and vice versa). For example, if you deploy version 1.0.0 of myface to staging, you’re eventually going to deploy that to production. However, if you deploy changes to “staging environment configuration” (perhaps network addresses?), you likely aren’t going to copy those configuration parameters to production any time soon. That configuration is strictly for the staging infrastructure environment!

So…

The key question is how you can then add in the ‘east’ and ‘west’ data center variances. Environment files for myface-prod-east and myface-prod-west would seem a decent option, if the variances are fairly simple attribute differences, and particularly if the variances were relatively independent/orthogonal to the application (e.g. updating networking information for one data center is relatively independent of changes to the myface application). You’d likely want to version the environment definitions externally in source control. You may need to coordinate cookbook changes with environment changes, in the case where an application change requires networking changes in the two data centers.

Based on your feedback, and the more I think about this, the more I think it does make sense to create more environment files in chef server and maintain them in revision control. These environment files would contain both application (managed by berkflow) and infrastructure environment configuration options (such as network configuration). Therefore, we would likely create something like:

myface-prod-us-east
myface-prod-us-west
myface-staging-us-east
myface-staging-us-west

and use berkflow to manage version pinning:

blo upgrade myface-prod-useast myface
blo upgrade myface-prod-uswest myface

…and simply build a wrapper tool to upgrade all “prod” environments at once that calls blo upgrade numerous times. Meanwhile, the environments above (myface-prod-useast) have specific configuration for servers in that environment, such as network configuration.

Thanks for your insight. I’m still curious to hear some other opinions but it’s good to know others have been confused on this and I’m not missing something simple.

-Matt

6

My favorite approach is to use a “top-level cookbook”. A top-level
cookbook roughly represents a VM. It is essentially the same as an
"environment cookbook" but uses metadata.rb for locking the dependency
graph rather than environments.

This lets you use environments for representing your infrastructure
environments rather than (ab?)using them for locking dependency
graphs. The only cookbook dependency you have in the environment is
the one to the top-level cookbook (all others come in transitively via
metadata.rb).

Related discussions:

Examples:

Word of caution: I’m using this with chef-solo and have not checked
whether this way of keeping metadata.rb and Berksfile.lock in sync
(https://gist.github.com/tknerr/4e3236d00ceba917abea) works with
chef-client too. If not, you might want to us a generative approach
instead.

HTH,
Torben

On Mon, Jan 5, 2015 at 12:04 AM, Matt Juszczak matt@atopia.net wrote:

Agreed!! Thanks for the help! It’s worth mentioning that "poise-appenv"
seems to try to solve this problem.

Matt

On Jan 4, 2015, at 7:08 AM, Christine Draper
christine_draper@thirdwaveinsights.com wrote:

Matt,

Your analysis makes a lot of sense to me. We need a way to separate the
inherent variances between infrastructure environments as one dimension; and
the through-time release (“code”) variances as another.

Christine

On Sat, Jan 3, 2015 at 10:09 PM, Matt Juszczak matt@atopia.net wrote:

Christine,

It makes sense to me to create a base ‘myface’ cookbook that knows how
to set up the logical components of the application like database servers,
application servers, the actual application code etc. I’d consider this to
be an application cookbook, one that you could then reuse in multiple
contexts (e.g. the different production data centers). Perhaps it might
even be reused in different lifecycle phases (prod, staging, systest).

The above might become an environment cookbook if you use it to lock
down the cookbook versioning for actual deployments.

Gotcha. So in this case, the application cookbook and environment cookbook
are literally the same thing and you don’t have BOTH. An application
cookbook simply becomes an environment cookbook once a Berksfile.lock is
checked in. Jamie mentions that application cookbooks and environment
cookbooks don’t differ much, except to me it’s been unclear whether having
an environment cookbook for an app replaces the functionality of the
application cookbook all-together (simply by now having a Berksfile.lock in
revision control). I think I got it now.

In your case it sounds like intend the east and west data centers to be
part of one overall production environment and want to keep them in sync
from a cookbook (and application software) version perspective. If so, the
environment cookbook could be ‘myface-prod’ and consist of little more than
a berksfile.lock plus attributes to customize the application cookbook
specifically for prod environment.

I’m thinking we would make our environment cookbooks code environments
only. In other words, while a prod environment might have static attributes
that differ from a staging environment and will always differ, it seems
the environment cookbook pattern that Jamie describes is about managing an
SDLC. You apply version 1.0.1 of a cookbook to your staging environment, and
eventually that makes its way to prod. Therefore, to me, it makes sense to
have a “myface” environment cookbook that can be versioned, and utilize the
berkflow tool to promote different versions of that cookbook through chef
server environments.

But that still doesn’t solve the problem of infrastructure environments
(IE: configuration that applies to servers in the staging environment but
never the prod environment and vice versa). For example, if you deploy
version 1.0.0 of myface to staging, you’re eventually going to deploy that
to production. However, if you deploy changes to “staging environment
configuration” (perhaps network addresses?), you likely aren’t going to copy
those configuration parameters to production any time soon. That
configuration is strictly for the staging infrastructure environment!

So…

The key question is how you can then add in the ‘east’ and ‘west’ data
center variances. Environment files for myface-prod-east and
myface-prod-west would seem a decent option, if the variances are fairly
simple attribute differences, and particularly if the variances were
relatively independent/orthogonal to the application (e.g. updating
networking information for one data center is relatively independent of
changes to the myface application). You’d likely want to version the
environment definitions externally in source control. You may need to
coordinate cookbook changes with environment changes, in the case where an
application change requires networking changes in the two data centers.

Based on your feedback, and the more I think about this, the more I think
it does make sense to create more environment files in chef server and
maintain them in revision control. These environment files would contain
both application (managed by berkflow) and infrastructure environment
configuration options (such as network configuration). Therefore, we would
likely create something like:

myface-prod-us-east
myface-prod-us-west
myface-staging-us-east
myface-staging-us-west

and use berkflow to manage version pinning:

blo upgrade myface-prod-useast myface
blo upgrade myface-prod-uswest myface

…and simply build a wrapper tool to upgrade all “prod” environments at
once that calls blo upgrade numerous times. Meanwhile, the environments
above (myface-prod-useast) have specific configuration for servers in that
environment, such as network configuration.

Thanks for your insight. I’m still curious to hear some other opinions but
it’s good to know others have been confused on this and I’m not missing
something simple.

-Matt

7

Thanks for the suggestion!

On Jan 5, 2015, at 6:02 AM, Torben Knerr mail@tknerr.de wrote:

My favorite approach is to use a “top-level cookbook”. A top-level
cookbook roughly represents a VM. It is essentially the same as an
"environment cookbook" but uses metadata.rb for locking the dependency
graph rather than environments.

This lets you use environments for representing your infrastructure
environments rather than (ab?)using them for locking dependency
graphs. The only cookbook dependency you have in the environment is
the one to the top-level cookbook (all others come in transitively via
metadata.rb).

Related discussions:

Examples:

Word of caution: I’m using this with chef-solo and have not checked
whether this way of keeping metadata.rb and Berksfile.lock in sync
(https://gist.github.com/tknerr/4e3236d00ceba917abea) works with
chef-client too. If not, you might want to us a generative approach
instead.

HTH,
Torben

On Mon, Jan 5, 2015 at 12:04 AM, Matt Juszczak matt@atopia.net wrote:

Agreed!! Thanks for the help! It’s worth mentioning that "poise-appenv"
seems to try to solve this problem.

Matt

On Jan 4, 2015, at 7:08 AM, Christine Draper
christine_draper@thirdwaveinsights.com wrote:

Matt,

Your analysis makes a lot of sense to me. We need a way to separate the
inherent variances between infrastructure environments as one dimension; and
the through-time release (“code”) variances as another.

Christine

On Sat, Jan 3, 2015 at 10:09 PM, Matt Juszczak matt@atopia.net wrote:

Christine,

It makes sense to me to create a base ‘myface’ cookbook that knows how
to set up the logical components of the application like database servers,
application servers, the actual application code etc. I’d consider this to
be an application cookbook, one that you could then reuse in multiple
contexts (e.g. the different production data centers). Perhaps it might
even be reused in different lifecycle phases (prod, staging, systest).

The above might become an environment cookbook if you use it to lock
down the cookbook versioning for actual deployments.

Gotcha. So in this case, the application cookbook and environment cookbook
are literally the same thing and you don’t have BOTH. An application
cookbook simply becomes an environment cookbook once a Berksfile.lock is
checked in. Jamie mentions that application cookbooks and environment
cookbooks don’t differ much, except to me it’s been unclear whether having
an environment cookbook for an app replaces the functionality of the
application cookbook all-together (simply by now having a Berksfile.lock in
revision control). I think I got it now.

In your case it sounds like intend the east and west data centers to be
part of one overall production environment and want to keep them in sync
from a cookbook (and application software) version perspective. If so, the
environment cookbook could be ‘myface-prod’ and consist of little more than
a berksfile.lock plus attributes to customize the application cookbook
specifically for prod environment.

I’m thinking we would make our environment cookbooks code environments
only. In other words, while a prod environment might have static attributes
that differ from a staging environment and will always differ, it seems
the environment cookbook pattern that Jamie describes is about managing an
SDLC. You apply version 1.0.1 of a cookbook to your staging environment, and
eventually that makes its way to prod. Therefore, to me, it makes sense to
have a “myface” environment cookbook that can be versioned, and utilize the
berkflow tool to promote different versions of that cookbook through chef
server environments.

But that still doesn’t solve the problem of infrastructure environments
(IE: configuration that applies to servers in the staging environment but
never the prod environment and vice versa). For example, if you deploy
version 1.0.0 of myface to staging, you’re eventually going to deploy that
to production. However, if you deploy changes to “staging environment
configuration” (perhaps network addresses?), you likely aren’t going to copy
those configuration parameters to production any time soon. That
configuration is strictly for the staging infrastructure environment!

So…

The key question is how you can then add in the ‘east’ and ‘west’ data
center variances. Environment files for myface-prod-east and
myface-prod-west would seem a decent option, if the variances are fairly
simple attribute differences, and particularly if the variances were
relatively independent/orthogonal to the application (e.g. updating
networking information for one data center is relatively independent of
changes to the myface application). You’d likely want to version the
environment definitions externally in source control. You may need to
coordinate cookbook changes with environment changes, in the case where an
application change requires networking changes in the two data centers.

Based on your feedback, and the more I think about this, the more I think
it does make sense to create more environment files in chef server and
maintain them in revision control. These environment files would contain
both application (managed by berkflow) and infrastructure environment
configuration options (such as network configuration). Therefore, we would
likely create something like:

myface-prod-us-east
myface-prod-us-west
myface-staging-us-east
myface-staging-us-west

and use berkflow to manage version pinning:

blo upgrade myface-prod-useast myface
blo upgrade myface-prod-uswest myface

…and simply build a wrapper tool to upgrade all “prod” environments at
once that calls blo upgrade numerous times. Meanwhile, the environments
above (myface-prod-useast) have specific configuration for servers in that
environment, such as network configuration.

Thanks for your insight. I’m still curious to hear some other opinions but
it’s good to know others have been confused on this and I’m not missing
something simple.

-Matt

8

we’ve been using an interesting pattern based on ideas that have been
around a while, that’s been working pretty well for us, but i haven’t seen
discussed much. i’ve been meaning to write up a blog post about (just as
soon as i find that 25th hour in the day). here’s a quick overview, happy
to flesh out any details if you have questions.

we’re a 2-person ops team in a medium-sized org with about 50 nodes, mixed
about 90/10 between VMs and physical gear. not sure how well this would
scale to a bigger setup, but not seeing any bottlenecks at present, except
that cookbook promotion is more manual than i would like, which we’re
planning on fixing with some CI pipeline stuff. anyways… the details:

we refuse to set attributes in roles, and just use them for runlist
management. nodes get roles in the runlist, roles get recipes (e.g.
app::default and app::monitoring). roles roughly correlate to suites in our
test-kitchen configuration (i.e. monitoring cookbook has both client and
server suites and corresponding roles)

we use wrapper, library, and application cookbooks. our application
cookbooks roughly map to roles, with a few exceptions.

we run our own berkshelf-api service that hooks into our hosted-chef org,
and use a berksfile with just the metadata config in each cookbook repo.
the cookbook-berksfile has our berks-api listed as a source, and the
berksfile.lock is in the .gitignore, and uploading new cookbooks is done
with berks upload from the cookbook repo.

we then have an infrastructure repo, where we keep documentation, runbooks,
maintenance plans (ops stuff that isn’t software), and “the” Berksfile that
lists all of our org-specific cookbooks. the infrastructure repo has one
branch per environment. promoting a cookbook is just a matter of berks
updating a cookbook (berks update myface) after it’s been uploaded, and
berks applying to the environment that corresponds to the relevant branch
(we initially envisioned promoting from staging->prod via PRs of the diff
to the Berksfile.lock, but it turned out to be a hassle, so we just handle
it manually for now). this also provides a nice audit-log in the form of
the git-log if something should go wrong, and visibility into any dependent
or contingent cookbooks that may also be changed.

overall, this gives us really good control (by locking the environment with
berks apply) over what changes are released, without getting much in the way

that said, we don’t have a way to release an updated library cookbook or
community cookbook to a specific role. one version of each cookbook per
environment, period. i’ve heard of people addressing this with
role-cookbooks, but it hasn’t been a problem for us in practice yet, so
we’re not using them. i expect we’ll switch to policyfiles before
introducing role-cookbooks, and we’re eagerly watching policyfile support
develop.

hope that helps!

– nathan

Regards,

Nathan Williams

On Mon, Jan 5, 2015 at 11:59 AM, Matt Juszczak matt@atopia.net wrote:

Thanks for the suggestion!

On Jan 5, 2015, at 6:02 AM, Torben Knerr mail@tknerr.de wrote:

My favorite approach is to use a “top-level cookbook”. A top-level
cookbook roughly represents a VM. It is essentially the same as an
"environment cookbook" but uses metadata.rb for locking the dependency
graph rather than environments.

This lets you use environments for representing your infrastructure
environments rather than (ab?)using them for locking dependency
graphs. The only cookbook dependency you have in the environment is
the one to the top-level cookbook (all others come in transitively via
metadata.rb).

Related discussions:

Examples:

Word of caution: I’m using this with chef-solo and have not checked
whether this way of keeping metadata.rb and Berksfile.lock in sync
(https://gist.github.com/tknerr/4e3236d00ceba917abea) works with
chef-client too. If not, you might want to us a generative approach
instead.

HTH,
Torben

On Mon, Jan 5, 2015 at 12:04 AM, Matt Juszczak matt@atopia.net wrote:

Agreed!! Thanks for the help! It’s worth mentioning that "poise-appenv"
seems to try to solve this problem.

Matt

On Jan 4, 2015, at 7:08 AM, Christine Draper
christine_draper@thirdwaveinsights.com wrote:

Matt,

Your analysis makes a lot of sense to me. We need a way to separate the
inherent variances between infrastructure environments as one
dimension; and

the through-time release (“code”) variances as another.

Christine

On Sat, Jan 3, 2015 at 10:09 PM, Matt Juszczak matt@atopia.net wrote:

Christine,

It makes sense to me to create a base ‘myface’ cookbook that knows how
to set up the logical components of the application like database
servers,

application servers, the actual application code etc. I’d consider
this to

be an application cookbook, one that you could then reuse in multiple
contexts (e.g. the different production data centers). Perhaps it
might

even be reused in different lifecycle phases (prod, staging, systest).

The above might become an environment cookbook if you use it to lock
down the cookbook versioning for actual deployments.

Gotcha. So in this case, the application cookbook and environment
cookbook

are literally the same thing and you don’t have BOTH. An application
cookbook simply becomes an environment cookbook once a Berksfile.lock
is

checked in. Jamie mentions that application cookbooks and environment
cookbooks don’t differ much, except to me it’s been unclear whether
having

an environment cookbook for an app replaces the functionality of the
application cookbook all-together (simply by now having a
Berksfile.lock in

revision control). I think I got it now.

In your case it sounds like intend the east and west data centers to
be

part of one overall production environment and want to keep them in
sync

from a cookbook (and application software) version perspective. If
so, the

environment cookbook could be ‘myface-prod’ and consist of little
more than

a berksfile.lock plus attributes to customize the application cookbook
specifically for prod environment.

I’m thinking we would make our environment cookbooks code
environments

only. In other words, while a prod environment might have static
attributes

that differ from a staging environment and will always differ, it
seems

the environment cookbook pattern that Jamie describes is about
managing an

SDLC. You apply version 1.0.1 of a cookbook to your staging
environment, and

eventually that makes its way to prod. Therefore, to me, it makes
sense to

have a “myface” environment cookbook that can be versioned, and
utilize the

berkflow tool to promote different versions of that cookbook through
chef

server environments.

But that still doesn’t solve the problem of infrastructure
environments

(IE: configuration that applies to servers in the staging environment
but

never the prod environment and vice versa). For example, if you deploy
version 1.0.0 of myface to staging, you’re eventually going to deploy
that

to production. However, if you deploy changes to “staging environment
configuration” (perhaps network addresses?), you likely aren’t going
to copy

those configuration parameters to production any time soon. That
configuration is strictly for the staging infrastructure environment!

So…

The key question is how you can then add in the ‘east’ and ‘west’ data
center variances. Environment files for myface-prod-east and
myface-prod-west would seem a decent option, if the variances are
fairly

simple attribute differences, and particularly if the variances were
relatively independent/orthogonal to the application (e.g. updating
networking information for one data center is relatively independent
of

changes to the myface application). You’d likely want to version the
environment definitions externally in source control. You may need to
coordinate cookbook changes with environment changes, in the case
where an

application change requires networking changes in the two data
centers.

Based on your feedback, and the more I think about this, the more I
think

it does make sense to create more environment files in chef server and
maintain them in revision control. These environment files would
contain

both application (managed by berkflow) and infrastructure environment
configuration options (such as network configuration). Therefore, we
would

likely create something like:

myface-prod-us-east
myface-prod-us-west
myface-staging-us-east
myface-staging-us-west

and use berkflow to manage version pinning:

blo upgrade myface-prod-useast myface
blo upgrade myface-prod-uswest myface

…and simply build a wrapper tool to upgrade all “prod” environments
at

once that calls blo upgrade numerous times. Meanwhile, the environments
above (myface-prod-useast) have specific configuration for servers in
that

environment, such as network configuration.

Thanks for your insight. I’m still curious to hear some other opinions
but

it’s good to know others have been confused on this and I’m not missing
something simple.

-Matt