Help understanding environment cookbook pattern, and how to generate it?


#1

We are using Chef 12 Enterprise, and have different organizations setup for development and QA.

We want to adapt to the Berkshelf way of managing cookbooks. The workflow puts a lot of emphasis on the “environment” cookbook, and I read numerous articles and watch numerous videos about it.

http://misheska.com/blog/2013/06/16/getting-started-writing-chef-cookbooks-the-berkshelf-way/
http://blog.vialstudios.com/the-environment-cookbook-pattern/


…and many more.

However, the concept of environment cookbook still eludes me. Can you please shed some light?

Say I have a product that has two VMs, a Web VM and a Router VM. Say I have the following cookbooks, (each cookbook is in its own repo and downloaded to the workstation)

.berkshelf/default/cookbooks/java
.berkshelf/default/cookbooks/apache
.berkshelf/default/cookbooks/web-appl
.berkshelf/default/cookbooks/iprouter-appl

My Web box needs these
.berkshelf/default/cookbooks/java
.berkshelf/default/cookbooks/apache
.berkshelf/default/cookbooks/web-appl

My Router box needs these
.berkshelf/default/cookbooks/java
.berkshelf/default/cookbooks/iprouter-appl

I want to setup “dev” and “qa” environments so I can control the versions I want for all the above cookbooks in each environment.

Where does the environment cookbook come into play?

Should I have the following environment cookbooks (I inferred this from this link http://bytearrays.com/manage-chef-cookbooks-organization-repo) which says environment cookbooks " Represent a whole node to be installed/managed"

  • web-dev
  • web-qa
  • router-dev
  • router-qa

If so, what would above cookbook’s metadata.rb and Berkshelf.lock file look like?

Chris


#2

On Thursday, February 12, 2015 at 12:51 PM, Fouts, Chris wrote:

We are using Chef 12 Enterprise, and have different organizations setup for development and QA.
We want to adapt to the Berkshelf way of managing cookbooks. The workflow puts a lot of emphasis on the “environment” cookbook, and I read numerous articles and watch numerous videos about it.
http://misheska.com/blog/2013/06/16/getting-started-writing-chef-cookbooks-the-berkshelf-way/
http://blog.vialstudios.com/the-environment-cookbook-pattern/
http://www.hurryupandwait.io/blog/chef-cookbook-dependency-management-and-the-environment-cookbook-pattern
http://bytearrays.com/manage-chef-cookbooks-organization-repo/
…and many more.
However, the concept of environment cookbook still eludes me. Can you please shed some light?
Say I have a product that has two VMs, a Web VM and a Router VM. Say I have the following cookbooks, (each cookbook is in its own repo and downloaded to the workstation)
.berkshelf/default/cookbooks/java
.berkshelf/default/cookbooks/apache
.berkshelf/default/cookbooks/web-appl
.berkshelf/default/cookbooks/iprouter-appl
My Web box needs these
.berkshelf/default/cookbooks/java
.berkshelf/default/cookbooks/apache
.berkshelf/default/cookbooks/web-appl
My Router box needs these
.berkshelf/default/cookbooks/java
.berkshelf/default/cookbooks/iprouter-appl
I want to setup “dev” and “qa” environments so I can control the versions I want for all the above cookbooks in each environment.
Where does the environment cookbook come into play?
Should I have the following environment cookbooks (I inferred this from this link http://bytearrays.com/manage-chef-cookbooks-organization-repo) which says environment cookbooks ” Represent a whole node to be installed/managed”

  • web-dev
  • web-qa
  • router-dev
  • router-qa

Yes, this is how it works. Instead of Chef environments representing the a stage in the deployment cycle for all of your apps, when you use the environment cookbook pattern, your Chef environments now represent the intersection of both a functional role and a deployment stage. So your Chef environments don’t map to your “real” environments one-to-one (some folks dislike this but if it works for you, go for it). The advantage of this is that each kind of application can have the versions of its dependencies managed independently (e.g., web and router can use different versions of the java cookbook in prod, which is hard to do safely and easily if you’re mapping your Chef environments one to one with your real environments). Berkshelf generates the Chef environment data for you, setting an exact version pin in the Chef environment which matches the dependency solution in your lock file.

If so, what would above cookbook’s metadata.rb and Berkshelf.lock file look like?

The blog post you referenced here http://blog.vialstudios.com/the-environment-cookbook-pattern/ covers this pretty well. You have a depends entry for each cookbook a particular application needs. E.g., for your example web box, your environment cookbook depends on java, apache, and web-appl. You can add other version constraints if you need them (e.g., web and router might only be compatible with different major versions of the java cookbook, which you’d manage by putting the correct squiggly constraints ‘~> x.y’ in your depends statements).

Chris

HTH,


Daniel DeLeo


#3

Reviving and old, but (I think) still relevant topic…

I too have been struggling with the concept of an environment cookbook. I understand how to create one and how to apply it, but in all the material I had come across, it wasn’t clear to me why this was better than traditional environments. In my thinking… yes, you can have many versions of an environment cookbook, but what good is that? A node is tied to an environment cookbook via it’s runlist, but it’s always going to get the latest of that environment cookbook. How is that different than traditional environments where they aren’t versioned - you’re always going to get the latest. Right? This statement from above shed a lot of light on the value:

So, it’s not really about environments being “versioned” so much as it’s about being able to have multiple apps co-existing within a single org on the chef server - each having their own definition of an “environment”. So, when one changes, other’s aren’t impacted. Is that right? Yes, that sounds useful.

We’re in the process of building out our pipeline and we’ve been planning to segment using a separate org per app. This would have a lot of advantages for us as - i.e. teams owning and operating the apps will only want to see their nodes, each will have dedicated users on the chef server, etc. We intend to use Visibility to give the umbrella view across all orgs for folks that want that. This seems like a good solution for us and with that, I don’t really see an advantage to the environment cookbook pattern because apps will not be collocated. When an environment needs to change, it needs to change for the entire org. We would also like to take advantage of the native filtering capabilities in Visibility on the traditional “environment” construct.

If you’re planning to segment by org like I’m describing, is there any advantage to the environment cookbook pattern?


#4

The first thing to do when trying to understand the environment cookbook pattern is to pretend that the word “environment” has no meaning whatsoever. In traditional use, it’s expected that your environments map to things like “dev,” “test,” “prod,” etc. When you use the environment cookbook pattern, you are re-using the functionality that the environments feature provides to do something different. Specifically, you are using the version-locking functionality to create locked cookbook sets that generally you will use on a per-application basis. The second bit of this is where you bolt back on the ability to have different locked cookbook sets in use at the same time by concatenating the development stage with the application name. So you name your Chef environments something like “mywebapp-dev,” “mywebapp-test,” and so on. This way you can have different locked cookbook sets at various stages of the software lifecycle.

This means your apps can’t use the search feature to find things. Also, the separation between different teams may be convenient most of the time, but it will make it difficult for teams to troubleshoot and interact if a change/deployment in app A breaks app B.

If these things aren’t a deal-breaker for you, then you wouldn’t get much benefit from environment cookbooks because you’re isolating the cookbook sets in a different way.


#5

Thanks for the reply. This makes sense. I’m still not totally convinced this will add much value to our world given the nature of our business, but leveraging this pattern is a very lightweight effort that could pay dividends down the road if/when my assumptions are violated. It’s an easy abstraction that gives you another level to pull.