Chef best practice for multiple developers


#1

Okay so I installed a Chef server, A chef workstation and a chef node.

I am able to create cookbook with a recipe and upload the results from the workstation to the server. The server can then update the node.

Everything is good.

Only one question

How do multiple developers make, update and recipes?
I know about cvs, git and all that but what it the best practice with Chef?

Ray Keckler
Unix Administator | Acclaris, Inc.http://acclaris.com/ | c 813.9515160
Transforming consumer-driven healthcare and reimbursement offerings.

PRIVACY/CONFIDENTIALITY NOTICE: This e-mail communication may contain private, confidential, or legally privileged information intended for the sole use of the designated and/or duly authorized recipient(s). If you are not the intended recipient or have received this communication in error, please notify the sender immediately by email to privacy.officer@acclaris.commailto:privacy.officer@acclaris.com or by telephone at 1-813-873-2020, and delete all copies of this e-mail, including all attachments, without reading them or saving them to your computer or any attached storage device. If you are the intended recipient, you will need to secure the contents conforming to all applicable state and/or federal requirements related to the privacy and confidentiality of such information, including the HIPAA Privacy guidelines.


#2

On Dec 27, 2013, at 7:50 AM, “Keckler, Ray” Ray.Keckler@acclaris.com wrote:

How do multiple developers make, update and recipes?
I know about cvs, git and all that but what it the best practice with Chef?

There are multiple facets to this question.

One has to do with the development of the code itself, and how you keep one developer from stepping on the toes of another one. The answer to that question depends on a lot of factors, including the size of the team of developers you’re working with, the methods that they tend to apply when developing code, and others.

For example, if it’s a two-man team of highly skilled/talented Agile developers who know each other well and are very familiar with their chosen shared methods, then the answers are likely to be very different compared to a large group of disparate people who are not well-versed in a given method.

Speaking only for myself, I would recommend the use of git, in part because there are some workflow tools out there that can help you with this job, and they were developed assuming that you would be using git. If you can host your cookbooks on github (maybe in private repos), then you can use the github standard methods of issue tracking, pull requests, etc… That would also allow you to relatively easily keep a small number of long-running branches (maybe just “master” and “develop”), and then for each “chunk” of code functionality could be done in a feature branch that starts off life as a fork of one of these branches, and ends up being merged back in and then being deleted immediately thereafter.

Another facet has to do with how you test your code to make sure that it’s working as desired. Here, I think the recommendation would be to use a variety of tools, including “knife cookbook test” to do basic syntax checking before you upload a cookbook, the “foodcritic” code style checker, writing chefspec tests for each cookbook before you actually write the cookbook code itself (i.e., Test-Driven Development), Berkshelf to help you manage your dependencies on other cookbooks that you use but for which you don’t want to maintain your own forks, and Test Kitchen to help you do integration testing on a vagrant-based VM on your local box before it gets put onto a separate VM or machine of its own.

You could also do some work to integrate these two facets, by using git commit hooks, so that no code is allowed to be committed to the repo unless it at least passes the basic syntax checking performed by “knife cookbook test” and basic code style checking as performed by foodcritic. You might also want to set up a Continuous Integration or maybe even a Continuous Delivery system using tools like Jenkins and/or Travis-CI, so that you can build up a large library of automated tests at various levels that are run against your code on a regular basis, and that may go so far as to include automated deployment of code that passes all the tests.

There’s lots of workflow tooling that is available, and you should check that out and see what makes sense in your environment.

The suggestions above are just a bunch of ideas thrown off the top of my head, based on work that my co-workers and I have done for some recent clients, and our ideas of what we think Best Practices are, and how we continue to tweak that concept as we do more development and we learn more about what works well for us in our environments.


Brad Knowles brad@shub-internet.org
LinkedIn Profile: http://tinyurl.com/y8kpxu


#3

I would say go with git. You can also have Gerrit as a gatekeeper for git.
This will provide review workflow. Or simplest way is using github for
managing repos.

On Friday, December 27, 2013, Keckler, Ray wrote:

Okay so I installed a Chef server, A chef workstation and a chef node.

I am able to create cookbook with a recipe and upload the results from
the workstation to the server. The server can then update the node.

Everything is good.

Only one question

How do multiple developers make, update and recipes?

I know about cvs, git and all that but what it the best practice with Chef?

Ray Keckler

Unix Administator | Acclaris, Inc. http://acclaris.com/ | c
813.9515160
Transforming consumer-driven healthcare and reimbursement offerings.

PRIVACY/CONFIDENTIALITY NOTICE: This e-mail communication may contain
private, confidential, or legally privileged information intended for the
sole use of the designated and/or duly authorized recipient(s). If you are
not the intended recipient or have received this communication in error,
please notify the sender immediately by email to
privacy.officer@acclaris.com <javascript:_e({}, ‘cvml’,
‘privacy.officer@acclaris.com’);> or by telephone at 1-813-873-2020, and
delete all copies of this e-mail, including all attachments, without
reading them or saving them to your computer or any attached storage
device. If you are the intended recipient, you will need to secure the
contents conforming to all applicable state and/or federal requirements
related to the privacy and confidentiality of such information, including
the HIPAA Privacy guidelines.


#4

Great question & thanks for the informed reply Brad.

I’ve been pondering about this myself. My team is in the initial stages of
our Chef implenetation. We’re currently a team of 15 developers.We have
staging & production environments for all our applications that directly
map to develop & master branches in their respective git repositories. We
also use Jenkins for integration & deployment across our services so a push
to develop or master triggers a deployment to our staging or production
servers.

My question is how do we do a similar setup for Chef? Does it make sense to
have two different chef servers per environment? My initial guess would be
no because chef itself is used to manage the different environments.

But, on the other hand it would be ideal to have git branches as the single
point of truth such that everything on master branch in the chef-repo is
the exact replica of what’s on the chef-server.

How are you guys doing it currently?

Ritesh

On Fri, Dec 27, 2013 at 10:57 PM, Ameya Varade ameya.varade@clogeny.comwrote:

I would say go with git. You can also have Gerrit as a gatekeeper for git.
This will provide review workflow. Or simplest way is using github for
managing repos.

On Friday, December 27, 2013, Keckler, Ray wrote:

Okay so I installed a Chef server, A chef workstation and a chef node.

I am able to create cookbook with a recipe and upload the results from
the workstation to the server. The server can then update the node.

Everything is good.

Only one question

How do multiple developers make, update and recipes?

I know about cvs, git and all that but what it the best practice with
Chef?

Ray Keckler

Unix Administator | Acclaris, Inc. http://acclaris.com/ | c
813.9515160
Transforming consumer-driven healthcare and reimbursement offerings.

PRIVACY/CONFIDENTIALITY NOTICE: This e-mail communication may contain
private, confidential, or legally privileged information intended for the
sole use of the designated and/or duly authorized recipient(s). If you are
not the intended recipient or have received this communication in error,
please notify the sender immediately by email to
privacy.officer@acclaris.com or by telephone at 1-813-873-2020, and
delete all copies of this e-mail, including all attachments, without
reading them or saving them to your computer or any attached storage
device. If you are the intended recipient, you will need to secure the
contents conforming to all applicable state and/or federal requirements
related to the privacy and confidentiality of such information, including
the HIPAA Privacy guidelines.


#5

On Jan 13, 2014, at 8:28 AM, Ritesh Angural ritesh.angural@gmail.com wrote:

Great question & thanks for the informed reply Brad.

I’ve been pondering about this myself. My team is in the initial stages of our Chef implenetation. We’re currently a team of 15 developers.We have staging & production environments for all our applications that directly map to develop & master branches in their respective git repositories. We also use Jenkins for integration & deployment across our services so a push to develop or master triggers a deployment to our staging or production servers.

Generally speaking, I would say that the bigger your development team, the more you’re going to need standards that are automatically enforced, and you may need more tools to help you cover all the various different areas of concern.

But each tool you use is going to have a certain cost to it, and you will want to make sure that you balance that against the benefits that it provides.

My question is how do we do a similar setup for Chef? Does it make sense to have two different chef servers per environment? My initial guess would be no because chef itself is used to manage the different environments.

Well, for one thing – application developers are not necessarily system/chef developers. So, you’ve presumably got a different community of people who would be your developers in this case. Everyone else would be a “user” of the services provided by your system developers.

You could certainly have separate Chef servers for your application developers to interface with, and then an “air gap” to the Chef servers for the production network that is only touched by your most trusted and experienced systems developers.

But, on the other hand it would be ideal to have git branches as the single point of truth such that everything on master branch in the chef-repo is the exact replica of what’s on the chef-server.

That’s basically what we’re doing with several of our customers. I think the issue is how do you get there, and what methods do you use along the way?

What gatekeepers do you have built into that process to try to help ensure that all of the code that makes it that far will actually do what you want, and that any code refactoring that might occur along the way doesn’t introduce any new bugs.

Of course, those gatekeepers could be automated tools, but they could also be things that are done by humans.

How are you guys doing it currently?

We start with git hooks for low-level syntax and style checking (e.g., knife cookbook test, foodcritic, rubocop, etc…). We are in the process of creating spec tests for all of our cookbooks, as well as test kitchen definitions for running all the tests that can be run.

That’s combined with proprietary tools to do automated discovery of what’s defined on the Chef server versus what is in the git repo, automatic creation of Jenkins jobs to run all the appropriate tests, creation of a dashboard to show the current status of all jobs, etc…

But we don’t do deployment of the whole multi-thousand node cluster that this is all designed to build.

We’re a small system tools developer team, but we’re supporting a much larger systems development/deployment team, and we’re hoping that all of these tools and methods can help them get more work done better and faster, and with less pain in both the near term and in the long term.


Brad Knowles brad@shub-internet.org
LinkedIn Profile: http://tinyurl.com/y8kpxu


#6

With Chef, the issue of workflows is almost as important as the actual
tooling. Here’s how we do it at Airbnb:

On Mon, Jan 13, 2014 at 7:14 AM, Brad Knowles brad@shub-internet.orgwrote:

On Jan 13, 2014, at 8:28 AM, Ritesh Angural ritesh.angural@gmail.com
wrote:

Great question & thanks for the informed reply Brad.

I’ve been pondering about this myself. My team is in the initial stages
of our Chef implenetation. We’re currently a team of 15 developers.We have
staging & production environments for all our applications that directly
map to develop & master branches in their respective git repositories. We
also use Jenkins for integration & deployment across our services so a push
to develop or master triggers a deployment to our staging or production
servers.

Generally speaking, I would say that the bigger your development team, the
more you’re going to need standards that are automatically enforced, and
you may need more tools to help you cover all the various different areas
of concern.

But each tool you use is going to have a certain cost to it, and you will
want to make sure that you balance that against the benefits that it
provides.

My question is how do we do a similar setup for Chef? Does it make sense
to have two different chef servers per environment? My initial guess would
be no because chef itself is used to manage the different environments.

Well, for one thing – application developers are not necessarily
system/chef developers. So, you’ve presumably got a different community of
people who would be your developers in this case. Everyone else would be a
"user" of the services provided by your system developers.

You could certainly have separate Chef servers for your application
developers to interface with, and then an “air gap” to the Chef servers for
the production network that is only touched by your most trusted and
experienced systems developers.

But, on the other hand it would be ideal to have git branches as the
single point of truth such that everything on master branch in the
chef-repo is the exact replica of what’s on the chef-server.

That’s basically what we’re doing with several of our customers. I think
the issue is how do you get there, and what methods do you use along the
way?

What gatekeepers do you have built into that process to try to help ensure
that all of the code that makes it that far will actually do what you want,
and that any code refactoring that might occur along the way doesn’t
introduce any new bugs.

Of course, those gatekeepers could be automated tools, but they could also
be things that are done by humans.

How are you guys doing it currently?

We start with git hooks for low-level syntax and style checking (e.g.,
knife cookbook test, foodcritic, rubocop, etc…). We are in the process
of creating spec tests for all of our cookbooks, as well as test kitchen
definitions for running all the tests that can be run.

That’s combined with proprietary tools to do automated discovery of what’s
defined on the Chef server versus what is in the git repo, automatic
creation of Jenkins jobs to run all the appropriate tests, creation of a
dashboard to show the current status of all jobs, etc…

But we don’t do deployment of the whole multi-thousand node cluster that
this is all designed to build.

We’re a small system tools developer team, but we’re supporting a much
larger systems development/deployment team, and we’re hoping that all of
these tools and methods can help them get more work done better and faster,
and with less pain in both the near term and in the long term.


Brad Knowles brad@shub-internet.org
LinkedIn Profile: http://tinyurl.com/y8kpxu


#7

Igor & Brad, Thank you so much for your response. Really appreciate the
transparent sharing of how it’s done at shub-internet & Airbnb.

I think for starters we’re going to continue with a gate keeper/ chef
server approach & as more and more of our developers are familiar with
chef, the workflow outlined by Igor sounds like where we’d want to be
strive towards with Chef Solo.

Thanks again!

Ritesh

On Tue, Jan 14, 2014 at 3:31 AM, Igor Serebryany <igor.serebryany@airbnb.com

wrote:

With Chef, the issue of workflows is almost as important as the actual
tooling. Here’s how we do it at Airbnb:

http://nerds.airbnb.com/making-breakfast-chef-airbnb/

On Mon, Jan 13, 2014 at 7:14 AM, Brad Knowles brad@shub-internet.orgwrote:

On Jan 13, 2014, at 8:28 AM, Ritesh Angural ritesh.angural@gmail.com
wrote:

Great question & thanks for the informed reply Brad.

I’ve been pondering about this myself. My team is in the initial stages
of our Chef implenetation. We’re currently a team of 15 developers.We have
staging & production environments for all our applications that directly
map to develop & master branches in their respective git repositories. We
also use Jenkins for integration & deployment across our services so a push
to develop or master triggers a deployment to our staging or production
servers.

Generally speaking, I would say that the bigger your development team,
the more you’re going to need standards that are automatically enforced,
and you may need more tools to help you cover all the various different
areas of concern.

But each tool you use is going to have a certain cost to it, and you will
want to make sure that you balance that against the benefits that it
provides.

My question is how do we do a similar setup for Chef? Does it make
sense to have two different chef servers per environment? My initial guess
would be no because chef itself is used to manage the different
environments.

Well, for one thing – application developers are not necessarily
system/chef developers. So, you’ve presumably got a different community of
people who would be your developers in this case. Everyone else would be a
"user" of the services provided by your system developers.

You could certainly have separate Chef servers for your application
developers to interface with, and then an “air gap” to the Chef servers for
the production network that is only touched by your most trusted and
experienced systems developers.

But, on the other hand it would be ideal to have git branches as the
single point of truth such that everything on master branch in the
chef-repo is the exact replica of what’s on the chef-server.

That’s basically what we’re doing with several of our customers. I think
the issue is how do you get there, and what methods do you use along the
way?

What gatekeepers do you have built into that process to try to help
ensure that all of the code that makes it that far will actually do what
you want, and that any code refactoring that might occur along the way
doesn’t introduce any new bugs.

Of course, those gatekeepers could be automated tools, but they could
also be things that are done by humans.

How are you guys doing it currently?

We start with git hooks for low-level syntax and style checking (e.g.,
knife cookbook test, foodcritic, rubocop, etc…). We are in the process
of creating spec tests for all of our cookbooks, as well as test kitchen
definitions for running all the tests that can be run.

That’s combined with proprietary tools to do automated discovery of
what’s defined on the Chef server versus what is in the git repo, automatic
creation of Jenkins jobs to run all the appropriate tests, creation of a
dashboard to show the current status of all jobs, etc…

But we don’t do deployment of the whole multi-thousand node cluster that
this is all designed to build.

We’re a small system tools developer team, but we’re supporting a much
larger systems development/deployment team, and we’re hoping that all of
these tools and methods can help them get more work done better and faster,
and with less pain in both the near term and in the long term.


Brad Knowles brad@shub-internet.org
LinkedIn Profile: http://tinyurl.com/y8kpxu


#8

On Jan 15, 2014, at 10:15 AM, Ritesh Angural ritesh.angural@gmail.com wrote:

Igor & Brad, Thank you so much for your response. Really appreciate the transparent sharing of how it’s done at shub-internet & Airbnb.

Actually, I currently work at an Austin-based cloud consulting company called Momentum SI.

My personal vanity domain shub-internet.org is one that I have owned since the 90s, and I use it for subscribing to most mailing lists because I don’t want people to make the mistake that I am officially speaking for the company in any capacity. In addition, this e-mail address hasn’t changed since the 90s, while I have changed employers a number of times in that same period.


Brad Knowles brad@shub-internet.org
LinkedIn Profile: http://tinyurl.com/y8kpxu