Managing a chef repository with multiple engineers

I’m wondering if anyone has come across any blog posts or anything
regarding best practices with managing the cookbooks in a chef repository
when you have multiple engineers working with the repository at the same
time?
Currently, I’m basically using the git-flow approach; however, I find it
cumbersome to merge multiple feature branches together for different
cookbooks (and sometimes the same cookbook), make sure there is an
appropriate version bump in each affected cookbook, updated changelog files
in each cookbook, and determine what actually changed so that I can upload
the changed cookbooks, roles, data bags to the server.
I’m just wondering what other folks are doing.

John

I just started a blog, and my first post is on this topic. It's a reasonable summary of the Berkshelf approach (including the link to Jamie's talk that Noah posted and some other resources), and some things to take in to consideration. We are just starting down this path, so we don't have it all figured out quite yet...

Also, this episode of the FoodFightShow, which has Etsy and Riot Games both discussing how they handle cookbook collaboration:

http://foodfight.libsyn.com/episode-32-there-s-a-spork-in-my-berkshelf-talkin-bout-workflow

I hope this is helpful!

Eric Reeves
Senior Systems Engineer
Alert Logic, Inc.

M 713.992.7993
O 713.351.1775
E eric@alertlogic.commailto:eric@alertlogic.com

[ C.H.A.M.S. - Culture, HUGS, Automation, Metrics, and Sharing ]

On May 6, 2013, at 3:09 PM, John Alberts wrote:

I'm wondering if anyone has come across any blog posts or anything regarding best practices with managing the cookbooks in a chef repository when you have multiple engineers working with the repository at the same time?
Currently, I'm basically using the git-flow approach; however, I find it cumbersome to merge multiple feature branches together for different cookbooks (and sometimes the same cookbook), make sure there is an appropriate version bump in each affected cookbook, updated changelog files in each cookbook, and determine what actually changed so that I can upload the changed cookbooks, roles, data bags to the server.
I'm just wondering what other folks are doing.

Having lived through this in real life, +1 to the Berkshelf way.

For those of you that use berkshelf, do you still keep a chef-repo that
holds your roles, data bags, environment files, etc?

On Mon, May 6, 2013 at 6:30 PM, Brian Akins brian@akins.org wrote:

Having lived through this in real life, +1 to the Berkshelf way.

--
John Alberts

Hey John,

We, the Berkshelf developers, actually strongly suggest against using roles and attempt to avoid data bags unless they really make sense (organization wide data).

We also advocate for not versioning environments or nodes in a chef repo since both of those pieces are manipulated by our orchestrator. We hope to open this up pretty soon to the community.

With Berkshelf and motherbrain we're trying to do away with the idea of a Chef Repo and instead provide tools which populate and manipulate the data in a Chef server. This will add an easier to use high level of abstraction on top of the typical Chef workflow which will hopefully lower the learning curve of getting into Chef.

I think a Chef repo containing data not managed by a higher level abstraction is a good in between goal. Just make sure you immediately evict those cookbooks from it :wink:

Jamie Winsor
@resetexistence

On May 6, 2013, at 9:20 PM, John Alberts john.m.alberts@gmail.com wrote:

For those of you that use berkshelf, do you still keep a chef-repo that holds your roles, data bags, environment files, etc?

On Mon, May 6, 2013 at 6:30 PM, Brian Akins brian@akins.org wrote:

Having lived through this in real life, +1 to the Berkshelf way.

--
John Alberts

Most people still use a chef-repo holding all the other pieces of infrastructure. FYI Spiceweasel has Berkshelf support so it can manage and validate your roles, environments, data bags and nodes in conjunction with Berkshelf managing the cookbooks.

Thanks,
Matt Ray
Senior Technical Evangelist :: Opscodehttp://opscode.com
512.731.2218 :: matt@opscode.com
mattray :: GitHubhttp://github.com/mattray/ :: IRC :: Twitterhttp://twitter.com/mattray


From: John Alberts
Sent: Monday, May 06, 2013 11:20 PM
To: chef@lists.opscode.com
Subject: [chef] Re: Re: Re: Managing a chef repository with multiple engineers

For those of you that use berkshelf, do you still keep a chef-repo that holds your roles, data bags, environment files, etc?

On Mon, May 6, 2013 at 6:30 PM, Brian Akins <brian@akins.orgmailto:brian@akins.org> wrote:

Having lived through this in real life, +1 to the Berkshelf way.


John Alberts

We don’t use roles or environments - wrapper cookbooks only. We do use
data bags but use a repo per data bag and use CI/CD tools to push them to
all the needed orgs.

Brian, what CD tools do you use?

Thanks!

On Tue, May 7, 2013 at 1:26 PM, Brian Akins brian@akins.org wrote:

We don't use roles or environments - wrapper cookbooks only. We do use
data bags but use a repo per data bag and use CI/CD tools to push them to
all the needed orgs.

On Tue, May 7, 2013 at 4:35 PM, Thor Wolpert thor@wolpert.ca wrote:

Brian, what CD tools do you use?

Bamboo with some custom plugins along with some ruby scripts. Calling them
CD tools is being generous, but they get the job done for now.

  1. Follow https://www.youtube.com/watch?v=hYt0E84kYUI
  2. There is no step two

:slight_smile:

--Noah

On May 6, 2013, at 1:09 PM, John Alberts wrote:

I'm wondering if anyone has come across any blog posts or anything regarding best practices with managing the cookbooks in a chef repository when you have multiple engineers working with the repository at the same time?
Currently, I'm basically using the git-flow approach; however, I find it cumbersome to merge multiple feature branches together for different cookbooks (and sometimes the same cookbook), make sure there is an appropriate version bump in each affected cookbook, updated changelog files in each cookbook, and determine what actually changed so that I can upload the changed cookbooks, roles, data bags to the server.
I'm just wondering what other folks are doing.

John

John,

I think you'll like knife-spork:

Something like Berkshelf might also help:

http://berkshelf.com/

-Nathen
@nathenharvey

From: John Alberts <john.m.alberts@gmail.commailto:john.m.alberts@gmail.com>
Reply-To: "chef@lists.opscode.commailto:chef@lists.opscode.com" <chef@lists.opscode.commailto:chef@lists.opscode.com>
Date: Monday, May 6, 2013 4:09 PM
To: "chef@lists.opscode.commailto:chef@lists.opscode.com" <chef@lists.opscode.commailto:chef@lists.opscode.com>
Subject: [chef] Managing a chef repository with multiple engineers

I'm wondering if anyone has come across any blog posts or anything regarding best practices with managing the cookbooks in a chef repository when you have multiple engineers working with the repository at the same time?
Currently, I'm basically using the git-flow approach; however, I find it cumbersome to merge multiple feature branches together for different cookbooks (and sometimes the same cookbook), make sure there is an appropriate version bump in each affected cookbook, updated changelog files in each cookbook, and determine what actually changed so that I can upload the changed cookbooks, roles, data bags to the server.
I'm just wondering what other folks are doing.

John

Hi John,

we are using knife-spork to manage our Chef workflow at Etsy and these
slides1 should give a rough overview how we do it. Hope that helps, if you
have any questions, let me know.

Cheers,
Daniel

On Mon, May 06, 2013 at 03:09:47PM -0500, John Alberts wrote:

I'm wondering if anyone has come across any blog posts or anything
regarding best practices with managing the cookbooks in a chef repository
when you have multiple engineers working with the repository at the same
time?
Currently, I'm basically using the git-flow approach; however, I find it
cumbersome to merge multiple feature branches together for different
cookbooks (and sometimes the same cookbook), make sure there is an
appropriate version bump in each affected cookbook, updated changelog
files in each cookbook, and determine what actually changed so that I can
upload the changed cookbooks, roles, data bags to the server.
I'm just wondering what other folks are doing.
John

Wow, 5 minutes and 4 replies. Awesome community. :slight_smile:
Already using the awesome knife-spork plugin, but I'll take a look at the
slides and also the youtube video and reply back afterwords.

Thanks
John

On Mon, May 6, 2013 at 3:16 PM, Daniel Schauenberg d@unwiredcouch.comwrote:

Hi John,

we are using knife-spork to manage our Chef workflow at Etsy and these
slides1 should give a rough overview how we do it. Hope that helps, if
you
have any questions, let me know.

Cheers,
Daniel

On Mon, May 06, 2013 at 03:09:47PM -0500, John Alberts wrote:

I'm wondering if anyone has come across any blog posts or anything
regarding best practices with managing the cookbooks in a chef
repository
when you have multiple engineers working with the repository at the
same
time?
Currently, I'm basically using the git-flow approach; however, I find
it
cumbersome to merge multiple feature branches together for different
cookbooks (and sometimes the same cookbook), make sure there is an
appropriate version bump in each affected cookbook, updated changelog
files in each cookbook, and determine what actually changed so that I
can
upload the changed cookbooks, roles, data bags to the server.
I'm just wondering what other folks are doing.
John

--
John Alberts