I just wanted to get a feeling for what standard practices people are
following for managing chef cookbooks.
We will be moving to git internally and I would like to plan out the
migration of our cookbooks to git. What is the best practice for this
layout?
One repo per cookbook? or all cookbooks in one repository?
How about databags, roles, environments?
Any feedback would be greatly appreciated.
–
Thanks,
Mark
We use one chef-repo git repository that contains our roles,
non-encrypted data bags and a cookbooks directory that only contains git
submodules that reference separate cookbook repositories (both
upstream/community as well as internal).
We store encrypted data bags in a separate repo that is
encrypted/decrypted using some Python automation code we have.
Takes a little getting used to git submodules, but overall it works
quite well for us. Use source control for what it's good at: versioning
related software. Use Chef for what it's good at: configuration
management. My personal opinion is it is best not to mix the two things...
Best,
-jay
On 03/13/2013 11:22 AM, Mark Pimentel wrote:
I just wanted to get a feeling for what standard practices people are
following for managing chef cookbooks.We will be moving to git internally and I would like to plan out the
migration of our cookbooks to git. What is the best practice for this
layout?One repo per cookbook? or all cookbooks in one repository?
How about databags, roles, environments?Any feedback would be greatly appreciated.
--
Thanks,Mark
I think it's an emerging best practice to go with having everything but
cookbooks in your master git repo: environments, data bags, roles, etc.
Then keep just all the cookbooks in their own repositories. You should look
into how people are beginning to use the wrapper cookbook pattern and what
berkshelf provides to tie everything together. After a year and a half of
doing it all in one big repository and feeling the pain of it, I can
unequivocally say that I think this is the future.
http://www.ustream.tv/recorded/27487546/highlight/309841
Best Regards,
-Kevin
On Wed, Mar 13, 2013 at 10:22 AM, Mark Pimentel markpimentel22@gmail.comwrote:
I just wanted to get a feeling for what standard practices people are
following for managing chef cookbooks.We will be moving to git internally and I would like to plan out the
migration of our cookbooks to git. What is the best practice for this
layout?One repo per cookbook? or all cookbooks in one repository?
How about databags, roles, environments?Any feedback would be greatly appreciated.
--
Thanks,Mark
We us one cook per repo (migrated from monolithic repos). With one repo per
project/infra that has a berksfile + env/bags/rolls. Internal dev projects
in this scenario can get their own version of the main repo, but all
cookbook work happens per cook and each dev/project/infra can lock their
versions down in environments + berksfile. This has been IMO the best
layout for working with diverse projects and needs. The main goal for us
was to capture all work in cooks and make sure there was single source of
truth for those cooks. This has worked out much better than the one repo
for everything model we moved from. We see less issues with people stepping
on toes, and individual cookbook quality has anecdotally gone up. We have
~15-20 people interacting with the cookbooks and a core of about 4 people
that are the "main" committers.
On Thu, Mar 14, 2013 at 12:22 AM, Mark Pimentel markpimentel22@gmail.comwrote:
I just wanted to get a feeling for what standard practices people are
following for managing chef cookbooks.We will be moving to git internally and I would like to plan out the
migration of our cookbooks to git. What is the best practice for this
layout?One repo per cookbook? or all cookbooks in one repository?
How about databags, roles, environments?Any feedback would be greatly appreciated.
--
Thanks,Mark
We are using:
This works quite well and is quite sound imho - cookbook dependency
management is king!
Within the project infrastructure repo we have a rake task which resolves
the cookbook dependencies per application cookbook into
/cookbooks/-. This honors the fact
that application cookbooks may use different versions of dependent library
cookbooks (e.g. app-a depends on lib-d 1.0 but app-b depends on lib-d 2.0
and both live in the same project infra)
The only thing that makes me a bit unhappy is that we can not use version
numbers like "1.1.0.dev" to differentiate between in-development and stable
versions or "1.1.0.my-fork" to differentiate between forks of and original
community cookbook versions.
Cheers, Torben
Am 13.03.2013 16:46 schrieb "Jesse Nelson" spheromak@gmail.com:
We us one cook per repo (migrated from monolithic repos). With one repo
per project/infra that has a berksfile + env/bags/rolls. Internal dev
projects in this scenario can get their own version of the main repo, but
all cookbook work happens per cook and each dev/project/infra can lock
their versions down in environments + berksfile. This has been IMO the best
layout for working with diverse projects and needs. The main goal for us
was to capture all work in cooks and make sure there was single source of
truth for those cooks. This has worked out much better than the one repo
for everything model we moved from. We see less issues with people stepping
on toes, and individual cookbook quality has anecdotally gone up. We have
~15-20 people interacting with the cookbooks and a core of about 4 people
that are the "main" committers.On Thu, Mar 14, 2013 at 12:22 AM, Mark Pimentel markpimentel22@gmail.comwrote:
I just wanted to get a feeling for what standard practices people are
following for managing chef cookbooks.We will be moving to git internally and I would like to plan out the
migration of our cookbooks to git. What is the best practice for this
layout?One repo per cookbook? or all cookbooks in one repository?
How about databags, roles, environments?Any feedback would be greatly appreciated.
--
Thanks,Mark
Am 13.03.2013 18:39 schrieb "Torben Knerr" ukio@gmx.de:
We are using:
- single git repo per cookbook
- one git repo per project infrastructure
- cookbook dependencies resolved via librarian-chef instead of berkshelf
(due to proxy support)- application cookbooks instead of roles
This works quite well and is quite sound imho - cookbook dependency
management is king!Within the project infrastructure repo we have a rake task which resolves
the cookbook dependencies per application cookbook into
/cookbooks/-.
P.S.: and we are using the application cookbook's Cheffile/Berksfile for
dependency resolution so we don't have to repeat overridden/custom
dependency sources in the project infrastructure repo (like you would if
you had a Cheffile/Berksfile with the application cookbooks there)
This honors the fact that application cookbooks may use different
versions of dependent library cookbooks (e.g. app-a depends on lib-d 1.0
but app-b depends on lib-d 2.0 and both live in the same project infra)The only thing that makes me a bit unhappy is that we can not use version
numbers like "1.1.0.dev" to differentiate between in-development and stable
versions or "1.1.0.my-fork" to differentiate between forks of and original
community cookbook versions.Cheers, Torben
Am 13.03.2013 16:46 schrieb "Jesse Nelson" spheromak@gmail.com:
We us one cook per repo (migrated from monolithic repos). With one repo
per project/infra that has a berksfile + env/bags/rolls. Internal dev
projects in this scenario can get their own version of the main repo, but
all cookbook work happens per cook and each dev/project/infra can lock
their versions down in environments + berksfile. This has been IMO the best
layout for working with diverse projects and needs. The main goal for us
was to capture all work in cooks and make sure there was single source of
truth for those cooks. This has worked out much better than the one repo
for everything model we moved from. We see less issues with people stepping
on toes, and individual cookbook quality has anecdotally gone up. We have
~15-20 people interacting with the cookbooks and a core of about 4 people
that are the "main" committers.On Thu, Mar 14, 2013 at 12:22 AM, Mark Pimentel markpimentel22@gmail.com
wrote:I just wanted to get a feeling for what standard practices people are
following for managing chef cookbooks.We will be moving to git internally and I would like to plan out the
migration of our cookbooks to git. What is the best practice for this
layout?One repo per cookbook? or all cookbooks in one repository?
How about databags, roles, environments?Any feedback would be greatly appreciated.
--
Thanks,Mark
A number of folks responded indicating a myriad of problems with having all
the cookbooks in one repo. I'd like to hear more about these problems. So
far (admittedly, without too many people working on the cookbooks) I've had
no trouble keeping all the cookbooks in one big repo together. Following
the standard practice of branch/work/merge, unless we're actually working
on the same part of the same cookbook at the same time (which would clearly
have issues even if each cookbook was in its own repo), the merges all
happen with no trouble.
Could one of you that migrated from one-big-repo to tons-of-tiny-repos
describe in more detail the issues you encountered?
My other question is how you deal with having so many repositories. When
you want to refresh your local copy, I'm used to just doing a 'git pull'.
Instead do you have to 'for i in *; do pushd $i; git pull; popd; done'
every time you want to get everything up to current?
Thanks,
-ben
On Wed, Mar 13, 2013 at 8:22 AM, Mark Pimentel markpimentel22@gmail.comwrote:
I just wanted to get a feeling for what standard practices people are
following for managing chef cookbooks.We will be moving to git internally and I would like to plan out the
migration of our cookbooks to git. What is the best practice for this
layout?One repo per cookbook? or all cookbooks in one repository?
How about databags, roles, environments?Any feedback would be greatly appreciated.
--
Thanks,Mark
this used to be a problem before berkshelf, librarian.
I keep the repository similar to how its documented in the wiki. Its a
single repo for cookbooks, roles, env etc. Everything is in ruby, no json.
cookbooks folder is gitignored with holds all the community cookbooks
(managed by berks) . Whenever we need to alter any community cookbooks,
standard github fork/pull request style workflow is practiced. Cheffile
points to the forked repos of the cookbooks till the time pull requests are
merged (you can also maintain them in your own repo in a separate directory
if you want, and point berks to pick it up from there, but u'll miss out on
the updates to those cookbooks).
These practices are pretty comparable to any other ruby project, except
bundler takes up the job of librarian/berkshelf.
Previous to librarian and berkshelf , several of us used git-submodule and
ultimate trick foo .. but with great pain and anguish. 
regards
ranjib
On Wed, Mar 13, 2013 at 11:35 AM, Ben Hartshorne ben@parse.com wrote:
A number of folks responded indicating a myriad of problems with having
all the cookbooks in one repo. I'd like to hear more about these
problems. So far (admittedly, without too many people working on the
cookbooks) I've had no trouble keeping all the cookbooks in one big repo
together. Following the standard practice of branch/work/merge, unless
we're actually working on the same part of the same cookbook at the same
time (which would clearly have issues even if each cookbook was in its own
repo), the merges all happen with no trouble.Could one of you that migrated from one-big-repo to tons-of-tiny-repos
describe in more detail the issues you encountered?My other question is how you deal with having so many repositories. When
you want to refresh your local copy, I'm used to just doing a 'git pull'.
Instead do you have to 'for i in *; do pushd $i; git pull; popd; done'
every time you want to get everything up to current?Thanks,
-ben
On Wed, Mar 13, 2013 at 8:22 AM, Mark Pimentel markpimentel22@gmail.comwrote:
I just wanted to get a feeling for what standard practices people are
following for managing chef cookbooks.We will be moving to git internally and I would like to plan out the
migration of our cookbooks to git. What is the best practice for this
layout?One repo per cookbook? or all cookbooks in one repository?
How about databags, roles, environments?Any feedback would be greatly appreciated.
--
Thanks,Mark
Am 13.03.2013 19:36 schrieb "Ben Hartshorne" ben@parse.com:
A number of folks responded indicating a myriad of problems with having
all the cookbooks in one repo. I'd like to hear more about these
problems. So far (admittedly, without too many people working on the
cookbooks) I've had no trouble keeping all the cookbooks in one big repo
together.
This might work well if you have only a single chef repo. If you have
multiple chef repos for multiple infrastructures you end up duplicating
cookbook sources and you are no longer having a single source of cookbook
truth as soon as you make the first chef-repo local modification to a
cookbook.
Following the standard practice of branch/work/merge, unless we're
actually working on the same part of the same cookbook at the same time
(which would clearly have issues even if each cookbook was in its own
repo), the merges all happen with no trouble.Could one of you that migrated from one-big-repo to tons-of-tiny-repos
describe in more detail the issues you encountered?My other question is how you deal with having so many repositories. When
you want to refresh your local copy, I'm used to just doing a 'git pull'.
Instead do you have to 'for i in *; do pushd $i; git pull; popd; done'
every time you want to get everything up to current?
You wouldn't do that. You'd rather have a Berksfile or Cheffile defining
your cookbook dependencies and then run 'berks install' or 'librarian-chef
install' and your cookbook dependencies get downloaded to ./cookbooks which
is typically .gitignore'd.
Thanks,
-ben
On Wed, Mar 13, 2013 at 8:22 AM, Mark Pimentel markpimentel22@gmail.com
wrote:I just wanted to get a feeling for what standard practices people are
following for managing chef cookbooks.We will be moving to git internally and I would like to plan out the
migration of our cookbooks to git. What is the best practice for this
layout?One repo per cookbook? or all cookbooks in one repository?
How about databags, roles, environments?Any feedback would be greatly appreciated.
--
Thanks,Mark