Git / Chef workflow

#1

Hey, All –

I’m working on figuring out how I’m going to implement Chef with git, and my
Google foo seems to be failing me. I know roughly how I want things to
work, but I’m not sure whether it’s possible, and I’m wondering if anyone
has links/pointers/experiences that they can share.

Here’s what I would like to be able to do

  • Have a single git repository, with the following branches
    • QA branch
    • production branch
  • Have any given change be made exactly once. That is, /etc/config.file
    will only be edited by hand once, that edit will be applied via git merges
    to every environment where it needs to be distributed.

This results in a standard workflow that looks like this:
- Changes are made in a local branch, and merged into QA
- QA chef server pulls down the changes, and applies them somehow
(knife cookbook upload -a /path/to/local/repo presumably, or at least
that’s the only option that I’ve found so far – it was pointed out to me on
the IRC channel.)
- Once changes have been validated in QA, they are merged from the QA
branch to the production branch
- Production chef servers pull down the changes, and apply them somehow
(presumably the same knife cookbook upload.... command)

I’m running into issues with this work flow, though, and I’m not sure if its
because I’m trying to do something that can’t be done, or simply because I’m
doing things in the wrong way, or what. Specifically:

  • This doesn’t handle anything other than cookbooks – data_bags get
    ignored, for example
  • Because pulling a delete in git doesn’t remove anything from the
    filesystem, old cookbooks stick around

Obviously, there are ways to work around both of these issues, but I don’t
want to invent the wheel if someone else has already dealt with this. So,
my questions:
Are other people using chef/git in this way (where git comes before, as
opposed to after, chef)?
If so, what’s the secret sauce? Are there scripts that I need to be looking
for that’ll make this all easy? Documentation that I can follow?
Am I totally misunderstanding anything about how Chef works, given what I’m
trying to do?

Thanks for any advice/pointers,

  • Ian
#2

I’m doing what you propose now to manage cookbook development from local -> Qa -> staging -> prod

However, git only manages the source code. Chef does not yet support environments, you have to ‘fake it’ by using a different chef server (or opscode platform org) for each environment.

This gist shows how I do it by reading in different knife configs based on the branch I’m on

You still need to upload your cookbooks, roles, etc into each server, so merging from local to qa won’t do it (unless you put in a git hook to do that for you)

Hope that helps,

Alex

On Nov 22, 2010, at 10:40 AM, Ian Marlier wrote:

Hey, All –

I’m working on figuring out how I’m going to implement Chef with git, and my Google foo seems to be failing me. I know roughly how I want things to work, but I’m not sure whether it’s possible, and I’m wondering if anyone has links/pointers/experiences that they can share.

Here’s what I would like to be able to do

  • Have a single git repository, with the following branches
    • QA branch
    • production branch
  • Have any given change be made exactly once. That is, /etc/config.file will only be edited by hand once, that edit will be applied via git merges to every environment where it needs to be distributed.

This results in a standard workflow that looks like this:
- Changes are made in a local branch, and merged into QA
- QA chef server pulls down the changes, and applies them somehow (knife cookbook upload -a /path/to/local/repo presumably, or at least that’s the only option that I’ve found so far – it was pointed out to me on the IRC channel.)
- Once changes have been validated in QA, they are merged from the QA branch to the production branch
- Production chef servers pull down the changes, and apply them somehow (presumably the same knife cookbook upload.... command)

I’m running into issues with this work flow, though, and I’m not sure if its because I’m trying to do something that can’t be done, or simply because I’m doing things in the wrong way, or what. Specifically:

  • This doesn’t handle anything other than cookbooks – data_bags get ignored, for example
  • Because pulling a delete in git doesn’t remove anything from the filesystem, old cookbooks stick around

Obviously, there are ways to work around both of these issues, but I don’t want to invent the wheel if someone else has already dealt with this. So, my questions:
Are other people using chef/git in this way (where git comes before, as opposed to after, chef)?
If so, what’s the secret sauce? Are there scripts that I need to be looking for that’ll make this all easy? Documentation that I can follow?
Am I totally misunderstanding anything about how Chef works, given what I’m trying to do?

Thanks for any advice/pointers,

  • Ian