Re: Re: What is the cookbook development workflow in your team?

Hi James, Peter, Zuhaib,

thanks for sharing your experience and sorry for the delayed response.

One thing I noticed is that we also have our chef repo managed in a git
repository, which makes it natural the version the whole chef repo instead
of individual cookbooks (which is what James is doing). However, cookbooks
have a version in their metadata.rb, which brings up the urge in me to
version them separately, ideally in their own git repo. Someone in Chef Infra (archive)
irc pointed me to librarian [1], which solves this issue by .gitignoring
the cookbooks/ dir and pulling them in from another location, e.g. a
separate git repo.

I have also discussed with some other people, and my current gut feeling
tends to an approach like this:

  • treat X.Y.Z versions with an odd Z as SNAPSHOT versions that might be
    overridden in the Chef server without notice

  • development and testing of SNAPSHOTs is done in a branch. Once it is
    ready to release it gets an even Z, is merged back to master, uploaded to
    the chef Server and frozen

  • allow SNAPSHOTs only in lower environments, e.g. in dev env and for the
    CI build they may be common, but in prod env you never have odd Z versions

  • use librarian [1] so that cookbooks can be versioned independently in
    their own git repos and can be pulled together as needed (e.g. branches for
    CI build, tags for prod env)

  • consider having developer private chef servers running in a local VM
    while developing / testing a SNAPSHOT (if interference becomes an issue)

  • some tooling would be required to automate the release process outlined
    above (a la maven-release-plugin), and to check that cookbooks to be
    released don't have dependencies to SNAPSHOT cookbooks, etc..

One thing that is different in maven is that you have SNAPSHOTs as an
explicit concept in the version number. Having this, you can easily tell
whether that thing is released or in development just by looking at the git
repo. In chef you have this concept only in the chef server (frozen vs not
frozen), but you can't tell from looking at the source. That's why you then
need to come up with conventions like odd Z ,versions etc...

Would introducing explicit SNAPSHOTs to Chef be an option? Or is this just
a special requirement which most people don't have in reality?

What do you think?

Cheers,
Torben

[1] GitHub - applicationsonline/librarian: Librarian - A Framework for Bundlers. Librarian-Chef is at: https://github.com/applicationsonline/librarian-chef.
On Fri, Jan 6, 2012 at 12:26 AM, Zuhaib Siddique <
zuhaib.siddique@leapfile.com> wrote:

Peter,

Your idea sounds similar to what we at Oxygen Cloud use, which is changes
made to Chef cookbooks and checked in to git and at the same time uploaded
to our chef server (we have a small bash script that helps you do both at
the same time, saves a lot of time). At which time using Jenkins we would
kick off an integration server to make sure it runs correctly and deploys
our code correctly. If that passes I manually bump the version and we call
it stable till I make some changes/hack. And then we have a script that
kills all old servers and clear their Node/Client.

The thing I would like to do is is stop deploying servers for quicker Chef
testing use Shef but from my short time testing the thing it seems only
good for unit testing blocks of code but it wont help you do a full test of
a cookbook or recipe. If someone has some tips or guides on doing that I
am very much open to it. Also it seems Shef is not sandbox which mean
changes it makes will apply to your system which is bad as you would need
to run it in a VM which from my point of view more of a PITA then spinning
up new servers and killing them.

Zuhaib
On Jan 5, 2012, at 1:32 PM, Peter Norton wrote:

I'm interested in using chef+gerrit+jenkins to test new cookbooks in
our environment. I'm thinking of a workflow like this:

git checkin of a cookbook.

The cookbook will have an associated unit test suite that jenkins will
run by deploying to a VM (probably an amazon EC2 t1.micro unless a
particular architecture is needed). The cookbook will be run and if
it checks out, the version will be recorded as a candidate to become
elevated to the next environment. I'm thinking of:

sandbox (least reliable)
dev (someone's at least looking at it, may get incorporated in
integration tests)
staging (will be used for pre-producting integration testing)
production (the real thing)

My only question is how to inject version numbers into the metadata in
a sane manner. I can create/manipulate metadata.rb, but I can't see
an easy and clean way to put basic assurances with the X.Y.Z
versioning.

The closest I've gotten is something like this:

checkin foo with metadata.rb's version == 0.1.1. All X.Y.Z releases
where (X%2 == false) are submitted by the dev/user, so can never be
part of dev, staging, or production.

When it's committed and pushed to the git/gerrit server a jenkins job
is kicked off which uploads the new cookbook and launches a server
with this cookbook version explicitly in the run list to verify that
the job passes. If it passes, then the jenkins server increments the
metadata so that version = 0.2.1 (matching Z values) and commits that
to a branch that doesn't require review. That branch then gets
uploaded to the chef server again, but now with the new version
number.

When moving from unit test to integration tests (e.g. with
cluster_chef/metachef, doing an entire environment or facet build)
then the Y value will get incremented to 3, 4, or 5 as testing have
created for veewee...
proceeds.

The thing I'm trying to solve is being able to use the same chef
account for the life of the cookbook so that version X.Y.Z will always
be the same if called in any environment. The issue I'm having is
that fencing off particular minor (Y) revision numbers seems to invite
mis-understandings and casual mistakes.

Has anyone else dealt with this issue, or does anyone have a more
enlightened way of looking at the problem?

-Peter

On Thu, Jan 5, 2012 at 2:15 PM, James Litton james@pagerduty.com
wrote:

The git repo clone on the host is not necessary. It could certainly be
done from a dev box or build server. Doing that from a build server
would make a lot of sense, actually. The reason has just been a matter
of convenience. We don't have to run multi-knife or a similar system
because dev boxes always push to dev, test to test and so on. This
also prevents an accidental push to prod in the case that someone
forgets to branch after push, etc.

versioning of cookbooks has been entirely haphazard(at best) with no
ill effects so far, but this approach has only been in place a month
or so. none of our cookbook depends specify version numbers. The
biggest problem with this approach has been that it makes it more
difficult to commit back to public cookbooks.

No, that situation has not come up. If it were to come up, we would
roll back to that point in the git repository not just the cookbook.

James

On Thu, Jan 5, 2012 at 1:12 AM, Torben Knerr ukio@gmx.de wrote:

Hi James,

thanks for sharing! Interesting to see the different approaches you
can go
for.

Few questions about your approach:

  1. as you describe you have a git repo clone of your chef repository
    on each
    of your chef servers, then check out locally on the chef server and
    then
    knife upload to localhost if i understood correctly. Why do you need a
    git
    repo clone of your chef repository on the chef server? Wouldn't it be
    enough
    to knife upload from an external box (e.g. developer's box, build
    server) to
    either of your chef servers?

  2. as far as i understand your cookbook dependencies are versionless
    during
    delevopment/upload/test/fix cycles and you always release the whole
    chef
    repo instead of single cookbooks. As soon as you do a release, does the
    script you mention update the Version number of all your cookbooks?
    Does it
    at this point in time also add this version number to the cookbook
    depends
    statements (and remove it afterwards for the next dev/upload/test/fix
    cycle)?

  3. do you ever have the situation that you need to set up a node with
    an
    older cookbook version (e.g. for regression testing)?

Cheers,
Torben

On Thu, Jan 5, 2012 at 4:24 AM, James Litton james@pagerduty.com
wrote:

Torben,

Thank you for bringing this up. I have also been puzzling about this
recently.

Right now the process that I am using involves 3 chef servers.

Development:

  • Branch git
  • Test/Deploy on Chef Solo (dedicated dev box with separate role in
    prod chef server)

Test:

  • Checkout branch on test chef server
  • Push entire configuration (via shell script that I wrote)
  • Force a chef-client run on a hosts in test by environment (Load
    Test, staging, preview)
  • Complete testing

Code Review

  • Review the changes in the branch as well as the test cases

Prod (hosts developer environment and production hosts):

  • Merge branch to master
  • git pull on prod
  • Push config via script
  • Force chef-client run here only if required due to external
    requirement

The only issue really is the Test Chef server sharing, but testing can
generally be done quickly by maintaining smaller updates.

cookbook versions number changes aren't as important due to re-upload
of the cookbook with each push and splitting environments across chef
servers

The release process is not currently automated other than the script
that updates all roles, environments, data bags and cookbooks with
each release.

Generally the entire chef repository is treated as a version/release
in order to avoid version dependencies within each cycle.

I'd also love to hear what other people are doing.

James
PagerDuty

On Wed, Jan 4, 2012 at 11:58 AM, Torben Knerr ukio@gmx.de wrote:

Hi everybody,

Happy New Year! :slight_smile:

This is my first post to this mailing list. I'm trying to implement a
proper
workflow for developing cookbooks within a small team. Coming from a
slightly different background (Java, Maven and the like) I'm looking
for
some experience on how to implement a typical develop/test/release
cycle
in
a Chef environment.

So, assuming you are developing cookbooks in a team, how would you go
for
adding a new feature to a cookbook:

(a) develop -> test (using Vagrant/Chef Solo) -> repeat
develop/fix/test
cycle until done -> upload to team's chef server
(b) develop -> test (upload to a private chef server) -> repeat
develop/fix/test cycle until done -> upload to team's chef server
(c) develop -> test (upload directly to team's chef server) ->
repeat
develop/fix/test cycle until done

I'm especially interested in:

  1. at which point in time do you share your sources (git push)?

    • are you working on feature branches, then merge + push when
      done?
    • when do you increase the cookbook version? in the commit where
      you
      merge back to master?
    • when do you create the tag? everytime you commit something on
      master?
  2. at which point in time do you upload the cookbook to the team's
    chef
    server (knife upload)?

    • whenever you want to test something?
    • strictly only after creating a tag (i.e. releases only)?
    • do you always freeze uploaded cookbooks?
    • do you allow to overwrite cookbooks at all?
  3. how do you handle releases?

    • is every upload to the chef repository automatically a
      (public?)
      release for you?
    • do you have the release process (roughly: "run tests -> create
      tag
      ->
      upload cookbook") automated?
    • how do you handle cookbook dependencies when you release? do
      you
      allow
      for versionless cookbook depends?

Eager to hear your experiences. Any help is appreciated!

Cheers,
Torben

Hi,

someone in Chef Infra (archive) irc pointed me to librarian [1], which solves this issue by .gitignoring the cookbooks/ dir and pulling them in from another location, e.g. a separate git repo.

You might want to have a look at git submodules for this (which does exactly that but doesn't require you to do anything with .gitignore or 3rd-party software.

Steffen

On 09.01.2012, at 09:34, Torben Knerr wrote:

Hi James, Peter, Zuhaib,

thanks for sharing your experience and sorry for the delayed response.

One thing I noticed is that we also have our chef repo managed in a git repository, which makes it natural the version the whole chef repo instead of individual cookbooks (which is what James is doing). However, cookbooks have a version in their metadata.rb, which brings up the urge in me to version them separately, ideally in their own git repo. Someone in Chef Infra (archive) irc pointed me to librarian [1], which solves this issue by .gitignoring the cookbooks/ dir and pulling them in from another location, e.g. a separate git repo.
I have also discussed with some other people, and my current gut feeling tends to an approach like this:

  • treat X.Y.Z versions with an odd Z as SNAPSHOT versions that might be overridden in the Chef server without notice

  • development and testing of SNAPSHOTs is done in a branch. Once it is ready to release it gets an even Z, is merged back to master, uploaded to the chef Server and frozen

  • allow SNAPSHOTs only in lower environments, e.g. in dev env and for the CI build they may be common, but in prod env you never have odd Z versions

  • use librarian [1] so that cookbooks can be versioned independently in their own git repos and can be pulled together as needed (e.g. branches for CI build, tags for prod env)

  • consider having developer private chef servers running in a local VM while developing / testing a SNAPSHOT (if interference becomes an issue)

  • some tooling would be required to automate the release process outlined above (a la maven-release-plugin), and to check that cookbooks to be released don't have dependencies to SNAPSHOT cookbooks, etc..

One thing that is different in maven is that you have SNAPSHOTs as an explicit concept in the version number. Having this, you can easily tell whether that thing is released or in development just by looking at the git repo. In chef you have this concept only in the chef server (frozen vs not frozen), but you can't tell from looking at the source. That's why you then need to come up with conventions like odd Z ,versions etc...

Would introducing explicit SNAPSHOTs to Chef be an option? Or is this just a special requirement which most people don't have in reality?

What do you think?

Cheers,
Torben

[1] GitHub - applicationsonline/librarian: Librarian - A Framework for Bundlers. Librarian-Chef is at: https://github.com/applicationsonline/librarian-chef.
On Fri, Jan 6, 2012 at 12:26 AM, Zuhaib Siddique zuhaib.siddique@leapfile.com wrote:
Peter,

Your idea sounds similar to what we at Oxygen Cloud use, which is changes made to Chef cookbooks and checked in to git and at the same time uploaded to our chef server (we have a small bash script that helps you do both at the same time, saves a lot of time). At which time using Jenkins we would kick off an integration server to make sure it runs correctly and deploys our code correctly. If that passes I manually bump the version and we call it stable till I make some changes/hack. And then we have a script that kills all old servers and clear their Node/Client.

The thing I would like to do is is stop deploying servers for quicker Chef testing use Shef but from my short time testing the thing it seems only good for unit testing blocks of code but it wont help you do a full test of a cookbook or recipe. If someone has some tips or guides on doing that I am very much open to it. Also it seems Shef is not sandbox which mean changes it makes will apply to your system which is bad as you would need to run it in a VM which from my point of view more of a PITA then spinning up new servers and killing them.

Zuhaib
On Jan 5, 2012, at 1:32 PM, Peter Norton wrote:

I'm interested in using chef+gerrit+jenkins to test new cookbooks in
our environment. I'm thinking of a workflow like this:

git checkin of a cookbook.

The cookbook will have an associated unit test suite that jenkins will
run by deploying to a VM (probably an amazon EC2 t1.micro unless a
particular architecture is needed). The cookbook will be run and if
it checks out, the version will be recorded as a candidate to become
elevated to the next environment. I'm thinking of:

sandbox (least reliable)
dev (someone's at least looking at it, may get incorporated in
integration tests)
staging (will be used for pre-producting integration testing)
production (the real thing)

My only question is how to inject version numbers into the metadata in
a sane manner. I can create/manipulate metadata.rb, but I can't see
an easy and clean way to put basic assurances with the X.Y.Z
versioning.

The closest I've gotten is something like this:

checkin foo with metadata.rb's version == 0.1.1. All X.Y.Z releases
where (X%2 == false) are submitted by the dev/user, so can never be
part of dev, staging, or production.

When it's committed and pushed to the git/gerrit server a jenkins job
is kicked off which uploads the new cookbook and launches a server
with this cookbook version explicitly in the run list to verify that
the job passes. If it passes, then the jenkins server increments the
metadata so that version = 0.2.1 (matching Z values) and commits that
to a branch that doesn't require review. That branch then gets
uploaded to the chef server again, but now with the new version
number.

When moving from unit test to integration tests (e.g. with
cluster_chef/metachef, doing an entire environment or facet build)
then the Y value will get incremented to 3, 4, or 5 as testing have created for veewee...
proceeds.

The thing I'm trying to solve is being able to use the same chef
account for the life of the cookbook so that version X.Y.Z will always
be the same if called in any environment. The issue I'm having is
that fencing off particular minor (Y) revision numbers seems to invite
mis-understandings and casual mistakes.

Has anyone else dealt with this issue, or does anyone have a more
enlightened way of looking at the problem?

-Peter

On Thu, Jan 5, 2012 at 2:15 PM, James Litton james@pagerduty.com wrote:

The git repo clone on the host is not necessary. It could certainly be
done from a dev box or build server. Doing that from a build server
would make a lot of sense, actually. The reason has just been a matter
of convenience. We don't have to run multi-knife or a similar system
because dev boxes always push to dev, test to test and so on. This
also prevents an accidental push to prod in the case that someone
forgets to branch after push, etc.

versioning of cookbooks has been entirely haphazard(at best) with no
ill effects so far, but this approach has only been in place a month
or so. none of our cookbook depends specify version numbers. The
biggest problem with this approach has been that it makes it more
difficult to commit back to public cookbooks.

No, that situation has not come up. If it were to come up, we would
roll back to that point in the git repository not just the cookbook.

James

On Thu, Jan 5, 2012 at 1:12 AM, Torben Knerr ukio@gmx.de wrote:

Hi James,

thanks for sharing! Interesting to see the different approaches you can go
for.

Few questions about your approach:

  1. as you describe you have a git repo clone of your chef repository on each
    of your chef servers, then check out locally on the chef server and then
    knife upload to localhost if i understood correctly. Why do you need a git
    repo clone of your chef repository on the chef server? Wouldn't it be enough
    to knife upload from an external box (e.g. developer's box, build server) to
    either of your chef servers?

  2. as far as i understand your cookbook dependencies are versionless during
    delevopment/upload/test/fix cycles and you always release the whole chef
    repo instead of single cookbooks. As soon as you do a release, does the
    script you mention update the Version number of all your cookbooks? Does it
    at this point in time also add this version number to the cookbook depends
    statements (and remove it afterwards for the next dev/upload/test/fix
    cycle)?

  3. do you ever have the situation that you need to set up a node with an
    older cookbook version (e.g. for regression testing)?

Cheers,
Torben

On Thu, Jan 5, 2012 at 4:24 AM, James Litton james@pagerduty.com wrote:

Torben,

Thank you for bringing this up. I have also been puzzling about this
recently.

Right now the process that I am using involves 3 chef servers.

Development:

  • Branch git
  • Test/Deploy on Chef Solo (dedicated dev box with separate role in
    prod chef server)

Test:

  • Checkout branch on test chef server
  • Push entire configuration (via shell script that I wrote)
  • Force a chef-client run on a hosts in test by environment (Load
    Test, staging, preview)
  • Complete testing

Code Review

  • Review the changes in the branch as well as the test cases

Prod (hosts developer environment and production hosts):

  • Merge branch to master
  • git pull on prod
  • Push config via script
  • Force chef-client run here only if required due to external requirement

The only issue really is the Test Chef server sharing, but testing can
generally be done quickly by maintaining smaller updates.

cookbook versions number changes aren't as important due to re-upload
of the cookbook with each push and splitting environments across chef
servers

The release process is not currently automated other than the script
that updates all roles, environments, data bags and cookbooks with
each release.

Generally the entire chef repository is treated as a version/release
in order to avoid version dependencies within each cycle.

I'd also love to hear what other people are doing.

James
PagerDuty

On Wed, Jan 4, 2012 at 11:58 AM, Torben Knerr ukio@gmx.de wrote:

Hi everybody,

Happy New Year! :slight_smile:

This is my first post to this mailing list. I'm trying to implement a
proper
workflow for developing cookbooks within a small team. Coming from a
slightly different background (Java, Maven and the like) I'm looking for
some experience on how to implement a typical develop/test/release cycle
in
a Chef environment.

So, assuming you are developing cookbooks in a team, how would you go
for
adding a new feature to a cookbook:

(a) develop -> test (using Vagrant/Chef Solo) -> repeat
develop/fix/test
cycle until done -> upload to team's chef server
(b) develop -> test (upload to a private chef server) -> repeat
develop/fix/test cycle until done -> upload to team's chef server
(c) develop -> test (upload directly to team's chef server) -> repeat
develop/fix/test cycle until done

I'm especially interested in:

  1. at which point in time do you share your sources (git push)?

    • are you working on feature branches, then merge + push when done?
    • when do you increase the cookbook version? in the commit where you
      merge back to master?
    • when do you create the tag? everytime you commit something on
      master?
  2. at which point in time do you upload the cookbook to the team's chef
    server (knife upload)?

    • whenever you want to test something?
    • strictly only after creating a tag (i.e. releases only)?
    • do you always freeze uploaded cookbooks?
    • do you allow to overwrite cookbooks at all?
  3. how do you handle releases?

    • is every upload to the chef repository automatically a (public?)
      release for you?
    • do you have the release process (roughly: "run tests -> create tag
      ->
      upload cookbook") automated?
    • how do you handle cookbook dependencies when you release? do you
      allow
      for versionless cookbook depends?

Eager to hear your experiences. Any help is appreciated!

Cheers,
Torben

Hi Steffen,

thanks for pointing this out, I will definitely consider that. The only
difference I can see so far is that with librarian you can also pull in
cookbooks from Chef REST API, not only from a git repository. For librarian
you need an extra gem... hmmm... interesting tradeoff...

Torben

On Mon, Jan 9, 2012 at 10:58 AM, Steffen Gebert st+opscode@st-g.de wrote:

Hi,

someone in Chef Infra (archive) irc pointed me to librarian [1], which solves this
issue by .gitignoring the cookbooks/ dir and pulling them in from another
location, e.g. a separate git repo.

You might want to have a look at git submodules for this (which does
exactly that but doesn't require you to do anything with .gitignore or
3rd-party software.

Steffen

On 09.01.2012, at 09:34, Torben Knerr wrote:

Hi James, Peter, Zuhaib,

thanks for sharing your experience and sorry for the delayed response.

One thing I noticed is that we also have our chef repo managed in a git
repository, which makes it natural the version the whole chef repo instead
of individual cookbooks (which is what James is doing). However, cookbooks
have a version in their metadata.rb, which brings up the urge in me to
version them separately, ideally in their own git repo. Someone in Chef Infra (archive)
irc pointed me to librarian [1], which solves this issue by .gitignoring
the cookbooks/ dir and pulling them in from another location, e.g. a
separate git repo.
I have also discussed with some other people, and my current gut feeling
tends to an approach like this:

  • treat X.Y.Z versions with an odd Z as SNAPSHOT versions that might be
    overridden in the Chef server without notice

  • development and testing of SNAPSHOTs is done in a branch. Once it is
    ready to release it gets an even Z, is merged back to master, uploaded to
    the chef Server and frozen

  • allow SNAPSHOTs only in lower environments, e.g. in dev env and for
    the CI build they may be common, but in prod env you never have odd Z
    versions

  • use librarian [1] so that cookbooks can be versioned independently in
    their own git repos and can be pulled together as needed (e.g. branches for
    CI build, tags for prod env)

  • consider having developer private chef servers running in a local VM
    while developing / testing a SNAPSHOT (if interference becomes an issue)

  • some tooling would be required to automate the release process
    outlined above (a la maven-release-plugin), and to check that cookbooks to
    be released don't have dependencies to SNAPSHOT cookbooks, etc..

One thing that is different in maven is that you have SNAPSHOTs as an
explicit concept in the version number. Having this, you can easily tell
whether that thing is released or in development just by looking at the git
repo. In chef you have this concept only in the chef server (frozen vs not
frozen), but you can't tell from looking at the source. That's why you then
need to come up with conventions like odd Z ,versions etc...

Would introducing explicit SNAPSHOTs to Chef be an option? Or is this
just a special requirement which most people don't have in reality?

What do you think?

Cheers,
Torben

[1] GitHub - applicationsonline/librarian: Librarian - A Framework for Bundlers. Librarian-Chef is at: https://github.com/applicationsonline/librarian-chef.
On Fri, Jan 6, 2012 at 12:26 AM, Zuhaib Siddique <
zuhaib.siddique@leapfile.com> wrote:
Peter,

Your idea sounds similar to what we at Oxygen Cloud use, which is
changes made to Chef cookbooks and checked in to git and at the same time
uploaded to our chef server (we have a small bash script that helps you do
both at the same time, saves a lot of time). At which time using Jenkins
we would kick off an integration server to make sure it runs correctly and
deploys our code correctly. If that passes I manually bump the version and
we call it stable till I make some changes/hack. And then we have a script
that kills all old servers and clear their Node/Client.

The thing I would like to do is is stop deploying servers for quicker
Chef testing use Shef but from my short time testing the thing it seems
only good for unit testing blocks of code but it wont help you do a full
test of a cookbook or recipe. If someone has some tips or guides on doing
that I am very much open to it. Also it seems Shef is not sandbox which
mean changes it makes will apply to your system which is bad as you would
need to run it in a VM which from my point of view more of a PITA then
spinning up new servers and killing them.

Zuhaib
On Jan 5, 2012, at 1:32 PM, Peter Norton wrote:

I'm interested in using chef+gerrit+jenkins to test new cookbooks in
our environment. I'm thinking of a workflow like this:

git checkin of a cookbook.

The cookbook will have an associated unit test suite that jenkins will
run by deploying to a VM (probably an amazon EC2 t1.micro unless a
particular architecture is needed). The cookbook will be run and if
it checks out, the version will be recorded as a candidate to become
elevated to the next environment. I'm thinking of:

sandbox (least reliable)
dev (someone's at least looking at it, may get incorporated in
integration tests)
staging (will be used for pre-producting integration testing)
production (the real thing)

My only question is how to inject version numbers into the metadata in
a sane manner. I can create/manipulate metadata.rb, but I can't see
an easy and clean way to put basic assurances with the X.Y.Z
versioning.

The closest I've gotten is something like this:

checkin foo with metadata.rb's version == 0.1.1. All X.Y.Z releases
where (X%2 == false) are submitted by the dev/user, so can never be
part of dev, staging, or production.

When it's committed and pushed to the git/gerrit server a jenkins job
is kicked off which uploads the new cookbook and launches a server
with this cookbook version explicitly in the run list to verify that
the job passes. If it passes, then the jenkins server increments the
metadata so that version = 0.2.1 (matching Z values) and commits that
to a branch that doesn't require review. That branch then gets
uploaded to the chef server again, but now with the new version
number.

When moving from unit test to integration tests (e.g. with
cluster_chef/metachef, doing an entire environment or facet build)
then the Y value will get incremented to 3, 4, or 5 as testing have
created for veewee...
proceeds.

The thing I'm trying to solve is being able to use the same chef
account for the life of the cookbook so that version X.Y.Z will always
be the same if called in any environment. The issue I'm having is
that fencing off particular minor (Y) revision numbers seems to invite
mis-understandings and casual mistakes.

Has anyone else dealt with this issue, or does anyone have a more
enlightened way of looking at the problem?

-Peter

On Thu, Jan 5, 2012 at 2:15 PM, James Litton james@pagerduty.com
wrote:

The git repo clone on the host is not necessary. It could certainly be
done from a dev box or build server. Doing that from a build server
would make a lot of sense, actually. The reason has just been a matter
of convenience. We don't have to run multi-knife or a similar system
because dev boxes always push to dev, test to test and so on. This
also prevents an accidental push to prod in the case that someone
forgets to branch after push, etc.

versioning of cookbooks has been entirely haphazard(at best) with no
ill effects so far, but this approach has only been in place a month
or so. none of our cookbook depends specify version numbers. The
biggest problem with this approach has been that it makes it more
difficult to commit back to public cookbooks.

No, that situation has not come up. If it were to come up, we would
roll back to that point in the git repository not just the cookbook.

James

On Thu, Jan 5, 2012 at 1:12 AM, Torben Knerr ukio@gmx.de wrote:

Hi James,

thanks for sharing! Interesting to see the different approaches you
can go
for.

Few questions about your approach:

  1. as you describe you have a git repo clone of your chef repository
    on each
    of your chef servers, then check out locally on the chef server and
    then
    knife upload to localhost if i understood correctly. Why do you need
    a git
    repo clone of your chef repository on the chef server? Wouldn't it
    be enough
    to knife upload from an external box (e.g. developer's box, build
    server) to
    either of your chef servers?

  2. as far as i understand your cookbook dependencies are versionless
    during
    delevopment/upload/test/fix cycles and you always release the whole
    chef
    repo instead of single cookbooks. As soon as you do a release, does
    the
    script you mention update the Version number of all your cookbooks?
    Does it
    at this point in time also add this version number to the cookbook
    depends
    statements (and remove it afterwards for the next dev/upload/test/fix
    cycle)?

  3. do you ever have the situation that you need to set up a node
    with an
    older cookbook version (e.g. for regression testing)?

Cheers,
Torben

On Thu, Jan 5, 2012 at 4:24 AM, James Litton james@pagerduty.com
wrote:

Torben,

Thank you for bringing this up. I have also been puzzling about
this
recently.

Right now the process that I am using involves 3 chef servers.

Development:

  • Branch git
  • Test/Deploy on Chef Solo (dedicated dev box with separate role in
    prod chef server)

Test:

  • Checkout branch on test chef server
  • Push entire configuration (via shell script that I wrote)
  • Force a chef-client run on a hosts in test by environment (Load
    Test, staging, preview)
  • Complete testing

Code Review

  • Review the changes in the branch as well as the test cases

Prod (hosts developer environment and production hosts):

  • Merge branch to master
  • git pull on prod
  • Push config via script
  • Force chef-client run here only if required due to external
    requirement

The only issue really is the Test Chef server sharing, but testing
can
generally be done quickly by maintaining smaller updates.

cookbook versions number changes aren't as important due to
re-upload
of the cookbook with each push and splitting environments across
chef
servers

The release process is not currently automated other than the script
that updates all roles, environments, data bags and cookbooks with
each release.

Generally the entire chef repository is treated as a version/release
in order to avoid version dependencies within each cycle.

I'd also love to hear what other people are doing.

James
PagerDuty

On Wed, Jan 4, 2012 at 11:58 AM, Torben Knerr ukio@gmx.de wrote:

Hi everybody,

Happy New Year! :slight_smile:

This is my first post to this mailing list. I'm trying to
implement a
proper
workflow for developing cookbooks within a small team. Coming from
a
slightly different background (Java, Maven and the like) I'm
looking for
some experience on how to implement a typical develop/test/release
cycle
in
a Chef environment.

So, assuming you are developing cookbooks in a team, how would you
go
for
adding a new feature to a cookbook:

(a) develop -> test (using Vagrant/Chef Solo) -> repeat
develop/fix/test
cycle until done -> upload to team's chef server
(b) develop -> test (upload to a private chef server) -> repeat
develop/fix/test cycle until done -> upload to team's chef server
(c) develop -> test (upload directly to team's chef server) ->
repeat
develop/fix/test cycle until done

I'm especially interested in:

  1. at which point in time do you share your sources (git push)?

    • are you working on feature branches, then merge + push when
      done?
    • when do you increase the cookbook version? in the commit
      where you
      merge back to master?
    • when do you create the tag? everytime you commit something on
      master?
  2. at which point in time do you upload the cookbook to the
    team's chef
    server (knife upload)?

    • whenever you want to test something?
    • strictly only after creating a tag (i.e. releases only)?
    • do you always freeze uploaded cookbooks?
    • do you allow to overwrite cookbooks at all?
  3. how do you handle releases?

    • is every upload to the chef repository automatically a
      (public?)
      release for you?
    • do you have the release process (roughly: "run tests ->
      create tag
      ->
      upload cookbook") automated?
    • how do you handle cookbook dependencies when you release? do
      you
      allow
      for versionless cookbook depends?

Eager to hear your experiences. Any help is appreciated!

Cheers,
Torben