Multiple versions of same cookbook in one chef git repo

Chef Infra (archive)
1

I’m dealing with some interesting configurations where one git repository is used for production and testing environments on a project. I can set up a different branch or tag for each environment, and set up a chef server for each environment, and keep testing or variant cookbooks separated that way. But it’s awkward.

I’d love to be able to use cookbook versions, specified for each environment as needed, to load or change the version of the cookbook. So “example” cookbook 1.0.0 gets loaded on most hosts, for example, but on testing hosts for the same server “example” cookbook “0.0.9” is specified and used. The difficulty is being able to record and load the distinct cookbooks from the same working git repository. If I try publishing them as “cookbooks/example-0.0.9” and “cookbooks/example-0.1.0”, they get uploaded as “example-0.0.9”, with version “0.0.9”, and that breaks things seriously.

Does anyone know how to publish and automatically upload distinct versions cookbooks of the same name in a working chef server environment, under git?

–
Nico Kadel-Garcia
Senior Systems Consultant
Email: nkadelgarcia-consultant@scholastic.com
Cell Phone: +1.339.368.2428

2

Chef 11.6.0 introduced the versioned_cookbooks knife configuration:
http://docs.opscode.com/release_notes.html#knife-rb-settings

Have not tried it yet, but that might help you?

On Thu, Aug 15, 2013 at 1:31 PM, Kadel-Garcia, Nico <
NKadelGarcia-consultant@scholastic.com> wrote:

I’m dealing with some interesting configurations where one git
repository is used for production and testing environments on a project. I
can set up a different branch or tag for each environment, and set up a
chef server for each environment, and keep testing or variant cookbooks
separated that way. But it’s awkward.****

I’d love to be able to use cookbook versions, specified for each
environment as needed, to load or change the version of the cookbook. So
“example” cookbook 1.0.0 gets loaded on most hosts, for example, but on
testing hosts for the same server “example” cookbook “0.0.9” is specified
and used. The difficulty is being able to record and load the distinct
cookbooks from the same working git repository. If I try publishing them as
“cookbooks/example-0.0.9” and “cookbooks/example-0.1.0”, they get uploaded
as “example-0.0.9”, with version “0.0.9”, and that breaks things seriously.

Does anyone know how to publish and automatically upload distinct versions
cookbooks of the same name in a working chef server environment, under git?

--
Nico Kadel-Garcia
Senior Systems Consultant
Email: nkadelgarcia-consultant@scholastic.com
Cell Phone: +1.339.368.2428****

3

On Aug 15, 2013, at 6:31 AM, "Kadel-Garcia, Nico" NKadelGarcia-consultant@Scholastic.com wrote:

I’d love to be able to use cookbook versions, specified for each environment as needed, to load or change the version of the cookbook. So “example” cookbook 1.0.0 gets loaded on most hosts, for example, but on testing hosts for the same server “example” cookbook “0.0.9” is specified and used. The difficulty is being able to record and load the distinct cookbooks from the same working git repository. If I try publishing them as “cookbooks/example-0.0.9” and “cookbooks/example-0.1.0”, they get uploaded as “example-0.0.9”, with version “0.0.9”, and that breaks things seriously.

Does anyone know how to publish and automatically upload distinct versions cookbooks of the same name in a working chef server environment, under git?

What you describe should "just work" by pinning versions in your environments or roles, and then uploading whatever versions are appropriate with "knife cookbook upload". The chef-server will keep track of what versions of the cookbook have been uploaded, and which versions are associated with which environments and roles.

The only way that git factors into this equation is as the basic source code management system, and you update versions of the cookbook as specified in metadata.rb, check that code into git, and then upload that to the server. IMO, the "spork" plugin for knife makes this kind of workflow a lot easier -- it can automate the git commit, the knife cookbook upload, etc.... It might even be able to auto-increment cookbook versions, but I'm not sure about that.

--
Brad Knowles brad@shub-internet.org
LinkedIn Profile: http://tinyurl.com/y8kpxu

4

Looking at it again, it looks as it is only useful for backup via knife cookbook download.

You should consider to not manage all your cookbooks in one monolithic git
repo, rather use individual git repos per cookbook. You then need a
cookbook dependency manager (e.g. Bershelf or librarian-chef) to pull in
the cookbooks in the specific versions you need them. Speaking of the
community, that's the commonly accepted and recommended way I would say.

Have a look at http://berkshelf.com/

HTH, Torben

On Thu, Aug 15, 2013 at 3:41 PM, Torben Knerr ukio@gmx.de wrote:

Chef 11.6.0 introduced the versioned_cookbooks knife configuration:
Chef Infra Client Release Notes

Have not tried it yet, but that might help you?

On Thu, Aug 15, 2013 at 1:31 PM, Kadel-Garcia, Nico <
NKadelGarcia-consultant@scholastic.com> wrote:

I’m dealing with some interesting configurations where one git
repository is used for production and testing environments on a project. I
can set up a different branch or tag for each environment, and set up a
chef server for each environment, and keep testing or variant cookbooks
separated that way. But it’s awkward.****

I’d love to be able to use cookbook versions, specified for each
environment as needed, to load or change the version of the cookbook. So
“example” cookbook 1.0.0 gets loaded on most hosts, for example, but on
testing hosts for the same server “example” cookbook “0.0.9” is specified
and used. The difficulty is being able to record and load the distinct
cookbooks from the same working git repository. If I try publishing them as
“cookbooks/example-0.0.9” and “cookbooks/example-0.1.0”, they get uploaded
as “example-0.0.9”, with version “0.0.9”, and that breaks things seriously.

Does anyone know how to publish and automatically upload distinct
versions cookbooks of the same name in a working chef server environment,
under git?****

--
Nico Kadel-Garcia
Senior Systems Consultant
Email: nkadelgarcia-consultant@scholastic.com
Cell Phone: +1.339.368.2428****

5

Torben wrote:

You should consider to not manage all your cookbooks in one monolithic git repo, rather use individual git repos

Doing that can get interesting. While it's possible to have an indeterminate number of versions of the same cookbook, scattered in different upstrem https or local filesystem locations, I'm afraid that tools like "librarian-chef" get very confused by this when they try to copy the cookbooks to "vendor_cookbooks or when they try to render dependencies in Cheffile.lock".

I really wish I could just rename the cookbook, such as "frenchfood", to store in "cookbooks/freenchfood-1.0", "cookbooks/frenchfood-1.1", "cookbooks/frenchfood-1.2", set the metadata.rb appropriately, and librarian-chef would just do the right tihings and load the new cookbooks based on metada.rb, not on the directory name. But I'm afraid that trying this simply yields up new cookbooks with names like "frenchfood-1.2", version 1.2, and "frenchfood-1.0", version 1.0. That way lies madness.

I'm also afraid that the same chef server is handling testing and production versoins of the same cookbooks. That makes it more awkward to run separate git repos: for the different environments, and almost demands running extra chef servers for those extra environments. It adds.... complexity to do that.

From: Torben Knerr [ukio@gmx.de]
Sent: Thursday, August 15, 2013 11:15 AM
To: chef@lists.opscode.com
Subject: [chef] Re: Multiple versions of same cookbook in one chef git repo

Looking at it again, it looks as it is only useful for backup via knife cookbook download.

You should consider to not manage all your cookbooks in one monolithic git repo, rather use individual git repos per cookbook. You then need a cookbook dependency manager (e.g. Bershelf or librarian-chef) to pull in the cookbooks in the specific versions you need them. Speaking of the community, that's the commonly accepted and recommended way I would say.

Have a look at http://berkshelf.com/

HTH, Torben

On Thu, Aug 15, 2013 at 3:41 PM, Torben Knerr <ukio@gmx.demailto:ukio@gmx.de> wrote:
Chef 11.6.0 introduced the versioned_cookbooks knife configuration:
Chef Infra Client Release Notes

Have not tried it yet, but that might help you?

On Thu, Aug 15, 2013 at 1:31 PM, Kadel-Garcia, Nico <NKadelGarcia-consultant@scholastic.commailto:NKadelGarcia-consultant@scholastic.com> wrote:
I’m dealing with some interesting configurations where one git repository is used for production and testing environments on a project. I can set up a different branch or tag for each environment, and set up a chef server for each environment, and keep testing or variant cookbooks separated that way. But it’s awkward.

I’d love to be able to use cookbook versions, specified for each environment as needed, to load or change the version of the cookbook. So “example” cookbook 1.0.0 gets loaded on most hosts, for example, but on testing hosts for the same server “example” cookbook “0.0.9” is specified and used. The difficulty is being able to record and load the distinct cookbooks from the same working git repository. If I try publishing them as “cookbooks/example-0.0.9” and “cookbooks/example-0.1.0”, they get uploaded as “example-0.0.9”, with version “0.0.9”, and that breaks things seriously.

Does anyone know how to publish and automatically upload distinct versions cookbooks of the same name in a working chef server environment, under git?

--
Nico Kadel-Garcia
Senior Systems Consultant
Email: nkadelgarcia-consultant@scholastic.commailto:nkadelgarcia-consultant@scholastic.com
Cell Phone: +1.339.368.2428tel:%2B1.339.368.2428