How to manage cookbook versions more efficiently?

Chris_F · April 23, 2015, 2:29pm

I use role cookbooks to pin down versions of the specific versions of the cookbooks they use. Since I have 25 nodes in my product and each node has a role, I have at least 25 role cookbooks. I just then add my role cookbooks to my nodes’ run list. For example I have: the following. I DO want to pin a particular cookbook version in my role cookbooks.

cookbook_role1/metadata.rb
depends ‘cookbook1’, ‘=1.0.0’ # Don’t want '~> 1.0.0’
depends ‘cookbook3’, '=1.0.0’
depends ‘cookbook4’, ‘=1.0.0’

cookbook_role2/metadata.rb
depends ‘cookbook1’, '=1.0.0’
depends ‘cookbook5’, '=1.0.0’
depends ‘cookbook6’, ‘=1.0.0’

My problem is, when I update cookboo1’s version, I have to go to EACH role and update its version there. I also update the role cookbooks versions.

What’s a more efficient way to do this, avoiding ‘~> x.y.z’ in the role cookbooks. I was thinking that since the metadata.rb files are just ruby file, I can do

cookbook_role1/metadata.rb
depends ‘cookbook1’, cookbook_versions[‘cookbook1’]
depends ‘cookbook3’, cookbook_versions[‘cookbook3’]
depends ‘cookbook4’, cookbook_versison[‘cookbook4’]

and have a central *.rb file that has

$cookbook_versions = {
‘cookbook1’ => ‘= 1.0.0’,
‘cookbook2’ => ‘= 1.0.0’,
‘cookbook4’ => ‘=1.0.0’
}

That is, I have a central location I can just update my cookbook versions, and my cookbooks’ metatdata.rb files will be updated accordingly.

So far I’m not able to make this hokey idea work.

Any ideas on how to alleviate this situation?

Chris

Nico_Kadel_Garcia · April 24, 2015, 3:41pm

From: Fouts, Chris [mailto:Chris.Fouts@Sensus.com]
Sent: Thursday, April 23, 2015 10:29 AM
To: chef@lists.opscode.com
Subject: [chef] How to manage cookbook versions more efficiently?

I use role cookbooks to pin down versions of the specific versions of the cookbooks they use. Since I have 25 nodes in my product and each node has a role, I have at least 25 role cookbooks. I just then add my role cookbooks to my nodes' run list. For example I have: the following. I DO want to pin a particular cookbook version in my role cookbooks.

Any ideas on how to alleviate this situation?

For complete consistency, you have to specify every single cookbook, including all dependencies for and on your desired cookbooks in your roles or run_lists. Your role based approach gets very nasty if you start mixing roles, and mixing conflicting cookbook versions or unintended dependencies. There were big problems when the yum and mysql cookbooks were updated and were incompatible with many older, stable, tested, production cookbooks that relied on them.

This is one of my major reasons for giving up on the "chef-server/chef-client" model, and preferring "chef-solo" for small environments. I can lock down every single cookbook in Berkshelf in a much more controlled fashion than mixing and matching and unweaving roles, cookbook, or environment wrappers, and I can apply an updated or testing cookbook on a single host with a locally updated or git branched Berkshelf.lock without potentially inflicting it on any other unexpected host. There are costs: using chefdk is a fast way to get a full Berkshelf enabled chef-solo environment, but it's not pre-built for all operating systems that Chef supports.

Nico Kadel-Garcia
Lead DevOps Engineer
nkadel@skyhookwireless.com

Chris_F · April 24, 2015, 4:25pm

" For complete consistency, you have to specify every single cookbook, including all dependencies for and on your desired cookbooks in your roles or run_lists. Your role based approach gets very nasty if you start mixing roles, and mixing conflicting cookbook versions or unintended dependencies. There were big problems when the yum and mysql cookbooks were updated and were incompatible with many older, stable, tested, production cookbooks that relied on them."

That's why I asked the question, you just seconded it.

Chris

-----Original Message-----
From: Nico Kadel-Garcia [mailto:nkadel@skyhookwireless.com]
Sent: Friday, April 24, 2015 11:41 AM
To: chef@lists.opscode.com
Subject: [chef] RE: How to manage cookbook versions more efficiently?

From: Fouts, Chris [mailto:Chris.Fouts@Sensus.com]
Sent: Thursday, April 23, 2015 10:29 AM
To: chef@lists.opscode.com
Subject: [chef] How to manage cookbook versions more efficiently?

I use role cookbooks to pin down versions of the specific versions of the cookbooks they use. Since I have 25 nodes in my product and each node has a role, I have at least 25 role cookbooks. I just then add my role cookbooks to my nodes' run list. For example I have: the following. I DO want to pin a particular cookbook version in my role cookbooks.

Any ideas on how to alleviate this situation?

For complete consistency, you have to specify every single cookbook, including all dependencies for and on your desired cookbooks in your roles or run_lists. Your role based approach gets very nasty if you start mixing roles, and mixing conflicting cookbook versions or unintended dependencies. There were big problems when the yum and mysql cookbooks were updated and were incompatible with many older, stable, tested, production cookbooks that relied on them.

This is one of my major reasons for giving up on the "chef-server/chef-client" model, and preferring "chef-solo" for small environments. I can lock down every single cookbook in Berkshelf in a much more controlled fashion than mixing and matching and unweaving roles, cookbook, or environment wrappers, and I can apply an updated or testing cookbook on a single host with a locally updated or git branched Berkshelf.lock without potentially inflicting it on any other unexpected host. There are costs: using chefdk is a fast way to get a full Berkshelf enabled chef-solo environment, but it's not pre-built for all operating systems that Chef supports.

Nico Kadel-Garcia
Lead DevOps Engineer
nkadel@skyhookwireless.com

Nathen_Harvey3 · April 24, 2015, 4:48pm

I'm not sure I understand why you have a single role for each node in your
infrastructure. Could you help me understand that? Or do you mean you
have 25
different types of nodes in your infrastructure where each type might have
N
instances?

Have you considered using Environments [1] for pinning your cookbook
versions?
I think this might be a more sustainable approach for what you've described.

-Nathen

1: About Environments

On Fri, Apr 24, 2015 at 12:25 PM, Fouts, Chris Chris.Fouts@sensus.com
wrote:

" For complete consistency, you have to specify every single cookbook,
including all dependencies for and on your desired cookbooks in your roles
or run_lists. Your role based approach gets very nasty if you start mixing
roles, and mixing conflicting cookbook versions or unintended dependencies.
There were big problems when the yum and mysql cookbooks were updated and
were incompatible with many older, stable, tested, production cookbooks
that relied on them."

That's why I asked the question, you just seconded it.

Chris

-----Original Message-----
From: Nico Kadel-Garcia [mailto:nkadel@skyhookwireless.com]
Sent: Friday, April 24, 2015 11:41 AM
To: chef@lists.opscode.com
Subject: [chef] RE: How to manage cookbook versions more efficiently?

From: Fouts, Chris [mailto:Chris.Fouts@Sensus.com]
Sent: Thursday, April 23, 2015 10:29 AM
To: chef@lists.opscode.com
Subject: [chef] How to manage cookbook versions more efficiently?

I use role cookbooks to pin down versions of the specific versions of
the cookbooks they use. Since I have 25 nodes in my product and each node
has a role, I have at least 25 role cookbooks. I just then add my role
cookbooks to my nodes' run list. For example I have: the following. I DO
want to pin a particular cookbook version in my role cookbooks.

Any ideas on how to alleviate this situation?

For complete consistency, you have to specify every single cookbook,
including all dependencies for and on your desired cookbooks in your roles
or run_lists. Your role based approach gets very nasty if you start mixing
roles, and mixing conflicting cookbook versions or unintended dependencies.
There were big problems when the yum and mysql cookbooks were updated and
were incompatible with many older, stable, tested, production cookbooks
that relied on them.

This is one of my major reasons for giving up on the
"chef-server/chef-client" model, and preferring "chef-solo" for small
environments. I can lock down every single cookbook in Berkshelf in a much
more controlled fashion than mixing and matching and unweaving roles,
cookbook, or environment wrappers, and I can apply an updated or testing
cookbook on a single host with a locally updated or git branched
Berkshelf.lock without potentially inflicting it on any other unexpected
host. There are costs: using chefdk is a fast way to get a full Berkshelf
enabled chef-solo environment, but it's not pre-built for all operating
systems that Chef supports.

Nico Kadel-Garcia
Lead DevOps Engineer
nkadel@skyhookwireless.com

Chris_F · April 24, 2015, 6:02pm

Here’s an abbreviated description of our product.

Say we have two different applications, say, a “cacheserver” and a “flexip” application, running in separate RHEL VMs. For each, I created a “cacheserver_role” and a “flexip_role” cookbooks to install their respective application. They do however, share some “common” books, say a “java” cookbook. So role cookbooks’ metadata.rb files look like

cacheserver_role
name ‘cacheserver_role’
depends ‘java’, ‘= 1.0.0’
depends ‘cacheserver’, ‘= 1.0.0’

flexip_role
name ‘flexip_role’
depends ‘java’, ‘= 1.0.0’
depends ‘flexip’, ‘= 1.0.0’

I want to purposely “pin” a specific version of the aggregate cookbooks in each role cookbook. This is an important requirement.

As you can see, if I want to update my java cookbook, I have to modify (at least) two cookbooks. In my real product, I have more than just two role cookbooks.

I’m purposely moving away from environment.json and role.json files because they are NOT versioned, that is, they do NOT have a metadata.rb file.

Chris

From: Nathen Harvey [mailto:nharvey@chef.io]
Sent: Friday, April 24, 2015 12:48 PM
To: chef@lists.opscode.com
Subject: [chef] Re: RE: How to manage cookbook versions more efficiently?

I'm not sure I understand why you have a single role for each node in your
infrastructure. Could you help me understand that? Or do you mean you have 25
different types of nodes in your infrastructure where each type might have N
instances?

Have you considered using Environments [1] for pinning your cookbook versions?
I think this might be a more sustainable approach for what you've described.

-Nathen

1: About Environments

On Fri, Apr 24, 2015 at 12:25 PM, Fouts, Chris <Chris.Fouts@sensus.com mailto:Chris.Fouts@sensus.com> wrote:
" For complete consistency, you have to specify every single cookbook, including all dependencies for and on your desired cookbooks in your roles or run_lists. Your role based approach gets very nasty if you start mixing roles, and mixing conflicting cookbook versions or unintended dependencies. There were big problems when the yum and mysql cookbooks were updated and were incompatible with many older, stable, tested, production cookbooks that relied on them."

That's why I asked the question, you just seconded it.

Chris

-----Original Message-----
From: Nico Kadel-Garcia [mailto:nkadel@skyhookwireless.com mailto:nkadel@skyhookwireless.com]
Sent: Friday, April 24, 2015 11:41 AM
To: chef@lists.opscode.com mailto:chef@lists.opscode.com
Subject: [chef] RE: How to manage cookbook versions more efficiently?

From: Fouts, Chris [mailto:Chris.Fouts@Sensus.com mailto:Chris.Fouts@Sensus.com]
Sent: Thursday, April 23, 2015 10:29 AM
To: chef@lists.opscode.com mailto:chef@lists.opscode.com
Subject: [chef] How to manage cookbook versions more efficiently?

I use role cookbooks to pin down versions of the specific versions of the cookbooks they use. Since I have 25 nodes in my product and each node has a role, I have at least 25 role cookbooks. I just then add my role cookbooks to my nodes' run list. For example I have: the following. I DO want to pin a particular cookbook version in my role cookbooks.

Any ideas on how to alleviate this situation?

For complete consistency, you have to specify every single cookbook, including all dependencies for and on your desired cookbooks in your roles or run_lists. Your role based approach gets very nasty if you start mixing roles, and mixing conflicting cookbook versions or unintended dependencies. There were big problems when the yum and mysql cookbooks were updated and were incompatible with many older, stable, tested, production cookbooks that relied on them.

This is one of my major reasons for giving up on the "chef-server/chef-client" model, and preferring "chef-solo" for small environments. I can lock down every single cookbook in Berkshelf in a much more controlled fashion than mixing and matching and unweaving roles, cookbook, or environment wrappers, and I can apply an updated or testing cookbook on a single host with a locally updated or git branched Berkshelf.lock without potentially inflicting it on any other unexpected host. There are costs: using chefdk is a fast way to get a full Berkshelf enabled chef-solo environment, but it's not pre-built for all operating systems that Chef supports.

Nico Kadel-Garcia
Lead DevOps Engineer
nkadel@skyhookwireless.com mailto:nkadel@skyhookwireless.com

bbaugher · April 24, 2015, 6:10pm

One thing I've played around with is have a cookbook's metadata.rb read a
Berksfile.lock which looks something like this,

require 'berkshelf'
berksfile = ::Berkshelf::Berksfile.new 'Berksfile'
berksfile.list.each do |dependency|
depends dependency.name, "= #{dependency.locked_version.to_s}"
end

The nice part about this is Berkshelf will calculate all of your
dependencies, you don't have to figure them all out. Additional for your
problem updating your cookbook's dependencies can be done with 'berkshelf
update [COOKBOOK]'. So from here you could write a script to update all of
them at once, or maybe even have your CI do it for you.

On Fri, Apr 24, 2015 at 1:02 PM Fouts, Chris Chris.Fouts@sensus.com wrote:

Here’s an abbreviated description of our product.

Say we have two different applications, say, a “cacheserver” and a
“flexip” application, running in separate RHEL VMs. For each, I created a
“cacheserver_role” and a “flexip_role” cookbooks to install their
respective application. They do however, share some “common” books, say a
“java” cookbook. So role cookbooks’ metadata.rb files look like

cacheserver_role

name ‘cacheserver_role’

depends ‘java’, ‘= 1.0.0’

depends ‘cacheserver’, ‘= 1.0.0’

flexip_role

name ‘flexip_role’

depends ‘java’, ‘= 1.0.0’

depends ‘flexip’, ‘= 1.0.0’

I want to purposely “pin” a specific version of the aggregate cookbooks in
each role cookbook. This is an important requirement.

As you can see, if I want to update my java cookbook, I have to modify (at
least) two cookbooks. In my real product, I have more than just two role
cookbooks.

I’m purposely moving away from environment.json and role.json files
because they are NOT versioned, that is, they do NOT have a metadata.rb
file.

Chris

From: Nathen Harvey [mailto:nharvey@chef.io]
Sent: Friday, April 24, 2015 12:48 PM
To: chef@lists.opscode.com
Subject: [chef] Re: RE: How to manage cookbook versions more
efficiently?

I'm not sure I understand why you have a single role for each node in your

infrastructure. Could you help me understand that? Or do you mean you
have 25

different types of nodes in your infrastructure where each type might have
N

instances?

Have you considered using Environments [1] for pinning your cookbook
versions?

I think this might be a more sustainable approach for what you've
described.

-Nathen

1: About Environments

On Fri, Apr 24, 2015 at 12:25 PM, Fouts, Chris Chris.Fouts@sensus.com
wrote:

" For complete consistency, you have to specify every single cookbook,
including all dependencies for and on your desired cookbooks in your roles
or run_lists. Your role based approach gets very nasty if you start mixing
roles, and mixing conflicting cookbook versions or unintended dependencies.
There were big problems when the yum and mysql cookbooks were updated and
were incompatible with many older, stable, tested, production cookbooks
that relied on them."

That's why I asked the question, you just seconded it.

Chris

-----Original Message-----
From: Nico Kadel-Garcia [mailto:nkadel@skyhookwireless.com]
Sent: Friday, April 24, 2015 11:41 AM
To: chef@lists.opscode.com

Subject: [chef] RE: How to manage cookbook versions more efficiently?

From: Fouts, Chris [mailto:Chris.Fouts@Sensus.com]
Sent: Thursday, April 23, 2015 10:29 AM
To: chef@lists.opscode.com
Subject: [chef] How to manage cookbook versions more efficiently?

I use role cookbooks to pin down versions of the specific versions of
the cookbooks they use. Since I have 25 nodes in my product and each node
has a role, I have at least 25 role cookbooks. I just then add my role
cookbooks to my nodes' run list. For example I have: the following. I DO
want to pin a particular cookbook version in my role cookbooks.

Any ideas on how to alleviate this situation?

For complete consistency, you have to specify every single cookbook,
including all dependencies for and on your desired cookbooks in your roles
or run_lists. Your role based approach gets very nasty if you start mixing
roles, and mixing conflicting cookbook versions or unintended dependencies.
There were big problems when the yum and mysql cookbooks were updated and
were incompatible with many older, stable, tested, production cookbooks
that relied on them.

This is one of my major reasons for giving up on the
"chef-server/chef-client" model, and preferring "chef-solo" for small
environments. I can lock down every single cookbook in Berkshelf in a much
more controlled fashion than mixing and matching and unweaving roles,
cookbook, or environment wrappers, and I can apply an updated or testing
cookbook on a single host with a locally updated or git branched
Berkshelf.lock without potentially inflicting it on any other unexpected
host. There are costs: using chefdk is a fast way to get a full Berkshelf
enabled chef-solo environment, but it's not pre-built for all operating
systems that Chef supports.

Nico Kadel-Garcia
Lead DevOps Engineer
nkadel@skyhookwireless.com

Marco_Betti · April 24, 2015, 6:30pm

For the same concern (roles, environments and data bags not versioned) AND
due to the fact we have small infrastructures (less then 10/20 nodes for
production environment) for each product on our vmware private cloud
infrastructure we are currently using chef-zero (actually chef client with
local mode) instead of chef-server. We pin each cookbook exact version
within Berksfile and our environment cookbook contains everything,
including list of nodes, environments, roles, databags. Everything is under
version control within each "environment" cookbook, even if we don't need
to pin cookbook versions within environments.
Each cookbook is then autonatically "built" (binary encoded) by jenkins as
payload of a shell script that is deployed on each node of its
infrastructure using a custom rails webapp.
Finally we developed an internal tool to automatically create such
cookbooks from a model described within a json file.
Regards,
Marco
Il 24/apr/2015 20:02 "Fouts, Chris" Chris.Fouts@sensus.com ha scritto:

Here’s an abbreviated description of our product.

Say we have two different applications, say, a “cacheserver” and a
“flexip” application, running in separate RHEL VMs. For each, I created a
“cacheserver_role” and a “flexip_role” cookbooks to install their
respective application. They do however, share some “common” books, say a
“java” cookbook. So role cookbooks’ metadata.rb files look like

cacheserver_role

name ‘cacheserver_role’

depends ‘java’, ‘= 1.0.0’

depends ‘cacheserver’, ‘= 1.0.0’

flexip_role

name ‘flexip_role’

depends ‘java’, ‘= 1.0.0’

depends ‘flexip’, ‘= 1.0.0’

I want to purposely “pin” a specific version of the aggregate cookbooks in
each role cookbook. This is an important requirement.

As you can see, if I want to update my java cookbook, I have to modify (at
least) two cookbooks. In my real product, I have more than just two role
cookbooks.

I’m purposely moving away from environment.json and role.json files
because they are NOT versioned, that is, they do NOT have a metadata.rb
file.

Chris

From: Nathen Harvey [mailto:nharvey@chef.io]
Sent: Friday, April 24, 2015 12:48 PM
To: chef@lists.opscode.com
Subject: [chef] Re: RE: How to manage cookbook versions more
efficiently?

I'm not sure I understand why you have a single role for each node in your

infrastructure. Could you help me understand that? Or do you mean you
have 25

different types of nodes in your infrastructure where each type might have
N

instances?

Have you considered using Environments [1] for pinning your cookbook
versions?

I think this might be a more sustainable approach for what you've
described.

-Nathen

1: About Environments

On Fri, Apr 24, 2015 at 12:25 PM, Fouts, Chris Chris.Fouts@sensus.com
wrote:

" For complete consistency, you have to specify every single cookbook,
including all dependencies for and on your desired cookbooks in your roles
or run_lists. Your role based approach gets very nasty if you start mixing
roles, and mixing conflicting cookbook versions or unintended dependencies.
There were big problems when the yum and mysql cookbooks were updated and
were incompatible with many older, stable, tested, production cookbooks
that relied on them."

That's why I asked the question, you just seconded it.

Chris

-----Original Message-----
From: Nico Kadel-Garcia [mailto:nkadel@skyhookwireless.com]
Sent: Friday, April 24, 2015 11:41 AM
To: chef@lists.opscode.com

Subject: [chef] RE: How to manage cookbook versions more efficiently?

From: Fouts, Chris [mailto:Chris.Fouts@Sensus.com]
Sent: Thursday, April 23, 2015 10:29 AM
To: chef@lists.opscode.com
Subject: [chef] How to manage cookbook versions more efficiently?

I use role cookbooks to pin down versions of the specific versions of
the cookbooks they use. Since I have 25 nodes in my product and each node
has a role, I have at least 25 role cookbooks. I just then add my role
cookbooks to my nodes' run list. For example I have: the following. I DO
want to pin a particular cookbook version in my role cookbooks.

Any ideas on how to alleviate this situation?

For complete consistency, you have to specify every single cookbook,
including all dependencies for and on your desired cookbooks in your roles
or run_lists. Your role based approach gets very nasty if you start mixing
roles, and mixing conflicting cookbook versions or unintended dependencies.
There were big problems when the yum and mysql cookbooks were updated and
were incompatible with many older, stable, tested, production cookbooks
that relied on them.

This is one of my major reasons for giving up on the
"chef-server/chef-client" model, and preferring "chef-solo" for small
environments. I can lock down every single cookbook in Berkshelf in a much
more controlled fashion than mixing and matching and unweaving roles,
cookbook, or environment wrappers, and I can apply an updated or testing
cookbook on a single host with a locally updated or git branched
Berkshelf.lock without potentially inflicting it on any other unexpected
host. There are costs: using chefdk is a fast way to get a full Berkshelf
enabled chef-solo environment, but it's not pre-built for all operating
systems that Chef supports.

Nico Kadel-Garcia
Lead DevOps Engineer
nkadel@skyhookwireless.com

Chris_F · April 24, 2015, 7:03pm

Thanks, but your environment does not apply to ours. We have a large infrastructure, that (may) require a different set of cookbooks per each organization.

Chris

From: Marco Betti [mailto:m.betti@gmail.com]
Sent: Friday, April 24, 2015 2:31 PM
To: chef@lists.opscode.com
Subject: [chef] Re: RE: Re: RE: How to manage cookbook versions more efficiently?

For the same concern (roles, environments and data bags not versioned) AND due to the fact we have small infrastructures (less then 10/20 nodes for production environment) for each product on our vmware private cloud infrastructure we are currently using chef-zero (actually chef client with local mode) instead of chef-server. We pin each cookbook exact version within Berksfile and our environment cookbook contains everything, including list of nodes, environments, roles, databags. Everything is under version control within each "environment" cookbook, even if we don't need to pin cookbook versions within environments.
Each cookbook is then autonatically "built" (binary encoded) by jenkins as payload of a shell script that is deployed on each node of its infrastructure using a custom rails webapp.
Finally we developed an internal tool to automatically create such cookbooks from a model described within a json file.
Regards,
Marco
Il 24/apr/2015 20:02 "Fouts, Chris" <Chris.Fouts@sensus.com mailto:Chris.Fouts@sensus.com> ha scritto:
Here’s an abbreviated description of our product.

Say we have two different applications, say, a “cacheserver” and a “flexip” application, running in separate RHEL VMs. For each, I created a “cacheserver_role” and a “flexip_role” cookbooks to install their respective application. They do however, share some “common” books, say a “java” cookbook. So role cookbooks’ metadata.rb files look like

cacheserver_role
name ‘cacheserver_role’
depends ‘java’, ‘= 1.0.0’
depends ‘cacheserver’, ‘= 1.0.0’

flexip_role
name ‘flexip_role’
depends ‘java’, ‘= 1.0.0’
depends ‘flexip’, ‘= 1.0.0’

I want to purposely “pin” a specific version of the aggregate cookbooks in each role cookbook. This is an important requirement.

As you can see, if I want to update my java cookbook, I have to modify (at least) two cookbooks. In my real product, I have more than just two role cookbooks.

I’m purposely moving away from environment.json and role.json files because they are NOT versioned, that is, they do NOT have a metadata.rb file.

Chris

From: Nathen Harvey [mailto:nharvey@chef.io mailto:nharvey@chef.io]
Sent: Friday, April 24, 2015 12:48 PM
To: chef@lists.opscode.com mailto:chef@lists.opscode.com
Subject: [chef] Re: RE: How to manage cookbook versions more efficiently?

I'm not sure I understand why you have a single role for each node in your
infrastructure. Could you help me understand that? Or do you mean you have 25
different types of nodes in your infrastructure where each type might have N
instances?

Have you considered using Environments [1] for pinning your cookbook versions?
I think this might be a more sustainable approach for what you've described.

-Nathen

1: About Environments

On Fri, Apr 24, 2015 at 12:25 PM, Fouts, Chris <Chris.Fouts@sensus.com mailto:Chris.Fouts@sensus.com> wrote:
" For complete consistency, you have to specify every single cookbook, including all dependencies for and on your desired cookbooks in your roles or run_lists. Your role based approach gets very nasty if you start mixing roles, and mixing conflicting cookbook versions or unintended dependencies. There were big problems when the yum and mysql cookbooks were updated and were incompatible with many older, stable, tested, production cookbooks that relied on them."

That's why I asked the question, you just seconded it.

Chris

-----Original Message-----
From: Nico Kadel-Garcia [mailto:nkadel@skyhookwireless.com mailto:nkadel@skyhookwireless.com]
Sent: Friday, April 24, 2015 11:41 AM
To: chef@lists.opscode.com mailto:chef@lists.opscode.com
Subject: [chef] RE: How to manage cookbook versions more efficiently?

From: Fouts, Chris [mailto:Chris.Fouts@Sensus.com mailto:Chris.Fouts@Sensus.com]
Sent: Thursday, April 23, 2015 10:29 AM
To: chef@lists.opscode.com mailto:chef@lists.opscode.com
Subject: [chef] How to manage cookbook versions more efficiently?

I use role cookbooks to pin down versions of the specific versions of the cookbooks they use. Since I have 25 nodes in my product and each node has a role, I have at least 25 role cookbooks. I just then add my role cookbooks to my nodes' run list. For example I have: the following. I DO want to pin a particular cookbook version in my role cookbooks.

Any ideas on how to alleviate this situation?

For complete consistency, you have to specify every single cookbook, including all dependencies for and on your desired cookbooks in your roles or run_lists. Your role based approach gets very nasty if you start mixing roles, and mixing conflicting cookbook versions or unintended dependencies. There were big problems when the yum and mysql cookbooks were updated and were incompatible with many older, stable, tested, production cookbooks that relied on them.

This is one of my major reasons for giving up on the "chef-server/chef-client" model, and preferring "chef-solo" for small environments. I can lock down every single cookbook in Berkshelf in a much more controlled fashion than mixing and matching and unweaving roles, cookbook, or environment wrappers, and I can apply an updated or testing cookbook on a single host with a locally updated or git branched Berkshelf.lock without potentially inflicting it on any other unexpected host. There are costs: using chefdk is a fast way to get a full Berkshelf enabled chef-solo environment, but it's not pre-built for all operating systems that Chef supports.

Nico Kadel-Garcia
Lead DevOps Engineer
nkadel@skyhookwireless.com mailto:nkadel@skyhookwireless.com

Chris_F · April 24, 2015, 7:04pm

Now this is an interesting idea. But isn’t this a chicken-egg problem? Doesn’t Berksfile.lock get generated based on the metadata.rb?

Chris

From: Bryan Baugher [mailto:bjbq4d@gmail.com]
Sent: Friday, April 24, 2015 2:11 PM
To: nharvey@chef.io; chef@lists.opscode.com
Subject: [chef] Re: RE: Re: RE: How to manage cookbook versions more efficiently?

One thing I've played around with is have a cookbook's metadata.rb read a Berksfile.lock which looks something like this,
require 'berkshelf'
berksfile = ::Berkshelf::Berksfile.new 'Berksfile'
berksfile.list.each do |dependency|
depends dependency.namehttp://dependency.name, "= #{dependency.locked_version.to_s}"
end

The nice part about this is Berkshelf will calculate all of your dependencies, you don't have to figure them all out. Additional for your problem updating your cookbook's dependencies can be done with 'berkshelf update [COOKBOOK]'. So from here you could write a script to update all of them at once, or maybe even have your CI do it for you.

On Fri, Apr 24, 2015 at 1:02 PM Fouts, Chris <Chris.Fouts@sensus.com mailto:Chris.Fouts@sensus.com> wrote:
Here’s an abbreviated description of our product.

Say we have two different applications, say, a “cacheserver” and a “flexip” application, running in separate RHEL VMs. For each, I created a “cacheserver_role” and a “flexip_role” cookbooks to install their respective application. They do however, share some “common” books, say a “java” cookbook. So role cookbooks’ metadata.rb files look like

cacheserver_role
name ‘cacheserver_role’
depends ‘java’, ‘= 1.0.0’
depends ‘cacheserver’, ‘= 1.0.0’

flexip_role
name ‘flexip_role’
depends ‘java’, ‘= 1.0.0’
depends ‘flexip’, ‘= 1.0.0’

I want to purposely “pin” a specific version of the aggregate cookbooks in each role cookbook. This is an important requirement.

As you can see, if I want to update my java cookbook, I have to modify (at least) two cookbooks. In my real product, I have more than just two role cookbooks.

I’m purposely moving away from environment.json and role.json files because they are NOT versioned, that is, they do NOT have a metadata.rb file.

Chris

From: Nathen Harvey [mailto:nharvey@chef.io mailto:nharvey@chef.io]
Sent: Friday, April 24, 2015 12:48 PM
To: chef@lists.opscode.com mailto:chef@lists.opscode.com
Subject: [chef] Re: RE: How to manage cookbook versions more efficiently?

I'm not sure I understand why you have a single role for each node in your
infrastructure. Could you help me understand that? Or do you mean you have 25
different types of nodes in your infrastructure where each type might have N
instances?

Have you considered using Environments [1] for pinning your cookbook versions?
I think this might be a more sustainable approach for what you've described.

-Nathen

1: About Environments

On Fri, Apr 24, 2015 at 12:25 PM, Fouts, Chris <Chris.Fouts@sensus.com mailto:Chris.Fouts@sensus.com> wrote:
" For complete consistency, you have to specify every single cookbook, including all dependencies for and on your desired cookbooks in your roles or run_lists. Your role based approach gets very nasty if you start mixing roles, and mixing conflicting cookbook versions or unintended dependencies. There were big problems when the yum and mysql cookbooks were updated and were incompatible with many older, stable, tested, production cookbooks that relied on them."

That's why I asked the question, you just seconded it.

Chris

-----Original Message-----
From: Nico Kadel-Garcia [mailto:nkadel@skyhookwireless.com mailto:nkadel@skyhookwireless.com]
Sent: Friday, April 24, 2015 11:41 AM
To: chef@lists.opscode.com mailto:chef@lists.opscode.com
Subject: [chef] RE: How to manage cookbook versions more efficiently?

From: Fouts, Chris [mailto:Chris.Fouts@Sensus.com mailto:Chris.Fouts@Sensus.com]
Sent: Thursday, April 23, 2015 10:29 AM
To: chef@lists.opscode.com mailto:chef@lists.opscode.com
Subject: [chef] How to manage cookbook versions more efficiently?

I use role cookbooks to pin down versions of the specific versions of the cookbooks they use. Since I have 25 nodes in my product and each node has a role, I have at least 25 role cookbooks. I just then add my role cookbooks to my nodes' run list. For example I have: the following. I DO want to pin a particular cookbook version in my role cookbooks.

Any ideas on how to alleviate this situation?

For complete consistency, you have to specify every single cookbook, including all dependencies for and on your desired cookbooks in your roles or run_lists. Your role based approach gets very nasty if you start mixing roles, and mixing conflicting cookbook versions or unintended dependencies. There were big problems when the yum and mysql cookbooks were updated and were incompatible with many older, stable, tested, production cookbooks that relied on them.

This is one of my major reasons for giving up on the "chef-server/chef-client" model, and preferring "chef-solo" for small environments. I can lock down every single cookbook in Berkshelf in a much more controlled fashion than mixing and matching and unweaving roles, cookbook, or environment wrappers, and I can apply an updated or testing cookbook on a single host with a locally updated or git branched Berkshelf.lock without potentially inflicting it on any other unexpected host. There are costs: using chefdk is a fast way to get a full Berkshelf enabled chef-solo environment, but it's not pre-built for all operating systems that Chef supports.

Nico Kadel-Garcia
Lead DevOps Engineer
nkadel@skyhookwireless.com mailto:nkadel@skyhookwireless.com

Maxime_Brugidou · April 24, 2015, 9:56pm

Have you looked at the new policy file thing?

github.com

chef-boneyard/chef-dk/blob/master/POLICYFILE_README.md

# ChefDK Policyfile  README

The Policyfile feature is now documented on the main documentation site.
See [https://docs.chef.io/policyfile.html](https://docs.chef.io/policyfile.html)
for complete and up-to-date documentation.

On Apr 24, 2015 9:04 PM, "Fouts, Chris" Chris.Fouts@sensus.com wrote:

Now this is an interesting idea. But isn’t this a chicken-egg problem?
Doesn’t Berksfile.lock get generated based on the metadata.rb?

Chris

From: Bryan Baugher [mailto:bjbq4d@gmail.com]
Sent: Friday, April 24, 2015 2:11 PM
To: nharvey@chef.io; chef@lists.opscode.com
Subject: [chef] Re: RE: Re: RE: How to manage cookbook versions more
efficiently?

One thing I've played around with is have a cookbook's metadata.rb read a
Berksfile.lock which looks something like this,

require 'berkshelf'

berksfile = ::Berkshelf::Berksfile.new 'Berksfile'

berksfile.list.each do |dependency|

depends dependency.name, "= #{dependency.locked_version.to_s}"

end

The nice part about this is Berkshelf will calculate all of your
dependencies, you don't have to figure them all out. Additional for your
problem updating your cookbook's dependencies can be done with 'berkshelf
update [COOKBOOK]'. So from here you could write a script to update all of
them at once, or maybe even have your CI do it for you.

On Fri, Apr 24, 2015 at 1:02 PM Fouts, Chris Chris.Fouts@sensus.com
wrote:

Here’s an abbreviated description of our product.

Say we have two different applications, say, a “cacheserver” and a
“flexip” application, running in separate RHEL VMs. For each, I created a
“cacheserver_role” and a “flexip_role” cookbooks to install their
respective application. They do however, share some “common” books, say a
“java” cookbook. So role cookbooks’ metadata.rb files look like

cacheserver_role

name ‘cacheserver_role’

depends ‘java’, ‘= 1.0.0’

depends ‘cacheserver’, ‘= 1.0.0’

flexip_role

name ‘flexip_role’

depends ‘java’, ‘= 1.0.0’

depends ‘flexip’, ‘= 1.0.0’

I want to purposely “pin” a specific version of the aggregate cookbooks in
each role cookbook. This is an important requirement.

As you can see, if I want to update my java cookbook, I have to modify (at
least) two cookbooks. In my real product, I have more than just two role
cookbooks.

I’m purposely moving away from environment.json and role.json files
because they are NOT versioned, that is, they do NOT have a metadata.rb
file.

Chris

From: Nathen Harvey [mailto:nharvey@chef.io]
Sent: Friday, April 24, 2015 12:48 PM
To: chef@lists.opscode.com
Subject: [chef] Re: RE: How to manage cookbook versions more
efficiently?

I'm not sure I understand why you have a single role for each node in your

infrastructure. Could you help me understand that? Or do you mean you
have 25

different types of nodes in your infrastructure where each type might have
N

instances?

Have you considered using Environments [1] for pinning your cookbook
versions?

I think this might be a more sustainable approach for what you've
described.

-Nathen

1: About Environments

On Fri, Apr 24, 2015 at 12:25 PM, Fouts, Chris Chris.Fouts@sensus.com
wrote:

" For complete consistency, you have to specify every single cookbook,
including all dependencies for and on your desired cookbooks in your roles
or run_lists. Your role based approach gets very nasty if you start mixing
roles, and mixing conflicting cookbook versions or unintended dependencies.
There were big problems when the yum and mysql cookbooks were updated and
were incompatible with many older, stable, tested, production cookbooks
that relied on them."

That's why I asked the question, you just seconded it.

Chris

-----Original Message-----
From: Nico Kadel-Garcia [mailto:nkadel@skyhookwireless.com]
Sent: Friday, April 24, 2015 11:41 AM
To: chef@lists.opscode.com

Subject: [chef] RE: How to manage cookbook versions more efficiently?

From: Fouts, Chris [mailto:Chris.Fouts@Sensus.com]
Sent: Thursday, April 23, 2015 10:29 AM
To: chef@lists.opscode.com
Subject: [chef] How to manage cookbook versions more efficiently?

I use role cookbooks to pin down versions of the specific versions of
the cookbooks they use. Since I have 25 nodes in my product and each node
has a role, I have at least 25 role cookbooks. I just then add my role
cookbooks to my nodes' run list. For example I have: the following. I DO
want to pin a particular cookbook version in my role cookbooks.

Any ideas on how to alleviate this situation?

For complete consistency, you have to specify every single cookbook,
including all dependencies for and on your desired cookbooks in your roles
or run_lists. Your role based approach gets very nasty if you start mixing
roles, and mixing conflicting cookbook versions or unintended dependencies.
There were big problems when the yum and mysql cookbooks were updated and
were incompatible with many older, stable, tested, production cookbooks
that relied on them.

This is one of my major reasons for giving up on the
"chef-server/chef-client" model, and preferring "chef-solo" for small
environments. I can lock down every single cookbook in Berkshelf in a much
more controlled fashion than mixing and matching and unweaving roles,
cookbook, or environment wrappers, and I can apply an updated or testing
cookbook on a single host with a locally updated or git branched
Berkshelf.lock without potentially inflicting it on any other unexpected
host. There are costs: using chefdk is a fast way to get a full Berkshelf
enabled chef-solo environment, but it's not pre-built for all operating
systems that Chef supports.

Nico Kadel-Garcia
Lead DevOps Engineer
nkadel@skyhookwireless.com

Chris_F · April 24, 2015, 10:23pm

That’s in my future, yes. It may be THE solution for me, but for now I’m looking at a “quicker” fix, if any. Bryan Baugher’s email is intriguing.

Chris

From: Maxime Brugidou [mailto:maxime.brugidou@gmail.com]
Sent: Friday, April 24, 2015 5:56 PM
To: chef@lists.opscode.com
Cc: nharvey@chef.io
Subject: [chef] Re: RE: Re: RE: Re: RE: How to manage cookbook versions more efficiently?

Have you looked at the new policy file thing?

github.com

chef-boneyard/chef-dk/blob/master/POLICYFILE_README.md

# ChefDK Policyfile  README

The Policyfile feature is now documented on the main documentation site.
See [https://docs.chef.io/policyfile.html](https://docs.chef.io/policyfile.html)
for complete and up-to-date documentation.

On Apr 24, 2015 9:04 PM, "Fouts, Chris" <Chris.Fouts@sensus.com mailto:Chris.Fouts@sensus.com> wrote:
Now this is an interesting idea. But isn’t this a chicken-egg problem? Doesn’t Berksfile.lock get generated based on the metadata.rb?

Chris

From: Bryan Baugher [mailto:bjbq4d@gmail.com mailto:bjbq4d@gmail.com]
Sent: Friday, April 24, 2015 2:11 PM
To: nharvey@chef.io mailto:nharvey@chef.io; chef@lists.opscode.com mailto:chef@lists.opscode.com
Subject: [chef] Re: RE: Re: RE: How to manage cookbook versions more efficiently?

One thing I've played around with is have a cookbook's metadata.rb read a Berksfile.lock which looks something like this,
require 'berkshelf'
berksfile = ::Berkshelf::Berksfile.new 'Berksfile'
berksfile.list.each do |dependency|
depends dependency.namehttp://dependency.name, "= #{dependency.locked_version.to_s}"
end

The nice part about this is Berkshelf will calculate all of your dependencies, you don't have to figure them all out. Additional for your problem updating your cookbook's dependencies can be done with 'berkshelf update [COOKBOOK]'. So from here you could write a script to update all of them at once, or maybe even have your CI do it for you.

On Fri, Apr 24, 2015 at 1:02 PM Fouts, Chris <Chris.Fouts@sensus.com mailto:Chris.Fouts@sensus.com> wrote:
Here’s an abbreviated description of our product.

Say we have two different applications, say, a “cacheserver” and a “flexip” application, running in separate RHEL VMs. For each, I created a “cacheserver_role” and a “flexip_role” cookbooks to install their respective application. They do however, share some “common” books, say a “java” cookbook. So role cookbooks’ metadata.rb files look like

cacheserver_role
name ‘cacheserver_role’
depends ‘java’, ‘= 1.0.0’
depends ‘cacheserver’, ‘= 1.0.0’

flexip_role
name ‘flexip_role’
depends ‘java’, ‘= 1.0.0’
depends ‘flexip’, ‘= 1.0.0’

I want to purposely “pin” a specific version of the aggregate cookbooks in each role cookbook. This is an important requirement.

As you can see, if I want to update my java cookbook, I have to modify (at least) two cookbooks. In my real product, I have more than just two role cookbooks.

I’m purposely moving away from environment.json and role.json files because they are NOT versioned, that is, they do NOT have a metadata.rb file.

Chris

From: Nathen Harvey [mailto:nharvey@chef.io mailto:nharvey@chef.io]
Sent: Friday, April 24, 2015 12:48 PM
To: chef@lists.opscode.com mailto:chef@lists.opscode.com
Subject: [chef] Re: RE: How to manage cookbook versions more efficiently?

I'm not sure I understand why you have a single role for each node in your
infrastructure. Could you help me understand that? Or do you mean you have 25
different types of nodes in your infrastructure where each type might have N
instances?

Have you considered using Environments [1] for pinning your cookbook versions?
I think this might be a more sustainable approach for what you've described.

-Nathen

1: About Environments

On Fri, Apr 24, 2015 at 12:25 PM, Fouts, Chris <Chris.Fouts@sensus.com mailto:Chris.Fouts@sensus.com> wrote:
" For complete consistency, you have to specify every single cookbook, including all dependencies for and on your desired cookbooks in your roles or run_lists. Your role based approach gets very nasty if you start mixing roles, and mixing conflicting cookbook versions or unintended dependencies. There were big problems when the yum and mysql cookbooks were updated and were incompatible with many older, stable, tested, production cookbooks that relied on them."

That's why I asked the question, you just seconded it.

Chris

-----Original Message-----
From: Nico Kadel-Garcia [mailto:nkadel@skyhookwireless.com mailto:nkadel@skyhookwireless.com]
Sent: Friday, April 24, 2015 11:41 AM
To: chef@lists.opscode.com mailto:chef@lists.opscode.com
Subject: [chef] RE: How to manage cookbook versions more efficiently?

From: Fouts, Chris [mailto:Chris.Fouts@Sensus.com mailto:Chris.Fouts@Sensus.com]
Sent: Thursday, April 23, 2015 10:29 AM
To: chef@lists.opscode.com mailto:chef@lists.opscode.com
Subject: [chef] How to manage cookbook versions more efficiently?

I use role cookbooks to pin down versions of the specific versions of the cookbooks they use. Since I have 25 nodes in my product and each node has a role, I have at least 25 role cookbooks. I just then add my role cookbooks to my nodes' run list. For example I have: the following. I DO want to pin a particular cookbook version in my role cookbooks.

Any ideas on how to alleviate this situation?

For complete consistency, you have to specify every single cookbook, including all dependencies for and on your desired cookbooks in your roles or run_lists. Your role based approach gets very nasty if you start mixing roles, and mixing conflicting cookbook versions or unintended dependencies. There were big problems when the yum and mysql cookbooks were updated and were incompatible with many older, stable, tested, production cookbooks that relied on them.

This is one of my major reasons for giving up on the "chef-server/chef-client" model, and preferring "chef-solo" for small environments. I can lock down every single cookbook in Berkshelf in a much more controlled fashion than mixing and matching and unweaving roles, cookbook, or environment wrappers, and I can apply an updated or testing cookbook on a single host with a locally updated or git branched Berkshelf.lock without potentially inflicting it on any other unexpected host. There are costs: using chefdk is a fast way to get a full Berkshelf enabled chef-solo environment, but it's not pre-built for all operating systems that Chef supports.

Nico Kadel-Garcia
Lead DevOps Engineer
nkadel@skyhookwireless.com mailto:nkadel@skyhookwireless.com

Nathen_Harvey3 · April 24, 2015, 10:45pm

I understand your desire to move away from roles. I'd argue environments are actually what you want to use to solve for this particular use case while you wait for Policyfile to be fully realized.

Environments will balance your desire to roll out versions in a stepwise way without the complexity and overhead of pinning your cookbooks in role cookbooks.

--
Nathen Harvey

On Apr 24, 2015, at 2:02 PM, Fouts, Chris Chris.Fouts@Sensus.com wrote:

Here’s an abbreviated description of our product.

Say we have two different applications, say, a “cacheserver” and a “flexip” application, running in separate RHEL VMs. For each, I created a “cacheserver_role” and a “flexip_role” cookbooks to install their respective application. They do however, share some “common” books, say a “java” cookbook. So role cookbooks’ metadata.rb files look like

cacheserver_role
name ‘cacheserver_role’
depends ‘java’, ‘= 1.0.0’
depends ‘cacheserver’, ‘= 1.0.0’

flexip_role
name ‘flexip_role’
depends ‘java’, ‘= 1.0.0’
depends ‘flexip’, ‘= 1.0.0’

I want to purposely “pin” a specific version of the aggregate cookbooks in each role cookbook. This is an important requirement.

As you can see, if I want to update my java cookbook, I have to modify (at least) two cookbooks. In my real product, I have more than just two role cookbooks.

I’m purposely moving away from environment.json and role.json files because they are NOT versioned, that is, they do NOT have a metadata.rb file.

Chris

From: Nathen Harvey [mailto:nharvey@chef.io]
Sent: Friday, April 24, 2015 12:48 PM
To: chef@lists.opscode.com
Subject: [chef] Re: RE: How to manage cookbook versions more efficiently?

I'm not sure I understand why you have a single role for each node in your
infrastructure. Could you help me understand that? Or do you mean you have 25
different types of nodes in your infrastructure where each type might have N
instances?

Have you considered using Environments [1] for pinning your cookbook versions?
I think this might be a more sustainable approach for what you've described.

-Nathen

1: About Environments

On Fri, Apr 24, 2015 at 12:25 PM, Fouts, Chris Chris.Fouts@sensus.com wrote:
" For complete consistency, you have to specify every single cookbook, including all dependencies for and on your desired cookbooks in your roles or run_lists. Your role based approach gets very nasty if you start mixing roles, and mixing conflicting cookbook versions or unintended dependencies. There were big problems when the yum and mysql cookbooks were updated and were incompatible with many older, stable, tested, production cookbooks that relied on them."

That's why I asked the question, you just seconded it.

Chris

-----Original Message-----
From: Nico Kadel-Garcia [mailto:nkadel@skyhookwireless.com]
Sent: Friday, April 24, 2015 11:41 AM
To: chef@lists.opscode.com
Subject: [chef] RE: How to manage cookbook versions more efficiently?

From: Fouts, Chris [mailto:Chris.Fouts@Sensus.com]
Sent: Thursday, April 23, 2015 10:29 AM
To: chef@lists.opscode.com
Subject: [chef] How to manage cookbook versions more efficiently?

I use role cookbooks to pin down versions of the specific versions of the cookbooks they use. Since I have 25 nodes in my product and each node has a role, I have at least 25 role cookbooks. I just then add my role cookbooks to my nodes' run list. For example I have: the following. I DO want to pin a particular cookbook version in my role cookbooks.

Any ideas on how to alleviate this situation?

For complete consistency, you have to specify every single cookbook, including all dependencies for and on your desired cookbooks in your roles or run_lists. Your role based approach gets very nasty if you start mixing roles, and mixing conflicting cookbook versions or unintended dependencies. There were big problems when the yum and mysql cookbooks were updated and were incompatible with many older, stable, tested, production cookbooks that relied on them.

This is one of my major reasons for giving up on the "chef-server/chef-client" model, and preferring "chef-solo" for small environments. I can lock down every single cookbook in Berkshelf in a much more controlled fashion than mixing and matching and unweaving roles, cookbook, or environment wrappers, and I can apply an updated or testing cookbook on a single host with a locally updated or git branched Berkshelf.lock without potentially inflicting it on any other unexpected host. There are costs: using chefdk is a fast way to get a full Berkshelf enabled chef-solo environment, but it's not pre-built for all operating systems that Chef supports.

Nico Kadel-Garcia
Lead DevOps Engineer
nkadel@skyhookwireless.com

Chris_F · April 24, 2015, 10:53pm

That’s what we’re doing now, and have been bitten by it, BECAUSE of their lack of metadata.rb file.

Chris

From: Nathen Harvey [mailto:nharvey@chef.io]
Sent: Friday, April 24, 2015 6:45 PM
To: Fouts, Chris
Cc: chef@lists.opscode.com
Subject: Re: [chef] Re: RE: How to manage cookbook versions more efficiently?

I understand your desire to move away from roles. I'd argue environments are actually what you want to use to solve for this particular use case while you wait for Policyfile to be fully realized.

Environments will balance your desire to roll out versions in a stepwise way without the complexity and overhead of pinning your cookbooks in role cookbooks.

--
Nathen Harvey

On Apr 24, 2015, at 2:02 PM, Fouts, Chris <Chris.Fouts@Sensus.com mailto:Chris.Fouts@Sensus.com> wrote:
Here’s an abbreviated description of our product.

Say we have two different applications, say, a “cacheserver” and a “flexip” application, running in separate RHEL VMs. For each, I created a “cacheserver_role” and a “flexip_role” cookbooks to install their respective application. They do however, share some “common” books, say a “java” cookbook. So role cookbooks’ metadata.rb files look like

cacheserver_role
name ‘cacheserver_role’
depends ‘java’, ‘= 1.0.0’
depends ‘cacheserver’, ‘= 1.0.0’

flexip_role
name ‘flexip_role’
depends ‘java’, ‘= 1.0.0’
depends ‘flexip’, ‘= 1.0.0’

I want to purposely “pin” a specific version of the aggregate cookbooks in each role cookbook. This is an important requirement.

As you can see, if I want to update my java cookbook, I have to modify (at least) two cookbooks. In my real product, I have more than just two role cookbooks.

I’m purposely moving away from environment.json and role.json files because they are NOT versioned, that is, they do NOT have a metadata.rb file.

Chris

From: Nathen Harvey [mailto:nharvey@chef.io]
Sent: Friday, April 24, 2015 12:48 PM
To: chef@lists.opscode.com mailto:chef@lists.opscode.com
Subject: [chef] Re: RE: How to manage cookbook versions more efficiently?

I'm not sure I understand why you have a single role for each node in your
infrastructure. Could you help me understand that? Or do you mean you have 25
different types of nodes in your infrastructure where each type might have N
instances?

Have you considered using Environments [1] for pinning your cookbook versions?
I think this might be a more sustainable approach for what you've described.

-Nathen

1: About Environments

On Fri, Apr 24, 2015 at 12:25 PM, Fouts, Chris <Chris.Fouts@sensus.com mailto:Chris.Fouts@sensus.com> wrote:
" For complete consistency, you have to specify every single cookbook, including all dependencies for and on your desired cookbooks in your roles or run_lists. Your role based approach gets very nasty if you start mixing roles, and mixing conflicting cookbook versions or unintended dependencies. There were big problems when the yum and mysql cookbooks were updated and were incompatible with many older, stable, tested, production cookbooks that relied on them."

That's why I asked the question, you just seconded it.

Chris

-----Original Message-----
From: Nico Kadel-Garcia [mailto:nkadel@skyhookwireless.com mailto:nkadel@skyhookwireless.com]
Sent: Friday, April 24, 2015 11:41 AM
To: chef@lists.opscode.com mailto:chef@lists.opscode.com
Subject: [chef] RE: How to manage cookbook versions more efficiently?

From: Fouts, Chris [mailto:Chris.Fouts@Sensus.com mailto:Chris.Fouts@Sensus.com]
Sent: Thursday, April 23, 2015 10:29 AM
To: chef@lists.opscode.com mailto:chef@lists.opscode.com
Subject: [chef] How to manage cookbook versions more efficiently?

I use role cookbooks to pin down versions of the specific versions of the cookbooks they use. Since I have 25 nodes in my product and each node has a role, I have at least 25 role cookbooks. I just then add my role cookbooks to my nodes' run list. For example I have: the following. I DO want to pin a particular cookbook version in my role cookbooks.

Any ideas on how to alleviate this situation?

For complete consistency, you have to specify every single cookbook, including all dependencies for and on your desired cookbooks in your roles or run_lists. Your role based approach gets very nasty if you start mixing roles, and mixing conflicting cookbook versions or unintended dependencies. There were big problems when the yum and mysql cookbooks were updated and were incompatible with many older, stable, tested, production cookbooks that relied on them.

This is one of my major reasons for giving up on the "chef-server/chef-client" model, and preferring "chef-solo" for small environments. I can lock down every single cookbook in Berkshelf in a much more controlled fashion than mixing and matching and unweaving roles, cookbook, or environment wrappers, and I can apply an updated or testing cookbook on a single host with a locally updated or git branched Berkshelf.lock without potentially inflicting it on any other unexpected host. There are costs: using chefdk is a fast way to get a full Berkshelf enabled chef-solo environment, but it's not pre-built for all operating systems that Chef supports.

Nico Kadel-Garcia
Lead DevOps Engineer
nkadel@skyhookwireless.com mailto:nkadel@skyhookwireless.com

bbaugher · April 25, 2015, 3:08am

You can just remove the 'metadata' keyword in your Berksfile and use your
Berksfile as your metadata.rb.

On Fri, Apr 24, 2015 at 2:04 PM Fouts, Chris Chris.Fouts@sensus.com wrote:

Now this is an interesting idea. But isn’t this a chicken-egg problem?
Doesn’t Berksfile.lock get generated based on the metadata.rb?

Chris

From: Bryan Baugher [mailto:bjbq4d@gmail.com]
Sent: Friday, April 24, 2015 2:11 PM
To: nharvey@chef.io; chef@lists.opscode.com
Subject: [chef] Re: RE: Re: RE: How to manage cookbook versions more
efficiently?

One thing I've played around with is have a cookbook's metadata.rb read a
Berksfile.lock which looks something like this,

require 'berkshelf'

berksfile = ::Berkshelf::Berksfile.new 'Berksfile'

berksfile.list.each do |dependency|

depends dependency.name, "= #{dependency.locked_version.to_s}"

end

The nice part about this is Berkshelf will calculate all of your
dependencies, you don't have to figure them all out. Additional for your
problem updating your cookbook's dependencies can be done with 'berkshelf
update [COOKBOOK]'. So from here you could write a script to update all of
them at once, or maybe even have your CI do it for you.

On Fri, Apr 24, 2015 at 1:02 PM Fouts, Chris Chris.Fouts@sensus.com
wrote:

Here’s an abbreviated description of our product.

Say we have two different applications, say, a “cacheserver” and a
“flexip” application, running in separate RHEL VMs. For each, I created a
“cacheserver_role” and a “flexip_role” cookbooks to install their
respective application. They do however, share some “common” books, say a
“java” cookbook. So role cookbooks’ metadata.rb files look like

cacheserver_role

name ‘cacheserver_role’

depends ‘java’, ‘= 1.0.0’

depends ‘cacheserver’, ‘= 1.0.0’

flexip_role

name ‘flexip_role’

depends ‘java’, ‘= 1.0.0’

depends ‘flexip’, ‘= 1.0.0’

I want to purposely “pin” a specific version of the aggregate cookbooks in
each role cookbook. This is an important requirement.

As you can see, if I want to update my java cookbook, I have to modify (at
least) two cookbooks. In my real product, I have more than just two role
cookbooks.

I’m purposely moving away from environment.json and role.json files
because they are NOT versioned, that is, they do NOT have a metadata.rb
file.

Chris

From: Nathen Harvey [mailto:nharvey@chef.io]
Sent: Friday, April 24, 2015 12:48 PM
To: chef@lists.opscode.com
Subject: [chef] Re: RE: How to manage cookbook versions more
efficiently?

I'm not sure I understand why you have a single role for each node in your

infrastructure. Could you help me understand that? Or do you mean you
have 25

different types of nodes in your infrastructure where each type might have
N

instances?

Have you considered using Environments [1] for pinning your cookbook
versions?

I think this might be a more sustainable approach for what you've
described.

-Nathen

1: About Environments

On Fri, Apr 24, 2015 at 12:25 PM, Fouts, Chris Chris.Fouts@sensus.com
wrote:

" For complete consistency, you have to specify every single cookbook,
including all dependencies for and on your desired cookbooks in your roles
or run_lists. Your role based approach gets very nasty if you start mixing
roles, and mixing conflicting cookbook versions or unintended dependencies.
There were big problems when the yum and mysql cookbooks were updated and
were incompatible with many older, stable, tested, production cookbooks
that relied on them."

That's why I asked the question, you just seconded it.

Chris

-----Original Message-----
From: Nico Kadel-Garcia [mailto:nkadel@skyhookwireless.com]
Sent: Friday, April 24, 2015 11:41 AM
To: chef@lists.opscode.com

Subject: [chef] RE: How to manage cookbook versions more efficiently?

From: Fouts, Chris [mailto:Chris.Fouts@Sensus.com]
Sent: Thursday, April 23, 2015 10:29 AM
To: chef@lists.opscode.com
Subject: [chef] How to manage cookbook versions more efficiently?

I use role cookbooks to pin down versions of the specific versions of
the cookbooks they use. Since I have 25 nodes in my product and each node
has a role, I have at least 25 role cookbooks. I just then add my role
cookbooks to my nodes' run list. For example I have: the following. I DO
want to pin a particular cookbook version in my role cookbooks.

Any ideas on how to alleviate this situation?

For complete consistency, you have to specify every single cookbook,
including all dependencies for and on your desired cookbooks in your roles
or run_lists. Your role based approach gets very nasty if you start mixing
roles, and mixing conflicting cookbook versions or unintended dependencies.
There were big problems when the yum and mysql cookbooks were updated and
were incompatible with many older, stable, tested, production cookbooks
that relied on them.

This is one of my major reasons for giving up on the
"chef-server/chef-client" model, and preferring "chef-solo" for small
environments. I can lock down every single cookbook in Berkshelf in a much
more controlled fashion than mixing and matching and unweaving roles,
cookbook, or environment wrappers, and I can apply an updated or testing
cookbook on a single host with a locally updated or git branched
Berkshelf.lock without potentially inflicting it on any other unexpected
host. There are costs: using chefdk is a fast way to get a full Berkshelf
enabled chef-solo environment, but it's not pre-built for all operating
systems that Chef supports.

Nico Kadel-Garcia
Lead DevOps Engineer
nkadel@skyhookwireless.com

Lamont_Granquist · April 27, 2015, 6:59pm

I still haven’t played with this yet, but I think what you want is an
environment cookbook:

http://blog.vialstudios.com/the-environment-cookbook-pattern/

And you want to berks apply the Berksfile.lock in that cookbook to the
environment.

bbaugher · April 27, 2015, 7:08pm

That then requires you either combine all of your roles into 1 environment
cookbook and hope you don't have conflicts/problems or have an environment
per role. I haven't actually used either of these approaches just played
around with them.

On Mon, Apr 27, 2015 at 1:59 PM Lamont Granquist lamont@chef.io wrote:

I still haven't played with this yet, but I think what you want is an
environment cookbook:

http://blog.vialstudios.com/the-environment-cookbook-pattern/

And you want to berks apply the Berksfile.lock in that cookbook to the
environment.

Chris_F · April 27, 2015, 7:16pm

If you carefully read my original post, this is indeed what I’m doing; hence the complication.

Chris

From: Bryan Baugher [mailto:bjbq4d@gmail.com]
Sent: Monday, April 27, 2015 3:08 PM
To: Lamont Granquist; chef@lists.opscode.com
Subject: [chef] Re: Re: RE: Re: RE: Re: RE: How to manage cookbook versions more efficiently?

That then requires you either combine all of your roles into 1 environment cookbook and hope you don’t have conflicts/problems or have an environment per role. I haven’t actually used either of these approaches just played around with them.

On Mon, Apr 27, 2015 at 1:59 PM Lamont Granquist <lamont@chef.io mailto:lamont@chef.io> wrote:
I still haven’t played with this yet, but I think what you want is an
environment cookbook:

http://blog.vialstudios.com/the-environment-cookbook-pattern/

And you want to berks apply the Berksfile.lock in that cookbook to the
environment.

Greg_Damiani · April 30, 2015, 1:20pm

Is there any documentation or discussion around this approach, i.e.
removing metadata.rb from your cookbooks? Can't seem to find a blog or
thread anywhere.

I've got cookbook developers asking me why they need to declare
dependencies in two different places right now, so this would be a great
convenience.

One question I have at the outset is, without metadata.rb, where do
cookbook name and version number get stored? Most "knife cookbook"
commands fail when I remove metadata.rb as a result.

On Sat, Apr 25, 2015 at 12:08 PM, Bryan Baugher bjbq4d@gmail.com wrote:

You can just remove the 'metadata' keyword in your Berksfile and use your
Berksfile as your metadata.rb.

On Fri, Apr 24, 2015 at 2:04 PM Fouts, Chris Chris.Fouts@sensus.com
wrote:

Now this is an interesting idea. But isn’t this a chicken-egg problem?
Doesn’t Berksfile.lock get generated based on the metadata.rb?

Chris

From: Bryan Baugher [mailto:bjbq4d@gmail.com]
Sent: Friday, April 24, 2015 2:11 PM
To: nharvey@chef.io; chef@lists.opscode.com
Subject: [chef] Re: RE: Re: RE: How to manage cookbook versions more
efficiently?

One thing I've played around with is have a cookbook's metadata.rb read a
Berksfile.lock which looks something like this,

require 'berkshelf'

berksfile = ::Berkshelf::Berksfile.new 'Berksfile'

berksfile.list.each do |dependency|

depends dependency.name, "= #{dependency.locked_version.to_s}"

end

The nice part about this is Berkshelf will calculate all of your
dependencies, you don't have to figure them all out. Additional for your
problem updating your cookbook's dependencies can be done with 'berkshelf
update [COOKBOOK]'. So from here you could write a script to update all of
them at once, or maybe even have your CI do it for you.

On Fri, Apr 24, 2015 at 1:02 PM Fouts, Chris Chris.Fouts@sensus.com
wrote:

Here’s an abbreviated description of our product.

Say we have two different applications, say, a “cacheserver” and a
“flexip” application, running in separate RHEL VMs. For each, I created a
“cacheserver_role” and a “flexip_role” cookbooks to install their
respective application. They do however, share some “common” books, say a
“java” cookbook. So role cookbooks’ metadata.rb files look like

cacheserver_role

name ‘cacheserver_role’

depends ‘java’, ‘= 1.0.0’

depends ‘cacheserver’, ‘= 1.0.0’

flexip_role

name ‘flexip_role’

depends ‘java’, ‘= 1.0.0’

depends ‘flexip’, ‘= 1.0.0’

I want to purposely “pin” a specific version of the aggregate cookbooks
in each role cookbook. This is an important requirement.

As you can see, if I want to update my java cookbook, I have to modify
(at least) two cookbooks. In my real product, I have more than just two
role cookbooks.

I’m purposely moving away from environment.json and role.json files
because they are NOT versioned, that is, they do NOT have a metadata.rb
file.

Chris

From: Nathen Harvey [mailto:nharvey@chef.io]
Sent: Friday, April 24, 2015 12:48 PM
To: chef@lists.opscode.com
Subject: [chef] Re: RE: How to manage cookbook versions more
efficiently?

I'm not sure I understand why you have a single role for each node in
your

infrastructure. Could you help me understand that? Or do you mean you
have 25

different types of nodes in your infrastructure where each type might
have N

instances?

Have you considered using Environments [1] for pinning your cookbook
versions?

I think this might be a more sustainable approach for what you've
described.

-Nathen

1: About Environments

On Fri, Apr 24, 2015 at 12:25 PM, Fouts, Chris Chris.Fouts@sensus.com
wrote:

" For complete consistency, you have to specify every single cookbook,
including all dependencies for and on your desired cookbooks in your roles
or run_lists. Your role based approach gets very nasty if you start mixing
roles, and mixing conflicting cookbook versions or unintended dependencies.
There were big problems when the yum and mysql cookbooks were updated and
were incompatible with many older, stable, tested, production cookbooks
that relied on them."

That's why I asked the question, you just seconded it.

Chris

-----Original Message-----
From: Nico Kadel-Garcia [mailto:nkadel@skyhookwireless.com]
Sent: Friday, April 24, 2015 11:41 AM
To: chef@lists.opscode.com

Subject: [chef] RE: How to manage cookbook versions more efficiently?

From: Fouts, Chris [mailto:Chris.Fouts@Sensus.com]
Sent: Thursday, April 23, 2015 10:29 AM
To: chef@lists.opscode.com
Subject: [chef] How to manage cookbook versions more efficiently?

I use role cookbooks to pin down versions of the specific versions of
the cookbooks they use. Since I have 25 nodes in my product and each node
has a role, I have at least 25 role cookbooks. I just then add my role
cookbooks to my nodes' run list. For example I have: the following. I DO
want to pin a particular cookbook version in my role cookbooks.

Any ideas on how to alleviate this situation?

For complete consistency, you have to specify every single cookbook,
including all dependencies for and on your desired cookbooks in your roles
or run_lists. Your role based approach gets very nasty if you start mixing
roles, and mixing conflicting cookbook versions or unintended dependencies.
There were big problems when the yum and mysql cookbooks were updated and
were incompatible with many older, stable, tested, production cookbooks
that relied on them.

This is one of my major reasons for giving up on the
"chef-server/chef-client" model, and preferring "chef-solo" for small
environments. I can lock down every single cookbook in Berkshelf in a much
more controlled fashion than mixing and matching and unweaving roles,
cookbook, or environment wrappers, and I can apply an updated or testing
cookbook on a single host with a locally updated or git branched
Berkshelf.lock without potentially inflicting it on any other unexpected
host. There are costs: using chefdk is a fast way to get a full Berkshelf
enabled chef-solo environment, but it's not pre-built for all operating
systems that Chef supports.

Nico Kadel-Garcia
Lead DevOps Engineer
nkadel@skyhookwireless.com

--

Greg Damiani | Senior System Administrator | @damianigreg

BuzzFeed: The Social News and Entertainment Company

40 Argyll Street, 2nd Floor, London, W1F 7EB

Chris_F · April 30, 2015, 6:20pm

Cookbooks still need metadata.rb

I believe the Bryan suggests that if you want to use Berkshelf, you can make it “independent” of the metadata.rb file but NOT putting the metadata line in it. Then you manage the versions in you metadata.rb file by putting in the code he suggested.

Chris

From: Greg Damiani [mailto:greg.damiani@buzzfeed.com]
Sent: Thursday, April 30, 2015 9:21 AM
To: chef@lists.opscode.com
Subject: [chef] Re: Re: RE: Re: RE: Re: RE: How to manage cookbook versions more efficiently?

Is there any documentation or discussion around this approach, i.e. removing metadata.rb from your cookbooks? Can't seem to find a blog or thread anywhere.

I've got cookbook developers asking me why they need to declare dependencies in two different places right now, so this would be a great convenience.

One question I have at the outset is, without metadata.rb, where do cookbook name and version number get stored? Most "knife cookbook" commands fail when I remove metadata.rb as a result.

On Sat, Apr 25, 2015 at 12:08 PM, Bryan Baugher <bjbq4d@gmail.com mailto:bjbq4d@gmail.com> wrote:
You can just remove the 'metadata' keyword in your Berksfile and use your Berksfile as your metadata.rb.

On Fri, Apr 24, 2015 at 2:04 PM Fouts, Chris <Chris.Fouts@sensus.com mailto:Chris.Fouts@sensus.com> wrote:
Now this is an interesting idea. But isn’t this a chicken-egg problem? Doesn’t Berksfile.lock get generated based on the metadata.rb?

Chris

From: Bryan Baugher [mailto:bjbq4d@gmail.com mailto:bjbq4d@gmail.com]
Sent: Friday, April 24, 2015 2:11 PM
To: nharvey@chef.io mailto:nharvey@chef.io; chef@lists.opscode.com mailto:chef@lists.opscode.com
Subject: [chef] Re: RE: Re: RE: How to manage cookbook versions more efficiently?

One thing I've played around with is have a cookbook's metadata.rb read a Berksfile.lock which looks something like this,
require 'berkshelf'
berksfile = ::Berkshelf::Berksfile.new 'Berksfile'
berksfile.list.each do |dependency|
depends dependency.namehttp://dependency.name, "= #{dependency.locked_version.to_s}"
end

The nice part about this is Berkshelf will calculate all of your dependencies, you don't have to figure them all out. Additional for your problem updating your cookbook's dependencies can be done with 'berkshelf update [COOKBOOK]'. So from here you could write a script to update all of them at once, or maybe even have your CI do it for you.

On Fri, Apr 24, 2015 at 1:02 PM Fouts, Chris <Chris.Fouts@sensus.com mailto:Chris.Fouts@sensus.com> wrote:
Here’s an abbreviated description of our product.

Say we have two different applications, say, a “cacheserver” and a “flexip” application, running in separate RHEL VMs. For each, I created a “cacheserver_role” and a “flexip_role” cookbooks to install their respective application. They do however, share some “common” books, say a “java” cookbook. So role cookbooks’ metadata.rb files look like

cacheserver_role
name ‘cacheserver_role’
depends ‘java’, ‘= 1.0.0’
depends ‘cacheserver’, ‘= 1.0.0’

flexip_role
name ‘flexip_role’
depends ‘java’, ‘= 1.0.0’
depends ‘flexip’, ‘= 1.0.0’

I want to purposely “pin” a specific version of the aggregate cookbooks in each role cookbook. This is an important requirement.

As you can see, if I want to update my java cookbook, I have to modify (at least) two cookbooks. In my real product, I have more than just two role cookbooks.

I’m purposely moving away from environment.json and role.json files because they are NOT versioned, that is, they do NOT have a metadata.rb file.

Chris

From: Nathen Harvey [mailto:nharvey@chef.io mailto:nharvey@chef.io]
Sent: Friday, April 24, 2015 12:48 PM
To: chef@lists.opscode.com mailto:chef@lists.opscode.com
Subject: [chef] Re: RE: How to manage cookbook versions more efficiently?

I'm not sure I understand why you have a single role for each node in your
infrastructure. Could you help me understand that? Or do you mean you have 25
different types of nodes in your infrastructure where each type might have N
instances?

Have you considered using Environments [1] for pinning your cookbook versions?
I think this might be a more sustainable approach for what you've described.

-Nathen

1: About Environments

On Fri, Apr 24, 2015 at 12:25 PM, Fouts, Chris <Chris.Fouts@sensus.com mailto:Chris.Fouts@sensus.com> wrote:
" For complete consistency, you have to specify every single cookbook, including all dependencies for and on your desired cookbooks in your roles or run_lists. Your role based approach gets very nasty if you start mixing roles, and mixing conflicting cookbook versions or unintended dependencies. There were big problems when the yum and mysql cookbooks were updated and were incompatible with many older, stable, tested, production cookbooks that relied on them."

That's why I asked the question, you just seconded it.

Chris

-----Original Message-----
From: Nico Kadel-Garcia [mailto:nkadel@skyhookwireless.com mailto:nkadel@skyhookwireless.com]
Sent: Friday, April 24, 2015 11:41 AM
To: chef@lists.opscode.com mailto:chef@lists.opscode.com
Subject: [chef] RE: How to manage cookbook versions more efficiently?

From: Fouts, Chris [mailto:Chris.Fouts@Sensus.com mailto:Chris.Fouts@Sensus.com]
Sent: Thursday, April 23, 2015 10:29 AM
To: chef@lists.opscode.com mailto:chef@lists.opscode.com
Subject: [chef] How to manage cookbook versions more efficiently?

I use role cookbooks to pin down versions of the specific versions of the cookbooks they use. Since I have 25 nodes in my product and each node has a role, I have at least 25 role cookbooks. I just then add my role cookbooks to my nodes' run list. For example I have: the following. I DO want to pin a particular cookbook version in my role cookbooks.

Any ideas on how to alleviate this situation?

For complete consistency, you have to specify every single cookbook, including all dependencies for and on your desired cookbooks in your roles or run_lists. Your role based approach gets very nasty if you start mixing roles, and mixing conflicting cookbook versions or unintended dependencies. There were big problems when the yum and mysql cookbooks were updated and were incompatible with many older, stable, tested, production cookbooks that relied on them.

This is one of my major reasons for giving up on the "chef-server/chef-client" model, and preferring "chef-solo" for small environments. I can lock down every single cookbook in Berkshelf in a much more controlled fashion than mixing and matching and unweaving roles, cookbook, or environment wrappers, and I can apply an updated or testing cookbook on a single host with a locally updated or git branched Berkshelf.lock without potentially inflicting it on any other unexpected host. There are costs: using chefdk is a fast way to get a full Berkshelf enabled chef-solo environment, but it's not pre-built for all operating systems that Chef supports.

Nico Kadel-Garcia
Lead DevOps Engineer
nkadel@skyhookwireless.com mailto:nkadel@skyhookwireless.com

--

Greg Damiani | Senior System Administrator | @damianigreg

BuzzFeed: The Social News and Entertainment Company

40 Argyll Street, 2nd Floor, London, W1F 7EB

someara · April 30, 2015, 10:17pm

You can't have cookbooks without metadata.

The metadata is used by Chef itself
The Berksfile is used by your development tooling. (test-kitchen)

For example, a common thing to do with test-kitchen is apply the
apt::default recipe to the machine before running recipes on debuntu
platforms.

Like this: https://github.com/chef-cookbooks/mysql/blob/c2d49a98ba9566967f464009c59a557b23086907/.kitchen.yml#L21

Because apt is not a strict dependency for the mysql cookbook, it
needs to be listed in the Berksfile.

github.com

sous-chefs/mysql/blob/c2d49a98ba9566967f464009c59a557b23086907/metadata.rb#L20-L21


      
          depends 'yum-mysql-community'
          depends 'smf'

and

github.com

sous-chefs/mysql/blob/c2d49a98ba9566967f464009c59a557b23086907/Berksfile#L6


      
          source 'https://supermarket.chef.io'
          
          metadata
          
          group :integration do
            cookbook 'apt'
            cookbook 'freebsd'
            cookbook 'ips-omniti'
            cookbook 'selinux'
          end
          
          cookbook 'mysql_client_test', path: 'test/fixtures/cookbooks/mysql_client_test'
          cookbook 'mysql_config_test', path: 'test/fixtures/cookbooks/mysql_config_test'
          cookbook 'mysql_replication_test', path: 'test/fixtures/cookbooks/mysql_replication_test'
          cookbook 'mysql_service_test', path: 'test/fixtures/cookbooks/mysql_service_test'

The other use case, is for when you're developing more than one
cookbook at once, and need to point at local disk paths or git repos.

https://github.com/someara/wordpress-uberdemo/blob/d8ed8d2ce53cb68518afc6cd2bfa1cda97140e63/Berksfile#L5-L6

^ After those two cookbooks are finished, they'll be released to an
artifact repository (supermarket or your chef-server), then the
Berksfile lines can be removed.

-s

On Thu, Apr 30, 2015 at 9:20 AM, Greg Damiani greg.damiani@buzzfeed.com wrote:

Is there any documentation or discussion around this approach, i.e. removing
metadata.rb from your cookbooks? Can't seem to find a blog or thread
anywhere.

I've got cookbook developers asking me why they need to declare dependencies
in two different places right now, so this would be a great convenience.

One question I have at the outset is, without metadata.rb, where do cookbook
name and version number get stored? Most "knife cookbook" commands fail
when I remove metadata.rb as a result.

On Sat, Apr 25, 2015 at 12:08 PM, Bryan Baugher bjbq4d@gmail.com wrote:

You can just remove the 'metadata' keyword in your Berksfile and use your
Berksfile as your metadata.rb.

On Fri, Apr 24, 2015 at 2:04 PM Fouts, Chris Chris.Fouts@sensus.com
wrote:

Now this is an interesting idea. But isn’t this a chicken-egg problem?
Doesn’t Berksfile.lock get generated based on the metadata.rb?

Chris

From: Bryan Baugher [mailto:bjbq4d@gmail.com]
Sent: Friday, April 24, 2015 2:11 PM
To: nharvey@chef.io; chef@lists.opscode.com
Subject: [chef] Re: RE: Re: RE: How to manage cookbook versions more
efficiently?

One thing I've played around with is have a cookbook's metadata.rb read a
Berksfile.lock which looks something like this,

require 'berkshelf'

berksfile = ::Berkshelf::Berksfile.new 'Berksfile'

berksfile.list.each do |dependency|

depends dependency.name, "= #{dependency.locked_version.to_s}"

end

The nice part about this is Berkshelf will calculate all of your
dependencies, you don't have to figure them all out. Additional for your
problem updating your cookbook's dependencies can be done with 'berkshelf
update [COOKBOOK]'. So from here you could write a script to update all of
them at once, or maybe even have your CI do it for you.

On Fri, Apr 24, 2015 at 1:02 PM Fouts, Chris Chris.Fouts@sensus.com
wrote:

Here’s an abbreviated description of our product.

Say we have two different applications, say, a “cacheserver” and a
“flexip” application, running in separate RHEL VMs. For each, I created a
“cacheserver_role” and a “flexip_role” cookbooks to install their respective
application. They do however, share some “common” books, say a “java”
cookbook. So role cookbooks’ metadata.rb files look like

cacheserver_role

name ‘cacheserver_role’

depends ‘java’, ‘= 1.0.0’

depends ‘cacheserver’, ‘= 1.0.0’

flexip_role

name ‘flexip_role’

depends ‘java’, ‘= 1.0.0’

depends ‘flexip’, ‘= 1.0.0’

I want to purposely “pin” a specific version of the aggregate cookbooks
in each role cookbook. This is an important requirement.

As you can see, if I want to update my java cookbook, I have to modify
(at least) two cookbooks. In my real product, I have more than just two role
cookbooks.

I’m purposely moving away from environment.json and role.json files
because they are NOT versioned, that is, they do NOT have a metadata.rb
file.

Chris

From: Nathen Harvey [mailto:nharvey@chef.io]
Sent: Friday, April 24, 2015 12:48 PM
To: chef@lists.opscode.com
Subject: [chef] Re: RE: How to manage cookbook versions more efficiently?

I'm not sure I understand why you have a single role for each node in
your

infrastructure. Could you help me understand that? Or do you mean you
have 25

different types of nodes in your infrastructure where each type might
have N

instances?

Have you considered using Environments [1] for pinning your cookbook
versions?

I think this might be a more sustainable approach for what you've
described.

-Nathen

1: About Environments

On Fri, Apr 24, 2015 at 12:25 PM, Fouts, Chris Chris.Fouts@sensus.com
wrote:

" For complete consistency, you have to specify every single cookbook,
including all dependencies for and on your desired cookbooks in your roles
or run_lists. Your role based approach gets very nasty if you start mixing
roles, and mixing conflicting cookbook versions or unintended dependencies.
There were big problems when the yum and mysql cookbooks were updated and
were incompatible with many older, stable, tested, production cookbooks that
relied on them."

That's why I asked the question, you just seconded it.

Chris

-----Original Message-----
From: Nico Kadel-Garcia [mailto:nkadel@skyhookwireless.com]
Sent: Friday, April 24, 2015 11:41 AM
To: chef@lists.opscode.com

Subject: [chef] RE: How to manage cookbook versions more efficiently?

From: Fouts, Chris [mailto:Chris.Fouts@Sensus.com]
Sent: Thursday, April 23, 2015 10:29 AM
To: chef@lists.opscode.com
Subject: [chef] How to manage cookbook versions more efficiently?

I use role cookbooks to pin down versions of the specific versions of
the cookbooks they use. Since I have 25 nodes in my product and each node
has a role, I have at least 25 role cookbooks. I just then add my role
cookbooks to my nodes' run list. For example I have: the following. I DO
want to pin a particular cookbook version in my role cookbooks.

Any ideas on how to alleviate this situation?

For complete consistency, you have to specify every single cookbook,
including all dependencies for and on your desired cookbooks in your roles
or run_lists. Your role based approach gets very nasty if you start mixing
roles, and mixing conflicting cookbook versions or unintended dependencies.
There were big problems when the yum and mysql cookbooks were updated and
were incompatible with many older, stable, tested, production cookbooks that
relied on them.

This is one of my major reasons for giving up on the
"chef-server/chef-client" model, and preferring "chef-solo" for small
environments. I can lock down every single cookbook in Berkshelf in a much
more controlled fashion than mixing and matching and unweaving roles,
cookbook, or environment wrappers, and I can apply an updated or testing
cookbook on a single host with a locally updated or git branched
Berkshelf.lock without potentially inflicting it on any other unexpected
host. There are costs: using chefdk is a fast way to get a full Berkshelf
enabled chef-solo environment, but it's not pre-built for all operating
systems that Chef supports.

Nico Kadel-Garcia
Lead DevOps Engineer
nkadel@skyhookwireless.com

--

Greg Damiani | Senior System Administrator | @damianigreg

BuzzFeed: The Social News and Entertainment Company

40 Argyll Street, 2nd Floor, London, W1F 7EB

Topic		Replies	Views
Which cookbook version wins in this case? Chef Infra (archive)	7	358	March 12, 2015
Opinions on Cookbook version contraints - environment or cookbook Chef Infra (archive)	4	1092	April 22, 2016
Role Versioning? Chef Infra (archive)	25	688	February 15, 2013
How to update cookbook version in environment cookbook Chef Infra (archive)	1	901	July 29, 2016
Roles and Versioning Chef Infra (archive)	15	409	November 27, 2012

How to manage cookbook versions more efficiently?

Related Topics