Cookbook granularity


#1

All,

I would like to find out how granular you keep your cookbooks. I feel like
I have to give a bit of a long-winded introduction to the point of my
question, but please bear with me for a minute or two whilst I illustrate.

I manage a number of Ubuntu systems, amongst other things, on various
levels of uptodateness. Let me use ‘lucid’ and ‘precise’ as cases in point
here. I use chef to create a number of cookbooks, assign roles to the
nodes, populate the roles with recipes according to their purpose, and
everything works fine.

Every machine periodically runs apticron, which notifies me of package
updates. I use these notifications to refine the cookbooks, making
decisions along the way which packages should receive an :update action in
addition to their :install action - some things I want updated
automatically, some things I don’t.

This is where the fun begins. As an example, let us assume that I have a
security update to libjpeg pending. Yes, that happens. On a system which
has various tools like graphviz and gnuplot installed, there are multiple
packages that depend on libjpeg. Now I have to make a decision which of
the receipes I should let update libjpeg. If I add it to only one, I can’t
rip out the other cookbook to use it in a different chef setup without
having to amend it later in the same way. Or I could add it to both, but
maintenance overhead may increase drastically depending on how many
recipes I have to amend this way.

The next logical step would be to create a libjpeg cookbook and make both
graphviz and gnuplot depend on it from the start.

Let us now assume that the decision is made to upgrade the target node in
question from lucid to precise. Ubuntu packaging has a few interesting
quirks, including the fact that there actually is no ‘libjpeg’ - version
numbers are being added to library package names in many cases, but not
necessarily in all of them. In the case at hand, the package name changes
from libjpeg62 to libjpeg8. So what I would have to do next is to
introduce a case statement in the libjpeg cookbook which switches over
node[:lsb][:codename] and installs the correct package. Note btw that the
versioning has only very little to do with the actual version of the
upstream software package; as an example, follow the versioning of
libgnutls package and sources from hardy through precise to get an idea
what I mean.

The next thing that creeps up is that libjpeg on precise is not
’self-contained’ anymore, but it now depends on another libjpeg
implementation package (libjpeg-turbo8). In other words, the dependency
chain has changed, too. Do I turn this into yet another separate cookbook
in case of updates? Or will it just become a dependency that I add to
libjpeg? Also, how can I be sure that such version changes only ever occur
between distro upgrades?

The conceptual problem that I have here is that I seem to end up
replicating the data structures that underlie the package management
system into a separate tool. Is that necessary? Does it even make sense?
Should I come up with a few lines of Perl code based on AptPkg (in the
vein of debtree) that take any given package name and create the two dozen
interdependent cookbooks as required?

How do you people deal with this? Do you even think it’s an issue? Am I
trying to be too detailed? Am I using the wrong tool to keep my systems up
to date? Should I maybe focus on using chef to get a system to a defined
initial state and manage updates somehow different? But then why would
chef offer an :update action in the first place?

Thanks for reading this far :slight_smile:

Cheers,

t


Error opening ~/.signature


#2

Personally, my package resources in recipes use a version that is found
from a node attribute, and then I set different versions as needed at
the environment level as an override attribute over the default as
needed. ie i manage all package updates through Chef. Because of
inheritance, there is only one recipe containing each packge resource,
which may be references as needed from one or hundreds of roles.

We get around any problem with package names changing (as opposed to
package versions) by building packages in-house from source. That’s part
of our standard process because its a corporate environment and the
packages are security checked etc. So this means we don’t encounter the
problem you describe, but also means I don’t have an easy answer for how
you can get around it.

It would seem very odd for a package to change name within a
distribution major release version, and normally where this happens you
have a parent/virtual package that translates to the version-specific
packge names. eg there would on most distributions always be a
"libjpeg-virtual" or something that is like a package “symlink” to
"libjpeg62" or whatever.

If this is not the case on your distribution, then you could always
engineer a similar mechanism yourself. I guess what you want is some
kind of meta-package definition to map from a package at one version to
another package at a different version (eg libjpeg62 maps to libjpeg
version6.2 and libpjg8 maps to libjpeg version8.0). Without thinking
about this too much (so this may be nonsense) I guess if I was in this
situation I would use LWRP to write some kind of metapackage resource
for this intermediary translation of package names.

You may be worrying about something unecessarily though because I’ve
not come across such a problem. If you say what distribution you are
using, we can check and see if parent/virtual package names are
implemented in it

Cheers

Dan

On 11.05.2012 13:21, Thorsten Fischer wrote:

All,

I would like to find out how granular you keep your cookbooks. I feel
like
I have to give a bit of a long-winded introduction to the point of my
question, but please bear with me for a minute or two whilst I
illustrate.

I manage a number of Ubuntu systems, amongst other things, on various
levels of uptodateness. Let me use ‘lucid’ and ‘precise’ as cases in
point
here. I use chef to create a number of cookbooks, assign roles to the
nodes, populate the roles with recipes according to their purpose,
and
everything works fine.

Every machine periodically runs apticron, which notifies me of
package
updates. I use these notifications to refine the cookbooks, making
decisions along the way which packages should receive an :update
action in
addition to their :install action - some things I want updated
automatically, some things I don’t.

This is where the fun begins. As an example, let us assume that I
have a
security update to libjpeg pending. Yes, that happens. On a system
which
has various tools like graphviz and gnuplot installed, there are
multiple
packages that depend on libjpeg. Now I have to make a decision which
of
the receipes I should let update libjpeg. If I add it to only one, I
can’t
rip out the other cookbook to use it in a different chef setup
without
having to amend it later in the same way. Or I could add it to both,
but
maintenance overhead may increase drastically depending on how many
recipes I have to amend this way.

The next logical step would be to create a libjpeg cookbook and make
both
graphviz and gnuplot depend on it from the start.

Let us now assume that the decision is made to upgrade the target
node in
question from lucid to precise. Ubuntu packaging has a few
interesting
quirks, including the fact that there actually is no ‘libjpeg’ -
version
numbers are being added to library package names in many cases, but
not
necessarily in all of them. In the case at hand, the package name
changes
from libjpeg62 to libjpeg8. So what I would have to do next is to
introduce a case statement in the libjpeg cookbook which switches
over
node[:lsb][:codename] and installs the correct package. Note btw that
the
versioning has only very little to do with the actual version of the
upstream software package; as an example, follow the versioning of
libgnutls package and sources from hardy through precise to get an
idea
what I mean.

The next thing that creeps up is that libjpeg on precise is not
’self-contained’ anymore, but it now depends on another libjpeg
implementation package (libjpeg-turbo8). In other words, the
dependency
chain has changed, too. Do I turn this into yet another separate
cookbook
in case of updates? Or will it just become a dependency that I add to
libjpeg? Also, how can I be sure that such version changes only ever
occur
between distro upgrades?

The conceptual problem that I have here is that I seem to end up
replicating the data structures that underlie the package management
system into a separate tool. Is that necessary? Does it even make
sense?
Should I come up with a few lines of Perl code based on AptPkg (in
the
vein of debtree) that take any given package name and create the two
dozen
interdependent cookbooks as required?

How do you people deal with this? Do you even think it’s an issue? Am
I
trying to be too detailed? Am I using the wrong tool to keep my
systems up
to date? Should I maybe focus on using chef to get a system to a
defined
initial state and manage updates somehow different? But then why
would
chef offer an :update action in the first place?

Thanks for reading this far :slight_smile:

Cheers,

t


#3

Personally, my package resources in recipes use a version that is found
from a node attribute, and then I set different versions as needed at
the environment level as an override attribute over the default as
needed. ie i manage all package updates through Chef. Because of
inheritance, there is only one recipe containing each packge resource,
which may be references as needed from one or hundreds of roles.

We get around any problem with package names changing (as opposed to
package versions) by building packages in-house from source. That’s part
of our standard process because its a corporate environment and the
packages are security checked etc. So this means we don’t encounter the
problem you describe, but also means I don’t have an easy answer for how
you can get around it.

It would seem very odd for a package to change name within a
distribution major release version, and normally where this happens you
have a parent/virtual package that translates to the version-specific
packge names. eg there would on most distributions always be a
"libjpeg-virtual" or something that is like a package “symlink” to
"libjpeg62" or whatever.

If this is not the case on your distribution, then you could always
engineer a similar mechanism yourself. I guess what you want is some
kind of meta-package definition to map from a package at one version to
another package at a different version (eg libjpeg62 maps to libjpeg
version6.2 and libpjg8 maps to libjpeg version8.0). Without thinking
about this too much (so this may be nonsense) I guess if I was in this
situation I would use LWRP to write some kind of metapackage resource
for this intermediary translation of package names.

You may be worrying about something unecessarily though because I’ve
not come across such a problem. If you say what distribution you are
using, we can check and see if parent/virtual package names are
implemented in it

Cheers

Dan

On 11.05.2012 13:21, Thorsten Fischer wrote:

All,

I would like to find out how granular you keep your cookbooks. I feel
like
I have to give a bit of a long-winded introduction to the point of my
question, but please bear with me for a minute or two whilst I
illustrate.

I manage a number of Ubuntu systems, amongst other things, on various
levels of uptodateness. Let me use ‘lucid’ and ‘precise’ as cases in
point
here. I use chef to create a number of cookbooks, assign roles to the
nodes, populate the roles with recipes according to their purpose,
and
everything works fine.

Every machine periodically runs apticron, which notifies me of
package
updates. I use these notifications to refine the cookbooks, making
decisions along the way which packages should receive an :update
action in
addition to their :install action - some things I want updated
automatically, some things I don’t.

This is where the fun begins. As an example, let us assume that I
have a
security update to libjpeg pending. Yes, that happens. On a system
which
has various tools like graphviz and gnuplot installed, there are
multiple
packages that depend on libjpeg. Now I have to make a decision which
of
the receipes I should let update libjpeg. If I add it to only one, I
can’t
rip out the other cookbook to use it in a different chef setup
without
having to amend it later in the same way. Or I could add it to both,
but
maintenance overhead may increase drastically depending on how many
recipes I have to amend this way.

The next logical step would be to create a libjpeg cookbook and make
both
graphviz and gnuplot depend on it from the start.

Let us now assume that the decision is made to upgrade the target
node in
question from lucid to precise. Ubuntu packaging has a few
interesting
quirks, including the fact that there actually is no ‘libjpeg’ -
version
numbers are being added to library package names in many cases, but
not
necessarily in all of them. In the case at hand, the package name
changes
from libjpeg62 to libjpeg8. So what I would have to do next is to
introduce a case statement in the libjpeg cookbook which switches
over
node[:lsb][:codename] and installs the correct package. Note btw that
the
versioning has only very little to do with the actual version of the
upstream software package; as an example, follow the versioning of
libgnutls package and sources from hardy through precise to get an
idea
what I mean.

The next thing that creeps up is that libjpeg on precise is not
’self-contained’ anymore, but it now depends on another libjpeg
implementation package (libjpeg-turbo8). In other words, the
dependency
chain has changed, too. Do I turn this into yet another separate
cookbook
in case of updates? Or will it just become a dependency that I add to
libjpeg? Also, how can I be sure that such version changes only ever
occur
between distro upgrades?

The conceptual problem that I have here is that I seem to end up
replicating the data structures that underlie the package management
system into a separate tool. Is that necessary? Does it even make
sense?
Should I come up with a few lines of Perl code based on AptPkg (in
the
vein of debtree) that take any given package name and create the two
dozen
interdependent cookbooks as required?

How do you people deal with this? Do you even think it’s an issue? Am
I
trying to be too detailed? Am I using the wrong tool to keep my
systems up
to date? Should I maybe focus on using chef to get a system to a
defined
initial state and manage updates somehow different? But then why
would
chef offer an :update action in the first place?

Thanks for reading this far :slight_smile:

Cheers,

t