How to tell if Chef is running?

Chef Infra (archive)
1

Is there a standard way to tell if your (ruby) code is being invoked (indirectly) by a Chef run? In this specific case the code is executing in a RubyGems pre-install hook.

Semi-relatedly, Chef seems strongly aimed at being run to configure the machine, and then mostly not run again on the same node except to add more software. For example, there aren’t a lot of uninstall-based resources, I don’t see an obvious way to “reverse” a cookbook, few cookbooks seem written to work incrementally or in stages, etc. Is that pretty much how it’s intended, or am I missing something?

To be fair, I haven’t come up with a good scenario yet for why you’d want to run it on a more continuous basis (at least, not one where you couldn’t do it better in some other way). But it’s odd because a lot of the individual resources seem to be kind of incremental in feel (specify how many backups to make, specify that you’re syncing the git repository rather than just creating it, etc) – almost as though they’re meant for repeated use as a deployment tool, for instance. But the cookbooks don’t seem to be written the same way. It’s slightly confusing.

2

On Sun, Jan 23, 2011 at 11:54 PM, Noah Gibbs noah_gibbs@yahoo.com wrote:

Is there a standard way to tell if your (ruby) code is being invoked
(indirectly) by a Chef run? In this specific case the code is executing in
a RubyGems pre-install hook.

Well, there's always the log resource.

Semi-relatedly, Chef seems strongly aimed at being run to configure the

machine, and then mostly not run again on the same node except to add more
software. For example, there aren't a lot of uninstall-based resources, I
don't see an obvious way to "reverse" a cookbook, few cookbooks seem written
to work incrementally or in stages, etc. Is that pretty much how it's
intended, or am I missing something?

Things which are meant to be uninstallable should be written as resource
providers with alternate actions. The LWRP mechanism makes this
straightforward.

That said -- cookbooks work incrementally as a matter of course, by virtue
of the resource providers they're built from being (where possible)
idempotent in their actions. The "install this package" providers do nothing
if the package is already there, the "fill out this file from this template"
does nothing (and generates no notification-type events) if the file already
has the desired contents, etc.

On the occasions where I want some magic -- such that removing role $foo
from a node will cause all the software installed by role $foo to be
removed, for instance -- I've had to roll my own thus far. The same is true
with Puppet; the only provisioning system I've seen which made that kind of
thing easy was one I wrote myself at a former employer half a decade ago,
and which I have no intent of ever resurrecting.

3

On Mon, Jan 24, 2011 at 12:54 AM, Noah Gibbs noah_gibbs@yahoo.com wrote:

Is there a standard way to tell if your (ruby) code is being invoked (indirectly) by a Chef run? In this specific case the code is executing in a RubyGems pre-install hook.

Semi-relatedly, Chef seems strongly aimed at being run to configure the machine, and then mostly not run again on the same node except to add more software. For example, there aren't a lot of uninstall-based resources, I don't see an obvious way to "reverse" a cookbook, few cookbooks seem written to work incrementally or in stages, etc. Is that pretty much how it's intended, or am I missing something?

To be fair, I haven't come up with a good scenario yet for why you'd want to run it on a more continuous basis (at least, not one where you couldn't do it better in some other way). But it's odd because a lot of the individual resources seem to be kind of incremental in feel (specify how many backups to make, specify that you're syncing the git repository rather than just creating it, etc) -- almost as though they're meant for repeated use as a deployment tool, for instance. But the cookbooks don't seem to be written the same way. It's slightly confusing.
Charles, already covered the second part a little bit but my solution
is to track those types of undo changes in a data bag. For instance,
for users, I have a deleted_users databag that is used as part of my
user management cookbook:

  • Search users and deleted users data bags.
  • prune any users in deleted users from users
  • run the standard user management tasks in the recipes
  • delete any users in the deleted users data bag

Realize that regardless of state of the system, over the course of a
few runs, any users who shouldnt be there (that chef is aware of) will
get removed. Eventually the system will converge in the way you want.
Now that strategy works for users but not, directly, for role changes.
While I have had to deal with it yet, my strategy is essentially to
manage that via via a transient role that removes itself at the end.
Assume I want to change a system from role[mysql] to role[pgsql]:

  • Add role[uninstall_mysql] to the system.
  • Have a mysql::uninstall recipe that, when it sees a system has a
    role of uninstall_mysql, it removes and cleans up after itself.
  • mysql::uninstall then removes the role uninstall_mysql from the node.

The benefit is you can do things like "backup all the database before
uninstalling" as part of that recipe. Yes, you have to manage some of
the idempotence yourself but that will be the case regardless of the
tool if it doesn't track those types of state changes.

But honestly the best approach is a "kick it/not fix it"* one. If you
have all of the recipes that can get you to role B from bare metal,
why spend the time rolling all the logic into a special recipe? Just
kick the node from scratch with role A.

John E. Vincent

4

You could potentially model the 'un' of every resource in an inverse
cookbook, but there are usually enough complicated corner cases that the
prevailing wisdom, when possible, is to build systems in such a way that
each node can be blow away and rebuilt.

Your mileage may vary.

On Sun, Jan 23, 2011 at 10:54 PM, Noah Gibbs noah_gibbs@yahoo.com wrote:

Is there a standard way to tell if your (ruby) code is being invoked
(indirectly) by a Chef run? In this specific case the code is executing in
a RubyGems pre-install hook.

Semi-relatedly, Chef seems strongly aimed at being run to configure the
machine, and then mostly not run again on the same node except to add more
software. For example, there aren't a lot of uninstall-based resources, I
don't see an obvious way to "reverse" a cookbook, few cookbooks seem written
to work incrementally or in stages, etc. Is that pretty much how it's
intended, or am I missing something?

To be fair, I haven't come up with a good scenario yet for why you'd want
to run it on a more continuous basis (at least, not one where you couldn't
do it better in some other way). But it's odd because a lot of the
individual resources seem to be kind of incremental in feel (specify how
many backups to make, specify that you're syncing the git repository rather
than just creating it, etc) -- almost as though they're meant for repeated
use as a deployment tool, for instance. But the cookbooks don't seem to be
written the same way. It's slightly confusing.

5

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Ohai!

On Jan 23, 2011, at 10:54 PM, Noah Gibbs wrote:

Is there a standard way to tell if your (ruby) code is being invoked (indirectly) by a Chef run? In this specific case the code is executing in a RubyGems pre-install hook.

You can use 'shef' to debug recipes.

http://wiki.opscode.com/display/chef/Shef

Semi-relatedly, Chef seems strongly aimed at being run to configure the machine, and then mostly not run again on the same node except to add more software. For example, there aren't a lot of uninstall-based resources, I don't see an obvious way to "reverse" a cookbook, few cookbooks seem written to work incrementally or in stages, etc. Is that pretty much how it's intended, or am I missing something?

As stated by others already, you would have to write "undo" style recipes that call the opposite actions on the various resources.

To be fair, I haven't come up with a good scenario yet for why you'd want to run it on a more continuous basis (at least, not one where you couldn't do it better in some other way). But it's odd because a lot of the individual resources seem to be kind of incremental in feel (specify how many backups to make, specify that you're syncing the git repository rather than just creating it, etc) -- almost as though they're meant for repeated use as a deployment tool, for instance. But the cookbooks don't seem to be written the same way. It's slightly confusing.

Dynamic infrastructure configurations that use a lot of search might want to ensure they're running on a somewhat continuous basis. For example, the Opscode Nagios cookbook0 does a lot of searches based on roles and nodes. To ensure that all systems are monitored, you might run the chef-client on the Nagios server every 10 minutes. In environments that do application deployment with a Chef recipe1, and practice continuous deployment, an interval may make sense as well.

In the case of cookbooks published by Opscode, we subscribe to the idea that infrastructure should be treated as disposable. It is generally[2] easier to simply terminate a VM/cloud instance and start a new one (in two commands, even), than it is to put effort into "undo" recipes, so our cookbooks do not have such recipes included.

[2]: John Vincent's users scenario is a good one to consider, we just haven't written similar in our own users cookbook, yet :-).

Opscode, Inc
Joshua Timberman, Technical Evangelist
IRC, Skype, Twitter, Github: jtimberman

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (Darwin)

iEYEARECAAYFAk09J7QACgkQO97WSdVpzT2b0gCeKxX9ryH5od6CFOch5Lw+E4Z+
ML0AoJmEGuN50S9KG28326+0RkUv+tr7
=VtQq
-----END PGP SIGNATURE-----