Life cycle of a chef run

Chef Infra (archive)
1

Is there a way to wrap some code around the chef run?

I have two specific goals.

  1. Catch exceptions and email them to me.
  2. Send some data to my monitoring service after the run (if nothing
    else just the fact that the run took place and went OK but I would
    like to include other data from ohai).
2

I second this. I had the same need, and I ended up rolling my own scripts to do this. Otherwise there is no visibility into what's "really" going on in the deployment. However, while this is certainly an important part of "managing your servers", it might be outside of the scope of Chef's goals. Chef is good at "configuring your servers", but is it the intent of Chef to (eventually) provide support for "monitoring your servers"?

-Nick

Tim Uckun wrote:

Is there a way to wrap some code around the chef run?

I have two specific goals.

  1. Catch exceptions and email them to me.
  2. Send some data to my monitoring service after the run (if nothing
    else just the fact that the run took place and went OK but I would
    like to include other data from ohai).
3

On Wed, Oct 21, 2009 at 2:56 PM, Tim Uckun timuckun@gmail.com wrote:

Is there a way to wrap some code around the chef run?

Not yet - we have an open Feature Proposal that has some examples of
how you can get this functionality today, though:

http://wiki.opscode.com/display/chef/After+hooks

I have two specific goals.

  1. Catch exceptions and email them to me.

  2. Send some data to my monitoring service after the run (if nothing
    else just the fact that the run took place and went OK but I would
    like to include other data from ohai).

In both these cases, you'll want to have a recipe that is at the tail
end of your run. Have it walk the collection (like in the example in
the After+Hooks) and go nuts. Stick it in a ruby block and inspect
whether each resource is "updated" to get end-of-run information about
what happened.

Improving this kind of reporting is definitely on them map.

Adam

--
Opscode, Inc.
Adam Jacob, CTO
T: (206) 508-7449 E: adam@opscode.com

4

Tim,One (really hacky, some might even say kludgy) way to accomplish what
you want, at least for reporting of failed runs via email, would be to
include a Ruby at_exit hook in a recipe. Note that this is only effective
if you run the client via cron or some mechanism where the chef-client
program exits--so anything using the interval (-i) option won't work with
this hack. Here's some code that demonstrates the various behaviors you can
see with at_exit

at_exit do

p $!.inspect

end

def raise_error

raise "an uncaught error"

end

def let_script_reach_eof

don't need to do anything

end

def raise_and_rescue

begin

raise "an error that will be rescued"

rescue => e

# don't need to do anything

end

end

def exit_with_exit

exit 42

end

In the code above, $! is the last "unrescued" error raised. I didn't have a
chance to see what the results are when using this in a chef run, so you'll
need to do additional research on your own, but from what I can tell you'll
have these cases:

  • for a successful run, you'll have $!.kind_of?(SystemExit) #=> true and
    $!.success? #=> true
  • Chef might kill itself due to a configuration problem or something, in
    this case you'll probably see $!.kind_of?(SystemExit) #=> true and
    $!.success #=> false
  • for a run that dies due to an error being raised in a recipe, you should
    that error in $!, i.e., $!.kind_of?(SystemExit) #=> false

Hope this helps,
Daniel DeLeo

On Thu, Oct 22, 2009 at 10:27 AM, Nick Ohanian nick@geodelic.com wrote:

I second this. I had the same need, and I ended up rolling my own scripts
to do this. Otherwise there is no visibility into what's "really" going on
in the deployment. However, while this is certainly an important part of
"managing your servers", it might be outside of the scope of Chef's goals.
Chef is good at "configuring your servers", but is it the intent of Chef to
(eventually) provide support for "monitoring your servers"?

-Nick

Tim Uckun wrote:

Is there a way to wrap some code around the chef run?

I have two specific goals.

  1. Catch exceptions and email them to me.
  2. Send some data to my monitoring service after the run (if nothing
    else just the fact that the run took place and went OK but I would
    like to include other data from ohai).
5

On Fri, Oct 23, 2009 at 6:14 AM, Daniel DeLeo dan@kallistec.com wrote:

Tim,
One (really hacky, some might even say kludgy) way to accomplish what you
want, at least for reporting of failed runs via email, would be to include a
Ruby at_exit hook in a recipe. Note that this is only effective if you run
the client via cron or some mechanism where the chef-client program
exits--so anything using the interval (-i) option won't work with this hack.
Here's some code that demonstrates the various behaviors you can see with
at_exit

Thanks. That's an approach I might try.

I use zabbix for monitoring so I could also send some data to the
monitoring server from those hooks too I suppose.

In any case it might be enough (for me) to know that something went
wrong even if it means I have to log into the server and investigate
further.