# Abusing Startup Handlers

**URL:** <https://discourse.chef.io/t/abusing-startup-handlers/2605>\
**Category:** Chef Infra (archive)\
**Created:** [March 28, 2012, 9:38am UTC](https://discourse.chef.io/t/abusing-startup-handlers/2605 "2012-03-28T09:38:05Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Os4839](https://avatars.discourse-cdn.com/v4/letter/o/b782af/32.png) [@Os4839](https://discourse.chef.io/u/Os4839)\
**Post date:** [March 28, 2012, 9:38am UTC](https://discourse.chef.io/t/abusing-startup-handlers/2605/1 "2012-03-28T09:38:05Z")

</div>

Hi,

I’m currently thinking about abusing the Chef startup handlers for more  
stability in a compute cluster.

We are running a compute cluster for scientific analysis with a resource  
management system where you have a master node responsible for scheduling  
analysis jobs to execution nodes. For each of the execution nodes I would  
like to do the following:

1. When they start a chef run, they send a “Don’t send any new workloads to  
me” message to the master node
2. When the chef run was successful they send a “Ok. I’m up for some more  
jobs now.” message to the master node.

My goal is to have nodes on which a chef run fails to be disabled in the  
resource management system because a failed chef run most of the time means  
that the node is in a bad/undefined state so chances are that new jobs  
scheduled there will fail.

I could obviously have the node disabled with an execute resource in the  
very first recipe, but what if one of the ohai plugins gets stuck and chef  
never makes it to that first recipe?

Therefore I would like to disable the execution node at the very beginning  
of the chef run. Is this possible with the startup handlers?

As I understood it, the start up handlers are run after ohai. Wouldn’t it  
be reasonable to run them before ohai starts, thus making the startup  
handlers the very first thing to run on a node?

Cheers,

Oscar

---

<div class="post-metadata">

**Author:** ![Ranjib\_Dey](https://avatars.discourse-cdn.com/v4/letter/r/8e8cbc/32.png) [@Ranjib\_Dey](https://discourse.chef.io/u/Ranjib_Dey)\
**Post date:** [March 28, 2012, 10:05am UTC](https://discourse.chef.io/t/abusing-startup-handlers/2605/2 "2012-03-28T10:05:06Z")

</div>

You have highlighted multiple things here, if you plan to orchestrate cross  
node services , where status of chef run should impact whether another chef  
run will be triggered or not, you can try mcollective.  
also, for custom solutions,  
node.ohai\_time stores the time of last successful chef run occurred , if  
your chef runs are scheduled periodically (or with a fixed splay time) you  
can exploit this attribute in chef startup handler to decide whether a node  
has converged successfully or not (and also you can use the rabbitmq that  
you are using with chef server to modle pub/sub like workflows).

On Wed, Mar 28, 2012 at 3:08 PM, oscar schneider [os4839@googlemail.com](mailto:os4839@googlemail.com)wrote:

> Hi,
> 
> I'm currently thinking about abusing the Chef startup handlers for more  
> stability in a compute cluster.
> 
> We are running a compute cluster for scientific analysis with a resource  
> management system where you have a master node responsible for scheduling  
> analysis jobs to execution nodes. For each of the execution nodes I would  
> like to do the following:
> 
> 1. When they start a chef run, they send a "Don't send any new workloads  
> to me" message to the master node
> 2. When the chef run was successful they send a "Ok. I'm up for some more  
> jobs now." message to the master node.
> 
> My goal is to have nodes on which a chef run fails to be disabled in the  
> resource management system because a failed chef run most of the time means  
> that the node is in a bad/undefined state so chances are that new jobs  
> scheduled there will fail.
> 
> I could obviously have the node disabled with an execute resource in the  
> very first recipe, but what if one of the ohai plugins gets stuck and chef  
> never makes it to that first recipe?
> 
> Therefore I would like to disable the execution node at the very beginning  
> of the chef run. Is this possible with the startup handlers?
> 
> As I understood it, the start up handlers are run after ohai. Wouldn't it  
> be reasonable to run them before ohai starts, thus making the startup  
> handlers the very first thing to run on a node?
> 
> Cheers,
> 
> Oscar