# Progress of a chef run

**URL:** <https://discourse.chef.io/t/progress-of-a-chef-run/4392>\
**Category:** Chef Infra (archive)\
**Created:** [August 6, 2013, 10:38am UTC](https://discourse.chef.io/t/progress-of-a-chef-run/4392 "2013-08-06T10:38:54Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Lucian\_Ciufudean](https://avatars.discourse-cdn.com/v4/letter/l/c5a1d2/32.png) [@Lucian\_Ciufudean](https://discourse.chef.io/u/Lucian_Ciufudean)\
**Post date:** [August 6, 2013, 10:38am UTC](https://discourse.chef.io/t/progress-of-a-chef-run/4392/1 "2013-08-06T10:38:54Z")

</div>

Guys, is there a mechanism (maybe a chef handler trick) so that I can report  
(in a file for ex) the report of a long running chef run?

Cheers  
Lucian

---

<div class="post-metadata">

**Author:** ![Mike](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/mike/32/18_2.png) [@Mike](https://discourse.chef.io/u/Mike)\
**Post date:** [August 6, 2013, 11:29am UTC](https://discourse.chef.io/t/progress-of-a-chef-run/4392/2 "2013-08-06T11:29:01Z")

</div>

Take a look at this handler:

> **[GitHub - jamesc/chef-handler-elapsed-time: A chef handler that reports on...](https://github.com/jamesc/chef-handler-elapsed-time)**
>
> A chef handler that reports on per-resource elasped times in a simple graphical form - GitHub - jamesc/chef-handler-elapsed-time: A chef handler that reports on per-resource elasped times in a simp...

-M

On Tuesday, August 6, 2013, wrote:

> Guys, is there a mechanism (maybe a chef handler trick) so that I can  
> report  
> (in a file for ex) the report of a long running chef run?
> 
> Cheers  
> Lucian

---

<div class="post-metadata">

**Author:** ![Lucian\_Ciufudean](https://avatars.discourse-cdn.com/v4/letter/l/c5a1d2/32.png) [@Lucian\_Ciufudean](https://discourse.chef.io/u/Lucian_Ciufudean)\
**Post date:** [August 8, 2013, 10:33am UTC](https://discourse.chef.io/t/progress-of-a-chef-run/4392/3 "2013-08-08T10:33:20Z")

</div>

I might have not been clear.  
Let's say I start a chef run with 1k recipes and it will take 24h to  
execute, is there a callback mechanism that chef uses while the run is in  
progress, not after its completion as is the case with report handlers? I  
want to be able to say to the user at one moment that "55 out of 1000  
resources executed".

Lucian

On Tue, Aug 6, 2013 at 2:29 PM, Mike [miketheman@gmail.com](mailto:miketheman@gmail.com) wrote:

> Take a look at this handler:  
> [GitHub - jamesc/chef-handler-elapsed-time: A chef handler that reports on per-resource elasped times in a simple graphical form](https://github.com/jamesc/chef-handler-elapsed-time)  
> -M
> 
> On Tuesday, August 6, 2013, wrote:
> 
> > Guys, is there a mechanism (maybe a chef handler trick) so that I can  
> > report  
> > (in a file for ex) the report of a long running chef run?
> > 
> > Cheers  
> > Lucian

---

<div class="post-metadata">

**Author:** ![Jeff\_Blaine](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/jeff_blaine/32/19_2.png) [@Jeff\_Blaine](https://discourse.chef.io/u/Jeff_Blaine)\
**Post date:** [August 8, 2013, 1:19pm UTC](https://discourse.chef.io/t/progress-of-a-chef-run/4392/4 "2013-08-08T13:19:23Z")

</div>

On 8/8/2013 6:33 AM, Lucian Ciufudean wrote:

> I might have not been clear.  
> Let's say I start a chef run with 1k recipes and it will take 24h to  
> execute, is there a callback mechanism that chef uses while the run is  
> in progress, not after its completion as is the case with report  
> handlers? I want to be able to say to the user at one moment that "55  
> out of 1000 resources executed".

I'm not aware of such a thing built into Chef, but one has to wonder  
what the real value of that is if step 56 is going to take 2hrs, 57-83  
will take 45 seconds, 84 will take 20 minutes, etc.

---

<div class="post-metadata">

**Author:** ![Ranjib](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/ranjib/32/1040_2.png) [@Ranjib](https://discourse.chef.io/u/Ranjib)\
**Post date:** [August 8, 2013, 3:03pm UTC](https://discourse.chef.io/t/progress-of-a-chef-run/4392/5 "2013-08-08T15:03:14Z")

</div>

Event handlers can do this.  
On Aug 8, 2013 3:34 AM, "Lucian Ciufudean" [lucian.ciufudean@gmail.com](mailto:lucian.ciufudean@gmail.com)  
wrote:

> I might have not been clear.  
> Let's say I start a chef run with 1k recipes and it will take 24h to  
> execute, is there a callback mechanism that chef uses while the run is in  
> progress, not after its completion as is the case with report handlers? I  
> want to be able to say to the user at one moment that "55 out of 1000  
> resources executed".
> 
> Lucian
> 
> On Tue, Aug 6, 2013 at 2:29 PM, Mike [miketheman@gmail.com](mailto:miketheman@gmail.com) wrote:
> 
> > Take a look at this handler:  
> > [GitHub - jamesc/chef-handler-elapsed-time: A chef handler that reports on per-resource elasped times in a simple graphical form](https://github.com/jamesc/chef-handler-elapsed-time)  
> > -M
> > 
> > On Tuesday, August 6, 2013, wrote:
> > 
> > > Guys, is there a mechanism (maybe a chef handler trick) so that I can  
> > > report  
> > > (in a file for ex) the report of a long running chef run?
> > > 
> > > Cheers  
> > > Lucian

---

<div class="post-metadata">

**Author:** ![steve](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/steve/32/294_2.png) [@steve](https://discourse.chef.io/u/steve)\
**Post date:** [August 8, 2013, 6:33pm UTC](https://discourse.chef.io/t/progress-of-a-chef-run/4392/6 "2013-08-08T18:33:32Z")

</div>

It looks like event handlers can now hook into one of three places:

- Start handlers hook in at the beginning of a run.
- Report handlers hook in at the end of a successful run.
- Exception handlers hook in after an exception is raised (i.e., after an  
_unsuccessful_ run).

It seems to me that you'd need your code to fire at the beginning of a run  
and monitor resources as they're converged. So that'd be a start handler.  
I don't know of any start handlers in the wild (or within my organization)  
... so I'm not sure how well-explored this area is in the community. Or  
even if there's any documentation on using them.

I suggest an alternative: Write your own Chef output formatter that  
streams a progress update to a remote service.

Observe the behavior of the nyancat Chef formatter:

> <https://github.com/andreacampi/nyan-cat-chef-formatter/blob/master/lib/nyan-cat-chef-formatter.rb>

Just take out the nyancat bits and replace them with posting a JSON object  
containing, perhaps, the run list being executed, start/end times of the  
run, the number of resources being considered and the number of resources  
processed and the final status of the run (success/fail).

If I had a Chef run that was taking 24 hours, I'd be looking at what was  
taking so long and trying to break those steps out into a separate  
orchestration tool, since I have a RunDeck server lying around that's  
pretty well-suited for scheduling and executing recurring batch jobs. Or,  
to keep it in the Opscode family, I'd be clamoring to take some of those  
long-running processes out of the "regular" Chef run and slot them into  
some kind of recurring Pushy task.

On Thu, Aug 8, 2013 at 8:03 AM, Ranjib Dey [dey.ranjib@gmail.com](mailto:dey.ranjib@gmail.com) wrote:

> Event handlers can do this.  
> On Aug 8, 2013 3:34 AM, "Lucian Ciufudean" [lucian.ciufudean@gmail.com](mailto:lucian.ciufudean@gmail.com)  
> wrote:
> 
> > I might have not been clear.  
> > Let's say I start a chef run with 1k recipes and it will take 24h to  
> > execute, is there a callback mechanism that chef uses while the run is in  
> > progress, not after its completion as is the case with report handlers? I  
> > want to be able to say to the user at one moment that "55 out of 1000  
> > resources executed".
> > 
> > Lucian
> > 
> > On Tue, Aug 6, 2013 at 2:29 PM, Mike [miketheman@gmail.com](mailto:miketheman@gmail.com) wrote:
> > 
> > > Take a look at this handler:  
> > > [GitHub - jamesc/chef-handler-elapsed-time: A chef handler that reports on per-resource elasped times in a simple graphical form](https://github.com/jamesc/chef-handler-elapsed-time)  
> > > -M
> > > 
> > > On Tuesday, August 6, 2013, wrote:
> > > 
> > > > Guys, is there a mechanism (maybe a chef handler trick) so that I can  
> > > > report  
> > > > (in a file for ex) the report of a long running chef run?
> > > > 
> > > > Cheers  
> > > > Lucian

---

<div class="post-metadata">

**Author:** ![Ranjib](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/ranjib/32/1040_2.png) [@Ranjib](https://discourse.chef.io/u/Ranjib)\
**Post date:** [August 8, 2013, 6:36pm UTC](https://discourse.chef.io/t/progress-of-a-chef-run/4392/7 "2013-08-08T18:36:44Z")

</div>

> <https://gist.github.com/ranjib/5960964>

i use it to get real time update, and also im working with another etcd  
based handler for remote notifications (incomplete).

On Thu, Aug 8, 2013 at 11:33 AM, steve . [leftathome@gmail.com](mailto:leftathome@gmail.com) wrote:

> It looks like event handlers can now hook into one of three places:
> 
> - Start handlers hook in at the beginning of a run.
> - Report handlers hook in at the end of a successful run.
> - Exception handlers hook in after an exception is raised (i.e., after an  
> _unsuccessful_ run).
> 
> It seems to me that you'd need your code to fire at the beginning of a run  
> and monitor resources as they're converged. So that'd be a start handler.  
> I don't know of any start handlers in the wild (or within my organization)  
> ... so I'm not sure how well-explored this area is in the community. Or  
> even if there's any documentation on using them.
> 
> I suggest an alternative: Write your own Chef output formatter that  
> streams a progress update to a remote service.
> 
> Observe the behavior of the nyancat Chef formatter:
> 
> [https://github.com/andreacampi/nyan-cat-chef-formatter/blob/master/lib/nyan-cat-chef-formatter.rb](https://github.com/andreacampi/nyan-cat-chef-formatter/blob/master/lib/nyan-cat-chef-formatter.rb)
> 
> Just take out the nyancat bits and replace them with posting a JSON object  
> containing, perhaps, the run list being executed, start/end times of the  
> run, the number of resources being considered and the number of resources  
> processed and the final status of the run (success/fail).
> 
> If I had a Chef run that was taking 24 hours, I'd be looking at what was  
> taking so long and trying to break those steps out into a separate  
> orchestration tool, since I have a RunDeck server lying around that's  
> pretty well-suited for scheduling and executing recurring batch jobs. Or,  
> to keep it in the Opscode family, I'd be clamoring to take some of those  
> long-running processes out of the "regular" Chef run and slot them into  
> some kind of recurring Pushy task.
> 
> On Thu, Aug 8, 2013 at 8:03 AM, Ranjib Dey [dey.ranjib@gmail.com](mailto:dey.ranjib@gmail.com) wrote:
> 
> > Event handlers can do this.  
> > On Aug 8, 2013 3:34 AM, "Lucian Ciufudean" [lucian.ciufudean@gmail.com](mailto:lucian.ciufudean@gmail.com)  
> > wrote:
> > 
> > > I might have not been clear.  
> > > Let's say I start a chef run with 1k recipes and it will take 24h to  
> > > execute, is there a callback mechanism that chef uses while the run is in  
> > > progress, not after its completion as is the case with report handlers? I  
> > > want to be able to say to the user at one moment that "55 out of 1000  
> > > resources executed".
> > > 
> > > Lucian
> > > 
> > > On Tue, Aug 6, 2013 at 2:29 PM, Mike [miketheman@gmail.com](mailto:miketheman@gmail.com) wrote:
> > > 
> > > > Take a look at this handler:  
> > > > [GitHub - jamesc/chef-handler-elapsed-time: A chef handler that reports on per-resource elasped times in a simple graphical form](https://github.com/jamesc/chef-handler-elapsed-time)  
> > > > -M
> > > > 
> > > > On Tuesday, August 6, 2013, wrote:
> > > > 
> > > > > Guys, is there a mechanism (maybe a chef handler trick) so that I can  
> > > > > report  
> > > > > (in a file for ex) the report of a long running chef run?
> > > > > 
> > > > > Cheers  
> > > > > Lucian

---

<div class="post-metadata">

**Author:** ![kallistec](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/kallistec/32/23_2.png) [@kallistec](https://discourse.chef.io/u/kallistec)\
**Post date:** [August 8, 2013, 6:52pm UTC](https://discourse.chef.io/t/progress-of-a-chef-run/4392/8 "2013-08-08T18:52:32Z")

</div>

On Thursday, August 8, 2013 at 11:33 AM, steve . wrote:

> It looks like event handlers can now hook into one of three places:
> 
> - Start handlers hook in at the beginning of a run.
> - Report handlers hook in at the end of a successful run.
> - Exception handlers hook in after an exception is raised (i.e., after an _unsuccessful_ run).
> 
> It seems to me that you'd need your code to fire at the beginning of a run and monitor resources as they're converged. So that'd be a start handler. I don't know of any start handlers in the wild (or within my organization) ... so I'm not sure how well-explored this area is in the community. Or even if there's any documentation on using them.
> 
> I suggest an alternative: Write your own Chef output formatter that streams a progress update to a remote service.
> 
> Observe the behavior of the nyancat Chef formatter:  
> FWIW, Output formatters are built on a lower level instrumentation framework that we added in 10.14.0:

> <https://github.com/chef/chef/blob/main/lib/chef/event_dispatch/base.rb>

Configuration of custom event handlers is a little rough around the edges, this will improve as people start to use them and we understand the use cases better.

--  
Daniel DeLeo

---

<div class="post-metadata">

**Author:** ![steve](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/steve/32/294_2.png) [@steve](https://discourse.chef.io/u/steve)\
**Post date:** [August 8, 2013, 7:47pm UTC](https://discourse.chef.io/t/progress-of-a-chef-run/4392/9 "2013-08-08T19:47:51Z")

</div>

That's neat. I'm totally digging into that once I get through this  
infinity of new product launches. We already have an internal metrics  
dashboard that has a GUI mechanism for displaying Growl-like notifications  
and I think it's about to get progress bars for Chef runs! 😃

On Thu, Aug 8, 2013 at 11:52 AM, Daniel DeLeo [dan@kallistec.com](mailto:dan@kallistec.com) wrote:

> On Thursday, August 8, 2013 at 11:33 AM, steve . wrote:
> 
> It looks like event handlers can now hook into one of three places:
> 
> - Start handlers hook in at the beginning of a run.
> - Report handlers hook in at the end of a successful run.
> - Exception handlers hook in after an exception is raised (i.e., after an  
> _unsuccessful_ run).
> 
> It seems to me that you'd need your code to fire at the beginning of a run  
> and monitor resources as they're converged. So that'd be a start handler.  
> I don't know of any start handlers in the wild (or within my organization)  
> ... so I'm not sure how well-explored this area is in the community. Or  
> even if there's any documentation on using them.
> 
> I suggest an alternative: Write your own Chef output formatter that  
> streams a progress update to a remote service.
> 
> Observe the behavior of the nyancat Chef formatter:
> 
> FWIW, Output formatters are built on a lower level instrumentation  
> framework that we added in 10.14.0:
> 
> [https://github.com/opscode/chef/blob/master/lib/chef/event\_dispatch/base.rb](https://github.com/opscode/chef/blob/master/lib/chef/event_dispatch/base.rb)
> 
> Configuration of custom event handlers is a little rough around the edges,  
> this will improve as people start to use them and we understand the use  
> cases better.
> 
> --  
> Daniel DeLeo