# Force recipe to run at the end

**URL:** <https://discourse.chef.io/t/force-recipe-to-run-at-the-end/3121>\
**Category:** Chef Infra (archive)\
**Created:** [September 22, 2012, 9:16pm UTC](https://discourse.chef.io/t/force-recipe-to-run-at-the-end/3121 "2012-09-22T21:16:11Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![Julien\_Vehent](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/julien_vehent/32/381_2.png) [@Julien\_Vehent](https://discourse.chef.io/u/Julien_Vehent)\
**Post date:** [September 22, 2012, 9:16pm UTC](https://discourse.chef.io/t/force-recipe-to-run-at-the-end/3121/1 "2012-09-22T21:16:11Z")

</div>

Hi everyone,

Is there a way to force a recipe to run at the very end of the run list ?  
I have this firewall cookbook (AFW) that creates iptables rules into a  
template, but it runs in a base role and gets called before everything else.

I have other cookbooks that create rules, but those rules won’t appear until  
the next chef run when AFW runs again. I would like to converge in one run  
instead, by forcing AFW to run after everyone else.

Is it doable ?

Thanks,  
Julien

–  
Julien Vehent - [http://jve.linuxwall.info](http://jve.linuxwall.info)

---

<div class="post-metadata">

**Author:** ![Ranjib\_Dey](https://avatars.discourse-cdn.com/v4/letter/r/8e8cbc/32.png) [@Ranjib\_Dey](https://discourse.chef.io/u/Ranjib_Dey)\
**Post date:** [September 22, 2012, 9:35pm UTC](https://discourse.chef.io/t/force-recipe-to-run-at-the-end/3121/2 "2012-09-22T21:35:12Z")

</div>

Can you break you base role as base-start , base-end ? And then group the  
recipes the way you want?  
Though there are pretty established usage of using resources in the compile  
time, its not clean. Its difficult to test . I wish we had something like  
initializers (like what we have in Rails) , where you can specify a set of  
requirements in chef dsl, but they'll be executed in compile time.  
I've asked similar question, you can either use an orchestrator(mco) along  
with multiple chef runs. This way the chef recipes will be clean. Though  
convergence will be a multi step process . Else, you can always do the  
raw chef api calls in a recipe (and this will be executed in compile time  
unless you wrap them in ruby\_block resource).

You can add a dummy end recipe, to do the hacks for you. i.e re order the  
resource. Look at runner and run\_context class. You can also use a report  
handler to do some thing weird , invoking the iptables reload .

On Sun, Sep 23, 2012 at 2:46 AM, Julien Vehent [julien@linuxwall.info](mailto:julien@linuxwall.info)wrote:

> Hi everyone,
> 
> Is there a way to force a recipe to run at the very end of the run list ?  
> I have this firewall cookbook (AFW) that creates iptables rules into a  
> template, but it runs in a base role and gets called before everything else.
> 
> I have other cookbooks that create rules, but those rules won't appear  
> until the next chef run when AFW runs again. I would like to converge in  
> one run instead, by forcing AFW to run after everyone else.
> 
> Is it doable ?
> 
> Thanks,  
> Julien
> 
> --  
> Julien Vehent - [http://jve.linuxwall.info](http://jve.linuxwall.info)

---

<div class="post-metadata">

**Author:** ![Phil\_Dibowitz](https://avatars.discourse-cdn.com/v4/letter/p/a88e4f/32.png) [@Phil\_Dibowitz](https://discourse.chef.io/u/Phil_Dibowitz)\
**Post date:** [September 22, 2012, 9:49pm UTC](https://discourse.chef.io/t/force-recipe-to-run-at-the-end/3121/3 "2012-09-22T21:49:11Z")

</div>

On 09/22/2012 02:16 PM, Julien Vehent wrote:

> Hi everyone,
> 
> Is there a way to force a recipe to run at the very end of the run list ?  
> I have this firewall cookbook (AFW) that creates iptables rules into a  
> template, but it runs in a base role and gets called before everything else.

Do the other cookbooks add stuff at runtime, or compile time?

I'm assuming it's at runtime, they just need to add the rules to the node  
object, and then everything will work the way you want - the template is  
processed at runtime, and the node object has all rules add by all other things.

If, on the other hand, you have, say, ruby\_block's that are adding rules, then  
you can do some manual mucking to make this work, but I question if you really  
should be modifying your data at runtime, that's SUPER late in the game. But  
if you absolutely must make a ruby\_block which creates resources manually and  
appends them to the resource\_collection:

t = Chef::Resource::Template.new('/etc/thissucks', run\_context)  
t.cookbook "awf"  
t.source "suckage.erb"

# Don't remember if this is exactly right:

run\_context.resource\_collection \<\< t

Now that template will be at the very end of the resource collection, so even  
if you fuck with the rules in the node object at runtime, you'll be fine  
(provided no one makes dynamic resources after this one).

But seriously... that's really the wrong approach unless you have a VERY good  
reason. The better approach is that your recipes do stuff like:

# Careful, when node attributes are array-like instead of hash-like

# you need to initialize that somewhere.

node.default['afw']['rules'] \<\< 'iptables -A INPUT -m tcp -p 22 -j ACCEPT'

or whatever, and then your very very first template that runs can still have  
access to all the stuff all your other cookbooks added:

> I have other cookbooks that create rules, but those rules won't appear until  
> the next chef run when AFW runs again. I would like to converge in one run  
> instead, by forcing AFW to run after everyone else.

This sounds like you're trying to pass in your rules using "variables" or  
something...

> Is it doable ?

It's Chef, of course it's doable. 🙂

--  
Phil Dibowitz [phil@ipom.com](mailto:phil@ipom.com)  
Open Source software and tech docs Insanity Palace of Metallica  
[http://www.phildev.net/](http://www.phildev.net/)[http://www.ipom.com/](http://www.ipom.com/)

"Be who you are and say what you feel, because those who mind don't matter  
and those who matter don't mind."

- Dr. Seuss

---

<div class="post-metadata">

**Author:** ![Phil\_Dibowitz](https://avatars.discourse-cdn.com/v4/letter/p/a88e4f/32.png) [@Phil\_Dibowitz](https://discourse.chef.io/u/Phil_Dibowitz)\
**Post date:** [September 22, 2012, 9:56pm UTC](https://discourse.chef.io/t/force-recipe-to-run-at-the-end/3121/4 "2012-09-22T21:56:59Z")

</div>

On 09/22/2012 02:49 PM, Phil Dibowitz wrote:

> On 09/22/2012 02:16 PM, Julien Vehent wrote:
> 
> > Hi everyone,
> > 
> > Is there a way to force a recipe to run at the very end of the run list ?  
> > I have this firewall cookbook (AFW) that creates iptables rules into a  
> > template, but it runs in a base role and gets called before everything else.
> 
> Do the other cookbooks add stuff at runtime, or compile time?
> 
> I'm assuming it's at runtime, they just need to add the rules to the node

Er, that's supposed to say "assuming it's at compile time", sorry.

--  
Phil Dibowitz [phil@ipom.com](mailto:phil@ipom.com)  
Open Source software and tech docs Insanity Palace of Metallica  
[http://www.phildev.net/](http://www.phildev.net/)[http://www.ipom.com/](http://www.ipom.com/)

"Be who you are and say what you feel, because those who mind don't matter  
and those who matter don't mind."

- Dr. Seuss

---

<div class="post-metadata">

**Author:** ![Julien\_Vehent](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/julien_vehent/32/381_2.png) [@Julien\_Vehent](https://discourse.chef.io/u/Julien_Vehent)\
**Post date:** [September 23, 2012, 12:22am UTC](https://discourse.chef.io/t/force-recipe-to-run-at-the-end/3121/5 "2012-09-23T00:22:18Z")

</div>

Thanks a lot for the {quick,efficient} replies ! I understand better the  
whole compile/runtime separation now.

> > I have other cookbooks that create rules, but those rules won't appear  
> > until  
> > the next chef run when AFW runs again. I would like to converge in one  
> > run  
> > instead, by forcing AFW to run after everyone else.
> 
> This sounds like you're trying to pass in your rules using "variables" or  
> something...

Correct. I have a recipe for haproxy that does this:

```
 def foo()
   # add server to outbound firewall rules
   node['afw']['rules']\
       ["Haproxy local LB to #{server['hostname']}:#{port}"] = {
         'protocol' => 'tcp',
         'direction' => 'out',
         'user' => 'haproxy',
         'destination' => "#{server['hostname']}",
         'dport' => "#{port}"
       }
 end

```

If I add a provider to the AFW cookbook and call the provider in ruby in the  
haproxy cookbook, then the haproxy rule should be added at compile time.  
correct ?

def foo()  
afw\_add\_rule(name, protocol, direction, user, destination, ...)  
end

Thanks,  
Julien

---

<div class="post-metadata">

**Author:** ![Phil\_Dibowitz](https://avatars.discourse-cdn.com/v4/letter/p/a88e4f/32.png) [@Phil\_Dibowitz](https://discourse.chef.io/u/Phil_Dibowitz)\
**Post date:** [September 23, 2012, 3:08am UTC](https://discourse.chef.io/t/force-recipe-to-run-at-the-end/3121/6 "2012-09-23T03:08:48Z")

</div>

On 09/22/2012 05:22 PM, Julien Vehent wrote:

> ```
> def foo()
> # add server to outbound firewall rules
> node['afw']['rules']\
> ["Haproxy local LB to #{server['hostname']}:#{port}"] = {
> 'protocol' => 'tcp',
> 'direction' => 'out',
> 'user' => 'haproxy',
> 'destination' => "#{server['hostname']}",
> 'dport' => "#{port}"
> }
> end
> 
> ```

OK, so that's fine, you're adding to the node object, which you have access to  
from the template. Don't pass that data into the template via "variables",  
just access node directly in the template ERB and you have access to all the  
data added anywhere in any cookbook.

--  
Phil Dibowitz [phil@ipom.com](mailto:phil@ipom.com)  
Open Source software and tech docs Insanity Palace of Metallica  
[http://www.phildev.net/](http://www.phildev.net/)[http://www.ipom.com/](http://www.ipom.com/)

"Be who you are and say what you feel, because those who mind don't matter  
and those who matter don't mind."

- Dr. Seuss

---

<div class="post-metadata">

**Author:** ![Julien\_Vehent](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/julien_vehent/32/381_2.png) [@Julien\_Vehent](https://discourse.chef.io/u/Julien_Vehent)\
**Post date:** [September 26, 2012, 2:02am UTC](https://discourse.chef.io/t/force-recipe-to-run-at-the-end/3121/7 "2012-09-26T02:02:12Z")

</div>

On 2012-09-22 23:08, Phil Dibowitz wrote:

> On 09/22/2012 05:22 PM, Julien Vehent wrote:
> 
> > ```
> > def foo()
> > # add server to outbound firewall rules
> > node['afw']['rules']\
> > ["Haproxy local LB to #{server['hostname']}:#{port}"] = {
> > 'protocol' => 'tcp',
> > 'direction' => 'out',
> > 'user' => 'haproxy',
> > 'destination' => "#{server['hostname']}",
> > 'dport' => "#{port}"
> > }
> > end
> > 
> > ```
> 
> OK, so that's fine, you're adding to the node object, which you have  
> access  
> to  
> from the template. Don't pass that data into the template via "variables",  
> just access node directly in the template ERB and you have access to all  
> the  
> data added anywhere in any cookbook.

Thanks. I updated the cookbook: [http://community.opscode.com/cookbooks/afw](http://community.opscode.com/cookbooks/afw)  
One thing that I'm not 100% comfortable with (despite the great help on  
#Chef Infra (archive)) is how to export a single function from a given module to other  
cookbooks. In this case, I have AFW.create\_rule() that simply calls the  
internal function AFWCore.process\_rule(). I'm guessing this could be done  
better, but being primarily a sysadmin/security and not really a rubyist, I'm  
not sure how.

Advices and comments are most welcome. The github repos is here:

> **[GitHub - jvehent/AFW: Advanced FireWall cookbook for Chef and Linux that uses...](https://github.com/jvehent/AFW/)**
>
> Advanced FireWall cookbook for Chef and Linux that uses Iptables and to dynamically configure inbound and outbound rules on each node. - GitHub - jvehent/AFW: Advanced FireWall cookbook for Chef an...

Cheers,  
Julien

---

<div class="post-metadata">

**Author:** ![Phil\_Dibowitz](https://avatars.discourse-cdn.com/v4/letter/p/a88e4f/32.png) [@Phil\_Dibowitz](https://discourse.chef.io/u/Phil_Dibowitz)\
**Post date:** [September 26, 2012, 7:15am UTC](https://discourse.chef.io/t/force-recipe-to-run-at-the-end/3121/8 "2012-09-26T07:15:56Z")

</div>

On 09/25/2012 07:02 PM, Julien Vehent wrote:

> Thanks. I updated the cookbook: [http://community.opscode.com/cookbooks/afw](http://community.opscode.com/cookbooks/afw)  
> One thing that I'm not 100% comfortable with (despite the great help on #Chef Infra (archive))  
> is how to export a single function from a given module to other cookbooks. In  
> this case, I have AFW.create\_rule() that simply calls the internal function  
> AFWCore.process\_rule(). I'm guessing this could be done better, but being  
> primarily a sysadmin/security and not really a rubyist, I'm not sure how.

That's totally fine. I only did a quick scan of the code, but if I read it  
correctly, create\_rule() is just a wrapper for some sanity checking and then  
adding the rule to the node object, right?

That's completely reasonable. That all then gets added a compile time, and at  
runtime when your template is processed all the data is there.

Again, I only did a very quick glance at the code, but it seems good in general.

--  
Phil Dibowitz [phil@ipom.com](mailto:phil@ipom.com)  
Open Source software and tech docs Insanity Palace of Metallica  
[http://www.phildev.net/](http://www.phildev.net/)[http://www.ipom.com/](http://www.ipom.com/)

"Be who you are and say what you feel, because those who mind don't matter  
and those who matter don't mind."

- Dr. Seuss

---

<div class="post-metadata">

**Author:** ![Julien\_Vehent](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/julien_vehent/32/381_2.png) [@Julien\_Vehent](https://discourse.chef.io/u/Julien_Vehent)\
**Post date:** [October 2, 2012, 1:35am UTC](https://discourse.chef.io/t/force-recipe-to-run-at-the-end/3121/9 "2012-10-02T01:35:26Z")

</div>

On 2012-09-26 03:15, Phil Dibowitz wrote:

> On 09/25/2012 07:02 PM, Julien Vehent wrote:
> 
> > Thanks. I updated the cookbook:  
> > [http://community.opscode.com/cookbooks/afw](http://community.opscode.com/cookbooks/afw)  
> > One thing that I'm not 100% comfortable with (despite the great help on  
> > #Chef Infra (archive))  
> > is how to export a single function from a given module to other  
> > cookbooks. In  
> > this case, I have AFW.create\_rule() that simply calls the internal  
> > function  
> > AFWCore.process\_rule(). I'm guessing this could be done better, but being  
> > primarily a sysadmin/security and not really a rubyist, I'm not sure how.
> 
> That's totally fine. I only did a quick scan of the code, but if I read it  
> correctly, create\_rule() is just a wrapper for some sanity checking and  
> then  
> adding the rule to the node object, right?
> 
> That's completely reasonable. That all then gets added a compile time, and  
> at  
> runtime when your template is processed all the data is there.
> 
> Again, I only did a very quick glance at the code, but it seems good in  
> general.

I have a problem with the "extend" function. This code:

module AFW  
extend AFWCore  
\<....\>  
end

(see [https://github.com/jvehent/AFW/blob/master/libraries/create\_rule.rb](https://github.com/jvehent/AFW/blob/master/libraries/create_rule.rb) )

works perfectly fine in chef 10.12, but not in chef \<= 10.10.

Quoting yfeldblum from #Chef Infra (archive) "the problem is that there's no total order  
imposed as part of the API or part of the contract, and there's no supported  
API way for cookbooks and libraries themselves to request a partial order"

What would be the right way to do this, while making certain that it won't  
break earlier/future versions of chef ?

Thanks,  
Julien

---

<div class="post-metadata">

**Author:** ![AJ\_Christensen](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/aj_christensen/32/525_2.png) [@AJ\_Christensen](https://discourse.chef.io/u/AJ_Christensen)\
**Post date:** [October 2, 2012, 1:39am UTC](https://discourse.chef.io/t/force-recipe-to-run-at-the-end/3121/10 "2012-10-02T01:39:33Z")

</div>

For structured code like this (requires, includes, extends.. code that  
spreads multiple files..) I'd suggest moving it out to a Ruby Gem and  
using the chef\_gem installation method / compile time access.

As Jay mentioned, the libraries are shipped from the server and  
evaluated as they are received (IIRC) so you can have potential  
ordering issues there when requiring a file that doesn't exist, or  
relying on a file to be autoloaded, etc.

Here's an additional example:

[http://rubygems.org/gems/flock\_of\_chefs](http://rubygems.org/gems/flock_of_chefs)

> **[GitHub - chrisroberts/flock\_of\_chefs](https://github.com/chrisroberts/flock_of_chefs/)**
>
> Contribute to chrisroberts/flock\_of\_chefs development by creating an account on GitHub.

> <https://github.com/chrisroberts/cookbook-flock_of_chefs/blob/master/recipes/default.rb>

Cheers,

--AJ

On 2 October 2012 14:35, Julien Vehent [julien@linuxwall.info](mailto:julien@linuxwall.info) wrote:

> On 2012-09-26 03:15, Phil Dibowitz wrote:
> 
> > On 09/25/2012 07:02 PM, Julien Vehent wrote:
> > 
> > > Thanks. I updated the cookbook:  
> > > [http://community.opscode.com/cookbooks/afw](http://community.opscode.com/cookbooks/afw)  
> > > One thing that I'm not 100% comfortable with (despite the great help on  
> > > #Chef Infra (archive))  
> > > is how to export a single function from a given module to other  
> > > cookbooks. In  
> > > this case, I have AFW.create\_rule() that simply calls the internal  
> > > function  
> > > AFWCore.process\_rule(). I'm guessing this could be done better, but being  
> > > primarily a sysadmin/security and not really a rubyist, I'm not sure how.
> > 
> > That's totally fine. I only did a quick scan of the code, but if I read it  
> > correctly, create\_rule() is just a wrapper for some sanity checking and  
> > then  
> > adding the rule to the node object, right?
> > 
> > That's completely reasonable. That all then gets added a compile time, and  
> > at  
> > runtime when your template is processed all the data is there.
> > 
> > Again, I only did a very quick glance at the code, but it seems good in  
> > general.
> 
> I have a problem with the "extend" function. This code:
> 
> module AFW  
> extend AFWCore  
> \<....\>  
> end
> 
> (see [https://github.com/jvehent/AFW/blob/master/libraries/create\_rule.rb](https://github.com/jvehent/AFW/blob/master/libraries/create_rule.rb) )
> 
> works perfectly fine in chef 10.12, but not in chef \<= 10.10.
> 
> Quoting yfeldblum from #Chef Infra (archive) "the problem is that there's no total order  
> imposed as part of the API or part of the contract, and there's no supported  
> API way for cookbooks and libraries themselves to request a partial order"
> 
> What would be the right way to do this, while making certain that it won't  
> break earlier/future versions of chef ?
> 
> Thanks,  
> Julien

---

<div class="post-metadata">

**Author:** ![Julien\_Vehent](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/julien_vehent/32/381_2.png) [@Julien\_Vehent](https://discourse.chef.io/u/Julien_Vehent)\
**Post date:** [October 2, 2012, 3:07am UTC](https://discourse.chef.io/t/force-recipe-to-run-at-the-end/3121/11 "2012-10-02T03:07:40Z")

</div>

On 2012-10-01 21:39, AJ Christensen wrote:

> For structured code like this (requires, includes, extends.. code that  
> spreads multiple files..) I'd suggest moving it out to a Ruby Gem and  
> using the chef\_gem installation method / compile time access.
> 
> As Jay mentioned, the libraries are shipped from the server and  
> evaluated as they are received (IIRC) so you can have potential  
> ordering issues there when requiring a file that doesn't exist, or  
> relying on a file to be autoloaded, etc.
> 
> Here's an additional example:
> 
> [flock\_of\_chefs | RubyGems.org | your community gem host](http://rubygems.org/gems/flock_of_chefs)  
> [GitHub - chrisroberts/flock\_of\_chefs](https://github.com/chrisroberts/flock_of_chefs/)
> 
> [cookbook-flock\_of\_chefs/recipes/default.rb at master · chrisroberts/cookbook-flock\_of\_chefs · GitHub](https://github.com/chrisroberts/cookbook-flock_of_chefs/blob/master/recipes/default.rb)
> 
> Cheers,
> 
> --AJ

Thanks for the explanation, AJ.  
I decided to take a simpler approach and store the exported function in the  
same module as the internal function. It works fine, and does what I want.

> <https://github.com/jvehent/AFW/commit/2117d19d64c568985eae5a855c11e42167d2e41e>

Cheers,  
Julien

---

<div class="post-metadata">

**Author:** ![steve](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/steve/32/294_2.png) [@steve](https://discourse.chef.io/u/steve)\
**Post date:** [October 7, 2012, 8:46am UTC](https://discourse.chef.io/t/force-recipe-to-run-at-the-end/3121/12 "2012-10-07T08:46:57Z")

</div>

I had this crazy idea once for a node to drop itself from an external  
load balancer device (which shall remain nameless) and re-add it at  
the end of a successful Chef run. Ended up implementing it as an  
_event handler_ -- ran the "disable myself and let connections drop to  
zero" code at init (this was before the start handler hook, sorry) and  
ran the "add myself back to the pool" code as event handler code.

I think it would have worked, except I apparently accidentally  
overlooked the fact that the load balancer API endpoint wasn't  
accessible from the Chef node. DOH.

An approach like this might work for your situation, though -- you  
have full access to the node object in the event handler code and  
you're guaranteed to be running at the end of the run. As long as  
you're not doing something that could get clobbered by another event  
handler, you'd be fine.

This might sound gonzo, but to my eye it's simpler than distributing a  
gem. Might be easier to troubleshoot, too.

Good luck!

On Mon, Oct 1, 2012 at 8:07 PM, Julien Vehent [julien@linuxwall.info](mailto:julien@linuxwall.info) wrote:

> On 2012-10-01 21:39, AJ Christensen wrote:
> 
> > For structured code like this (requires, includes, extends.. code that  
> > spreads multiple files..) I'd suggest moving it out to a Ruby Gem and  
> > using the chef\_gem installation method / compile time access.
> > 
> > As Jay mentioned, the libraries are shipped from the server and  
> > evaluated as they are received (IIRC) so you can have potential  
> > ordering issues there when requiring a file that doesn't exist, or  
> > relying on a file to be autoloaded, etc.
> > 
> > Here's an additional example:
> > 
> > [flock\_of\_chefs | RubyGems.org | your community gem host](http://rubygems.org/gems/flock_of_chefs)  
> > [GitHub - chrisroberts/flock\_of\_chefs](https://github.com/chrisroberts/flock_of_chefs/)
> > 
> > [cookbook-flock\_of\_chefs/recipes/default.rb at master · chrisroberts/cookbook-flock\_of\_chefs · GitHub](https://github.com/chrisroberts/cookbook-flock_of_chefs/blob/master/recipes/default.rb)
> > 
> > Cheers,
> > 
> > --AJ
> 
> Thanks for the explanation, AJ.  
> I decided to take a simpler approach and store the exported function in the  
> same module as the internal function. It works fine, and does what I want.  
> [Move `create_rule()` to core library and rename it AFW · jvehent/AFW@2117d19 · GitHub](https://github.com/jvehent/AFW/commit/2117d19d64c568985eae5a855c11e42167d2e41e)
> 
> Cheers,  
> Julien