Compile time order

Hi All,

I’m rather new Chef but already impressed with its power.
For my current Chef investigations I need some way to put recipe at the
end of recipes list ignoring the real order of it in node’s run_list.
I’m working on automated firewall configuration and now using attribute
files of recipes to declare what ports this recipe needs to be opened.
“Firewall” recipe then parses all ports at compile time and creates
initial data for a template. But if I declare settings not in attribute
file but in recipe itself (via node.default/node.set) - then this change
is invisible to “firewall” recipe because it is already compiled. My
thoughts were to have some callback at the end of compile time or a way
of putting dynamically some code at the end of run_list (without saving
it to node state) - so “firewall” recipe declared elsewhere could put
callback and read the final attributes state.

Maybe there is some workaround to do this, or I need to change logic
completely to fit to current Chef run model?

wbr,
Dmitry.

On Monday, November 7, 2011 at 7:43 AM, avenger@dmz.org.ua wrote:

Hi All,

I'm rather new Chef but already impressed with its power.
For my current Chef investigations I need some way to put recipe at the
end of recipes list ignoring the real order of it in node's run_list.
I'm working on automated firewall configuration and now using attribute
files of recipes to declare what ports this recipe needs to be opened.
"Firewall" recipe then parses all ports at compile time and creates
initial data for a template. But if I declare settings not in attribute
file but in recipe itself (via node.default/node.set) - then this change
is invisible to "firewall" recipe because it is already compiled. My
thoughts were to have some callback at the end of compile time or a way
of putting dynamically some code at the end of run_list (without saving
it to node state) - so "firewall" recipe declared elsewhere could put
callback and read the final attributes state.

Maybe there is some workaround to do this, or I need to change logic
completely to fit to current Chef run model?

wbr,
Dmitry.

This happens because node attributes get recomputed and produce a new object when you read them, so when you modify an attribute later it does not affect the previously computed value.

One way to handle this use case is to modify the variables passed in to a template:

template "/path/to/file" do
variables || variables(:tcp_ports => )
variables[:tcp_ports] << 80 << 443
end

elsewhere:

template "/path/to/file" do
variables || variables(:tcp_ports => )

variables[:tcp_ports] << 8000 << 8080
end

In case it's not clear, what's happening here is: when you declare a resource with the same type and "name argument" (i.e., argument passed in before do), chef will monkey patch the existing resource instead of creating a new one. What the variables lines are doing is setting the variables to a default setting ( {:tcp_ports => [] } if not already set, then adding the desired ports to the array.

Clearly there's more than a little duplication here, so you might want to wrap this up in a resource definition (http://wiki.opscode.com/display/chef/Definitions).

That said, it might be better to restructure your recipes and roles so that the required ports will be known ahead of time. But if that's not an option, you can make it work using the above.

--
Dan DeLeo

If you're using the firewall and ufw cookbooks provided by Opscode,
take a look at using the default ufw recipe. You can just use
attributes in roles and cookbook attribute files to define your
firewall rules and the recipe will merge and apply them. There are
more complicated setups supported by the ufw cookbook as well.
https://github.com/opscode/cookbooks/tree/master/ufw

Thanks,
Matt Ray
Senior Technical Evangelist | Opscode Inc.
matt@opscode.com | (512) 731-2218
Twitter, IRC, GitHub: mattray

On Mon, Nov 7, 2011 at 9:43 AM, avenger@dmz.org.ua wrote:

Hi All,

I'm rather new Chef but already impressed with its power.
For my current Chef investigations I need some way to put recipe at the end
of recipes list ignoring the real order of it in node's run_list.
I'm working on automated firewall configuration and now using attribute
files of recipes to declare what ports this recipe needs to be opened.
"Firewall" recipe then parses all ports at compile time and creates initial
data for a template. But if I declare settings not in attribute file but in
recipe itself (via node.default/node.set) - then this change is invisible to
"firewall" recipe because it is already compiled. My thoughts were to have
some callback at the end of compile time or a way of putting dynamically
some code at the end of run_list (without saving it to node state) - so
"firewall" recipe declared elsewhere could put callback and read the final
attributes state.

Maybe there is some workaround to do this, or I need to change logic
completely to fit to current Chef run model?

wbr,
Dmitry.

Thanks Matt,

I'm using Shorewall as my primary firewall management script. And I
already using attributes in environment/roles and cookbook attribute
files to define rules. But things starting to become more complicated,
so I need to put some attribute definitions inside "recipe" itself, and
I need a way to get them in "firewall" recipe. My current setup, nor ufw
recipe is not capable of doing this.

I think I will stick with "resource patching" (very cool feature!) and
Definitions for some time.

On Mon, 7 Nov 2011 11:09:19 -0600, Matt Ray wrote:

If you're using the firewall and ufw cookbooks provided by Opscode,
take a look at using the default ufw recipe. You can just use
attributes in roles and cookbook attribute files to define your
firewall rules and the recipe will merge and apply them. There are
more complicated setups supported by the ufw cookbook as well.
https://github.com/opscode/cookbooks/tree/master/ufw

Thanks,
Matt Ray
Senior Technical Evangelist | Opscode Inc.
matt@opscode.com | (512) 731-2218
Twitter, IRC, GitHub: mattray

On Mon, Nov 7, 2011 at 9:43 AM, avenger@dmz.org.ua wrote:

Hi All,

I'm rather new Chef but already impressed with its power.
For my current Chef investigations I need some way to put recipe at
the end
of recipes list ignoring the real order of it in node's run_list.
I'm working on automated firewall configuration and now using
attribute
files of recipes to declare what ports this recipe needs to be
opened.
"Firewall" recipe then parses all ports at compile time and creates
initial
data for a template. But if I declare settings not in attribute file
but in
recipe itself (via node.default/node.set) - then this change is
invisible to
"firewall" recipe because it is already compiled. My thoughts were
to have
some callback at the end of compile time or a way of putting
dynamically
some code at the end of run_list (without saving it to node state) -
so
"firewall" recipe declared elsewhere could put callback and read the
final
attributes state.

Maybe there is some workaround to do this, or I need to change logic
completely to fit to current Chef run model?

wbr,
Dmitry.

Thanks for the suggestion!
I will stick with "resource patching" and Definitions for a while -
this will do the job in price of dependency on "firewall" recipe.

But will also search for "plain attributes" solution - so if there is
no "firewall recipe" on node - firewall attributes will be just ignored.

On Mon, 7 Nov 2011 08:07:36 -0800, Daniel DeLeo wrote:

On Monday, November 7, 2011 at 7:43 AM, avenger@dmz.org.ua wrote:

Hi All,

I'm rather new Chef but already impressed with its power.
For my current Chef investigations I need some way to put recipe at
the
end of recipes list ignoring the real order of it in node's
run_list.
I'm working on automated firewall configuration and now using
attribute
files of recipes to declare what ports this recipe needs to be
opened.
"Firewall" recipe then parses all ports at compile time and creates
initial data for a template. But if I declare settings not in
attribute
file but in recipe itself (via node.default/node.set) - then this
change
is invisible to "firewall" recipe because it is already compiled. My
thoughts were to have some callback at the end of compile time or a
way
of putting dynamically some code at the end of run_list (without
saving
it to node state) - so "firewall" recipe declared elsewhere could
put
callback and read the final attributes state.

Maybe there is some workaround to do this, or I need to change logic
completely to fit to current Chef run model?

wbr,
Dmitry.

This happens because node attributes get recomputed and produce a new
object when you read them, so when you modify an attribute later it
does not affect the previously computed value.

One way to handle this use case is to modify the variables passed in
to a template:

template "/path/to/file" do
variables || variables(:tcp_ports => )
variables[:tcp_ports] << 80 << 443
end

elsewhere:

template "/path/to/file" do
variables || variables(:tcp_ports => )

variables[:tcp_ports] << 8000 << 8080
end

In case it's not clear, what's happening here is: when you declare a
resource with the same type and "name argument" (i.e., argument
passed
in before do), chef will monkey patch the existing resource instead
of creating a new one. What the variables lines are doing is setting
the variables to a default setting ( {:tcp_ports => [] } if not
already set, then adding the desired ports to the array.

Clearly there's more than a little duplication here, so you might
want to wrap this up in a resource definition
(http://wiki.opscode.com/display/chef/Definitions).

That said, it might be better to restructure your recipes and roles
so that the required ports will be known ahead of time. But if that's
not an option, you can make it work using the above.

The way the Firewall cookbook works allows for multiple providers
under the covers. It would be pretty slick for someone to write a
shorewall provider to go with it (ufw is already there as an example).

Just saying :slight_smile:

Thanks,
Matt Ray
Senior Technical Evangelist | Opscode Inc.
matt@opscode.com | (512) 731-2218
Twitter, IRC, GitHub: mattray

On Mon, Nov 7, 2011 at 11:21 AM, avenger@dmz.org.ua wrote:

Thanks Matt,

I'm using Shorewall as my primary firewall management script. And I already
using attributes in environment/roles and cookbook attribute files to define
rules. But things starting to become more complicated, so I need to put some
attribute definitions inside "recipe" itself, and I need a way to get them
in "firewall" recipe. My current setup, nor ufw recipe is not capable of
doing this.

I think I will stick with "resource patching" (very cool feature!) and
Definitions for some time.

On Mon, 7 Nov 2011 11:09:19 -0600, Matt Ray wrote:

If you're using the firewall and ufw cookbooks provided by Opscode,
take a look at using the default ufw recipe. You can just use
attributes in roles and cookbook attribute files to define your
firewall rules and the recipe will merge and apply them. There are
more complicated setups supported by the ufw cookbook as well.
https://github.com/opscode/cookbooks/tree/master/ufw

Thanks,
Matt Ray
Senior Technical Evangelist | Opscode Inc.
matt@opscode.com | (512) 731-2218
Twitter, IRC, GitHub: mattray

On Mon, Nov 7, 2011 at 9:43 AM, avenger@dmz.org.ua wrote:

Hi All,

I'm rather new Chef but already impressed with its power.
For my current Chef investigations I need some way to put recipe at the
end
of recipes list ignoring the real order of it in node's run_list.
I'm working on automated firewall configuration and now using attribute
files of recipes to declare what ports this recipe needs to be opened.
"Firewall" recipe then parses all ports at compile time and creates
initial
data for a template. But if I declare settings not in attribute file but
in
recipe itself (via node.default/node.set) - then this change is invisible
to
"firewall" recipe because it is already compiled. My thoughts were to
have
some callback at the end of compile time or a way of putting dynamically
some code at the end of run_list (without saving it to node state) - so
"firewall" recipe declared elsewhere could put callback and read the
final
attributes state.

Maybe there is some workaround to do this, or I need to change logic
completely to fit to current Chef run model?

wbr,
Dmitry.