Dynamically building attributes (ie. via search) for cross-service, cross-node configuration

Howdy!

I’m trying to figure out the best way to leverage Chef’s attributes for
configuring a firewall. For the scenario in question, I’m using the
following cookbooks:

  • memcached <- unmodified upstream (except for CentOS support)
  • shorewall <- local, intend to contribute eventually
  • appserver-mysite
  • memcached-mysite

memcached-mysite includes both memcached and shorewall, and needs to tell
shorewall to add rules allowing incoming connections to the local memcached
instance from all systems having role[appserver-mysite]. Thus, I would like
to do something like the following in memcached-mysite’s
attributes/default.rb:

set[:shorewall][:rules] = [] # this will get merged, not overwrite, correct?
search(:node, ‘role[appserver-mysite]’).each do |matching_node|
local_addresses = []
matching_node[“network”][“interfaces”].each_pair do |ifname, ifdata|
interface_addresses = ifdata[“addresses”].find{ |k,v| k =~
/^192[.]168[.]/ }
if interface_addresses; then
local_addresses = local_addresses + interface_addresses
end
end
internal_ips =
matching_node[“network”][“interfaces”][“eth0”][“addresses”].find{ |k,v| k =~
/^192[.]168[.]/ } # FIXME: support other private ranges
if ! internal_ips; then break; end
internal_ips.each do |internal_ip|
set[:shorewall][:rules] << {
:description => “Allow app server #{node[“name”]} access to
memcached”,
:action => :accept,
:source => “lan:#{internal_ip}”,
:dest => :fw,
:proto => “tcp”,
:port => “11211”
}
end
end

…such that the shorewall recipe, in creating the rules file, can have its
template simply iterate over the full contents of node[:shorewall][:rules],
and use information appended by the memcached-mysite recipe (as well as
dbserver-mysite, and any other such cookbooks as may be applied). (By the
way – while this uses search, I’m not sure I like that – trusting
something as important as firewall configuration to a search engine which
could potentially be out-of-date makes me a little uncomfortable).

Unfortunately, it doesn’t appear to me that I can do this quite the way I
want. I understand that search is only available in recipes, not attributes,
and while I could update the attributes from within a recipe, this loses
ordering guarantees (such that if the same node is both a memcache server
and a database server – very likely on a development system – only the
first set of attributes might be in place at the time when the firewall
rules are applied).

How can I build up a description of my firewall rules using knowledge of the
other nodes’ configuration, and ensure that this information is available
when the templates are applied?

Thanks!

On 28 Dec 2010, at 16:25, Charles Duffy wrote:

Howdy!

I'm trying to figure out the best way to leverage Chef's attributes for configuring a firewall. For the scenario in question, I'm using the following cookbooks:

  • memcached <- unmodified upstream (except for CentOS support)
  • shorewall <- local, intend to contribute eventually
  • appserver-mysite
  • memcached-mysite

memcached-mysite includes both memcached and shorewall, and needs to tell shorewall to add rules allowing incoming connections to the local memcached instance from all systems having role[appserver-mysite]. Thus, I would like to do something like the following in memcached-mysite's attributes/default.rb:

set[:shorewall][:rules] = # this will get merged, not overwrite, correct?
search(:node, 'role[appserver-mysite]').each do |matching_node|
local_addresses =
matching_node["network"]["interfaces"].each_pair do |ifname, ifdata|
interface_addresses = ifdata["addresses"].find{ |k,v| k =~ /^192[.]168[.]/ }
if interface_addresses; then
local_addresses = local_addresses + interface_addresses
end
end
internal_ips = matching_node["network"]["interfaces"]["eth0"]["addresses"].find{ |k,v| k =~ /^192[.]168[.]/ } # FIXME: support other private ranges
if ! internal_ips; then break; end
internal_ips.each do |internal_ip|
set[:shorewall][:rules] << {
:description => "Allow app server #{node["name"]} access to memcached",
:action => :accept,
:source => "lan:#{internal_ip}",
:dest => :fw,
:proto => "tcp",
:port => "11211"
}
end
end

...such that the shorewall recipe, in creating the rules file, can have its template simply iterate over the full contents of node[:shorewall][:rules], and use information appended by the memcached-mysite recipe (as well as dbserver-mysite, and any other such cookbooks as may be applied). (By the way -- while this uses search, I'm not sure I like that -- trusting something as important as firewall configuration to a search engine which could potentially be out-of-date makes me a little uncomfortable).

Unfortunately, it doesn't appear to me that I can do this quite the way I want. I understand that search is only available in recipes, not attributes, and while I could update the attributes from within a recipe, this loses ordering guarantees (such that if the same node is both a memcache server and a database server -- very likely on a development system -- only the first set of attributes might be in place at the time when the firewall rules are applied).

How can I build up a description of my firewall rules using knowledge of the other nodes' configuration, and ensure that this information is available when the templates are applied?

Thanks!

I have a similar sort of thing for setting up DB users/pass from the app servers to my DB servers - I use data bags:

https://gist.github.com/757395

Does this give you some pointers?

Further: why do you need it in an attribute? Can you not just have the template block setup the variables as needed?

-ash

Ash, thanks for your response, and the sample code -- there's a lot there to
digest.

I have a few up-front goals influencing my design --

(1) - avoid needing any reconfiguration when adding new nodes beyond
bootstrapping and assigning roles
(2) - prevent the shorewall recipe from needing to know anything about
app-server semantics (knowledge about how mysite-memcached and
mysite-appserver interact should be in one of those two cookbooks, not in
the "shorewall" one, and should be flexible enough that arbitrary Ruby code
can be used for defining rules).

The former goal makes data bags the wrong tool for storing the final,
generated rules (as these rules need to change when new nodes are added, and
as I understand it, data bags are intended to store user-provided rather
than generated configuration). The latter goal means that the template block
in the shorewall module can't be running the application logic to generate
rules, though it can certainly run logic which collects or finds rules
which were set up by other cookbooks.

I certainly shouldn't have posed my question in such a way to present
attributes as an a priori solution, but given the constraints above and my
(presently) limited understanding of the tools chef provides, they look like
the best available fit to the problem space. That said -- if any of the
assumptions I reached this conclusion on is invalid, I'd be very glad to
hear it.

On Tue, Dec 28, 2010 at 10:37 AM, Ash Berlin ash_opscode@firemirror.comwrote:

On 28 Dec 2010, at 16:25, Charles Duffy wrote:

Howdy!

I'm trying to figure out the best way to leverage Chef's attributes for
configuring a firewall. For the scenario in question, I'm using the
following cookbooks:

  • memcached <- unmodified upstream (except for CentOS support)
  • shorewall <- local, intend to contribute eventually
  • appserver-mysite
  • memcached-mysite

memcached-mysite includes both memcached and shorewall, and needs to tell
shorewall to add rules allowing incoming connections to the local memcached
instance from all systems having role[appserver-mysite]. Thus, I would like
to do something like the following in memcached-mysite's
attributes/default.rb:

set[:shorewall][:rules] = # this will get merged, not overwrite,
correct?
search(:node, 'role[appserver-mysite]').each do |matching_node|
local_addresses =
matching_node["network"]["interfaces"].each_pair do |ifname, ifdata|
interface_addresses = ifdata["addresses"].find{ |k,v| k =~
/^192[.]168[.]/ }
if interface_addresses; then
local_addresses = local_addresses + interface_addresses
end
end
internal_ips =
matching_node["network"]["interfaces"]["eth0"]["addresses"].find{ |k,v| k =~
/^192[.]168[.]/ } # FIXME: support other private ranges
if ! internal_ips; then break; end
internal_ips.each do |internal_ip|
set[:shorewall][:rules] << {
:description => "Allow app server #{node["name"]} access to
memcached",
:action => :accept,
:source => "lan:#{internal_ip}",
:dest => :fw,
:proto => "tcp",
:port => "11211"
}
end
end

...such that the shorewall recipe, in creating the rules file, can have
its template simply iterate over the full contents of
node[:shorewall][:rules], and use information appended by the
memcached-mysite recipe (as well as dbserver-mysite, and any other such
cookbooks as may be applied). (By the way -- while this uses search, I'm not
sure I like that -- trusting something as important as firewall
configuration to a search engine which could potentially be out-of-date
makes me a little uncomfortable).

Unfortunately, it doesn't appear to me that I can do this quite the way I
want. I understand that search is only available in recipes, not attributes,
and while I could update the attributes from within a recipe, this loses
ordering guarantees (such that if the same node is both a memcache server
and a database server -- very likely on a development system -- only the
first set of attributes might be in place at the time when the firewall
rules are applied).

How can I build up a description of my firewall rules using knowledge of
the other nodes' configuration, and ensure that this information is
available when the templates are applied?

Thanks!

I have a similar sort of thing for setting up DB users/pass from the app
servers to my DB servers - I use data bags:

https://gist.github.com/757395

Does this give you some pointers?

Further: why do you need it in an attribute? Can you not just have the
template block setup the variables as needed?

-ash

Hello!

On Tue, Dec 28, 2010 at 9:25 AM, Charles Duffy charles@dyfis.net wrote:

How can I build up a description of my firewall rules using knowledge of the
other nodes' configuration, and ensure that this information is available
when the templates are applied?

We recommend using the attributes files in the cookbook simply for
setting default values.

default[:shorewall][:rules] =

So they don't cause nil errors when accessed elsewhere.

Then you can use the recipe to do a search, generate the rules as
hashes (per your code example, but assign to generated_rules), then
something like:

node.set[:shorewall][:rules] = generated_rules

Near the end.

--
Opscode, Inc
Joshua Timberman, Technical Evangelist
IRC, Skype, Twitter, Github: jtimberman

Thanks -- that's exactly what I was looking for.

The remaining piece is ensuring that the shorewall recipe is in the run_list
after anything/everything which might update the attributes it uses.

It would be nice if there a way to do that automatically ("include_later"?),
but worst-case I should be able to hack something together myself.

On Tue, Dec 28, 2010 at 12:52 PM, Joshua Timberman joshua@opscode.comwrote:

Hello!

On Tue, Dec 28, 2010 at 9:25 AM, Charles Duffy charles@dyfis.net wrote:

How can I build up a description of my firewall rules using knowledge of
the
other nodes' configuration, and ensure that this information is available
when the templates are applied?

We recommend using the attributes files in the cookbook simply for
setting default values.

default[:shorewall][:rules] =

So they don't cause nil errors when accessed elsewhere.

Then you can use the recipe to do a search, generate the rules as
hashes (per your code example, but assign to generated_rules), then
something like:

node.set[:shorewall][:rules] = generated_rules

Near the end.

--
Opscode, Inc
Joshua Timberman, Technical Evangelist
IRC, Skype, Twitter, Github: jtimberman