Defining a global variable to avoid sprinkles

Chef Infra (archive)
1

i’m finding myself sprinkling my cookbooks with repeated code, and wondering
how i can do this differently. i’m in ec2. let’s say i want to define
subdomains keyed off of 2 things: a userdata setting and
placement_availability_zone:

if userdata contains “-e dev” and av zone matches us-east-1, subdomain is dev.abfabreunion.com.
if userdata contains “-e prod” and av zone matches us-east-1, subdomain is prod.abfabreunion.com.
and so on.

i’d want to make that derived subdomain available to all of my cookbooks. is
there a way to do that in chef without sprinkling my code with the following?

mrepo/attributes/default.rb:

if node[:ec2][:userdata] =~ /-e dev/ && node[:ec2][:placement_availability_zone] =~ /us-east-1/
node.default[:mrepo][:smtpserver] = "mail1.dev.abfabreunion.com"
elsif node[:ec2][:userdata] =~ /-e prod/ && node[:ec2][:placement_availability_zone] =~ /us-east-1/
node.default[:mrepo][:smtpserver] = "mail1.prod.abfabreunion.com"
elsif node[:ec2][:userdata] =~ /-e prod/ && node[:ec2][:placement_availability_zone] =~ /us-west-2/
node.default[:mrepo][:smtpserver] = "mail1.prod2.abfabreunion.com"
end

openvpn/attributes/default.rb:

if node[:ec2][:userdata] =~ /-e dev/ && node[:ec2][:placement_availability_zone] =~ /us-east-1/
node.default[:openvpn][:dhcpdomain] = "dev.abfabreunion.com"
elsif node[:ec2][:userdata] =~ /-e prod/ && node[:ec2][:placement_availability_zone] =~ /us-east-1/
node.default[:openvpn][:dhcpdomain] = "prod.abfabreunion.com"
elsif node[:ec2][:userdata] =~ /-e prod/ && node[:ec2][:placement_availability_zone] =~ /us-west-2/
node.default[:openvpn][:dhcpdomain] = "prod2.abfabreunion.com"
end

and so on.

thanks,
kallen

2

Yo,

Couple of anti-patterns emerging in this post, quick attempt to enlighten:

On 10 January 2012 09:40, kallen@groknaut.net wrote:

i'm finding myself sprinkling my cookbooks with repeated code, and wondering
how i can do this differently. i'm in ec2. let's say i want to define
subdomains keyed off of 2 things: a userdata setting and
placement_availability_zone:

if userdata contains "-e dev" and av zone matches us-east-1, subdomain is dev.abfabreunion.com.
if userdata contains "-e prod" and av zone matches us-east-1, subdomain is prod.abfabreunion.com.
and so on.

i'd want to make that derived subdomain available to all of my cookbooks. is
there a way to do that in chef without sprinkling my code with the following?

In general the logical identifier of a node as a hostname is often set
up by sysadmins.

Are the hostnames you mention below registered automatically via API
or similar, or manually managed?

mrepo/attributes/default.rb:

if node[:ec2][:userdata] =~ /-e dev/ && node[:ec2][:placement_availability_zone] =~ /us-east-1/
node.default[:mrepo][:smtpserver] = "mail1.dev.abfabreunion.com"
elsif node[:ec2][:userdata] =~ /-e prod/ && node[:ec2][:placement_availability_zone] =~ /us-east-1/
node.default[:mrepo][:smtpserver] = "mail1.prod.abfabreunion.com"
elsif node[:ec2][:userdata] =~ /-e prod/ && node[:ec2][:placement_availability_zone] =~ /us-west-2/
node.default[:mrepo][:smtpserver] = "mail1.prod2.abfabreunion.com"
end

In this particular case, the obvious solution to me is to use Chef
Search to locate the smtp server for the environment. Consider
something like

just a node, so you can access the usual stuff

mrepo_smtpserver = search(:node, "role:whatever_smtp_role AND
ec2_placement_availability_zone:us-east-1 AND
chef_enviornment:#{node.chef_environment}")
mrepo_smtpserver[:ec2][:public_hostname]

You could easily store these attributes in a databag, instead of on
the node itself, too.. e.g.:

mrepo_smtpserver = data_bag_item "mrepo_smtpserver", node.chef_environment

Anyway I'm circling back to my original point: you should avoid hard
coding "target" attributes on client systems and instead make use of
one of many methods for dynamically registering or locating parts of
your infrastructure.

Hope this helps?

Cheers,

--AJ

openvpn/attributes/default.rb:

if node[:ec2][:userdata] =~ /-e dev/ && node[:ec2][:placement_availability_zone] =~ /us-east-1/
node.default[:openvpn][:dhcpdomain] = "dev.abfabreunion.com"
elsif node[:ec2][:userdata] =~ /-e prod/ && node[:ec2][:placement_availability_zone] =~ /us-east-1/
node.default[:openvpn][:dhcpdomain] = "prod.abfabreunion.com"
elsif node[:ec2][:userdata] =~ /-e prod/ && node[:ec2][:placement_availability_zone] =~ /us-west-2/
node.default[:openvpn][:dhcpdomain] = "prod2.abfabreunion.com"
end

and so on.

thanks,
kallen

3

On Tue, 10 Jan 2012, AJ Christensen wrote:

Yo,

Couple of anti-patterns emerging in this post, quick attempt to enlighten:

hi. yeah i was feeling kinda anti-patterny here... and realizing my example
here wrt setting a subdomain isn't a good example. anyway, good learning
opportunity. see inline (not much more to add, just responses)..

On 10 January 2012 09:40, kallen@groknaut.net wrote:

i'm finding myself sprinkling my cookbooks with repeated code, and wondering
how i can do this differently. i'm in ec2. let's say i want to define
subdomains keyed off of 2 things: a userdata setting and
placement_availability_zone:

if userdata contains "-e dev" and av zone matches us-east-1, subdomain is dev.abfabreunion.com.
if userdata contains "-e prod" and av zone matches us-east-1, subdomain is prod.abfabreunion.com.
and so on.

i'd want to make that derived subdomain available to all of my cookbooks. is
there a way to do that in chef without sprinkling my code with the following?

In general the logical identifier of a node as a hostname is often set
up by sysadmins.

Are the hostnames you mention below registered automatically via API
or similar, or manually managed?

at this time hostnames such as mail1.dev.abfabreunion.com are manually
managed. one goal here is to have easily derivable smtp relay hosts "per
environment", however the bounds of an environment are defined.

mrepo/attributes/default.rb:

if node[:ec2][:userdata] =~ /-e dev/ && node[:ec2][:placement_availability_zone] =~ /us-east-1/
node.default[:mrepo][:smtpserver] = "mail1.dev.abfabreunion.com"
elsif node[:ec2][:userdata] =~ /-e prod/ && node[:ec2][:placement_availability_zone] =~ /us-east-1/
node.default[:mrepo][:smtpserver] = "mail1.prod.abfabreunion.com"
elsif node[:ec2][:userdata] =~ /-e prod/ && node[:ec2][:placement_availability_zone] =~ /us-west-2/
node.default[:mrepo][:smtpserver] = "mail1.prod2.abfabreunion.com"
end

In this particular case, the obvious solution to me is to use Chef
Search to locate the smtp server for the environment.

ah yes, Chef Search. i'm just starting to scratch at that..

Consider something like

just a node, so you can access the usual stuff

mrepo_smtpserver = search(:node, "role:whatever_smtp_role AND
ec2_placement_availability_zone:us-east-1 AND
chef_enviornment:#{node.chef_environment}")
mrepo_smtpserver[:ec2][:public_hostname]

You could easily store these attributes in a databag, instead of on
the node itself, too.. e.g.:

mrepo_smtpserver = data_bag_item "mrepo_smtpserver", node.chef_environment

i will give both of these suggestions a try. it's still not clear to
me when to use attrs vs data bags.

Anyway I'm circling back to my original point: you should avoid hard
coding "target" attributes on client systems and instead make use of
one of many methods for dynamically registering or locating parts of
your infrastructure.

absolutely agreed. this is a part of chef i need to start getting
practice on.

Hope this helps?

it does. thanks!

Cheers,

--AJ

4

Hey hey,

On 10 January 2012 10:25, kallen@groknaut.net wrote:

On Tue, 10 Jan 2012, AJ Christensen wrote:

Yo,

Couple of anti-patterns emerging in this post, quick attempt to enlighten:

hi. yeah i was feeling kinda anti-patterny here... and realizing my example
here wrt setting a subdomain isn't a good example. anyway, good learning
opportunity. see inline (not much more to add, just responses)..

On 10 January 2012 09:40, kallen@groknaut.net wrote:

i'm finding myself sprinkling my cookbooks with repeated code, and wondering
how i can do this differently. i'm in ec2. let's say i want to define
subdomains keyed off of 2 things: a userdata setting and
placement_availability_zone:

if userdata contains "-e dev" and av zone matches us-east-1, subdomain is dev.abfabreunion.com.
if userdata contains "-e prod" and av zone matches us-east-1, subdomain is prod.abfabreunion.com.
and so on.

i'd want to make that derived subdomain available to all of my cookbooks. is
there a way to do that in chef without sprinkling my code with the following?

In general the logical identifier of a node as a hostname is often set
up by sysadmins.

Are the hostnames you mention below registered automatically via API
or similar, or manually managed?

at this time hostnames such as mail1.dev.abfabreunion.com are manually
managed. one goal here is to have easily derivable smtp relay hosts "per
environment", however the bounds of an environment are defined.

Something to consider here may be the dnsimple or dynect cookbooks,
check them out. Once you can logically identify these machines by
role/environment, you could easily have them register their own host
names.

https://github.com/heavywater/chef-dnsimple
https://github.com/opscode/cookbooks/tree/master/dynect

disclaimer: I am partnered with http://www.hw-ops.com/ who manage
operations for dnsimple, we use this cookbook for the majority of our
clients that require automated DNS

mrepo/attributes/default.rb:

if node[:ec2][:userdata] =~ /-e dev/ && node[:ec2][:placement_availability_zone] =~ /us-east-1/
node.default[:mrepo][:smtpserver] = "mail1.dev.abfabreunion.com"
elsif node[:ec2][:userdata] =~ /-e prod/ && node[:ec2][:placement_availability_zone] =~ /us-east-1/
node.default[:mrepo][:smtpserver] = "mail1.prod.abfabreunion.com"
elsif node[:ec2][:userdata] =~ /-e prod/ && node[:ec2][:placement_availability_zone] =~ /us-west-2/
node.default[:mrepo][:smtpserver] = "mail1.prod2.abfabreunion.com"
end

In this particular case, the obvious solution to me is to use Chef
Search to locate the smtp server for the environment.

ah yes, Chef Search. i'm just starting to scratch at that..

Consider something like

just a node, so you can access the usual stuff

mrepo_smtpserver = search(:node, "role:whatever_smtp_role AND
ec2_placement_availability_zone:us-east-1 AND
chef_enviornment:#{node.chef_environment}")
mrepo_smtpserver[:ec2][:public_hostname]

You could easily store these attributes in a databag, instead of on
the node itself, too.. e.g.:

mrepo_smtpserver = data_bag_item "mrepo_smtpserver", node.chef_environment

i will give both of these suggestions a try. it's still not clear to
me when to use attrs vs data bags.

I tend to use data bags when the information I'm storing is illogical
to store on a node - when it is not node-centric at all, more business
centric, or human-knowledge-required.. Credentials and stuff like
that. EBS device grouping/mappings.. SSL certificates.. that kind of
stuff.

Anyway I'm circling back to my original point: you should avoid hard
coding "target" attributes on client systems and instead make use of
one of many methods for dynamically registering or locating parts of
your infrastructure.

absolutely agreed. this is a part of chef i need to start getting
practice on.

There are some more advanced tools which may be worth researching down
the track, or alongside, in order to gain a better view of dynamic
infrastructure composition.

John Vincent's Noah springs to mind, and the zookeeper example
cookbooks that Netflix published may be handy too.

https://github.com/Netflix/curator/tree/master/curator-recipes/src/main/java/com/netflix/curator/framework/recipes

Hope this helps?

--AJ

5

On Tue, 10 Jan 2012, AJ Christensen wrote:

Hey hey,

hey, thanks for the additional info. after i succeed with this
exercise, i think i'll move on to using a data bag for this. but,

i'm having trouble with rendering the search results. can ya help?
i figure it's something silly i'm just not seeing.

let's say i'm looking for this rendered result:

smtp-foo = admin1.dev.abfabreunion.com

if i use this in my attributes:

mrepo_smtpserver = search("node", "role:#{node['postfix-server']} AND ec2_placement_availability_zone:us-east-1b AND chef_environment:#{node.chef_environment}")
default[:mrepo][:smtpserverfoo] = "#{mrepo_smtpserver}"

the template contains:
smtp-foo = <%= node[:mrepo][:smtpserverfoo] %>

the rendered config file is:

[admin]# grep smtp-foo /etc/mrepo.conf
smtp-foo = noderole: AND ec2_placement_availability_zone:us-east-1b AND chef_environment:dev

that's wrong..

i also used the syntax from your example, but the result in the rendered
template was the same wrongness:
mrepo_smtpserver = search(:node, "role:postfix-server AND ec2_placement_availability_zone:us-east-1 AND chef_enviornment:#{node.chef_environment}")

result:
smtp-foo = noderole:postfix-server AND ec2_placement_availability_zone:us-east-1 AND chef_enviornment:dev

illustrations and more questions:

[chef-repo]$ knife search node 'role:postfix-server' -i
4 items found
admin2.dev.abfabreunion.com
admin1.prod.abfabreunion.com
admin1.dev.abfabreunion.com
admin2.prod.abfabreunion.com

i'm not sure what the difference is between these 2 invocations, but for
fun:

[chef-repo]$ knife exec -E 'nodes.search("role:postfix-server") { |n| puts n.fqdn }'
admin2.dev.abfabreunion.com
admin1.dev.abfabreunion.com
admin1.prod.abfabreunion.com
admin2.prod.abfabreunion.com

[chef-repo]$ knife search node "ec2_placement_availability_zone:us-east-1b AND role:postfix-server"| grep ^Node
Node Name: admin1.dev.abfabreunion.com
Node Name: admin2.prod.abfabreunion.com

so, the data is available for me to grab via search..

as an aside, in the plethora of cookbook examples i'm combing thru, i
notice places where ".first" is appended. i wonder if that's like "head
-1" .. take the first one from the list. i wonder if i'll need that
since i have multiple nodes of role:postfix-server.

an additional question -- in the search syntax in the attributes, how
would i match on "us-east-1" for ec2_placement_availability_zone?
available are us-east-1a, 1b, 1c.. maybe i'll see that a wildcard match
will work as soon as i get some basic rendering to succeed.

ok yay. thanks!
kallen

6

search doesn't work in attributes. you don't need to store the result
of the search in an attribute. do it in a recipe and stuff it in a
local var

wildcards and stuff are available, it's standard lucene with some
additions; I'm sure they are covered on the wiki.

Cheers,

--AJ

On 10 January 2012 12:23, kallen@groknaut.net wrote:

On Tue, 10 Jan 2012, AJ Christensen wrote:

Hey hey,

hey, thanks for the additional info. after i succeed with this
exercise, i think i'll move on to using a data bag for this. but,

i'm having trouble with rendering the search results. can ya help?
i figure it's something silly i'm just not seeing.

let's say i'm looking for this rendered result:

smtp-foo = admin1.dev.abfabreunion.com

if i use this in my attributes:

mrepo_smtpserver = search("node", "role:#{node['postfix-server']} AND ec2_placement_availability_zone:us-east-1b AND chef_environment:#{node.chef_environment}")
default[:mrepo][:smtpserverfoo] = "#{mrepo_smtpserver}"

the template contains:
smtp-foo = <%= node[:mrepo][:smtpserverfoo] %>

the rendered config file is:

[admin]# grep smtp-foo /etc/mrepo.conf
smtp-foo = noderole: AND ec2_placement_availability_zone:us-east-1b AND chef_environment:dev

that's wrong..

i also used the syntax from your example, but the result in the rendered
template was the same wrongness:
mrepo_smtpserver = search(:node, "role:postfix-server AND ec2_placement_availability_zone:us-east-1 AND chef_enviornment:#{node.chef_environment}")

result:
smtp-foo = noderole:postfix-server AND ec2_placement_availability_zone:us-east-1 AND chef_enviornment:dev

illustrations and more questions:

[chef-repo]$ knife search node 'role:postfix-server' -i
4 items found
admin2.dev.abfabreunion.com
admin1.prod.abfabreunion.com
admin1.dev.abfabreunion.com
admin2.prod.abfabreunion.com

i'm not sure what the difference is between these 2 invocations, but for
fun:

[chef-repo]$ knife exec -E 'nodes.search("role:postfix-server") { |n| puts n.fqdn }'
admin2.dev.abfabreunion.com
admin1.dev.abfabreunion.com
admin1.prod.abfabreunion.com
admin2.prod.abfabreunion.com

[chef-repo]$ knife search node "ec2_placement_availability_zone:us-east-1b AND role:postfix-server"| grep ^Node
Node Name: admin1.dev.abfabreunion.com
Node Name: admin2.prod.abfabreunion.com

so, the data is available for me to grab via search..

as an aside, in the plethora of cookbook examples i'm combing thru, i
notice places where ".first" is appended. i wonder if that's like "head
-1" .. take the first one from the list. i wonder if i'll need that
since i have multiple nodes of role:postfix-server.

an additional question -- in the search syntax in the attributes, how
would i match on "us-east-1" for ec2_placement_availability_zone?
available are us-east-1a, 1b, 1c.. maybe i'll see that a wildcard match
will work as soon as i get some basic rendering to succeed.

ok yay. thanks!
kallen