Preferred Attribute locations


#1

At my day job, we’re having a discussion on the proper location for
attributes and it was suggested to ask the question on this list to see
what the community has to say.

The assertion initially made was to not use attributes at the Environment
level (since there’s one and only one Environment per node it is a limiting
place to put attributes). The questions that came up from this assertion
were as follows:

  • At what level should attributes be set?

  • At the file, role or node level?

  • Does it matter if we’re using default, normal/set or override for setting
    the attributes (other than for precedence’s sake)?

  • Also, does the answer change depending on what the particular attribute
    is?

  • Is there a logical set of attributes that should be defaulted at the role
    level and overridden at the file level?

  • Or is the usage wide open with the exception of using attributes at the
    environment level?

Looking at several cookbooks, it appears most cookbooks that have
attributes have “default” attributes (with exceptions being relatively old
ones). Also, role attributes tend to be “override” attributes. Is this
what can be considered “best practice”? Are there cases where this approach
is not the preferred approach?


Dylan


#2

One idea that has surfaced is the preferred cookbook:

I tend to place all default, “sane” attributes at the node level, then
default at the role level, and override at the environment level.

Some cookbooks use “set/normal” and you should be aware of what
implication that has on setting attributes at another level.

Reference: http://wiki.opscode.com/display/chef/Attributes#Attributes-Precedence

Consider:

cookbook/recipe defaults for every node that is going to have the
attribute. e.g. "default[‘logdir’] = “/var/log/”

role defaults/overrides for role-specific behavior, e.g.
default_attributes(‘logdir’ => “/var/log/myrole/”)

environment overrides for something specific to this particular
environment, e.g override_attributes('logdir =>
"/var/log/myrole/staging/")

Hope this helps,
-Michael

On Tue, Jul 17, 2012 at 9:47 AM, Dylan Northrup chef@doc-x.net wrote:

At my day job, we’re having a discussion on the proper location for
attributes and it was suggested to ask the question on this list to see what
the community has to say.

The assertion initially made was to not use attributes at the Environment
level (since there’s one and only one Environment per node it is a limiting
place to put attributes). The questions that came up from this assertion
were as follows:

  • At what level should attributes be set?

  • At the file, role or node level?

  • Does it matter if we’re using default, normal/set or override for setting
    the attributes (other than for precedence’s sake)?

  • Also, does the answer change depending on what the particular attribute
    is?

  • Is there a logical set of attributes that should be defaulted at the role
    level and overridden at the file level?

  • Or is the usage wide open with the exception of using attributes at the
    environment level?

Looking at several cookbooks, it appears most cookbooks that have attributes
have “default” attributes (with exceptions being relatively old ones).
Also, role attributes tend to be “override” attributes. Is this what can be
considered “best practice”? Are there cases where this approach is not the
preferred approach?


Dylan


#3

When we first structured our attributes, we found this table to be
invaluable:
http://wiki.opscode.com/display/chef/Attributes#Attributes-Precedence

And in fact it is explicitly referred to in several places in our source
code :wink:

The “not to use attributes at environment” assertion seems overly broad.
I’d say most of our attributes are defined at the environment level,
since our environments (prod, staging, qa) are virtually identical except
for the variants which are controlled by these environment attributes.

For example, our application has an archive database, but except when we’re
explicitly testing it, we only stand up an archiving database in our
production environment; that’s controlled via environment attributes.

In practice, we find we define attributes pretty much only at environment
and role. But YMMV.


Denis Haskin
cell: 781-258-7414

On Tue, Jul 17, 2012 at 9:47 AM, Dylan Northrup chef@doc-x.net wrote:

At my day job, we’re having a discussion on the proper location for
attributes and it was suggested to ask the question on this list to see
what the community has to say.

The assertion initially made was to not use attributes at the Environment
level (since there’s one and only one Environment per node it is a limiting
place to put attributes). The questions that came up from this assertion
were as follows:

  • At what level should attributes be set?

  • At the file, role or node level?

  • Does it matter if we’re using default, normal/set or override for
    setting the attributes (other than for precedence’s sake)?

  • Also, does the answer change depending on what the particular attribute
    is?

  • Is there a logical set of attributes that should be defaulted at the
    role level and overridden at the file level?

  • Or is the usage wide open with the exception of using attributes at the
    environment level?

Looking at several cookbooks, it appears most cookbooks that have
attributes have “default” attributes (with exceptions being relatively old
ones). Also, role attributes tend to be “override” attributes. Is this
what can be considered “best practice”? Are there cases where this approach
is not the preferred approach?


Dylan


#4

Are you opposed to databags? These are a nice alternative to setting
attributes and potentially have them overwritten by other levels.

On Tue, Jul 17, 2012 at 10:13 AM, Denis Haskin denis@constantorbit.comwrote:

When we first structured our attributes, we found this table to be
invaluable:
http://wiki.opscode.com/display/chef/Attributes#Attributes-Precedence

And in fact it is explicitly referred to in several places in our source
code :wink:

The “not to use attributes at environment” assertion seems overly broad.
I’d say most of our attributes are defined at the environment level,
since our environments (prod, staging, qa) are virtually identical except
for the variants which are controlled by these environment attributes.

For example, our application has an archive database, but except when
we’re explicitly testing it, we only stand up an archiving database in our
production environment; that’s controlled via environment attributes.

In practice, we find we define attributes pretty much only at environment
and role. But YMMV.


Denis Haskin
cell: 781-258-7414

On Tue, Jul 17, 2012 at 9:47 AM, Dylan Northrup chef@doc-x.net wrote:

At my day job, we’re having a discussion on the proper location for
attributes and it was suggested to ask the question on this list to see
what the community has to say.

The assertion initially made was to not use attributes at the Environment
level (since there’s one and only one Environment per node it is a limiting
place to put attributes). The questions that came up from this assertion
were as follows:

  • At what level should attributes be set?

  • At the file, role or node level?

  • Does it matter if we’re using default, normal/set or override for
    setting the attributes (other than for precedence’s sake)?

  • Also, does the answer change depending on what the particular attribute
    is?

  • Is there a logical set of attributes that should be defaulted at the
    role level and overridden at the file level?

  • Or is the usage wide open with the exception of using attributes at the
    environment level?

Looking at several cookbooks, it appears most cookbooks that have
attributes have “default” attributes (with exceptions being relatively old
ones). Also, role attributes tend to be “override” attributes. Is this
what can be considered “best practice”? Are there cases where this approach
is not the preferred approach?


Dylan


#5

And, as a followup (motivated by the two posts so far that have provided
pointers to the precedence rules), I’m aware of the precedence rules both
from the opscode site, but also from the quite good talk given at ChefConf
this year by Nathan Harvey. I’m primarily interested in how these
particular rules are applied in the real world.


Dylan


#6

i prefer to have default values of all the attributes in the cookbook,
the next level where we aggregate recipes within a role , we still prefer
default
while on the environment wide customization (like environment specific dns
server etc) we use override.

On Tue, Jul 17, 2012 at 7:37 PM, Mike miketheman@gmail.com wrote:

One idea that has surfaced is the preferred cookbook:
https://github.com/dcrosta/cookbook-preferred

I tend to place all default, “sane” attributes at the node level, then
default at the role level, and override at the environment level.

Some cookbooks use “set/normal” and you should be aware of what
implication that has on setting attributes at another level.

Reference:
http://wiki.opscode.com/display/chef/Attributes#Attributes-Precedence

Consider:

cookbook/recipe defaults for every node that is going to have the
attribute. e.g. "default[‘logdir’] = “/var/log/”

role defaults/overrides for role-specific behavior, e.g.
default_attributes(‘logdir’ => “/var/log/myrole/”)

environment overrides for something specific to this particular
environment, e.g override_attributes('logdir =>
"/var/log/myrole/staging/")

Hope this helps,
-Michael

On Tue, Jul 17, 2012 at 9:47 AM, Dylan Northrup chef@doc-x.net wrote:

At my day job, we’re having a discussion on the proper location for
attributes and it was suggested to ask the question on this list to see
what
the community has to say.

The assertion initially made was to not use attributes at the Environment
level (since there’s one and only one Environment per node it is a
limiting
place to put attributes). The questions that came up from this assertion
were as follows:

  • At what level should attributes be set?

  • At the file, role or node level?

  • Does it matter if we’re using default, normal/set or override for
    setting
    the attributes (other than for precedence’s sake)?

  • Also, does the answer change depending on what the particular attribute
    is?

  • Is there a logical set of attributes that should be defaulted at the
    role
    level and overridden at the file level?

  • Or is the usage wide open with the exception of using attributes at the
    environment level?

Looking at several cookbooks, it appears most cookbooks that have
attributes
have “default” attributes (with exceptions being relatively old ones).
Also, role attributes tend to be “override” attributes. Is this what
can be
considered “best practice”? Are there cases where this approach is not
the
preferred approach?


Dylan


#7

Hello,

This topic comes up quite often, and we spend a lot of time discussing this in our Chef Fundamentals class. There’s a number of reasons for the level of flexibility, most importantly because Chef doesn’t know what the right thing for your systems is, so it provides a lot of options.

The nearest we get to a “best practice” is to use the following guidelines:

  • Use “default” attributes in cookbooks’ attributes files to set “sane defaults” such that you do not get a “NilClass” error in your recipes or templates.
  • Use “default_attributes” in roles that are applied to the node(s).
  • Use “default” attributes in recipes for values that are calculated, or might combine other attributes.
  • Use “override_attributes” in environments for things that are truly environment specific (like, use this resolver for DNS in prod, but a different one in staging).

In general, override attributes should be used sparingly. Consider the kind of data that your modeling about the node in determining where the attribute should go.

These aren’t set in stone, and you’re welcome to modify your use of attribute setting based on how your infrastructure should be built. You’ll probably find yourself growing out of this fairly narrow set, but probably stick it to for most things.

Cheers!
Joshua

On Jul 17, 2012, at 7:47 AM, Dylan Northrup wrote:

At my day job, we’re having a discussion on the proper location for attributes and it was suggested to ask the question on this list to see what the community has to say.

The assertion initially made was to not use attributes at the Environment level (since there’s one and only one Environment per node it is a limiting place to put attributes). The questions that came up from this assertion were as follows:

  • At what level should attributes be set?

  • At the file, role or node level?

  • Does it matter if we’re using default, normal/set or override for setting the attributes (other than for precedence’s sake)?

  • Also, does the answer change depending on what the particular attribute is?

  • Is there a logical set of attributes that should be defaulted at the role level and overridden at the file level?

  • Or is the usage wide open with the exception of using attributes at the environment level?

Looking at several cookbooks, it appears most cookbooks that have attributes have “default” attributes (with exceptions being relatively old ones). Also, role attributes tend to be “override” attributes. Is this what can be considered “best practice”? Are there cases where this approach is not the preferred approach?


Dylan


#8

Hi,

We tend to do the following.

  • In the cookbook if an attribute is modifiable then it is set as a “default”
  • In the cookbook if an attribute is not modifiable then it is set as
    a “override”. It is usually used to “publish” information to other
    cookbooks. (i.e. The java cookbook may publish the path to the java
    command line tool)
  • In environments we only use override attributes all the time as our
    environment ~= cluster
  • We are moving from not having any attributes in roles (See below)
  • We rarely if ever use normal attributes and usually it is only for
    things like point of contact, business owner, or other cataloguing
    information.

Our role specific configuration we tend to put in a recipe in a
"policy" cookbook as we often use a combination of searches, data bag
lookup and rules to syntesize the attributes and then we set them as
override. i.e. Just say we have a role “metric_server” that sets up a
graphite server that we can send metric data to. We would create a
"fisg-graphite" cookbook that generates the environment specific
configuration and then includes the appropriate recipes from the
graphite cookbook.


Cheers,

Peter Donald


#9

I like your thinking around publishing and mutability. It seems Chef is growing up with the community’s recent focus on modularity, and attributes are part of the interface to a cookbook.

I only tend to use role attributes to integrate cookbooks. I.e. if Varnish and Nginx are to run on the same node, use a role attribute to integrate them.

Setting sane defaults in cookbooks helps with modularization, and my recent cookbooks have a Vagrantfile and some code to test interacting with a vm from the outside. The Vagrantfile should work with chef-solo, and shouldn’t require setting many attributes, if any.

We prefer to only store ephemeral or external data in data bags, such as credentials or the location of a service not managed by Chef. We also sometimes use it to override values that apply to an entire cluster.

Generally, I try to set attributes at the lowest possible preference because I don’t know what I’ll need to change later.

Jeremy

On Tuesday, July 17, 2012 at 4:44 PM, Peter Donald wrote:

Hi,

We tend to do the following.

  • In the cookbook if an attribute is modifiable then it is set as a “default”
  • In the cookbook if an attribute is not modifiable then it is set as
    a “override”. It is usually used to “publish” information to other
    cookbooks. (i.e. The java cookbook may publish the path to the java
    command line tool)
  • In environments we only use override attributes all the time as our
    environment ~= cluster
  • We are moving from not having any attributes in roles (See below)
  • We rarely if ever use normal attributes and usually it is only for
    things like point of contact, business owner, or other cataloguing
    information.

Our role specific configuration we tend to put in a recipe in a
"policy" cookbook as we often use a combination of searches, data bag
lookup and rules to syntesize the attributes and then we set them as
override. i.e. Just say we have a role “metric_server” that sets up a
graphite server that we can send metric data to. We would create a
"fisg-graphite" cookbook that generates the environment specific
configuration and then includes the appropriate recipes from the
graphite cookbook.


Cheers,

Peter Donald


#10

On Tue, Jul 17, 2012 at 2:36 PM, Joshua Timberman joshua@opscode.com wrote:

In general, override attributes should be used sparingly. Consider the kind of data that your modeling about the node in determining where the attribute should go.

Interesting. I think a year or so ago when this topic came up,
override attributes were all the rage :wink: Now that I think about this,
it makes sense. The info that Joshua just gave us seems like it
should replace the content here:
http://wiki.opscode.com/display/chef/Attributes#Attributes-UsageBestPractices