Chef 11 Attribute Changes -- Computed Attributes Edition

Ohai Chefs,

We recently merged some additional changes to attributes behavior for
Chef 11 that I’d like to make you aware of.

TLDR

  • Attributes from roles and environments are available when attribute
    files are evaluated.
  • Chef::Node now includes platform introspection methods, so you can use
    plaform?() and friends in attributes files.
  • Default and Override attributes from cookbooks, roles, and
    environments are stored separately during the Chef run.
  • As a consequence of the above, setting a default or override attribute
    in a recipe will no longer overwrite a value set by a role or
    environment.
  • CHEF-2936 http://tickets.opscode.com/browse/CHEF-2936

Accessing Role and Environment Attributes in Attributes Files

In Chef 10.x and previous, you cannot set attribute values that depend
on other attributes in attributes files, because you do not have access
to attributes from roles. For example, suppose you want to configure the
database for your app. You’d expect code like this to work (in an
attributes file):

    node.default[:app][:name] = "my_app"
    node.default[:app][:env] = "development"

    node.default[:app][:database] ="#{node.app.name}_#{node.app.env}"

However, this will not work if you intend to set node[:app][:name] or
node[:app][:env] via roles or environments, because Chef will not have
seen those attributes when evaluating the attributes files.

In Chef 11, role and environment attributes are applied when the
run_list is expanded, so the above code will work the way you want it
to.

Setting Platform-Specific Attributes in Attributes Files

In Chef 10.x and previous, the utility methods for making choices based
on a node’s platform or platform_family are available in recipes, but
not in attributes files. Some users have expressed a desire to be able
to set attributes based on platform in their attributes files. In Chef
11, these methods are now available on node objects, so you can use them
in attributes files. E.g.,:

if platform?(:debian, :ubuntu)
  default[:foo] = "debian value"
elsif platform?(:centos, :rhel)
  default[:foo] = "rhel value"
else
  default[:foo] = "generic value"
end

Cookbook Style

Together, these changes mean you can move attributes computations out of
recipes and into attributes files. I personally think this style will be
much better than what you must do currently, but we (Opscode) haven’t
made any official recommendation on this question yet. Do keep in mind
that any cookbook written using these features won’t be usable on Chef
10.x systems if you intend to make your cookbook public.

If you wish to write your cookbooks with computed attributes in
attributes files, don’t forget to use include_attribute() to enforce
the desired attribute file load order.

Breaking Changes

The implementation change that makes these features possible is a
potentially breaking change.

During a Chef run, default and override attributes from cookbooks,
roles, and environments are now stored separately. In the following
cases, you may get a different result when computing the value of an
attribute:

Attributes Files and Recipes Have the Same Precedence

In Chef 10.x and previous, setting a default attribute value in a recipe
would overwrite a value set by a role or environment. In Chef 11, values
set from a cookbook are stored separately, so the value set by the role
or environment will not be overwritten.

Consider a role like this:

default_attributes "app_name" => "from-role"

And a recipe file like this:

node.default["app_name"] = "from-recipe"

In Chef 10.x and lower, the value of node["app_name"] will be
"from-recipe". In Chef 11, the value will be “from-role”

Role and Environment Attributes are Visible to Attributes Files

If, for some reason, you have been depending on attributes from roles
and environments not being visible during attribute file evaluation, you
will need to adjust your cookbooks or roles for this change.

What to Look For

Unlike the other changes to attributes in Chef 11, this one is a
behavior change, meaning that valid usage can lead to different results
in Chef 11 compared to Chef 10. I expect the vast majority of users will
not be affected, since it’s unlikely you were depending on the old
behavior intentionally.

To see if you’re affected, you’ll need to take a look at your cookbooks
and roles and look for attribute conflicts.

One strategy is to inspect your recipes to see if you’re setting
defaults:

find . -name recipes -exec grep -n 'node.default' -r {} \;

…and compare with your roles to see if there’s a conflict.

Another option is to use why-run mode and inspect the output for any
unexpected configuration changes.


Daniel DeLeo

This is awesome, looking forward to it!

Hope that cookbook authors would adopt the new "attribute style" rather
than keeping the Chef 10.x "cookbook style" which is much less cleaner and
intentional imho.

Cheers, Torben
Am 06.11.2012 01:07 schrieb "Daniel DeLeo" dan@kallistec.com:

Ohai Chefs,

We recently merged some additional changes to attributes behavior for
Chef 11 that I'd like to make you aware of.

TLDR

  • Attributes from roles and environments are available when attribute
    files are evaluated.
  • Chef::Node now includes platform introspection methods, so you can use
    plaform?() and friends in attributes files.
  • Default and Override attributes from cookbooks, roles, and
    environments are stored separately during the Chef run.
  • As a consequence of the above, setting a default or override attribute
    in a recipe will no longer overwrite a value set by a role or
    environment.
  • CHEF-2936 http://tickets.opscode.com/browse/CHEF-2936

Accessing Role and Environment Attributes in Attributes Files

In Chef 10.x and previous, you cannot set attribute values that depend
on other attributes in attributes files, because you do not have access
to attributes from roles. For example, suppose you want to configure the
database for your app. You'd expect code like this to work (in an
attributes file):

    node.default[:app][:name] = "my_app"
    node.default[:app][:env] = "development"

    node.default[:app][:database] ="#{node.app.name}_#{node.app.env}"

However, this will not work if you intend to set node[:app][:name] or
node[:app][:env] via roles or environments, because Chef will not have
seen those attributes when evaluating the attributes files.

In Chef 11, role and environment attributes are applied when the
run_list is expanded, so the above code will work the way you want it
to.

Setting Platform-Specific Attributes in Attributes Files

In Chef 10.x and previous, the utility methods for making choices based
on a node's platform or platform_family are available in recipes, but
not in attributes files. Some users have expressed a desire to be able
to set attributes based on platform in their attributes files. In Chef
11, these methods are now available on node objects, so you can use them
in attributes files. E.g.,:

if platform?(:debian, :ubuntu)
  default[:foo] = "debian value"
elsif platform?(:centos, :rhel)
  default[:foo] = "rhel value"
else
  default[:foo] = "generic value"
end

Cookbook Style

Together, these changes mean you can move attributes computations out of
recipes and into attributes files. I personally think this style will be
much better than what you must do currently, but we (Opscode) haven't
made any official recommendation on this question yet. Do keep in mind
that any cookbook written using these features won't be usable on Chef
10.x systems if you intend to make your cookbook public.

If you wish to write your cookbooks with computed attributes in
attributes files, don't forget to use include_attribute() to enforce
the desired attribute file load order.

Breaking Changes

The implementation change that makes these features possible is a
potentially breaking change.

During a Chef run, default and override attributes from cookbooks,
roles, and environments are now stored separately. In the following
cases, you may get a different result when computing the value of an
attribute:

Attributes Files and Recipes Have the Same Precedence

In Chef 10.x and previous, setting a default attribute value in a recipe
would overwrite a value set by a role or environment. In Chef 11, values
set from a cookbook are stored separately, so the value set by the role
or environment will not be overwritten.

Consider a role like this:

default_attributes "app_name" => "from-role"

And a recipe file like this:

node.default["app_name"] = "from-recipe"

In Chef 10.x and lower, the value of node["app_name"] will be
"from-recipe". In Chef 11, the value will be "from-role"

Role and Environment Attributes are Visible to Attributes Files

If, for some reason, you have been depending on attributes from roles
and environments not being visible during attribute file evaluation, you
will need to adjust your cookbooks or roles for this change.

What to Look For

Unlike the other changes to attributes in Chef 11, this one is a
behavior change, meaning that valid usage can lead to different results
in Chef 11 compared to Chef 10. I expect the vast majority of users will
not be affected, since it's unlikely you were depending on the old
behavior intentionally.

To see if you're affected, you'll need to take a look at your cookbooks
and roles and look for attribute conflicts.

One strategy is to inspect your recipes to see if you're setting
defaults:

find . -name recipes -exec grep -n 'node.default' -r {} \;

...and compare with your roles to see if there's a conflict.

Another option is to use why-run mode and inspect the output for any
unexpected configuration changes.

--
Daniel DeLeo

That's really great news! I got bit by this more than once...

Regards, Matthias

Am 06.11.2012 um 01:07 schrieb Daniel DeLeo dan@kallistec.com:

Ohai Chefs,

We recently merged some additional changes to attributes behavior for
Chef 11 that I'd like to make you aware of.

TLDR

  • Attributes from roles and environments are available when attribute
    files are evaluated.
  • Chef::Node now includes platform introspection methods, so you can use
    plaform?() and friends in attributes files.
  • Default and Override attributes from cookbooks, roles, and
    environments are stored separately during the Chef run.
  • As a consequence of the above, setting a default or override attribute
    in a recipe will no longer overwrite a value set by a role or
    environment.
  • CHEF-2936 http://tickets.opscode.com/browse/CHEF-2936

Accessing Role and Environment Attributes in Attributes Files

In Chef 10.x and previous, you cannot set attribute values that depend
on other attributes in attributes files, because you do not have access
to attributes from roles. For example, suppose you want to configure the
database for your app. You'd expect code like this to work (in an
attributes file):

    node.default[:app][:name] = "my_app"
    node.default[:app][:env] = "development"

    node.default[:app][:database] ="#{node.app.name}_#{node.app.env}"

However, this will not work if you intend to set node[:app][:name] or
node[:app][:env] via roles or environments, because Chef will not have
seen those attributes when evaluating the attributes files.

In Chef 11, role and environment attributes are applied when the
run_list is expanded, so the above code will work the way you want it
to.

Setting Platform-Specific Attributes in Attributes Files

In Chef 10.x and previous, the utility methods for making choices based
on a node's platform or platform_family are available in recipes, but
not in attributes files. Some users have expressed a desire to be able
to set attributes based on platform in their attributes files. In Chef
11, these methods are now available on node objects, so you can use them
in attributes files. E.g.,:

if platform?(:debian, :ubuntu)
  default[:foo] = "debian value"
elsif platform?(:centos, :rhel)
  default[:foo] = "rhel value"
else
  default[:foo] = "generic value"
end

Cookbook Style

Together, these changes mean you can move attributes computations out of
recipes and into attributes files. I personally think this style will be
much better than what you must do currently, but we (Opscode) haven't
made any official recommendation on this question yet. Do keep in mind
that any cookbook written using these features won't be usable on Chef
10.x systems if you intend to make your cookbook public.

If you wish to write your cookbooks with computed attributes in
attributes files, don't forget to use include_attribute() to enforce
the desired attribute file load order.

Breaking Changes

The implementation change that makes these features possible is a
potentially breaking change.

During a Chef run, default and override attributes from cookbooks,
roles, and environments are now stored separately. In the following
cases, you may get a different result when computing the value of an
attribute:

Attributes Files and Recipes Have the Same Precedence

In Chef 10.x and previous, setting a default attribute value in a recipe
would overwrite a value set by a role or environment. In Chef 11, values
set from a cookbook are stored separately, so the value set by the role
or environment will not be overwritten.

Consider a role like this:

default_attributes "app_name" => "from-role"

And a recipe file like this:

node.default["app_name"] = "from-recipe"

In Chef 10.x and lower, the value of node["app_name"] will be
"from-recipe". In Chef 11, the value will be "from-role"

Role and Environment Attributes are Visible to Attributes Files

If, for some reason, you have been depending on attributes from roles
and environments not being visible during attribute file evaluation, you
will need to adjust your cookbooks or roles for this change.

What to Look For

Unlike the other changes to attributes in Chef 11, this one is a
behavior change, meaning that valid usage can lead to different results
in Chef 11 compared to Chef 10. I expect the vast majority of users will
not be affected, since it's unlikely you were depending on the old
behavior intentionally.

To see if you're affected, you'll need to take a look at your cookbooks
and roles and look for attribute conflicts.

One strategy is to inspect your recipes to see if you're setting
defaults:

find . -name recipes -exec grep -n 'node.default' -r {} \;

...and compare with your roles to see if there's a conflict.

Another option is to use why-run mode and inspect the output for any
unexpected configuration changes.

--
Daniel DeLeo

On Mon, Nov 05, 2012 at 04:07:29PM -0800, Daniel DeLeo wrote:

Attributes Files and Recipes Have the Same Precedence

In Chef 10.x and previous, setting a default attribute value in a recipe
would overwrite a value set by a role or environment. In Chef 11, values
set from a cookbook are stored separately, so the value set by the role
or environment will not be overwritten.

Consider a role like this:

default_attributes "app_name" => "from-role"

And a recipe file like this:

node.default["app_name"] = "from-recipe"

In Chef 10.x and lower, the value of node["app_name"] will be
"from-recipe". In Chef 11, the value will be "from-role"

Wait, this is really bad. It's one thing to store them separately, but you're

  1. changing predecence and 2. giving no way to override roles.

Since roles are non-static, that's REALLY bad. roles are upload-time static
data. The thing that can do dynmaic logic later on should absolutely have a
higher precedence. At read-time this should collapse the way it does now, and
I should get the recipe value.

--
Phil Dibowitz phil@ipom.com
Open Source software and tech docs Insanity Palace of Metallica
http://www.phildev.net/ http://www.ipom.com/

"Be who you are and say what you feel, because those who mind don't matter
and those who matter don't mind."

  • Dr. Seuss

On Tuesday, November 6, 2012 at 6:07 PM, Phil Dibowitz wrote:

On Mon, Nov 05, 2012 at 04:07:29PM -0800, Daniel DeLeo wrote:

Attributes Files and Recipes Have the Same Precedence

In Chef 10.x and previous, setting a default attribute value in a recipe
would overwrite a value set by a role or environment. In Chef 11, values
set from a cookbook are stored separately, so the value set by the role
or environment will not be overwritten.

Consider a role like this:

default_attributes "app_name" => "from-role"

And a recipe file like this:

node.default["app_name"] = "from-recipe"

In Chef 10.x and lower, the value of node["app_name"] will be
"from-recipe". In Chef 11, the value will be "from-role"

Wait, this is really bad. It's one thing to store them separately, but you're

  1. changing predecence and 2. giving no way to override roles.

Since roles are non-static, that's REALLY bad. roles are upload-time static
data. The thing that can do dynmaic logic later on should absolutely have a
higher precedence. At read-time this should collapse the way it does now, and
I should get the recipe value.

In what cases do you use this behavior?

And, in the case of defaults, you can go to normal or override attributes if you must. Do you find yourself frequently overriding role overrides in recipes?

--
Phil Dibowitz phil@ipom.com (mailto:phil@ipom.com)
Open Source software and tech docs Insanity Palace of Metallica
http://www.phildev.net/ http://www.ipom.com/

"Be who you are and say what you feel, because those who mind don't matter
and those who matter don't mind."

  • Dr. Seuss

--
Daniel DeLeo

On Tue, Nov 06, 2012 at 06:12:43PM -0800, Daniel DeLeo wrote:

In what cases do you use this behavior?

And, in the case of defaults, you can go to normal or override attributes if you must. Do you find yourself frequently overriding role overrides in recipes?

All over the place... roles request stuff we want, and in certain cases we
determine based on the runtime environment that we can't do that.

But I shouldn't have to use override. Obviously I can, but it defeats the
whole purpose of the predecence hierarchy within a given precedence level.

Plus, as Adam would say, you have no runway. We went from a 14-level runway,
to a 2-level runway. Someone goes "override" in a role and you're fucked.

I see no reason why recipes shouldn't win - we can accomplish everything
desired and still have the Attribute collapse such that recipes win.

Runtime data should always win over static data.

--
Phil Dibowitz phil@ipom.com
Open Source software and tech docs Insanity Palace of Metallica
http://www.phildev.net/ http://www.ipom.com/

"Be who you are and say what you feel, because those who mind don't matter
and those who matter don't mind."

  • Dr. Seuss

I'm curious about this... Phil, can you give a more concrete example?

On Tue, Nov 6, 2012 at 9:19 PM, Phil Dibowitz phil@ipom.com wrote:

On Tue, Nov 06, 2012 at 06:12:43PM -0800, Daniel DeLeo wrote:

In what cases do you use this behavior?

And, in the case of defaults, you can go to normal or override
attributes if you must. Do you find yourself frequently overriding role
overrides in recipes?

All over the place... roles request stuff we want, and in certain cases we
determine based on the runtime environment that we can't do that.

But I shouldn't have to use override. Obviously I can, but it defeats the
whole purpose of the predecence hierarchy within a given precedence level.

Plus, as Adam would say, you have no runway. We went from a 14-level
runway,
to a 2-level runway. Someone goes "override" in a role and you're fucked.

I see no reason why recipes shouldn't win - we can accomplish everything
desired and still have the Attribute collapse such that recipes win.

Runtime data should always win over static data.

--
Phil Dibowitz phil@ipom.com
Open Source software and tech docs Insanity Palace of Metallica
http://www.phildev.net/ http://www.ipom.com/

"Be who you are and say what you feel, because those who mind don't matter
and those who matter don't mind."

  • Dr. Seuss

On Wed, Nov 07, 2012 at 12:46:40PM -0500, Jesse Campbell wrote:

I'm curious about this... Phil, can you give a more concrete example?

The classic example is:

default_attributes 'sysctl' => {'foo' => 'bar'}

and in a recipe

if memory < X

ensure node['sysctl']['foo'] isn't greater than Y

end

Here's a slightly contrived example...

Role:

default_attributes 'want_service_x_private_key' => true

Recipe:

unless query_external_source('can_have_service_x_private_key')
node.default['want_service_x_private_key'] = false
end

Doesn't mean I dont' want to be be able to override later. Yes, I can force
that to normal/override but those have specific meaning: I'm OVERRIDING the
standard logic. In this case I"m not, I'm just using logic to determine a
better state of the world. This is the standard logic. It's smarter than the
static world.

In both of these examples I can go into override, but that's non-intuitive and
we work very hard to make sure people can just update the node attribute in
cookbooks as they go along and the right things happen. This is a HORRIBLE
breakage.

--
Phil Dibowitz phil@ipom.com
Open Source software and tech docs Insanity Palace of Metallica
http://www.phildev.net/ http://www.ipom.com/

"Be who you are and say what you feel, because those who mind don't matter
and those who matter don't mind."

  • Dr. Seuss

On Wednesday, November 7, 2012 at 9:46 AM, Jesse Campbell wrote:

I'm curious about this... Phil, can you give a more concrete example?

On Tue, Nov 6, 2012 at 9:19 PM, Phil Dibowitz <phil@ipom.com (mailto:phil@ipom.com)> wrote:

On Tue, Nov 06, 2012 at 06:12:43PM -0800, Daniel DeLeo wrote:

In what cases do you use this behavior?

And, in the case of defaults, you can go to normal or override attributes if you must. Do you find yourself frequently overriding role overrides in recipes?

All over the place... roles request stuff we want, and in certain cases we
determine based on the runtime environment that we can't do that.

But I shouldn't have to use override. Obviously I can, but it defeats the
whole purpose of the predecence hierarchy within a given precedence level.

Plus, as Adam would say, you have no runway. We went from a 14-level runway,
to a 2-level runway. Someone goes "override" in a role and you're fucked.
To be fair, we went from 11 to 9 de facto attribute precedence levels, or 7 if you exclude environments. But it's also the case that normal attributes can be less useful, since they persist to the node and you have to manually wipe them if you don't want them anymore, which leaves you with 6 "useful" levels of attributes.

I see no reason why recipes shouldn't win - we can accomplish everything
desired and still have the Attribute collapse such that recipes win.
Mostly it comes down to the fact that attributes files are implemented via instance_eval on the node object, rather than having a defined DSL. I'd initially written this change so that attributes files would be evaluated in the context of a special DSL object, but we decided that this change needed more design and research so we had to drop it in the interest of shipping Chef 11.

The only other obvious way to accomplish what you're asking for is to put the node object into "attribute-file-mode" or "recipe-mode", but this carries the risk that you get stuck in the wrong state. Given all the ways people use Chef's libraries to automate mangling of node data, it seems pretty likely that this would occur.

Another alternative is to add to the existing api, so you could do node.recipe_default[:foo] = "bar"

Runtime data should always win over static data.
There are quite a few ways to accomplish what you want given the code as it is now.

If what you really wanted in the first place was something like a runtime-evaluated role, you could implement that as a cookbook with all the logic you want in the attributes file.

Alternatively, you can use local variables instead of node attributes as the final input to your recipes. Your logic would be the same, and in the same place, you just assign the final value to a local instead.

If you really want the value to be set as a node attributes, you can simply go around the precedence system by accessing node.attributes.role_override.

Of these, dynamic-role-as-a-cookbook feels the most correct, but of course may not be appropriate for you.

--
Phil Dibowitz phil@ipom.com (mailto:phil@ipom.com)
Open Source software and tech docs Insanity Palace of Metallica
http://www.phildev.net/ http://www.ipom.com/

"Be who you are and say what you feel, because those who mind don't matter
and those who matter don't mind."

  • Dr. Seuss

--
Daniel DeLeo

On Wed, Nov 07, 2012 at 11:48:44AM -0800, Daniel DeLeo wrote:

Mostly it comes down to the fact that attributes files are implemented via instance_eval on the node object, rather than having a defined DSL. I'd initially written this change so that attributes files would be evaluated in the context of a special DSL object, but we decided that this change needed more design and research so we had to drop it in the interest of shipping Chef 11.

The only other obvious way to accomplish what you're asking for is to put the node object into "attribute-file-mode" or "recipe-mode", but this carries the risk that you get stuck in the wrong state. Given all the ways people use Chef's libraries to automate mangling of node data, it seems pretty likely that this would occur.

But you keep track of them separately, right? So at read time, why can't you
just pick the same one you used to pick?

If what you really wanted in the first place was something like a
runtime-evaluated role, you could implement that as a cookbook with all the
logic you want in the attributes file.

No, because those aren't evaulated in run_list order. Our attributes files
setup all basic data-structures. Then our recipes all add/change/munge data in
a sane way, based on runlist order: basic company-wide cookbooks, then
cookbooks that apply to large swaths of servers, then cookbooks that are more
specific. Each one can change the data.

We rely heavily on templates, so I all the early cookbooks setup those
templates, everyone has plenty of time to add to the data in recipes or roles
as necessary, and then the template will get the most appropriate data.

Alternatively, you can use local variables instead of node attributes as the
final input to your recipes. Your logic would be the same, and in the same
place, you just assign the final value to a local instead.

No, because that's all compile-time. I need the templates to get the data
provided by cookbooks that run after them.

If you really want the value to be set as a node attributes, you can simply
go around the precedence system by accessing
node.attributes.role_override.

I could also do node.override, but that's silly. The precedence system was
logical before. Having data that is evaluated at upload time override data
computed at runtime is counter-intuitive and limiting and defeats the
original beauty of the predence system.

--
Phil Dibowitz phil@ipom.com
Open Source software and tech docs Insanity Palace of Metallica
http://www.phildev.net/ http://www.ipom.com/

"Be who you are and say what you feel, because those who mind don't matter
and those who matter don't mind."

  • Dr. Seuss

You still have a higher precedence option here - it's just precedence,
instead of merge order.

Adam

On 11/6/12 6:07 PM, "Phil Dibowitz" phil@ipom.com wrote:

On Mon, Nov 05, 2012 at 04:07:29PM -0800, Daniel DeLeo wrote:

Attributes Files and Recipes Have the Same Precedence

In Chef 10.x and previous, setting a default attribute value in a recipe
would overwrite a value set by a role or environment. In Chef 11, values
set from a cookbook are stored separately, so the value set by the role
or environment will not be overwritten.

Consider a role like this:

default_attributes "app_name" => "from-role"

And a recipe file like this:

node.default["app_name"] = "from-recipe"

In Chef 10.x and lower, the value of node["app_name"] will be
"from-recipe". In Chef 11, the value will be "from-role"

Wait, this is really bad. It's one thing to store them separately, but
you're

  1. changing predecence and 2. giving no way to override roles.

Since roles are non-static, that's REALLY bad. roles are upload-time
static
data. The thing that can do dynmaic logic later on should absolutely have
a
higher precedence. At read-time this should collapse the way it does now,
and
I should get the recipe value.

--
Phil Dibowitz phil@ipom.com
Open Source software and tech docs Insanity Palace of Metallica
http://www.phildev.net/ http://www.ipom.com/

"Be who you are and say what you feel, because those who mind don't matter
and those who matter don't mind."

  • Dr. Seuss

On Wed, Nov 07, 2012 at 10:50:24PM +0000, Adam Jacob wrote:

You still have a higher precedence option here - it's just precedence,
instead of merge order.

Yes, I'm aware, but that cuts down your "runway" as you would put it.

The merge order was intuitive and obvious and meant that everyone could use
the same precedence unless they had to override the accepted state of the
sytem. Now this is no longer the case.

--
Phil Dibowitz phil@ipom.com
Open Source software and tech docs Insanity Palace of Metallica
http://www.phildev.net/ http://www.ipom.com/

"Be who you are and say what you feel, because those who mind don't matter
and those who matter don't mind."

  • Dr. Seuss

On Wednesday, November 7, 2012 at 12:03 PM, Phil Dibowitz wrote:

On Wed, Nov 07, 2012 at 11:48:44AM -0800, Daniel DeLeo wrote:

Mostly it comes down to the fact that attributes files are implemented via instance_eval on the node object, rather than having a defined DSL. I'd initially written this change so that attributes files would be evaluated in the context of a special DSL object, but we decided that this change needed more design and research so we had to drop it in the interest of shipping Chef 11.

The only other obvious way to accomplish what you're asking for is to put the node object into "attribute-file-mode" or "recipe-mode", but this carries the risk that you get stuck in the wrong state. Given all the ways people use Chef's libraries to automate mangling of node data, it seems pretty likely that this would occur.

But you keep track of them separately, right? So at read time, why can't you
just pick the same one you used to pick?

In Chef 8-10, everything is based on merge order. There is a "combined_default" method on Chef::Node::Attribute, but it's a computed value, based on a merge of cookbook, role, and environment defaults.

If what you really wanted in the first place was something like a
runtime-evaluated role, you could implement that as a cookbook with all the
logic you want in the attributes file.

No, because those aren't evaulated in run_list order. Our attributes files
setup all basic data-structures. Then our recipes all add/change/munge data in
a sane way, based on runlist order: basic company-wide cookbooks, then
cookbooks that apply to large swaths of servers, then cookbooks that are more
specific. Each one can change the data.

We rely heavily on templates, so I all the early cookbooks setup those
templates, everyone has plenty of time to add to the data in recipes or roles
as necessary, and then the template will get the most appropriate data.

This is valid, so we fixed it. Attributes files will load in run_list order in Chef 11. Caveats to be aware of:

  • dependencies load before the cookbook that depends on them, so if you have cookbooks in your run_list with dependencies on each other, the depended-on cookbook could load before its position in the run list.
  • The expanded run_list doesn't know if/how you're using include_recipe(). Generally when using include_recipe you also specify a cookbook level dependency, so you can count on the depended-on cookbook being evaluated first, but the order may not match exactly.
  • Dependencies in cookbook metadata are not ordered, so there is no way to specify in which order dependencies get evaluated. We sort them to enforce a repeatable order.

include_attribute can still be used to force an attribute file to load before another.

Alternatively, you can use local variables instead of node attributes as the
final input to your recipes. Your logic would be the same, and in the same
place, you just assign the final value to a local instead.

No, because that's all compile-time. I need the templates to get the data
provided by cookbooks that run after them.

If you really want the value to be set as a node attributes, you can simply
go around the precedence system by accessing
node.attributes.role_override.

I could also do node.override, but that's silly. The precedence system was
logical before. Having data that is evaluated at upload time override data
computed at runtime is counter-intuitive and limiting and defeats the
original beauty of the predence system.

Implementing it as merge order was counterintuitive and artificially limiting for many people, so we're not going back to that, but you're right that there should be a way to have runtime logic win over role data. To that end, we've added a precedence level to replace the "recipe default" and "recipe override". You can access it with node.default![:key] = "value" and node.override![:key] = "value". This is available everywhere, so you could use it in an attributes file if you had a need.

--
Phil Dibowitz phil@ipom.com (mailto:phil@ipom.com)
Open Source software and tech docs Insanity Palace of Metallica
http://www.phildev.net/ http://www.ipom.com/

"Be who you are and say what you feel, because those who mind don't matter
and those who matter don't mind."

  • Dr. Seuss

--
Daniel DeLeo