Nested node attributes not working as expected when using overrides?

Chef Infra (archive)
1

Hi Chefs!

I’m often using something like this in my attributes/default.rb:

default['my_recipe']['base_dir']  = '/path/to/basedir'
default['my_recipe']['other_dir']  =

“#{node[‘my_recipe’][‘base_dir’]}/otherdir”

This looks sane at first, but as soon as I’m overriding the default value
for [‘my_recipe’][‘base_dir’], e.g. in a role like so…

...
default_attributes({
  :my_recipe => {
    :base_dir => '/somewhere/else'
  }
})
...

…I get a weird behavior when using the node attributes in my recipe:

=> node[‘my_recipe’][‘base_dir’] renders to ‘/somewhere/else’
=> node[‘my_recipe’][‘other_dir’] renders to ‘/path/to/basedir/otherdir’

Is this the expected behaviour?!

Cheers,
Torben

P.S.: this was under omnibus chef 10.14.4 on a 64 bit Ubuntu 12.04

2

Yes I believe this is expected. I ran across this problem and the solution
is that if you ever use nested attributes, you should always use them in
your recipes and not in attributes as Chef cannot guarantee the ordering of
when things are loaded (i.e. environments, attributes files, roles...)

On Fri, Oct 19, 2012 at 9:43 AM, Torben Knerr ukio@gmx.de wrote:

Hi Chefs!

I'm often using something like this in my attributes/default.rb:

default['my_recipe']['base_dir']  = '/path/to/basedir'
default['my_recipe']['other_dir']  =

"#{node['my_recipe']['base_dir']}/otherdir"

This looks sane at first, but as soon as I'm overriding the default value
for ['my_recipe']['base_dir'], e.g. in a role like so...

...
default_attributes({
  :my_recipe => {
    :base_dir => '/somewhere/else'
  }
})
...

...I get a weird behavior when using the node attributes in my recipe:

=> node['my_recipe']['base_dir'] renders to '/somewhere/else'
=> node['my_recipe']['other_dir'] renders to '/path/to/basedir/otherdir'

Is this the expected behaviour?!

Cheers,
Torben

P.S.: this was under omnibus chef 10.14.4 on a 64 bit Ubuntu 12.04

--
-Bryan

3

Waah, really? I hoped I did something wrong. Wouldn't that violate the
principle of least surprise? Or is it just me who is "surprised" by that...

Btw: I'm seeing this pattern in lots of recipes, guess this would be a
surprise to some of the authors as well.

Is this documented anywhere in the Wiki?

On Fri, Oct 19, 2012 at 4:47 PM, Bryan Baugher bjbq4d@gmail.com wrote:

Yes I believe this is expected. I ran across this problem and the solution
is that if you ever use nested attributes, you should always use them in
your recipes and not in attributes as Chef cannot guarantee the ordering of
when things are loaded (i.e. environments, attributes files, roles...)

On Fri, Oct 19, 2012 at 9:43 AM, Torben Knerr ukio@gmx.de wrote:

Hi Chefs!

I'm often using something like this in my attributes/default.rb:

default['my_recipe']['base_dir']  = '/path/to/basedir'
default['my_recipe']['other_dir']  =

"#{node['my_recipe']['base_dir']}/otherdir"

This looks sane at first, but as soon as I'm overriding the default value
for ['my_recipe']['base_dir'], e.g. in a role like so...

...
default_attributes({
  :my_recipe => {
    :base_dir => '/somewhere/else'
  }
})
...

...I get a weird behavior when using the node attributes in my recipe:

=> node['my_recipe']['base_dir'] renders to '/somewhere/else'
=> node['my_recipe']['other_dir'] renders to '/path/to/basedir/otherdir'

Is this the expected behaviour?!

Cheers,
Torben

P.S.: this was under omnibus chef 10.14.4 on a 64 bit Ubuntu 12.04

--
-Bryan

4

I looked and didn't find anything. I have been using nested attributes for
quite some time and haven't had any issues using them in recipes.

On Fri, Oct 19, 2012 at 9:53 AM, Torben Knerr ukio@gmx.de wrote:

Waah, really? I hoped I did something wrong. Wouldn't that violate the
principle of least surprise? Or is it just me who is "surprised" by that...

Btw: I'm seeing this pattern in lots of recipes, guess this would be a
surprise to some of the authors as well.

Is this documented anywhere in the Wiki?

On Fri, Oct 19, 2012 at 4:47 PM, Bryan Baugher bjbq4d@gmail.com wrote:

Yes I believe this is expected. I ran across this problem and the
solution is that if you ever use nested attributes, you should always use
them in your recipes and not in attributes as Chef cannot guarantee the
ordering of when things are loaded (i.e. environments, attributes files,
roles...)

On Fri, Oct 19, 2012 at 9:43 AM, Torben Knerr ukio@gmx.de wrote:

Hi Chefs!

I'm often using something like this in my attributes/default.rb:

default['my_recipe']['base_dir']  = '/path/to/basedir'
default['my_recipe']['other_dir']  =

"#{node['my_recipe']['base_dir']}/otherdir"

This looks sane at first, but as soon as I'm overriding the default
value for ['my_recipe']['base_dir'], e.g. in a role like so...

...
default_attributes({
  :my_recipe => {
    :base_dir => '/somewhere/else'
  }
})
...

...I get a weird behavior when using the node attributes in my recipe:

=> node['my_recipe']['base_dir'] renders to '/somewhere/else'
=> node['my_recipe']['other_dir'] renders to '/path/to/basedir/otherdir'

Is this the expected behaviour?!

Cheers,
Torben

P.S.: this was under omnibus chef 10.14.4 on a 64 bit Ubuntu 12.04

--
-Bryan

--
-Bryan

5

On Friday, October 19, 2012 at 8:11 AM, Bryan Baugher wrote:

I looked and didn't find anything. I have been using nested attributes for quite some time and haven't had any issues using them in recipes.

On Fri, Oct 19, 2012 at 9:53 AM, Torben Knerr <ukio@gmx.de (mailto:ukio@gmx.de)> wrote:

Waah, really? I hoped I did something wrong. Wouldn't that violate the principle of least surprise? Or is it just me who is "surprised" by that...

Btw: I'm seeing this pattern in lots of recipes, guess this would be a surprise to some of the authors as well.

Is this documented anywhere in the Wiki?

Currently, precedence within an attribute level is implemented by order of application. This means that the only safe time to generate a computed value is in a recipe, since attributes from roles will not have been applied until later.

We recently fixed a different but related issue, CHEF-1804[0] with an eye towards making this issue (CHEF-2936[1]) easier to resolve; it should be fixed in master in the next couple of weeks. Note that these are potentially breaking changes so they will go in Chef 11.0 but not 10.x.

--
Daniel DeLeo

  1. http://tickets.opscode.com/browse/CHEF-1804
  2. http://tickets.opscode.com/browse/CHEF-2936
6

Am 19.10.2012 17:36 schrieb "Daniel DeLeo" dan@kallistec.com:

On Friday, October 19, 2012 at 8:11 AM, Bryan Baugher wrote:

I looked and didn't find anything. I have been using nested attributes
for quite some time and haven't had any issues using them in recipes.

On Fri, Oct 19, 2012 at 9:53 AM, Torben Knerr ukio@gmx.de wrote:

Waah, really? I hoped I did something wrong. Wouldn't that violate the
principle of least surprise? Or is it just me who is "surprised" by that...

Btw: I'm seeing this pattern in lots of recipes, guess this would be a
surprise to some of the authors as well.

Is this documented anywhere in the Wiki?

Currently, precedence within an attribute level is implemented by order
of application. This means that the only safe time to generate a computed
value is in a recipe, since attributes from roles will not have been
applied until later.

We recently fixed a different but related issue, CHEF-1804[0] with an eye
towards making this issue (CHEF-2936[1]) easier to resolve; it should be
fixed in master in the next couple of weeks. Note that these are
potentially breaking changes so they will go in Chef 11.0 but not 10.x.

--
Daniel DeLeo

  1. http://tickets.opscode.com/browse/CHEF-1804
  2. http://tickets.opscode.com/browse/CHEF-2936

Ah, thanks! CHEF-2936 is exactly what hits me. CHEF-1804 is weird as well,
I'll try to keep that in the back of my head...

As this behaviour is quite counter-intuitive, would you agree that a
foodcritic rule would make sense here?

And a side question: what's the way to dump a node's "resolved" attributes?
Is it possible via knife? I'm currently doing it via shef on the node...

Cheers, Torben

7

Since failed-run-data.json has it, you could probably write a really
simple Chef handler to dump/deliver node data...

On Mon, Oct 22, 2012 at 12:40 AM, Torben Knerr ukio@gmx.de wrote:

Am 19.10.2012 17:36 schrieb "Daniel DeLeo" dan@kallistec.com:

On Friday, October 19, 2012 at 8:11 AM, Bryan Baugher wrote:

I looked and didn't find anything. I have been using nested attributes
for quite some time and haven't had any issues using them in recipes.

On Fri, Oct 19, 2012 at 9:53 AM, Torben Knerr ukio@gmx.de wrote:

Waah, really? I hoped I did something wrong. Wouldn't that violate the
principle of least surprise? Or is it just me who is "surprised" by that...

Btw: I'm seeing this pattern in lots of recipes, guess this would be a
surprise to some of the authors as well.

Is this documented anywhere in the Wiki?

Currently, precedence within an attribute level is implemented by order of
application. This means that the only safe time to generate a computed value
is in a recipe, since attributes from roles will not have been applied until
later.

We recently fixed a different but related issue, CHEF-1804[0] with an eye
towards making this issue (CHEF-2936[1]) easier to resolve; it should be
fixed in master in the next couple of weeks. Note that these are potentially
breaking changes so they will go in Chef 11.0 but not 10.x.

--
Daniel DeLeo

  1. http://tickets.opscode.com/browse/CHEF-1804
  2. http://tickets.opscode.com/browse/CHEF-2936

Ah, thanks! CHEF-2936 is exactly what hits me. CHEF-1804 is weird as well,
I'll try to keep that in the back of my head...

As this behaviour is quite counter-intuitive, would you agree that a
foodcritic rule would make sense here?

And a side question: what's the way to dump a node's "resolved" attributes?
Is it possible via knife? I'm currently doing it via shef on the node...

Cheers, Torben

8

Thanks Steve, but this is only for failed runs, is it?
Am 22.10.2012 18:53 schrieb "steve ." leftathome@gmail.com:

Since failed-run-data.json has it, you could probably write a really
simple Chef handler to dump/deliver node data...

On Mon, Oct 22, 2012 at 12:40 AM, Torben Knerr ukio@gmx.de wrote:

Am 19.10.2012 17:36 schrieb "Daniel DeLeo" dan@kallistec.com:

On Friday, October 19, 2012 at 8:11 AM, Bryan Baugher wrote:

I looked and didn't find anything. I have been using nested attributes
for quite some time and haven't had any issues using them in recipes.

On Fri, Oct 19, 2012 at 9:53 AM, Torben Knerr ukio@gmx.de wrote:

Waah, really? I hoped I did something wrong. Wouldn't that violate the
principle of least surprise? Or is it just me who is "surprised" by
that...

Btw: I'm seeing this pattern in lots of recipes, guess this would be a
surprise to some of the authors as well.

Is this documented anywhere in the Wiki?

Currently, precedence within an attribute level is implemented by order
of
application. This means that the only safe time to generate a computed
value
is in a recipe, since attributes from roles will not have been applied
until
later.

We recently fixed a different but related issue, CHEF-1804[0] with an
eye
towards making this issue (CHEF-2936[1]) easier to resolve; it should be
fixed in master in the next couple of weeks. Note that these are
potentially
breaking changes so they will go in Chef 11.0 but not 10.x.

--
Daniel DeLeo

  1. http://tickets.opscode.com/browse/CHEF-1804
  2. http://tickets.opscode.com/browse/CHEF-2936

Ah, thanks! CHEF-2936 is exactly what hits me. CHEF-1804 is weird as
well,
I'll try to keep that in the back of my head...

As this behaviour is quite counter-intuitive, would you agree that a
foodcritic rule would make sense here?

And a side question: what's the way to dump a node's "resolved"
attributes?
Is it possible via knife? I'm currently doing it via shef on the node...

Cheers, Torben

9

On Monday, October 22, 2012 at 10:36 AM, Torben Knerr wrote:

Ah, thanks! CHEF-2936 is exactly what hits me. CHEF-1804 is weird as well,
I'll try to keep that in the back of my head...

As this behaviour is quite counter-intuitive, would you agree that a
foodcritic rule would make sense here?
Yeah, a foodcritic rule would be nice. I was poking around in foodcritic recently and found that it doesn't detect some types of attribute access[0], so fixing that would be a good starting point.

We intend to fix this for Chef 11, with the intention of allowing more logic (especially computed/derived attribute values and platform-specific attribute values) in attributes files so any foodcritic rule ought to branch on Chef version above/below 11.0.

And a side question: what's the way to dump a node's "resolved" attributes?
Is it possible via knife? I'm currently doing it via shef on the node...

Cheers, Torben
Knife can give you a section of a node's attribute tree with the -a option, after chef client has run and saved that node's data. To expand the run_list and eval attributes, you're better off doing this on the node, since cookbooks could have platform branching logic which would set different attributes.

--
Daniel DeLeo

  1. Rule to detect that node attribute setting methods are used · Issue #81 · Foodcritic/foodcritic · GitHub
10

On Mon, Oct 22, 2012 at 9:11 PM, Daniel DeLeo dan@kallistec.com wrote:

On Monday, October 22, 2012 at 10:36 AM, Torben Knerr wrote:

Ah, thanks! CHEF-2936 is exactly what hits me. CHEF-1804 is weird as
well,
I'll try to keep that in the back of my head...

As this behaviour is quite counter-intuitive, would you agree that a
foodcritic rule would make sense here?

Yeah, a foodcritic rule would be nice. I was poking around in foodcritic
recently and found that it doesn't detect some types of attribute
access[0], so fixing that would be a good starting point.

We intend to fix this for Chef 11, with the intention of allowing more
logic (especially computed/derived attribute values and platform-specific
attribute values) in attributes files so any foodcritic rule ought to
branch on Chef version above/below 11.0.

FYI: added Rule to detect nested node attributes in attributes.rb · Issue #82 · Foodcritic/foodcritic · GitHub for the nested
node attributes issue specifically

11

(Sorry for the delay, had a few stone-age deployments take up my
list-reading time)

That particular handler only hooks failed runs. You could either
extend it so that it runs on successful runs as well ... or you could
intentionally write a recipe bogus resource right there at the end of
the run list and intentionally trigger a run failure. Or write a
failing test if you use chef-minitest-handler...

On Mon, Oct 22, 2012 at 10:36 AM, Torben Knerr ukio@gmx.de wrote:

Thanks Steve, but this is only for failed runs, is it?

Am 22.10.2012 18:53 schrieb "steve ." leftathome@gmail.com:

Since failed-run-data.json has it, you could probably write a really
simple Chef handler to dump/deliver node data...

On Mon, Oct 22, 2012 at 12:40 AM, Torben Knerr ukio@gmx.de wrote:

Am 19.10.2012 17:36 schrieb "Daniel DeLeo" dan@kallistec.com:

On Friday, October 19, 2012 at 8:11 AM, Bryan Baugher wrote:

I looked and didn't find anything. I have been using nested attributes
for quite some time and haven't had any issues using them in recipes.

On Fri, Oct 19, 2012 at 9:53 AM, Torben Knerr ukio@gmx.de wrote:

Waah, really? I hoped I did something wrong. Wouldn't that violate
the
principle of least surprise? Or is it just me who is "surprised" by
that...

Btw: I'm seeing this pattern in lots of recipes, guess this would be
a
surprise to some of the authors as well.

Is this documented anywhere in the Wiki?

Currently, precedence within an attribute level is implemented by order
of
application. This means that the only safe time to generate a computed
value
is in a recipe, since attributes from roles will not have been applied
until
later.

We recently fixed a different but related issue, CHEF-1804[0] with an
eye
towards making this issue (CHEF-2936[1]) easier to resolve; it should
be
fixed in master in the next couple of weeks. Note that these are
potentially
breaking changes so they will go in Chef 11.0 but not 10.x.

--
Daniel DeLeo

  1. http://tickets.opscode.com/browse/CHEF-1804
  2. http://tickets.opscode.com/browse/CHEF-2936

Ah, thanks! CHEF-2936 is exactly what hits me. CHEF-1804 is weird as
well,
I'll try to keep that in the back of my head...

As this behaviour is quite counter-intuitive, would you agree that a
foodcritic rule would make sense here?

And a side question: what's the way to dump a node's "resolved"
attributes?
Is it possible via knife? I'm currently doing it via shef on the node...

Cheers, Torben