Understanding node attributes set from 'role' cookbooks

Chef Infra (archive)
1

Hello,

I’m in the process of converting my roles into role cookbooks. A
minority of these roles set attributes and these are causing me
problems.

Assuming my cookbook a has the following in an attributes file:

default[‘a’][‘b’] = true

I can override this in a cookbook recipe as expected:

node.default[‘a’][‘b’] = false

However, as soon as attributes files start performing logic based on
attributes things break down as demonstrated in the following example
cookbook (roleattributes):

attributes file:

default[‘roleattributes’][‘test’] = true
if node[‘roleattributes’][‘test’] == true
default[‘roleattributes’][‘testistrue’] = true
else
default[‘roleattributes’][‘testistrue’] = false
end

default recipe:

node.normal[‘roleattributes’][‘test’] = false
require 'pp’
pp node.debug_value(:roleattributes, :test)
pp node.debug_value(:roleattributes, :testistrue)

The output from the pp command during a Chef run is (with all
not_present stripped for clarity):

pp node.debug_value(:roleattributes, :test)

[“default”, true],
[“normal”, false],

pp node.debug_value(:roleattributes, :testistrue)

[“default”, true],

As can be seen node[‘roleattributes’][‘test’] is correctly false while
node[‘roleattributes’][‘testistrue’] is true. By comparison when I
define roleattributes.test=false in my role normal attributes I get a
different result:

pp node.debug_value(:roleattributes, :test)

[“default”, true],
[“normal”, false],

pp node.debug_value(:roleattributes, :testistrue)

[“default”, false],

Is there any way around this somewhat unexpected (to me) behaviour?
At the moment it is holding up a full migration towards role cookbooks
(to allow versioning, etc.)

Many thanks

Lewis

2

I think you want node.override here

post back if this doesn't work for you. generally a good idea to avoid overrides, but when dealing with cookbooks from various sources, sadly this happens more then one would want.

Greg

Sent from my iPhone

On Mar 14, 2014, at 8:44 AM, Lewis Thompson purple@lewiz.net wrote:

Hello,

I'm in the process of converting my roles into role cookbooks. A
minority of these roles set attributes and these are causing me
problems.

Assuming my cookbook a has the following in an attributes file:

default['a']['b'] = true

I can override this in a cookbook recipe as expected:

node.default['a']['b'] = false

However, as soon as attributes files start performing logic based on
attributes things break down as demonstrated in the following example
cookbook (roleattributes):

attributes file:

default['roleattributes']['test'] = true
if node['roleattributes']['test'] == true
default['roleattributes']['testistrue'] = true
else
default['roleattributes']['testistrue'] = false
end

default recipe:

node.normal['roleattributes']['test'] = false
require 'pp'
pp node.debug_value(:roleattributes, :test)
pp node.debug_value(:roleattributes, :testistrue)

The output from the pp command during a Chef run is (with all
not_present stripped for clarity):

pp node.debug_value(:roleattributes, :test)

["default", true],
["normal", false],

pp node.debug_value(:roleattributes, :testistrue)

["default", true],

As can be seen node['roleattributes']['test'] is correctly false while
node['roleattributes']['testistrue'] is true. By comparison when I
define roleattributes.test=false in my role normal attributes I get a
different result:

pp node.debug_value(:roleattributes, :test)

["default", true],
["normal", false],

pp node.debug_value(:roleattributes, :testistrue)

["default", false],

Is there any way around this somewhat unexpected (to me) behaviour?
At the moment it is holding up a full migration towards role cookbooks
(to allow versioning, etc.)

Many thanks

Lewis

3

On 14 March 2014 12:51, Gregory Patmore gregorypatmore@gmail.com wrote:

I think you want node.override here

About Attributes

Hi Greg

Thanks for this suggestion but I have already tried various override
levels (including node.force_override) but this does not resolve my
problem.

cheers, Lewis

4

On Mar 14, 2014, at 5:44 AM, Lewis Thompson purple@lewiz.net wrote:

Hello,

I'm in the process of converting my roles into role cookbooks. A
minority of these roles set attributes and these are causing me
problems.

Assuming my cookbook a has the following in an attributes file:

default['a']['b'] = true

I can override this in a cookbook recipe as expected:

node.default['a']['b'] = false

However, as soon as attributes files start performing logic based on
attributes things break down as demonstrated in the following example
cookbook (roleattributes):

attributes file:

default['roleattributes']['test'] = true
if node['roleattributes']['test'] == true
default['roleattributes']['testistrue'] = true
else
default['roleattributes']['testistrue'] = false
end

default recipe:

node.normal['roleattributes']['test'] = false
require 'pp'
pp node.debug_value(:roleattributes, :test)
pp node.debug_value(:roleattributes, :testistrue)

The output from the pp command during a Chef run is (with all
not_present stripped for clarity):

pp node.debug_value(:roleattributes, :test)

["default", true],
["normal", false],

pp node.debug_value(:roleattributes, :testistrue)

["default", true],

As can be seen node['roleattributes']['test'] is correctly false while
node['roleattributes']['testistrue'] is true. By comparison when I
define roleattributes.test=false in my role normal attributes I get a
different result:

pp node.debug_value(:roleattributes, :test)

["default", true],
["normal", false],

pp node.debug_value(:roleattributes, :testistrue)

["default", false],

Is there any way around this somewhat unexpected (to me) behaviour?
At the moment it is holding up a full migration towards role cookbooks
(to allow versioning, etc.)

Move that logic into recipe or resource code so it can be lazily evaluated. The problem is that attribute files are run once at the start a run so having any logic based on values that might change later (either due to overrides or anything else) doesn't change the outcome of that execution (since it already happened).

--Noah

5

On 14 March 2014 16:44, Noah Kantrowitz noah@coderanger.net wrote:

Move that logic into recipe or resource code so it can be lazily evaluated. The problem is that attribute files are run once at the start a run so having any logic based on values that might change later (either due to overrides or anything else) doesn't change the outcome of that execution (since it already happened).

Hi Noah

Thanks for your reply. Am I correct in understanding that you mean
the attributes file logic should be moved into a recipe for lazy
evaluation?

I can understand how this would change the behaviour but in this
instance I am wrapping a third-party cookbook. Adjusting that
cookbook would make tracking upstream changes difficult.

Are there any other tricks that I can do to force my wrapping
cookbook's recipe to be evaluated before that of the cookbook with
attribute logic?

thanks, Lewis

6

Ensure load order of attributes by using incliude_attribute[1] in your
wrapper attributes. You can then replace the values of any of the 'library'
cooks attributes in your wrapper with the same attribute level. I.e.
default.

In your wrapper you include_recipe[2] the library cook, and your attributes
will have taken precedence. This imo Is the sane way to wrap things.

I personally avoid node.set as much as possible in my wrappers as it
persists to the node, and It gets hard to reason about what is going on.
Enforcing load order of attributes, recipes, and using default 'most' of
the time will make your life much more sane IMO.

[1]
https://github.com/cloudware-cookbooks/ktc-messaging/blob/master/attributes/rabbit.rb
[2]
https://github.com/cloudware-cookbooks/ktc-messaging/blob/master/recipes/default.rb

  • Jesse

On Fri, Mar 14, 2014 at 11:59 AM, Lewis Thompson purple@lewiz.net wrote:

On 14 March 2014 16:44, Noah Kantrowitz noah@coderanger.net wrote:

Move that logic into recipe or resource code so it can be lazily
evaluated. The problem is that attribute files are run once at the start a
run so having any logic based on values that might change later (either due
to overrides or anything else) doesn't change the outcome of that execution
(since it already happened).

Hi Noah

Thanks for your reply. Am I correct in understanding that you mean
the attributes file logic should be moved into a recipe for lazy
evaluation?

I can understand how this would change the behaviour but in this
instance I am wrapping a third-party cookbook. Adjusting that
cookbook would make tracking upstream changes difficult.

Are there any other tricks that I can do to force my wrapping
cookbook's recipe to be evaluated before that of the cookbook with
attribute logic?

thanks, Lewis

7

On Mar 14, 2014, at 12:10 PM, Jesse Nelson spheromak@gmail.com wrote:

Ensure load order of attributes by using incliude_attribute[1] in your wrapper attributes. You can then replace the values of any of the 'library' cooks attributes in your wrapper with the same attribute level. I.e. default.

This isn't needed, they are already loaded in topographic-sort-order :slight_smile: load_attributes used to be needed more in the Chef 10 days when load order was undefined, but it should almost never come up anymore.

In your wrapper you include_recipe[2] the library cook, and your attributes will have taken precedence. This imo Is the sane way to wrap things.

The problem is you still can't (easily) force things into derived attributes without duplicating the logic.

I personally avoid node.set as much as possible in my wrappers as it persists to the node, and It gets hard to reason about what is going on. Enforcing load order of attributes, recipes, and using default 'most' of the time will make your life much more sane IMO.

Are there any other tricks that I can do to force my wrapping
cookbook's recipe to be evaluated before that of the cookbook with
attribute logic?

Unfortunately no, the load order of these is fixed where all attribute files are run (in topo order) and then recipes specified by the run list are run. My solution to this is unfortunately a tad complex and involves moving derived data to methods on my resource objects.

--Noah

8

On Fri, Mar 14, 2014 at 12:39 PM, Noah Kantrowitz noah@coderanger.netwrote:

On Mar 14, 2014, at 12:10 PM, Jesse Nelson spheromak@gmail.com wrote:

Ensure load order of attributes by using incliude_attribute[1] in your
wrapper attributes. You can then replace the values of any of the 'library'
cooks attributes in your wrapper with the same attribute level. I.e.
default.

This isn't needed, they are already loaded in topographic-sort-order :slight_smile:
load_attributes used to be needed more in the Chef 10 days when load order
was undefined, but it should almost never come up anymore.

For me this is more about indicating intent, and readability for someone
else on my team. As we discussed on IRC this is also needed if you need to
load specific attribute files in a specific order from the same cookbook.

I also just feel safer being explicit vs implicit ordering.

In your wrapper you include_recipe[2] the library cook, and your
attributes will have taken precedence. This imo Is the sane way to wrap
things.

The problem is you still can't (easily) force things into derived
attributes without duplicating the logic.

My approach here is to try and avoid deriving the attrib in the attributes
file(s). I think we agree there.

I personally avoid node.set as much as possible in my wrappers as it
persists to the node, and It gets hard to reason about what is going on.
Enforcing load order of attributes, recipes, and using default 'most' of
the time will make your life much more sane IMO.

Are there any other tricks that I can do to force my wrapping
cookbook's recipe to be evaluated before that of the cookbook with
attribute logic?

Unfortunately no, the load order of these is fixed where all attribute
files are run (in topo order) and then recipes specified by the run list
are run. My solution to this is unfortunately a tad complex and involves
moving derived data to methods on my resource objects.

--Noah

9

On 14 March 2014 19:39, Noah Kantrowitz noah@coderanger.net wrote:

Are there any other tricks that I can do to force my wrapping
cookbook's recipe to be evaluated before that of the cookbook with
attribute logic?

Unfortunately no, the load order of these is fixed where all attribute files are run (in topo order) and then recipes specified by the run list are run. My solution to this is unfortunately a tad complex and involves moving derived data to methods on my resource objects.

Hi Noah,

Thanks again for clarifying.

The following post is just to help explain for anybody else seeing
this. In my example but the attributes & the recipe override were in
a single cookbook; this was my attempt at creating a concise example.

In actual fact I was attempting to override some attributes in one
cookbook from another. It's not possible to do so from within the
recipe, however Noah's answer here helped clarify that it is possible
from within the attributes of the role cookbook.

Hence I now have the following example:

roleattributes/attributes/default.rb:
default['roleattributes']['test'] = true
if node['roleattributes']['test'] == true
 default['roleattributes']['testistrue'] = true
else
 default['roleattributes']['testistrue'] = false
end

roleattributes2/attributes/default.rb:
default['roleattributes']['test'] = false

roleattributes2/recipes/default.rb:
include_recipe 'roleattributes'
require 'pp'
pp node.debug_value(:roleattributes, :test)
pp node.debug_value(:roleattributes, :testistrue)

And the result I receive is exactly what I wanted:

# pp node.debug_value(:roleattributes, :test)
["default", false],
["normal", false],
# pp node.debug_value(:roleattributes, :testistrue)
["default", false],

Admittedly this is not as clean as having it defined entirely within a
recipe, but I can still make this work.

The key here for me was that cookbook attributes files are all
evaluated in their entirety before recipes are evaluated, hence any
login in attributes files cannot be manipulated from recipes.

Many thanks for the help guys.

Lewis

10

The key here for me was that cookbook attributes files are all
evaluated in their entirety before recipes are evaluated, hence any
login in attributes files cannot be manipulated from recipes.

FYI: you can still reload the attributes file from within a recipe, see
here:

For example:

# override some node attributes here
node.set ....

# re-evaluate attributes files whose attributes are computed based on the
ones above
node.from_file(run_context.resolve_attribute("postfix", "default"))

# include actual recipes which need the attributes
include_recipe 'postfix::default'
include_recipe 'postfix::sasl_auth'

Maybe this works in your situation as well?

HTH, Torben

Many thanks for the help guys.

Lewis