Nested attributes in policyfiles

Chef Infra (archive)
1

In Nested Attributes section of POLICYFILE_README.md it talks about nesting attributes one level deeper and then ‘hoisting’ them. How exactly? I was playing around with it and can’t get it to work. I tried something like this:

In my chef-repo:
Policyfile.rb

default['staging']['domain'] = 'foo'
default['prod']['domain'] = 'bar'

cookbooks/server-provision/recipes/setup_server.rb

context = ChefDK::ProvisioningData.context
# is this happening on the servers where chef-client runs? Or on provisioning node?
node.default['env'] = context.opts.env
policy_group = context.policy_group
policy_name = context.policy_name
machine_batch do
  1.upto(2) do |i|
    machine "#{context.node_name}-#{i}" do
      chef_config "use_policyfile true\nversioned_cookbooks true\npolicy_group '#{policy_group}'\npolicy_name '#{policy_name}'"
      converge true
    end
  end
end

In different/separate cookbook that uses the attribute domain, say my-base-cookbook:
in the attributes/default.rb:

default['domain']='undefined'

in recipes/default.rb:

case node['env']
when 'prod'
  node.default['domain'] = node['prod']['domain']
when 'staging'
  node.default['domain'] = node['staging']['domain']
else
  Chef::Log.fatal('unknown env')
end

I run commands like this:

chef update Policyfile.rb
chef push staging Policyfile.rb
chef provision --no-policy -n my-server --recipe setup_server --cookbook cookbooks/server-provision -o env='staging'

I don’t see the ‘domain’ attribute change from ‘undefined’. When I examine the node on chef server I don’t see the env attribute set. What am I doing wrong? And/or is there another way I can ‘hoist’? What would be some ways to figure out if in prod, staging, qa, dev?

Thanks.
-Maciej

2

The basic idea is you have the attributes in your policyfile set pretty much like you do in your example. Then, in the attributes file of a cookbook, you do something like this:

default.merge!(node["env_specific"])

Where I have the string "env_specific" you probably want Chef::Config.policy_group or something like that (in Chef 12.5 you could do node.policy_group).

3

Thanks Dan. That worked. Sort of. I’m using the system cookbook to set hostname and timezone. I run into problems using both the system::hostname and system::timezone recipes. I tried to debug the system::hostname problem…

The error during chef-client run:

 ================================================================================
                      Error executing action `set` on resource 'system_hostname[stage_1]'
                      ================================================================================
                      
                      NoMethodError
                      -------------
                      undefined method `[]' for nil:NilClass
                      
                      Cookbook Trace:
                      ---------------
                      /var/chef/cache/cookbooks/system/providers/hostname.rb:105:in `block in class_from_file'
                      
                      Resource Declaration:
                      ---------------------
                      # In /var/chef/cache/cookbooks/system/recipes/hostname.rb
                      
                       20: system_hostname node['system']['short_hostname'] do
                       21:   short_hostname    node['system']['short_hostname']
                       22:   domain_name       node['system']['domain_name']
                       23:   static_hosts      node['system']['static_hosts']
                       24:   netbios_name      node['system']['netbios_name']
                       25:   workgroup         node['system']['workgroup']
                       26:   manage_hostsfile  node['system']['manage_hostsfile']
                       27: end
                      
                      Compiled Resource:
                      ------------------
                      # Declared in /var/chef/cache/cookbooks/system/recipes/hostname.rb:20:in `from_file'/recipes/hostname.rb:20:in `from_file'

I’m struggling to understand exactly what the problem is and how to fix it. I believe the error msg I see means some attribute is not set. I tracked the exception down to this line: https://github.com/xhost-cookbooks/system/blob/master/providers/hostname.rb#L97

primary_if is set to nothing on line 96? I added some debug stmts to check. [node[‘system’][‘primary_interface’] is set to nil. In the attributes/default.rb file it is set to node[‘network’[‘default_interface’], see https://github.com/xhost-cookbooks/system/blob/master/attributes/default.rb#L37

If I print out node[‘network’[‘default_interface’] in providers/hostname.rb I see it is set. So… this is where I’m stuck now. If I remove the call to default.merge() using the system::hostname recipe works.

Any suggestions? I’m open to suggestions to use another cookbook or method of setting hostname and timezone as well. I haven’t dug as deep into timezone recipe problem with this cookbook.

Thanks.
-Maciej

4

Can you use debug_value to check the data before and after the merge!? Joshua has a guide for how to use that here: http://jtimberman.housepub.org/blog/2014/09/02/chef-node-dot-debug-value/

5

Not sure what I found helps much, or maybe I’m doing something wrong.

Here’s what I did:

  • ran chef-shell -z on the node.
  • used debug_value to check some data
  • ran run_chef in chef-shell, run failed as before.
  • used debug_value to check data again.

It looks to me like the line https://github.com/xhost-cookbooks/system/blob/master/attributes/default.rb#L37
is just not being run/set.

The actual output:

> chef (12.5.1)> pp node.debug_value('system','primary_interface')
> [["set_unless_enabled?", false],
>  ["default", :not_present],
>  ["env_default", :not_present],
>  ["role_default", :not_present],
>  ["force_default", :not_present],
>  ["normal", :not_present],
>  ["override", :not_present],
>  ["role_override", :not_present],
>  ["env_override", :not_present],
>  ["force_override", :not_present],
>  ["automatic", :not_present]]
>  => [["set_unless_enabled?", false], ["default", :not_present], ["env_default", :not_present], ["role_default", :not_present], ["force_default", :not_present], [
> "normal", :not_present], ["override", :not_present], ["role_override", :not_present], ["env_override", :not_present], ["force_override", :not_present], ["automat
> ic", :not_present]]                                                                                                                                             
> chef (12.5.1)> pp node.debug_value('network','default_interface')
> [["set_unless_enabled?", false],
>  ["default", :not_present],
>  ["env_default", :not_present],
>  ["role_default", :not_present],
>  ["force_default", :not_present],
>  ["normal", :not_present],
>  ["override", :not_present],
>  ["role_override", :not_present],
>  ["env_override", :not_present],
>  ["force_override", :not_present],
>  ["automatic", "enp0s3"]]
>  => [["set_unless_enabled?", false], ["default", :not_present], ["env_default", :not_present], ["role_default", :not_present], ["force_default", :not_present], [
> "normal", :not_present], ["override", :not_present], ["role_override", :not_present], ["env_override", :not_present], ["force_override", :not_present], ["automat
> ic", "enp0s3"]]                                                                                                                                                 
> chef (12.5.1)> 
> chef > pp node['network']['interfaces'][node['system']['primary_interface']]
> nil
>  => nil 
> chef (12.5.1)> pp node['network']['interfaces'][node['network']['default_interface']]
> {"type"=>"enp0s",
>  "number"=>"3",
>  "mtu"=>"1500",
>  "flags"=>["BROADCAST", "MULTICAST", "UP", "LOWER_UP"],
>  "encapsulation"=>"Ethernet",
>  "addresses"=>
>   {"08:00:27:89:34:17"=>{"family"=>"lladdr"},
>    "10.0.2.15"=>
>     {"family"=>"inet",
>      "prefixlen"=>"24",
>      "netmask"=>"255.255.255.0",
>      "broadcast"=>"10.0.2.255",
>      "scope"=>"Global"},
>    "fe80::a00:27ff:fe89:3417"=>
>     {"family"=>"inet6", "prefixlen"=>"64", "scope"=>"Link"}},
>  "state"=>"up",
>  "arp"=>{"10.0.2.3"=>"52:54:00:12:35:03", "10.0.2.2"=>"52:54:00:12:35:02"},
>  "routes"=>
>   [{"destination"=>"default",
>     "family"=>"inet",
>     "via"=>"10.0.2.2",
>     "metric"=>"100",
>     "proto"=>"static"},
>    {"destination"=>"10.0.2.0/24",
>     "family"=>"inet",
>     "scope"=>"link",
>     "metric"=>"100",
>     "proto"=>"kernel",
>     "src"=>"10.0.2.15"},
>    {"destination"=>"fe80::/64",
>     "family"=>"inet6",
>     "metric"=>"256",
>     "proto"=>"kernel"}]}
>  => {"type"=>"enp0s", "number"=>"3", "mtu"=>"1500", "flags"=>["BROADCAST", "MULTICAST", "UP", "LOWER_UP"], "encapsulation"=>"Ethernet", "addresses"=>{"08:00:27:8
> 9:34:17"=>{"family"=>"lladdr"}, "10.0.2.15"=>{"family"=>"inet", "prefixlen"=>"24", "netmask"=>"255.255.255.0", "broadcast"=>"10.0.2.255", "scope"=>"Global"}, "fe
> 80::a00:27ff:fe89:3417"=>{"family"=>"inet6", "prefixlen"=>"64", "scope"=>"Link"}}, "state"=>"up", "arp"=>{"10.0.2.3"=>"52:54:00:12:35:03", "10.0.2.2"=>"52:54:00:
> 12:35:02"}, "routes"=>[{"destination"=>"default", "family"=>"inet", "via"=>"10.0.2.2", "metric"=>"100", "proto"=>"static"}, {"destination"=>"10.0.2.0/24", "famil
> y"=>"inet", "scope"=>"link", "metric"=>"100", "proto"=>"kernel", "src"=>"10.0.2.15"}, {"destination"=>"fe80::/64", "family"=>"inet6", "metric"=>"256", "proto"=>"
> kernel"}]}                                                                                                                                                      
> chef (12.5.1)> run_chef
> [2015-11-18T18:02:38+00:00] INFO: Processing system_hostname[production_2] action set (system::hostname line 20)
> [2015-11-18T18:02:38+00:00] DEBUG: Providers for generic system_hostname resource enabled on node include: [LWRP provider system_hostname from cookbook system]
> [2015-11-18T18:02:38+00:00] DEBUG: Provider for action set on resource system_hostname[production_2] is LWRP provider system_hostname from cookbook system
> [2015-11-18T18:02:38+00:00] DEBUG: FQDN determined to be: production_2.production.cartera.com
> [2015-11-18T18:02:38+00:00] DEBUG: Resources for generic ruby_block resource enabled on node include: [Chef::Resource::RubyBlock]
> [2015-11-18T18:02:38+00:00] DEBUG: Resource for ruby_block is Chef::Resource::RubyBlock

> ================================================================================
> Error executing action `set` on resource 'system_hostname[production_2]'
> ================================================================================

> NoMethodError
> -------------
> undefined method `[]' for nil:NilClass

> Cookbook Trace:
> ---------------
> /home/vagrant/.chef/cache/cookbooks/system/providers/hostname.rb:97:in `block in class_from_file'

> Resource Declaration:
> ---------------------
> # In /home/vagrant/.chef/cache/cookbooks/system/recipes/hostname.rb

>  20: system_hostname node['system']['short_hostname'] do
>  21:   short_hostname    node['system']['short_hostname']
>  22:   domain_name       node['system']['domain_name']
>  23:   static_hosts      node['system']['static_hosts']
>  24:   netbios_name      node['system']['netbios_name']
>  25:   workgroup         node['system']['workgroup']
>  26:   manage_hostsfile  node['system']['manage_hostsfile']
>  27: end

> Compiled Resource:
> ------------------
> # Declared in /home/vagrant/.chef/cache/cookbooks/system/recipes/hostname.rb:20:in `from_file'

> system_hostname("production_2") do
>   action :set
>   retries 0
>   retry_delay 2
>   default_guard_interpreter :default
>   declared_type :system_hostname
>   cookbook_name "system"
>   recipe_name "hostname"
>   short_hostname "production_2"
>   domain_name "production.cartera.com"
>   manage_hostsfile true
>   hostname "production_2"
> end

> [2015-11-18T18:02:38+00:00] INFO: Running queued delayed notifications before re-raising exception
> NoMethodError: system_hostname[production_2] (system::hostname line 20) had an error: NoMethodError: undefined method `[]' for nil:NilClass
>         from /home/vagrant/.chef/cache/cookbooks/system/providers/hostname.rb:97:in `block in class_from_file'
>         from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/provider/lwrp_base.rb:86:in `instance_eval'
>         from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/provider/lwrp_base.rb:86:in `block in action'
>         from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/provider.rb:144:in `run_action'
>         from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource.rb:585:in `run_action'
>         from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/runner.rb:49:in `run_action'
>         from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/runner.rb:81:in `block (2 levels) in converge'
>         from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/runner.rb:81:in `each'
>         from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/runner.rb:81:in `block in converge'
>         from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/resource_list.rb:83:in `block in execute_each_resource'
>         from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/stepable_iterator.rb:116:in `call'
>         from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/stepable_iterator.rb:116:in `call_iterator_block'
>         from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'
>         from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/stepable_iterator.rb:104:in `iterate'
>         from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/stepable_iterator.rb:55:in `each_with_index'
>         from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/resource_list.rb:81:in `execute_each_resource'
>         from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/runner.rb:80:in `converge'
>         from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/shell/ext.rb:236:in `run_chef'
>         from (irb):6
>         from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/shell.rb:76:in `block in start'
>         from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/shell.rb:75:in `catch'
>         from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/shell.rb:75:in `start'
>         from /opt/chef/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/bin/chef-shell:37:in `<top (required)>'
>         from /usr/bin/chef-shell:54:in `load'
>         from /usr/bin/chef-shell:54:in `<main>'chef (12.5.1)> 
> chef > pp node.debug_value('network','default_interface')
> [["set_unless_enabled?", false],
>  ["default", :not_present],
>  ["env_default", :not_present],
>  ["role_default", :not_present],
>  ["force_default", :not_present],
>  ["normal", :not_present],
>  ["override", :not_present],
>  ["role_override", :not_present],
>  ["env_override", :not_present],
>  ["force_override", :not_present],
>  ["automatic", "enp0s3"]]
>  => [["set_unless_enabled?", false], ["default", :not_present], ["env_default", :not_present], ["role_default", :not_present], ["force_default", :not_present], [
> "normal", :not_present], ["override", :not_present], ["role_override", :not_present], ["env_override", :not_present], ["force_override", :not_present], ["automat
> ic", "enp0s3"]]                                                                                                                                                 
> chef (12.5.1)> pp node.debug_value('system','primary_interface')
> [["set_unless_enabled?", false],
>  ["default", :not_present],
>  ["env_default", :not_present],
>  ["role_default", :not_present],
>  ["force_default", :not_present],
>  ["normal", :not_present],
>  ["override", :not_present],
>  ["role_override", :not_present],
>  ["env_override", :not_present],
>  ["force_override", :not_present],
>  ["automatic", :not_present]]
>  => [["set_unless_enabled?", false], ["default", :not_present], ["env_default", :not_present], ["role_default", :not_present], ["force_default", :not_present], [
> "normal", :not_present], ["override", :not_present], ["role_override", :not_present], ["env_override", :not_present], ["force_override", :not_present], ["automat
> ic", :not_present]]                                                                                                                                             
> chef (12.5.1)>
6

I tried same thing as my last post, but commented out the line:

default.merge!(node[env])

in my base cookbook's attributes/default.rb file.

I get different output from chef-shell. Right away when I start chef-shell and view data i see values are set:

chef (12.5.1)> pp node.debug_value('network','default_interface')
[["set_unless_enabled?", false],
["default", :not_present],
["env_default", :not_present],
["role_default", :not_present],
["force_default", :not_present],
["normal", :not_present],
["override", :not_present],
["role_override", :not_present],
["env_override", :not_present],
["force_override", :not_present],
["automatic", "enp0s3"]]
=> [["set_unless_enabled?", false], ["default", :not_present], ["env_default", :not_present], ["role_default", :not_present], ["force_default", :not_present], [
"normal", :not_present], ["override", :not_present], ["role_override", :not_present], ["env_override", :not_present], ["force_override", :not_present], ["automat
ic", "enp0s3"]]
chef (12.5.1)> pp node.debug_value('system','primary_interface')
[["set_unless_enabled?", false],
["default", "enp0s3"],
["env_default", :not_present],
["role_default", :not_present],
["force_default", :not_present],
["normal", :not_present],
["override", :not_present],
["role_override", :not_present],
["env_override", :not_present],
["force_override", :not_present],
["automatic", :not_present]]
=> [["set_unless_enabled?", false], ["default", "enp0s3"], ["env_default", :not_present], ["role_default", :not_present], ["force_default", :not_present], ["nor
mal", :not_present], ["override", :not_present], ["role_override", :not_present], ["env_override", :not_present], ["force_override", :not_present], ["automatic",
:not_present]]

And the chef run succeeds.

When I start chef-shell -z, does that automatically evaluate/exec lines in attribute files? If so, how would I insert a breakpoint or stop that so I can print values before/after?

-Maciej