Querying other client attributes in a recipe

Chef Infra (archive)
1

I cannot seem to get the syntax right for using search in a recipe. I am hoping someone can help!

What I am ultimately trying to accomplish:

When chef-client runs on each machine, it "discovers" what needs to be monitored on each machine and sets the attribute node.default['monitoring']['nagios']['nrpe']['commands'] = [ 'x' , 'y', 'z' ]. This part seems to be working fine.

Next I want to run a recipe on the nagios server that generates the configs for each machine. To do this I need to query default['monitoring']['nagios']['nrpe']['commands'] for each server to generate what needs to be monitored.

Running this from the command line retrieves what I am looking for:

knife search node "name:tempmachine2 AND monitoring:*" -a monitoring.nagios.nrpe.commands

tempmachine2:
  monitoring.nagios.nrpe.commands:
    check_cron_procs
    check_nfs_mounts
    check_gmond_procs
    check_mailqueue
    check_postfix_proc
    ...

But I am not sure how to do this in a recipe. I thought this is what I needed, but this does not work:

nrpe_hosts = search(:node, "name:tempmachine2 and monitoring:*")
nrpe_hosts.each do |nrpe_list|
  nrpe_list['monitoring']['nagios']['nrpe']['commands'].each do |cmd|
    Chef::Log.warn("NRPE CMD: " + cmd)
  end
end

but that ends with a:

NoMethodError
-------------
undefined method `[]' for nil:NilClass

What am I missing? Or maybe I am trying to do something that is not possible?

I 'll also mention that I am running this in test kitchen, but I have dumped tempmachine2 (and other machines) configs into test/integration/nodes/tempmachine2.json with knife -Fj. Other search queries in test kitchen seem to work fine - and I also see all the monitoring attributes in the json files, so I know they are there!

example of working search in test kitchen:

newton_servers = search(:node, 'tags:auto_newton')
newton_servers.each do | curr_server |
  hostname = curr_server.name.split(".")
  Chef::Log.warn("NAGIOS HOST: #{hostname[0]}")
end

Thanks!

2

Here is a generic example I have for finding a list of ips for a given role which is quite useful for clustered solutions such as elasticsearch:

def get_server_ips(env, role)
  servers = []

  if Chef::Config[:solo]
    Chef::Log.warn 'Warning: we do not support chef solo in this recipe'
  else
      host_search = search(:node, "chef_environment:#{env} AND role:#{role}")
      host_search.each do |nodeobj|
        servers << nodeobj.node.ipaddress
      end
  end

  return servers.sort
end

Looking at your example you need to iterate over the object returned, node, and then attributes as you normally would access them.

3

Thanks for the reply MajorMoses!

The search that I posted at the end of my post is pretty much the same as the one you posted. That DOES seem to work fine. The problem I am having is figuring out how to query a multidimensional attribute.

If I have this set for each node:

default[‘monitoring’][‘nagios’][‘nrpe’][‘commands’] = [ ‘x’ , ‘y’, ‘z’ ]

How do I query that in the search and get the results set [ ‘x’ , ‘y’, ‘z’ ]?

Thanks!

4

Your example is still missing node, I think it should look like this:

newton_servers = search(:node, 'tags:auto_newton')
newton_servers.each do | curr_server |
  hostname = curr_server.node.name.split(".") # added .node object
  Chef::Log.warn("NAGIOS HOST: #{hostname[0]}")
end

I can’t recall but maybe there is a .name method on the object but you don’t want the name of the object I am assuming you want the node attribute inside the node object. It is a subtle difference. I tend to use chef-shell to help explore the object that contains node objects in a more ad-hoc manner when working out queries.

It is a fairly long gist as it includes many examples on various checks that can be setup in various fashions. You will find a recipe in there with the logic and any of the attribute files included should help you understand the data structure.

5

Interestingly your example does not work for me either.

servers = []
host_search = search(:node, "tags:auto_newton")
host_search.each do |nodeobj|
  servers << nodeobj.node.ipaddress
end

================================================================================
Recipe Compile Error in /tmp/kitchen/cache/cookbooks/acs_monitoring/recipes/default.rb
================================================================================
   
NoMethodError
-------------
Undefined method or attribute `ipaddress' on `node'

I was swamped for the last couple days, but let me look at the other stuff you posted and see if I can make sense of it.

Thanks!

6

I cannot believe how frustrating chef is. It turns out that this code just does not work in test-kitchen, but works fine on all the other machines.

nrpe_servers = search(:node, 'name:tempmachine2')
nrpe_servers.each do |nrpe_list|
    nrpe_list['monitoring']['nagios']['nrpe']['commands'].each do |nrpe_command|
        Chef::Log.warn("NRPE COMMAND: " + nrpe_command)
   end
end

I see the attribute fine in test/integration/nodes/tempmachine2.json, but for some reason test kitchen search does not return it.

But I KNOW that test kitchen IS looking at SOME of the data in these json files when I search… because I dumped 5 machine json files in there, and when I run this in test kitchen:

newton_servers = search(:node, 'tags:auto_newton')
newton_servers.each do |s|
  Chef::Log.warn("NEWTON SERVER: " + s.name)
end

It returns the list of the 5 machines that I have json files for.

Can someone please explain to me why test kitchen search cannot return some attributes listed in test/integration/nodes/*.json files?!?!?!?

7

I cannot believe how frustrating chef is. It turns out that this code just does not work in test-kitchen, but works fine on all the other machines.

Sorry you feel that way, chef is an excellent automation framework which has allowed businesses I have worked at deliver consistent and repeatable infrastructure automation for many years.

I see the attribute fine in test/integration/nodes/tempmachine2.json, but for some reason test kitchen search does not return it.

Can you share your .kitchen.yml? I know it does work in test-kitchen with the right setup.

For example in ./test/nodes/rabbitmq_server2.json looks like this:

{
  "name": "node2",
  "chef_type": "node",
  "json_class": "Chef::Node",
  "chef_environment": "kitchen",
  "automatic": {
    "ipaddress": "192.168.33.31",
    "hostname": "rabbitmq_server2",
    "fqdn": "rabbitmq_server2.kitchen"
  },
  "run_list": [
    "role[rabbitmq_server]"
  ]
}

In the .kitchen.yml

provisioner:
  name: dokken
  roles_path: test/roles
  nodes_path: test/nodes

in a recipe:

# Find peers to join cluster
rabbitmq_servers = []
host_search = search(:node, "chef_environment:#{node.chef_environment} AND role:rabbitmq_server")
host_search.sort.each do |srv| # there are peers and... maybe self
  # We have a problem comparing self's hostname when running this on Docker (testkitchen) as the
  # self's hostname returned from search is different from that of the final image.
  # Vagrant and AWS however work just fine. In our use case, hostname and ipaddress are equivalent
  # so we use ipaddress for now.
  next if srv.node.ipaddress == node.ipaddress
  rabbitmq_servers << {
    :name => "rmq-#{Digest::SHA256.hexdigest(srv.node.hostname)}@#{srv.node.hostname}",
    :type => 'disc'
  }
8

Given that the search works under non-kitchen circumstances my hunch is that the content of the JSON itself (i.e. does it have name set properly, the name of the file isn’t used) is where the disconnect happens. Is there any chance we could get a gist of that so we can replicate?

Kitchen is, by and large, just a harness - it doesn’t change the mechanics of how chef works as such but allows you to load data that leverages existing chef functionality. I’m pointing this out because test-kitchen itself shouldn’t change the core mechanics.

1 Like
9

Part of the problem here I think is that I have inherited a setup from someone else. My assumption was that it was all setup correctly, but maybe that is not the case…

Here is the provisioner section of the .kitchen.yml:

provisioner:
  name: chef_zero
  client_rb: 
    environment: test
  http_proxy: '<%= ENV['http_proxy'] %>'
  https_proxy: '<%= ENV['https_proxy'] %>'
  no_proxy: '<%= ENV['no_proxy'] %>'
  require_chef_omnibus: 12.16.42

The tempmachine2.json was generated by running
knife search node “name:tempmachine2” -Fj > tempmachine2.json , so I think it should be formed correctly (ie it is not a hand written json file)

{
  "results": 1,
  "rows": [
    {
  "name": "tempmachine2",
  "chef_environment": "prod",
  "json_class": "Chef::Node",
  "automatic": {
    "kernel": {
      "name": "Linux",

…

Though, I notice mine starts with:

{
  "results": 1,
  "rows": [
    {
  "name": "tempmachine2",

But since the search on “auto_newton” servers (in kitchen) works and returns the same list that has json files in:

> ls test/integration/nodes/
jumpnt1.json  oradevnt1.json  rbint1.json  tempmachine2.json  uniteapps1.json  unitedev1.json

It seems like the json files are being noticed/searched, otherwise it wouldn’t have returned the 6 servers, no? Thoughts?

Thanks!

10

Hi cheeseplus! I have opened case 18050 with Chef Support (we have a contract), so someone there is already looking into it I think. The ticket has the json file (28000+ lines) and code snippets. I'll also attach the .kitchen.yml too now that majormoses mentioned that is important.

Thanks!

11

Okay, I have seemed to narrow down the problem, though it does not make much sense to me.

When using this command to generate the json:

knife search node “name:xxx” -Fj > xxx.json

It appears to add extra stuff:

{
  "results": 1,
  "rows": [
    {

and

  ]
}

to the top and bottom of the JSON file. But oddly enough, this only causes some of the searches to fail.
Specifically

host_search = search(:node, "tags:xxx")

works fine. ugh.