Undefined method error (testing chef recipe)

Hi:

Chef newbie here. I'm testing the vault cookbook (GitHub - bloomberg/vault-cluster-cookbook: Application cookbook which installs and configures Vault with Consul as a backend.). When I attempt to do a bootstrap with the recipe added to the run-list, I get the following error:

Recipe Compile Error in /var/chef/cache/cookbooks/vault-cluster/recipes/default.rb

NoMethodError

undefined method `bag_item' for VaultCookbook::Resource::VaultConfig

Cookbook Trace:

 /var/chef/cache/cookbooks/hashicorp-vault/recipes/default.rb:23:in `block (2 levels) in from_file'
 /var/chef/cache/cookbooks/hashicorp-vault/recipes/default.rb:23:in `each_pair'
 /var/chef/cache/cookbooks/hashicorp-vault/recipes/default.rb:23:in `block in from_file'
 /var/chef/cache/cookbooks/hashicorp-vault/recipes/default.rb:18:in `from_file'
 /var/chef/cache/cookbooks/compat_resource/files/lib/chef_compat/monkeypatches/chef/run_context.rb:347:in `load_recipe'
 /var/chef/cache/cookbooks/compat_resource/files/lib/chef_compat/monkeypatches/chef/run_context.rb:303:in `block in include_recipe'
 /var/chef/cache/cookbooks/compat_resource/files/lib/chef_compat/monkeypatches/chef/run_context.rb:302:in `each'
 /var/chef/cache/cookbooks/compat_resource/files/lib/chef_compat/monkeypatches/chef/run_context.rb:302:in `include_recipe'
 /var/chef/cache/cookbooks/vault-cluster/recipes/default.rb:35:in `from_file'
 /var/chef/cache/cookbooks/compat_resource/files/lib/chef_compat/monkeypatches/chef/run_context.rb:347:in `load_recipe'

Can someone tell me what the NoMethodError is about? I've been doing research in the forums. I know once possibility is the chef client version I'm using, but I don't think that's the problem (using chef version 12.15.19).

Thanks,

jon

Which version of the hashicorp-vault cookbook are you using? It seems that in older versions of this cookbook they had this bag_item attribute which was removed in newer ones.

Could you post some more of your setup and code? The info you gave is a good starter but not enough to really debug the problem.

Ok. Thanks for the help! I'm not sure where to start since I'm new to all of this. An older cookbook was listed in my metadata.rb file, so I adjusted it to hashicorp-vault 2.4.0. Here is how my metadata.rb file looks:

name 'vault-cluster'
maintainer 'John Bellone'
maintainer_email 'jbellone@bloomberg.net'
source_url 'GitHub - bloomberg/vault-cluster-cookbook: Application cookbook which installs and configures Vault with Consul as a backend.'
issues_url 'Issues · bloomberg/vault-cluster-cookbook · GitHub'
license 'Apache 2.0'
description 'Wrapper cookbook which installs and configures Vault with Consul as its backend.'
long_description 'Wrapper cookbook which installs and configures Vault with Consul as its backend.'
version '2.0.0'

supports 'ubuntu', '>= 12.04'
supports 'redhat', '>= 6.6'
supports 'centos', '>= 6.6'

depends 'ssl_certificate', '~> 1.11'
depends 'consul-cluster', '~> 2.0'
depends 'hashicorp-vault', '~> 2.4.0'
~

Does this help?

I guess the version of vault it’s installing is to new? In this case, what do developers/admins usually do? Should I specify a version of vault to install? Or does the cookbook need to be rewritten to ignore the bag_item attribute?

Hmm, I’ve never used any of those cookbooks but my guess is that since the hashicorp-vault cookbook was rewritten the vault-cluser cookbook is broken. In the vault-cluser cookbook the version pin of hashicorp vault probably should be ‘~> 2.1.0’ instead of ‘~> 2.1’.
So I guess you are left with two choices:

  • Either pin hashicorp-vault to a specific version that works (probably 2.1.x).
  • Fork the vault-cluster cookbook and fix it…

If you are testing with kitchen then the version pin should be the faster way to get everything running and see if this cookbook actually does what you want/need.
In the long run fixing the vault-cluster cookbook and sending a pull request to the author might be the better option.

Thank you very much? I'm getting a better idea of how this works. Making more progress now. I changed the version to 2.1.0 for now and it's gotten me farther, but I now have the following error:

Error executing action create on resource 'ssl_certificate[consul]'

RuntimeError

Cannot read SSL key from chef-vault: secrets.consul[private_key]

I have a test chef server. I suppose this means that I need to set up an ssl key in chef-vault? I've never done this before and I can start looking into it, but I just wanted to make sure I'm going in the right direction.

Thoughts?

The ssl_certificate cookbook will fall back to reading an unencrypted DataBag if no Vault exists. So for testing you can just provide a DataBag in kitchen.

Thanks. Sorry for all the questions…

When you say … if no vault exists… are you talking about ‘Vault’ or chef-vault? I’m not sure how you came to that conclusion. Sorry, you’re dealing with a newbie here:)

Oh, sorry. I ment chef-vault. Just look at the few lines of code here: https://github.com/chef-cookbooks/chef-vault/blob/master/libraries/helpers.rb#L37

So basically if you have the chef-vault cookbook somewhere in your run_list or as a dependency of a cookbook in your run_list you do have access to this definition chef_vault_item. So if in any cookbook you call chef_vault_item(my_data_bag, my_item) it will try to load a chef-vault and if that Bag/Item combination is not a chef-vault but a simple unencrypted DataBag it will load that one. This is great for kitchen testing, because there you can simply provide unencrypted DataBags as test data.

Thanks - this is super helpful!

Okay, I think I'm making some progress. I'm trying to do bootstrap this cookbook now with a server. The consul cluster cookbook documentation says the following:

Chef Vault and TLS

By default, the Consul Cluster cookbook uses Chef Vault to encrypt the TLS certificates and keys that are deployed to the nodes. This data should only be capable of being decrypted by the nodes the Consul Cluster itself.

{
"ca_certificate": "-----BEGIN CERTIFICATE-----\nMIIE [...]",
"certificate": "-----BEGIN CERTIFICATE-----\nMIIE [...]",
"private_key": "-----BEGIN RSA PRIVATE KEY-----\nMIIE [...]"
}

So I went ahead and created a certinfo.json file and included the key info as listed above and ran the following command:

knife vault create secrets consul -A "username, nodename" -M client -J ./certinfo.json

I believe I created the encrypted databag successfully. However, when bootstrapping, I still see the following errors:

Error executing action create on resource 'ssl_certificate[consul]'
.
.
.

secrets/consul is encrypted for you, but your private key failed to decrypt the contents. (if you regenerated your client key, have an administrator of the vault run 'knife vault refresh')
.
.
.
Cannot read SSL key from chef-vault: secrets.consul[private_key]

I noticed you mentioned earlier that the ssl_certificate cookbook would fall back to reading an unencrypted DataBag if no Vault exists. I still get the error executing action create on resource 'ssl_certificate[consul]' message. Perhaps I did not configure the databag properly?

Hi,

For Vaults please read the manual carefully. Encryptions always is an overhead but it is necessary and it never is easy.
Have a look at the DataBag that was created by Vault, especially the <data_bag_item_name>_keys item. This _keys contains all users/admins and nodes that will have access to the encrypted data. I think you will see that your node does not have access. This is because you have to specify a --search when creating a vault and the return of that search will be the nodes that will get access to the vault.