Classic convergence/compile problems?

Chef Infra (archive)
1

Hi all and thanks in advance for any help you can provide.

I’ve got an encrypted data bag storing sensitive data (passwords).

I’ve got a recipe which contains a cookbook_file resource which copies myFile.pem to my node, which is the --secret-file used to decrypt the aforementioned data bag - when I add only this cookbook to a node’s run_list, it works perfectly.

I’ve got additional recipes which need to decrypt our encrypted data bag in order to get a PW to use in conjunction with a remote_file resource block.

For those recipes which require the encrypted data bag secret file myFile.pem, I’ve added “include_recipe AddMyFilePem::default” in the recipe, as well as add “depends ‘AddMyFilePem’” to the cookbook’s metadata.rb … and although I see the AddMyFilePem cookbook added to my node’s run list, the Chef client never enters the execute phase because the convergence (or is it compile?) phase fails on this line:

encrypted_vars = data_bag_item(‘encryptedDataBag’, ‘encryptedVars’, IO.read(EDBS))

where the file represented by the EDBS variable is MyFile.pem …

My expectation was that by including the recipe, it would be run before attempting to set the encrypted_vars variable … but that does not seem to be the case.

Is there a way to force the execution of a recipe being included during the convergence/compile phase? I found this link https://docs.chef.io/resource_common.html#run-in-compile-phase - so I tried wrapping my include recipe in a Ruby block like this:

ruby_block “include_AddMyFilePem” do
block do
run_context.include_recipe "AddMyFilePem::default"
end
end

but this also failed to resolve my problem.

I’ve tried making this file’s creation a custom resource, but run into the now all-too-familiar error:

Recipe Compile Error in C:/chef/cache/cookbooks/DownloadBldTools/recipes/default.rb

Code snippets available if that will help. Thanks.

2

Hello @NoobSkywalker

The compile phase is when Chef will load your cookbooks into memory. What is actually happening is Ruby is interpreting the code and executing it. Therefore any Ruby code is executed in this phase. Your Chef DSL resources actually get run by Ruby in this phase too

directory 'c:\temp\stuff' do
  recursive true
end

The code is saying call a method called directory and pass into it a code block (which is evaluated in the compile phase). The resources get placed into a resource queue.

The converge phase happens after compile and will go through the resource queue and run the actions on the resources.

Your issue does sound like it could be compile/converge. You won’t be able to run a recipe or cookbook in compile phase but you can run a resource in compile phase, two easiest ways is to use at_compile_time in the Chef Sugar cookbook/gem https://github.com/sethvargo/chef-sugar#filters

Alternatively set the resource action to nothing in the code block and after declaring the resource call the run_action method

directory 'c:\temp\stuff' do
  recursive true
  action :nothing
end.run_action(:create)

Or obtain a reference to the resource and call run action

d = directory 'c:\temp\stuff' do
  action :nothing
end
d.recursive = true
d.run_action(:create)

Hope that helps

3

Thanks Chris, but I don’t think the line of code where I’m tripping up is a Chef DSL resource. Let me try to clarify. I have a cookbook named AddMyFilePem, which makes a call to a cookbook_file resource & copies foobar.pem from the cookbook to /etc/chef/foobar.pem.

I then have a second cookbook, DownloadSomething, where I’m including the AddMyFilePem recipe, expecting it to be run & have foobar.pem copied to /etc/chef.

After including the recipe to copy foobar.pem from my cookbook, I then use foobar.pem to decrypt an encrypted data bag, and attempt to set an array, as such:

include_recipe "AddMyFilePem"
encrypted_vars = data_bag_item('encryptedDataBag', 'encryptedVars', IO.read('/etc/chef/foobar.pem'))

however, when executing DownloadSomething, it fails during the compile phase when attempting to define the encrypted_vars array due to /etc/chef/foobar.pem not yet existing.

How can I tell Chef … “hey, this is fine … foobar.pem will exist eventually … trust me.” ?

Appreciate the help.

-NS

4

Hi @NoobSkywalker

The issue is that when your encrypted_vars code runs your resource will not have downloaded the file you require.

In your AddMyFilePem you need to co-erce the resource to run at compile time, i.e. when Chef loads the recipes.

All parts of your recipe are executed at compile time but the actions of the Chef DSL resources are actioned in Converge phase.

What is in your AddMyFilePem recipe? Have you made the cookbook_file action of :nothing and then force the resource to be run at compile time with either of the techniques above?

1 Like
5

AGHHHHH … I was looking at the include_recipe line as the location where a fix was required, but as you pointed out, the fix needed to be within the recipe I was attempting to include … using the technique you described in your first reply:

directory 'c:\temp\stuff' do
recursive true
action :nothing
end.run_action(:create)

worked perfectly. Thank you so much Chris … if you’re on the east coast I’d love to buy you a beer.

6

Alternatively you could look at forcing the databag encryption key to be lazily loaded, that might also work.

Nathan Clemons
DevOps Engineer
Moxie Cloud Services (MCS)

O +1.425.467.5075
M +1.360.861.6291
E nclemons@gomoxie.com
W www.gomoxie.comhttp://www.gomoxie.com/

7

Hi @nclemons,

Can you provide an example of a lazy load using data from a data bag please.

Would be useful to myself and others in the community.

Thanks