Problem with remote bootstrap Chef server with policy file


I’m currently using Knife Zero to remotely bootstrap a Linux machine to be a Chef server with policy file by following this guide:

These are the list of commands I used:

# in cookbooks/chef-automate/policies directory
chef install chef-automate.rb --chef-license accept
chef export chef-automate.rb ./ -f

[in another terminal] sudo knife serve 
chef push -c knife.rb local chef-automate.rb

sudo knife zero bootstrap <HOST> --yes --policy-name chef-automate --policy-group local -N <NODENAME> -U adminuser -P psswd --sudo --use-sudo-password --secret-file /var/chef/encrypted_data_bag_secret

This is my file structure

This is my knife.rb located in ???

knife[:chef_repo_path] = "#{ENV['HOME']}/.chef"

current_dir = File.dirname(__FILE__)
use_policyfile true
versioned_cookbooks true
policy_document_native_api false
chef_server_url "http://localhost:8889"
cookbook_path ["#{current_dir}/../cookbooks"]

knife[:secret_file] = "/etc/chef/encrypted_data_bag_secret"

After running the knife zero bootstrap, it seems like the remote machine can run the cookbook through the policy file. However, it keeps failing when trying to load data bag with the below error.

#error message has been shorten to highlight key messages
ERROR: Failed to load data bag item: "tls_cert" "automate"
Recipe Compile Error in /var/chef/cache/cookbooks/chef-automate/recipes/default.rb
404 "Not Found "

#error line 
>> tls_cert_data = data_bag_item("tls_cert", automate)

The data_bag folder is at the same level as recipes/ policies/etc under cookbooks/chef-automate directory

What I have done so far?

1.I have checked the destination machine and can confirm that cookbook/recipes/databags/secret file all have been transferred to the destination machine
2.I can confirm that the cookbook as well as the recipe that has the function to retrieve data from data bag are working fine since I did test them in test kitchen or other places

What I suspect about the problem?

1.I think it might be because when executing knife zero and running chef zero remote in destination machine, I have missed out a configuration somewhere that it could not find the data bag, even thought it’s clearly locating beside the recipes folder, but I am not too sure since I still haven't figured out why it could not find the data bag or failed to load it.

I would appreciate if anyone could help me point out what I have missed to fix this. :slight_smile:

You might need to add cache folder inside your Chef dir while running Chef zero.

How can I add cache folder inside your Chef dir (in target machine or bootstraper) ? Do I need to reference it somewhere in knife.rb ?

Hmm I think after running the knife zero bootstrap, it did add cache folder in the target machine at /var/chef/cache/cookbooks