I’m following along the chef-delivery tutorial and I am stuck on brining up the machine during the provisioning phase. (its about halfway down the page here: https://learn.chef.io/build-a-delivery-pipeline/aws/write-the-build-cookbook/)
The specific error I’m getting through the chef-delivery gui shows:
`
Error executing action 'setup' on resource 'machine[acceptance-deliver-customers-rhel]'
================================================================================
Chef::Exceptions::InsufficientPermissions
-----------------------------------------
aws_key_pair[chef_default] (basic_chef_client::block line 1126) had an error: Chef::Exceptions::InsufficientPermissions: directory[/var/opt/delivery/workspace/172.29.1.50/test/Development/deliver-customers-rhel/master/acceptance/provision/chef/keys] (basic_chef_client::block line 30) had an error: Chef::Exceptions::InsufficientPermissions: Cannot create directory[/var/opt/delivery/workspace/172.29.1.50/test/Development/deliver-customers-rhel/master/acceptance/provision/chef/keys] at /var/opt/delivery/workspace/172.29.1.50/test/Development/deliver-customers-rhel/master/acceptance/provision/chef/keys due to insufficient permissions
Resource Declaration:
---------------------
# In /var/opt/delivery/workspace/172.29.1.50/test/Development/deliver-customers-rhel/master/acceptance/provision/cache/cookbooks/build-cookbook/recipes/provision.rb
57: machine "#{stage}-#{project}" do
58: action [:setup]
59: chef_environment delivery_environment
60: converge false
61: files '/etc/chef/encrypted_data_bag_secret' => File.join(database_passwords_key_path, 'database_passwords_key')
62: run_list node[project]['run_list']
63: machine_options Cheffish::MergedConfig.new({"bootstrap_options" => {"key_name" => ssh_key['name'], "key_path" => ssh_private_key_path}}, node[project][stage]['aws']['config']['machine_options'])
64: end
`
What appears to be the issue is that the build node is trying to write to the directory (/var/opt/delivery/workspace/172.29.1.50/test/Development/deliver-customers-rhel/master/acceptance/provision/chef/keys) but can’t for some reason. I don’t understand why not because it created the directory and subdirectories. Also it’s happening on the machine resource which somewhat baffles me. The stack trace is at the bottom of the document.
So I guess my question is how would I further troubleshoot the issue to resolve? Is it possible to run the chef-client run on the build node? What exactly is going on - is the chef-client run on the build node trying to download the database key and attempting to save it to a directory its creating and NOT have permission to do that? If so what’s a good way to troubleshoot? If not please explain.
I’ve tried removing the files attribute from the machine but the error still persisted. I’ve also tried deleting the build node’s provision directory - (/var/opt/delivery/workspace/172.29.1.50/test/Development/deliver-customers-rhel/master/acceptance/provision) no luck. What could I try next?
stack trace:
root@build-node1:~# cat /var/opt/delivery/workspace/172.29.1.50/test/Development/deliver-customers-rhel/master/acceptance/provision/cache/chef-stacktrace.out
Generated at 2016-01-15 22:28:13 +0000
Chef::Exceptions::InsufficientPermissions: machine[acceptance-deliver-customers-rhel] (build-cookbook::provision line 107) had an error: Chef::Exceptions::InsufficientPermissions: aws_key_pair[chef_default] (basic_chef_client::block line 1126) had an error: Chef::Exceptions::InsufficientPermissions: directory[/var/opt/delivery/workspace/172.29.1.50/test/Development/deliver-customers-rhel/master/acceptance/provision/chef/keys] (basic_chef_client::block line 30) had an error: Chef::Exceptions::InsufficientPermissions: Cannot create directory[/var/opt/delivery/workspace/172.29.1.50/test/Development/deliver-customers-rhel/master/acceptance/provision/chef/keys] at /var/opt/delivery/workspace/172.29.1.50/test/Development/deliver-customers-rhel/master/acceptance/provision/chef/keys due to insufficient permissions
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/mixin/why_run.rb:241:in run' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/mixin/why_run.rb:322:inblock in run’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/mixin/why_run.rb:321:in each' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/mixin/why_run.rb:321:inrun’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/provider.rb:154:in process_resource_requirements' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/provider.rb:132:inrun_action’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource.rb:585:in run_action' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/runner.rb:49:inrun_action’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/runner.rb:81:in block (2 levels) in converge' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/runner.rb:81:ineach’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/runner.rb:81:in block in converge' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/resource_list.rb:83:inblock in execute_each_resource’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/stepable_iterator.rb:116:in call' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/stepable_iterator.rb:116:incall_iterator_block’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/stepable_iterator.rb:85:in step' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/stepable_iterator.rb:104:initerate’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/stepable_iterator.rb:55:in each_with_index' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/resource_list.rb:81:inexecute_each_resource’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/runner.rb:80:in converge' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/cheffish-1.6.0/lib/cheffish/basic_chef_client.rb:69:inblock in converge’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/cheffish-1.6.0/lib/cheffish/basic_chef_client.rb:133:in call' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/cheffish-1.6.0/lib/cheffish/basic_chef_client.rb:133:inwith_chef_config’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/cheffish-1.6.0/lib/cheffish/basic_chef_client.rb:68:in converge' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/cheffish-1.6.0/lib/cheffish/basic_chef_client.rb:102:ininline_resource’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/cheffish-1.6.0/lib/cheffish.rb:5:in inline_resource' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-provisioning-aws-1.6.1/lib/chef/provider/aws_key_pair.rb:29:increate_key’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-provisioning-aws-1.6.1/lib/chef/provider/aws_key_pair.rb:10:in block in <class:AwsKeyPair>' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/provider.rb:362:inaction_create’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/provider.rb:144:in run_action' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource.rb:585:inrun_action’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/runner.rb:49:in run_action' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/runner.rb:81:inblock (2 levels) in converge’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/runner.rb:81:in each' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/runner.rb:81:inblock in converge’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/resource_list.rb:83:in block in execute_each_resource' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/stepable_iterator.rb:116:incall’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/stepable_iterator.rb:116:in call_iterator_block' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/stepable_iterator.rb:85:instep’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/stepable_iterator.rb:104:in iterate' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/stepable_iterator.rb:55:ineach_with_index’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/resource_list.rb:81:in execute_each_resource' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/runner.rb:80:inconverge’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/cheffish-1.6.0/lib/cheffish/basic_chef_client.rb:69:in block in converge' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/cheffish-1.6.0/lib/cheffish/basic_chef_client.rb:133:incall’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/cheffish-1.6.0/lib/cheffish/basic_chef_client.rb:133:in with_chef_config' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/cheffish-1.6.0/lib/cheffish/basic_chef_client.rb:68:inconverge’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/cheffish-1.6.0/lib/cheffish/basic_chef_client.rb:109:in converge_block' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-provisioning-1.5.0/lib/chef/provisioning.rb:12:ininline_resource’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-provisioning-aws-1.6.1/lib/chef/provisioning/aws_driver/driver.rb:1125:in block in default_aws_keypair' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-provisioning-aws-1.6.1/lib/chef/provisioning/aws_driver/driver.rb:1124:insynchronize’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-provisioning-aws-1.6.1/lib/chef/provisioning/aws_driver/driver.rb:1124:in default_aws_keypair' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-provisioning-aws-1.6.1/lib/chef/provisioning/aws_driver/driver.rb:755:inbootstrap_options_for’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-provisioning-aws-1.6.1/lib/chef/provisioning/aws_driver/driver.rb:551:in allocate_machine' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-provisioning-1.5.0/lib/chef/provider/machine.rb:31:inblock in class:Machine’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/provider.rb:362:in action_allocate' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-provisioning-1.5.0/lib/chef/provider/machine.rb:38:inblock in class:Machine’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/provider.rb:362:in action_ready' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-provisioning-1.5.0/lib/chef/provider/machine.rb:45:inblock in class:Machine’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/provider.rb:362:in action_setup' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/provider.rb:144:inrun_action’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource.rb:585:in run_action' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/runner.rb:49:inrun_action’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/runner.rb:81:in block (2 levels) in converge' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/runner.rb:81:ineach’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/runner.rb:81:in block in converge' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/resource_list.rb:83:inblock in execute_each_resource’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/stepable_iterator.rb:116:in call' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/stepable_iterator.rb:116:incall_iterator_block’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/stepable_iterator.rb:85:in step' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/stepable_iterator.rb:104:initerate’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/stepable_iterator.rb:55:in each_with_index' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/resource_collection/resource_list.rb:81:inexecute_each_resource’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/runner.rb:80:in converge' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/client.rb:653:inblock in converge’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/client.rb:648:in catch' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/client.rb:648:inconverge’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/client.rb:687:in converge_and_save' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/client.rb:269:inrun’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/application.rb:270:in block in fork_chef_client' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/application.rb:258:infork’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/application.rb:258:in fork_chef_client' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/application.rb:224:inblock in run_chef_client’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/local_mode.rb:44:in with_server_connectivity' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/application.rb:212:inrun_chef_client’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/application/client.rb:408:in block in interval_run_chef_client' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/application/client.rb:398:inloop’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/application/client.rb:398:in interval_run_chef_client' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/application/client.rb:388:inrun_application’
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/application.rb:60:in run' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/bin/chef-client:26:in<top (required)>’
/opt/chefdk/bin/chef-client:54:in load' /opt/chefdk/bin/chef-client:54:in’