Hi Chef experts:
I'm kind of stumped on this one... I am quite aware of the concept between the Chef run stage and compile stage but I still cannot seem to get what I need to accomplish figured out.
I think the solution listed in How to store array in node object chef? may have helped in a normal recipe but I am trying to create a custom resource that basically reads a databag that has the following structure:
{
"id": "local_files",
"development":
[
{
"source": "https://sourcecode.mydomain.com/projects/ASX-NBJG/repos/sapops/raw/config/lama/Z_LVM_CheckBackup.conf?at=refs%2Fheads%2Ffeature%2FADVR-575-as-a-basis-
user-i-want-to-be-able-to-use-chef-to-copy-files-locally-to-servers",
"target": "/tmp/some_folder/Z_LVM_CheckBackup.conf",
"owner": "root",
"group": "root",
"mode": "0777",
"action": "create",
"run_command" : "echo 'hello my beautiful world' > /tmp/helloworld.txt"
},
{
"source": "https://sourcecode.mydomain.com/projects/ABCD/repos/build/raw/copylocal/sudoerspolicy?at=refs%2Fheads%2Ffeature%2FADVR575CopyLocal",
"target": "/etc/sudoers.d/mysudoers",
"owner": "root",
"group": "root",
"mode": "0440",
"action": "create",
"run_command" : "echo \"sudoers file updated on `date`\" |tee -a /tmp/sudoers_updated.log"
},
{
"source": "file:///erpsoftware/auto/copylocal/testtar.tar",
"target": "/tmp/testtar.tar",
"owner": "root",
"group": "root",
"mode": "0440",
"action": "create",
"run_command" : "mkdir -p /tmp/letsSeeHowItGoes && tar -xvf /tmp/testtar.tar -C /tmp/letsSeeHowItGoes"
},
],
"qa":
[
{
"source": "https://sourcecode.mydomain.com/projects/ASX/repos/ops/raw/config/stuff/Z_LVM_CheckBackup.conf?at=refs%2Fheads%2Ffeature%2FADVR-575-as-a-basis-
user-i-want-to-be-able-to-use-chef-to-copy-files-locally-to-servers",
"target": "/tmp/some_folder/Z_LVM_CheckBackup.conf",
"owner": "root",
"group": "root",
"mode": "0777",
"action": "create",
"run_command" : "echo 'hello my beautiful world' > /tmp/helloworld.txt"
}
]
The resource call:
copy_local 'Ensure all files that need to be copied locally are handled...' do
databag_itemid node['srv']['copylocal_databag_id']
action :from_databag
end
The problem I have is, even if I delete the first file, the "run_command" that my code chooses is always the last item in the array of hashes.
I have a simple loop like this:
resource_name :copy_local
property :databag_itemid, String
property :databag_auth_required, [true, false], default: true
property :databag_name, String, default: node['srv']['databag_name']
property :databag_env, String, default: node['scm_appbranch'].downcase
property :secret_databag_itemid, String, default: 'sa'
property :secret_keysrc, String, default: node['srv']['sa_sec_key_src']
property :secret_key, String, default: node['srv']['sa_secret_key']
property :service_acctname, String, default: 'service_account'
property :service_acctpwname, String, default: 'service_account_pw'
property :files, Hash
property :files_env, String, default: node['scm_appbranch'].downcase
property :debug, [true, false], default: false
action :from_databag do
# load variables
skey = new_resource.secret_key
skeysrc = new_resource.secret_keysrc
sdbid = new_resource.secret_databag_itemid
dbenv = new_resource.databag_env
dbid = new_resource.databag_itemid
dbname = new_resource.databag_name
dbauthreq = new_resource.databag_auth_required
saname = new_resource.service_acctname
sapwname = new_resource.service_acctpwname
dbg = new_resource.debug
# DL the secret
remote_file skey do
source skeysrc
action :nothing
sensitive true
end.run_action(:create)
# Load the secret and try to decrypt
secret = Chef::EncryptedDataBagItem.load_secret(skey)
begin
credentials = data_bag_item(dbname, sdbid, secret)
user = credentials[saname]
password = credentials[sapwname]
rescue StandardError => msg
puts 'ERROR :: Could not get credentials from encrypted databag!!!'
raise msg
end
# load the data bag item for copy local functionality
all_local_files = data_bag_item(dbname, dbid)
# load the hash
my_files = missing?(all_local_files[dbenv]) ? all_local_files : all_local_files[dbenv]
# now loop through files and begin local copy via the remote_file resource
# This loop does not work, need to find a way to loop through and not only get the last item of the array
auth = "Basic #{Base64.encode64("#{user}:#{password}")}"
my_files.each do |file_obj|
Array.wrap(file_obj).each do |file|
# check all flavors for each file object
checks = check_all_flavors(file)
unless checks.has_value?(false)
# Debug
puts " :: D E B U G :: ==> file source : #{file['source']}" if dbg
puts " :: D E B U G :: ==> file target : #{file['target']}" if dbg
puts " :: D E B U G :: ==> file mode : #{file['mode']}" if dbg
puts " :: D E B U G :: ==> file owner : #{file['owner']}" if dbg
puts " :: D E B U G :: ==> file group : #{file['group']}" if dbg
puts " :: D E B U G :: ==> file action : #{file['action']}" if dbg
puts " :: D E B U G :: ==> file run_command: #{file['run_command']}" if dbg
# Create the directory for the parent folder of the file['target'] if it doesn't exist
directory dir_name(file['target']) do
recursive true
mode file['mode']
owner file['owner']
group file['group']
action :create
not_if { dir_exists?(dir_name(file['target'])) }
end
# use remote_file resource to copy the file locally
remote_file file['target'] do
source file['source']
mode file['mode']
owner file['owner']
group file['group']
headers('Authorization' => auth) if dbauthreq
action file['action']
notifies :run, 'execute[run-command-for-copy-local-databag]', :immediately unless missing?(file['run_command'])
end
# resource to execute any command specified in copy local attributes
# allows us to "copy local and execute a command"
execute 'run-command-for-copy-local-databag' do
command file['run_command']
environment file['run_env']
creates file['run_creates']
cwd file['run_cwd']
group file['run_group']
user file['run_user']
action :nothing
end
end
end
end
# # delete secret file once data bag is decrypted successfully
# file skey do
# action :delete
# end
end
But as I mentioned, instead of the affiliated "run_command" executing, it is always the last "run_command" in the list. To me, this seems like a pretty straight forward loop, and things are working for owner, group and permissions, but my "run_command" execute resource block seems to always be picking up the last item in the databag. Meaning, even if I purposely delete the /tmp/some_folder/Z_LVM_CheckBackup.conf file, when it should be triggering the affiliated echo "hello my beautiful world" command, it is running the last command in the list for the rum_command key: "run_command" : "mkdir -p /tmp/letsSeeHowItGoes && tar -xvf /tmp/testtar.tar -C /tmp/letsSeeHowItGoes"
If I test this on a fresh node, the array of hash elements is processed in order and the files are created and the correct commands I expect run. The problem only arises when I test a deletion of one or more files, then I see that chef-client is taking the last run_command element in my databag object.
I tried to implement the strategy laid out in How to store array in node object chef? with using the node.run_state but this didn't get me any luck either. Pretty stumped at this point and would greatly appreciate any help.