How to prevent chef from replacing my files

boss6of9 · October 4, 2016, 7:19pm

Hi,

I have a chef recipe that downloads a tar file, that then gets extracted. As part of that recipe, I also update a config file, based upon certain settings that I determine. My problem is that after the file gets updated, chef puts it back to it’s original default setting. How can I prevent this from happening?

phanikumar · October 4, 2016, 7:29pm

I think you are using the execute resource to extract file my suggestion is to check whether the file exists or not if the file exist exclude the file with -X in tar by specifying the path or else extract entire tar file.

boss6of9 · October 4, 2016, 7:31pm

Hi,

I am using the execute resources. I extract the entire tar file (it’s only 2 directories and 3 files). That part works fine - the problem is that chef replaces my updates.

kawolfe · October 4, 2016, 7:37pm

As phanikumar mentioned, it sounds like chef is extracting the tar every run. You can avoid this by checking if the file exists before executing the tar extraction. Try adding something like not_if { File.exist?('path_to_your_file') } into your execute block.

phanikumar · October 4, 2016, 7:40pm

Can you provide your execute resource code here once. I do suggest implement the code in this way.

execute ‘extract’ do
File.exist?(‘path_to_file’)
command "tar -xvf java-1.42.tar.gz --exclude=‘java/LICENCE’"
else
command "tar -xvf java-1.42.tar.gz"
end
cwd "path_to_tar_file"
end

Hope the above code will help you.

boss6of9 · October 4, 2016, 8:05pm

Thanks. I’d agree, but I have some code at the beginning of my recipe that should prevent it from even running:

f File.exists?("/var/chef/eisplunk_full_install_completed_do_not_delete")
puts “### Splunk chef recipe has already run on this client… exiting…###”.upcase
else
#log “=====Log Set=========” do

phanikumar · October 4, 2016, 8:46pm

From above you are trying to send the stdout some info that client-run has already performed. But how you are avoiding the extracting. Have you implemented the code that I provided you before and what was the result. It would be nice if you could paste the code here so that I can have a look at it and help you out on that.

boss6of9 · October 4, 2016, 8:51pm

It’s a lot of code… under that else is the logic for the entire recipe. I’m somewhat of a newbie, but my understanding is that if will prevent the rest of the code from even running, so the tar should never execute a second time.

At the end of the recipe, I create the tag file, so if chef-client tries to run again, it will stop at the opening if statement.

Create Tag file

file “/var/chef/eisplunk_full_install_completed_do_not_delete” do
action :create
end

phanikumar · October 4, 2016, 9:01pm

So you have two parts in which in the first statement you are verifying whether the file exists or not. If the file exist then you want to avoid the run by only executing if statement or else the else condition will execute and the entire recipe will run right. So what exactly is the condition in if statement. I am assuming the code would be like this

if File.exist?(‘path_to_file’)
puts "chef-recipe …"
else
entire recipe code
end

Am I correct?

boss6of9 · October 4, 2016, 9:05pm

Yes, exactly. The if statement is at the beginning:

chefk to see if splunk already installed

if File.exists?("/var/chef/eisplunk_full_install_completed_do_not_delete")
puts “### Splunk chef recipe has already run on this client… exiting…###”.upcase
else
#log “=====Log Set=========” do

rest of code…

At the end of the recipe, I create the file that is referenced:

Create Tag file

file “/var/chef/eisplunk_full_install_completed_do_not_delete” do
action :create
end

So, in theory, the recipe should only “run” once. The tar logic is part of the code in the middle.

phanikumar · October 4, 2016, 9:10pm

Are you getting ### Splunk chef recipe has already run on this client… exiting…### on your console. If I am not wrong the if condition is getting failed and the path of the file should have cache/cookbooks it seems if you want to cross-check with cookbooks path.

boss6of9 · October 7, 2016, 2:25am

OK. Did a re-write, but the same thing is happening. Part of my code updates a config file (deploymentclient.conf), and it works, but less than 5 minutes later, that file has been replaced with the original.

Cookbook Name:: eisplunk_full_install

Recipe:: default

Copyright 2016, YOUR_COMPANY_NAME

All rights reserved - Do Not Redistribute

installer = splunk binary

configFile = hfw/shc/idx config

prodType = deployment file

output_config = outputs file

unless File.exist?("/etc/.eisplunk_full_install_completed_do_not_delete")

Chef::Log.info("#{cookbook_name} – Starting ===============")

Chef::Log.info(“Environment ==> #{node[‘c2c_vm_environment’]}”)

#Define Variables

set user to splunk

splunk_user = node[‘c2c’][‘recipe_args’][‘eisplunk_full_install’][‘splunk_user’]

set tar file for splunk binary

installer = node[‘c2c’][‘recipe_args’][‘eisplunk_full_install’][‘installer’]

set to /apps/splunk/etc/apps directory for download location

tmp_dir = node[:c2c][:recipe_args][:eisplunk_full_install][:tmp_dir]

set to /tmp directory for full download

tmp = node[:c2c][:recipe_args][:eisplunk_full_install][:tmp]

set to mymonitoring website and path for downloads - managed by APM

repo_url = node[‘c2c’][‘recipe_args’][‘eisplunk_full_install’][‘repo_url’]

set to local hostname

HOSTNAME = node[‘hostname’]

set to c2c environment

myenv = node[‘c2c_vm_environment’]

set to requestor corpid

requestor = node[‘vm_creator_corpid’]

set to “none” but passed through gui/rest call - hfw/sh/idx

type = node[‘c2c’][‘recipe_args’][‘eisplunk_full_install’][‘type’]

@@devEnvir = false
aliasFile = “”

#dev by default
enviroPathType = “dev”

set config file to grab default - this may contain settings specific to the config type - hfw/sh/idx

configFile = node[‘c2c’][‘recipe_args’][‘eisplunk_full_install’][‘dev_hfw’]

case “#{node[‘c2c_vm_environment’]}”

when “Development”
@@devEnvir = true

Chef::Log.info("Recipe: eisplunk_full_install Step:Environment -->Â  #{node['c2c_vm_environment']} ")
Chef::Log.info("Recipe: eisplunk_full_install Step:Type of Install -->Â  #{type} ")
 
 ## set dev deployment tar file
 prodType = node[:c2c][:recipe_args][:eisplunk_full_install][:config_dev]
 
  ## set dev alias tar file
 aliasFile = node[:c2c][:recipe_args][:eisplunk_full_install][:a_linux_dev] 
  
  ## set location of dev deployment.conf file, which will get updated
 filepath =  "/apps/splunk/etc/apps/fmrei_all_deploymentclient_dev/local/deploymentclient.conf"
enviroPathType = "dev"

if type == "idx"
        configFile = node['c2c']['recipe_args']['eisplunk_full_install']['dev_idx']

end
 if type == "sh"
        configFile = node['c2c']['recipe_args']['eisplunk_full_install']['dev_sh']
end
if type == "hfw"
         ## set dev config file to grab default - this may contain settings specific to the config type - hfw/sh/idx
        configFile = node['c2c']['recipe_args']['eisplunk_full_install']['dev_hfw']
end

print(" “)
print(type)
print(” ")
print(configFile)

when “Production”
@@devEnvir = false

Chef::Log.info("Recipe: eisplunk_full_install Step:  Environment -->Â  #{node['c2c_vm_environment']} ")

Chef::Log.info("Recipe: eisplunk_full_install Step:Type of Install -->Â #{type} ")

set prod deployment tar file

prodType = node[:c2c][:recipe_args][:eisplunk_full_install][:config_prod]

set prod alias tar file

aliasFile = node[:c2c][:recipe_args][:eisplunk_full_install][:a_linux_prod]

set path to prod deploymentclient file, which will get updated

filepath = "/apps/splunk/etc/apps/fmrei_all_deploymentclient/local/deploymentclient.conf"
enviroPathType = “prod”

if type == "idx"
        configFile = node['c2c']['recipe_args']['eisplunk_full_install']['prod_idx']

end
 if type == "sh"
        configFile = node['c2c']['recipe_args']['eisplunk_full_install']['prod_sh']
end
if type == "hfw"
        configFile = node['c2c']['recipe_args']['eisplunk_full_install']['prod_hfw']
end

print(" “)
print(type)
print(” ")
print(configFile)

when “Test”
@@devEnvir = true

Chef::Log.info("Recipe: eisplunk_full_install Step:  Environment -->Â  #{node['c2c_vm_environment']} ")
Chef::Log.info("Recipe: eisplunk_full_install Step:Type of Install -->Â  #{type} ")

prodType = node[:c2c][:recipe_args][:eisplunk_full_install][:config_dev]
aliasFile = node[:c2c][:recipe_args][:eisplunk_full_install][:a_linux_dev]
filepath = "/apps/splunk/etc/apps/fmrei_all_deploymentclient_dev/local/deploymentclient.conf"
enviroPathType = “dev”

if type == "idx"
     configFile = node['c2c']['recipe_args']['eisplunk_full_install']['dev_idx']

end
 if type == "sh"
        configFile = node['c2c']['recipe_args']['eisplunk_full_install']['dev_sh']
end
if type == "hfw"
        configFile = node['c2c']['recipe_args']['eisplunk_full_install']['dev_hfw']
end

print(" ")
print(type)
print(" ")
print(configFile)

when “Chef Development”
@@devEnvir = true
Chef::Log.info("Recipe: eisplunk_full_install Step: Environment -->Â #{node[‘c2c_vm_environment’]} ")
Chef::Log.info("Recipe: eisplunk_full_install Step:Type of Install -->Â #{type} ")

 prodType = node[:c2c][:recipe_args][:eisplunk_full_install][:config_dev]
   aliasFile = node[:c2c][:recipe_args][:eisplunk_full_install][:a_linux_dev] 
   filepath =  "/apps/splunk/etc/apps/fmrei_all_deploymentclient_dev/local/deploymentclient.conf"
   
enviroPathType = "dev"
if type == "idx"
  configFile = node['c2c']['recipe_args']['eisplunk_full_install']['dev_idx']

end
 if type == "sh"
        configFile = node['c2c']['recipe_args']['eisplunk_full_install']['dev_sh']
end
if type == "hfw"
        configFile = node['c2c']['recipe_args']['eisplunk_full_install']['dev_hfw']
end

print(" “)
print(type)
print(” ")
print(configFile)

else

end

Determine the location

if node[:fqdn].include? 'rtp’
location = "RTP"
Message = ""
if @@devEnvir == true
## dev rtp outputs
output_config = node[:c2c][:recipe_args][:eisplunk_full_install][:output_rtp_d]
else
## prod rtp outputs
output_config = node[:c2c][:recipe_args][:eisplunk_full_install][:output_rtp]
end
Chef::Log.info("Recipe:splunk Step:Detecting setup tar, need #{output_config} ")

elsif node[:fqdn].include? 'oma’
location = "OMA"
Message = ""
if @@devEnvir == true
## dev oma outputs
output_config = node[:c2c][:recipe_args][:eisplunk_full_install][:output_oma_d]

else
## prod oma outputs
output_config = node[:c2c][:recipe_args][:eisplunk_full_install][:output_oma]

end
Chef::Log.info("Recipe:splunk Step:Detecting setup tar, need #{output_config} ")

elsif node[:fqdn].include? 'mmk’
location = "MMK"
Message = ""
if @@devEnvir == true
## dev mmk outputs
output_config = node[:c2c][:recipe_args][:eisplunk_full_install][:output_mmk_d]

else
## prod mmk outputs
output_config = node[:c2c][:recipe_args][:eisplunk_full_install][:output_mmk]
end
Chef::Log.info("Recipe:splunk Step:Detecting setup tar, need #{output_config} ")

else
#Error State
#Send error messag eand close script
Message = ""
Chef::Log.fatal(“Recipe:splunk Step:Unable to detect tar from MMK, OMA, RTP Status:ABORT”)
return

end

#Get Time
time2 = Time.now
startTime = time2.inspect

sending http message to splunk dev server

http_request ‘posting data1’ do
action :post
url "https://abc.com:8088/services/collector/event"
message ({:event => “#{startTime} splunk installer starting. type=#{type} env=#{myenv} requestor=#{requestor}”, :host => "#{HOSTNAME} ", :index => “main”}.to_json)
headers({
‘Authorization’ => ‘Splunksdflksjdf4246-93B9-81931D303A58’
})
end

download blocks should have xx steps

1: download file

2: wait for file

3: verify file exists

4: extract file to appropriate location

5: remove file

###################################### start binary download block

Chef::Log.info(“Recipe:splunk Step:starting binary download block…”)

1 ) download splunk binary from mymon and put it in /tmp directory

remote_file “installerDownload” do
source "#{repo_url}/#{installer}"
path "/tmp/#{installer}"
retries 5
mode '0777’
action :create_if_missing
end

2) waits until binary file is downloaded

ruby_block ‘wait for installer to download’ do
block do
true until ::File.exists?("/tmp/#{installer}")
end
end

3) verify binary file exists

ruby_block ‘verifyInstallerDownloadSuccess’ do
block do
#Code Below Here

       if File.exist?("/tmp/#{installer}")
       
  #File does exist
    Chef::Log.info("Recipe:eisplunk_full_install Step:Downloaded Installer to /tmp ")
    
  else
  #File Does not Exist

    Chef::Log.info("Recipe:eisplunk_full_install Step:Unable to download installer to /tmp  Status:ERROR")
    
    Chef::Log.info("Recipe:eisplunk_full_install Step:Binary LOCAL Path was: /tmp/#{installer} Status:ERROR")
     
    Chef::Log.info("Recipe:eisplunk_full_install Step:Source Path was: #{repo_url}/#{installer}  Status:ERROR")
    
   Chef::Log.info("Recipe:eisplunk_full_install Step:Exiting due to no /tmp/#{installer} Status:ERROR")
  
    return
  end

#Code Above Here
end
action :run
end

4) extract our binary file

execute ‘extractTarFile’ do
Chef::Log.info("Recipe:eisplunk_full_install Step:Extracting #{installer} ")
user node[:c2c][:recipe_args][:eisplunk_full_install][:user]
group node[:c2c][:recipe_args][:eisplunk_full_install][:user]
cwd "/apps/"
command "tar -xvzf /tmp/#{installer}"
end

5) remove our downloaded binary file

execute ‘removeFile’ do
Chef::Log.info("Recipe:eisplunk_full_install deleting #{installer} ")
command “rm /tmp/#{installer}”

command “df”

end
###################################### end of binary download block
Chef::Log.info(“Recipe:splunk Step:Finished binary download block…”)

####################################### deployment file download block

tar deployment file

Chef::Log.info(“Recipe:splunk Step:starting deployment file download block…”)

1) deployment config download block

remote_file “downloadProdTypeFile” do
source "#{repo_url}/#{enviroPathType}/#{prodType}"
path "/apps/splunk/etc/apps/#{prodType}"
retries 5
mode '0777’
action :create_if_missing
end

2) waits until deployment file is downloaded

ruby_block ‘wait for deploymentFile to download’ do
block do
true until ::File.exists?("/apps/splunk/etc/apps/#{prodType}")
end
end

3) verify config file exists

ruby_block ‘verifyDeploymentFileDownloadSuccess’ do
block do
#Code Below Here

       if File.exist?("/apps/splunk/etc/apps/#{prodType}")
       
  #File does exist
    Chef::Log.info("Recipe:eisplunk_full_install Step:Downloaded #{prodType} to /apps/splunk/etc/apps ")
    
  else
  #File Does not Exist

    Chef::Log.info("Recipe:eisplunk_full_install Step:Unable to download #{prodType} to /apps/splunk/etc/apps") 
    return
  end

#Code Above Here
end
action :run
end

4A extract our deployment file

execute ‘extractConfigTarFile’ do
Chef::Log.info("Recipe:eisplunk_full_install Step:Extracting #{prodType} ")
user node[:c2c][:recipe_args][:eisplunk_full_install][:user]
group node[:c2c][:recipe_args][:eisplunk_full_install][:user]
cwd "/apps/splunk/etc/apps/"
command "tar -xvf /apps/splunk/etc/apps/#{prodType}"
end

5) remove our downloaded deployment file

execute ‘removeConfigTarFile’ do
Chef::Log.info("Recipe:eisplunk_full_install deleting #{prodType} ")
command “rm -v /apps/splunk/etc/apps/#{prodType}”

command “df”

end
###################################### end of binary download block
Chef::Log.info(“Recipe:splunk Step:Finished deployment config download block…”)

####################################### start output download block
Chef::Log.info(“Recipe:splunk Step:starting outputs file download block…”)

1) download output file and put in /apps/splunk/etc/apps logic here

remote_file “forwarderOutputConfig” do
source "#{repo_url}/#{enviroPathType}/#{output_config}"
path "/apps/splunk/etc/apps/#{output_config}"
retries 5
mode '0777’
action :create_if_missing
end

2) waits until outputs file is downloaded

ruby_block ‘wait for deploymentFile to download’ do
block do
true until ::File.exists?("/apps/splunk/etc/apps/#{output_config}")
end
end

3) verify outputs file exists

ruby_block ‘verifyOutPutsFileDownloadSuccess’ do
block do
#Code Below Here

       if File.exist?("/apps/splunk/etc/apps/#{output_config}")
       
  #File does exist
    Chef::Log.info("Recipe:eisplunk_full_install Step:Downloaded #{output_config} to /apps/splunk/etc/apps ")
    
  else
  #File Does not Exist

    Chef::Log.info("Recipe:eisplunk_full_install Step:Unable to download #{output_config} to /apps/splunk/etc/apps") 
    return
  end

#Code Above Here
end
action :run
end

4) extract our outputs file

execute ‘extractOutPutsTarFile’ do
Chef::Log.info("Recipe:eisplunk_full_install Step:Extracting #{output_config} ")
user node[:c2c][:recipe_args][:eisplunk_full_install][:user]
group node[:c2c][:recipe_args][:eisplunk_full_install][:user]
cwd "/apps/splunk/etc/apps/"
command "tar -xvf /apps/splunk/etc/apps/#{output_config}"
end

5) remove our downloaded outputs file

execute ‘removeOutPutsTarFile’ do
Chef::Log.info("Recipe:eisplunk_full_install deleting #{output_config} ")
command “rm -v /apps/splunk/etc/apps/#{output_config}”

command “df”

end
###################################### end of outputs download block
Chef::Log.info(“Recipe:splunk Step:Finished outputs config download block…”)

execute ‘writeaccess’ do
cwd "/apps/splunk"
command "chown -R splunk:splunk ./*"
end

execute ‘changepermissions’ do
cwd "/apps/splunk"
command "chmod -R 750 ./*"
end

Chef::Log.info(“Recipe:splunk Step:Updating deployment.conf file …”)
ruby_block ‘replaceFile’ do
block do

updateLine = “clientName=#{HOSTNAME}” + "_heavyforwarder"
Chef::Log.info("Recipe:eisplunk_full_install filepath is #{filepath} ")
Chef::Log.info("Recipe:eisplunk_full_install updateLine is #{updateLine} ")
File.open(filepath, ‘a’) do |file|
file.write(updateLine)
end
end
action :run
end

#start the splunk service
execute “startSplunk” do
user node[:c2c][:recipe_args][:eisplunk_full_install][:splunk_user]
group node[:c2c][:recipe_args][:eisplunk_full_install][:splunk_user]
cwd "/apps/splunk"
command "/apps/splunk/bin/splunk start --answer-yes --accept-license"
end

#Run Splunk Command
execute “run_splunk” do
command "/apps/splunk/bin/splunk enable boot-start -user splunk"
end

#Get Time
time3 = Time.now
endTime = time3.inspect

endeventtime = Time.new.strftime("%Y%m%d%H%M%S")
http_request ‘posting data2’ do
action :post
url "https://abc…com:8088/services/collector/event"
message ({:event => “#{endTime} splunk installer complete. type=#{type} env=#{myenv} requestor=#{requestor}”, :host => "#{HOSTNAME} ", :index => “main”}.to_json)
headers({
‘Authorization’ => ‘Splunksdfsdf213E-03E1-4246-93B9-81931D303A58’
})
end

ruby_block ‘startingSPlunkCom’ do
block do
#Code below here
Chef::Log.info(“Recipe:eisplunk_full_install… finished.”)
#Code Above Here
end
action :run
end

file “/etc/.eisplunk_full_install_completed_do_not_delete” do
action :create
owner "root"
group "root"
end

end

Topic		Replies	Views
Skip particular resource from execution Chef Infra (archive)	4	584	October 30, 2017
How to compare files in recipe Chef Infra (archive)	3	1285	May 10, 2018
Chef-client run making unwanted backup copy of file in file_edit.rb Chef Infra (archive)	0	684	May 4, 2017
not_if not being respected in execute resource? Chef Infra (archive)	3	1271	June 25, 2012
How to tell if my config file is latest release Chef Infra (archive)	1	547	July 28, 2016

How to prevent chef from replacing my files

Create Tag file

chefk to see if splunk already installed

rest of code…

Create Tag file

Cookbook Name:: eisplunk_full_install

Recipe:: default

Copyright 2016, YOUR_COMPANY_NAME

All rights reserved - Do Not Redistribute

installer = splunk binary

configFile = hfw/shc/idx config

prodType = deployment file

output_config = outputs file

set user to splunk

set tar file for splunk binary

set to /apps/splunk/etc/apps directory for download location

set to /tmp directory for full download

set to mymonitoring website and path for downloads - managed by APM

set to local hostname

set to c2c environment

set to requestor corpid

set to “none” but passed through gui/rest call - hfw/sh/idx

set config file to grab default - this may contain settings specific to the config type - hfw/sh/idx

set prod deployment tar file

set prod alias tar file

set path to prod deploymentclient file, which will get updated

Determine the location

sending http message to splunk dev server

download blocks should have xx steps

1: download file

2: wait for file

3: verify file exists

4: extract file to appropriate location

5: remove file

1 ) download splunk binary from mymon and put it in /tmp directory

2) waits until binary file is downloaded

3) verify binary file exists

4) extract our binary file

5) remove our downloaded binary file

command “df”

tar deployment file

1) deployment config download block

2) waits until deployment file is downloaded

3) verify config file exists

4A extract our deployment file

5) remove our downloaded deployment file

command “df”

1) download output file and put in /apps/splunk/etc/apps logic here

2) waits until outputs file is downloaded

3) verify outputs file exists

4) extract our outputs file

5) remove our downloaded outputs file

command “df”

Related topics