Is [chef-client::task] recipe supposed to be idempotent?

artisticcheese · May 23, 2016, 8:51pm

Hello,

I run chef-client as windows scheduled task and inclusion of [chef-client::task] in my runlist and I expected it to create scheduled task only on first run and somehow knew not to create a new task each time chef-client runs but I’m seeing behavior where chef-client recreates task on each run. What would be the proper way to handle this.

G

chris_sullivan · May 24, 2016, 8:31am

Hi artisticcheese,

My original response did not come through for some reason so here is an abbreviated version.

The task is being created by windows_task resource which is declared in the windows cookbook.

If the schedule task action is :create there are three ways the resource can be updated

Using the force setting of the resource
The schedule task user changes
The schedule task command changes

Suspect that the resource is not idempotent because of 3. Why?

This is likely to be down to a Ruby/PowerShell issue around quotes and/or backslashes which need to be escaped out.

Your options

Log an issue at https://github.com/chef-cookbooks/windows/issues
Pull the resource from the resource queue and add an only_if, if the task exists then do not update
Monkey patch the provider

chris_sullivan · May 24, 2016, 8:48am

I’ve just pushed the chef-client cookbook across to a VM and tested with Chef-Zero just running the task recipe and added a pry breakpoint in the Windows cookbook task provider.

The code that does the idempotency check is

def task_need_update?
  # gsub needed as schtasks converts single quotes to double quotes on creation
  @current_resource.command != @new_resource.command.tr("'", "\"") ||
    @current_resource.user != @new_resource.user
end

The values for user in the current and new resource definitions are the same.

The command being run by the schedule task is

@current_resource.command=> "cmd /c \" C:/opscode/chef/bin/chef-client   -L C:/chef/log/client.log   -c C:/chef/client.rb -s 300 > NUL 2>&1\""
@new_resource.command=> "cmd /c ' C:/opscode/chef/bin/chef-client   -L C:/chef/log/client.log   -c C:/chef/client.rb -s 300 > NUL 2>&1'"
@new_resource.command.tr("'", "\"") => "cmd /c \" C:/opscode/chef/bin/chef-client   -L C:/chef/log/client.log   -c C:/chef/client.rb -s 300 > NUL 2>&1\""

With the default attribute settings I’m not getting any resources updated and it looks idempotent to me. have you overridden any of the attributes below?

node['chef_client']['ruby_bin']} 
node['chef_client']['bin']
node['chef_client']['log_dir']
node['chef_client']['conf_dir']
node['chef_client']['splay']
node['chef_client']['task']['user']

Can you supply an extract of the log (debug mode preferably) to show that the resource is not idempotent, chef-client version, cookbook versions of chef-client and windows.

artisticcheese · May 24, 2016, 11:46am

I have a environment override for [task][frequency_modifier]. Would that trigger chef-client to recreate task on each run? If it is then how do modify frequency for different environments?

{
  "chef_client": {
    "task": {
      "frequency_modifier": 17
    }
  }
}

debug log is below which shows new task is being created

[2016-05-24T19:12:56-06:00] INFO: Processing windows_task[chef-client] action create (chef-client::task line 32)
[2016-05-24T19:12:56-06:00] DEBUG: Providers for generic windows_task resource enabled on node include: [LWRP provider windows_task from cookbook windows]
[2016-05-24T19:12:56-06:00] DEBUG: Provider for action create on resource windows_task[chef-client] is LWRP provider windows_task from cookbook windows
[2016-05-24T19:12:56-06:00] DEBUG: Looking for existing tasks
[2016-05-24T19:12:56-06:00] DEBUG: running: 
[2016-05-24T19:12:56-06:00] DEBUG:     schtasks /CREATE /TN "chef-client" /F /SC "minute" /MO "27" /ST "19:39" /TR "cmd /c \" C:/opscode/chef/bin/chef-client   -L C:/chef/log/client.log   -c C:/chef/client.rb -s 300 > NUL 2>&1\"" /RU "SYSTEM" /RL "HIGHEST" 
[2016-05-24T19:12:56-06:00] INFO: windows_task[chef-client] task created
[2016-05-24T19:12:56-06:00] DEBUG: Saving the current state of node CLDPOCCS0062

Chef-client 12.10.24, Windows 2012 R2, windows cookbook 1.41.0, chef-client cookbook 4.5.0

artisticcheese · May 25, 2016, 2:07pm

here is showing that task is being recreate on each run

chris_sullivan · May 26, 2016, 8:34am

Hi artisticcheese,

I’ve had another look just calling chef-client task … sure enough it is not idempotent, perhaps my original tests were using an older version of the cookbook? Using version 4.5.2 of the chef-client cookbook the windows_task resource in the task recipe is not idempotent. I’d probably log an issue on the Github of the chef-client cookbook.

FYI, the windows_task resource runs schtasks (although there is a pending pull for using OLE32, personally I think it would be better if a PowerShell provider was written for Windows 2012/8.0). On my test box I get the following

C:\>schtasks /Query /FO LIST /V /TN "chef-client"

Folder: \
HostName:                             TEST-WIN20-2
TaskName:                             \chef-client
Next Run Time:                        5/26/2016 8:47:00 AM
Status:                               Ready
Logon Mode:                           Interactive/Background
Last Run Time:                        N/A
Last Result:                          1
Author:                               vagrant
Task To Run:                          cmd /c " C:/opscode/chef/bin/chef-client -L C:/chef/log/client.log   -c C:/chef/client.rb -s 300 "
Start In:                             N/A
Comment:                              N/A
Scheduled Task State:                 Enabled
Idle Time:                            Disabled
Power Management:                     Stop On Battery Mode, No Start On Batteries
Run As User:                          SYSTEM
Delete Task If Not Rescheduled:       Disabled
Stop Task If Runs X Hours and X Mins: 72:00:00
Schedule:                             Scheduling data is not available in this format.
Schedule Type:                        One Time Only, Minute
Start Time:                           8:47:00 AM
Start Date:                           5/26/2016
End Date:                             N/A
Days:                                 N/A
Months:                               N/A
Repeat: Every:                        0 Hour(s), 30 Minute(s)
Repeat: Until: Time:                  None
Repeat: Until: Duration:              Disabled
Repeat: Stop If Still Running:        Disabled

The windows_task provider checks the current value of the task_to_run against the one specified in the new resource.

The new resource looks like this

windows_task 'chef-client' do
  run_level :highest
  command "cmd /c \\\"#{node['chef_client']['ruby_bin']} #{node['chef_client']['bin']} \
  -L #{File.join(node['chef_client']['log_dir'], 'client.log')} \
  -c #{File.join(node['chef_client']['conf_dir'], 'client.rb')} -s #{node['chef_client']['splay']} > NUL 2>&1\\\""

  user               node['chef_client']['task']['user']
  password           node['chef_client']['task']['password']
  frequency          node['chef_client']['task']['frequency'].to_sym
  frequency_modifier node['chef_client']['task']['frequency_modifier']
  start_time         node['chef_client']['task']['start_time'] || start_time
end

As you can see the command has a re-direct which is being lost when the schedule tasks is being saved, you can also see this by opening the task scheduler, locating chef-client, double clicking, select actions tab, edit. Sticking a breakpoint on the provider I can see the following

@current_resource.command    => "cmd /c \" C:/opscode/chef/bin/chef-client   -L C:/chef/log/client.log   -c C:/chef/client.rb -s 300 \""
@new_resource.command        => "cmd /c \\\" C:/opscode/chef/bin/chef-client   -L C:/chef/log/client.log   -c C:/chef/client.rb -s 300 > NUL 2>&1\\\""

chris_sullivan · May 26, 2016, 8:51am

The command that is generated by chef-client is pretty complex with regards to backslashes and quotes

cmd => "schtasks /CREATE /TN \"chef-client\" /F /SC \"minute\" /MO \"30\" /ST \"09:10\" /TR \"cmd /c \\\" C:/opscode/chef/bin/chef-client   -L C:/chef/log/client.log   -c C:/chef/client.rb -s 300 > NUL 2>&1\\\"\" /RU \"SYSTEM\" /RL \"HIGHEST\""

Started to debug but I suspect this will burn a couple of hours, definitely ping over an issue on the chef-client github, if I get time I’ll have another look myself.

Chris.

artisticcheese · May 26, 2016, 2:15pm

I created issue on github but I’m not sure I did it for correct cookbook.

Matt_Wrock · May 27, 2016, 7:17am

This requires fixes in both the windows cookbook as well as chef-client. I have submitted https://github.com/chef-cookbooks/windows/pull/365 and https://github.com/chef-cookbooks/chef-client/pull/398 to address this.

chris_sullivan · May 27, 2016, 7:34am

Thank you @Matt_Wrock

artisticcheese · May 27, 2016, 4:22pm

I updated Windows Cookbook to version 1.42.0 and now it still tries to create a new task on each run but fails with error below.

#<Mixlib::ShellOut::ShellCommandFailed: windows_task[chef-client] (chef-client::task line 32) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received ‘-2147467259’ ---- Begin output of schtasks /CREATE /TN “chef-client” /F /SC “minute” /MO “30” /ST “10:48” /TR “cmd /c \” C:/opscode/chef/bin/chef-client -L C:/chef/log/client.log -c C:/chef/client.rb -s 300 > NUL 2>&1\"" /RU “SYSTEM” /RL “HIGHEST” ---- STDOUT: STDERR: ---- End output of schtasks /CREATE /TN “chef-client” /F /SC “minute” /MO “30” /ST “10:48” /TR “cmd /c \” C:/opscode/chef/bin/chef-client -L C:/chef/log/client.log -c C:/chef/client.rb -s 300 > NUL 2>&1\"" /RU “SYSTEM” /RL “HIGHEST” ---- Ran schtasks /CREATE /TN “chef-client” /F /SC “minute” /MO “30” /ST “10:48” /TR “cmd /c \” C:/opscode/chef/bin/chef-client -L C:/chef/log/client.log -c C:/chef/client.rb -s 300 > NUL 2>&1\"" /RU “SYSTEM” /RL “HIGHEST” returned -2147467259> had an error:
windows_task[chef-client] (chef-client::task line 32) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received ‘-2147467259’ ---- Begin output of schtasks /CREATE /TN “chef-client” /F /SC “minute” /MO “30” /ST “10:48” /TR “cmd /c \” C:/opscode/chef/bin/chef-client -L C:/chef/log/client.log -c C:/chef/client.rb -s 300 > NUL 2>&1\"" /RU “SYSTEM” /RL “HIGHEST” ---- STDOUT: STDERR: ---- End output of schtasks /CREATE /TN “chef-client” /F /SC “minute” /MO “30” /ST “10:48” /TR “cmd /c \” C:/opscode/chef/bin/chef-client -L C:/chef/log/client.log -c C:/chef/client.rb -s 300 > NUL 2>&1\"" /RU “SYSTEM” /RL “HIGHEST” ---- Ran schtasks /CREATE /TN “chef-client” /F /SC “minute” /MO “30” /ST “10:48” /TR “cmd /c \” C:/opscode/chef/bin/chef-client -L C:/chef/log/client.log -c C:/chef/client.rb -s 300 > NUL 2>&1\"" /RU “SYSTEM” /RL “HIGHEST” returned -2147467259

Matt_Wrock · May 27, 2016, 4:33pm

we are in the midst of changing the chef-client cookbook to work with the windows changes. chef-client 4.5.3 has just ben released so please try now with that version.

artisticcheese · May 27, 2016, 5:06pm

Well it fixed issue with idempotency but broke ability to override task attributes. Previously it will recreate task but will honor task attributes overriden but now it does not.
If I have a task with default attribute of 30 mins and then override default in environment, it will not be recreated with new attribute.

{
  "chef_client": {
    "task": {
      "frequency_modifier": 17
    }
  }
}

Matt_Wrock · May 27, 2016, 5:41pm

ugh yes. For some reason the task idempotency (and I don’t think this is new) is based solely on the command and user name and it ignores all other attribute changes. I just created this issue to track this problem. In the meantime, you could work around the issue by removing the task prior to the chef-client task recipe.

artisticcheese · May 27, 2016, 6:20pm

Well if I delete task then chef-client will not run. So I need to delete task and then manually run chef-client. Chicken and egg scenario.

Matt_Wrock · May 27, 2016, 6:39pm

Sorry for the confusion. What I mean is to delete the task and then run the chef-client recipe in the same run. I think that should effectively put you back in your original position which is not ideal but should “work”.

Topic		Replies	Views
References for writing Chef idempotent Chef Infra (archive)	1	352	September 13, 2018
Chef-server run list questions Chef Infra (archive)	3	273	May 19, 2010
Can't modify frozen String Chef Infra (archive)	6	2677	December 29, 2017
New user feedback and questions Chef Infra (archive)	8	335	October 7, 2009
Recipes run order versus resource usage/cloning Chef Infra (archive)	2	688	April 7, 2014

Is [chef-client::task] recipe supposed to be idempotent?

Related Topics