Is [chef-client::task] recipe supposed to be idempotent?


#1

Hello,

I run chef-client as windows scheduled task and inclusion of [chef-client::task] in my runlist and I expected it to create scheduled task only on first run and somehow knew not to create a new task each time chef-client runs but I’m seeing behavior where chef-client recreates task on each run. What would be the proper way to handle this.

G


#2

Hi artisticcheese,

My original response did not come through for some reason so here is an abbreviated version.

The task is being created by windows_task resource which is declared in the windows cookbook.

If the schedule task action is :create there are three ways the resource can be updated

  1. Using the force setting of the resource
  2. The schedule task user changes
  3. The schedule task command changes

Suspect that the resource is not idempotent because of 3. Why?

This is likely to be down to a Ruby/PowerShell issue around quotes and/or backslashes which need to be escaped out.

Your options

  1. Log an issue at https://github.com/chef-cookbooks/windows/issues
  2. Pull the resource from the resource queue and add an only_if, if the task exists then do not update
  3. Monkey patch the provider

#3

I’ve just pushed the chef-client cookbook across to a VM and tested with Chef-Zero just running the task recipe and added a pry breakpoint in the Windows cookbook task provider.

The code that does the idempotency check is

def task_need_update?
  # gsub needed as schtasks converts single quotes to double quotes on creation
  @current_resource.command != @new_resource.command.tr("'", "\"") ||
    @current_resource.user != @new_resource.user
end

The values for user in the current and new resource definitions are the same.

The command being run by the schedule task is

@current_resource.command=> "cmd /c \" C:/opscode/chef/bin/chef-client   -L C:/chef/log/client.log   -c C:/chef/client.rb -s 300 > NUL 2>&1\""
@new_resource.command=> "cmd /c ' C:/opscode/chef/bin/chef-client   -L C:/chef/log/client.log   -c C:/chef/client.rb -s 300 > NUL 2>&1'"
@new_resource.command.tr("'", "\"") => "cmd /c \" C:/opscode/chef/bin/chef-client   -L C:/chef/log/client.log   -c C:/chef/client.rb -s 300 > NUL 2>&1\""

With the default attribute settings I’m not getting any resources updated and it looks idempotent to me. have you overridden any of the attributes below?

node['chef_client']['ruby_bin']} 
node['chef_client']['bin']
node['chef_client']['log_dir']
node['chef_client']['conf_dir']
node['chef_client']['splay']
node['chef_client']['task']['user']

Can you supply an extract of the log (debug mode preferably) to show that the resource is not idempotent, chef-client version, cookbook versions of chef-client and windows.


#4

I have a environment override for [task][frequency_modifier]. Would that trigger chef-client to recreate task on each run? If it is then how do modify frequency for different environments?

{
  "chef_client": {
    "task": {
      "frequency_modifier": 17
    }
  }
}

debug log is below which shows new task is being created

[2016-05-24T19:12:56-06:00] INFO: Processing windows_task[chef-client] action create (chef-client::task line 32)
[2016-05-24T19:12:56-06:00] DEBUG: Providers for generic windows_task resource enabled on node include: [LWRP provider windows_task from cookbook windows]
[2016-05-24T19:12:56-06:00] DEBUG: Provider for action create on resource windows_task[chef-client] is LWRP provider windows_task from cookbook windows
[2016-05-24T19:12:56-06:00] DEBUG: Looking for existing tasks
[2016-05-24T19:12:56-06:00] DEBUG: running: 
[2016-05-24T19:12:56-06:00] DEBUG:     schtasks /CREATE /TN "chef-client" /F /SC "minute" /MO "27" /ST "19:39" /TR "cmd /c \" C:/opscode/chef/bin/chef-client   -L C:/chef/log/client.log   -c C:/chef/client.rb -s 300 > NUL 2>&1\"" /RU "SYSTEM" /RL "HIGHEST" 
[2016-05-24T19:12:56-06:00] INFO: windows_task[chef-client] task created
[2016-05-24T19:12:56-06:00] DEBUG: Saving the current state of node CLDPOCCS0062

Chef-client 12.10.24, Windows 2012 R2, windows cookbook 1.41.0, chef-client cookbook 4.5.0


#5

here is showing that task is being recreate on each run


#6

Hi artisticcheese,

I’ve had another look just calling chef-client task … sure enough it is not idempotent, perhaps my original tests were using an older version of the cookbook? Using version 4.5.2 of the chef-client cookbook the windows_task resource in the task recipe is not idempotent. I’d probably log an issue on the Github of the chef-client cookbook.

FYI, the windows_task resource runs schtasks (although there is a pending pull for using OLE32, personally I think it would be better if a PowerShell provider was written for Windows 2012/8.0). On my test box I get the following

C:\>schtasks /Query /FO LIST /V /TN "chef-client"

Folder: \
HostName:                             TEST-WIN20-2
TaskName:                             \chef-client
Next Run Time:                        5/26/2016 8:47:00 AM
Status:                               Ready
Logon Mode:                           Interactive/Background
Last Run Time:                        N/A
Last Result:                          1
Author:                               vagrant
Task To Run:                          cmd /c " C:/opscode/chef/bin/chef-client -L C:/chef/log/client.log   -c C:/chef/client.rb -s 300 "
Start In:                             N/A
Comment:                              N/A
Scheduled Task State:                 Enabled
Idle Time:                            Disabled
Power Management:                     Stop On Battery Mode, No Start On Batteries
Run As User:                          SYSTEM
Delete Task If Not Rescheduled:       Disabled
Stop Task If Runs X Hours and X Mins: 72:00:00
Schedule:                             Scheduling data is not available in this format.
Schedule Type:                        One Time Only, Minute
Start Time:                           8:47:00 AM
Start Date:                           5/26/2016
End Date:                             N/A
Days:                                 N/A
Months:                               N/A
Repeat: Every:                        0 Hour(s), 30 Minute(s)
Repeat: Until: Time:                  None
Repeat: Until: Duration:              Disabled
Repeat: Stop If Still Running:        Disabled

The windows_task provider checks the current value of the task_to_run against the one specified in the new resource.

The new resource looks like this

windows_task 'chef-client' do
  run_level :highest
  command "cmd /c \\\"#{node['chef_client']['ruby_bin']} #{node['chef_client']['bin']} \
  -L #{File.join(node['chef_client']['log_dir'], 'client.log')} \
  -c #{File.join(node['chef_client']['conf_dir'], 'client.rb')} -s #{node['chef_client']['splay']} > NUL 2>&1\\\""

  user               node['chef_client']['task']['user']
  password           node['chef_client']['task']['password']
  frequency          node['chef_client']['task']['frequency'].to_sym
  frequency_modifier node['chef_client']['task']['frequency_modifier']
  start_time         node['chef_client']['task']['start_time'] || start_time
end

As you can see the command has a re-direct which is being lost when the schedule tasks is being saved, you can also see this by opening the task scheduler, locating chef-client, double clicking, select actions tab, edit. Sticking a breakpoint on the provider I can see the following

@current_resource.command    => "cmd /c \" C:/opscode/chef/bin/chef-client   -L C:/chef/log/client.log   -c C:/chef/client.rb -s 300 \""
@new_resource.command        => "cmd /c \\\" C:/opscode/chef/bin/chef-client   -L C:/chef/log/client.log   -c C:/chef/client.rb -s 300 > NUL 2>&1\\\""

#7

The command that is generated by chef-client is pretty complex with regards to backslashes and quotes

cmd => "schtasks /CREATE /TN \"chef-client\" /F /SC \"minute\" /MO \"30\" /ST \"09:10\" /TR \"cmd /c \\\" C:/opscode/chef/bin/chef-client   -L C:/chef/log/client.log   -c C:/chef/client.rb -s 300 > NUL 2>&1\\\"\" /RU \"SYSTEM\" /RL \"HIGHEST\""

Started to debug but I suspect this will burn a couple of hours, definitely ping over an issue on the chef-client github, if I get time I’ll have another look myself.

Chris.


#8

I created issue on github but I’m not sure I did it for correct cookbook.


#9

This requires fixes in both the windows cookbook as well as chef-client. I have submitted https://github.com/chef-cookbooks/windows/pull/365 and https://github.com/chef-cookbooks/chef-client/pull/398 to address this.


#10

Thank you @Matt_Wrock


#11

I updated Windows Cookbook to version 1.42.0 and now it still tries to create a new task on each run but fails with error below.

#<Mixlib::ShellOut::ShellCommandFailed: windows_task[chef-client] (chef-client::task line 32) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received ‘-2147467259’ ---- Begin output of schtasks /CREATE /TN “chef-client” /F /SC “minute” /MO “30” /ST “10:48” /TR “cmd /c \” C:/opscode/chef/bin/chef-client -L C:/chef/log/client.log -c C:/chef/client.rb -s 300 > NUL 2>&1\"" /RU “SYSTEM” /RL “HIGHEST” ---- STDOUT: STDERR: ---- End output of schtasks /CREATE /TN “chef-client” /F /SC “minute” /MO “30” /ST “10:48” /TR “cmd /c \” C:/opscode/chef/bin/chef-client -L C:/chef/log/client.log -c C:/chef/client.rb -s 300 > NUL 2>&1\"" /RU “SYSTEM” /RL “HIGHEST” ---- Ran schtasks /CREATE /TN “chef-client” /F /SC “minute” /MO “30” /ST “10:48” /TR “cmd /c \” C:/opscode/chef/bin/chef-client -L C:/chef/log/client.log -c C:/chef/client.rb -s 300 > NUL 2>&1\"" /RU “SYSTEM” /RL “HIGHEST” returned -2147467259> had an error:
windows_task[chef-client] (chef-client::task line 32) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received ‘-2147467259’ ---- Begin output of schtasks /CREATE /TN “chef-client” /F /SC “minute” /MO “30” /ST “10:48” /TR “cmd /c \” C:/opscode/chef/bin/chef-client -L C:/chef/log/client.log -c C:/chef/client.rb -s 300 > NUL 2>&1\"" /RU “SYSTEM” /RL “HIGHEST” ---- STDOUT: STDERR: ---- End output of schtasks /CREATE /TN “chef-client” /F /SC “minute” /MO “30” /ST “10:48” /TR “cmd /c \” C:/opscode/chef/bin/chef-client -L C:/chef/log/client.log -c C:/chef/client.rb -s 300 > NUL 2>&1\"" /RU “SYSTEM” /RL “HIGHEST” ---- Ran schtasks /CREATE /TN “chef-client” /F /SC “minute” /MO “30” /ST “10:48” /TR “cmd /c \” C:/opscode/chef/bin/chef-client -L C:/chef/log/client.log -c C:/chef/client.rb -s 300 > NUL 2>&1\"" /RU “SYSTEM” /RL “HIGHEST” returned -2147467259


#12

we are in the midst of changing the chef-client cookbook to work with the windows changes. chef-client 4.5.3 has just ben released so please try now with that version.


#13

Well it fixed issue with idempotency but broke ability to override task attributes. Previously it will recreate task but will honor task attributes overriden but now it does not.
If I have a task with default attribute of 30 mins and then override default in environment, it will not be recreated with new attribute.

{
  "chef_client": {
    "task": {
      "frequency_modifier": 17
    }
  }
}

#14

ugh yes. For some reason the task idempotency (and I don’t think this is new) is based solely on the command and user name and it ignores all other attribute changes. I just created this issue to track this problem. In the meantime, you could work around the issue by removing the task prior to the chef-client task recipe.


#15

Well if I delete task then chef-client will not run. So I need to delete task and then manually run chef-client. Chicken and egg scenario.


#16

Sorry for the confusion. What I mean is to delete the task and then run the chef-client recipe in the same run. I think that should effectively put you back in your original position which is not ideal but should “work”.