When trying to run chef-client on a particular node (server) we are getting no output. The command appears to run for a few seconds then just completes with no output or results.
Here is the command:
knife winrm fqdn chef-client --manual-list --winrm-user <user> --winrm-password <pwd>
The cookbooks set in the run_list are also not running. We’ve only seen this issue with this one particular node.
This is often the result of a
log_location property in the node’s client.rb. When this property exists, the chef client will log to that location and also
STDOUT if a
TTY is present. Unfortunately a winrm session does not expose a tty, and therefore chef client runs over winrm will yield no output when a
log_location is present in the client.rb. So you can either remove the
log_location if only temporarily or you could run chef-client locally on the node.
log_location from the client.rb did not help. But I was able to run chef-client locally on the client machine and it did run the cookbooks set in the run_list.
The thing is we need to run chef-client remotely via our automated processes. Is there anything we can do to fix this problem?
What version of windows is on the node? This could be the windows 2008 bug that does not run .bat files over winrm if you are using 2008 (R1). If thats the case you can work around it using the powershell shell by adding
--winrm-shell powershell to the command.
The key question is if chef-client is not running or if its just not emitting output.
The client is Windows Server 2008 Standard SP2. I don’t think the chef-client is running when we run our usual remote commands.
--winrm-shell powershell worked! Thanks for that tip!
Will we need to add this command for all our Server 2008 Standard machines, or do you think this is just an isolated incident?
Ahh yes! Windows Server 2008 Standard SP2 explains it. Yes you will need to add that for all machines using that OS (should work for others too). Keep in mind that any command you use must be runnable in powershell.
Note that the powershell does not work for
knife bootstrap. If you are using
knife bootstrap then you need to add
Yes, we love PowerShell. PowerShell is great! PowerShell is the driving force behind our automated workflow with Chef.
I will make sure to add
--winrm-shell powershell for our chef-client runs for all our 2008 Standard, 2008 R2, and 2012 machines.
And yes, we also like our buddy
--winrm-codepage 437. It was the only way we could bootstrap our 2008 Standard servers.
@Matt_Wrock, we’re also presently on win2008r2. Our chef-client is 12.13.37. The knife bootstrap places a log_location entry in the client.rb with a value of STDOUT.
We use --winrm-shell elevated because of other grief winrm brings to a client run, but I’m not seeing the output. We do the converge in a separate winrm call (with a runlist) to allow this (bootstrap has no run_list because otherwise it’s bitten by winrm).
This initial bootstrap/converge run happens via orchestration (Jenkins). Can you think of a means to overcome this situation?
UPDATE: It looks like the job’s configuration in Jenkins was the culprit here. Jenkins showed a twirly symbol as though it were doing something, but the fact was the job was waiting for a winrm password. The client wasn’t actually doing anything because of this. Once I recognized and resolved that, I see output as desired. WhooHoo!