Upgrading chef-client on Windows not supported?

Jeppe_Nejsum_Madsen · February 27, 2014, 9:15pm

Ohai,

I’ve been trying to figure out how to best upgrade chef-client on Windows
machines, but can’t find any info about it.

This seems a bit strange since Chef is clearly being pushed as being
windows friendly (http://www.getchef.com/solutions/windows/).

The official(?) way of upgrading chef-client (
http://www.getchef.com/blog/2013/12/20/managing-chef-with-chef/)
unfortunately doesn’t work:
https://github.com/hw-cookbooks/omnibus_updater/issues/32

So for this not to end up too whiny, I decided to actually fix the
omnibus_updater issue.

What I got seems to work to some degree in that the MSI gets downloaded and
the installation starts. But during the installation it seems like the
running chef-client process gets a SIGINT and stops with a failure. But the
installation continues (and the RestartManager says a reboot is required)

Recipe: omnibus_updater::downloader←[0m

remote_file[omnibus_remote[chef-client-11.6.2-1.windows.msi]] action
create_if_missing←[0m←[32m
- copy file downloaded from [] into
  C:/chef/cache/chef-client-11.6.2-1.windows.msi←[0m←[37m
  (file sizes exceed 10000000 bytes, diff output suppressed)←[0m
  ←[0m
  Recipe: omnibus_updater::installer←[0m
ruby_block[Omnibus Chef install notifier] action create←[0m←[32m
- execute the ruby block Omnibus Chef install notifier←[0m
  ←[0m
file[/tmp/nocheck] action create←[0m (skipped due to only_if)←[0m
ruby_block[omnibus chef killer] action nothing←[0m (up to date)←[0m
windows_package[stfu] action nothing←[0m (up to date)←[0m
ruby_block[Omnibus Chef install notifier] action nothing←[0m (up to
date)←[0m
windows_package[chef-client] action
install←[0m[2014-02-27T22:04:13+01:00] FATAL: SIGINT received, stopping
[2014-02-27T22:04:51+01:00] FATAL: SIGINT received, stopping

================================================================================←[0m
←[31mError executing action install on resource
’windows_package[chef-client]'←[0m
================================================================================←[0m

←[0m
SystemExit←[0m
----------←[0m
exit←[0m

After a while the installation does seems to complete…

Is the only alternative really to install the MSI through some other means?

/Jeppe

coderanger · February 27, 2014, 9:21pm

Windows, unlike all other OSes, locks executable files while they are running. The usual solution is to just move the old executables aside and write the new ones in their place. Unfortunately there is no way to do this atomically, so you often end up with tiny updater.exe binaries that the original process launches (and then kills itself) to manage the update process. I'm sure a pull-request would be happily accepted to add this dance to the omnibus_updater cookbook and I'm happy to talk anyone through other strategies for doing binary updates on Windows (might as well get some benefit out of doing that for 3 solid years :-).

--Noah

On Feb 27, 2014, at 1:15 PM, Jeppe Nejsum Madsen jeppe@ingolfs.dk wrote:

Ohai,

I've been trying to figure out how to best upgrade chef-client on Windows machines, but can't find any info about it.

This seems a bit strange since Chef is clearly being pushed as being windows friendly (Chef Automation Solutions | Chef).

The official(?) way of upgrading chef-client (Upgrading Chef using Chef - Chef Blog | Chef) unfortunately doesn't work: https://github.com/hw-cookbooks/omnibus_updater/issues/32

So for this not to end up too whiny, I decided to actually fix the omnibus_updater issue.

What I got seems to work to some degree in that the MSI gets downloaded and the installation starts. But during the installation it seems like the running chef-client process gets a SIGINT and stops with a failure. But the installation continues (and the RestartManager says a reboot is required)

Recipe: omnibus_updater::downloader←[0m

remote_file[omnibus_remote[chef-client-11.6.2-1.windows.msi]] action create_if_missing←[0m←[32m

copy file downloaded from into C:/chef/cache/chef-client-11.6.2-1.windows.msi←[0m←[37m
(file sizes exceed 10000000 bytes, diff output suppressed)←[0m
←[0m
Recipe: omnibus_updater::installer←[0m

ruby_block[Omnibus Chef install notifier] action create←[0m←[32m

execute the ruby block Omnibus Chef install notifier←[0m
←[0m

file[/tmp/nocheck] action create←[0m (skipped due to only_if)←[0m

ruby_block[omnibus chef killer] action nothing←[0m (up to date)←[0m

windows_package[stfu] action nothing←[0m (up to date)←[0m

ruby_block[Omnibus Chef install notifier] action nothing←[0m (up to date)←[0m

windows_package[chef-client] action install←[0m[2014-02-27T22:04:13+01:00] FATAL: SIGINT received, stopping
[2014-02-27T22:04:51+01:00] FATAL: SIGINT received, stopping

================================================================================←[0m
←[31mError executing action install on resource 'windows_package[chef-client]'←[0m
================================================================================←[0m

←[0m
SystemExit←[0m
----------←[0m
exit←[0m

After a while the installation does seems to complete...

Is the only alternative really to install the MSI through some other means?

/Jeppe

Jeppe_Nejsum_Madsen · February 27, 2014, 9:29pm

On Thu, Feb 27, 2014 at 10:21 PM, Noah Kantrowitz noah@coderanger.netwrote:

Windows, unlike all other OSes, locks executable files while they are
running. The usual solution is to just move the old executables aside and
write the new ones in their place. Unfortunately there is no way to do this
atomically, so you often end up with tiny updater.exe binaries that the
original process launches (and then kills itself) to manage the update
process. I'm sure a pull-request would be happily accepted to add this
dance to the omnibus_updater cookbook and I'm happy to talk anyone through
other strategies for doing binary updates on Windows (might as well get
some benefit out of doing that for 3 solid years :-).

Yeah I guess that it is not easily done. I don't think this should be in
the omnibus_updater though as it's not just the .exe but the entire
embedded ruby env that might need replacing.

I think the chef-client MSI should handle this. Like many other windows
services it should be able to upgrade itself (preferably without a reboot,
but hey this is windows after all

/Jeppe

coderanger · February 27, 2014, 9:33pm

On Feb 27, 2014, at 1:29 PM, Jeppe Nejsum Madsen jeppe@ingolfs.dk wrote:

On Thu, Feb 27, 2014 at 10:21 PM, Noah Kantrowitz noah@coderanger.net wrote:
Windows, unlike all other OSes, locks executable files while they are running. The usual solution is to just move the old executables aside and write the new ones in their place. Unfortunately there is no way to do this atomically, so you often end up with tiny updater.exe binaries that the original process launches (and then kills itself) to manage the update process. I'm sure a pull-request would be happily accepted to add this dance to the omnibus_updater cookbook and I'm happy to talk anyone through other strategies for doing binary updates on Windows (might as well get some benefit out of doing that for 3 solid years :-).

Yeah I guess that it is not easily done. I don't think this should be in the omnibus_updater though as it's not just the .exe but the entire embedded ruby env that might need replacing.

Yes, the problem is almost certainly the ruby.exe and other DLLs that are active while Chef is running.

I think the chef-client MSI should handle this. Like many other windows services it should be able to upgrade itself (preferably without a reboot, but hey this is windows after all

That would work too, I defer to others with more WiX experience as to how hard it is to augment the MSI do this though.

--Noah

Kapil_Shardha1 · February 27, 2014, 9:36pm

The MSI installer platform supports "Upgrade" scenario as long as the MSI has an unique "Product Code" and "Version Number" included. I am not sure what chef-client MSI does or how it has been authored but I have authored installers using WiX that support "Upgrade scenario".

Thanks

-Kapil

-----Original Message-----
From: Noah Kantrowitz [mailto:noah@coderanger.net]
Sent: Thursday, February 27, 2014 4:34 PM
To: chef@lists.opscode.com
Subject: [chef] Re: Re: Re: Upgrading chef-client on Windows not supported?

On Feb 27, 2014, at 1:29 PM, Jeppe Nejsum Madsen jeppe@ingolfs.dk wrote:

On Thu, Feb 27, 2014 at 10:21 PM, Noah Kantrowitz noah@coderanger.net wrote:
Windows, unlike all other OSes, locks executable files while they are running. The usual solution is to just move the old executables aside and write the new ones in their place. Unfortunately there is no way to do this atomically, so you often end up with tiny updater.exe binaries that the original process launches (and then kills itself) to manage the update process. I'm sure a pull-request would be happily accepted to add this dance to the omnibus_updater cookbook and I'm happy to talk anyone through other strategies for doing binary updates on Windows (might as well get some benefit out of doing that for 3 solid years :-).

Yeah I guess that it is not easily done. I don't think this should be in the omnibus_updater though as it's not just the .exe but the entire embedded ruby env that might need replacing.

Yes, the problem is almost certainly the ruby.exe and other DLLs that are active while Chef is running.

I think the chef-client MSI should handle this. Like many other windows services it should be able to upgrade itself (preferably without a reboot, but hey this is windows after all

That would work too, I defer to others with more WiX experience as to how hard it is to augment the MSI do this though.

--Noah

This email and any accompanying documents may contain privileged or otherwise confidential information of, and/or is the property of Education Management Solutions, Inc. If you are not the intended recipient, please immediately advise the sender by reply email & delete the message & any attachments without using, copying or disclosing the contents. Thank you.

Lamont_Granquist_OLD · February 28, 2014, 1:23am

there's a known problem where i think we're using unique product codes
for every different chef msi so that you can have multiple chef-client
packages installed at the same time (even though they overwrite each
other's files).

https://tickets.opscode.com/browse/CHEF-4657

not sure if that's related, but it probably doesn't help any...

On Thu Feb 27 13:36:45 2014, Kapil Shardha wrote:

The MSI installer platform supports "Upgrade" scenario as long as the MSI has an unique "Product Code" and "Version Number" included. I am not sure what chef-client MSI does or how it has been authored but I have authored installers using WiX that support "Upgrade scenario".

Kapil_Shardha1 · February 28, 2014, 2:09am

Wondering if the MSI's are being created using Visual Studio installer projects or WiX. If that is the case, VS generates a random GUID as PRODUCT CODE property.

Perhaps, checking the registry on a system with multiple versions of chef-client will tell if the Product Code is different.

-Kapil

-----Original Message-----
From: Lamont Granquist [mailto:lamont@opscode.com]
Sent: Thursday, February 27, 2014 8:23 PM
To: chef@lists.opscode.com
Cc: Kapil Shardha
Subject: Re: [chef] RE: Re: Re: Re: Upgrading chef-client on Windows not supported?

there's a known problem where i think we're using unique product codes for every different chef msi so that you can have multiple chef-client packages installed at the same time (even though they overwrite each other's files).

https://tickets.opscode.com/browse/CHEF-4657

not sure if that's related, but it probably doesn't help any...

On Thu Feb 27 13:36:45 2014, Kapil Shardha wrote:

The MSI installer platform supports "Upgrade" scenario as long as the MSI has an unique "Product Code" and "Version Number" included. I am not sure what chef-client MSI does or how it has been authored but I have authored installers using WiX that support "Upgrade scenario".

This email and any accompanying documents may contain privileged or otherwise confidential information of, and/or is the property of Education Management Solutions, Inc. If you are not the intended recipient, please immediately advise the sender by reply email & delete the message & any attachments without using, copying or disclosing the contents. Thank you.

Lamont_Granquist_OLD · February 28, 2014, 2:28am

they're built with wix:

https://github.com/opscode/omnibus-chef/blob/master/config/software/chef-client-msi.rb

On 2/27/14 6:09 PM, Kapil Shardha wrote:

Wondering if the MSI's are being created using Visual Studio installer projects or WiX. If that is the case, VS generates a random GUID as PRODUCT CODE property.

Perhaps, checking the registry on a system with multiple versions of chef-client will tell if the Product Code is different.

-Kapil

-----Original Message-----
From: Lamont Granquist [mailto:lamont@opscode.com]
Sent: Thursday, February 27, 2014 8:23 PM
To: chef@lists.opscode.com
Cc: Kapil Shardha
Subject: Re: [chef] RE: Re: Re: Re: Upgrading chef-client on Windows not supported?

there's a known problem where i think we're using unique product codes for every different chef msi so that you can have multiple chef-client packages installed at the same time (even though they overwrite each other's files).

https://tickets.opscode.com/browse/CHEF-4657

not sure if that's related, but it probably doesn't help any...

On Thu Feb 27 13:36:45 2014, Kapil Shardha wrote:

The MSI installer platform supports "Upgrade" scenario as long as the MSI has an unique "Product Code" and "Version Number" included. I am not sure what chef-client MSI does or how it has been authored but I have authored installers using WiX that support "Upgrade scenario".

This email and any accompanying documents may contain privileged or otherwise confidential information of, and/or is the property of Education Management Solutions, Inc. If you are not the intended recipient, please immediately advise the sender by reply email & delete the message & any attachments without using, copying or disclosing the contents. Thank you.

Adam_Edwards1 · February 28, 2014, 5:28pm

Lamont is correct about the root cause -- the build always produces a new
msi, and there are no upgrade codes in the generated package. And
omnibus-updater doesn't support Windows as you've found out Jeppe.

Jeppe, great start on this. So until the scheme for handling productid's /
upgrade logic for the msi across versions is resolved, a version of
omnibus-updater that works for Windows is the way to go.

At the point in omnibus-updated at which we know the package is not
installed and you need to install it, we can create a scheduled task to run
an upgrade script of some sort after the chef-client run completes. A script
like that would do something like the following:

net stop chef-client # so in case the service is running
Check to make sure there are no chef-client processes running
a. slightly tricky, you need to look at all processes, find each ruby.exe
process, and make sure the arguments don't contain 'chef-client'. In newer
versions of chef-client, you may be able to look for a process lock file.
Find the existing chef-client installed product by name via wmi -- I can
send some examples if it will help
Use wmi's msi handler to uninstall all chef-clients that you find
Run the installation for the new chef-client package
If the service was originally enabled (easy to find during the
chef-client run and then pass on the cli to the scheduled task), install the
service with chef-service-manager
a. This will also cause a chef-client run
If the service was not enabled, tell it to run chef-client. This step
should run regardless of whether the previous steps succeeded
The task should delete itself to keep things clean, alternatively, it
could delete itself only if #7 fails and configure itself to retry.

Since this would need to work on systems without powershell (because prior
to Win2k8R2 powershell is not installed by default :(), it's best to do this
via vbscript / jscript with cscript.exe. Powershell would be much easier
though if your focus is just getting things to work in your scenario.

Note that at the next chef-client run, the correct version should be
detected and then nothing happens.

If anyone else has implemented this, we should consider that as a starting
point for a PR. Otherwise, we can work with you Jeppe to get something
together before we finally get to it in our backlog of work.

Thanks.

-Adam

-----Original Message-----
From: Lamont Granquist [mailto:lamont@opscode.com]
Sent: Thursday, February 27, 2014 6:28 PM
To: chef@lists.opscode.com
Subject: [chef] Re: RE: RE: Re: Re: Re: Upgrading chef-client on Windows not
supported?

they're built with wix:

https://github.com/opscode/omnibus-chef/blob/master/config/software/chef-client-msi.rb

On 2/27/14 6:09 PM, Kapil Shardha wrote:

Wondering if the MSI's are being created using Visual Studio installer
projects or WiX. If that is the case, VS generates a random GUID as
PRODUCT CODE property.

Perhaps, checking the registry on a system with multiple versions of
chef-client will tell if the Product Code is different.

-Kapil

-----Original Message-----
From: Lamont Granquist [mailto:lamont@opscode.com]
Sent: Thursday, February 27, 2014 8:23 PM
To: chef@lists.opscode.com
Cc: Kapil Shardha
Subject: Re: [chef] RE: Re: Re: Re: Upgrading chef-client on Windows not
supported?

there's a known problem where i think we're using unique product codes for
every different chef msi so that you can have multiple chef-client
packages installed at the same time (even though they overwrite each
other's files).

https://tickets.opscode.com/browse/CHEF-4657

not sure if that's related, but it probably doesn't help any...

On Thu Feb 27 13:36:45 2014, Kapil Shardha wrote:

The MSI installer platform supports "Upgrade" scenario as long as the MSI
has an unique "Product Code" and "Version Number" included. I am not sure
what chef-client MSI does or how it has been authored but I have authored
installers using WiX that support "Upgrade scenario".

This email and any accompanying documents may contain privileged or
otherwise confidential information of, and/or is the property of Education
Management Solutions, Inc. If you are not the intended recipient, please
immediately advise the sender by reply email & delete the message & any
attachments without using, copying or disclosing the contents. Thank you.

Jeppe_Nejsum_Madsen · March 3, 2014, 10:13am

Adam,

Thanks for the heads up and recipe. Any idea where this ends up in the back
log given the focus on supporting Windows in chef?

The amount of windows hackery needed here probably exceeds my capabilities.
And while I have some colleagues that might be able to get the job done, I
think we'll cop out and upgrade chef-client manually (and note we are using
hosted chef, so we are actually paying for the product which buys us a
limited amount of whining imo

/Jeppe

On Fri, Feb 28, 2014 at 6:28 PM, Adam Edwards adamed@getchef.com wrote:

Lamont is correct about the root cause -- the build always produces a new
msi, and there are no upgrade codes in the generated package. And
omnibus-updater doesn't support Windows as you've found out Jeppe.

Jeppe, great start on this. So until the scheme for handling productid's /
upgrade logic for the msi across versions is resolved, a version of
omnibus-updater that works for Windows is the way to go.

At the point in omnibus-updated at which we know the package is not
installed and you need to install it, we can create a scheduled task to run
an upgrade script of some sort after the chef-client run completes. A
script
like that would do something like the following:

net stop chef-client # so in case the service is running

Check to make sure there are no chef-client processes running
a. slightly tricky, you need to look at all processes, find each
ruby.exe
process, and make sure the arguments don't contain 'chef-client'. In newer
versions of chef-client, you may be able to look for a process lock file.

Find the existing chef-client installed product by name via wmi -- I can
send some examples if it will help

Use wmi's msi handler to uninstall all chef-clients that you find

Run the installation for the new chef-client package

If the service was originally enabled (easy to find during the
chef-client run and then pass on the cli to the scheduled task), install
the
service with chef-service-manager
a. This will also cause a chef-client run

If the service was not enabled, tell it to run chef-client. This step
should run regardless of whether the previous steps succeeded

The task should delete itself to keep things clean, alternatively, it
could delete itself only if #7 fails and configure itself to retry.

Since this would need to work on systems without powershell (because prior
to Win2k8R2 powershell is not installed by default :(), it's best to do
this
via vbscript / jscript with cscript.exe. Powershell would be much easier
though if your focus is just getting things to work in your scenario.

Note that at the next chef-client run, the correct version should be
detected and then nothing happens.

If anyone else has implemented this, we should consider that as a starting
point for a PR. Otherwise, we can work with you Jeppe to get something
together before we finally get to it in our backlog of work.

Thanks.

-Adam

-----Original Message-----
From: Lamont Granquist [mailto:lamont@opscode.com]
Sent: Thursday, February 27, 2014 6:28 PM
To: chef@lists.opscode.com
Subject: [chef] Re: RE: RE: Re: Re: Re: Upgrading chef-client on Windows
not
supported?

they're built with wix:

https://github.com/opscode/omnibus-chef/blob/master/config/software/chef-client-msi.rb

On 2/27/14 6:09 PM, Kapil Shardha wrote:

Wondering if the MSI's are being created using Visual Studio installer
projects or WiX. If that is the case, VS generates a random GUID as
PRODUCT CODE property.

Perhaps, checking the registry on a system with multiple versions of
chef-client will tell if the Product Code is different.

-Kapil

-----Original Message-----
From: Lamont Granquist [mailto:lamont@opscode.com]
Sent: Thursday, February 27, 2014 8:23 PM
To: chef@lists.opscode.com
Cc: Kapil Shardha
Subject: Re: [chef] RE: Re: Re: Re: Upgrading chef-client on Windows not
supported?

there's a known problem where i think we're using unique product codes
for
every different chef msi so that you can have multiple chef-client
packages installed at the same time (even though they overwrite each
other's files).

https://tickets.opscode.com/browse/CHEF-4657

not sure if that's related, but it probably doesn't help any...

On Thu Feb 27 13:36:45 2014, Kapil Shardha wrote:

The MSI installer platform supports "Upgrade" scenario as long as the
MSI
has an unique "Product Code" and "Version Number" included. I am not
sure
what chef-client MSI does or how it has been authored but I have
authored
installers using WiX that support "Upgrade scenario".

This email and any accompanying documents may contain privileged or
otherwise confidential information of, and/or is the property of
Education
Management Solutions, Inc. If you are not the intended recipient, please
immediately advise the sender by reply email & delete the message & any
attachments without using, copying or disclosing the contents. Thank you.

Topic		Replies	Views
Version of windows chef client Chef Infra (archive)	2	283	July 12, 2013
Re: Upgrade process Chef Infra (archive)	2	287	November 9, 2012
Compiling Chef Client source on Windows Chef Infra (archive)	14	1732	August 21, 2018
Cookbook to manage upgrades of Chef client (and from gem Chef -> Omnibus Chef?) Chef Infra (archive)	4	387	July 3, 2013
Chef 10.12.0 Release Candidate 1 Chef Infra (archive)	3	364	June 4, 2012

Upgrading chef-client on Windows not supported?

Related topics