Upgrade caused chef client to stop


#1

So last night all my chef clients seem to finally pick up the upgrade to chef 0.9.14 (I guess this is when .14 got pushed to the opscode apt repo which i’m using)

Sadly they didn’t upgrade cleanly:

[Thu, 10 Mar 2011 23:32:28 +0000] INFO: Upgrading package[chef] version from 0.9.12+dfsg-2 to 0.9.14+dfsg-1
[Thu, 10 Mar 2011 23:32:30 +0000] FATAL: SIGTERM received, stopping
[Thu, 10 Mar 2011 23:32:30 +0000] ERROR: Running exception handlers
[Thu, 10 Mar 2011 23:32:30 +0000] ERROR: Exception handlers complete

Thats it - after that point chef stopped running.

  1. Why?
  2. How do i stop it from happening again? :wink:

-ash


#2

On 11 Mar 2011, at 09:29, Ash Berlin wrote:

So last night all my chef clients seem to finally pick up the upgrade to chef 0.9.14 (I guess this is when .14 got pushed to the opscode apt repo which i’m using)

Sadly they didn’t upgrade cleanly:

[Thu, 10 Mar 2011 23:32:28 +0000] INFO: Upgrading package[chef] version from 0.9.12+dfsg-2 to 0.9.14+dfsg-1
[Thu, 10 Mar 2011 23:32:30 +0000] FATAL: SIGTERM received, stopping
[Thu, 10 Mar 2011 23:32:30 +0000] ERROR: Running exception handlers
[Thu, 10 Mar 2011 23:32:30 +0000] ERROR: Exception handlers complete

Thats it - after that point chef stopped running.

  1. Why?
  2. How do i stop it from happening again? :wink:

-ash

I just noticed that apt-get update was failing with this error:

E: dpkg was interrupted, you must manually run ‘sudo dpkg --configure -a’ to correct the problem.

Which might implies it aborted part way through the update process when it SIGTERMed the old process.

How should I be upgrading chef using chef?

-ash


#3

On Friday, March 11, 2011 at 3:56 AM, Ash Berlin wrote:
On 11 Mar 2011, at 09:29, Ash Berlin wrote:

So last night all my chef clients seem to finally pick up the upgrade to chef 0.9.14 (I guess this is when .14 got pushed to the opscode apt repo which i’m using)

Sadly they didn’t upgrade cleanly:

[Thu, 10 Mar 2011 23:32:28 +0000] INFO: Upgrading package[chef] version from 0.9.12+dfsg-2 to 0.9.14+dfsg-1
[Thu, 10 Mar 2011 23:32:30 +0000] FATAL: SIGTERM received, stopping
[Thu, 10 Mar 2011 23:32:30 +0000] ERROR: Running exception handlers
[Thu, 10 Mar 2011 23:32:30 +0000] ERROR: Exception handlers complete

Thats it - after that point chef stopped running.

  1. Why?
  2. How do i stop it from happening again? :wink:

-ash

I just noticed that apt-get update was failing with this error:

E: dpkg was interrupted, you must manually run ‘sudo dpkg --configure -a’ to correct the problem.

Which might implies it aborted part way through the update process when it SIGTERMed the old process.

How should I be upgrading chef using chef?

-ash
Right, I believe that the upgrade causes apt to restart chef-client. In order to update chef via apt with chef, you’ll need to fork off a new process that can survive the chef process being killed. You would need to do something like this:

fork do
Process.setsid

code to upgrade the package

end

Chef::Log.info(“Exiting for upgrade…”)
exit! 0

Clearly, not as nice as using the package resource, but you could maybe make a LWRP of it.


Dan DeLeo


#4

On Fri, Mar 11, 2011 at 1:29 AM, Ash Berlin ash_opscode@firemirror.com wrote:

So last night all my chef clients seem to finally pick up the upgrade to chef 0.9.14 (I guess this is when .14 got pushed to the opscode apt repo which i’m using)

Yup, the 0.9.14 debs were pushed up last night.

Sadly they didn’t upgrade cleanly:

As Dan mentioned, it’s a little tricky to have chef upgrade itself.
For most software that gets significant updates, chef included, I
wouldn’t have the system update automatically. So when using apt
packages I would use action :install, not :upgrade. This way you’re
not surprised by anything in a new release. You can use knife to
upgrade individual environments when you’re ready to test and deploy.

A few knife examples for the benefit of the list:

knife ssh ‘name:[* TO *]’ ‘sudo apt-get update ; sudo apt-get install
chef’ # all systems

knife ssh ‘environment:preprod’ ‘sudo apt-get update ; sudo apt-get
install chef’ # if you’ve set up an attribute for each enviroment

knife ssh -E preprod ‘name:[* TO *]’ ‘sudo apt-get update ; sudo
apt-get install chef’ # Chef 0.10 environments

Bryan