0.9.0 -> 0.9.6 results in eventmachine reporting "no acceptor"


#1

Hello,

This morning I did a gem upgrade from chef 0.9.0 to 0.9.6. After
starting the services again, I first of all noticed that the web
template from the webui was not used. I only had raw HTML. With it, I
couldn’t login anymore either. Looking at the chef-server-webui logs,
I found this:

Any idea how to resolve this?

Ringo


#2

This is generally because something is already listening on the port. Try “lsof -nP | grep LISTEN” and kill the pid that’s listening on the port.

To that end has anyone noticed this happening when starting/stopping with runit? I’ve seen a couple times where this happens on a runit restart.

-Joe

On Jul 16, 2010, at 7:57 AM, Ringo De Smet wrote:

Hello,

This morning I did a gem upgrade from chef 0.9.0 to 0.9.6. After
starting the services again, I first of all noticed that the web
template from the webui was not used. I only had raw HTML. With it, I
couldn’t login anymore either. Looking at the chef-server-webui logs,
I found this:

http://gist.github.com/478185

Any idea how to resolve this?

Ringo

Name: Joseph A. Williams
Email: joe@joetify.com
Blog: http://www.joeandmotorboat.com/
Twitter: http://twitter.com/williamsjoe


#3

On 16 July 2010 17:10, Joe Williams joe@joetify.com wrote:

This is generally because something is already listening on the port. Try “lsof -nP | grep LISTEN” and kill the pid that’s listening on the port.

To that end has anyone noticed this happening when starting/stopping with runit? I’ve seen a couple times where this happens on a runit restart.

Tnx. I did “sv stop chef-XXX” for several chef subsystems before the
upgrade of the gems, but as you mentioned, some of the services didn’t
stop at all. All OK now after a real restart.

Ringo


#4

Hello!

On Jul 16, 2010, at 9:31 AM, Ringo De Smet wrote:

On 16 July 2010 17:10, Joe Williams joe@joetify.com wrote:

This is generally because something is already listening on the port. Try “lsof -nP | grep LISTEN” and kill the pid that’s listening on the port.

To that end has anyone noticed this happening when starting/stopping with runit? I’ve seen a couple times where this happens on a runit restart.

Tnx. I did “sv stop chef-XXX” for several chef subsystems before the
upgrade of the gems, but as you mentioned, some of the services didn’t
stop at all. All OK now after a real restart.

When started by runit, the chef-server and chef-server-webui services spawn merb master processes, which in turn start the merb workers (which by default are ‘thin’ web servers).

The merb master doesn’t always stop/start/restart the workers properly, related to a way that merb does its own internal process handling. If you send an ‘int’ signal, a la:

sudo sv int chef-server

It should behave properly for restarting the Chef Server. If you need to ensure a full stop before restarting such as when doing an upgrade, then you may need to check for running ‘merb’ processes and kill them.


Opscode, Inc
Joshua Timberman, Senior Solutions Engineer
C: 720.334.RUBY E: joshua@opscode.com


#5

Maybe there’s a better way but to work around this pesky problem I’ve the
recipe add a ‘finish’ file to kill merb:

/etc/sv/chef-server/sv-chef-server-finish.erb :

#!/bin/sh
exec 2>&1
exec kill -9 lsof -i :<%= node[:chef][:server_port] %> -t

/etc/sv/chef-server-webui/sv-chef-server-webui-finish.erb :

#!/bin/sh
exec 2>&1
exec kill -9 lsof -i :<%= node[:chef][:webui_port] %> -t

Make sure it’s chmodded to 755

On Fri, Jul 16, 2010 at 8:56 AM, Joshua Timberman joshua@opscode.comwrote:

Hello!

On Jul 16, 2010, at 9:31 AM, Ringo De Smet wrote:

On 16 July 2010 17:10, Joe Williams joe@joetify.com wrote:

This is generally because something is already listening on the port.
Try “lsof -nP | grep LISTEN” and kill the pid that’s listening on the port.

To that end has anyone noticed this happening when starting/stopping
with runit? I’ve seen a couple times where this happens on a runit restart.

Tnx. I did “sv stop chef-XXX” for several chef subsystems before the
upgrade of the gems, but as you mentioned, some of the services didn’t
stop at all. All OK now after a real restart.

When started by runit, the chef-server and chef-server-webui services spawn
merb master processes, which in turn start the merb workers (which by
default are ‘thin’ web servers).

The merb master doesn’t always stop/start/restart the workers properly,
related to a way that merb does its own internal process handling. If you
send an ‘int’ signal, a la:

sudo sv int chef-server

It should behave properly for restarting the Chef Server. If you need to
ensure a full stop before restarting such as when doing an upgrade, then you
may need to check for running ‘merb’ processes and kill them.


Opscode, Inc
Joshua Timberman, Senior Solutions Engineer
C: 720.334.RUBY E: joshua@opscode.com