Stop a chef server

Chef Infra (archive)
1

Hi,
I was trying to install, start and then stop the chef server which I installed on an AWS platform. As part of the installation, I also installed chef-manage.

Now, when I try to stop the chef-server, I see chef-processes still running which I am unable to kill. I tried these commands to stop the server -

  • chef-server-ctl stop
  • chef-manage-ctl stop

The commands executed without error, but I still see chef-specific processes on the server.
ps -ef | grep chef
root 8583 3541 0 04:20 pts/1 00:00:00 grep --color=auto chef
root 21772 1 0 04:06 ? 00:00:00 /opt/chef-manage/embedded/bin/svlogd -tt /var/log/chef-manage/worker
opscode 21776 1 0 04:06 ? 00:00:05 sidekiq 2.5.2 chef-manage [0 of 25 busy]
root 21784 1 0 04:06 ? 00:00:00 runsvdir -P /opt/chef-manage/service log: ng: unable to lock directory: /var/log/chef-manage/worker: temporary failure?svlogd: fatal: no functional log directories.?svlogd: warning: unable to lock directory: /var/log/chef-manage/worker: temporary failure?svlogd: fatal: no functional log directories.?svlogd: warning: unable to lock directory: /var/log/chef-manage/worker: temporary failure?svlogd: fatal: no functional log directories.?
root 21785 5855 0 04:06 ? 00:00:00 runsv opscode-erchef
root 21786 5855 0 04:06 ? 00:00:00 runsv opscode-chef-mover
opscode 21787 21785 0 04:06 ? 00:00:00 svlogd -tt /var/log/opscode/opscode-erchef
opscode 21788 21786 0 04:06 ? 00:00:00 svlogd -tt /var/log/opscode/opscode-chef-mover
root 21797 21791 0 04:06 ? 00:00:00 /opt/chef-manage/embedded/bin/svlogd -tt /var/log/chef-manage/redis
root 22983 21789 0 04:07 ? 00:00:00 /opt/chef-manage/embedded/bin/svlogd -tt /var/log/chef-manage/web

I also tried the the uninstall commands, but with no success -

  • chef-server-ctl stop
  • chef-manage-ctl stop
2

Chef server is run under runit, it is normal there’s still the top runit service in the process list.

If you run chef-server-ctl status you’ll see all services with a state stopped
If you stop it for a backup, there’s a chef-server-ctl backup command which is here exactly for that.

Without more details about what you’re trying to do while stopping the chef-server it’s hard to give better advice.

3

Thanks Tensibai!

Just to provide some context, I am hosting a ‘chef server’ on an AWS instance. For some reasons, I had to stop the instance and bring it back again. When I did that, I noticed some chef-services running on the server

ps -ef | grep chef
root      1896  1881  0 01:17 ?        00:00:00 runsv opscode-erchef
root      1904  1881  0 01:17 ?        00:00:00 runsv opscode-chef-mover
opscode   1920  1896  0 01:17 ?        00:00:00 svlogd -tt /var/log/opscode/opscode-erchef
opscode   1922  1904  0 01:17 ?        00:00:00 svlogd -tt /var/log/opscode/opscode-chef-mover
opscode   1928  1896  6 01:17 ?        00:00:05 /opt/opscode/embedded/service/opscode-erchef/erts-6.4/bin/beam.smp -Bd -K true -A 5 -- -root /opt/opscode/embedded/service/opscode-erchef -progname opt/opscode/embedded/service/opscode-erchef/bin/oc_erchef -- -home /var/opt/opscode/opscode-erchef -- -noshell -noshell -noinput -boot /opt/opscode/embedded/service/opscode-erchef/releases/12.8.0/oc_erchef -mode embedded -config /opt/opscode/embedded/service/opscode-erchef/sys.config -boot_var ERTS_LIB_DIR /opt/opscode/embedded/service/opscode-erchef/erts-6.4/../lib -name erchef@127.0.0.1 -setcookie erchef -smp enable -pa lib/patches -- foreground
opscode   1930  1904  4 01:17 ?        00:00:04 /opt/opscode/embedded/service/opscode-chef-mover/erts-6.4/bin/beam.smp -Bd -K true -A 5 -- -root /opt/opscode/embedded/service/opscode-chef-mover -progname opt/opscode/embedded/service/opscode-chef-mover/bin/mover -- -home /var/opt/opscode/opscode-chef-mover -- -noshell -noshell -noinput -boot /opt/opscode/embedded/service/opscode-chef-mover/releases/12.8.0/mover -mode embedded -config /opt/opscode/embedded/service/opscode-chef-mover/sys.config -boot_var ERTS_LIB_DIR /opt/opscode/embedded/service/opscode-chef-mover/erts-6.4/../lib -name mover@127.0.0.1 -setcookie mover -- foreground
497       2765  1932  0 01:17 ?        00:00:00 postgres: opscode_chef opscode_chef 127.0.0.1(49535) idle
497       2766  1932  0 01:17 ?        00:00:00 postgres: opscode_chef opscode_chef 127.0.0.1(47282) idle
497       2767  1932  0 01:17 ?        00:00:00 postgres: opscode_chef opscode_chef 127.0.0.1(43618) idle
497       2768  1932  0 01:17 ?        00:00:00 postgres: opscode_chef opscode_chef 127.0.0.1(41646) idle
497       2769  1932  0 01:17 ?        00:00:00 postgres: opscode_chef opscode_chef 127.0.0.1(54743) idle
497       2969  1932  0 01:18 ?        00:00:00 postgres: opscode_chef opscode_chef 127.0.0.1(38905) idle
497       2970  1932  0 01:18 ?        00:00:00 postgres: opscode_chef opscode_chef 127.0.0.1(57844) idle
497       2971  1932  0 01:18 ?        00:00:00 postgres: opscode_chef opscode_chef 127.0.0.1(55156) idle
497       2972  1932  0 01:18 ?        00:00:00 postgres: opscode_chef opscode_chef 127.0.0.1(33066) idle
497       2973  1932  0 01:18 ?        00:00:00 postgres: opscode_chef opscode_chef 127.0.0.1(54376) idle
497       2974  1932  0 01:18 ?        00:00:00 postgres: opscode_chef opscode_chef 127.0.0.1(55774) idle
497       2975  1932  0 01:18 ?        00:00:00 postgres: opscode_chef opscode_chef 127.0.0.1(53504) idle
497       2976  1932  0 01:18 ?        00:00:00 postgres: opscode_chef opscode_chef 127.0.0.1(37414) idle
497       2978  1932  0 01:18 ?        00:00:00 postgres: opscode_chef opscode_chef 127.0.0.1(38371) idle
497       2979  1932  0 01:18 ?        00:00:00 postgres: opscode_chef opscode_chef 127.0.0.1(45048) idle
497       2981  1932  0 01:18 ?        00:00:00 postgres: opscode_chef opscode_chef 127.0.0.1(42955) idle
497       2982  1932  0 01:18 ?        00:00:00 postgres: opscode_chef opscode_chef 127.0.0.1(59671) idle
497       2983  1932  0 01:18 ?        00:00:00 postgres: opscode_chef opscode_chef 127.0.0.1(41183) idle
497       2984  1932  0 01:18 ?        00:00:00 postgres: opscode_chef opscode_chef 127.0.0.1(44613) idle
497       2985  1932  0 01:18 ?        00:00:00 postgres: opscode_chef opscode_chef 127.0.0.1(38537) idle
497       2986  1932  0 01:18 ?        00:00:00 postgres: opscode_chef opscode_chef 127.0.0.1(55877) idle
497       2987  1932  0 01:18 ?        00:00:00 postgres: opscode_chef opscode_chef 127.0.0.1(46699) idle
497       2990  1932  0 01:18 ?        00:00:00 postgres: opscode_chef opscode_chef 127.0.0.1(57717) idle
497       2991  1932  0 01:18 ?        00:00:00 postgres: opscode_chef opscode_chef 127.0.0.1(53013) idle
497       2992  1932  0 01:18 ?        00:00:00 postgres: opscode_chef opscode_chef 127.0.0.1(43308) idle
opscode   2993  1928  0 01:18 ?        00:00:00 ruby /opt/opscode/embedded/service/opscode-erchef/lib/chef_objects-12.8.0/priv/depselector_rb/depselector.rb
opscode   2995  1928  0 01:18 ?        00:00:00 ruby /opt/opscode/embedded/service/opscode-erchef/lib/chef_objects-12.8.0/priv/depselector_rb/depselector.rb
opscode   2997  1928  0 01:18 ?        00:00:00 ruby /opt/opscode/embedded/service/opscode-erchef/lib/chef_objects-12.8.0/priv/depselector_rb/depselector.rb
opscode   2999  1928  0 01:18 ?        00:00:00 ruby /opt/opscode/embedded/service/opscode-erchef/lib/chef_objects-12.8.0/priv/depselector_rb/depselector.rb
opscode   3001  1928  0 01:18 ?        00:00:00 ruby /opt/opscode/embedded/service/opscode-erchef/lib/chef_objects-12.8.0/priv/depselector_rb/depselector.rb
root      3074  3054  0 01:19 pts/0    00:00:00 grep --color=auto chef

I was hoping to issue a single command to bring down these services. I was expecting chef-server-ctl stop to help with this, but apparently it isnt.

the output of the chef-server-ctl status command is -

run: bookshelf: (pid 1925) 257s; run: log: (pid 1918) 257s
run: nginx: (pid 1924) 257s; run: log: (pid 1914) 257s
run: oc_bifrost: (pid 1927) 257s; run: log: (pid 1921) 257s
run: oc_id: (pid 1912) 257s; run: log: (pid 1907) 257s
run: opscode-erchef: (pid 1928) 257s; run: log: (pid 1920) 257s
run: opscode-expander: (pid 1917) 257s; run: log: (pid 1908) 257s
run: opscode-solr4: (pid 1923) 257s; run: log: (pid 1913) 257s
run: postgresql: (pid 1932) 257s; run: log: (pid 1929) 257s
run: rabbitmq: (pid 1915) 257s; run: log: (pid 1909) 257s
run: redis_lb: (pid 1926) 257s; run: log: (pid 1919) 257s

I issued a ‘chef-server-ctl stop’ command for each of these services and now we have all the services down.

chef-server-ctl status
down: bookshelf: 2428s, normally up; run: log: (pid 1918) 2810s
down: nginx: 2419s, normally up; run: log: (pid 1914) 2810s
down: oc_bifrost: 2344s, normally up; run: log: (pid 1921) 2810s
down: oc_id: 2334s, normally up; run: log: (pid 1907) 2810s
down: opscode-erchef: 2327s, normally up; run: log: (pid 1920) 2810s
down: opscode-expander: 2319s, normally up; run: log: (pid 1908) 2810s
down: opscode-solr4: 4s, normally up; run: log: (pid 1913) 2810s
down: postgresql: 2457s, normally up; run: log: (pid 1929) 2810s
down: rabbitmq: 16s, normally up; run: log: (pid 1909) 2951s
down: redis_lb: 6s, normally up; run: log: (pid 1919) 2951s

However, when I grep for the chef keyword in the process list, I still see some services running.

ps -ef | grep chef
root      1896  1881  0 01:17 ?        00:00:00 runsv opscode-erchef
root      1904  1881  0 01:17 ?        00:00:00 runsv opscode-chef-mover
opscode   1920  1896  0 01:17 ?        00:00:00 svlogd -tt /var/log/opscode/opscode-erchef
opscode   1922  1904  0 01:17 ?        00:00:00 svlogd -tt /var/log/opscode/opscode-chef-mover
opscode   1930  1904  0 01:17 ?        00:00:04 /opt/opscode/embedded/service/opscode-chef-mover/erts-6.4/bin/beam.smp -Bd -K true -A 5 -- -root /opt/opscode/embedded/service/opscode-chef-mover -progname opt/opscode/embedded/service/opscode-chef-mover/bin/mover -- -home /var/opt/opscode/opscode-chef-mover -- -noshell -noshell -noinput -boot /opt/opscode/embedded/service/opscode-chef-mover/releases/12.8.0/mover -mode embedded -config /opt/opscode/embedded/service/opscode-chef-mover/sys.config -boot_var ERTS_LIB_DIR /opt/opscode/embedded/service/opscode-chef-mover/erts-6.4/../lib -name mover@127.0.0.1 -setcookie mover -- foreground
root      3836  3054  0 02:04 pts/0    00:00:00 grep --color=auto chef

If I try to kill these processes, they tend to spawn new ones in its place.

What am I missing? Is there a way to cleanly stop a chef server?

Thanks for the help!

4

When you see all services down, your chef server is stopped.

The remaining processes in the process list are runit processes (supervisor for the services).
This mailing list is not the place to write a full tutorial about runit, the doc about it is abundant on the web.

I still don’t understand why the processes did bother you at first. It sounds healthy that the services restart automatically on boot, that’s what services are supposed to do.

1 Like
5

Thanks and apologies for a delayed reply.

I was hoping to find a way to get the runit services to be down as well. Thats where the confusion was. Thanks for pointing me to the right direction. We are good now!

Mrinal