How to properly upgrade chef-server from 11.08

I’ve hit a known bug with the web-ui, where the show action on a cookbook results in “ERROR: undefined method `close!’ for nil:NilClass”. On a somewhat related note, the same cookbooks’ changes are not being picked up on a client when converging. Due to the error, I cannot confirm (or I don’t know how) whether or not the knife upload is actually resulting in my changes being persisted on the server, although the knife command returns success. So, I’m thinking it might be best to upgrade the server. I’ve been looking for a definitive resource on the proper means of upgrading chef server, but not finding anything very concrete.

Per the /version, I’ve got:

chef-server 11.0.8

Component Installed Version Version GUID

bookshelf 0.2.1 git:0a01f74ffd1313c4dc9bf0d236e03a871add4e01
chef-expander 11.0.0 git:14b11a96da1273b362f39ab11c411470688a8bd6
chef-gem 11.4.0
chef-pedant 1.0.3 git:15de6cd06f16ee5dee501d6aba36f4ba60162e62
chef-server-cookbooks 11.0.8
chef-server-ctl 11.0.8
chef-server-scripts 11.0.8
chef-server-webui 11.0.4 git:498097c0793e51e4f4e7df9f35ee1a3ed3282841
chef-solr 11.0.1 git:bcd45175fd402f3082e7146f94c5d571b0620434
erchef 1.2.6 git:77ade20f166367b5f0cde468e3c6066b8a327475
nginx 1.2.3 md5:0a986e60826d9e3b453dbefc36bf8f6c
postgresql 9.2.4 md5:6ee5bb53b97da7c6ad9cb0825d3300dd
preparation 11.0.8
rabbitmq 2.7.1 md5:34a5f9fb6f22e6681092443fcc80324f
runit 2.1.1 md5:8fa53ea8f71d88da9503f62793336bc3
unicorn 4.2.0
version-manifest 11.0.8

I’m on ubuntu 12.04

Are there any solid resources that explain this, or brief summary on steps involved? Pointers to good resources, or brief walk through would be much appreciated. Thanks.

Bill

Hi Bill,

go to http://www.getchef.com/chef/install/ and pick the "Chef Server"
tab. Then download the package and do what you already did when
installing it:

  • install the packet using your OS's package management utility (apt,
    yum, ..)
  • run chef-server-ctl reconfigure

Like described here: Install the Chef Infra Server

But I agree that an "Upgrading Chef Server" section the docs would be of
help. I also had to search a lot while doing the update.

Yours
Steffen

On 18/01/14 07:41, William Hatch wrote:

I've hit a known bug with the web-ui, where the show action on a cookbook results in "ERROR: undefined method `close!' for nil:NilClass". On a somewhat related note, the same cookbooks' changes are not being picked up on a client when converging. Due to the error, I cannot confirm (or I don't know how) whether or not the knife upload is actually resulting in my changes being persisted on the server, although the knife command returns success. So, I'm thinking it might be best to upgrade the server. I've been looking for a definitive resource on the proper means of upgrading chef server, but not finding anything very concrete.

Per the /version, I've got:

chef-server 11.0.8

Component Installed Version Version GUID

bookshelf 0.2.1 git:0a01f74ffd1313c4dc9bf0d236e03a871add4e01
chef-expander 11.0.0 git:14b11a96da1273b362f39ab11c411470688a8bd6
chef-gem 11.4.0
chef-pedant 1.0.3 git:15de6cd06f16ee5dee501d6aba36f4ba60162e62
chef-server-cookbooks 11.0.8
chef-server-ctl 11.0.8
chef-server-scripts 11.0.8
chef-server-webui 11.0.4 git:498097c0793e51e4f4e7df9f35ee1a3ed3282841
chef-solr 11.0.1 git:bcd45175fd402f3082e7146f94c5d571b0620434
erchef 1.2.6 git:77ade20f166367b5f0cde468e3c6066b8a327475
nginx 1.2.3 md5:0a986e60826d9e3b453dbefc36bf8f6c
postgresql 9.2.4 md5:6ee5bb53b97da7c6ad9cb0825d3300dd
preparation 11.0.8
rabbitmq 2.7.1 md5:34a5f9fb6f22e6681092443fcc80324f
runit 2.1.1 md5:8fa53ea8f71d88da9503f62793336bc3
unicorn 4.2.0
version-manifest 11.0.8

I'm on ubuntu 12.04

Are there any solid resources that explain this, or brief summary on steps involved? Pointers to good resources, or brief walk through would be much appreciated. Thanks.

Bill

Hi Bill,

The latest release of chef-server is 11.0.10. The upgrade path from 11.0.8
is as simple as Steffen mentioned:

dpkg -i PACKAGE.deb

chef-server-ctl reconfigure

You mentioned that the bug is a known bug. Without knowing the specifics of
the bug (a link to JIRA would help), it's unclear on whether an upgrade to
11.0.10 will solve the problem you are seeing. Here is the omnibus-level
diff between .8 and .10:

Of specific note is the upgrade to chef-server-webui:

Do any of the bugs in that commit message align with the bug that you are
seeing?

HTH!
Stephen

On Sat, Jan 18, 2014 at 2:32 AM, Steffen Gebert st+gmane@st-g.de wrote:

Hi Bill,

go to http://www.getchef.com/chef/install/ and pick the "Chef Server"
tab. Then download the package and do what you already did when
installing it:

  • install the packet using your OS's package management utility (apt,
    yum, ..)
  • run chef-server-ctl reconfigure

Like described here: Install the Chef Infra Server

But I agree that an "Upgrading Chef Server" section the docs would be of
help. I also had to search a lot while doing the update.

Yours
Steffen

On 18/01/14 07:41, William Hatch wrote:

I've hit a known bug with the web-ui, where the show action on a
cookbook results in "ERROR: undefined method `close!' for nil:NilClass". On
a somewhat related note, the same cookbooks' changes are not being picked
up on a client when converging. Due to the error, I cannot confirm (or I
don't know how) whether or not the knife upload is actually resulting in my
changes being persisted on the server, although the knife command returns
success. So, I'm thinking it might be best to upgrade the server. I've been
looking for a definitive resource on the proper means of upgrading chef
server, but not finding anything very concrete.

Per the /version, I've got:

chef-server 11.0.8

Component Installed Version Version GUID


bookshelf 0.2.1
git:0a01f74ffd1313c4dc9bf0d236e03a871add4e01
chef-expander 11.0.0
git:14b11a96da1273b362f39ab11c411470688a8bd6
chef-gem 11.4.0
chef-pedant 1.0.3
git:15de6cd06f16ee5dee501d6aba36f4ba60162e62
chef-server-cookbooks 11.0.8
chef-server-ctl 11.0.8
chef-server-scripts 11.0.8
chef-server-webui 11.0.4
git:498097c0793e51e4f4e7df9f35ee1a3ed3282841
chef-solr 11.0.1
git:bcd45175fd402f3082e7146f94c5d571b0620434
erchef 1.2.6
git:77ade20f166367b5f0cde468e3c6066b8a327475
nginx 1.2.3
md5:0a986e60826d9e3b453dbefc36bf8f6c
postgresql 9.2.4
md5:6ee5bb53b97da7c6ad9cb0825d3300dd
preparation 11.0.8
rabbitmq 2.7.1
md5:34a5f9fb6f22e6681092443fcc80324f
runit 2.1.1
md5:8fa53ea8f71d88da9503f62793336bc3
unicorn 4.2.0
version-manifest 11.0.8

I'm on ubuntu 12.04

Are there any solid resources that explain this, or brief summary on
steps involved? Pointers to good resources, or brief walk through would be
much appreciated. Thanks.

Bill

--
Stephen Delano
Software Development Engineer
Opscode, Inc.
1008 Western Avenue
Suite 601
Seattle, WA 98104

Thanks Stephen;

Yes, that's the same git hash I found last night, although it doesn't seem to mention the web-ui bug in the commit, but same revision from the issue I found here: https://tickets.opscode.com/browse/CHEF-3952

I'm upgrading now; thanks for the pointers and help Steffen and Stephen!

Bill

On Jan 18, 2014, at 12:24 PM, Stephen Delano stephen@opscode.com wrote:

Hi Bill,

The latest release of chef-server is 11.0.10. The upgrade path from 11.0.8 is as simple as Steffen mentioned:

dpkg -i PACKAGE.deb
chef-server-ctl reconfigure

You mentioned that the bug is a known bug. Without knowing the specifics of the bug (a link to JIRA would help), it's unclear on whether an upgrade to 11.0.10 will solve the problem you are seeing. Here is the omnibus-level diff between .8 and .10: Comparing 11.0.8...11.0.10 · chef-boneyard/omnibus-chef-server · GitHub

Of specific note is the upgrade to chef-server-webui: chef-server-webui 11.0.10 · chef-boneyard/omnibus-chef-server@60dc79b · GitHub

Do any of the bugs in that commit message align with the bug that you are seeing?

HTH!
Stephen

On Sat, Jan 18, 2014 at 2:32 AM, Steffen Gebert st+gmane@st-g.de wrote:
Hi Bill,

go to http://www.getchef.com/chef/install/ and pick the "Chef Server"
tab. Then download the package and do what you already did when
installing it:

  • install the packet using your OS's package management utility (apt,
    yum, ..)
  • run chef-server-ctl reconfigure

Like described here: Install the Chef Infra Server

But I agree that an "Upgrading Chef Server" section the docs would be of
help. I also had to search a lot while doing the update.

Yours
Steffen

On 18/01/14 07:41, William Hatch wrote:

I've hit a known bug with the web-ui, where the show action on a cookbook results in "ERROR: undefined method `close!' for nil:NilClass". On a somewhat related note, the same cookbooks' changes are not being picked up on a client when converging. Due to the error, I cannot confirm (or I don't know how) whether or not the knife upload is actually resulting in my changes being persisted on the server, although the knife command returns success. So, I'm thinking it might be best to upgrade the server. I've been looking for a definitive resource on the proper means of upgrading chef server, but not finding anything very concrete.

Per the /version, I've got:

chef-server 11.0.8

Component Installed Version Version GUID

bookshelf 0.2.1 git:0a01f74ffd1313c4dc9bf0d236e03a871add4e01
chef-expander 11.0.0 git:14b11a96da1273b362f39ab11c411470688a8bd6
chef-gem 11.4.0
chef-pedant 1.0.3 git:15de6cd06f16ee5dee501d6aba36f4ba60162e62
chef-server-cookbooks 11.0.8
chef-server-ctl 11.0.8
chef-server-scripts 11.0.8
chef-server-webui 11.0.4 git:498097c0793e51e4f4e7df9f35ee1a3ed3282841
chef-solr 11.0.1 git:bcd45175fd402f3082e7146f94c5d571b0620434
erchef 1.2.6 git:77ade20f166367b5f0cde468e3c6066b8a327475
nginx 1.2.3 md5:0a986e60826d9e3b453dbefc36bf8f6c
postgresql 9.2.4 md5:6ee5bb53b97da7c6ad9cb0825d3300dd
preparation 11.0.8
rabbitmq 2.7.1 md5:34a5f9fb6f22e6681092443fcc80324f
runit 2.1.1 md5:8fa53ea8f71d88da9503f62793336bc3
unicorn 4.2.0
version-manifest 11.0.8

I'm on ubuntu 12.04

Are there any solid resources that explain this, or brief summary on steps involved? Pointers to good resources, or brief walk through would be much appreciated. Thanks.

Bill

--
Stephen Delano
Software Development Engineer
Opscode, Inc.
1008 Western Avenue
Suite 601
Seattle, WA 98104

Ok, did the following, which seems to be the common approach, and what you suggested:

wget https://opscode-omnibus-packages.s3.amazonaws.com/ubuntu/12.04/x86_64/chef-server_11.0.10-1.ubuntu.12.04_amd64.deb

Then installed using the system package manager:

sudo dpkg -i chef-server_11.0.10-1.ubuntu.12.04_amd64.deb

And then ran the chef-server-ctl reconfigure command like:

sudo chef-server-ctl reconfigure, which hit an error on solr; here's the output:

Recipe: chef-server::default

  • file[/etc/chef-server/chef-server-running.json] action create
    • update content in file /etc/chef-server/chef-server-running.json from 9a16b2 to 25d78a
      --- /etc/chef-server/chef-server-running.json 2013-10-21 02:33:17.167171003 +0000
      +++ /tmp/.chef-server-running.json20140118-23607-8ra0ps 2014-01-18 20:06:17.503171012 +0000
      @@ -45,7 +45,7 @@
      "max_commit_docs": 1000,
      "commit_interval": 60000,
      "poll_seconds": 20,
      •  "command": "java -Xmx413M -Xms413M -XX:NewSize=41M -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=8086 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dsolr.data.dir=/var/opt/chef-server/chef-solr/data -Dsolr.solr.home=/var/opt/chef-server/chef-solr/home -server -jar '/var/opt/chef-server/chef-solr/jetty/start.jar'"
        
      •  "command": "java -Xmx413M -Xms413M -XX:NewSize=41M -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -Dsolr.data.dir=/var/opt/chef-server/chef-solr/data -Dsolr.solr.home=/var/opt/chef-server/chef-solr/home -server -jar '/var/opt/chef-server/chef-solr/jetty/start.jar'"
        
        },
        "chef-expander": {
        "enable": true,

Recipe: chef-server::chef-solr

  • service[chef-solr] action restart
    ================================================================================
    Error executing action restart on resource 'service[chef-solr]'
    ================================================================================

Mixlib::ShellOut::ShellCommandFailed

Expected process to exit with [0], but received '1'
---- Begin output of /opt/chef-server/embedded/bin/chpst -u root /opt/chef-server/embedded/bin/sv restart /opt/chef-server/service/chef-solr ----
STDOUT: timeout: down: /opt/chef-server/service/chef-solr: 1s, normally up, want up
STDERR:
---- End output of /opt/chef-server/embedded/bin/chpst -u root /opt/chef-server/embedded/bin/sv restart /opt/chef-server/service/chef-solr ----
Ran /opt/chef-server/embedded/bin/chpst -u root /opt/chef-server/embedded/bin/sv restart /opt/chef-server/service/chef-solr returned 1

Resource Declaration:

In /opt/chef-server/embedded/cookbooks/runit/definitions/runit_service.rb

172: service params[:name] do
173: control_cmd = node[:runit][:sv_bin]
174: if params[:owner]
175: control_cmd = "#{node[:runit][:chpst_bin]} -u #{params[:owner]} #{control_cmd}"
176: end
177: provider Chef::Provider::Service::Simple

Compiled Resource:

Declared in /opt/chef-server/embedded/cookbooks/runit/definitions/runit_service.rb:172:in `block in from_file'

service("chef-solr") do
params {:directory=>"/opt/chef-server/sv", :only_if=>false, :finish_script=>false, :control=>, :run_restart=>true, :active_directory=>"/opt/chef-server/service", :init_script_template=>nil, :owner=>"root", :group=>"root", :template_name=>"chef-solr", :start_command=>"start", :stop_command=>"stop", :restart_command=>"restart", :status_command=>"status", :options=>{:log_directory=>"/var/log/chef-server/chef-solr", :directory=>nil, :only_if=>false, :finish_script=>false, :control=>, :run_restart=>true, :active_directory=>nil, :init_script_template=>nil, :owner=>"root", :group=>"root", :template_name=>nil, :start_command=>"start", :stop_command=>"stop", :restart_command=>"restart", :status_command=>"status", :options=>{}, :env=>{}, :action=>:enable, :down=>false}, :env=>{}, :action=>:enable, :down=>false, :name=>"chef-solr"}
provider Chef::Provider::Service::Simple
action [:nothing]
supports {:restart=>true, :status=>true}
retries 0
retry_delay 2
service_name "chef-solr"
pattern "chef-solr"
start_command "/opt/chef-server/embedded/bin/chpst -u root /opt/chef-server/embedded/bin/sv start /opt/chef-server/service/chef-solr"
stop_command "/opt/chef-server/embedded/bin/chpst -u root /opt/chef-server/embedded/bin/sv stop /opt/chef-server/service/chef-solr"
status_command "/opt/chef-server/embedded/bin/chpst -u root /opt/chef-server/embedded/bin/sv status /opt/chef-server/service/chef-solr"
restart_command "/opt/chef-server/embedded/bin/chpst -u root /opt/chef-server/embedded/bin/sv restart /opt/chef-server/service/chef-solr"
startup_type :automatic
cookbook_name :"chef-server"
recipe_name "chef-solr"
end

Recipe: chef-server::bookshelf

  • service[bookshelf] action restart
    • restart service service[bookshelf]

Recipe: chef-server::erchef

  • service[erchef] action restart
    • restart service service[erchef]

[2014-01-18T20:06:27+00:00] ERROR: Running exception handlers
[2014-01-18T20:06:27+00:00] ERROR: Exception handlers complete
[2014-01-18T20:06:27+00:00] FATAL: Stacktrace dumped to /opt/chef-server/embedded/cookbooks/cache/chef-stacktrace.out
Chef Client failed. 14 resources updated
[2014-01-18T20:06:28+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

the rest of the services restarted properly, but my server is still showing as running 11.0.8, and hitting the cookbook show link still results in the error.

Any ideas?

Bill

On Jan 18, 2014, at 5:32 AM, Steffen Gebert st+gmane@st-g.de wrote:

Hi Bill,

go to http://www.getchef.com/chef/install/ and pick the "Chef Server"
tab. Then download the package and do what you already did when
installing it:

  • install the packet using your OS's package management utility (apt,
    yum, ..)
  • run chef-server-ctl reconfigure

Like described here: Install the Chef Infra Server

But I agree that an "Upgrading Chef Server" section the docs would be of
help. I also had to search a lot while doing the update.

Yours
Steffen

On 18/01/14 07:41, William Hatch wrote:

I've hit a known bug with the web-ui, where the show action on a cookbook results in "ERROR: undefined method `close!' for nil:NilClass". On a somewhat related note, the same cookbooks' changes are not being picked up on a client when converging. Due to the error, I cannot confirm (or I don't know how) whether or not the knife upload is actually resulting in my changes being persisted on the server, although the knife command returns success. So, I'm thinking it might be best to upgrade the server. I've been looking for a definitive resource on the proper means of upgrading chef server, but not finding anything very concrete.

Per the /version, I've got:

chef-server 11.0.8

Component Installed Version Version GUID

bookshelf 0.2.1 git:0a01f74ffd1313c4dc9bf0d236e03a871add4e01
chef-expander 11.0.0 git:14b11a96da1273b362f39ab11c411470688a8bd6
chef-gem 11.4.0
chef-pedant 1.0.3 git:15de6cd06f16ee5dee501d6aba36f4ba60162e62
chef-server-cookbooks 11.0.8
chef-server-ctl 11.0.8
chef-server-scripts 11.0.8
chef-server-webui 11.0.4 git:498097c0793e51e4f4e7df9f35ee1a3ed3282841
chef-solr 11.0.1 git:bcd45175fd402f3082e7146f94c5d571b0620434
erchef 1.2.6 git:77ade20f166367b5f0cde468e3c6066b8a327475
nginx 1.2.3 md5:0a986e60826d9e3b453dbefc36bf8f6c
postgresql 9.2.4 md5:6ee5bb53b97da7c6ad9cb0825d3300dd
preparation 11.0.8
rabbitmq 2.7.1 md5:34a5f9fb6f22e6681092443fcc80324f
runit 2.1.1 md5:8fa53ea8f71d88da9503f62793336bc3
unicorn 4.2.0
version-manifest 11.0.8

I'm on ubuntu 12.04

Are there any solid resources that explain this, or brief summary on steps involved? Pointers to good resources, or brief walk through would be much appreciated. Thanks.

Bill

stephen@opscode.com writes:

Hi Bill,

The latest release of chef-server is 11.0.10. The upgrade path from 11.0.8
is as simple as Steffen mentioned:

dpkg -i PACKAGE.deb

chef-server-ctl reconfigure

I would recommend stopping all services before upgrading:

chef-server-ctl stop
dpkg -i PACKAGE.deb
chef-server-ctl reconfigure
chef-server-ctl start
  • seth

Thanks Seth, I tried that, but now I get errors during reconfigure pertaining to postgres not running; trace below:

  • execute[/opt/chef-server/embedded/bin/createdb -T template0 -E UTF-8 opscode_chef] action run
    ================================================================================
    Error executing action run on resource 'execute[/opt/chef-server/embedded/bin/createdb -T template0 -E UTF-8 opscode_chef]'
    ================================================================================

Mixlib::ShellOut::ShellCommandFailed

Expected process to exit with [0], but received '1'
---- Begin output of /opt/chef-server/embedded/bin/createdb -T template0 -E UTF-8 opscode_chef ----
STDOUT:
STDERR: createdb: could not connect to database template1: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
---- End output of /opt/chef-server/embedded/bin/createdb -T template0 -E UTF-8 opscode_chef ----
Ran /opt/chef-server/embedded/bin/createdb -T template0 -E UTF-8 opscode_chef returned 1

Resource Declaration:

In /opt/chef-server/embedded/cookbooks/chef-server/recipes/postgresql.rb

141: execute "/opt/chef-server/embedded/bin/createdb -T template0 -E UTF-8 opscode_chef" do
142: user node['chef_server']['postgresql']['username']
143: not_if database_exists
144: retries 30
145: notifies :run, "execute[migrate_database]", :immediately
146: end
147:

Compiled Resource:

Declared in /opt/chef-server/embedded/cookbooks/chef-server/recipes/postgresql.rb:141:in `from_file'

execute("/opt/chef-server/embedded/bin/createdb -T template0 -E UTF-8 opscode_chef") do
action "run"
retries 0
retry_delay 2
command "/opt/chef-server/embedded/bin/createdb -T template0 -E UTF-8 opscode_chef"
backup 5
returns 0
user "opscode-pgsql"
cookbook_name :"chef-server"
recipe_name "postgresql"
not_if "/opt/chef-server/embedded/bin/chpst -u opscode-pgsql /opt/chef-server/embedded/bin/psql -d 'template1' -c 'select datname from pg_database' -x|grep opscode_chef"
end

[2014-01-18T21:28:28+00:00] ERROR: Running exception handlers
[2014-01-18T21:28:28+00:00] ERROR: Exception handlers complete
[2014-01-18T21:28:28+00:00] FATAL: Stacktrace dumped to /opt/chef-server/embedded/cookbooks/cache/chef-stacktrace.out
Chef Client failed. 2 resources updated
[2014-01-18T21:28:28+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

And, I'm still on 11.0.8 after starting again.

Bill

On Jan 18, 2014, at 3:30 PM, Seth Falcon seth@getchef.com wrote:

stephen@opscode.com writes:

Hi Bill,

The latest release of chef-server is 11.0.10. The upgrade path from 11.0.8
is as simple as Steffen mentioned:

dpkg -i PACKAGE.deb

chef-server-ctl reconfigure

I would recommend stopping all services before upgrading:

chef-server-ctl stop
dpkg -i PACKAGE.deb
chef-server-ctl reconfigure
chef-server-ctl start

  • seth

Tried running reconfigure again after starting, after the failed attempt, and it completed successfully. /version now shows as 11.0.10, and the web-ui bug is no longer present. Yay! Thanks for all the help and info!

Bill

On Jan 18, 2014, at 4:31 PM, William Hatch wkhatch@me.com wrote:

Thanks Seth, I tried that, but now I get errors during reconfigure pertaining to postgres not running; trace below:

  • execute[/opt/chef-server/embedded/bin/createdb -T template0 -E UTF-8 opscode_chef] action run
    ================================================================================
    Error executing action run on resource 'execute[/opt/chef-server/embedded/bin/createdb -T template0 -E UTF-8 opscode_chef]'
    ================================================================================

Mixlib::ShellOut::ShellCommandFailed

Expected process to exit with [0], but received '1'
---- Begin output of /opt/chef-server/embedded/bin/createdb -T template0 -E UTF-8 opscode_chef ----
STDOUT:
STDERR: createdb: could not connect to database template1: could not connect to server: No such file or directory
Is the server running locally and accepting
connections on Unix domain socket "/tmp/.s.PGSQL.5432"?
---- End output of /opt/chef-server/embedded/bin/createdb -T template0 -E UTF-8 opscode_chef ----
Ran /opt/chef-server/embedded/bin/createdb -T template0 -E UTF-8 opscode_chef returned 1

Resource Declaration:

In /opt/chef-server/embedded/cookbooks/chef-server/recipes/postgresql.rb

141: execute "/opt/chef-server/embedded/bin/createdb -T template0 -E UTF-8 opscode_chef" do
142: user node['chef_server']['postgresql']['username']
143: not_if database_exists
144: retries 30
145: notifies :run, "execute[migrate_database]", :immediately
146: end
147:

Compiled Resource:

Declared in /opt/chef-server/embedded/cookbooks/chef-server/recipes/postgresql.rb:141:in `from_file'

execute("/opt/chef-server/embedded/bin/createdb -T template0 -E UTF-8 opscode_chef") do
action "run"
retries 0
retry_delay 2
command "/opt/chef-server/embedded/bin/createdb -T template0 -E UTF-8 opscode_chef"
backup 5
returns 0
user "opscode-pgsql"
cookbook_name :"chef-server"
recipe_name "postgresql"
not_if "/opt/chef-server/embedded/bin/chpst -u opscode-pgsql /opt/chef-server/embedded/bin/psql -d 'template1' -c 'select datname from pg_database' -x|grep opscode_chef"
end

[2014-01-18T21:28:28+00:00] ERROR: Running exception handlers
[2014-01-18T21:28:28+00:00] ERROR: Exception handlers complete
[2014-01-18T21:28:28+00:00] FATAL: Stacktrace dumped to /opt/chef-server/embedded/cookbooks/cache/chef-stacktrace.out
Chef Client failed. 2 resources updated
[2014-01-18T21:28:28+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)

And, I'm still on 11.0.8 after starting again.

Bill

On Jan 18, 2014, at 3:30 PM, Seth Falcon seth@getchef.com wrote:

stephen@opscode.com writes:

Hi Bill,

The latest release of chef-server is 11.0.10. The upgrade path from 11.0.8
is as simple as Steffen mentioned:

dpkg -i PACKAGE.deb

chef-server-ctl reconfigure

I would recommend stopping all services before upgrading:

chef-server-ctl stop
dpkg -i PACKAGE.deb
chef-server-ctl reconfigure
chef-server-ctl start

  • seth

Apologies on the upgrade procedures. Like Seth said, stopping the running
services should be a best practice for upgrading.

It appears that the restart command to Solr failed with a timeout during
your reconfigure. It could very well be that the restart command eventually
completed, but if search isn't working for you then it might be beneficial
to tail the Solr logs to see what happened during the restart. If you have
a large infrastructure, it's not uncommon for Solr to take a while to stop
cleanly before attempting a restart.

As it pertains to the server version, what leads you to believe that 11.0.8
is still running?

Also, since the reconfigure failed midway through and didn't completely
reconfigure all of the services, it would be a good idea to try running the
reconfigure to completion with all of the services down:

chef-server-ctl stop

chef-server-ctl reconfigure

chef-server-ctl start

-Stephen

On Sat, Jan 18, 2014 at 12:30 PM, Seth Falcon seth@getchef.com wrote:

stephen@opscode.com writes:

Hi Bill,

The latest release of chef-server is 11.0.10. The upgrade path from
11.0.8
is as simple as Steffen mentioned:

dpkg -i PACKAGE.deb

chef-server-ctl reconfigure

I would recommend stopping all services before upgrading:

chef-server-ctl stop
dpkg -i PACKAGE.deb
chef-server-ctl reconfigure
chef-server-ctl start
  • seth

--
Stephen Delano
Software Development Engineer
Opscode, Inc.
1008 Western Avenue
Suite 601
Seattle, WA 98104

Hi Bill,

Indeed, it appears that the reconfigure needs to connect to running
services (psql) to determine if it needs to create a database, and also to
create one.

You're going to need to run the reconfigure with the services online,
apologies for the miscommunication. Before attempting another reconfigure,
can you see what the state of the chef-solr service is? If it's up and
running again correctly, it should not receive a restart command on the
next reconfigure (no config change) and you should be able to proceed past
the point at which the run had failed the first time.

Again, what data paint are you using to determine what version of the chef
server is running?

On Sat, Jan 18, 2014 at 1:36 PM, Stephen Delano stephen@opscode.com wrote:

Apologies on the upgrade procedures. Like Seth said, stopping the running
services should be a best practice for upgrading.

It appears that the restart command to Solr failed with a timeout during
your reconfigure. It could very well be that the restart command eventually
completed, but if search isn't working for you then it might be beneficial
to tail the Solr logs to see what happened during the restart. If you have
a large infrastructure, it's not uncommon for Solr to take a while to stop
cleanly before attempting a restart.

As it pertains to the server version, what leads you to believe that
11.0.8 is still running?

Also, since the reconfigure failed midway through and didn't completely
reconfigure all of the services, it would be a good idea to try running the
reconfigure to completion with all of the services down:

chef-server-ctl stop

chef-server-ctl reconfigure

chef-server-ctl start

-Stephen

On Sat, Jan 18, 2014 at 12:30 PM, Seth Falcon seth@getchef.com wrote:

stephen@opscode.com writes:

Hi Bill,

The latest release of chef-server is 11.0.10. The upgrade path from
11.0.8
is as simple as Steffen mentioned:

dpkg -i PACKAGE.deb

chef-server-ctl reconfigure

I would recommend stopping all services before upgrading:

chef-server-ctl stop
dpkg -i PACKAGE.deb
chef-server-ctl reconfigure
chef-server-ctl start
  • seth

--
Stephen Delano
Software Development Engineer
Opscode, Inc.
1008 Western Avenue
Suite 601
Seattle, WA 98104

--
Stephen Delano
Software Development Engineer
Opscode, Inc.
1008 Western Avenue
Suite 601
Seattle, WA 98104

On Jan 18, 2014, at 4:42 PM, Stephen Delano stephen@opscode.com wrote:

Again, what data paint are you using to determine what version of the chef server is running?

I just hit the /version path in the web-ui, which now returns this:

chef-server 11.0.10

Component Installed Version Version GUID

bookshelf 0.2.1 git:0a01f74ffd1313c4dc9bf0d236e03a871add4e01
bundler 1.1.5
cacerts 2012.12.19-1 md5:349ba2d6964db9ca558c9e1daf38e428
chef-expander 11.0.0 git:14b11a96da1273b362f39ab11c411470688a8bd6
chef-gem 11.6.0
chef-pedant 1.0.3 git:15de6cd06f16ee5dee501d6aba36f4ba60162e62
chef-server-cookbooks 11.0.10
chef-server-ctl 11.0.10
chef-server-scripts 11.0.10
chef-server-webui 11.0.10 git:3334f2a577f1cc792b3ca960508bc7e4ac12ee9d
chef-solr 11.0.1 git:bcd45175fd402f3082e7146f94c5d571b0620434
curl 7.23.1 md5:8e23151f569fb54afef093ac0695077d
erchef 1.2.6 git:77ade20f166367b5f0cde468e3c6066b8a327475
erlang R15B02 md5:ccbe5e032a2afe2390de8913bfe737a1
jre 7u3-b04 md5:3d3e206cea84129f1daa8e62bf656a28
libedit 20120601-3.0 md5:e50f6a7afb4de00c81650f7b1a0f5aea
libgcc 11.0.10
libiconv 1.14 md5:e34509b1623cec449dfeb73d7ce9c6c6
libxml2 2.7.8 md5:8127a65e8c3b08856093099b52599c86
libxslt 1.1.26 md5:e61d0364a30146aaa3001296f853b2b9
libyaml 0.1.4 md5:36c852831d02cf90508c29852361d01b
ncurses 5.9 md5:8cb9c412e5f2d96bc6f459aa8c6282a1
nginx 1.4.4 md5:5dfaba1cbeae9087f3949860a02caa9f
omnibus-ctl 0.0.7 git:0dae72b0f55f804294e004632ffaea4418d094a5
openssl 1.0.1e md5:66bf6f10f060d561929de96f9dfe5b8c
pcre 8.31 md5:fab1bb3b91a4c35398263a5c1e0858c1
popt 1.16 md5:3743beefa3dd6247a73f8f7a32c14c33
postgresql 9.2.4 md5:6ee5bb53b97da7c6ad9cb0825d3300dd
preparation 11.0.10
rabbitmq 2.7.1 md5:34a5f9fb6f22e6681092443fcc80324f
rebar retry_use_source git:f21561caf3400a78ee5c17e2032292896fa021ab
rsync 3.0.9 md5:5ee72266fe2c1822333c407e1761b92b
ruby 1.9.3-p484 md5:8ac0dee72fe12d75c8b2d0ef5d0c2968
rubygems 1.8.24 md5:3a555b9d579f6a1a1e110628f5110c6b
runit 2.1.1 md5:8fa53ea8f71d88da9503f62793336bc3
unicorn 4.2.0
version-manifest 11.0.10
zlib 1.2.6 md5:618e944d7c7cd6521551e30b32322f4a

Regarding the solr issues, it looks like after running reconfigure with the service stopped, I hit the postgres related error, and then started chef-server again using ...ctl, then reran reconfigure again and everything was fine. I'm sure it's a bit unorthodox, but it got the job done. Just to be clear, you're saying that perhaps the best option would have been to retry reconfigure again after hitting the initial solr related error, but without first stopping? Thanks again, Stephen.

Bill,

Just to be clear, you're saying that perhaps the best option would have
been to retry reconfigure again after hitting the initial solr related
error, but without first stopping?

Correct, or at least waited until the Solr service had completely restarted
itself. You would have been able to tell by using chef-server-ctl status chef-solr. Anyways, I'm glad to hear that you've got everything working
now.

Cheers!

On Sat, Jan 18, 2014 at 2:03 PM, William Hatch wkhatch@me.com wrote:

On Jan 18, 2014, at 4:42 PM, Stephen Delano stephen@opscode.com wrote:

Again, what data paint are you using to determine what version of the
chef server is running?

I just hit the /version path in the web-ui, which now returns this:

chef-server 11.0.10

Component Installed Version Version GUID


bookshelf 0.2.1
git:0a01f74ffd1313c4dc9bf0d236e03a871add4e01
bundler 1.1.5
cacerts 2012.12.19-1
md5:349ba2d6964db9ca558c9e1daf38e428
chef-expander 11.0.0
git:14b11a96da1273b362f39ab11c411470688a8bd6
chef-gem 11.6.0
chef-pedant 1.0.3
git:15de6cd06f16ee5dee501d6aba36f4ba60162e62
chef-server-cookbooks 11.0.10
chef-server-ctl 11.0.10
chef-server-scripts 11.0.10
chef-server-webui 11.0.10
git:3334f2a577f1cc792b3ca960508bc7e4ac12ee9d
chef-solr 11.0.1
git:bcd45175fd402f3082e7146f94c5d571b0620434
curl 7.23.1
md5:8e23151f569fb54afef093ac0695077d
erchef 1.2.6
git:77ade20f166367b5f0cde468e3c6066b8a327475
erlang R15B02
md5:ccbe5e032a2afe2390de8913bfe737a1
jre 7u3-b04
md5:3d3e206cea84129f1daa8e62bf656a28
libedit 20120601-3.0
md5:e50f6a7afb4de00c81650f7b1a0f5aea
libgcc 11.0.10
libiconv 1.14
md5:e34509b1623cec449dfeb73d7ce9c6c6
libxml2 2.7.8
md5:8127a65e8c3b08856093099b52599c86
libxslt 1.1.26
md5:e61d0364a30146aaa3001296f853b2b9
libyaml 0.1.4
md5:36c852831d02cf90508c29852361d01b
ncurses 5.9
md5:8cb9c412e5f2d96bc6f459aa8c6282a1
nginx 1.4.4
md5:5dfaba1cbeae9087f3949860a02caa9f
omnibus-ctl 0.0.7
git:0dae72b0f55f804294e004632ffaea4418d094a5
openssl 1.0.1e
md5:66bf6f10f060d561929de96f9dfe5b8c
pcre 8.31
md5:fab1bb3b91a4c35398263a5c1e0858c1
popt 1.16
md5:3743beefa3dd6247a73f8f7a32c14c33
postgresql 9.2.4
md5:6ee5bb53b97da7c6ad9cb0825d3300dd
preparation 11.0.10
rabbitmq 2.7.1
md5:34a5f9fb6f22e6681092443fcc80324f
rebar retry_use_source
git:f21561caf3400a78ee5c17e2032292896fa021ab
rsync 3.0.9
md5:5ee72266fe2c1822333c407e1761b92b
ruby 1.9.3-p484
md5:8ac0dee72fe12d75c8b2d0ef5d0c2968
rubygems 1.8.24
md5:3a555b9d579f6a1a1e110628f5110c6b
runit 2.1.1
md5:8fa53ea8f71d88da9503f62793336bc3
unicorn 4.2.0
version-manifest 11.0.10
zlib 1.2.6
md5:618e944d7c7cd6521551e30b32322f4a

Regarding the solr issues, it looks like after running reconfigure with
the service stopped, I hit the postgres related error, and then started
chef-server again using ...ctl, then reran reconfigure again and everything
was fine. I'm sure it's a bit unorthodox, but it got the job done. Just to
be clear, you're saying that perhaps the best option would have been to
retry reconfigure again after hitting the initial solr related error, but
without first stopping? Thanks again, Stephen.

--
Stephen Delano
Software Development Engineer
Opscode, Inc.
1008 Western Avenue
Suite 601
Seattle, WA 98104