RE: Re: RE: Re: Chef server update went wrong - how to repair?


#1

Thank you! The upgrade has run halfway through, but now redis_lb times out while attempting to restart. I tried chef-server-ctl start, but that left me with the same internal error problem.

Any idea how to resolve this?

Kevin Keane

The NetTech

http://www.4nettech.com

Our values: Privacy, Liberty, Justice

See https://www.4nettech.com/corp/the-nettech-values.html

-----Original message-----
From: Mark Mzyk mmzyk@chef.io
Sent: Monday 29th June 2015 8:14
To: chef@lists.opscode.com
Subject: [chef] Re: RE: Re: Chef server update went wrong - how to repair?

Hi Kevin,

When you upgrade from the open source Chef 11 server to Chef server 12, it doesn’t remove Chef server 11 from the system. The upgrade to Chef server 12 is done next to Chef server 11 on the system, since they live at different locations in the file system. It is done this way so you can fall back to Chef server 11 if something goes wrong with the upgrade. In your case, the upgrade went fine, but it seems like you missed removing Chef server 11 from the system.

If you didn’t remove the old chef server package from the system the files will still be present on disk and this is what the upgrade process is detecting. Specifically, the upgrade process is picking up that /opt/chef-server still exists and that is why it thinks that Chef server 11 is on the system (Chef server 12 installs to /opt/opscode, which is a bit of legacy from it being based off the old enterprise code base).

So I recommend you remove the Chef server 11 package and then run the upgrade command again. It should no longer pick up Chef server 11 and will instead upgrade you from 12.0.3 to 12.1. Make sure you remove the correct package, since they have similar names.

The reason that you likely don’t realize Chef server 11 is still on the system is that all the processes for it are stopped, and on the upgrade chef-server-ctl was switched to point at Chef server 12. It’s actually still possible to access Chef server 11 if you use direct paths to the tooling.

Hopefully that clears things up and gives you a way forward. Sorry for the confusion on this.

Mark Mzyk
Chef Software Engineer

Kevin Keane Subscription

June 28, 2015 at 4:00 PM

It does not seem to give me that option. If I tell it not to upgrade, it simply aborts. And if I tell it to upgrade, it seems to go right into the upgrade from chef 11 to chef 12 (even though chef11 does not even exist on that server any more, as far as I can tell)

chef-server-ctl upgrade

Upgrading with options #<OpenStruct skip_confirmation=false, chef11_server_url=“https://localhost”, chef12_server_url=“https://localhost”, upload_threads=10, chef11_admin_client_name=“admin”, chef11_admin_client_key="/etc/chef-server/admin.pem">
Open source Chef 11 server detected.
Would you like to upgrade? [yN]
n
Aborting upgrade.

chef-server-ctl upgrade

Upgrading with options #<OpenStruct skip_confirmation=false, chef11_server_url=“https://localhost”, chef12_server_url=“https://localhost”, upload_threads=10, chef11_admin_client_name=“admin”, chef11_admin_client_key="/etc/chef-server/admin.pem">
Open source Chef 11 server detected.
Would you like to upgrade? [yN]
y
Upgrading the open source Chef 11 server.
Chef 12 short organization name?
Chef 12 full organization name?
Creating /tmp/chef11-server-data20150628-5878-13zhtja as the location to save the open source Chef 11 server data
Ensuring Chef 12 server is stopped
ok: down: bookshelf: 0s, normally up
ok: down: nginx: 0s, normally up
ok: down: oc_bifrost: 1s, normally up
ok: down: oc_id: 0s, normally up
ok: down: opscode-chef-mover: 0s, normally up
ok: down: opscode-erchef: 1s, normally up
ok: down: opscode-expander: 0s, normally up
ok: down: opscode-expander-reindexer: 0s, normally up
ok: down: opscode-solr4: 0s, normally up
ok: down: postgresql: 1s, normally up
ok: down: rabbitmq: 0s, normally up
ok: down: redis_lb: 1s, normally up
sed: can’t read /opt/chef-server/bin/wait-for-rabbit: No such file or directory
Failed to write fix to wait-for-rabbit script

Kevin Keane

The NetTech

http://www.4nettech.com

Our values: Privacy, Liberty, Justice

See https://www.4nettech.com/corp/the-nettech-values.html


#2

Output of the stacktrace around the error would be helpful Kevin, as
well as if there is any info in the logs.

  • Mark

Kevin Keane Subscription mailto:subscription@kkeane.com
June 29, 2015 at 12:47 PM

Thank you! The upgrade has run halfway through, but now redis_lb
times out while attempting to restart. I tried chef-server-ctl start,
but that left me with the same internal error problem.

Any idea how to resolve this?

Kevin Keane

The NetTech

http://www.4nettech.com

Our values: Privacy, Liberty, Justice

See https://www.4nettech.com/corp/the-nettech-values.html


#3

Which log should I be looking at?

I’m not sure if there is a stack trace, becasue it was simply a timeout, not a crash per se.

Kevin Keane

The NetTech

760-721-8339

http://www.4nettech.com

Our values: Privacy, Liberty, Justice

See https://www.4nettech.com/corp/the-nettech-values.html

-----Original message-----
From: Mark Mzyk mmzyk@chef.io
Sent: Monday 29th June 2015 12:35
To: chef@lists.opscode.com
Subject: [chef] Re: RE: Re: RE: Re: Chef server update went wrong - how to repair?

Output of the stacktrace around the error would be helpful Kevin, as well as if there is any info in the logs.

  • Mark

Kevin Keane Subscription

June 29, 2015 at 12:47 PM

Thank you! The upgrade has run halfway through, but now redis_lb times out while attempting to restart. I tried chef-server-ctl start, but that left me with the same internal error problem.

Any idea how to resolve this?

Kevin Keane

The NetTech

http://www.4nettech.com

Our values: Privacy, Liberty, Justice

See https://www.4nettech.com/corp/the-nettech-values.html


#4

The output from running the reconfigure where it failed would be
helpful, just so I can get a better idea where in the process it failed
and try and help you trace it back from there. I think I was wrong to
suggest looking at a log, as I’m not sure what log might contain
information on giving it a second thought.

  • Mark

Kevin Keane Subscription mailto:subscription@kkeane.com
June 29, 2015 at 6:50 PM

Which log should I be looking at?

I’m not sure if there is a stack trace, becasue it was simply a
timeout, not a crash per se.

Kevin Keane

The NetTech

760-721-8339

http://www.4nettech.com

Our values: Privacy, Liberty, Justice

See https://www.4nettech.com/corp/the-nettech-values.html

Kevin Keane Subscription mailto:subscription@kkeane.com
June 29, 2015 at 12:47 PM

Thank you! The upgrade has run halfway through, but now redis_lb
times out while attempting to restart. I tried chef-server-ctl start,
but that left me with the same internal error problem.

Any idea how to resolve this?

Kevin Keane

The NetTech

http://www.4nettech.com

Our values: Privacy, Liberty, Justice

See https://www.4nettech.com/corp/the-nettech-values.html