Trouble upgrading 11 to 12; can't start

While trying to upgrade chef from 11.1 to 12, I seem to have broken chef 11.

The error I get is:

Timeout waiting for Chef 11 server to start. Received exception Connection refused - connect(2) for “localhost” port 8000

Where can I find logs regarding this failure to start?

Thanks,

-Ian

For others who find this searching:

  • I found that it’s erchef that runs on port 8000.
  • I ran lsof to find that the logs are in /var/log/chef-server/erchef/erchef.log.
  • The logs there showed me that erchef was waiting for rabbitmq process to appear. Google helped me figure out why: a pid file didn’t have any contents.
  • Stopping rabbitmq fixed this: sudo /opt/chef-server/embedded/bin/rabbitmqctl stop.

The upgrade script starts it up again, and seems to be working.

One oddity here is that the command line option --chef11-data-dir is ignored: the Chef 11 export goes to /tmp. Is this a known bug?

Hi Ian,

Sorry for the really late reply on this. Stopping rabbitmq likely fixed the issue b/c both Chef 12 and Chef 11 run rabbitmq, but they run it in slightly different places - so during the upgrade the Chef 11 rabbitmq is shut down and the Chef 12 is started (the upgrade process installs Chef 12 on disk next to Chef 11 and does a data migration). If for some reason rabbit from Chef 11 fails to stop or clean up properly what you saw can result. The upgrade tries really hard to make sure this doesn’t happen, but sometimes rabbitmq just refuses to behave. Glad you found the information you needed to get around it.

I’m not aware of any bug around the chef11-data-dir. A search of the chef-server repository github issues didn’t reveal anything. A cursory glance of the code also doesn’t reveal anything to my eyes that would immediately cause an issue. I know we tested this fairly extensively. Can you provide any more information? We’re you just running the upgrade command with the option, or where you trying the multi-step individual command process where you specify each of the three upgrades steps separately (if you have no idea what this is, you’re almost certainly just running the upgrade command)? I realize this was all a while ago and you might not have relevant information any more. It’s also possible you just had a subtle typo.

If you determine it was a legitimate issue, feel free to file an issue agains the chef-server github repository, https://github.com/chef/chef-server

If you’re really motivated you can follow the code here: https://github.com/chef/chef-server/tree/master/omnibus/files/private-chef-ctl-commands. Look in upgrade.rb and follow the code path for upgrading the open source Chef 11 server. All the upgrade code lives in files in that folder and should be easy to find, since each relevant file has upgrade in it’s name somewhere.

  • Mark Mzyk
    Engineering Manager, Chef