Gateway Time-out Upgrading Chef 11 to Chef 12

Chef Infra (archive)
1

I’m trying to upgrade chef 11 server to chef 12. I’m following the
directions at https://docs.chef.io/upgrade_server.html#from-chef-server-osc,
and party way through the upgrade, apparently while it’s copying content,
it fails with a “Gateway Time-out”. See below.

ERROR: Gateway Time-out
/opt/chef-server/embedded/lib/ruby/1.9.1/net/http.rb:2789:in
stream_check': undefined methodclosed?’ for nil:NilClass (NoMethodError)
from /opt/chef-server/embedded/lib/ruby/1.9.1/net/http.rb:2709:in
read_body' from /opt/chef-server/embedded/lib/ruby/1.9.1/net/http.rb:2736:inbody’
from
/opt/chef-server/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/knife.rb:572:in
rescue in format_rest_error' from /opt/chef-server/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/knife.rb:570:informat_rest_error’
from
/opt/chef-server/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/knife.rb:553:in
humanize_http_exception' from /opt/chef-server/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/knife.rb:507:inhumanize_exception’
from
/opt/chef-server/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/knife.rb:498:in
rescue in run_with_pretty_exceptions' from /opt/chef-server/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/knife.rb:486:inrun_with_pretty_exceptions’
from
/opt/chef-server/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/knife.rb:174:in
run' from /opt/chef-server/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/lib/chef/application/knife.rb:135:inrun’
from
/opt/chef-server/embedded/lib/ruby/gems/1.9.1/gems/chef-11.12.2/bin/knife:25:in
<top (required)>' from /opt/chef-server/embedded/bin/knife:23:inload’
from /opt/chef-server/embedded/bin/knife:23:in `'
knife download failed with pid 7601 exit 1

It’s reproduceable. I had thought it was due to us having a lot of data to
copy over but I’ve realised the error happens at arbitrary points in the
process. Google has yielded nothing useful. Anyone got any ideas?

2

That’s a hard one Doug, as there isn’t much to go by in the stacktrace. It could be related to a large volume of data, as the process should pick up where you left off before, so it might be picking up and getting farther before failing again.

Have you managed to process further? Is there anything that might be unique about your environment? Depending how comfortable you are looking through code, I can point you to the upgrade code.

  • Mark Mzyk
    Chef Engineering Manager
3

I’m experiencing the same, it’s always with a random cookbook. Is grepping the source code the best we can do?

Can I do the download manually? Is there some way to resume if it fails the first time?

4

In case it could be of some help here is what I had to do to fix cookbooks after download and preparation from chef 11 and before upload to chef12. It’s a little messy as I did that “live” but I think the Readme describe properly what has been done.

5

For me, it never even finishes the download.

6

Ok, if you get here from googling, this is the solution. It’s crashing during a knife download. Knife download supports resume, but the chef-server-ctl upgrade command generates a random directory by default. So grepping through the code, I found out (it’s also in the --help if you look) that you can specify a chef11-data-directory and that means you can resume. It doesn’t help understand why you get a timeout in the first place … but meh.

chef-server-ctl upgrade --org-name name --full-org-name "Full Name" -d /tmp/chef11-data -e chef12-data --yes

Cheers,
Rob

7

Glad you got that figured out Rob. Sorry that it wasn’t more clear that if the same data is used that it can pick up where it left off. Is there a place in the docs that you think we could update that might have helped you figure this out sooner?

  • Mark Mzyk
    Chef Engineering Manager