We have one host that is being testy while getting itself
set up with Chef 11.4.0. Any ideas would be welcome. Using
–log_level debug is not proving helpful.
240 others have self-registered via this method just fine.
/etc/chef/validation.pem exists and is correct.
/etc/chef/boot.json file:
{
"run_list": ["role[rcf-basic-rhel]"]
}
/etc/chef/client.rb we drop in place before the 1st run:
log_level :auto
log_location STDOUT
chef_server_url "https://cm-master.our.org"
validation_client_name "chef-validator"
# Using default node name (fqdn)
http_proxy "http://gatekeeper.our.org"
https_proxy "http://gatekeeper.our.org"
file_backup_path "/var/chef/backup"
file_cache_path "/var/chef/cache"
cache_options({ :path => "/var/chef/cache/checksums", :skip_expires
=> true })
# set the proxy env variable so rubygems works correctly
ENV['http_proxy'] = "http://gatekeeper.our.org"
ENV['HTTP_PROXY'] = "http://gatekeeper.our.org"
Ohai::Config[:disabled_plugins] = ["passwd"]
The 1st run on the node and its failure:
chef-client --log_level debug --environment production -j
/etc/chef/boot.json
…
[2013-07-08T15:15:09-04:00] INFO: Client key /etc/chef/client.pem is not
present - registering
[2013-07-08T15:15:09-04:00] DEBUG: Signing the request as chef-validator
[2013-07-08T15:15:09-04:00] DEBUG: Using gatekeeper.our.org:80 for proxy
[2013-07-08T15:15:09-04:00] DEBUG: Sending HTTP Request via POST to
cm-master.our.org:443/clients
[2013-07-08T15:15:09-04:00] FATAL: Stacktrace dumped to
/var/chef/cache/chef-stacktrace.out
[2013-07-08T15:15:09-04:00] DEBUG: NoMethodError: undefined method
closed?' for nil:NilClass /opt/chef/embedded/lib/ruby/1.9.1/net/http.rb:2788:instream_check’
/opt/chef/embedded/lib/ruby/1.9.1/net/http.rb:2708:in read_body' /opt/chef/embedded/lib/ruby/1.9.1/net/http.rb:2735:inbody’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/formatters/error_inspectors/registration_error_inspector.rb:131:in
rescue in format_rest_error' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/formatters/error_inspectors/registration_error_inspector.rb:129:informat_rest_error’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/formatters/error_inspectors/registration_error_inspector.rb:85:in
humanize_http_exception' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/formatters/error_inspectors/registration_error_inspector.rb:25:inadd_explanation’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/formatters/error_mapper.rb:32:in
registration_failed' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/formatters/base.rb:135:inregistration_failed’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/event_dispatch/dispatcher.rb:29:in
block in registration_failed' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/event_dispatch/dispatcher.rb:29:ineach’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/event_dispatch/dispatcher.rb:29:in
registration_failed' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/client.rb:361:inrescue in register’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/client.rb:345:in
register' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/client.rb:454:indo_run’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/client.rb:200:in
run' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/application.rb:190:inrun_chef_client’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/application/client.rb:297:in
block in run_application' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/application/client.rb:290:inloop’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/application/client.rb:290:in
run_application' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/application.rb:73:inrun’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/bin/chef-client:26:in
<top (required)>' /usr/bin/chef-client:23:inload’
/usr/bin/chef-client:23:in <main>' [2013-07-08T15:15:09-04:00] FATAL: NoMethodError: undefined methodclosed?’ for nil:NilClass
On Monday, July 8, 2013 at 12:33 PM, Jeff Blaine wrote:
We have one host that is being testy while getting itself
set up with Chef 11.4.0. Any ideas would be welcome. Using
--log_level debug is not proving helpful.
240 others have self-registered via this method just fine.
/etc/chef/validation.pem exists and is correct.
/etc/chef/boot.json file:
{
"run_list": ["role[rcf-basic-rhel]"]
}
/etc/chef/client.rb we drop in place before the 1st run:
log_level :auto
log_location STDOUT
chef_server_url "https://cm-master.our.org"
validation_client_name "chef-validator"
Using default node name (fqdn)
http_proxy "http://gatekeeper.our.org"
https_proxy "http://gatekeeper.our.org"
file_backup_path "/var/chef/backup"
file_cache_path "/var/chef/cache"
cache_options({ :path => "/var/chef/cache/checksums", :skip_expires
=> true })
set the proxy env variable so rubygems works correctly
ENV['http_proxy'] = "http://gatekeeper.our.org"
ENV['HTTP_PROXY'] = "http://gatekeeper.our.org"
Ohai::Config[:disabled_plugins] = ["passwd"]
The 1st run on the node and its failure:
chef-client --log_level debug --environment production -j
/etc/chef/boot.json
...
[2013-07-08T15:15:09-04:00] INFO: Client key /etc/chef/client.pem is not
present - registering
[2013-07-08T15:15:09-04:00] DEBUG: Signing the request as chef-validator
[2013-07-08T15:15:09-04:00] DEBUG: Using gatekeeper.our.org:80 (http://gatekeeper.our.org:80) for proxy
[2013-07-08T15:15:09-04:00] DEBUG: Sending HTTP Request via POST to
cm-master.our.org:443/clients (http://cm-master.our.org:443/clients)
[2013-07-08T15:15:09-04:00] FATAL: Stacktrace dumped to
/var/chef/cache/chef-stacktrace.out
[2013-07-08T15:15:09-04:00] DEBUG: NoMethodError: undefined method
closed?' for nil:NilClass /opt/chef/embedded/lib/ruby/1.9.1/net/http.rb:2788:in stream_check'
/opt/chef/embedded/lib/ruby/1.9.1/net/http.rb:2708:in read_body' /opt/chef/embedded/lib/ruby/1.9.1/net/http.rb:2735:in body'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/formatters/error_inspectors/registration_error_inspector.rb:131:in
rescue in format_rest_error' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/formatters/error_inspectors/registration_error_inspector.rb:129:in format_rest_error'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/formatters/error_inspectors/registration_error_inspector.rb:85:in
humanize_http_exception' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/formatters/error_inspectors/registration_error_inspector.rb:25:in add_explanation'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/formatters/error_mapper.rb:32:in
registration_failed' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/formatters/base.rb:135:in registration_failed'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/event_dispatch/dispatcher.rb:29:in
block in registration_failed' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/event_dispatch/dispatcher.rb:29:in each'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/event_dispatch/dispatcher.rb:29:in
registration_failed' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/client.rb:361:in rescue in register'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/client.rb:345:in
register' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/client.rb:454:in do_run'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/client.rb:200:in
run' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/application.rb:190:in run_chef_client'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/application/client.rb:297:in
block in run_application' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/application/client.rb:290:in loop'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/application/client.rb:290:in
run_application' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/application.rb:73:in run'
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/bin/chef-client:26:in
<top (required)>' /usr/bin/chef-client:23:in load'
/usr/bin/chef-client:23:in <main>' [2013-07-08T15:15:09-04:00] FATAL: NoMethodError: undefined method closed?' for nil:NilClass
Groan Some Ruby net/http bug is blowing up the thing that's supposed to give you a pretty/humane error message. As best I can tell, you're getting a HTTP 400 back from the server. Anything in the server logs (chef-server-ctl tail if you're on Chef 11)?
--
Daniel DeLeo
On 7/8/2013 4:37 PM, Daniel DeLeo wrote:
Groan Some Ruby net/http bug is blowing up the thing that's supposed
to give you a pretty/humane error message. As best I can tell, you're
getting a HTTP 400 back from the server. Anything in the server logs
(chef-server-ctl tail if you're on Chef 11)?
Daniel,
Thanks for the reply.
Nothing of use in chef-server-ctl tail. The request never reached the
Chef server from what I can tell.
I did, however, solve this instance of this ugliness. I learned (perhaps
now that it is documented?) of the no_proxy client.rb setting and
configured things to make use of that the first time around and forever
afterwards via the chef-client cookbook we were already using.
It still does not explain how everything has been working fine for 5
months without the use of no_proxy. I don't see myself digging into that
further though due to lack of time.