Sending HTTP Request ... NoMethodError: undefined method `closed?' for nil:NilClass


#1

We have one host that is being testy while getting itself
set up with Chef 11.4.0. Any ideas would be welcome. Using
–log_level debug is not proving helpful.

240 others have self-registered via this method just fine.

/etc/chef/validation.pem exists and is correct.

/etc/chef/boot.json file:

 {
     "run_list": ["role[rcf-basic-rhel]"]
 }

/etc/chef/client.rb we drop in place before the 1st run:

 log_level        :auto
 log_location     STDOUT
 chef_server_url  "https://cm-master.our.org"
 validation_client_name "chef-validator"
 # Using default node name (fqdn)
 http_proxy        "http://gatekeeper.our.org"
 https_proxy       "http://gatekeeper.our.org"

 file_backup_path   "/var/chef/backup"
 file_cache_path    "/var/chef/cache"
 cache_options({ :path => "/var/chef/cache/checksums", :skip_expires 

=> true })

 # set the proxy env variable so rubygems works correctly
 ENV['http_proxy'] = "http://gatekeeper.our.org"
 ENV['HTTP_PROXY'] = "http://gatekeeper.our.org"

 Ohai::Config[:disabled_plugins] =  ["passwd"]

The 1st run on the node and its failure:

chef-client --log_level debug --environment production -j
/etc/chef/boot.json


[2013-07-08T15:15:09-04:00] INFO: Client key /etc/chef/client.pem is not
present - registering
[2013-07-08T15:15:09-04:00] DEBUG: Signing the request as chef-validator
[2013-07-08T15:15:09-04:00] DEBUG: Using gatekeeper.our.org:80 for proxy
[2013-07-08T15:15:09-04:00] DEBUG: Sending HTTP Request via POST to
cm-master.our.org:443/clients
[2013-07-08T15:15:09-04:00] FATAL: Stacktrace dumped to
/var/chef/cache/chef-stacktrace.out
[2013-07-08T15:15:09-04:00] DEBUG: NoMethodError: undefined method
closed?' for nil:NilClass /opt/chef/embedded/lib/ruby/1.9.1/net/http.rb:2788:instream_check’
/opt/chef/embedded/lib/ruby/1.9.1/net/http.rb:2708:in read_body' /opt/chef/embedded/lib/ruby/1.9.1/net/http.rb:2735:inbody’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/formatters/error_inspectors/registration_error_inspector.rb:131:in
rescue in format_rest_error' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/formatters/error_inspectors/registration_error_inspector.rb:129:informat_rest_error’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/formatters/error_inspectors/registration_error_inspector.rb:85:in
humanize_http_exception' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/formatters/error_inspectors/registration_error_inspector.rb:25:inadd_explanation’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/formatters/error_mapper.rb:32:in
registration_failed' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/formatters/base.rb:135:inregistration_failed’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/event_dispatch/dispatcher.rb:29:in
block in registration_failed' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/event_dispatch/dispatcher.rb:29:ineach’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/event_dispatch/dispatcher.rb:29:in
registration_failed' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/client.rb:361:inrescue in register’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/client.rb:345:in
register' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/client.rb:454:indo_run’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/client.rb:200:in
run' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/application.rb:190:inrun_chef_client’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/application/client.rb:297:in
block in run_application' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/application/client.rb:290:inloop’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/application/client.rb:290:in
run_application' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/application.rb:73:inrun’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/bin/chef-client:26:in
<top (required)>' /usr/bin/chef-client:23:inload’
/usr/bin/chef-client:23:in <main>' [2013-07-08T15:15:09-04:00] FATAL: NoMethodError: undefined methodclosed?’ for nil:NilClass


#2

On Monday, July 8, 2013 at 12:33 PM, Jeff Blaine wrote:

We have one host that is being testy while getting itself
set up with Chef 11.4.0. Any ideas would be welcome. Using
–log_level debug is not proving helpful.

240 others have self-registered via this method just fine.

/etc/chef/validation.pem exists and is correct.

/etc/chef/boot.json file:

{
“run_list”: [“role[rcf-basic-rhel]”]
}

/etc/chef/client.rb we drop in place before the 1st run:

log_level :auto
log_location STDOUT
chef_server_url "https://cm-master.our.org"
validation_client_name “chef-validator”

Using default node name (fqdn)

http_proxy "http://gatekeeper.our.org"
https_proxy “http://gatekeeper.our.org

file_backup_path "/var/chef/backup"
file_cache_path "/var/chef/cache"
cache_options({ :path => “/var/chef/cache/checksums”, :skip_expires
=> true })

set the proxy env variable so rubygems works correctly

ENV[‘http_proxy’] = "http://gatekeeper.our.org"
ENV[‘HTTP_PROXY’] = “http://gatekeeper.our.org

Ohai::Config[:disabled_plugins] = [“passwd”]

The 1st run on the node and its failure:

chef-client --log_level debug --environment production -j
/etc/chef/boot.json


[2013-07-08T15:15:09-04:00] INFO: Client key /etc/chef/client.pem is not
present - registering
[2013-07-08T15:15:09-04:00] DEBUG: Signing the request as chef-validator
[2013-07-08T15:15:09-04:00] DEBUG: Using gatekeeper.our.org:80 (http://gatekeeper.our.org:80) for proxy
[2013-07-08T15:15:09-04:00] DEBUG: Sending HTTP Request via POST to
cm-master.our.org:443/clients (http://cm-master.our.org:443/clients)
[2013-07-08T15:15:09-04:00] FATAL: Stacktrace dumped to
/var/chef/cache/chef-stacktrace.out
[2013-07-08T15:15:09-04:00] DEBUG: NoMethodError: undefined method
closed?' for nil:NilClass /opt/chef/embedded/lib/ruby/1.9.1/net/http.rb:2788:instream_check’
/opt/chef/embedded/lib/ruby/1.9.1/net/http.rb:2708:in read_body' /opt/chef/embedded/lib/ruby/1.9.1/net/http.rb:2735:inbody’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/formatters/error_inspectors/registration_error_inspector.rb:131:in
rescue in format_rest_error' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/formatters/error_inspectors/registration_error_inspector.rb:129:informat_rest_error’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/formatters/error_inspectors/registration_error_inspector.rb:85:in
humanize_http_exception' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/formatters/error_inspectors/registration_error_inspector.rb:25:inadd_explanation’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/formatters/error_mapper.rb:32:in
registration_failed' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/formatters/base.rb:135:inregistration_failed’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/event_dispatch/dispatcher.rb:29:in
block in registration_failed' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/event_dispatch/dispatcher.rb:29:ineach’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/event_dispatch/dispatcher.rb:29:in
registration_failed' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/client.rb:361:inrescue in register’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/client.rb:345:in
register' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/client.rb:454:indo_run’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/client.rb:200:in
run' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/application.rb:190:inrun_chef_client’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/application/client.rb:297:in
block in run_application' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/application/client.rb:290:inloop’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/application/client.rb:290:in
run_application' /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/lib/chef/application.rb:73:inrun’
/opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.4.0/bin/chef-client:26:in
<top (required)>' /usr/bin/chef-client:23:inload’
/usr/bin/chef-client:23:in <main>' [2013-07-08T15:15:09-04:00] FATAL: NoMethodError: undefined methodclosed?’ for nil:NilClass

Groan Some Ruby net/http bug is blowing up the thing that’s supposed to give you a pretty/humane error message. As best I can tell, you’re getting a HTTP 400 back from the server. Anything in the server logs (chef-server-ctl tail if you’re on Chef 11)?


Daniel DeLeo


#3

On 7/8/2013 4:37 PM, Daniel DeLeo wrote:

Groan Some Ruby net/http bug is blowing up the thing that’s supposed
to give you a pretty/humane error message. As best I can tell, you’re
getting a HTTP 400 back from the server. Anything in the server logs
(chef-server-ctl tail if you’re on Chef 11)?
Daniel,

Thanks for the reply.

Nothing of use in chef-server-ctl tail. The request never reached the
Chef server from what I can tell.

I did, however, solve this instance of this ugliness. I learned (perhaps
now that it is documented?) of the no_proxy client.rb setting and
configured things to make use of that the first time around and forever
afterwards via the chef-client cookbook we were already using.

It still does not explain how everything has been working fine for 5
months without the use of no_proxy. I don’t see myself digging into that
further though due to lack of time.