Test-kitchen causing HttpClient-KeepAlive disconnected while trying to run kitchen- converge

darknight · March 28, 2016, 12:20am

Hi,

I was trying to run “kitchen converge” on centos-6.5 box using test-kitchen to install a http package, but it keeps on failing during converge action.I am not sure what could be wrong here and would appreciate on any suggestions. could this to be due to any missing gems. In my metadata file i currently have berkshelf, test-kitchen,kitchen-vagrant.

Thank you for your time

Log:

I, [2016-03-27T19:04:33.576992 #72939] INFO – Kitchen: -----> Starting Kitchen (v1.6.0)
I, [2016-03-27T19:04:34.270469 #72939] INFO – Kitchen: -----> Creating …
I, [2016-03-27T19:05:14.416384 #72939] INFO – Kitchen: -----> Converging …
E, [2016-03-27T19:05:14.500624 #72939] ERROR – Kitchen: ------Exception-------
E, [2016-03-27T19:05:14.500659 #72939] ERROR – Kitchen: Class: Kitchen::ActionFailed
E, [2016-03-27T19:05:14.500682 #72939] ERROR – Kitchen: Message: Failed to complete #converge action: [HTTPClient::KeepAliveDisconnected: ]
E, [2016-03-27T19:05:14.500692 #72939] ERROR – Kitchen: —Nested Exception—
E, [2016-03-27T19:05:14.500702 #72939] ERROR – Kitchen: Class: HTTPClient::KeepAliveDisconnected
E, [2016-03-27T19:05:14.500710 #72939] ERROR – Kitchen: Message: HTTPClient::KeepAliveDisconnected:
E, [2016-03-27T19:05:14.500719 #72939] ERROR – Kitchen: ------Backtrace-------
E, [2016-03-27T19:05:14.500728 #72939] ERROR – Kitchen: /Users/abc1245/.rvm/gems/ruby-2.2.3/gems/httpclient-2.7.1/lib/httpclient/session.rb:795:in block in parse_header' E, [2016-03-27T19:05:14.500737 #72939] ERROR -- Kitchen: /Users/abc1245/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/timeout.rb:88:inblock in timeout’
E, [2016-03-27T19:05:14.500747 #72939] ERROR – Kitchen: /Users/abc1245/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/timeout.rb:98:in call' E, [2016-03-27T19:05:14.500756 #72939] ERROR -- Kitchen: /Users/abc1245/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/timeout.rb:98:intimeout’
E, [2016-03-27T19:05:14.500776 #72939] ERROR – Kitchen: /Users/abc1245/.rvm/gems/ruby-2.2.3/gems/httpclient-2.7.1/lib/httpclient/session.rb:788:in parse_header' E, [2016-03-27T19:05:14.500785 #72939] ERROR -- Kitchen: /Users/abc1245/.rvm/gems/ruby-2.2.3/gems/httpclient-2.7.1/lib/httpclient/session.rb:640:inconnect_ssl_proxy’
E, [2016-03-27T19:05:14.500795 #72939] ERROR – Kitchen: /Users/abc1245/.rvm/gems/ruby-2.2.3/gems/httpclient-2.7.1/lib/httpclient/ssl_socket.rb:21:in create_socket' E, [2016-03-27T19:05:14.500804 #72939] ERROR -- Kitchen: /Users/abc1245/.rvm/gems/ruby-2.2.3/gems/httpclient-2.7.1/lib/httpclient/session.rb:739:inblock in connect’
E, [2016-03-27T19:05:14.500823 #72939] ERROR – Kitchen: /Users/abc1245/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/timeout.rb:88:in block in timeout' E, [2016-03-27T19:05:14.500842 #72939] ERROR -- Kitchen: /Users/abc1245/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/timeout.rb:98:incall’
E, [2016-03-27T19:05:14.500861 #72939] ERROR – Kitchen: /Users/abc1245/.rvm/rubies/ruby-2.2.3/lib/ruby/2.2.0/timeout.rb:98:in timeout' E, [2016-03-27T19:05:14.500870 #72939] ERROR -- Kitchen: /Users/abc1245/.rvm/gems/ruby-2.2.3/gems/httpclient-2.7.1/lib/httpclient/session.rb:735:inconnect’
E, [2016-03-27T19:05:14.500879 #72939] ERROR – Kitchen: /Users/abc1245/.rvm/gems/ruby-2.2.3/gems/httpclient-2.7.1/lib/httpclient/session.rb:497:in query' E, [2016-03-27T19:05:14.500888 #72939] ERROR -- Kitchen: /Users/abc1245/.rvm/gems/ruby-2.2.3/gems/httpclient-2.7.1/lib/httpclient/session.rb:170:inquery’
E, [2016-03-27T19:05:14.500897 #72939] ERROR – Kitchen: /Users/abc1245/.rvm/gems/ruby-2.2.3/gems/httpclient-2.7.1/lib/httpclient.rb:1238:in do_get_block' E, [2016-03-27T19:05:14.500906 #72939] ERROR -- Kitchen: /Users/abc1245/.rvm/gems/ruby-2.2.3/gems/httpclient-2.7.1/lib/httpclient.rb:1021:inblock in do_request’
E, [2016-03-27T19:05:14.500915 #72939] ERROR – Kitchen: /Users/abc1245/.rvm/gems/ruby-2.2.3/gems/httpclient-2.7.1/lib/httpclient.rb:1134:in rescue in protect_keep_alive_disconnected' E, [2016-03-27T19:05:14.500924 #72939] ERROR -- Kitchen: /Users/abc1245/.rvm/gems/ruby-2.2.3/gems/httpclient-2.7.1/lib/httpclient.rb:1128:inprotect_keep_alive_disconnected’
E, [2016-03-27T19:05:14.500933 #72939] ERROR – Kitchen: /Users/abc1245/.rvm/gems/ruby-2.2.3/gems/httpclient-2.7.1/lib/httpclient.rb:1016:in do_request' E, [2016-03-27T19:05:14.500942 #72939] ERROR -- Kitchen: /Users/abc1245/.rvm/gems/ruby-2.2.3/gems/httpclient-2.7.1/lib/httpclient.rb:858:inrequest’
E, [2016-03-27T19:05:14.500990 #72939] ERROR – Kitchen: /Users/abc1245/.rvm/gems/ruby-2.2.3/gems/faraday-0.9.2/lib/faraday/adapter/httpclient.rb:36:in call' E, [2016-03-27T19:05:14.501006 #72939] ERROR -- Kitchen: /Users/abc1245/.rvm/gems/ruby-2.2.3/gems/faraday-0.9.2/lib/faraday/request/retry.rb:116:incall’
E, [2016-03-27T19:05:14.501016 #72939] ERROR – Kitchen: /Users/abc1245/.rvm/gems/ruby-2.2.3/gems/faraday-0.9.2/lib/faraday/response.rb:8:in call' E, [2016-03-27T19:05:14.501026 #72939] ERROR -- Kitchen: /Users/abc1245/.rvm/gems/ruby-2.2.3/gems/faraday-0.9.2/lib/faraday/rack_builder.rb:139:inbuild_response’
E, [2016-03-27T19:05:14.501035 #72939] ERROR – Kitchen: /Users/abc1245/.rvm/gems/ruby-2.2.3/gems/faraday-0.9.2/lib/faraday/connection.rb:377:in run_request' E, [2016-03-27T19:05:14.501044 #72939] ERROR -- Kitchen: /Users/abc1245/.rvm/gems/ruby-2.2.3/gems/faraday-0.9.2/lib/faraday/connection.rb:140:inget’
E, [2016-03-27T19:05:14.501053 #72939] ERROR – Kitchen: /Users/abc1245/.rvm/gems/ruby-2.2.3/gems/berkshelf-api-client-2.0.2/lib/berkshelf/api_client/connection.rb:60:in universe' E, [2016-03-27T19:05:14.501062 #72939] ERROR -- Kitchen: /Users/abc1245/.rvm/gems/ruby-2.2.3/gems/berkshelf-4.3.1/lib/berkshelf/source.rb:49:inbuild_universe’
E, [2016-03-27T19:05:14.501071 #72939] ERROR – Kitchen: /Users/abc1245/.rvm/gems/ruby-2.2.3/gems/berkshelf-4.3.1/lib/berkshelf/installer.rb:21:in `block (2 levels) in build_universe’

nukepuppy · March 28, 2016, 6:39pm

does this happen when using ChefDK by any chance?

darknight · March 28, 2016, 6:54pm

I was using the following installed chefdk version and also not sure why berks,kitchen versions are showing version errors at this moment.

$rvm list

ruby-2.2.3 [ x86_64 ]

$chef --version
Chef Development Kit Version: 0.11.2
chef-client version: 12.7.2
berks version: ERROR
kitchen version: ERROR

nukepuppy · March 29, 2016, 2:33pm

what exaclty is your platform?
have you tried using just chefDK via the installation packages?
then just selectively loading it up to test…

eval "$(chef shell-init bash)"

Just a suggestion to see if that works…

I know for me i have configuration in knife.rb that doesn’t work for test kitchen and the ones that work for test kitchen wont work for chef-provisioning. So I use knife-block to switch between cookbook development vs chef-provisioning cookbook development.

But you should get to the root cause before trying to go too deep into it…

paste your knife.rb maybe

darknight · March 30, 2016, 2:32am

Hi nukepuppy,

Thank you for looking into the error. As you advised,I have decided to start from clean state by uninstalling the components of rvm, chefdk, vagrant, virtualbox and downloaded latest version of chefdk from https://downloads.chef.io/chef-dk/mac/ - 12.0. I thought this might be easier to get to the root cause before going too deep.

I am currently on OS X El Capitan 10.11.3

chef -version

Chef Development Kit Version: 0.12.0
chef-client version: 12.8.1 
berks version: 4.3.0 
kitchen version: ERROR

rvm env

export PATH="/Users/local/.rvm/gems/ruby-2.3.0/bin:/Users/local/.rvm/gems/ruby- 
2.3.0@global/bin:/Users/local/.rvm/rubies/ruby-2.3.0/bin:$PATH"
export GEM_HOME='/Users/local/.rvm/gems/ruby-2.3.0'
export GEM_PATH='/Users/local/.rvm/gems/ruby-2.3.0:/Users/local/.rvm/gems/ruby-2.3.0@global'
export MY_RUBY_HOME='/Users/local/.rvm/rubies/ruby-2.3.0'
export IRBRC='/Users/local/.rvm/rubies/ruby-2.3.0/.irbrc'

vagrant --version

Vagrant 1.8.1

virtualbox --version

5.0.14r105127

I don't have knife.rb file in my cook book structure and i have used berks to create cookbook and installed the following vagrant plugins to provision box using kitchen.yml.

vagrant plugin list

vagrant-berkshelf (4.1.0)
vagrant-omnibus (1.4.1)

kitchen.yml

---
driver:
name: vagrant
customize:
  natdnshostresolver1: "on"
network:
  - ["forwarded_port", { guest: 81, host: 8082 }]
  - ["forwarded_port", { guest: 80, host: 8087 }]
provisioner:
chef_omnibus_url: http://www.chef.io/chef/install.sh
name: chef_zero
platforms:
  - name: centos-6.7
suites:
 - name: default
run_list:
 -  recipe[chef-boxtest::default] 
attributes:

When I tried to run kitchen converge, i am getting the following error.

kitchen converge

-----> Starting Kitchen (v1.6.0)
!!!!!! The berkshelf' gem is missing and must be installed or cannot be properly activated. Run gem install > berkshelfor add the following to your Gemfile if you are using Bundler:gem 'berkshelf'`.
------Exception-------
Class: Kitchen::UserError
Message: Could not load or activate Berkshelf (cannot load such file -- berkshelf)

I tried to install berkshelf gem using bundle but, it gives me different error. should i be using chefdk ruby or ruby through rvm. Please correct me on this.

$ gem install berkshelf
ERROR: While executing gem ... (NoMethodError)
undefined method `dump' for nil:NilClass
Did you mean? dup

nukepuppy · March 30, 2016, 2:38am

The error you saw about Berkshelf gem missing kind of leans me to say something is very odd with your environment setup.

Before you proceed do ‘which ruby’ see if its pointing to the system ruby / some rvm ruby/ or something else?

to initialize chefDK you have to run the following:

eval "$(chef shell-init bash)"

an example of before / after running that command

$ which ruby
/usr/bin/ruby
$ eval "$(chef shell-init bash)"
$ which ruby
/opt/chefdk/embedded/bin/ruby

you can see its pointing to the chefdk ruby now after that command… i suspect somethings interfering with things on your end causing you some gem path conflicts…

you can make the chefDK permanent by adding it to your ~/.bash_profile or you can selectively use by executing that command when needed.

for what its worth chefdk can also install gems

chef gem install whatever

just keep in mind they will go away when/if you upgrade to a newer chefDK.

In the end think of the chefDK embedded ruby as a safe zone from your system’s ruby that shouldn’t impact your day to day ruby work (if you do that kind of work) on my end I just use chefDK exclusively.

darknight · March 30, 2016, 1:48pm

@nukepuppy I have changed my local path to point to the chefdk ruby as i was having rvm in my path from previous installation. I was able to launch a centos-7 box using vagrant in my virtual box once i updated that. Thank you for correcting me on this.

The other question i have is about being able to use bundle to install the gem dependencies, can that also be done using chef gem? if i have the gems listed in my Gemfile. I was able to use chef gem install and got a gem but not sure how bundle behaves with chefdk ruby.

which ruby
/opt/chefdk/embedded/bin/ruby

GEM PATHS:
- /Users/local/.chefdk/gem/ruby/2.1.0
- /opt/chefdk/embedded/lib/ruby/gems/2.1.0

nukepuppy · March 30, 2016, 2:11pm

Good to hear that you got most of it sorted out.

just do a

which gem

it will probably tell you its in /opt/chefdk/embedded/bin/gem

bundler will work just as it normally would…

darknight · March 30, 2016, 2:25pm

@nukepuppy Yep, It did work.Thank you so much..Glad to get it resolved.Thanks for your time.

which gem
/opt/chefdk/embedded/bin/gem

Topic		Replies	Views
[RESOLVED] Kitchen converge with dependencies failing Chef Infra (archive)	3	2122	April 14, 2017
I'm getting errors when trying to pass the encrypted data bag key to kitchen Chef Infra (archive)	1	749	October 11, 2017
Kitchen converge failing with ssl error Chef Infra (archive)	15	5645	April 14, 2017
Kitchen converge failed with Class: Kitchen::ActionFailed Chef Infra (archive)	3	2779	November 21, 2017
Trying git clone-erroring out Chef Infra (archive)	2	696	December 2, 2016

Test-kitchen causing HttpClient-KeepAlive disconnected while trying to run kitchen- converge

Related Topics