Knife list client: SSL Validation failure connecting to host


#1

The odd thing is that it shows that verification successfully using “ssl check”

$ knife ssl check
Connecting to host chefserver.example.com:443
Successfully verified certificates from `chefserver.example.com

But when I run “client list” I got

$ knife client list
ERROR: SSL Validation failure connecting to host: chefserver.example.com - SSL_connect SYSCALL returned=5 errno=0 state=SSLv2/v3 read server hello A
ERROR: Could not establish a secure connection to the server.
Use knife ssl check to troubleshoot your SSL configuration.
If your Chef Server uses a self-signed certificate, you can use
knife ssl fetch to make knife trust the server’s certificates.

Original Exception: OpenSSL::SSL::SSLError: SSL_connect SYSCALL returned=5 errno=0 state=SSLv2/v3 read server hello A

================

I can telnet to the server 443 without problem so proxy should not be an issue.

Any suggestion?

thanks


#2

I bet your proxy does MITM SSL inspection and downgrade TLS to use SSLV3 which is not supported anymore by chef-server out of the box since poodle security release.

A telnet gives you no more information than ‘the port is open’, the error is about SSL negociation (the read server hello A part).

Without more information about your infrastructure it would be divination to give an advice but let’s say what come on top of my head:

  1. Get your netadmin to configure the proxy properly to not use SSLV3 anymore (best idea)
  2. Update the ssl protocols allowed on chef-server nginx by updating the chef-server.rb see here

#3

Tensibai, I really appreciate your quick response.

I tried your suggestion 2) (since I can not make any change the corp proxy) with the following

add the following content to /etc/opscode/chef-server.rb and /var/opt/opscode/chef-server.rb

ginx[‘url’] = "https://chefserver"
nginx[‘server_name’] = "chefserver"
nginx[‘ssl_certificate’] = "/var/opt/opscode/nginx/ca/chefserver.crt"
nginx[‘ssl_certificate_key’] = “/var/opt/opscode/nginx/ca/chefserver.key”

and ensure the crt and key are really there.

But I still got the same error:

mqian@chefwork:~/chef-repo$ knife client list
ERROR: SSL Validation failure connecting to host: chefserver - SSL_connect SYSCALL returned=5 errno=0 state=SSLv2/v3 read server hello A
ERROR: Could not establish a secure connection to the server.
Use knife ssl check to troubleshoot your SSL configuration.
If your Chef Server uses a self-signed certificate, you can use
knife ssl fetch to make knife trust the server’s certificates.

And still the “check” return normally:

mqian@chefwork:~/chef-repo$ knife ssl check
Connecting to host chefserver:443
Successfully verified certificates from `chefserver’

and “fetch” return normally too:

mqian@chefwork:~/chef-repo$ knife ssl fetch
WARNING: Certificates from chefserver will be fetched and placed in your trusted_cert
directory (/home/mqian/chef-repo/.chef/trusted_certs).

Knife has no means to verify these are the correct certificates. You should
verify the authenticity of these certificates after downloading.

Adding certificate for chefserver in /home/mqian/chef-repo/.chef/trusted_certs/chefserver.crt

Any further suggestion?


#4

Yep, I was talking about the SSL protocols allowed, something like this(in chef-server.rb) should do:

nginx['ssl_protocols'] = 'TLSv1 TLSv1.1 TLSv1.2 SSLv3' 

don’t forget to call chef-server-ctl reconfigure after modifying the file.


#5

Tensibai, thank you for the response!

After I changed to the following the output did change but still not working:

nginx[‘ssl_certificate’] = "/var/opt/opscode/nginx/ca/chefserver.crt"
nginx[‘ssl_certificate_key’] = "/var/opt/opscode/nginx/ca/chefserver.key"
nginx[‘ssl_ciphers’] = "HIGH:MEDIUM:!LOW:!kEDH:!aNULL:!ADH:!eNULL:!EXP:!SSLv2:!SEED:!CAMELLIA:!PSK"
nginx[‘ssl_protocols’] = ‘TLSv1 TLSv1.1 TLSv1.2 SSLv3’

mqian@chefwork:~/chef-repo$ knife ssl fetch
WARNING: Certificates from chefserver.example.com will be fetched and placed in your trusted_cert
directory (/home/mqian/chef-repo/.chef/trusted_certs).

Knife has no means to verify these are the correct certificates. You should
verify the authenticity of these certificates after downloading.

Adding certificate for chefserver in /home/mqian/chef-repo/.chef/trusted_certs/chefserver.crt
mqian@chefwork:~/chef-repo$ knife ssl cehck
FATAL: Cannot find sub command for: 'ssl cehck’
Available ssl subcommands: (for details, knife SUB-COMMAND --help)

** SSL COMMANDS **
knife ssl check [URL] (options)
knife ssl fetch [URL] (options)

mqian@chefwork:~/chef-repo$ knife client list
ERROR: Service temporarily unavailable
/opt/chefdk/embedded/lib/ruby/2.1.0/net/http/response.rb:325:in stream_check': undefined methodclosed?’ for nil:NilClass (NoMethodError)
from /opt/chefdk/embedded/lib/ruby/2.1.0/net/http/response.rb:199:in read_body' from /opt/chefdk/embedded/lib/ruby/2.1.0/net/http/response.rb:226:inbody’
from /opt/chefdk/embedded/apps/chef/lib/chef/knife.rb:507:in rescue in format_rest_error' from /opt/chefdk/embedded/apps/chef/lib/chef/knife.rb:505:informat_rest_error’
from /opt/chefdk/embedded/apps/chef/lib/chef/knife.rb:485:in humanize_http_exception' from /opt/chefdk/embedded/apps/chef/lib/chef/knife.rb:435:inhumanize_exception’
from /opt/chefdk/embedded/apps/chef/lib/chef/knife.rb:426:in rescue in run_with_pretty_exceptions' from /opt/chefdk/embedded/apps/chef/lib/chef/knife.rb:417:inrun_with_pretty_exceptions’
from /opt/chefdk/embedded/apps/chef/lib/chef/knife.rb:214:in run' from /opt/chefdk/embedded/apps/chef/lib/chef/application/knife.rb:142:inrun’
from /opt/chefdk/embedded/apps/chef/bin/knife:25:in <top (required)>' from /opt/chefdk/bin/knife:51:inload’
from /opt/chefdk/bin/knife:51:in `’

Ang suggestion?


#6

I am sorry: there was a type error command for “check” and here is the result

mqian@chefwork:~/chef-repo$ knife ssl check

Configuration Info:

OpenSSL Configuration:

  • Version: OpenSSL 1.0.1m 19 Mar 2015
  • Certificate file: /opt/chefdk/embedded/ssl/cert.pem
  • Certificate directory: /opt/chefdk/embedded/ssl/certs
    Chef SSL Configuration:
  • ssl_ca_path: nil
  • ssl_ca_file: nil
  • trusted_certs_dir: "/home/mqian/chef-repo/.chef/trusted_certs"
    WARNING: There are invalid certificates in your trusted_certs_dir.
    OpenSSL will not use the following certificates when verifying SSL connections:

/home/mqian/chef-repo/.chef/trusted_certs/chefserver.crt: certificate is not yet valid

TO FIX THESE WARNINGS:

We are working on documentation for resolving common issues uncovered here.

  • If the certificate is generated by the server, you may try redownloading the
    server’s certificate. By default, the certificate is stored in the following
    location on the host where your chef-server runs:

    /var/opt/opscode/nginx/ca/SERVER_HOSTNAME.crt

Copy that file to your trusted_certs_dir (currently: /home/mqian/chef-repo/.chef/trusted_certs)
using SSH/SCP or some other secure method, then re-run this command to confirm
that the server’s certificate is now trusted.

Connecting to host chefserver.example.com:443
ERROR: The SSL certificate of chefserver.example.com could not be verified
Certificate issuer data: /C=US/ST=WA/L=Seattle/O=YouCorp/OU=Operations/CN=chefserver/emailAddress=you@example.com

Configuration Info:

OpenSSL Configuration:

  • Version: OpenSSL 1.0.1m 19 Mar 2015
  • Certificate file: /opt/chefdk/embedded/ssl/cert.pem
  • Certificate directory: /opt/chefdk/embedded/ssl/certs
    Chef SSL Configuration:
  • ssl_ca_path: nil
  • ssl_ca_file: nil
  • trusted_certs_dir: “/home/mqian/chef-repo/.chef/trusted_certs”

TO FIX THIS ERROR:

If the server you are connecting to uses a self-signed certificate, you must
configure chef to trust that server’s certificate.

By default, the certificate is stored in the following location on the host
where your chef-server runs:

/var/opt/opscode/nginx/ca/SERVER_HOSTNAME.crt

Copy that file to your trusted_certs_dir (currently: /home/mqian/chef-repo/.chef/trusted_certs)
using SSH/SCP or some other secure method, then re-run this command to confirm
that the server’s certificate is now trusted.


#7

I then followed the instructions above to manually copy the chefserver.crt over and run the check again I always got the same check result with the error and running client list result the same…


#8

That’s expected, fetch gets only the chef certificate, you’ll have to get the proxy certificate wich doing a MITM will be the CA for the server certificate.

Best option is to use a browser to open the server webui if you have it, or any other https site, then in your browser show certificates, and save them in file as x509 in the trusted_certs dir (last time I tried I had to add it to chef cacert.pem file for this case)

Related StackOverflow answer: http://stackoverflow.com/questions/25604784/chef-berkshelf3-proxy-settings-windows-platform/25605415#25605415