How tdo I configure the ssl to make the chef client and server happy


#1

I am setting up a chef server for a project in AWS, and I am trying to find the best way to make the server and client happy wrt the ssl settings.

I want to do the following.

  •    Create a chef server with a private IP address and a public (elastic) IP address.
    
  •    I want to use chef01.some.dom.com as the DNS/hostname for the private IP address, and I want to use chef01-eip.some.dom.com as the DNS for the public IP address,  AND I want to create  a CNAME of just chef.some.dom.com which I will use as the name of the chef server in the client.rb files.
    

When I do that, the ssl checking fails. The name of the server in client.rb (che.some.dom.com) does not match the name on the certificate, which was generated with the machines hostname, chef01.some.dom.com.

I try to outsmart the “chef-server-ctl reconfigure” command by temporarily changing the host name to my CNAME (chef.some.dom.com). Well, chef-server-ctl is too clever for me. It does a DNS lookup and finds the actual name associated with the address so it now generates a certificate with name chef01-eip.some.dom.com.

What should I be doing? I am perfectly happy with the self-generated certificate.

On a related note, will I also have to copy that cert to the trusted_certs/ directory on all the clients?


Stephen Corbesero, DevOps Engineer
Synchronoss - Mobile Innovation for a Connected World
stephen.corbesero@synchronoss.com | www.synchronoss.com


#2

Just generate the certificate/key yourself and provide it to the server. To make a self-signed cert:

$ openssl req -x509 -newkey rsa:4096 -keyout chef.key -out chef.pem -nodes -days 365

And then in your /etc/chef-server/chef-server.rb (you may have to create it):

nginx[‘ssl_certificate’] = '/etc/chef-server/chef.pem’
nginx[‘ssl_certificate_key’] = ‘/etc/chef-server/chef.key’

You’ll need to distribute the chef.pem to all clients as well, and configure it as a trusted CA certificate.

–Noah

On Jul 24, 2014, at 6:27 AM, Stephen Corbesero Stephen.Corbesero@synchronoss.com wrote:

I am setting up a chef server for a project in AWS, and I am trying to find the best way to make the server and client happy wrt the ssl settings.

I want to do the following.

· Create a chef server with a private IP address and a public (elastic) IP address.
· I want to use chef01.some.dom.com as the DNS/hostname for the private IP address, and I want to use chef01-eip.some.dom.com as the DNS for the public IP address, AND I want to create a CNAME of just chef.some.dom.com which I will use as the name of the chef server in the client.rb files.

When I do that, the ssl checking fails. The name of the server in client.rb (che.some.dom.com) does not match the name on the certificate, which was generated with the machines hostname, chef01.some.dom.com.

I try to outsmart the “chef-server-ctl reconfigure” command by temporarily changing the host name to my CNAME (chef.some.dom.com). Well, chef-server-ctl is too clever for me. It does a DNS lookup and finds the actual name associated with the address so it now generates a certificate with name chef01-eip.some.dom.com.

What should I be doing? I am perfectly happy with the self-generated certificate.

On a related note, will I also have to copy that cert to the trusted_certs/ directory on all the clients?


Stephen Corbesero, DevOps Engineer
Synchronoss - Mobile Innovation for a Connected World
stephen.corbesero@synchronoss.com | www.synchronoss.com


#3

On Thursday, July 24, 2014 at 10:54 AM, Noah Kantrowitz wrote:

Just generate the certificate/key yourself and provide it to the server. To make a self-signed cert:

$ openssl req -x509 -newkey rsa:4096 -keyout chef.key -out chef.pem -nodes -days 365

And then in your /etc/chef-server/chef-server.rb (you may have to create it):

nginx[‘ssl_certificate’] = '/etc/chef-server/chef.pem’
nginx[‘ssl_certificate_key’] = ‘/etc/chef-server/chef.key’

You’ll need to distribute the chef.pem to all clients as well, and configure it as a trusted CA certificate.

–Noah
I think you’ll also need to set the SubjectAltName field to include each of the hostnames you wish to use.


Daniel DeLeo


#4

Thank you Noah and Daniel,

Follow-up questions:

Do I need to use the openssl inside the /opt/chef-server/… dirs so it gets the right openssl cnf file?

-----Original Message-----
From: Daniel DeLeo [mailto:ddeleo@kallistec.com] On Behalf Of Daniel DeLeo
Sent: Thursday, July 24, 2014 1:57 PM
To: chef@lists.opscode.com
Subject: [chef] Re: Re: How tdo I configure the ssl to make the chef client and server happy

On Thursday, July 24, 2014 at 10:54 AM, Noah Kantrowitz wrote:

Just generate the certificate/key yourself and provide it to the server. To make a self-signed cert:

$ openssl req -x509 -newkey rsa:4096 -keyout chef.key -out chef.pem -nodes -days 365

And then in your /etc/chef-server/chef-server.rb (you may have to create it):

nginx[‘ssl_certificate’] = '/etc/chef-server/chef.pem’
nginx[‘ssl_certificate_key’] = ‘/etc/chef-server/chef.key’

You’ll need to distribute the chef.pem to all clients as well, and configure it as a trusted CA certificate.

–Noah
I think you’ll also need to set the SubjectAltName field to include each of the hostnames you wish to use.


Daniel DeLeo


#5

More follow-ups…

I’ve built my pem & key just using the system openssl with a conf file to generate the SANs. I fetch it via “knife ssl fetch” which puts it in the /etc/chef/trusted-certs/ dir. But chef-client is still failing. And ‘knife ssl check’ fails saying I need the cert there, but it is there.

Am I still missing a step?

-----Original Message-----
From: Stephen Corbesero [mailto:Stephen.Corbesero@synchronoss.com]
Sent: Friday, July 25, 2014 11:16 AM
To: chef@lists.opscode.com
Subject: [chef] RE: Re: Re: How do I configure the ssl to make the chef client and server happy

Thank you Noah and Daniel,

Follow-up questions:

Do I need to use the openssl inside the /opt/chef-server/… dirs so it gets the right openssl cnf file?

-----Original Message-----
From: Daniel DeLeo [mailto:ddeleo@kallistec.com] On Behalf Of Daniel DeLeo
Sent: Thursday, July 24, 2014 1:57 PM
To: chef@lists.opscode.com
Subject: [chef] Re: Re: How tdo I configure the ssl to make the chef client and server happy

On Thursday, July 24, 2014 at 10:54 AM, Noah Kantrowitz wrote:

Just generate the certificate/key yourself and provide it to the server. To make a self-signed cert:

$ openssl req -x509 -newkey rsa:4096 -keyout chef.key -out chef.pem -nodes -days 365

And then in your /etc/chef-server/chef-server.rb (you may have to create it):

nginx[‘ssl_certificate’] = '/etc/chef-server/chef.pem’
nginx[‘ssl_certificate_key’] = ‘/etc/chef-server/chef.key’

You’ll need to distribute the chef.pem to all clients as well, and configure it as a trusted CA certificate.

–Noah
I think you’ll also need to set the SubjectAltName field to include each of the hostnames you wish to use.


Daniel DeLeo


#6

On Friday, July 25, 2014 at 10:18 AM, Stephen Corbesero wrote:

More follow-ups…

I’ve built my pem & key just using the system openssl with a conf file to generate the SANs. I fetch it via “knife ssl fetch” which puts it in the /etc/chef/trusted-certs/ dir. But chef-client is still failing. And ‘knife ssl check’ fails saying I need the cert there, but it is there.

Am I still missing a step?

What failure message are you getting from 'knife ssl check’ ?


Daniel DeLeo

-----Original Message-----
From: Stephen Corbesero [mailto:Stephen.Corbesero@synchronoss.com]
Sent: Friday, July 25, 2014 11:16 AM
To: chef@lists.opscode.com (mailto:chef@lists.opscode.com)
Subject: [chef] RE: Re: Re: How do I configure the ssl to make the chef client and server happy

Thank you Noah and Daniel,

Follow-up questions:

Do I need to use the openssl inside the /opt/chef-server/… dirs so it gets the right openssl cnf file?

-----Original Message-----
From: Daniel DeLeo [mailto:ddeleo@kallistec.com] On Behalf Of Daniel DeLeo
Sent: Thursday, July 24, 2014 1:57 PM
To: chef@lists.opscode.com (mailto:chef@lists.opscode.com)
Subject: [chef] Re: Re: How tdo I configure the ssl to make the chef client and server happy

On Thursday, July 24, 2014 at 10:54 AM, Noah Kantrowitz wrote:

Just generate the certificate/key yourself and provide it to the server. To make a self-signed cert:

$ openssl req -x509 -newkey rsa:4096 -keyout chef.key -out chef.pem -nodes -days 365

And then in your /etc/chef-server/chef-server.rb (you may have to create it):

nginx[‘ssl_certificate’] = '/etc/chef-server/chef.pem’
nginx[‘ssl_certificate_key’] = ‘/etc/chef-server/chef.key’

You’ll need to distribute the chef.pem to all clients as well, and configure it as a trusted CA certificate.

–Noah

I think you’ll also need to set the SubjectAltName field to include each of the hostnames you wish to use.


Daniel DeLeo


#7

This is the error I get when I specify the actual hostname or its subject alternative name.

[root@oh-chef01 sbin]# knife ssl check -c /etc/chef/client.rb

Connecting to host oh-chef01.devops.dev.cloud.synchronoss.net:443
ERROR: The SSL certificate of oh-chef01.devops.dev.cloud.synchronoss.net could not be verified
Certificate issuer data: /C=US/ST=PA/L=Bethlehem/O=Synchronoss Technologies, Inc./OU=IT/CN=oh-chef01.devops.dev.cloud.synchronoss.net/emailAddress=stephen.corbesero@synchronoss.com

Configuration Info:

OpenSSL Configuration:

  • Version: OpenSSL 1.0.1h 5 Jun 2014
  • Certificate file: /opt/chef/embedded/ssl/cert.pem
  • Certificate directory: /opt/chef/embedded/ssl/certs
    Chef SSL Configuration:
  • ssl_ca_path: nil
  • ssl_ca_file: nil
  • trusted_certs_dir: “/etc/chef/trusted_certs”

Followed by the ‘To fix this error…’

It looks like it doesn’t like the certificate at all.

I have generated certs w/ SANs before, but not very often. I even use openssl to dump the text of my cert and do see the proper CN and the SAN.

Also, If I generate a simple cert using the command given to me originally, that works just fine. If my client (or knife ssl check) tries to connect on the CN in the certificate, it succeeds. If I try the other name, “knife ssl check” gives me a very polite message that I am trying one name, but the server is reporting the other.

Just to make sure that the server is in the correct state, I do a reconfigure and restart every time I generate a new cert.

-----Original Message-----
From: Daniel DeLeo [mailto:ddeleo@kallistec.com] On Behalf Of Daniel DeLeo
Sent: Friday, July 25, 2014 4:06 PM
To: chef@lists.opscode.com
Subject: [chef] Re: RE: RE: Re: Re: How do I configure the ssl to make the chef client and server happy

On Friday, July 25, 2014 at 10:18 AM, Stephen Corbesero wrote:

More follow-ups…

I’ve built my pem & key just using the system openssl with a conf file to generate the SANs. I fetch it via “knife ssl fetch” which puts it in the /etc/chef/trusted-certs/ dir. But chef-client is still failing. And ‘knife ssl check’ fails saying I need the cert there, but it is there.

Am I still missing a step?

What failure message are you getting from 'knife ssl check’ ?


Daniel DeLeo

-----Original Message-----
From: Stephen Corbesero [mailto:Stephen.Corbesero@synchronoss.com]
Sent: Friday, July 25, 2014 11:16 AM
To: chef@lists.opscode.com (mailto:chef@lists.opscode.com)
Subject: [chef] RE: Re: Re: How do I configure the ssl to make the chef client and server happy

Thank you Noah and Daniel,

Follow-up questions:

Do I need to use the openssl inside the /opt/chef-server/… dirs so it gets the right openssl cnf file?

-----Original Message-----
From: Daniel DeLeo [mailto:ddeleo@kallistec.com] On Behalf Of Daniel DeLeo
Sent: Thursday, July 24, 2014 1:57 PM
To: chef@lists.opscode.com (mailto:chef@lists.opscode.com)
Subject: [chef] Re: Re: How tdo I configure the ssl to make the chef client and server happy

On Thursday, July 24, 2014 at 10:54 AM, Noah Kantrowitz wrote:

Just generate the certificate/key yourself and provide it to the server. To make a self-signed cert:

$ openssl req -x509 -newkey rsa:4096 -keyout chef.key -out chef.pem -nodes -days 365

And then in your /etc/chef-server/chef-server.rb (you may have to create it):

nginx[‘ssl_certificate’] = '/etc/chef-server/chef.pem’
nginx[‘ssl_certificate_key’] = ‘/etc/chef-server/chef.key’

You’ll need to distribute the chef.pem to all clients as well, and configure it as a trusted CA certificate.

–Noah

I think you’ll also need to set the SubjectAltName field to include each of the hostnames you wish to use.


Daniel DeLeo


#8

Looks like ruby ought to be checking your subject alt name entries. The code for this in ruby 1.9.3 (that ships with omnibus) is here: https://github.com/ruby/ruby/blob/5fc2510a654e764a9d6c5a261fc345ca2c1982c9/ext/openssl/lib/openssl/ssl-internal.rb#L87

You can find your local copy of it by running the command:

/opt/chef/embedded/bin/gem which openssl/ssl-internal

Unfortunately, I can’t think of anything better to do to debug this than to add puts statements in the code to see how ruby/openssl are handling your cert. If you manage to figure it out, I’d love to hear what you found so I could maybe add subject alt name debugging to knife ssl check.

Also, on the chef-client side, SSL certs are getting added to the list used for HTTPS connections in this code: https://github.com/opscode/chef/blob/2cc728f2dd85e11835d23d03f76e0e4c75ca2510/lib/chef/http/ssl_policies.rb#L109

You can find that locally with

/opt/chef/embedded/bin/gem which chef/http/ssl_policies

We really ought to have some debug logging there so you can see which certs are getting loaded. If you could file a ticket, or, even better, contribute a patch, that would be much appreciated.


Daniel DeLeo

On Friday, July 25, 2014 at 1:20 PM, Stephen Corbesero wrote:

This is the error I get when I specify the actual hostname or its subject alternative name.

[root@oh-chef01 sbin]# knife ssl check -c /etc/chef/client.rb

Connecting to host oh-chef01.devops.dev.cloud.synchronoss.net:443 (http://oh-chef01.devops.dev.cloud.synchronoss.net:443)
ERROR: The SSL certificate of oh-chef01.devops.dev.cloud.synchronoss.net (http://oh-chef01.devops.dev.cloud.synchronoss.net) could not be verified
Certificate issuer data: /C=US/ST=PA/L=Bethlehem/O=Synchronoss Technologies, Inc./OU=IT/CN=oh-chef01.devops.dev.cloud.synchronoss.net/emailAddress=stephen.corbesero@synchronoss.com (mailto:stephen.corbesero@synchronoss.com)

Configuration Info:

OpenSSL Configuration:

  • Version: OpenSSL 1.0.1h 5 Jun 2014
  • Certificate file: /opt/chef/embedded/ssl/cert.pem
  • Certificate directory: /opt/chef/embedded/ssl/certs
    Chef SSL Configuration:
  • ssl_ca_path: nil
  • ssl_ca_file: nil
  • trusted_certs_dir: “/etc/chef/trusted_certs”

Followed by the ‘To fix this error…’

It looks like it doesn’t like the certificate at all.

I have generated certs w/ SANs before, but not very often. I even use openssl to dump the text of my cert and do see the proper CN and the SAN.

Also, If I generate a simple cert using the command given to me originally, that works just fine. If my client (or knife ssl check) tries to connect on the CN in the certificate, it succeeds. If I try the other name, “knife ssl check” gives me a very polite message that I am trying one name, but the server is reporting the other.

Just to make sure that the server is in the correct state, I do a reconfigure and restart every time I generate a new cert.

-----Original Message-----
From: Daniel DeLeo [mailto:ddeleo@kallistec.com] On Behalf Of Daniel DeLeo
Sent: Friday, July 25, 2014 4:06 PM
To: chef@lists.opscode.com (mailto:chef@lists.opscode.com)
Subject: [chef] Re: RE: RE: Re: Re: How do I configure the ssl to make the chef client and server happy

On Friday, July 25, 2014 at 10:18 AM, Stephen Corbesero wrote:

More follow-ups…

I’ve built my pem & key just using the system openssl with a conf file to generate the SANs. I fetch it via “knife ssl fetch” which puts it in the /etc/chef/trusted-certs/ dir. But chef-client is still failing. And ‘knife ssl check’ fails saying I need the cert there, but it is there.

Am I still missing a step?

What failure message are you getting from 'knife ssl check’ ?


Daniel DeLeo

-----Original Message-----
From: Stephen Corbesero [mailto:Stephen.Corbesero@synchronoss.com]
Sent: Friday, July 25, 2014 11:16 AM
To: chef@lists.opscode.com (mailto:chef@lists.opscode.com)
Subject: [chef] RE: Re: Re: How do I configure the ssl to make the chef client and server happy

Thank you Noah and Daniel,

Follow-up questions:

Do I need to use the openssl inside the /opt/chef-server/… dirs so it gets the right openssl cnf file?

-----Original Message-----
From: Daniel DeLeo [mailto:ddeleo@kallistec.com] On Behalf Of Daniel DeLeo
Sent: Thursday, July 24, 2014 1:57 PM
To: chef@lists.opscode.com (mailto:chef@lists.opscode.com)
Subject: [chef] Re: Re: How tdo I configure the ssl to make the chef client and server happy

On Thursday, July 24, 2014 at 10:54 AM, Noah Kantrowitz wrote:

Just generate the certificate/key yourself and provide it to the server. To make a self-signed cert:

$ openssl req -x509 -newkey rsa:4096 -keyout chef.key -out chef.pem -nodes -days 365

And then in your /etc/chef-server/chef-server.rb (you may have to create it):

nginx[‘ssl_certificate’] = '/etc/chef-server/chef.pem’
nginx[‘ssl_certificate_key’] = ‘/etc/chef-server/chef.key’

You’ll need to distribute the chef.pem to all clients as well, and configure it as a trusted CA certificate.

–Noah

I think you’ll also need to set the SubjectAltName field to include each of the hostnames you wish to use.


Daniel DeLeo


#9

Daniel,

I have submitted a new issue on the new github/opscode/chef project. I will be away at Chef Intermediate Topics training for the next two days, so I won’t have a chance to do much more debugging on this until I get back.

Thanks for your insights so far. If I stumble across anything, I will certainly pass it on.

-----Original Message-----
From: Daniel DeLeo [mailto:ddeleo@kallistec.com] On Behalf Of Daniel DeLeo
Sent: Friday, July 25, 2014 4:53 PM
To: chef@lists.opscode.com
Subject: [chef] Re: RE: Re: RE: RE: Re: Re: How do I configure the ssl to make the chef client and server happy

Looks like ruby ought to be checking your subject alt name entries. The code for this in ruby 1.9.3 (that ships with omnibus) is here: https://github.com/ruby/ruby/blob/5fc2510a654e764a9d6c5a261fc345ca2c1982c9/ext/openssl/lib/openssl/ssl-internal.rb#L87

You can find your local copy of it by running the command:

/opt/chef/embedded/bin/gem which openssl/ssl-internal

Unfortunately, I can’t think of anything better to do to debug this than to add puts statements in the code to see how ruby/openssl are handling your cert. If you manage to figure it out, I’d love to hear what you found so I could maybe add subject alt name debugging to knife ssl check.

Also, on the chef-client side, SSL certs are getting added to the list used for HTTPS connections in this code: https://github.com/opscode/chef/blob/2cc728f2dd85e11835d23d03f76e0e4c75ca2510/lib/chef/http/ssl_policies.rb#L109

You can find that locally with

/opt/chef/embedded/bin/gem which chef/http/ssl_policies

We really ought to have some debug logging there so you can see which certs are getting loaded. If you could file a ticket, or, even better, contribute a patch, that would be much appreciated.


Daniel DeLeo

On Friday, July 25, 2014 at 1:20 PM, Stephen Corbesero wrote:

This is the error I get when I specify the actual hostname or its subject alternative name.

[root@oh-chef01 sbin]# knife ssl check -c /etc/chef/client.rb

Connecting to host oh-chef01.devops.dev.cloud.synchronoss.net:443
(http://oh-chef01.devops.dev.cloud.synchronoss.net:443)
ERROR: The SSL certificate of
oh-chef01.devops.dev.cloud.synchronoss.net
(http://oh-chef01.devops.dev.cloud.synchronoss.net) could not be
verified Certificate issuer data:
/C=US/ST=PA/L=Bethlehem/O=Synchronoss Technologies,
Inc./OU=IT/CN=oh-chef01.devops.dev.cloud.synchronoss.net/emailAddress=
stephen.corbesero@synchronoss.com
(mailto:stephen.corbesero@synchronoss.com)

Configuration Info:

OpenSSL Configuration:

  • Version: OpenSSL 1.0.1h 5 Jun 2014
  • Certificate file: /opt/chef/embedded/ssl/cert.pem
  • Certificate directory: /opt/chef/embedded/ssl/certs Chef SSL
    Configuration:
  • ssl_ca_path: nil
  • ssl_ca_file: nil
  • trusted_certs_dir: “/etc/chef/trusted_certs”

Followed by the ‘To fix this error…’

It looks like it doesn’t like the certificate at all.

I have generated certs w/ SANs before, but not very often. I even use openssl to dump the text of my cert and do see the proper CN and the SAN.

Also, If I generate a simple cert using the command given to me originally, that works just fine. If my client (or knife ssl check) tries to connect on the CN in the certificate, it succeeds. If I try the other name, “knife ssl check” gives me a very polite message that I am trying one name, but the server is reporting the other.

Just to make sure that the server is in the correct state, I do a reconfigure and restart every time I generate a new cert.

-----Original Message-----
From: Daniel DeLeo [mailto:ddeleo@kallistec.com] On Behalf Of Daniel
DeLeo
Sent: Friday, July 25, 2014 4:06 PM
To: chef@lists.opscode.com (mailto:chef@lists.opscode.com)
Subject: [chef] Re: RE: RE: Re: Re: How do I configure the ssl to make
the chef client and server happy

On Friday, July 25, 2014 at 10:18 AM, Stephen Corbesero wrote:

More follow-ups…

I’ve built my pem & key just using the system openssl with a conf file to generate the SANs. I fetch it via “knife ssl fetch” which puts it in the /etc/chef/trusted-certs/ dir. But chef-client is still failing. And ‘knife ssl check’ fails saying I need the cert there, but it is there.

Am I still missing a step?

What failure message are you getting from 'knife ssl check’ ?


Daniel DeLeo

-----Original Message-----
From: Stephen Corbesero [mailto:Stephen.Corbesero@synchronoss.com]
Sent: Friday, July 25, 2014 11:16 AM
To: chef@lists.opscode.com (mailto:chef@lists.opscode.com)
Subject: [chef] RE: Re: Re: How do I configure the ssl to make the
chef client and server happy

Thank you Noah and Daniel,

Follow-up questions:

Do I need to use the openssl inside the /opt/chef-server/… dirs so it gets the right openssl cnf file?

-----Original Message-----
From: Daniel DeLeo [mailto:ddeleo@kallistec.com] On Behalf Of Daniel
DeLeo
Sent: Thursday, July 24, 2014 1:57 PM
To: chef@lists.opscode.com (mailto:chef@lists.opscode.com)
Subject: [chef] Re: Re: How tdo I configure the ssl to make the chef
client and server happy

On Thursday, July 24, 2014 at 10:54 AM, Noah Kantrowitz wrote:

Just generate the certificate/key yourself and provide it to the server. To make a self-signed cert:

$ openssl req -x509 -newkey rsa:4096 -keyout chef.key -out
chef.pem -nodes -days 365

And then in your /etc/chef-server/chef-server.rb (you may have to create it):

nginx[‘ssl_certificate’] = '/etc/chef-server/chef.pem’
nginx[‘ssl_certificate_key’] = ‘/etc/chef-server/chef.key’

You’ll need to distribute the chef.pem to all clients as well, and configure it as a trusted CA certificate.

–Noah

I think you’ll also need to set the SubjectAltName field to include each of the hostnames you wish to use.


Daniel DeLeo