http_request resource using client certificates


#1

Something new has popped in my environment whereby I need to make
http_requests while supplying client side certificates for authentication.
This is relatively easy to do using native Ruby code (snippet below) but no
so easy using the existing http_request resource. Using the built in
resource would require constructing all of the headers by hand which is a
black art as far as I can tell.

Before I write my own LWRP I want to know if anyone else has come across
this requirement and solved it with either custom code or figuring out the
additional headers that need to be supplied to http_request.

As always, any help is greatly appreciated.

#!/opt/chef/embedded/bin/ruby

require "net/https"
require "openssl"
require “uri”

uri = URI.parse(“SOME_URL”)

cert = File.open(“THE_CERT”) { |file| file.read }
key = File.open(“THE_KEY”) { |file| file.read }

http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.cert = OpenSSL::X509::Certificate.new(cert)
http.key = OpenSSL::PKey::RSA.new(key)
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
res = http.get(uri.request_uri)


#2

Would the right solution be extending http_request (
https://docs.chef.io/resource_http_request.html) to have attributes for
client_cert and client_key?

On Fri, Feb 27, 2015 at 5:11 PM, Mark Selby mselby@thenextbigsound.com
wrote:

Something new has popped in my environment whereby I need to make
http_requests while supplying client side certificates for authentication.
This is relatively easy to do using native Ruby code (snippet below) but no
so easy using the existing http_request resource. Using the built in
resource would require constructing all of the headers by hand which is a
black art as far as I can tell.

Before I write my own LWRP I want to know if anyone else has come across
this requirement and solved it with either custom code or figuring out the
additional headers that need to be supplied to http_request.

As always, any help is greatly appreciated.

#!/opt/chef/embedded/bin/ruby

require "net/https"
require "openssl"
require “uri”

uri = URI.parse(“SOME_URL”)

cert = File.open(“THE_CERT”) { |file| file.read }
key = File.open(“THE_KEY”) { |file| file.read }

http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.cert = OpenSSL::X509::Certificate.new(cert)
http.key = OpenSSL::PKey::RSA.new(key)
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
res = http.get(uri.request_uri)

Peter Burkholder — Customer Success Engineer

Unavailability: Travel March 2-3; Vacation March 16-20; ChefConf March
30-April 3

301-204-5767 – pburkholder@chef.io – *my: *Linkedin
http://www.linkedin.com/in/pburkholder Twitter
http://www.twitter.com/pburkholder Cal
https://www.google.com/calendar/embed?src=pburkholder%40chef.io&mode=WEEK
endar

CHEF

CHEF.IO http://www.chef.io/

TM

chef.io http://www.chef.io/ Blog http://www.chef.io/blog/ Facebook
https://www.facebook.com/getchefdotcom Twitter
https://twitter.com/chef Youtube https://www.youtube.com/getchef


#3

On Friday, February 27, 2015 at 3:37 PM, Peter Burkholder wrote:

Would the right solution be extending http_request (https://docs.chef.io/resource_http_request.html) to have attributes for client_cert and client_key?

On Fri, Feb 27, 2015 at 5:11 PM, Mark Selby <mselby@thenextbigsound.com (mailto:mselby@thenextbigsound.com)> wrote:

Something new has popped in my environment whereby I need to make http_requests while supplying client side certificates for authentication. This is relatively easy to do using native Ruby code (snippet below) but no so easy using the existing http_request resource. Using the built in resource would require constructing all of the headers by hand which is a black art as far as I can tell.

Before I write my own LWRP I want to know if anyone else has come across this requirement and solved it with either custom code or figuring out the additional headers that need to be supplied to http_request.
Chef can already configure this globally. ssl_client_cert and ssl_client_key are the settings you want. See https://docs.chef.io/chef_client_security.html#client-rb-settings The relevant code is here: https://github.com/chef/chef/blob/master/lib/chef/http/ssl_policies.rb

So you could try configuring that globally and see if it works. Though it’s kinda ugly since clearly it’d be better to configure for just the one request. It’s probably possible to do that by munging Chef::Config in ruby_blocks, though that’s ugly as well.

As always, any help is greatly appreciated.

#!/opt/chef/embedded/bin/ruby

require "net/https"
require "openssl"
require “uri”

uri = URI.parse(“SOME_URL”)

cert = File.open(“THE_CERT”) { |file| file.read }
key = File.open(“THE_KEY”) { |file| file.read }

http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.cert = OpenSSL::X509::Certificate.new(cert)
http.key = OpenSSL::PKey::RSA.new(key)
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
res = http.get(uri.request_uri)

I think you really want VERIFY_PEER here. If the server has a self-signed certificate, you can pull that down with knife ssl fetch or use some other means to copy it to Chef’s trusted certs dir. Seems kinda crazy that you’d go through the trouble to have the server verify the client like that and not have the client verify the server.

As Peter suggests, the most elegant thing to do would be to add support for mutual auth into the http_request resource. Feel free to reach out if you’d like to contribute such a thing but need help to do so.

HTH,


Daniel DeLeo


#4

I would most definitely like to extend the current http_request resource to
allow to use a custom ssl_policy defined outside of the Chef::Config space.

I will preface all this with the fact that I know enough ruby to write some
moderately complex LWRPs but would not consider myself a truly experienced
Ruby developer. I may need some pointers along the way.

Please let me know if you want to go private with this thread while we work
though the questions.

Here is my first question. I have taken a look at the current
http_request.rb provider and see the following requirement model I pasted
below.

I see that the basic_client seems to apply the default policy whenever the
scheme is HTTPS and it uses the values from Chef::Config. I am having a
little problem following the code path of the http object but I am
wondering since the http_request provider actually uses the basic_client
library, why is the ssl_policy not getting applied when the HTTPS is in
effect for the general http_request resource?

It seems to me that again the best way to get the behavior I want to to use
the already existing logic in basic_client.rb but figure out a way to
supply an alternate config. Right now it seems that the options from
Chef::Config are the only ones that can be used.

if url.scheme == HTTPS
      configure_ssl(http_client)
    end

provider http_request.rb

require ‘chef/http/simple’

chef/http/simple.rb

require 'chef/http’
require 'chef/http/authenticator’
require 'chef/http/decompressor’
require 'chef/http/cookie_manager’
require ‘chef/http/validate_content_length’

chef/http.rb

require 'tempfile’
require 'net/https’
require 'uri’
require 'chef/http/basic_client’
require 'chef/monkey_patches/string’
require 'chef/monkey_patches/net_http’
require 'chef/config’
require 'chef/platform/query_helpers’
require ‘chef/exceptions’

I have taken a look at the current

On Fri, Feb 27, 2015 at 5:17 PM, Daniel DeLeo dan@kallistec.com wrote:

On Friday, February 27, 2015 at 3:37 PM, Peter Burkholder wrote:

Would the right solution be extending http_request (
https://docs.chef.io/resource_http_request.html) to have attributes for
client_cert and client_key?

On Fri, Feb 27, 2015 at 5:11 PM, Mark Selby <mselby@thenextbigsound.com
(mailto:mselby@thenextbigsound.com)> wrote:

Something new has popped in my environment whereby I need to make
http_requests while supplying client side certificates for authentication.
This is relatively easy to do using native Ruby code (snippet below) but no
so easy using the existing http_request resource. Using the built in
resource would require constructing all of the headers by hand which is a
black art as far as I can tell.

Before I write my own LWRP I want to know if anyone else has come
across this requirement and solved it with either custom code or figuring
out the additional headers that need to be supplied to http_request.
Chef can already configure this globally. ssl_client_cert and
ssl_client_key are the settings you want. See
https://docs.chef.io/chef_client_security.html#client-rb-settings The
relevant code is here:
https://github.com/chef/chef/blob/master/lib/chef/http/ssl_policies.rb

So you could try configuring that globally and see if it works. Though
it’s kinda ugly since clearly it’d be better to configure for just the one
request. It’s probably possible to do that by munging Chef::Config in
ruby_blocks, though that’s ugly as well.

As always, any help is greatly appreciated.

#!/opt/chef/embedded/bin/ruby

require "net/https"
require "openssl"
require “uri”

uri = URI.parse(“SOME_URL”)

cert = File.open(“THE_CERT”) { |file| file.read }
key = File.open(“THE_KEY”) { |file| file.read }

http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.cert = OpenSSL::X509::Certificate.new(cert)
http.key = OpenSSL::PKey::RSA.new(key)
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
res = http.get(uri.request_uri)

I think you really want VERIFY_PEER here. If the server has a self-signed
certificate, you can pull that down with knife ssl fetch or use some
other means to copy it to Chef’s trusted certs dir. Seems kinda crazy that
you’d go through the trouble to have the server verify the client like that
and not have the client verify the server.

As Peter suggests, the most elegant thing to do would be to add support
for mutual auth into the http_request resource. Feel free to reach out if
you’d like to contribute such a thing but need help to do so.

HTH,


Daniel DeLeo


#5

On Sunday, March 1, 2015 at 5:53 PM, Mark Selby wrote:

I would most definitely like to extend the current http_request resource to allow to use a custom ssl_policy defined outside of the Chef::Config space.

I will preface all this with the fact that I know enough ruby to write some moderately complex LWRPs but would not consider myself a truly experienced Ruby developer. I may need some pointers along the way.

Please let me know if you want to go private with this thread while we work though the questions.
Awesome! The chef-dev mailing list would be the right place for this if you want to stick with an email thread. If you prefer you can start a github pull request with whatever work you have, and at-me @danielsdeleo in the description so I get the notifications and we can work through questions there.

I’ll reply to your questions here, if you want to stick with email, just send your follow-up reply to chef-dev@lists.opscode.com

Here is my first question. I have taken a look at the current http_request.rb provider and see the following requirement model I pasted below.

I see that the basic_client seems to apply the default policy whenever the scheme is HTTPS and it uses the values from Chef::Config. I am having a little problem following the code path of the http object but I am wondering since the http_request provider actually uses the basic_client library, why is the ssl_policy not getting applied when the HTTPS is in effect for the general http_request resource?
It looks like it would be applied, what makes you say it is not?

It seems to me that again the best way to get the behavior I want to to use the already existing logic in basic_client.rb but figure out a way to supply an alternate config. Right now it seems that the options from Chef::Config are the only ones that can be used.
Yep, this is what you’ll have to do. Chef’s HTTP code is designed around the idea that you create a HTTP object for a particular server host and port (this is the stuff in lib/chef/http.rb) and then the details of a particular request are handled by a different object (BasicClient). There’s a decent bit of cruft because everything used to be in lib/chef/rest.rb (which has some behaviors that are convenient for talking to the server API but are annoying for general HTTP usage). Anyway, what you’ll need to do is add an option to Chef:HTTP#initialize that customizes the SSL behavior and thread that through to BasicClient and then on to DefaultSSLPolicy.

HTH,


Daniel DeLeo