Distribute ssl certs using encrypted data bags

Chef Infra (archive)
1

anyone have examples they can share on how they’re disting ssl
certificates from encrypted data bags? or, if not from data bags,
how are you doing it?

i’m studying 37 signals’ ssl_certificates cookbook, but there’s no
example therein using data bags.

i’m starting by trying to toss in the cert info into a data bag, but
this json is invalid?

[chef-repo]$ knife data bag from file --secret-file ~/.chef/encrypted_data_bag_secret hush data_bags/hush/aws-creds-gurf.json
ERROR: JSON::ParserError: 705: unexpected token at '{

[chef-repo]$ cat data_bags/hush/aws-creds-gurf.json
{
“id”:“aws-creds-gurf”,
“AWS_ACCOUNT_ID”:“OU81-2222-5150”,
“ROOT_AWS_ACCESS_KEY_ID”:“gurfaccesskeysplicersplicer”,
“ROOT_AWS_SECRET_ACCESS_KEY”:“gurfsecretkeydaddydaddy”,
“DORQ_AWS_ACCESS_KEY_ID”:“dorqaccesskeysistersister”,
“DORQ_AWS_SECRET_ACCESS_KEY”:“dorqsecretkeyplasmidplasmid”,
“servercert”:"-----BEGIN CERTIFICATE-----
MIICeqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqAYTbqqqqq
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
GEFXUyBMaW1pdGVkLUFzc3VyYW5jZSBDQTAeFw0wOTA3MDIwMTIyNTZaFw0xMDA3
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
nzANBgkqhkiG9w0BAQEFAAOBjQAwgYkCgYEAqNBSjUvc/oQgN8kRja+8d6b4z1hD
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
bWUr4dNogq3lqet04444EL+pLtMUEA52ZIdRG2w2Sn03CpUbbpoh5CpCyVcpYmcl
ZDAwaSx2D0+jadcCAwEAAaNXMFUwDgYDVR0PAQH/BAQDAgWgMBYGA1UdJQEB/wQM
MAoGCCsGAQUFBwMCMAwGA1UdEwEB/wQCMAAwHQYDVR0OBBYEFDypRtGMWzc86nOf
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
Y5Lgnkm7ESir9BSLnIFIhD9ibnaZuWUn+BNB5kzugsbPCnak7uyxFBSUp0rsEmOr
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
vPH8Bow18I4d6u7g
-----END CERTIFICATE-----"
}

can such certs be one continuous line? i don’t think so, but i admit
i didn’t verify that.

what to do?

schlanks,
kallen

2

On Jan 6, 2012, at 4:52 PM, kallen@groknaut.net wrote:

anyone have examples they can share on how they're disting ssl
certificates from encrypted data bags? or, if not from data bags,
how are you doing it?

We're using encrypted data bags. Basically, just just cut-n-paste the key and the cert into an encrypted data bag, and then you have to put together a recipe to pull that stuff out of the encrypted data bag and hand the decrypted secret as a variable to your web.conf.erb template that is being used by the apache2 cookbook to create the actual website(s) you're serving.

For a sample certs.rb file, see https://gist.github.com/b775da964e97a8b8c185. The corresponding .crt and .key files look like this:

-----BEGIN CERTIFICATE-----
<%= @ssl_cert %>
-----END CERTIFICATE-----
Tue Dec 20 15:03:52 CST 2011

-----BEGIN RSA PRIVATE KEY-----
<%= @ssl_key %>
-----END RSA PRIVATE KEY-----
Tue Dec 20 15:05:49 CST 2011

The corresponding domain_ssl.json file might look something like this (note that this is encrypted):

{
"cert": "za90CmUqQhVYYwP4xgxoGnCdKMvjqOBl7e52mcgoBmkA6ofsFnjR/9oQ0P9x\n...LOTS.OF.GARBAGE.DELETED...\n5F1/WZjp8LyIAooI9Cz2w1eDQwuax0w3xAvc\n",
"id": "domain_net_ssl",
"key": "xRp8e8PNWfJFYzViJpqxF5+zwWrdm2roLxNtp75wh2so06nKzcuy3P0JvFV+\n...LOTS.OF.GARBAGE.DELETED...\nsgZdBrEsxAtgcH5A\n"
}

When stored in .json format, you will get the single continuous line like that, with what appear to be embedded newline characters. However, you cut-n-paste the ASCII text version of the certificates and keys when you're creating the encrypted databag, and internally Chef will do the conversion for you. Of course, once you create the encrypted data bag directly on the chef server, you might want to then re-export that back out to a local file that you can store in the encrypted format, which is how I was able to provide the listing above.

Works well for us.

Of course, we did come very late to the party where we discovered that wildcard certs like *.domain.tld doesn't mean that you can use them to serve up proper SSL-secured sites with names like service.subdomain.subdomain.domain.tld, because the wildcard explicitly EXCLUDES the dot character. So, if you also want to serve up subdomains and subsubdomains and use wildcard certs for those as well, then you'll need to buy extra wildcard certs at each level. Blech.

--
Brad Knowles bknowles@ihiji.com
SAGE Level IV, Chef Level 0.0.1

3

excellent! thanks brad. that got me the push i needed. this is working
for me now. but i have a couple questions for you / the list.

question 1 --

when i cut n pasted the ASCII version of the cert into my json file,
then "data bag from file'd" it, chef did not do any conversion on
those newlines... if i understood you correctly. to illustrate:

i pasted my cert:

{
"id":"aws-creds-gurf",
"servercert":"MIICeDblahblahblahblahblahblah
AlVTMRblahblahblahblahblahblah
aklejblah="
}

knife data bag from file --secret-file ~/.chef/encrypted_data_bag_secret hush data_bags/hush/aws-creds-gurf.json
ERROR: JSON::ParserError: 705: unexpected token at '{

so to get this to work, i had to insert \n by hand:
{
"id":"aws-creds-gurf",
"servercert":"MIICeDblahblahblahblahblahblah\nAlVTMRblahblahblahblahblahblah\naklejblah="
}

then json was happy. i wonder if i misunderstood what you meant by
copy-pasting and chef doing some internal conversion. maybe the
conversion happens at rendering time. did you insert \n by hand?

question 2 --

if it were important, how would i get rid of the final newline in
my resulting pem file? the app that uses the cert and key are behaving
ok with there being a final newline. but istr some anecdote about
needing to NOT have a final newline character. or maybe i'm
misremembering and this is a silly question.

On Fri, 06 Jan 2012, Brad Knowles wrote:

On Jan 6, 2012, at 4:52 PM, kallen@groknaut.net wrote:

anyone have examples they can share on how they're disting ssl
certificates from encrypted data bags? or, if not from data bags,
how are you doing it?

We're using encrypted data bags. Basically, just just cut-n-paste the key and the cert into an encrypted data bag, and then you have to put together a recipe to pull that stuff out of the encrypted data bag and hand the decrypted secret as a variable to your web.conf.erb template that is being used by the apache2 cookbook to create the actual website(s) you're serving.

For a sample certs.rb file, see https://gist.github.com/b775da964e97a8b8c185. The corresponding .crt and .key files look like this:

-----BEGIN CERTIFICATE-----
<%= @ssl_cert %>
-----END CERTIFICATE-----
Tue Dec 20 15:03:52 CST 2011

-----BEGIN RSA PRIVATE KEY-----
<%= @ssl_key %>
-----END RSA PRIVATE KEY-----
Tue Dec 20 15:05:49 CST 2011

The corresponding domain_ssl.json file might look something like this (note that this is encrypted):

{
"cert": "za90CmUqQhVYYwP4xgxoGnCdKMvjqOBl7e52mcgoBmkA6ofsFnjR/9oQ0P9x\n...LOTS.OF.GARBAGE.DELETED...\n5F1/WZjp8LyIAooI9Cz2w1eDQwuax0w3xAvc\n",
"id": "domain_net_ssl",
"key": "xRp8e8PNWfJFYzViJpqxF5+zwWrdm2roLxNtp75wh2so06nKzcuy3P0JvFV+\n...LOTS.OF.GARBAGE.DELETED...\nsgZdBrEsxAtgcH5A\n"
}

When stored in .json format, you will get the single continuous line like that, with what appear to be embedded newline characters. However, you cut-n-paste the ASCII text version of the certificates and keys when you're creating the encrypted databag, and internally Chef will do the conversion for you. Of course, once you create the encrypted data bag directly on the chef server, you might want to then re-export that back out to a local file that you can store in the encrypted format, which is how I was able to provide the listing above.

Works well for us.

1 Like
4

Re: Sticking certificates in databags:

cat /your/private/key | sed s/$/\\n/ | tr -d '\n'

copy and paste into your JSON =)

-s

On Sat, Jan 7, 2012 at 12:19 AM, kallen@groknaut.net wrote:

excellent! thanks brad. that got me the push i needed. this is working
for me now. but i have a couple questions for you / the list.

question 1 --

when i cut n pasted the ASCII version of the cert into my json file,
then "data bag from file'd" it, chef did not do any conversion on
those newlines... if i understood you correctly. to illustrate:

i pasted my cert:

{
"id":"aws-creds-gurf",
"servercert":"MIICeDblahblahblahblahblahblah
AlVTMRblahblahblahblahblahblah
aklejblah="
}

knife data bag from file --secret-file ~/.chef/encrypted_data_bag_secret hush data_bags/hush/aws-creds-gurf.json
ERROR: JSON::ParserError: 705: unexpected token at '{

so to get this to work, i had to insert \n by hand:
{
"id":"aws-creds-gurf",
"servercert":"MIICeDblahblahblahblahblahblah\nAlVTMRblahblahblahblahblahblah\naklejblah="
}

then json was happy. i wonder if i misunderstood what you meant by
copy-pasting and chef doing some internal conversion. maybe the
conversion happens at rendering time. did you insert \n by hand?

question 2 --

if it were important, how would i get rid of the final newline in
my resulting pem file? the app that uses the cert and key are behaving
ok with there being a final newline. but istr some anecdote about
needing to NOT have a final newline character. or maybe i'm
misremembering and this is a silly question.

On Fri, 06 Jan 2012, Brad Knowles wrote:

On Jan 6, 2012, at 4:52 PM, kallen@groknaut.net wrote:

anyone have examples they can share on how they're disting ssl
certificates from encrypted data bags? or, if not from data bags,
how are you doing it?

We're using encrypted data bags. Basically, just just cut-n-paste the key and the cert into an encrypted data bag, and then you have to put together a recipe to pull that stuff out of the encrypted data bag and hand the decrypted secret as a variable to your web.conf.erb template that is being used by the apache2 cookbook to create the actual website(s) you're serving.

For a sample certs.rb file, see https://gist.github.com/b775da964e97a8b8c185. The corresponding .crt and .key files look like this:

-----BEGIN CERTIFICATE-----
<%= @ssl_cert %>
-----END CERTIFICATE-----
Tue Dec 20 15:03:52 CST 2011

-----BEGIN RSA PRIVATE KEY-----
<%= @ssl_key %>
-----END RSA PRIVATE KEY-----
Tue Dec 20 15:05:49 CST 2011

The corresponding domain_ssl.json file might look something like this (note that this is encrypted):

{
"cert": "za90CmUqQhVYYwP4xgxoGnCdKMvjqOBl7e52mcgoBmkA6ofsFnjR/9oQ0P9x\n...LOTS.OF.GARBAGE.DELETED...\n5F1/WZjp8LyIAooI9Cz2w1eDQwuax0w3xAvc\n",
"id": "domain_net_ssl",
"key": "xRp8e8PNWfJFYzViJpqxF5+zwWrdm2roLxNtp75wh2so06nKzcuy3P0JvFV+\n...LOTS.OF.GARBAGE.DELETED...\nsgZdBrEsxAtgcH5A\n"
}

When stored in .json format, you will get the single continuous line like that, with what appear to be embedded newline characters. However, you cut-n-paste the ASCII text version of the certificates and keys when you're creating the encrypted databag, and internally Chef will do the conversion for you. Of course, once you create the encrypted data bag directly on the chef server, you might want to then re-export that back out to a local file that you can store in the encrypted format, which is how I was able to provide the listing above.

Works well for us.

5

On Jan 6, 2012, at 6:19 PM, kallen@groknaut.net wrote:

question 1 --

when i cut n pasted the ASCII version of the cert into my json file,
then "data bag from file'd" it, chef did not do any conversion on
those newlines... if i understood you correctly. to illustrate:

The instructions from jtimberman that I followed do not involve creating a file from which you can do a "knife data bag from file". You do a "knife data bag create" instead, and there is no permanent file created that has the data stored in unencrypted plaintext format. His instructions are at http://jtimberman.posterous.com/64227128.

then json was happy. i wonder if i misunderstood what you meant by
copy-pasting and chef doing some internal conversion. maybe the
conversion happens at rendering time. did you insert \n by hand?

When you do the "knife data bag show ... -Fj", that will do the conversion for you into json format -- that's what the "-Fj" is for. You don't use json format for input into the creation process, it's an output format.

question 2 --

if it were important, how would i get rid of the final newline in
my resulting pem file? the app that uses the cert and key are behaving
ok with there being a final newline. but istr some anecdote about
needing to NOT have a final newline character. or maybe i'm
misremembering and this is a silly question.

I'm not sure I've got any useful advice for this case.

--
Brad Knowles bknowles@ihiji.com
SAGE Level IV, Chef Level 0.0.1