Populating encrypted_data_bag_secret?

Folks,

We’ve recently started using encrypted data bags, and we’re stumbling over the problem of how to populate this file early in the chef-client run, so that we don’t bomb out on the initial install.

Of course, I’ve read the posts at http://jtimberman.github.com/, as that was one of the primary sources of information I used to help get us bootstrapped on using encrypted data bags to begin with, but one thing that was missing from these posts was any discussion of how to do the population of the /etc/chef/encrypted_data_bag_secret file during an initial install.

Looking through the release notes, I found http://tickets.opscode.com/browse/CHEF-2434 which supposedly is fixed in 0.10.6, but it’s not clear to me how to make use of this new feature or how to upgrade knife on my workstation to allow me to make use of this new feature. Is this as simple as doing a “gem upgrade knife”? Does that also install the new templates that allow me to pre-populate this file during the initial install of a machine? I’m also curious to learn what happens to the old template files that I’ve been using (especially omnibus.erb).

Any and all assistance you can provide will be appreciated, especially including where in TFM that I should be R’ing. Thanks!


Brad Knowles bknowles@ihiji.com
SAGE Level IV, Chef Level 0.0.1

With 0.10.6 or later, once you're upgraded, add this to ~/.chef/knife.rb:

encrypted_data_bag_secret "#{current_dir}/data_bag_key"

Of course, if your file is called something besides "data_bag_key"
then change the name of the file above.

I'm not sure if these changes come in with 0.10.6 y default. If they
don't, or if you've got templates that you've invested time into
customizing, then add this to the template sometime around when it's
creating the client.rb:

See [CHEF-2434] add encrypted_data_bag_secret support to bootstrap subcom… · schisamo/chef@d0789ff · GitHub

The below relies on the config parameter "encrypted_data_bag_secret"

existing in the

knife config. This allows secrets to be stored on a less trusted server.

echo -e "date \n\n**** \n**** Knifing in the chef encrypted data bag
secret file:\n****\n"
(
cat <<EOP
<%= encrypted_data_bag_secret %>
EOP
) > /tmp/knife-bootstrap/encrypted_data_bag_secret
awk NF /tmp/knife-bootstrap/encrypted_data_bag_secret >
/etc/chef/encrypted_data_bag_secret
chmod 600 /tmp/knife-bootstrap/encrypted_data_bag_secret
chmod 600 /etc/chef/encrypted_data_bag_secret

This chunk depends on /tmp/knife-bootstrap existing in your template.

I don't know for sure what will happen with your current templates if
you've modified them but I suspect that in the common case you're
installing a new gem so all that will happen is that you'll have your
old templates in the old gem directory and the new ones in the new
directory. You'll probably have to manually reconcile them.

-Peter

On Tue, Dec 27, 2011 at 7:13 PM, Brad Knowles bknowles@ihiji.com wrote:

Folks,

We've recently started using encrypted data bags, and we're stumbling over the problem of how to populate this file early in the chef-client run, so that we don't bomb out on the initial install.

Of course, I've read the posts at http://jtimberman.github.com/, as that was one of the primary sources of information I used to help get us bootstrapped on using encrypted data bags to begin with, but one thing that was missing from these posts was any discussion of how to do the population of the /etc/chef/encrypted_data_bag_secret file during an initial install.

Looking through the release notes, I found http://tickets.opscode.com/browse/CHEF-2434 which supposedly is fixed in 0.10.6, but it's not clear to me how to make use of this new feature or how to upgrade knife on my workstation to allow me to make use of this new feature. Is this as simple as doing a "gem upgrade knife"? Does that also install the new templates that allow me to pre-populate this file during the initial install of a machine? I'm also curious to learn what happens to the old template files that I've been using (especially omnibus.erb).

Any and all assistance you can provide will be appreciated, especially including where in TFM that I should be R'ing. Thanks!

--
Brad Knowles bknowles@ihiji.com
SAGE Level IV, Chef Level 0.0.1

On Tue, Dec 27, 2011 at 7:13 PM, Brad Knowles bknowles@ihiji.com wrote:

Folks,

We've recently started using encrypted data bags, and we're stumbling over the problem of how to populate this file early in the chef-client run, so that we don't bomb out on the initial install.

Of course, I've read the posts at http://jtimberman.github.com/, as that was one of the primary sources of information I used to help get us bootstrapped on using encrypted data bags to begin with, but one thing that was missing from these posts was any discussion of how to do the population of the /etc/chef/encrypted_data_bag_secret file during an initial install.

Looking through the release notes, I found http://tickets.opscode.com/browse/CHEF-2434 which supposedly is fixed in 0.10.6, but it's not clear to me how to make use of this new feature or how to upgrade knife on my workstation to allow me to make use of this new feature. Is this as simple as doing a "gem upgrade knife"? Does that also install the new templates that allow me to pre-populate this file during the initial install of a machine? I'm also curious to learn what happens to the old template files that I've been using (especially omnibus.erb).

Any and all assistance you can provide will be appreciated, especially including where in TFM that I should be R'ing. Thanks!

--
Brad Knowles bknowles@ihiji.com
SAGE Level IV, Chef Level 0.0.1

You could do it the way my original databag_decrypt LWRP worked which
is use a remote host to store the key. We're still using my original
LWRP but I'm pretty sure my patch for the official method got pulled
in quite some time ago. That is, store the key on a private host
accessible only from your servers.

We store the key in a Noah ephemeral but you can store it anywhere
that will respond to HTTP. You can see the source code here:

https://github.com/opscode/chef/blob/master/chef/lib/chef/encrypted_data_bag_item.rb#L105-127

Essentially instead of passing a file path, put an http url.
Technically it doesn't have to be an http endpoint. Anything that
open-uri can handle will work.