Encrypted data bag questions

Chef Infra (archive)
1

Folks,

So, I’ve gone through the documentation at http://docs.opscode.com/chef/essentials_data_bags.html, http://docs.opscode.com/essentials_data_bags_encrypt.html, and http://docs.opscode.com/knife_data_bag.html, and I’m a little confused by the behaviour we’re seeing.

From what I can tell, encrypted data bags are implemented on top of regular data bags, and since the goal is primarily to protect the data while it’s on the Chef server (e.g., in case of compromise of Hosted Chef), it seems to me that all of the crypto should be done on the client side.

OTOH, from what I can tell, it seems that you have to actually have a Chef server in order to be able to interact with and use encrypted data bags, which precludes their use with Chef Solo.

Is that correct? If so, can someone explain to me why that is the case? What is keeping Chef Solo from being able to use encrypted data bags with the same shared secret?

Also, any word on http://tickets.opscode.com/browse/CHEF-4233? From what we can see, if you create the encrypted data bag on a workstation running knife where Chef 10.18.2 is installed, this situation doesn’t seem to happen. In contrast, if you use knife from a Chef 11.x box, you get the extra keys which bork everything.

–
Brad Knowles brad@shub-internet.org
LinkedIn Profile: http://tinyurl.com/y8kpxu

2

On Tuesday, June 11, 2013 at 7:04 AM, Brad Knowles wrote:

Folks,

So, I've gone through the documentation at http://docs.opscode.com/chef/essentials_data_bags.html, http://docs.opscode.com/essentials_data_bags_encrypt.html, and http://docs.opscode.com/knife_data_bag.html, and I'm a little confused by the behaviour we're seeing.

From what I can tell, encrypted data bags are implemented on top of regular data bags, and since the goal is primarily to protect the data while it's on the Chef server (e.g., in case of compromise of Hosted Chef), it seems to me that all of the crypto should be done on the client side.

OTOH, from what I can tell, it seems that you have to actually have a Chef server in order to be able to interact with and use encrypted data bags, which precludes their use with Chef Solo.

Is that correct? If so, can someone explain to me why that is the case? What is keeping Chef Solo from being able to use encrypted data bags with the same shared secret?
The only thing that's missing is the plumbing. Knife's support for encrypting data bags is built in to the commands that upload data bags. We use a knife exec script to encrypt plaintext data bag items like so:

data_bag_path = "data_bags/#{your_data_bag_name_here"

plaintext_data = File.open(plaintext_json_path) {|f| Yajl::Parser.parse(f) }
encrypted_item = Chef::EncryptedDataBagItem.encrypt_data_bag_item(plaintext_data, dev_secret)

encrypted_json_path = "#{data_bag_path}/#{File.basename(plaintext_json_path)}"
File.open(encrypted_json_path, "w") {|f| Yajl::Encoder.encode(encrypted_item.to_hash, f, :pretty => true) }

Adding this capability to knife would be a welcome patch.

Also, any word on http://tickets.opscode.com/browse/CHEF-4233? From what we can see, if you create the encrypted data bag on a workstation running knife where Chef 10.18.2 is installed, this situation doesn't seem to happen. In contrast, if you use knife from a Chef 11.x box, you get the extra keys which bork everything.
This looks to me like either a knife essentials bug or incompatibility between knife essentials and Chef 10.x. We just merged a ton of knife-essentials enhancements to (what will become) Chef 11.6, so it may be fixed already--I'll check with John on that. In any case, the knife data bag commands should work fine.

--
Brad Knowles <brad@shub-internet.org (mailto:brad@shub-internet.org)>

--
Daniel DeLeo