Get chef-vault shared secret

Chef Infra (archive)
1

Can anybody tell me how can I get chef-vault shared secret, used to encrypt data in encrypted data bags.
As far as I understand i can see secret keys, encryped by public key of admin and/or clients by issuing following command: knife data bag show $VAULT $ITEM_keys

So theoretically I can do something like that:

  1. Copy encrypted data for example for admin users
  2. openssl enc -in /root/encrypted_data -out /root/encrypted_data.binary -d -a
  3. openssl rsautl -decrypt -in /root/encrypted_data.binary -out /root/plaintext -inkey admin.pem
  4. base64 /root/plaintext > /root/plaintext.base64

But unfortunately something is wrong with this workflow, since I can decrypt data bag using /root/plaintext.base64 file by issuing following command:
knife data bag show $VAULT $ITEM --secret-file /root/plaintest.bas64

Any ideas ?

Thanks in advance.

2

I’m finally got to the state, when I can visually verify, that binary representation of shared secret are the same of what I’ve got on step 3 and in file ‘lib/chef-vault/item.rb’, line 288. The only problem right now is:

  1. when I try to use the binary representation of unencrypted shared secret using knife data bag show … --secret-file /root/plaintext command I get error “ERROR: ArgumentError: invalid byte sequence in UTF-8”
  2. When I add step 4 and use command “knife data bag show credentials database --secret-file /root/plaintext.base64” I get error “ERROR: Chef::EncryptedDataBagItem::DecryptionFailure: Error decrypting data bag value: ‘’”

So what I think is probably correctly encoding things to pass it to the knife data bag subcommand. Any comments on that ?

3

I can not help you directly with your problem, but could you maybe tell us why you want to manually encrypt/decrypt a vault? You could just use the chef-vault gem and knife to do the same thing locally which would save you a lot of painful work.

4

Hi. Thanks for replay. The main idea behind all this is the ability to keep
encrypted data inside of VCS and since, as far as I can see, vault is
basically an encrypted data bag, we can use this shared key to
encrypt/decrypt and/or directly update only vault’s data bag, without
touching metadata and key part. And after that we can use directly update
data bag by uploading already encrypted file, that comes from VCS.

Juris Krumins

5

I thought so. :slight_smile: The problem I see it that you still need the public key of the Node/Admin to encrypt the _keys DataBag.
But, you can still use chef vault in combination with chef zero to do so. This older but still valid Blog pos describes the procedure: http://jtimberman.housepub.org/blog/2013/09/10/managing-secrets-with-chef-vault/ . The only thing you need to change is setting the mode from client to solo althoug I am not sure right now how client/admin public keys are handled in this scenario.
But usually the procedure is to change a vault and then download it as a data bag (in encrypted format) to the VCS.

6

As far as I understand I don’t need any public/private key of any
node/client. In my dev environment I can develop data bag, upload, test and
then commit it to VCS in encrypted form. And the deployment process will
proceed without knowing any keys. And the only thing I need to encrypt data
bag before commiting to VCS is a shared secret I’m trying to get now.
I’m not going to manage VAULT_keys data bag, that’s the vault’s problem.

Juris Krumins

7

Well then you still could have the _keys locally in you VCS too and run “knife vault update … --local-mode --mode solo --key solo_user.pem --user ‘solo_user’” and have the solo_user in your _keys.

8

You proposal is good. But by keeping _keys data bag in VCS I immediately get into public key sync issue, since we don’t manage pub/priv keys of nodes in VCS. And sooner or later they will get out of sync with what Chef server have and what we have in VCS.
With encrypted data bag I think it’s easier. All you need is knife data bag create … --secret-file /etc/filename.

9

But in your case you will have the secret to decrypt the vault lying around in plain text…

10

Yes, you’re right. It’s trade-off. But unfortunately I can’t event my case case I can’t figure out how to correctly convert binary representation of shared secret key to the form knife data bag will accept. And thus I can’t test all possible solutions.

11

Well thats a massive tradeoff but if you really think its a good idea you probably have to use the Ruby API to encrypt and decrypt the Data Bag: https://github.com/chef/chef-vault/blob/895fb67e278bb86c732157abdd1c2eed3fb436a7/lib/chef-vault/item.rb#L380
One of the strengths of Vaults is that the secret can contain non printable characters which is hard to do in Bash.

12

Yes. I’ve seen this.
And the problem is, when I try to do following in the same file:

197     # private
198     def generate_secret(key_size = 32)
199       # Defaults to 32 bytes, as this is the size that a Chef
200       # Encrypted Data Bag Item will digest all secrets down to anyway
201       sec=SecureRandom.random_bytes(key_size)
202       File.open('/tmp/shared_secret', 'wb') { |file| file.write(sec) }
203       sec
204     end

I get binary representation of shared secret in /tmp/shared_secret file and then using base64 or openssl encI can convert it to base64 representation. Only after that I can use it in knife data bag file. But it keeps saying that the shared key is incorrect and it’s not possible to decrypt data in data bag.

13

As I said, you will have to use Ruby to decrypt and encrypt the DataBag. Knife was not designed to work with non human readable keys.

14

Ok. I’ll try. Thanks for the tip.

Juris Krumins

15

I utilize chef-vault, with a VCS management style but for the integration with VCS we use https://github.com/elasticdog/transcrypt which allows any information to be encrypted and decrypted in our git repo.

16

This is interesting. Thanks pmonson, I’ll consider this as a possible solution.