# Encrypted data bags

**URL:** <https://discourse.chef.io/t/encrypted-data-bags/3987>\
**Category:** Chef Infra (archive)\
**Created:** [May 20, 2013, 9:09pm UTC](https://discourse.chef.io/t/encrypted-data-bags/3987 "2013-05-20T21:09:31Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Astlock](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/astlock/32/441_2.png) [@Astlock](https://discourse.chef.io/u/Astlock)\
**Post date:** [May 20, 2013, 9:09pm UTC](https://discourse.chef.io/t/encrypted-data-bags/3987/1 "2013-05-20T21:09:31Z")

</div>

Hello.

I have a few questions about using encrypted things in Chef.

1)What is the best practice to distribute encrypted\_data\_bag\_secret to servers?  
2)What to do with bootstrapped servers? Does everyone put it in advance?

---

<div class="post-metadata">

**Author:** ![BradKnowles](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/bradknowles/32/99_2.png) [@BradKnowles](https://discourse.chef.io/u/BradKnowles)\
**Post date:** [May 21, 2013, 1:08am UTC](https://discourse.chef.io/t/encrypted-data-bags/3987/2 "2013-05-21T01:08:04Z")

</div>

On May 20, 2013, at 3:09 PM, [nowarry@gmail.com](mailto:nowarry@gmail.com) wrote:

> I have a few questions about using encrypted things in Chef.
> 
> 1)What is the best practice to distribute encrypted\_data\_bag\_secret to servers?

The short answer is that these things do NOT work like you think they do.

Keep in mind that the only purpose of encrypted data bags is to keep the contents encrypted while it remains on the Chef server. This was meant primarily for use with Hosted Chef, so that people could feel comfortable that their secrets were not exposed if Opscode should happen to have a security problem and the contents of these data bags were somehow leaked.

The contents have to be unencrypted on the Chef Workstation, so that you can encrypt them with the shared key, and then upload the encrypted data bag to the Chef Server.

The contents have to be unencrypted on the nodes, because otherwise it would be impossible to access the information stored in them. And since a shared key is used, once a single node is compromised, then all data bags that were encrypted with that same shared key are now vulnerable across your entire infrastructure.

I repeat, the short answer is that these things do NOT work like you think they do.

> 2)What to do with bootstrapped servers? Does everyone put it in advance?

Yup. It's a shared secret file. It has to be copied over from the Chef Workstation to the nodes at bootstrap time.

--  
Brad Knowles [brad@shub-internet.org](mailto:brad@shub-internet.org)  
LinkedIn Profile: [http://tinyurl.com/y8kpxu](http://tinyurl.com/y8kpxu)

---

<div class="post-metadata">

**Author:** ![kallistec](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/kallistec/32/23_2.png) [@kallistec](https://discourse.chef.io/u/kallistec)\
**Post date:** [May 21, 2013, 3:21pm UTC](https://discourse.chef.io/t/encrypted-data-bags/3987/3 "2013-05-21T15:21:43Z")

</div>

On Monday, May 20, 2013 at 6:08 PM, Brad Knowles wrote:

> On May 20, 2013, at 3:09 PM, [nowarry@gmail.com](mailto:nowarry@gmail.com) ([mailto:nowarry@gmail.com](mailto:nowarry@gmail.com)) wrote:
> 
> > I have a few questions about using encrypted things in Chef.
> > 
> > 1)What is the best practice to distribute encrypted\_data\_bag\_secret to servers?
> 
> The short answer is that these things do NOT work like you think they do.
> 
> Keep in mind that the only purpose of encrypted data bags is to keep the contents encrypted while it remains on the Chef server. This was meant primarily for use with Hosted Chef, so that people could feel comfortable that their secrets were not exposed if Opscode should happen to have a security problem and the contents of these data bags were somehow leaked.
> 
> The contents have to be unencrypted on the Chef Workstation, so that you can encrypt them with the shared key, and then upload the encrypted data bag to the Chef Server.
> 
> The contents have to be unencrypted on the nodes, because otherwise it would be impossible to access the information stored in them. And since a shared key is used, once a single node is compromised, then all data bags that were encrypted with that same shared key are now vulnerable across your entire infrastructure.
> 
> I repeat, the short answer is that these things do NOT work like you think they do.  
> This isn't strictly true. At Opscode we have some S3 keys that we only want to make available to one or two machines. Since this is a low churn part of our infrastructure and protecting the keys is worth the hassle, we manually copy the keys over when bootstrapping these nodes. The cookbooks that handle these keys have conditional logic to emit warnings and skip some resources if the encrypted data bag secret isn't available.

The key thing to understand is that there's not a single best practice for all use cases. If you just want to hide your data from Opscode or anyone who might have access to your Chef server's database, then a single shared secret distributed to all nodes will accomplish that. If you only want Chef client on your database servers to see the root database password, then something like we do at Opscode may make more sense.

> --  
> Brad Knowles \<[brad@shub-internet.org](mailto:brad@shub-internet.org) ([mailto:brad@shub-internet.org](mailto:brad@shub-internet.org))\>  
> LinkedIn Profile: [http://tinyurl.com/y8kpxu](http://tinyurl.com/y8kpxu)

--  
Daniel DeLeo

---

<div class="post-metadata">

**Author:** ![Jesse\_Nelson](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/jesse_nelson/32/262_2.png) [@Jesse\_Nelson](https://discourse.chef.io/u/Jesse_Nelson)\
**Post date:** [May 21, 2013, 5:17pm UTC](https://discourse.chef.io/t/encrypted-data-bags/3987/4 "2013-05-21T17:17:36Z")

</div>

Gonna toss a shout out to knife-vault[1] here. Which will let you encrypt  
a bag using the key from a host (or set of hosts). In this way you don't  
have to keep/distribute another secret, and only those hosts can get at  
that bag.

1. [GitHub - moserke/chef-vault: Development repository for Opscode's chef-vault cookbook](https://github.com/moserke/chef-vault)

On Tue, May 21, 2013 at 8:21 AM, Daniel DeLeo [dan@kallistec.com](mailto:dan@kallistec.com) wrote:

> On Monday, May 20, 2013 at 6:08 PM, Brad Knowles wrote:
> 
> On May 20, 2013, at 3:09 PM, [nowarry@gmail.com](mailto:nowarry@gmail.com) wrote:
> 
> I have a few questions about using encrypted things in Chef.
> 
> 1)What is the best practice to distribute encrypted\_data\_bag\_secret to  
> servers?
> 
> The short answer is that these things do NOT work like you think they do.
> 
> Keep in mind that the only purpose of encrypted data bags is to keep the  
> contents encrypted while it remains on the Chef server. This was meant  
> primarily for use with Hosted Chef, so that people could feel comfortable  
> that their secrets were not exposed if Opscode should happen to have a  
> security problem and the contents of these data bags were somehow leaked.
> 
> The contents have to be unencrypted on the Chef Workstation, so that you  
> can encrypt them with the shared key, and then upload the encrypted data  
> bag to the Chef Server.
> 
> The contents have to be unencrypted on the nodes, because otherwise it  
> would be impossible to access the information stored in them. And since a  
> shared key is used, once a single node is compromised, then all data bags  
> that were encrypted with that same shared key are now vulnerable across  
> your entire infrastructure.
> 
> I repeat, the short answer is that these things do NOT work like you think  
> they do.
> 
> This isn't strictly true. At Opscode we have some S3 keys that we only  
> want to make available to one or two machines. Since this is a low churn  
> part of our infrastructure and protecting the keys is worth the hassle, we  
> manually copy the keys over when bootstrapping these nodes. The cookbooks  
> that handle these keys have conditional logic to emit warnings and skip  
> some resources if the encrypted data bag secret isn't available.
> 
> The key thing to understand is that there's not a single best practice for  
> all use cases. If you just want to hide your data from Opscode or anyone  
> who might have access to your Chef server's database, then a single shared  
> secret distributed to all nodes will accomplish that. If you only want Chef  
> client on your database servers to see the root database password, then  
> something like we do at Opscode may make more sense.
> 
> --  
> Brad Knowles [brad@shub-internet.org](mailto:brad@shub-internet.org)  
> LinkedIn Profile: [http://tinyurl.com/y8kpxu](http://tinyurl.com/y8kpxu)
> 
> --  
> Daniel DeLeo

---

<div class="post-metadata">

**Author:** ![steve](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/steve/32/294_2.png) [@steve](https://discourse.chef.io/u/steve)\
**Post date:** [May 21, 2013, 6:45pm UTC](https://discourse.chef.io/t/encrypted-data-bags/3987/5 "2013-05-21T18:45:31Z")

</div>

For that matter, there's nothing that says you _have_ to encrypt/decrypt  
the data bag item with the key that's in the default location on the  
chef-client. You can distribute keys through a web service that performs  
some other means of authentication. You can distribute keys through your  
favorite orchestration layer, assuming you have one. (Or Pushy, though I  
haven't seen it running anywhere outside of Opscodeland yet... )

The nice thing about doing it that way is that re-keying is fairly  
straightforward, and the easier it is to re-key, the more likely it is that  
you'll do so on a regular basis.

That having been said, yeah, it's still a shared key and you're still  
compromising the data bag item if the key on disk gets compromised. But as  
long as you can write some Ruby code to deliver the key contents somehow to  
the function that loads the encrypted data bag item in your recipe (or  
library), you can solve this problem any way you like...

On Tue, May 21, 2013 at 10:17 AM, Jesse Nelson [spheromak@gmail.com](mailto:spheromak@gmail.com) wrote:

> Gonna toss a shout out to knife-vault[1] here. Which will let you encrypt  
> a bag using the key from a host (or set of hosts). In this way you don't  
> have to keep/distribute another secret, and only those hosts can get at  
> that bag.
> 
> 1. [GitHub - moserke/chef-vault: Development repository for Opscode's chef-vault cookbook](https://github.com/moserke/chef-vault)
> 
> On Tue, May 21, 2013 at 8:21 AM, Daniel DeLeo [dan@kallistec.com](mailto:dan@kallistec.com) wrote:
> 
> > On Monday, May 20, 2013 at 6:08 PM, Brad Knowles wrote:
> > 
> > On May 20, 2013, at 3:09 PM, [nowarry@gmail.com](mailto:nowarry@gmail.com) wrote:
> > 
> > I have a few questions about using encrypted things in Chef.
> > 
> > 1)What is the best practice to distribute encrypted\_data\_bag\_secret to  
> > servers?
> > 
> > The short answer is that these things do NOT work like you think they do.
> > 
> > Keep in mind that the only purpose of encrypted data bags is to keep the  
> > contents encrypted while it remains on the Chef server. This was meant  
> > primarily for use with Hosted Chef, so that people could feel comfortable  
> > that their secrets were not exposed if Opscode should happen to have a  
> > security problem and the contents of these data bags were somehow leaked.
> > 
> > The contents have to be unencrypted on the Chef Workstation, so that you  
> > can encrypt them with the shared key, and then upload the encrypted data  
> > bag to the Chef Server.
> > 
> > The contents have to be unencrypted on the nodes, because otherwise it  
> > would be impossible to access the information stored in them. And since a  
> > shared key is used, once a single node is compromised, then all data bags  
> > that were encrypted with that same shared key are now vulnerable across  
> > your entire infrastructure.
> > 
> > I repeat, the short answer is that these things do NOT work like you  
> > think they do.
> > 
> > This isn't strictly true. At Opscode we have some S3 keys that we only  
> > want to make available to one or two machines. Since this is a low churn  
> > part of our infrastructure and protecting the keys is worth the hassle, we  
> > manually copy the keys over when bootstrapping these nodes. The cookbooks  
> > that handle these keys have conditional logic to emit warnings and skip  
> > some resources if the encrypted data bag secret isn't available.
> > 
> > The key thing to understand is that there's not a single best practice  
> > for all use cases. If you just want to hide your data from Opscode or  
> > anyone who might have access to your Chef server's database, then a single  
> > shared secret distributed to all nodes will accomplish that. If you only  
> > want Chef client on your database servers to see the root database  
> > password, then something like we do at Opscode may make more sense.
> > 
> > --  
> > Brad Knowles [brad@shub-internet.org](mailto:brad@shub-internet.org)  
> > LinkedIn Profile: [http://tinyurl.com/y8kpxu](http://tinyurl.com/y8kpxu)
> > 
> > --  
> > Daniel DeLeo

---

<div class="post-metadata">

**Author:** ![BradKnowles](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/bradknowles/32/99_2.png) [@BradKnowles](https://discourse.chef.io/u/BradKnowles)\
**Post date:** [May 21, 2013, 7:05pm UTC](https://discourse.chef.io/t/encrypted-data-bags/3987/6 "2013-05-21T19:05:49Z")

</div>

On May 21, 2013, at 12:45 PM, steve . [leftathome@gmail.com](mailto:leftathome@gmail.com) wrote:

> For that matter, there's nothing that says you _have_ to encrypt/decrypt the data bag item with the key that's in the default location on the chef-client. You can distribute keys through a web service that performs some other means of authentication. You can distribute keys through your favorite orchestration layer, assuming you have one. (Or Pushy, though I haven't seen it running anywhere outside of Opscodeland yet... )

True enough, but where are the standard procedures or tools documented for doing any of this?

> That having been said, yeah, it's still a shared key and you're still compromising the data bag item if the key on disk gets compromised. But as long as you can write some Ruby code to deliver the key contents somehow to the function that loads the encrypted data bag item in your recipe (or library), you can solve this problem any way you like...

Sure, it's just code. You can do anything you want with code. You can re-write the entire operating system if you want -- it's just code.

A small matter of programming, that's all.

The key is that we don't want to have to invent or reinvent any wheels here.

Most of the people on this list are smarter and more advanced with Chef than we are, so if you've got any existing documented standard procedures or tools for how you do these sorts of things, we'd love to hear about it.

--  
Brad Knowles [brad@shub-internet.org](mailto:brad@shub-internet.org)  
LinkedIn Profile: [http://tinyurl.com/y8kpxu](http://tinyurl.com/y8kpxu)