Question about data bags - best practice

Chef Infra (archive)
1

Now i write cookbook bacula for setup bacula server and clients.

I have multiple recipes: database - for setup mysql db, server for setup
bacula-director and storage and client for setup bacula-fd on clients.

I write next code for load passwords for encrypted data bags:

encrypted_data_bag_secret_path =
"#{node[‘chef’][‘encrypted_data_bag_secret_path’]}/encrypted_data_bag_secret"
bacula_secret =
Chef::EncryptedDataBagItem.load_secret("#{encrypted_data_bag_secret_path}")

------- Setup mysql service users passwords used in cookbook for

succesful installation of MYSQL
item = Chef::EncryptedDataBagItem.load(“bacula”,“mysql_root”, bacula_secret)
node.set[‘mysql’][‘server_root_password’] = item[‘password’]
node.set[‘mysql’][‘server_repl_password’] = item[‘password’]
node.set[‘mysql’][‘server_debian_password’] = item[‘password’]

------- Setup mysql user “bacula”

item = Chef::EncryptedDataBagItem.load(“bacula”,“mysql_bacula”,
bacula_secret)
node.set[‘bacula’][‘mysql’][‘name’] = item[‘name’]
node.set[‘bacula’][‘mysql’][‘password’] = item[‘password’]

------- Setup bacula passwords used at director and storage machine

item = Chef::EncryptedDataBagItem.load(“bacula”,“director”, bacula_secret)
node.set[‘bacula’][‘director’][‘password’] = item[‘password’]

item = Chef::EncryptedDataBagItem.load(“bacula”,“monitor”, bacula_secret)
node.set[‘bacula’][‘monitor’][‘password’] = item[‘password’]

item = Chef::EncryptedDataBagItem.load(“bacula”,“storage”, bacula_secret)
node.set[‘bacula’][‘storage’][‘password’] = item[‘password’]

item = Chef::EncryptedDataBagItem.load(“bacula”,“root”, bacula_secret)
node.set[‘bacula’][‘root’][‘password’] = item[‘password’]

As we can see this single recipe set node parametrs, which later used in
my other recipes.

I think that this is not best practice. Because:

  1. This code place in /recipes/_loaddatabags.rb And this recipe run with
    include_recipe in several recipes for example in database, server and
    storage recipe.

I think it’s code must run once. This is a main problem.

  1. This code setup node vars. What happens after the recipe is executed
    ? Where traces remain with our passwords in an open way. Except of
    course the configuration files ?

  2. I think that should be a funtion or lwrp for this entitys. But I
    cannot represent to self how can i do it. Chef examples on site is very
    small.

  3. As my small practice I cannot represent how can i architect/design
    this in best way?

  4. Where better to define these variables
    ([‘mysql’][‘server_root_password’], [‘bacula’][‘director’][‘password’])
    ? In what part of cookbook ? What do you think ?

  5. I use chef 10.x version. Maybe there are other more elegant ways to
    work with data bag’s

  6. I plan to lay out your cookbook on opscode in the future. And I would
    like it to be ideologically correct.

  7. I have not seen in cookbooks from opscode use of data bags. Why ?

Thank you very much.

–
Best regards,

CVision Lab System Administrator
Vladmir Skubriev

  • English - detected

  • English

  • English

<javascript:void(0);>

2

Take a look at chef-vault (https://github.com/nordstrom/chef-vault) Keeps the data encrypted using the private key of the client itself.

You are right however that setting the value into a node attribute persists that data, in a chef-client run, back to the chef server at the end of a successful run. You don’t want to decrypt your passwords and then put them in an attribute. You want to decrypt and use that value to define a resource at compile time. At the end of convergence that resource leaves memory and your decrypted values go away.

Kevin

From: Vladimir Skubriev <skubriev@cvisionlab.commailto:skubriev@cvisionlab.com>
Reply-To: "chef@lists.opscode.commailto:chef@lists.opscode.com" <chef@lists.opscode.commailto:chef@lists.opscode.com>
Date: Tuesday, August 6, 2013 6:20 AM
To: "chef@lists.opscode.commailto:chef@lists.opscode.com" <chef@lists.opscode.commailto:chef@lists.opscode.com>
Subject: [chef] question about data bags - best practice

Now i write cookbook bacula for setup bacula server and clients.

I have multiple recipes: database - for setup mysql db, server for setup bacula-director and storage and client for setup bacula-fd on clients.

I write next code for load passwords for encrypted data bags:

encrypted_data_bag_secret_path = “#{node[‘chef’][‘encrypted_data_bag_secret_path’]}/encrypted_data_bag_secret"
bacula_secret = Chef::EncryptedDataBagItem.load_secret(”#{encrypted_data_bag_secret_path}")

------- Setup mysql service users passwords used in cookbook for succesful installation of MYSQL

item = Chef::EncryptedDataBagItem.load(“bacula”,“mysql_root”, bacula_secret)
node.set[‘mysql’][‘server_root_password’] = item[‘password’]
node.set[‘mysql’][‘server_repl_password’] = item[‘password’]
node.set[‘mysql’][‘server_debian_password’] = item[‘password’]

------- Setup mysql user “bacula”

item = Chef::EncryptedDataBagItem.load(“bacula”,“mysql_bacula”, bacula_secret)
node.set[‘bacula’][‘mysql’][‘name’] = item[‘name’]
node.set[‘bacula’][‘mysql’][‘password’] = item[‘password’]

------- Setup bacula passwords used at director and storage machine

item = Chef::EncryptedDataBagItem.load(“bacula”,“director”, bacula_secret)
node.set[‘bacula’][‘director’][‘password’] = item[‘password’]

item = Chef::EncryptedDataBagItem.load(“bacula”,“monitor”, bacula_secret)
node.set[‘bacula’][‘monitor’][‘password’] = item[‘password’]

item = Chef::EncryptedDataBagItem.load(“bacula”,“storage”, bacula_secret)
node.set[‘bacula’][‘storage’][‘password’] = item[‘password’]

item = Chef::EncryptedDataBagItem.load(“bacula”,“root”, bacula_secret)
node.set[‘bacula’][‘root’][‘password’] = item[‘password’]

As we can see this single recipe set node parametrs, which later used in my other recipes.

I think that this is not best practice. Because:

  1. This code place in /recipes/_loaddatabags.rb And this recipe run with include_recipe in several recipes for example in database, server and storage recipe.

I think it’s code must run once. This is a main problem.

  1. This code setup node vars. What happens after the recipe is executed ? Where traces remain with our passwords in an open way. Except of course the configuration files ?

  2. I think that should be a funtion or lwrp for this entitys. But I cannot represent to self how can i do it. Chef examples on site is very small.

  3. As my small practice I cannot represent how can i architect/design this in best way?

  4. Where better to define these variables ([‘mysql’][‘server_root_password’], [‘bacula’][‘director’][‘password’]) ? In what part of cookbook ? What do you think ?

  5. I use chef 10.x version. Maybe there are other more elegant ways to work with data bag’s

  6. I plan to lay out your cookbook on opscode in the future. And I would like it to be ideologically correct.

  7. I have not seen in cookbooks from opscode use of data bags. Why ?

Thank you very much.

–
Best regards,

CVision Lab System Administrator
Vladmir Skubriev

  • English - detected

  • English

  • English

<javascript:void(0);>

3

On Tuesday, August 6, 2013 at 7:37 AM, Moser, Kevin wrote:

Take a look at chef-vault (GitHub - chef/chef-vault: Securely manage passwords, certs, and other secrets in Chef) Keeps the data encrypted using the private key of the client itself.

You are right however that setting the value into a node attribute persists that data, in a chef-client run, back to the chef server at the end of a successful run. You don't want to decrypt your passwords and then put them in an attribute. You want to decrypt and use that value to define a resource at compile time. At the end of convergence that resource leaves memory and your decrypted values go away.

Kevin

There is a hash on the node object you can use for per-run state info. It is not persisted to the server. To access it:

node.run_state[:my_key] = "my value"

HTH,

--
Daniel DeLeo

4

On 06.08.2013 18:37, Moser, Kevin wrote:

Take a look at chef-vault (GitHub - chef/chef-vault: Securely manage passwords, certs, and other secrets in Chef) Keeps the data encrypted using the private key of the client itself.
This is a very interesting idea of crypt/decrypt with an existing
private key.
I understand that you proposed to use chef private key -
/etc/chef/client.pem

Maybe just use ecsdsa ssh private key?
I think that ecdsa is more than enough.
While the relationship with key chef server too very good idea of course.

You are right however that setting the value into a node attribute persists that data, in a chef-client run, back to the chef server at the end of a successful run.
I.e. when chef run my code node attributes should be setupped right way.
And I don't need to run this code in all recipes. Only in the first recipe.
How long this attributes are still stored on the chef-server?
You don't want to decrypt your passwords and then put them in an attribute.
Stop. How will I be able to set the attributes for the cookbook mysql.
for example

['mysql']['server_root_password']
'mysql']['server_repl_password']
['mysql']['server_debian_password']

I think that define this attribute in a role is no good idea. May be I don't right. Because only node has access to its attributes?

How is the procedure for obtaining the attributes associated host and the private key is stored on the client in the folder / etc / chef / client.pem?

I think that between them there is a connection. But now I find it difficult to understand this

You want to decrypt and use that value to define a resource at compile time.
May be yes, may be not. I can't understand how is right.
At the end of convergence that resource leaves memory and your decrypted values go away.
On the client after chef run will be nothing ?

What else cleans chef-client at the end of the run?

Kevin

Thank you very much for answers.
I am a newbie in chef. And want to understand it mush more.
I have language barriers - excusme for bad english.

--
Best regards,

CVision Lab System Administrator
Vladmir Skubriev

5

On 06.08.2013 19:01, Daniel DeLeo wrote:

There is a hash on the node object you can use for per-run state info.
It is not persisted to the server. To access it:

node.run_state[:my_key] = "my value"

how can i link run_state attributes to mysql cookbook attributes ?

example:

node.run_state['mysql']['server_root_password'] = item['password']

then it is not persisted to the server ?

Who can access to node attributes on the server? In previous message i
suggested that only node, which run their recipes.

Then why i must use node.run_state ?

is i am right ?

--
Best regards,

CVision Lab System Administrator
Vladmir Skubriev

6

On Tuesday, August 6, 2013 at 8:27 AM, Vladimir Skubriev wrote:

On 06.08.2013 19:01, Daniel DeLeo wrote:

There is a hash on the node object you can use for per-run state info.
It is not persisted to the server. To access it:

node.run_state[:my_key] = "my value"
how can i link run_state attributes to mysql cookbook attributes ?

example:

node.run_state['mysql']['server_root_password'] = item['password']

then it is not persisted to the server ?
node.run_state is a separate field in the node object that is not included in the serialized json. You can see for yourself by printing node.to_json.

Who can access to node attributes on the server? In previous message i
suggested that only node, which run their recipes.

All node attributes are publicly readable by all other nodes via search.

Then why i must use node.run_state ?
The run_state field is generally useful for storing any data that you only need to keep for the duration of the chef run.

is i am right ?

--
Best regards,

CVision Lab System Administrator
Vladmir Skubriev

--
Daniel DeLeo

7

On 06.08.2013 19:37, Daniel DeLeo wrote:

node.run_state is a separate field in the node object that is not
included in the serialized json. You can see for yourself by printing
node.to_json.

Just in case:

I must use node.run_state['mysql']['server_root_password'] instead of
node['mysql']['server_root_password'] ?

Thank you!

--
Best regards,

CVision Lab System Administrator
Vladmir Skubriev

8

On 06.08.2013 19:37, Daniel DeLeo wrote:

node.run_state is a separate field in the node object that is not
included in the serialized json. You can see for yourself by printing
node.to_json.
when i was do in recipe:
node.run_state['mysql']['server_root_password'] = 'secretpassword'

i recive error:
undefined method `=' for nil:NilClass

Why ?

--
Best regards,

CVision Lab System Administrator
Vladmir Skubriev

9

I'll take a guess here:

Because node.run_state['mysql'] wasn't defined first.

On Tue, Aug 6, 2013 at 10:56 PM, Vladimir Skubriev
skubriev@cvisionlab.comwrote:

On 06.08.2013 19:37, Daniel DeLeo wrote:

node.run_state is a separate field in the node object that is not
included in the serialized json. You can see for yourself by printing
node.to_json.

when i was do in recipe:
node.run_state['mysql']['**server_root_password'] = 'secretpassword'

i recive error:
undefined method `=' for nil:NilClass

Why ?

--
Best regards,

CVision Lab System Administrator
Vladmir Skubriev

10

Vladimir,

This previous posthttp://lists.opscode.com/sympa/arc/chef/2013-05/msg00361.html
describes
the approach I've taken to set a library cookbook's password attribute
while preventing it from living on Chef server.

Thanks,
Kevin Christen

On Tue, Aug 6, 2013 at 10:27 AM, Vladimir Skubriev
skubriev@cvisionlab.comwrote:

On 06.08.2013 19:01, Daniel DeLeo wrote:

There is a hash on the node object you can use for per-run state info. It
is not persisted to the server. To access it:

node.run_state[:my_key] = "my value"

how can i link run_state attributes to mysql cookbook attributes ?

example:

node.run_state['mysql']['**server_root_password'] = item['password']

then it is not persisted to the server ?

Who can access to node attributes on the server? In previous message i
suggested that only node, which run their recipes.

Then why i must use node.run_state ?

is i am right ?

--
Best regards,

CVision Lab System Administrator
Vladmir Skubriev