Handling Passwords with Chef Solo

From: Stuart Ellis stuart@stuartellis.eu
Date: 5 June 2010 10:13:00 GMT+01:00
To: chef@lists.opscode.com
Subject: [chef] Handling Passwords with Chef Solo
Reply-To: chef@lists.opscode.com

Hello,

I've just started to use Chef Solo and have been really pleased with it, but am curious as to how people are currently managing passwords with it. This hasn't been an issue for most things, but I've now built up a fairly complete cookbook for a Rails server, and can see that I need to manage the MySQL root password and the password for a system account in order to fully automate the setup.

I'm sorry - I ought to have said "handling passwords securely". The facilities for setting system account and MySQL root passwords work fine, but I've so far used them by embedded these passwords into the JSON and test cookbook, which feels like the wrong approach.


Stuart Ellis
stuart@stuartellis.eu

I like Chef solo a lot. :-). wrt. credentials, what I have done is to use a KVP store which exposes a web service which is script friendly, thus no complicated XML answers. This way the scripts get the password only when they need it and you never need to persist the info into your SCM. If you prefer a pure WS, then you'll need to create a client for it.

There are a couple of issues to deal with thus:

  1. How do you secure access to the KVP store? -- For this you can use things like firewall rules or keys that are pre-injected into you images to authenticate clients against the KVP store.

  2. You're adding one more service, which means you'll need to manage it. In order to minimize this, make your KVP store use an already existing store? IOW, instead of XML or a WS, say you already have LDAP, maybe you just need a client program that queries LDAP for your credentials and you need to populate properly your LDAP hierarchy.

From: stuart@stuartellis.eu
Date: Sat, 5 Jun 2010 10:22:12 +0100
To: chef@lists.opscode.com
Subject: [chef] Handling Passwords with Chef Solo

From: Stuart Ellis stuart@stuartellis.eu
Date: 5 June 2010 10:13:00 GMT+01:00
To: chef@lists.opscode.com
Subject: [chef] Handling Passwords with Chef Solo
Reply-To: chef@lists.opscode.com

Hello,

I've just started to use Chef Solo and have been really pleased with it, but am curious as to how people are currently managing passwords with it. This hasn't been an issue for most things, but I've now built up a fairly complete cookbook for a Rails server, and can see that I need to manage the MySQL root password and the password for a system account in order to fully automate the setup.

I'm sorry - I ought to have said "handling passwords securely". The facilities for setting system account and MySQL root passwords work fine, but I've so far used them by embedded these passwords into the JSON and test cookbook, which feels like the wrong approach.


Stuart Ellis
stuart@stuartellis.eu


Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox.
http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_1

I like the approach taken in the 37s "users" cookbook:
http://github.com/37signals/37s_cookbooks/blob/master/users/attributes/default.rb

passwords are kept in chef, but as hashed passwords with salts
(openssl passwd -l)

Is there a problem storing a new-style mysql password?

On Sat, Jun 5, 2010 at 12:59 PM, Pierre Jacomet pierrj@hotmail.com wrote:

I like Chef solo a lot. :-). wrt. credentials, what I have done is to use a
KVP store which exposes a web service which is script friendly, thus no
complicated XML answers. This way the scripts get the password only when
they need it and you never need to persist the info into your SCM. If you
prefer a pure WS, then you'll need to create a client for it.

There are a couple of issues to deal with thus:

  1. How do you secure access to the KVP store? -- For this you can use things
    like firewall rules or keys that are pre-injected into you images to
    authenticate clients against the KVP store.

  2. You're adding one more service, which means you'll need to manage it. In
    order to minimize this, make your KVP store use an already existing store?
    IOW, instead of XML or a WS, say you already have LDAP, maybe you just need
    a client program that queries LDAP for your credentials and you need to
    populate properly your LDAP hierarchy.

From: stuart@stuartellis.eu
Date: Sat, 5 Jun 2010 10:22:12 +0100
To: chef@lists.opscode.com
Subject: [chef] Handling Passwords with Chef Solo

From: Stuart Ellis stuart@stuartellis.eu
Date: 5 June 2010 10:13:00 GMT+01:00
To: chef@lists.opscode.com
Subject: [chef] Handling Passwords with Chef Solo
Reply-To: chef@lists.opscode.com

Hello,

I've just started to use Chef Solo and have been really pleased with it,
but am curious as to how people are currently managing passwords with it.
This hasn't been an issue for most things, but I've now built up a fairly
complete cookbook for a Rails server, and can see that I need to manage the
MySQL root password and the password for a system account in order to fully
automate the setup.

I'm sorry - I ought to have said "handling passwords securely". The
facilities for setting system account and MySQL root passwords work fine,
but I've so far used them by embedded these passwords into the JSON and test
cookbook, which feels like the wrong approach.


Stuart Ellis
stuart@stuartellis.eu


Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox.
Learn more.

On 5 Jun 2010, at 18:09, Erik Kastner wrote:

I like the approach taken in the 37s "users" cookbook:
http://github.com/37signals/37s_cookbooks/blob/master/users/attributes/default.rb

passwords are kept in chef, but as hashed passwords with salts
(openssl passwd -l)

Is there a problem storing a new-style mysql password?

Thanks - I'm now specifying the password hash with a custom attribute:

http://github.com/stuartellis/automation/blob/master/chef/site-cookbooks/se/recipes/deployer_user.rb#L27

The MySQL root password is peculiar because the Opscode MySQL recipe requires it to be stored unencrypted so that it can be used to run a setup script:

http://github.com/opscode/cookbooks/blob/master/mysql/recipes/server.rb#L98

I think that the real problem was that I hadn't grasped that the node configuration file needs to be treated differently to the chef-solo configuration file and the cookbooks, and kept out of the repository.

There seems to be a similar issue with SSL - ideally the files need to be provided to Chef Solo as it runs, but not kept in repositories, so some kind of secure store definitely looks useful.

On Sat, Jun 5, 2010 at 12:59 PM, Pierre Jacomet pierrj@hotmail.com wrote:

I like Chef solo a lot. :-). wrt. credentials, what I have done is to use a
KVP store which exposes a web service which is script friendly, thus no
complicated XML answers. This way the scripts get the password only when
they need it and you never need to persist the info into your SCM. If you
prefer a pure WS, then you'll need to create a client for it.

There are a couple of issues to deal with thus:

  1. How do you secure access to the KVP store? -- For this you can use things
    like firewall rules or keys that are pre-injected into you images to
    authenticate clients against the KVP store.

  2. You're adding one more service, which means you'll need to manage it. In
    order to minimize this, make your KVP store use an already existing store?
    IOW, instead of XML or a WS, say you already have LDAP, maybe you just need
    a client program that queries LDAP for your credentials and you need to
    populate properly your LDAP hierarchy.

From: stuart@stuartellis.eu
Date: Sat, 5 Jun 2010 10:22:12 +0100
To: chef@lists.opscode.com
Subject: [chef] Handling Passwords with Chef Solo

From: Stuart Ellis stuart@stuartellis.eu
Date: 5 June 2010 10:13:00 GMT+01:00
To: chef@lists.opscode.com
Subject: [chef] Handling Passwords with Chef Solo
Reply-To: chef@lists.opscode.com

Hello,

I've just started to use Chef Solo and have been really pleased with it,
but am curious as to how people are currently managing passwords with it.
This hasn't been an issue for most things, but I've now built up a fairly
complete cookbook for a Rails server, and can see that I need to manage the
MySQL root password and the password for a system account in order to fully
automate the setup.

I'm sorry - I ought to have said "handling passwords securely". The
facilities for setting system account and MySQL root passwords work fine,
but I've so far used them by embedded these passwords into the JSON and test
cookbook, which feels like the wrong approach.


Stuart Ellis
stuart@stuartellis.eu


Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox.
Learn more.


Stuart Ellis
stuart@stuartellis.eu

I do not know whether I understand this:

I think that the real problem was that I hadn't grasped that the
node configuration file needs to be treated differently to the chef-solo
configuration file and the cookbooks, and kept out of the repository.

There seems to be a similar issue with SSL - ideally the files
need to be provided to Chef Solo as it runs, but not kept in
repositories, so some kind of secure store definitely looks useful.

So, here goes my observation: One of the drivers why I switched to chef-solo was that I was able to use different versions or snapshots taken from a repository, i.e. a tag or similar. Yes, indeed, you need a way to shuttle the files to the guest or alternately make them available to chef-solo using a remote tarball - although I have not tried the latter. -- As I mentioned I used the approach of shuttling the files to a local folder in the guest as this would make development / debugging easier.

Now, notice that the way you shuttle things could be a snapshot like svn export or could also be an svn checkout from within the guest, using SSL and in that case you would have a copy of the repository in the guest and you can point chef-solo to the set of cookbooks / roles in that repository. Handy, because now you can fix things in the guest, and when things work you commit and the next guest will get the fixes. A checkout operation however would not be as handy in a DVCS like git because in that case you would be shuttling too much unneeded stuff into the guest.

HTH
Pierre

From: stuart@stuartellis.eu
Date: Wed, 9 Jun 2010 01:10:05 +0100
To: chef@lists.opscode.com
Subject: [chef] Re: Re: RE: Handling Passwords with Chef Solo

On 5 Jun 2010, at 18:09, Erik Kastner wrote:

I like the approach taken in the 37s "users" cookbook:
http://github.com/37signals/37s_cookbooks/blob/master/users/attributes/default.rb

passwords are kept in chef, but as hashed passwords with salts
(openssl passwd -l)

Is there a problem storing a new-style mysql password?

Thanks - I'm now specifying the password hash with a custom attribute:

http://github.com/stuartellis/automation/blob/master/chef/site-cookbooks/se/recipes/deployer_user.rb#L27

The MySQL root password is peculiar because the Opscode MySQL recipe requires it to be stored unencrypted so that it can be used to run a setup script:

http://github.com/opscode/cookbooks/blob/master/mysql/recipes/server.rb#L98

I think that the real problem was that I hadn't grasped that the node configuration file needs to be treated differently to the chef-solo configuration file and the cookbooks, and kept out of the repository.

There seems to be a similar issue with SSL - ideally the files need to be provided to Chef Solo as it runs, but not kept in repositories, so some kind of secure store definitely looks useful.

On Sat, Jun 5, 2010 at 12:59 PM, Pierre Jacomet pierrj@hotmail.com wrote:

I like Chef solo a lot. :-). wrt. credentials, what I have done is to use a
KVP store which exposes a web service which is script friendly, thus no
complicated XML answers. This way the scripts get the password only when
they need it and you never need to persist the info into your SCM. If you
prefer a pure WS, then you'll need to create a client for it.

There are a couple of issues to deal with thus:

  1. How do you secure access to the KVP store? -- For this you can use things
    like firewall rules or keys that are pre-injected into you images to
    authenticate clients against the KVP store.

  2. You're adding one more service, which means you'll need to manage it. In
    order to minimize this, make your KVP store use an already existing store?
    IOW, instead of XML or a WS, say you already have LDAP, maybe you just need
    a client program that queries LDAP for your credentials and you need to
    populate properly your LDAP hierarchy.

From: stuart@stuartellis.eu
Date: Sat, 5 Jun 2010 10:22:12 +0100
To: chef@lists.opscode.com
Subject: [chef] Handling Passwords with Chef Solo

From: Stuart Ellis stuart@stuartellis.eu
Date: 5 June 2010 10:13:00 GMT+01:00
To: chef@lists.opscode.com
Subject: [chef] Handling Passwords with Chef Solo
Reply-To: chef@lists.opscode.com

Hello,

I've just started to use Chef Solo and have been really pleased with it,
but am curious as to how people are currently managing passwords with it.
This hasn't been an issue for most things, but I've now built up a fairly
complete cookbook for a Rails server, and can see that I need to manage the
MySQL root password and the password for a system account in order to fully
automate the setup.

I'm sorry - I ought to have said "handling passwords securely". The
facilities for setting system account and MySQL root passwords work fine,
but I've so far used them by embedded these passwords into the JSON and test
cookbook, which feels like the wrong approach.


Stuart Ellis
stuart@stuartellis.eu


Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox.
Learn more.


Stuart Ellis
stuart@stuartellis.eu


The New Busy is not the old busy. Search, chat and e-mail from your inbox.
http://www.windowslive.com/campaign/thenewbusy?ocid=PID28326::T:WLMTAGL:ON:WL:en-US:WM_HMP:042010_3

On 9 Jun 2010, at 02:27, Pierre Jacomet wrote:

I do not know whether I understand this:

I think that the real problem was that I hadn't grasped that the node configuration file needs to be treated differently to the chef-solo configuration file and the cookbooks, and kept out of the repository.

There seems to be a similar issue with SSL - ideally the files need to be provided to Chef Solo as it runs, but not kept in repositories, so some kind of secure store definitely looks useful.

So, here goes my observation: One of the drivers why I switched to chef-solo was that I was able to use different versions or snapshots taken from a repository, i.e. a tag or similar. Yes, indeed, you need a way to shuttle the files to the guest or alternately make them available to chef-solo using a remote tarball - although I have not tried the latter. -- As I mentioned I used the approach of shuttling the files to a local folder in the guest as this would make development / debugging easier.

Now, notice that the way you shuttle things could be a snapshot like svn export or could also be an svn checkout from within the guest, using SSL and in that case you would have a copy of the repository in the guest and you can point chef-solo to the set of cookbooks / roles in that repository. Handy, because now you can fix things in the guest, and when things work you commit and the next guest will get the fixes. A checkout operation however would not be as handy in a DVCS like git because in that case you would be shuttling too much unneeded stuff into the guest.

Sorry if I was unclear - it was a little late. I meant that the same security issue applies to both the node JSON file and any SSL certificate and key files needed for the system being configured. Chef Solo needs to access the files from a defined location at runtime to copy them into place, but they can't be stored with the cookbooks.

The situation is that I have a laptop with development VMs and a single live server, and have a Git repository of the Chef and Vagrant setup that I use to configure them. There are currently three copies of the Git repository - one on the laptop, one on the server and one now on Github, so that other people can reuse it (http://github.com/stuartellis/automation).

This is really a toy project for learning Chef, but presumably this issue applies in other situations where you want to distribute reusable setups (not just cookbooks) to external people.

From: stuart@stuartellis.eu
Date: Wed, 9 Jun 2010 01:10:05 +0100
To: chef@lists.opscode.com
Subject: [chef] Re: Re: RE: Handling Passwords with Chef Solo

On 5 Jun 2010, at 18:09, Erik Kastner wrote:

I like the approach taken in the 37s "users" cookbook:
http://github.com/37signals/37s_cookbooks/blob/master/users/attributes/default.rb

passwords are kept in chef, but as hashed passwords with salts
(openssl passwd -l)

Is there a problem storing a new-style mysql password?

Thanks - I'm now specifying the password hash with a custom attribute:

http://github.com/stuartellis/automation/blob/master/chef/site-cookbooks/se/recipes/deployer_user.rb#L27

The MySQL root password is peculiar because the Opscode MySQL recipe requires it to be stored unencrypted so that it can be used to run a setup script:

http://github.com/opscode/cookbooks/blob/master/mysql/recipes/server.rb#L98

I think that the real problem was that I hadn't grasped that the node configuration file needs to be treated differently to the chef-solo configuration file and the cookbooks, and kept out of the repository.

There seems to be a similar issue with SSL - ideally the files need to be provided to Chef Solo as it runs, but not kept in repositories, so some kind of secure store definitely looks useful.

On Sat, Jun 5, 2010 at 12:59 PM, Pierre Jacomet pierrj@hotmail.com wrote:

I like Chef solo a lot. :-). wrt. credentials, what I have done is to use a
KVP store which exposes a web service which is script friendly, thus no
complicated XML answers. This way the scripts get the password only when
they need it and you never need to persist the info into your SCM. If you
prefer a pure WS, then you'll need to create a client for it.

There are a couple of issues to deal with thus:

  1. How do you secure access to the KVP store? -- For this you can use things
    like firewall rules or keys that are pre-injected into you images to
    authenticate clients against the KVP store.

  2. You're adding one more service, which means you'll need to manage it. In
    order to minimize this, make your KVP store use an already existing store?
    IOW, instead of XML or a WS, say you already have LDAP, maybe you just need
    a client program that queries LDAP for your credentials and you need to
    populate properly your LDAP hierarchy.

From: stuart@stuartellis.eu
Date: Sat, 5 Jun 2010 10:22:12 +0100
To: chef@lists.opscode.com
Subject: [chef] Handling Passwords with Chef Solo

From: Stuart Ellis stuart@stuartellis.eu
Date: 5 June 2010 10:13:00 GMT+01:00
To: chef@lists.opscode.com
Subject: [chef] Handling Passwords with Chef Solo
Reply-To: chef@lists.opscode.com

Hello,

I've just started to use Chef Solo and have been really pleased with it,
but am curious as to how people are currently managing passwords with it.
This hasn't been an issue for most things, but I've now built up a fairly
complete cookbook for a Rails server, and can see that I need to manage the
MySQL root password and the password for a system account in order to fully
automate the setup.

I'm sorry - I ought to have said "handling passwords securely". The
facilities for setting system account and MySQL root passwords work fine,
but I've so far used them by embedded these passwords into the JSON and test
cookbook, which feels like the wrong approach.


Stuart Ellis
stuart@stuartellis.eu


Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox.
Learn more.


Stuart Ellis
stuart@stuartellis.eu

The New Busy is not the old busy. Search, chat and e-mail from your inbox. Get started.


Stuart Ellis
stuart@stuartellis.eu