Deploying User Private Keys

Chef Infra (archive)
1

I’m currently using the opscode-cookbooks/users cookbook to deploy system users.

The users_manage resource uses data bags to deploy users, and this does allow you to set an ‘ssh_private_key’ attribute, however, I don’t want to store our private keys as plain text in our repository.

My plan is to use ChefVault to deploy users private keys (encrypted data bags and a template/file resource), after the users have been created with the users_manage resource.

Does anyone have suggestions for a better method, or best practice for this case?

Thanks,
Curtis

2

On Mon, 3 Feb 2014 21:06:54 +0000 Curtis Stewart
cstewart@momentumsi.com wrote:

I’m currently using the opscode-cookbooks/users cookbook to deploy
system users.

The users_manage resource uses data bags to deploy users, and this
does allow you to set an ‘ssh_private_key’ attribute, however, I
don’t want to store our private keys as plain text in our repository.

My plan is to use ChefVault to deploy users private keys (encrypted
data bags and a template/file resource), after the users have been
created with the users_manage resource.

Does anyone have suggestions for a better method, or best practice
for this case?

Why should your users hand their private keys to you as the admin to
feed it into some automated system???

The users (the fleshy ones) will give you their public keys to include
on machines so they don't need passwords when ssh-ing.

The private keys you store in chef is for automated things like
deployments getting source-code from a ssh-accessible git. Or for
jenkins-server logging into machines (but even that I did do with a
jenkins-cookbook creating the public/private keys and storing just the
public keys in chef).

  • Arnold
3

Thanks for the response, Arnold, I agree with your comments below about the ‘fleshy’ users.

In regard to the other users, like a deploy user with access to a private git repo, do you store those keys in your repository as plain text?

Curtis

On Feb 3, 2014, at 3:32 PM, Arnold Krille <arnold@arnoldarts.demailto:arnold@arnoldarts.de> wrote:

On Mon, 3 Feb 2014 21:06:54 +0000 Curtis Stewart
<cstewart@momentumsi.commailto:cstewart@momentumsi.com> wrote:
I’m currently using the opscode-cookbooks/users cookbook to deploy
system users.

The users_manage resource uses data bags to deploy users, and this
does allow you to set an ‘ssh_private_key’ attribute, however, I
don’t want to store our private keys as plain text in our repository.

My plan is to use ChefVault to deploy users private keys (encrypted
data bags and a template/file resource), after the users have been
created with the users_manage resource.

Does anyone have suggestions for a better method, or best practice
for this case?

Why should your users hand their private keys to you as the admin to
feed it into some automated system???

The users (the fleshy ones) will give you their public keys to include
on machines so they don’t need passwords when ssh-ing.

The private keys you store in chef is for automated things like
deployments getting source-code from a ssh-accessible git. Or for
jenkins-server logging into machines (but even that I did do with a
jenkins-cookbook creating the public/private keys and storing just the
public keys in chef).

  • Arnold
4

Forgive the shameless plug, but if you’re deploying via Github, Bitbucket or Gitlab, there’s the deploy_key cookbook [0][1] that can help you manage those.

The cookbook will automate the creation of an SSH key and registering it in the repository as a deploy key (read-only). The LWRP also supports unregistering the key from the repo and deleting from disk if you feel like cleaning up.

The idea is to somehow enforce the creating of local private keys per application that never leave the machine where they will be used, reducing the chance of MITM and other attacks. If you feel like any of your keys have been compromised, it’s easy enough to get rid of all of them and create new ones as needed.

The Gitlab provider was recently contributed by dwradcliffe and pull requests for other providers are welcome.

[0] http://community.opscode.com/cookbooks/deploy_key
[1] GitHub - c10l/cookbook-deploy_key: Idempotent Chef LWRP for creating, adding, removing and deleting SSH Deploy Keys on Bitbucket and Github

--
Cassiano Leal
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

On Monday, February 3, 2014 at 21:59, Curtis Stewart wrote:

Thanks for the response, Arnold, I agree with your comments below about the ‘fleshy’ users.

In regard to the other users, like a deploy user with access to a private git repo, do you store those keys in your repository as plain text?

Curtis

On Feb 3, 2014, at 3:32 PM, Arnold Krille <arnold@arnoldarts.demailto:arnold@arnoldarts.de> wrote:

On Mon, 3 Feb 2014 21:06:54 +0000 Curtis Stewart
<cstewart@momentumsi.commailto:cstewart@momentumsi.com> wrote:
I’m currently using the opscode-cookbooks/users cookbook to deploy
system users.

The users_manage resource uses data bags to deploy users, and this
does allow you to set an ‘ssh_private_key’ attribute, however, I
don’t want to store our private keys as plain text in our repository.

My plan is to use ChefVault to deploy users private keys (encrypted
data bags and a template/file resource), after the users have been
created with the users_manage resource.

Does anyone have suggestions for a better method, or best practice
for this case?

Why should your users hand their private keys to you as the admin to
feed it into some automated system???

The users (the fleshy ones) will give you their public keys to include
on machines so they don't need passwords when ssh-ing.

The private keys you store in chef is for automated things like
deployments getting source-code from a ssh-accessible git. Or for
jenkins-server logging into machines (but even that I did do with a
jenkins-cookbook creating the public/private keys and storing just the
public keys in chef).

  • Arnold
5

yeah, be real careful about deploying private keys widely and creating
'full mesh' of passwordless ssh logins on a user. this problem is
actually much worse on the non-fleshy users where you might not have any
password encryption on the key.

a company that i worked at previously had a set of perl scripts which
did log collection and was engineered so that the individual hosts
needed to be able to ssh into the centralized log collection
infrastructure (bad enough) and then the centralized servers needed to
be able to ssh back out (making the problem worse), and this was
implemented across the same user id with the same keys creating a
complete full mesh (horrible). as a result of that, having access to
any server and the ability to change to the log scraping user meant
having ssh access to all 30,000 servers in the company. and a few
members of the developer community had figured this out and were using
it to be able to access servers belonging to other dev teams in order to
debug across systems (arguably they should have just been granted
access, but they were technically using it to circumvent access controls...)

anyway, for github deploy users you do tend to wind up with the deploy
private key blasted out to a lot of your hosts, but you should at least
make sure that the user is read-only.

if you're using private enterprise chef or the open source server that
you host yourself you should consider about the amount of trouble that
you want to go to in order to protect the key. similarly, if most of
your servers are going to have access to the key, the benefit of
chef-vault becomes much lower. going through all the trouble to setup
chef-vault and then blowing the key out to every single server you have
does not accomplish very much. similarly, trying to protect that key
against your chef server getting compromised doesn't accomplish too much
when it can be used to push arbitrary code to your entire environment as
root already.

an encrypted databag is most useful when you're on hosted enterprise
chef. it can also, though, protect the key against leaking out of
developer checkouts of your git repo on laptops, so may be useful anyway
even on PEC and OSS (although if you just saved the key into a databag
that you didn't store in git that would mitigate a big chunk of that
problem as well). chef-vault is going to be most useful the smaller the
subset of your environment you are pushing the key to, i'm not sure i'd
use that for a github deploy key unless i had some kind of regulatory
necessity to do so.

On 2/3/14 1:59 PM, Curtis Stewart wrote:

Thanks for the response, Arnold, I agree with your comments below about the ‘fleshy’ users.

In regard to the other users, like a deploy user with access to a private git repo, do you store those keys in your repository as plain text?

Curtis

On Feb 3, 2014, at 3:32 PM, Arnold Krille <arnold@arnoldarts.demailto:arnold@arnoldarts.de> wrote:

On Mon, 3 Feb 2014 21:06:54 +0000 Curtis Stewart
<cstewart@momentumsi.commailto:cstewart@momentumsi.com> wrote:
I’m currently using the opscode-cookbooks/users cookbook to deploy
system users.

The users_manage resource uses data bags to deploy users, and this
does allow you to set an ‘ssh_private_key’ attribute, however, I
don’t want to store our private keys as plain text in our repository.

My plan is to use ChefVault to deploy users private keys (encrypted
data bags and a template/file resource), after the users have been
created with the users_manage resource.

Does anyone have suggestions for a better method, or best practice
for this case?

Why should your users hand their private keys to you as the admin to
feed it into some automated system???

The users (the fleshy ones) will give you their public keys to include
on machines so they don't need passwords when ssh-ing.

The private keys you store in chef is for automated things like
deployments getting source-code from a ssh-accessible git. Or for
jenkins-server logging into machines (but even that I did do with a
jenkins-cookbook creating the public/private keys and storing just the
public keys in chef).

  • Arnold
6

Hi Cassiano,

Nice plug! I'm just checking out the cookbook and my first impressions are
that this is pretty cool.

I have one question, as the recipe is idempotent it's meant to run on every
deploy right?

So at the start of every chef run it would check the git provider and add
the configured ssh key or do nothing .. correct?

Thanks

On Tue, Feb 4, 2014 at 6:24 AM, Cassiano Leal cassianoleal@gmail.comwrote:

Forgive the shameless plug, but if you're deploying via Github, Bitbucket
or Gitlab, there's the deploy_key cookbook [0][1] that can help you manage
those.

The cookbook will automate the creation of an SSH key and registering it
in the repository as a deploy key (read-only). The LWRP also supports
unregistering the key from the repo and deleting from disk if you feel like
cleaning up.

The idea is to somehow enforce the creating of local private keys per
application that never leave the machine where they will be used, reducing
the chance of MITM and other attacks. If you feel like any of your keys
have been compromised, it's easy enough to get rid of all of them and
create new ones as needed.

The Gitlab provider was recently contributed by dwradcliffe and pull
requests for other providers are welcome.

[0] http://community.opscode.com/cookbooks/deploy_key
[1] GitHub - c10l/cookbook-deploy_key: Idempotent Chef LWRP for creating, adding, removing and deleting SSH Deploy Keys on Bitbucket and Github

--
Cassiano Leal
Sent with Sparrow http://www.sparrowmailapp.com/?sig

On Monday, February 3, 2014 at 21:59, Curtis Stewart wrote:

Thanks for the response, Arnold, I agree with your comments below about
the 'fleshy' users.

In regard to the other users, like a deploy user with access to a private
git repo, do you store those keys in your repository as plain text?

Curtis

On Feb 3, 2014, at 3:32 PM, Arnold Krille <arnold@arnoldarts.de<
mailto:arnold@arnoldarts.de arnold@arnoldarts.de>> wrote:

On Mon, 3 Feb 2014 21:06:54 +0000 Curtis Stewart
<cstewart@momentumsi.com<mailto:cstewart@momentumsi.comcstewart@momentumsi.com>>
wrote:
I'm currently using the opscode-cookbooks/users cookbook to deploy
system users.

The users_manage resource uses data bags to deploy users, and this
does allow you to set an 'ssh_private_key' attribute, however, I
don't want to store our private keys as plain text in our repository.

My plan is to use ChefVault to deploy users private keys (encrypted
data bags and a template/file resource), after the users have been
created with the users_manage resource.

Does anyone have suggestions for a better method, or best practice
for this case?

Why should your users hand their private keys to you as the admin to
feed it into some automated system???

The users (the fleshy ones) will give you their public keys to include
on machines so they don't need passwords when ssh-ing.

The private keys you store in chef is for automated things like
deployments getting source-code from a ssh-accessible git. Or for
jenkins-server logging into machines (but even that I did do with a
jenkins-cookbook creating the public/private keys and storing just the
public keys in chef).

  • Arnold
7

I do something similar with the sshkey Ruby gem, and generate them
programattically. Generate the keys and save private key to disk. Save
the public key to disk, and then chef-server so it will be in search.

Example, Generate deploy key for Jenkins with Chef and sshkey gem. · GitHub

Didn't know there was a deploy_key resource, so I might have to give
that a whirl.

Eric G. Wolfe
Senior Linux Administrator,
IT Infrastructure Systems

Marshall University Computing Services
Drinko Library 428-K
One John Marshall Dr.
Huntington, WV 25755
Phone: 304.942.3970
Email: eric.wolfe@marshall.edu

The world has many unintentionally cruel mechanisms that are not
designed for people who walk on their hands.
-- John Irving, "The World According to Garp"

On 02/03/2014 05:24 PM, Cassiano Leal wrote:

Forgive the shameless plug, but if you’re deploying via Github,
Bitbucket or Gitlab, there’s the deploy_key cookbook [0][1] that can
help you manage those.

The cookbook will automate the creation of an SSH key and registering
it in the repository as a deploy key (read-only). The LWRP also
supports unregistering the key from the repo and deleting from disk if
you feel like cleaning up.

The idea is to somehow enforce the creating of local private keys per
application that never leave the machine where they will be used,
reducing the chance of MITM and other attacks. If you feel like any of
your keys have been compromised, it’s easy enough to get rid of all of
them and create new ones as needed.

The Gitlab provider was recently contributed by dwradcliffe and pull
requests for other providers are welcome.

[0] http://community.opscode.com/cookbooks/deploy_key
[1] GitHub - c10l/cookbook-deploy_key: Idempotent Chef LWRP for creating, adding, removing and deleting SSH Deploy Keys on Bitbucket and Github

--
Cassiano Leal
Sent with Sparrow http://www.sparrowmailapp.com/?sig

On Monday, February 3, 2014 at 21:59, Curtis Stewart wrote:

Thanks for the response, Arnold, I agree with your comments below
about the ‘fleshy’ users.

In regard to the other users, like a deploy user with access to a
private git repo, do you store those keys in your repository as plain
text?

Curtis

On Feb 3, 2014, at 3:32 PM, Arnold Krille
<arnold@arnoldarts.demailto:arnold@arnoldarts.de> wrote:

On Mon, 3 Feb 2014 21:06:54 +0000 Curtis Stewart
<cstewart@momentumsi.commailto:cstewart@momentumsi.com> wrote:
I’m currently using the opscode-cookbooks/users cookbook to deploy
system users.

The users_manage resource uses data bags to deploy users, and this
does allow you to set an ‘ssh_private_key’ attribute, however, I
don’t want to store our private keys as plain text in our repository.

My plan is to use ChefVault to deploy users private keys (encrypted
data bags and a template/file resource), after the users have been
created with the users_manage resource.

Does anyone have suggestions for a better method, or best practice
for this case?

Why should your users hand their private keys to you as the admin to
feed it into some automated system???

The users (the fleshy ones) will give you their public keys to include
on machines so they don't need passwords when ssh-ing.

The private keys you store in chef is for automated things like
deployments getting source-code from a ssh-accessible git. Or for
jenkins-server logging into machines (but even that I did do with a
jenkins-cookbook creating the public/private keys and storing just the
public keys in chef).

  • Arnold
8

Thanks!

Yes, that's correct. It first queries the deploy key API to check if the key is already there. If it is, it will skip and no Chef resources will have been updated. If it's not, then it gets added and the resource is updated.

It should be compatible with whyrun, although I regret to say that I haven't tested this feature. :-/

--
Cassiano Leal
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

On Tuesday, February 4, 2014 at 2:12, Rudi wrote:

Hi Cassiano,

Nice plug! I'm just checking out the cookbook and my first impressions are that this is pretty cool.

I have one question, as the recipe is idempotent it's meant to run on every deploy right?

So at the start of every chef run it would check the git provider and add the configured ssh key or do nothing .. correct?

Thanks

On Tue, Feb 4, 2014 at 6:24 AM, Cassiano Leal <cassianoleal@gmail.com (mailto:cassianoleal@gmail.com)> wrote:

Forgive the shameless plug, but if you’re deploying via Github, Bitbucket or Gitlab, there’s the deploy_key cookbook [0][1] that can help you manage those.

The cookbook will automate the creation of an SSH key and registering it in the repository as a deploy key (read-only). The LWRP also supports unregistering the key from the repo and deleting from disk if you feel like cleaning up.

The idea is to somehow enforce the creating of local private keys per application that never leave the machine where they will be used, reducing the chance of MITM and other attacks. If you feel like any of your keys have been compromised, it’s easy enough to get rid of all of them and create new ones as needed.

The Gitlab provider was recently contributed by dwradcliffe and pull requests for other providers are welcome.

[0] http://community.opscode.com/cookbooks/deploy_key
[1] GitHub - c10l/cookbook-deploy_key: Idempotent Chef LWRP for creating, adding, removing and deleting SSH Deploy Keys on Bitbucket and Github

--
Cassiano Leal
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

On Monday, February 3, 2014 at 21:59, Curtis Stewart wrote:

Thanks for the response, Arnold, I agree with your comments below about the ‘fleshy’ users.

In regard to the other users, like a deploy user with access to a private git repo, do you store those keys in your repository as plain text?

Curtis

On Feb 3, 2014, at 3:32 PM, Arnold Krille <arnold@arnoldarts.de (mailto:arnold@arnoldarts.de)mailto:arnold@arnoldarts.de> wrote:

On Mon, 3 Feb 2014 21:06:54 +0000 Curtis Stewart
<cstewart@momentumsi.com (mailto:cstewart@momentumsi.com)mailto:cstewart@momentumsi.com> wrote:
I’m currently using the opscode-cookbooks/users cookbook to deploy
system users.

The users_manage resource uses data bags to deploy users, and this
does allow you to set an ‘ssh_private_key’ attribute, however, I
don’t want to store our private keys as plain text in our repository.

My plan is to use ChefVault to deploy users private keys (encrypted
data bags and a template/file resource), after the users have been
created with the users_manage resource.

Does anyone have suggestions for a better method, or best practice
for this case?

Why should your users hand their private keys to you as the admin to
feed it into some automated system???

The users (the fleshy ones) will give you their public keys to include
on machines so they don't need passwords when ssh-ing.

The private keys you store in chef is for automated things like
deployments getting source-code from a ssh-accessible git. Or for
jenkins-server logging into machines (but even that I did do with a
jenkins-cookbook creating the public/private keys and storing just the
public keys in chef).

  • Arnold
9

Cassiano,

Sweet .. I'm going to check this recipe and give it a whirl over in an open
source AngularJS/MongoDB web stack :

It's a boilerplate MEAN stack.

Cheers.

On Tue, Feb 4, 2014 at 6:11 PM, Cassiano Leal cassianoleal@gmail.comwrote:

Thanks!

Yes, that's correct. It first queries the deploy key API to check if the
key is already there. If it is, it will skip and no Chef resources will
have been updated. If it's not, then it gets added and the resource is
updated.

It should be compatible with whyrun, although I regret to say that I
haven't tested this feature. :-/

--
Cassiano Leal
Sent with Sparrow http://www.sparrowmailapp.com/?sig

On Tuesday, February 4, 2014 at 2:12, Rudi wrote:

Hi Cassiano,

Nice plug! I'm just checking out the cookbook and my first impressions are
that this is pretty cool.

I have one question, as the recipe is idempotent it's meant to run on
every deploy right?

So at the start of every chef run it would check the git provider and add
the configured ssh key or do nothing .. correct?

Thanks

On Tue, Feb 4, 2014 at 6:24 AM, Cassiano Leal cassianoleal@gmail.comwrote:

Forgive the shameless plug, but if you're deploying via Github, Bitbucket
or Gitlab, there's the deploy_key cookbook [0][1] that can help you manage
those.

The cookbook will automate the creation of an SSH key and registering it
in the repository as a deploy key (read-only). The LWRP also supports
unregistering the key from the repo and deleting from disk if you feel like
cleaning up.

The idea is to somehow enforce the creating of local private keys per
application that never leave the machine where they will be used, reducing
the chance of MITM and other attacks. If you feel like any of your keys
have been compromised, it's easy enough to get rid of all of them and
create new ones as needed.

The Gitlab provider was recently contributed by dwradcliffe and pull
requests for other providers are welcome.

[0] http://community.opscode.com/cookbooks/deploy_key
[1] GitHub - c10l/cookbook-deploy_key: Idempotent Chef LWRP for creating, adding, removing and deleting SSH Deploy Keys on Bitbucket and Github

--
Cassiano Leal
Sent with Sparrow http://www.sparrowmailapp.com/?sig

On Monday, February 3, 2014 at 21:59, Curtis Stewart wrote:

Thanks for the response, Arnold, I agree with your comments below about
the 'fleshy' users.

In regard to the other users, like a deploy user with access to a private
git repo, do you store those keys in your repository as plain text?

Curtis

On Feb 3, 2014, at 3:32 PM, Arnold Krille <arnold@arnoldarts.de<
mailto:arnold@arnoldarts.de arnold@arnoldarts.de>> wrote:

On Mon, 3 Feb 2014 21:06:54 +0000 Curtis Stewart
<cstewart@momentumsi.com<mailto:cstewart@momentumsi.comcstewart@momentumsi.com>>
wrote:
I'm currently using the opscode-cookbooks/users cookbook to deploy
system users.

The users_manage resource uses data bags to deploy users, and this
does allow you to set an 'ssh_private_key' attribute, however, I
don't want to store our private keys as plain text in our repository.

My plan is to use ChefVault to deploy users private keys (encrypted
data bags and a template/file resource), after the users have been
created with the users_manage resource.

Does anyone have suggestions for a better method, or best practice
for this case?

Why should your users hand their private keys to you as the admin to
feed it into some automated system???

The users (the fleshy ones) will give you their public keys to include
on machines so they don't need passwords when ssh-ing.

The private keys you store in chef is for automated things like
deployments getting source-code from a ssh-accessible git. Or for
jenkins-server logging into machines (but even that I did do with a
jenkins-cookbook creating the public/private keys and storing just the
public keys in chef).

  • Arnold
10

Hello,

I have a sort of similar question and I think its unecessary to open another thread.

On my taskwarrior cookbook[1] I generate the keys for the users on the server.
I would like to distribute them to the client machines that have the taskwarrior client in it.

What would be a “good” way to do it. I tought on creating an user and scp them, but that would require
and ssh key of itself, that the user would need to set in a data bag.

[1] https://github.com/masterkorp/cookbook-taskwarrior

–
Regards,
Alfredo Palhares

11

On Tue, 04 Feb 2014 17:42:49 +0100 Alfredo Palhares
masterkorp@masterkorp.net wrote:

Hello,

I have a sort of similar question and I think its unecessary to open
another thread.

On my taskwarrior cookbook[1] I generate the keys for the users on
the server. I would like to distribute them to the client machines
that have the taskwarrior client in it.

What would be a "good" way to do it. I tought on creating an user and
scp them, but that would require and ssh key of itself, that the user
would need to set in a data bag.

If its the server contacting the clients: The server should publish its
public-key within chef and the clients will fetch that on their
chef-run and configure it.

If its keys so the clients can contact the server: Create the key-pairs
on the client and publish the public keys in chef so the server can
search for them and add them to the local store.

  • Arnold
12

Hello Arnold,

If its the server contacting the clients: The server should publish its
public-key within chef and the clients will fetch that on their
chef-run and configure it.
Yes but that would need to for me to publicate the CA key(private) too, now only the cert(public key)

If its keys so the clients can contact the server: Create the key-pairs
on the client and publish the public keys in chef so the server can
search for them and add them to the local store.
They're are gnutls (certtool generated) keys.

--
Regards,
Alfredo Palhares

13

On Thu, 06 Feb 2014 15:54:30 +0100 Alfredo Palhares
masterkorp@masterkorp.net wrote:

Hello Arnold,

If its the server contacting the clients: The server should publish
its public-key within chef and the clients will fetch that on their
chef-run and configure it.
Yes but that would need to for me to publicate the CA key(private)
too, now only the cert(public key)

If its keys so the clients can contact the server: Create the
key-pairs on the client and publish the public keys in chef so the
server can search for them and add them to the local store.
They're are gnutls (certtool generated) keys.

So that looks more like a use-case for the x509-cookbook and the
contained knife-ssl: GitHub - VendaTech/chef-cookbook-ssl: Deploy a Chef-managed Certificate Authority

Have the server create its private key and a self-signed public key,
the publish a csr that is signed by knife-ssl on a machine with the CA.
Then the public-cert is pushed to Chef-server where both the server
gets it to replace its own self-signed one and the clients can fetch it
if needed. Or they just trust the public part of the CA.

I might not understand your concret problem (mostly because I have no
clue what taskwarrior is), but rule of thumb:
A private key that leaves the machine where its created is rendered
insecure and not trustworthy anymore.
Only exception might be when its an ssl-certificate for load-balanced
services.

Have fun,

Arnold