Securing Knife

I am puzzled how to secure the use of knife in open source chef. If I add a password to the user PEM I am forced to enter the password multiple times for each knife command. So that’s not a very user friendly option. Someone else suggested storing the pem on an encrypted file system/device/etc. What is the best practice to provide controlled admin access to the chef server? It’s a little unnerving that someone with a copy of any admin PEM file gains complete control over your infrastructure. I feel like I must be missing something.

Thanks,
-Joe

Hi,

a different idea: Some people use jenkins to submit updates from Git to
the Chef Server.

This would also help to mitigate this problem, as admins don't work as
Chef admin clients anymore (although I'm still not sure, how useful this
approach is). But you would delegate the authentication problem to Git.

Yours
Steffen

On 06/11/13 23:40, Kemp, Joseph A. (JKEMP) wrote:

I am puzzled how to secure the use of knife in open source chef. If I add a password to the user PEM I am forced to enter the password multiple times for each knife command. So that's not a very user friendly option. Someone else suggested storing the pem on an encrypted file system/device/etc. What is the best practice to provide controlled admin access to the chef server? It's a little unnerving that someone with a copy of any admin PEM file gains complete control over your infrastructure. I feel like I must be missing something.

Thanks,
-Joe

Have individual/personal admin-level pem files - don't share a centralized
one.

knife client create new_person --admin

Ref: http://docs.opscode.com/chef/knife.html#create

-M

On Wed, Nov 6, 2013 at 5:40 PM, Kemp, Joseph A. (JKEMP) JKEMP@arinc.comwrote:

I am puzzled how to secure the use of knife in open source chef. If I
add a password to the user PEM I am forced to enter the password multiple
times for each knife command. So that’s not a very user friendly option.
Someone else suggested storing the pem on an encrypted file
system/device/etc. What is the best practice to provide controlled admin
access to the chef server? It’s a little unnerving that someone with a
copy of any admin PEM file gains complete control over your
infrastructure. I feel like I must be missing something.

Thanks,

-Joe

My concern is with the pem file. Right now it is basically like storing my password in the clear in a text file. I need to either add a password to the private key or I need the chef-server to require the user password to be provided before it allows access to the chef-server. Why does the chef server allow a user to perform operations against the server without the user’s password?

-----Original Message-----
From: Mike [mailto:miketheman@gmail.com]
Sent: Wednesday, November 06, 2013 5:45 PM
To: chef@lists.opscode.com
Subject: [chef] Re: Securing Knife

Have individual/personal admin-level pem files - don’t share a centralized one.

knife client create new_person --admin

Ref: http://docs.opscode.com/chef/knife.html#create

-M

On Wed, Nov 6, 2013 at 5:40 PM, Kemp, Joseph A. (JKEMP) JKEMP@arinc.com wrote:

I am puzzled how to secure the use of knife in open source chef.  If I add a password to the user PEM I am forced to enter the password multiple times for each knife command.  So that's not a very user friendly option.  Someone else suggested storing the pem on an encrypted file system/device/etc.  What is the best practice to provide controlled admin access to the chef server?  It's a little unnerving that someone with a copy of any admin PEM file gains complete control over your infrastructure.  I feel like I must be missing something.

 

Thanks,

-Joe

On Nov 6, 2013, at 4:49 PM, "Kemp, Joseph A. (JKEMP)" JKEMP@arinc.com wrote:

My concern is with the pem file. Right now it is basically like storing my password in the clear in a text file. I need to either add a password to the private key or I need the chef-server to require the user password to be provided before it allows access to the chef-server. Why does the chef server allow a user to perform operations against the server without the user's password?

With ssh, there is ssh-agent that can be used to temporarily cache the password in memory, once it has been entered from the keyboard. Through sshd_config, you can control how long they are cached before being purged.

If you're using "knife ssh ...", then I would expect that the ssh-agent would work the same for you as if you were typing a naked "ssh" command.

However, I don't know if there is an equivalent to ssh-agent for all the PEM-related things that might need to be done that do not actually use ssh.

--
Brad Knowles brad@shub-internet.org
LinkedIn Profile: http://tinyurl.com/y8kpxu

Such a private pem file is still stored locally, effectively in plain-text, with no password protection. For home directories on poorly secured NFS mounts it’s even worse because any host connected to the relevant network can NFS mount the directory, “sudo” to the relevant uid, and gain access to the unencrypted keys. NFSv4 with Kerberized authentication can help with that, as can proper CIFS configurations for Windows based fileshares, but the key is still available on all backup media in plaintext.

I’d recommend using a highly secured local disk area, such as an encrypted partition, and a symlink from the relevant workspace to the locally encrypted partition. And I’d suggest running chef server operations only from that secured workspace, especially for sensitive environments and source code manipulation. Since the source code for the cookbooks can often be used to manipulate or ruin deployed systems, similar precautions should be used for SSH keys for any central source repository.

And as mentioned, don’t forget to passphrase protect SSH keys? The old “keychain” perl script works well for managing personal SSH keys in command-line environments, and many modern window manager environments like Gnome and KDE have built-in tools for SSH key management.


From: Mike [miketheman@gmail.com]
Sent: Wednesday, November 06, 2013 5:45 PM
To: chef@lists.opscode.com
Subject: [chef] Re: Securing Knife

Have individual/personal admin-level pem files - don’t share a centralized one.

knife client create new_person --admin

Ref: http://docs.opscode.com/chef/knife.html#create

-M

On Wed, Nov 6, 2013 at 5:40 PM, Kemp, Joseph A. (JKEMP) <JKEMP@arinc.commailto:JKEMP@arinc.com> wrote:
I am puzzled how to secure the use of knife in open source chef. If I add a password to the user PEM I am forced to enter the password multiple times for each knife command. So that’s not a very user friendly option. Someone else suggested storing the pem on an encrypted file system/device/etc. What is the best practice to provide controlled admin access to the chef server? It’s a little unnerving that someone with a copy of any admin PEM file gains complete control over your infrastructure. I feel like I must be missing something.

Thanks,
-Joe

Joe,

It's the first time I've heard this raised as a concern, but that
doesn't mean it's not valid. I think the use case so far has been that
each Chef admin has the PEM on their local workstations as opposed to
a shared workstation/jumpbox.

However, one has to balance usability versus security. Even if Knife
only prompted once per command for the user's passphrase, that still
seems like a PITA. Doesn't that get in the way of operations?

Again, I think it's a feature request that we would consider if you
can define how you see PEM passphrases would work without being too
intrusive.

  • Julian

On Thu, Nov 7, 2013 at 7:33 AM, Kadel-Garcia, Nico
NKadelGarcia-consultant@scholastic.com wrote:

Such a private pem file is still stored locally, effectively in plain-text,
with no password protection. For home directories on poorly secured NFS
mounts it's even worse because any host connected to the relevant network
can NFS mount the directory, "sudo" to the relevant uid, and gain access to
the unencrypted keys. NFSv4 with Kerberized authentication can help with
that, as can proper CIFS configurations for Windows based fileshares, but
the key is still available on all backup media in plaintext.

I'd recommend using a highly secured local disk area, such as an encrypted
partition, and a symlink from the relevant workspace to the locally
encrypted partition. And I'd suggest running chef server operations only
from that secured workspace, especially for sensitive environments and
source code manipulation. Since the source code for the cookbooks can often
be used to manipulate or ruin deployed systems, similar precautions should
be used for SSH keys for any central source repository.

And as mentioned, don't forget to passphrase protect SSH keys? The old
"keychain" perl script works well for managing personal SSH keys in
command-line environments, and many modern window manager environments like
Gnome and KDE have built-in tools for SSH key management.


From: Mike [miketheman@gmail.com]

Sent: Wednesday, November 06, 2013 5:45 PM
To: chef@lists.opscode.com
Subject: [chef] Re: Securing Knife

Have individual/personal admin-level pem files - don't share a centralized
one.

knife client create new_person --admin

Ref: http://docs.opscode.com/chef/knife.html#create

-M

On Wed, Nov 6, 2013 at 5:40 PM, Kemp, Joseph A. (JKEMP) JKEMP@arinc.com
wrote:

I am puzzled how to secure the use of knife in open source chef. If I add
a password to the user PEM I am forced to enter the password multiple times
for each knife command. So that’s not a very user friendly option. Someone
else suggested storing the pem on an encrypted file system/device/etc. What
is the best practice to provide controlled admin access to the chef server?
It’s a little unnerving that someone with a copy of any admin PEM file gains
complete control over your infrastructure. I feel like I must be missing
something.

Thanks,

-Joe

--
[ Julian C. Dunn jdunn@aquezada.com * Sorry, I'm ]
[ WWW: Julian Dunn's Blog - Commentary on media, technology, and everything in between. * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]

Julian,

My understanding of the Chef infrastructure is that if someone gets a copy of the physical pem file and has access to your chef server they have the keys to your kingdom. This would be complete control over every managed device. It's just like keeping all of the root passwords in a text file on your laptop drive. I know I can use disk encryption to protect the pem but that seems like a pretty significant requirement to be able to use chef securely.

There are a handful of spots that load the key with a call like this: OpenSSL::PKey::RSA.new(rest.signing_key). Right now the code relies on the underlying openssl library to ask for the password but it can be easily passed in as a second parameter to this existing call. So in a sense knife already supports encrypted pems but it just isn't implemented very well.

  1. So the first option would be to allow the password to be passed on the command line to knife. Better would be to get the password from stdin. There are some significant risks to allowing the password on the command line but it is better than nothing.

  2. The second option is for knife to check to see if the pem is encrypted, if so ask for the password once and then pass it into all subsequent openssl calls for the user.

  3. Finally, ssh-agent has addressed this same type of issue for many years. It's a good balance between security and usability. Figuring out how to build on that model would most likely be the most elegant solution.

  4. Just thought of another angle. All of this "security" is built on top of openssl. Looking at the man page (/docs/manmaster/man1/openssl.html) it lists 5 different techniques to get the password into the openssl process. These look to be good options also.

-Joe

-----Original Message-----
From: Julian C. Dunn [mailto:jdunn@aquezada.com]
Sent: Thursday, November 07, 2013 6:40 PM
To: chef@lists.opscode.com
Subject: [chef] Re: RE: Re: Securing Knife

Joe,

It's the first time I've heard this raised as a concern, but that doesn't mean it's not valid. I think the use case so far has been that each Chef admin has the PEM on their local workstations as opposed to a shared workstation/jumpbox.

However, one has to balance usability versus security. Even if Knife only prompted once per command for the user's passphrase, that still seems like a PITA. Doesn't that get in the way of operations?

Again, I think it's a feature request that we would consider if you can define how you see PEM passphrases would work without being too intrusive.

  • Julian

On Thu, Nov 7, 2013 at 7:33 AM, Kadel-Garcia, Nico NKadelGarcia-consultant@scholastic.com wrote:

Such a private pem file is still stored locally, effectively in
plain-text, with no password protection. For home directories on
poorly secured NFS mounts it's even worse because any host connected
to the relevant network can NFS mount the directory, "sudo" to the
relevant uid, and gain access to the unencrypted keys. NFSv4 with
Kerberized authentication can help with that, as can proper CIFS
configurations for Windows based fileshares, but the key is still available on all backup media in plaintext.

I'd recommend using a highly secured local disk area, such as an
encrypted partition, and a symlink from the relevant workspace to the
locally encrypted partition. And I'd suggest running chef server
operations only from that secured workspace, especially for sensitive
environments and source code manipulation. Since the source code for
the cookbooks can often be used to manipulate or ruin deployed
systems, similar precautions should be used for SSH keys for any central source repository.

And as mentioned, don't forget to passphrase protect SSH keys? The old
"keychain" perl script works well for managing personal SSH keys in
command-line environments, and many modern window manager environments
like Gnome and KDE have built-in tools for SSH key management.


From: Mike [miketheman@gmail.com]

Sent: Wednesday, November 06, 2013 5:45 PM
To: chef@lists.opscode.com
Subject: [chef] Re: Securing Knife

Have individual/personal admin-level pem files - don't share a
centralized one.

knife client create new_person --admin

Ref: http://docs.opscode.com/chef/knife.html#create

-M

On Wed, Nov 6, 2013 at 5:40 PM, Kemp, Joseph A. (JKEMP)
JKEMP@arinc.com
wrote:

I am puzzled how to secure the use of knife in open source chef. If
I add a password to the user PEM I am forced to enter the password
multiple times for each knife command. So that's not a very user
friendly option. Someone else suggested storing the pem on an
encrypted file system/device/etc. What is the best practice to provide controlled admin access to the chef server?
It's a little unnerving that someone with a copy of any admin PEM
file gains complete control over your infrastructure. I feel like I
must be missing something.

Thanks,

-Joe

--
[ Julian C. Dunn jdunn@aquezada.com * Sorry, I'm ]
[ WWW: Julian Dunn's Blog - Commentary on media, technology, and everything in between. * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]

i like the idea. i dont think it will be lot of work to implement this.
though i find the whole password/ssh-agent/gnome bit strange (keyloggers?
each tool will add 1+ vector), but this will help in general 2 factor auth/
ldap backed etc.

On Thu, Nov 7, 2013 at 5:29 PM, Kemp, Joseph A. (JKEMP) JKEMP@arinc.comwrote:

Julian,

My understanding of the Chef infrastructure is that if someone gets a copy
of the physical pem file and has access to your chef server they have the
keys to your kingdom. This would be complete control over every managed
device. It's just like keeping all of the root passwords in a text file on
your laptop drive. I know I can use disk encryption to protect the pem but
that seems like a pretty significant requirement to be able to use chef
securely.

There are a handful of spots that load the key with a call like this:
OpenSSL::PKey::RSA.new(rest.signing_key). Right now the code relies on the
underlying openssl library to ask for the password but it can be easily
passed in as a second parameter to this existing call. So in a sense knife
already supports encrypted pems but it just isn't implemented very well.

  1. So the first option would be to allow the password to be passed on the
    command line to knife. Better would be to get the password from stdin.
    There are some significant risks to allowing the password on the command
    line but it is better than nothing.

  2. The second option is for knife to check to see if the pem is
    encrypted, if so ask for the password once and then pass it into all
    subsequent openssl calls for the user.

  3. Finally, ssh-agent has addressed this same type of issue for many
    years. It's a good balance between security and usability. Figuring out
    how to build on that model would most likely be the most elegant solution.

  4. Just thought of another angle. All of this "security" is built on top
    of openssl. Looking at the man page (
    /docs/manmaster/man1/openssl.html) it
    lists 5 different techniques to get the password into the openssl process.
    These look to be good options also.

-Joe

-----Original Message-----
From: Julian C. Dunn [mailto:jdunn@aquezada.com]
Sent: Thursday, November 07, 2013 6:40 PM
To: chef@lists.opscode.com
Subject: [chef] Re: RE: Re: Securing Knife

Joe,

It's the first time I've heard this raised as a concern, but that doesn't
mean it's not valid. I think the use case so far has been that each Chef
admin has the PEM on their local workstations as opposed to a shared
workstation/jumpbox.

However, one has to balance usability versus security. Even if Knife only
prompted once per command for the user's passphrase, that still seems like
a PITA. Doesn't that get in the way of operations?

Again, I think it's a feature request that we would consider if you can
define how you see PEM passphrases would work without being too intrusive.

  • Julian

On Thu, Nov 7, 2013 at 7:33 AM, Kadel-Garcia, Nico <
NKadelGarcia-consultant@scholastic.com> wrote:

Such a private pem file is still stored locally, effectively in
plain-text, with no password protection. For home directories on
poorly secured NFS mounts it's even worse because any host connected
to the relevant network can NFS mount the directory, "sudo" to the
relevant uid, and gain access to the unencrypted keys. NFSv4 with
Kerberized authentication can help with that, as can proper CIFS
configurations for Windows based fileshares, but the key is still
available on all backup media in plaintext.

I'd recommend using a highly secured local disk area, such as an
encrypted partition, and a symlink from the relevant workspace to the
locally encrypted partition. And I'd suggest running chef server
operations only from that secured workspace, especially for sensitive
environments and source code manipulation. Since the source code for
the cookbooks can often be used to manipulate or ruin deployed
systems, similar precautions should be used for SSH keys for any central
source repository.

And as mentioned, don't forget to passphrase protect SSH keys? The old
"keychain" perl script works well for managing personal SSH keys in
command-line environments, and many modern window manager environments
like Gnome and KDE have built-in tools for SSH key management.


From: Mike [miketheman@gmail.com]

Sent: Wednesday, November 06, 2013 5:45 PM
To: chef@lists.opscode.com
Subject: [chef] Re: Securing Knife

Have individual/personal admin-level pem files - don't share a
centralized one.

knife client create new_person --admin

Ref: http://docs.opscode.com/chef/knife.html#create

-M

On Wed, Nov 6, 2013 at 5:40 PM, Kemp, Joseph A. (JKEMP)
JKEMP@arinc.com
wrote:

I am puzzled how to secure the use of knife in open source chef. If
I add a password to the user PEM I am forced to enter the password
multiple times for each knife command. So that's not a very user
friendly option. Someone else suggested storing the pem on an
encrypted file system/device/etc. What is the best practice to provide
controlled admin access to the chef server?
It's a little unnerving that someone with a copy of any admin PEM
file gains complete control over your infrastructure. I feel like I
must be missing something.

Thanks,

-Joe

--
[ Julian C. Dunn jdunn@aquezada.com * Sorry, I'm ]
[ WWW: Julian Dunn's Blog - Commentary on media, technology, and everything in between. * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]

Hi,

There certainly seem to be other users who would like something along
these lines.

Joe, you might want to take a look at CHEF-2381 (link below). Attached
to it are some patches that allow knife to use ssh-agent.

https://tickets.opscode.com/browse/CHEF-2381 

Changes were made to the authN code in Chef Server to work with this,
but the client side changes got bogged down trying to find the right
form to add the feature and not break back compat.

If this approach would meet your needs, it would be great to revive the
patches and see what we can do.

  • seth


Seth Falcon | Development Lead | Opscode | @sfalcon

This looks like what I want. I am trying filter through all of the changes. Did the backend work get included in 11.0.8 of the server? I am going to try to patch in the client side changes and see if I can get it to work. How can we get this effort moving again?

-Joe

-----Original Message-----
From: Seth Falcon [mailto:seth@opscode.com]
Sent: Friday, November 08, 2013 1:03 AM
To: chef@lists.opscode.com
Subject: [chef] Re: RE: Re: RE: Re: Securing Knife

Hi,

There certainly seem to be other users who would like something along these lines.

Joe, you might want to take a look at CHEF-2381 (link below). Attached to it are some patches that allow knife to use ssh-agent.

https://tickets.opscode.com/browse/CHEF-2381 

Changes were made to the authN code in Chef Server to work with this, but the client side changes got bogged down trying to find the right form to add the feature and not break back compat.

If this approach would meet your needs, it would be great to revive the patches and see what we can do.

  • seth


Seth Falcon | Development Lead | Opscode | @sfalcon

If you're worried about keyloggers or remote control tools on admin's
workstations, then you've lost the war already.

There is a clear risk vector in stolen laptops and in drive-by hacks of
laptops snarfing unencrypted credentials.

Making knife encrypt the existing user.pem file would be fairly easy.

Making knife, and the chef-server, use ssh identities and integrate with
ssh-agent would be very cool, but obviously more work. Since Dan is
doing work that will eliminate the need for validation keys and leverage
the user creds for provisioning servers, if we could pick up existing
ssh keys then that would make chef a lot easier to use -- the signup
process becomes "paste in your public ssh key" which people should be
trained to do from interacting with AWS and other cloud services. That
key starts to be a lot of eggs in one basket, but for admins with root
access, compromise of their ssh credentials is usually enough to own the
entire shop anyway -- admin laptops should have encrypted drives and use
ssh-agent at that point.

On 11/7/13 9:43 PM, Ranjib Dey wrote:

i like the idea. i dont think it will be lot of work to implement
this. though i find the whole password/ssh-agent/gnome bit strange
(keyloggers? each tool will add 1+ vector), but this will help in
general 2 factor auth/ ldap backed etc.

On Thu, Nov 7, 2013 at 5:29 PM, Kemp, Joseph A. (JKEMP)
<JKEMP@arinc.com mailto:JKEMP@arinc.com> wrote:

Julian,

My understanding of the Chef infrastructure is that if someone
gets a copy of the physical pem file and has access to your chef
server they have the keys to your kingdom.  This would be complete
control over every managed device.  It's just like keeping all of
the root passwords in a text file on your laptop drive.  I know I
can use disk encryption to protect the pem but that seems like a
pretty significant requirement to be able to use chef securely.

There are a handful of spots that load the key with a call like
this: OpenSSL::PKey::RSA.new(rest.signing_key).  Right now the
code relies on the underlying openssl library to ask for the
password but it can be easily passed in as a second parameter to
this existing call.  So in a sense knife already supports
encrypted pems but it just isn't implemented very well.

1.  So the first option would be to allow the password to be
passed on the command line to knife.  Better would be to get the
password from stdin.  There are some significant risks to allowing
the password on the command line but it is better than nothing.

2.  The second option is for knife to check to see if the pem is
encrypted, if so ask for the password once and then pass it into
all subsequent openssl calls for the user.

3. Finally, ssh-agent has addressed this same type of issue for
many years.  It's a good balance between security and usability.
Figuring out how to build on that model would most likely be the
most elegant solution.

4. Just thought of another angle.  All of this "security" is built
on top of openssl.  Looking at the man page
(http://www.openssl.org/docs/apps/openssl.html#PASS_PHRASE_ARGUMENTS)
it lists 5 different techniques to get the password into the
openssl process.  These look to be good options also.


-Joe


-----Original Message-----
From: Julian C. Dunn [mailto:jdunn@aquezada.com
<mailto:jdunn@aquezada.com>]
Sent: Thursday, November 07, 2013 6:40 PM
To: chef@lists.opscode.com <mailto:chef@lists.opscode.com>
Subject: [chef] Re: RE: Re: Securing Knife

Joe,

It's the first time I've heard this raised as a concern, but that
doesn't mean it's not valid. I think the use case so far has been
that each Chef admin has the PEM on their local workstations as
opposed to a shared workstation/jumpbox.

However, one has to balance usability versus security. Even if
Knife only prompted once per command for the user's passphrase,
that still seems like a PITA. Doesn't that get in the way of
operations?

Again, I think it's a feature request that we would consider if
you can define how you see PEM passphrases would work without
being too intrusive.

- Julian

On Thu, Nov 7, 2013 at 7:33 AM, Kadel-Garcia, Nico
<NKadelGarcia-consultant@scholastic.com
<mailto:NKadelGarcia-consultant@scholastic.com>> wrote:
> Such a private pem file is still stored locally, effectively in
> plain-text, with no password protection. For home directories on
> poorly secured NFS mounts it's even worse because any host connected
> to the relevant network can NFS mount the directory, "sudo" to the
> relevant uid, and gain access to the unencrypted keys. NFSv4 with
> Kerberized authentication can help with that, as can proper CIFS
> configurations for Windows based fileshares, but the key is
still available on all backup media in plaintext.
>
> I'd recommend using a highly secured local disk area, such as an
> encrypted partition, and a symlink from the relevant workspace
to the
> locally encrypted partition. And I'd suggest running chef server
> operations only from that secured workspace, especially for
sensitive
> environments and source code manipulation. Since the source code for
> the cookbooks can often be used to manipulate or ruin deployed
> systems, similar precautions should be used for SSH keys for any
central source repository.
>
> And as mentioned, don't forget to passphrase protect SSH keys?
The old
> "keychain" perl script works well for managing personal SSH keys in
> command-line environments, and many modern window manager
environments
> like Gnome and KDE have built-in tools for SSH key management.
>
> ________________________________
> From: Mike [miketheman@gmail.com <mailto:miketheman@gmail.com>]
>
> Sent: Wednesday, November 06, 2013 5:45 PM
> To: chef@lists.opscode.com <mailto:chef@lists.opscode.com>
> Subject: [chef] Re: Securing Knife
>
> Have individual/personal admin-level pem files - don't share a
> centralized one.
>
>   knife client create new_person --admin
>
> Ref: http://docs.opscode.com/chef/knife.html#create
>
> -M
>
>
> On Wed, Nov 6, 2013 at 5:40 PM, Kemp, Joseph A. (JKEMP)
> <JKEMP@arinc.com <mailto:JKEMP@arinc.com>>
> wrote:
>>
>> I am puzzled how to secure the use of knife in open source
chef.  If
>> I add a password to the user PEM I am forced to enter the password
>> multiple times for each knife command.  So that's not a very user
>> friendly option.  Someone else suggested storing the pem on an
>> encrypted file system/device/etc.  What is the best practice to
provide controlled admin access to the chef server?
>> It's a little unnerving that someone with a copy of any admin PEM
>> file gains complete control over your infrastructure.  I feel
like I
>> must be missing something.
>>
>>
>>
>> Thanks,
>>
>> -Joe
>
>



--
[ Julian C. Dunn <jdunn@aquezada.com <mailto:jdunn@aquezada.com>>
         * Sorry, I'm    ]
[ WWW: http://www.aquezada.com/staff/julian    * only Web 1.0  ]
[ gopher://sdf.org/1/users/keymaker/
<http://sdf.org/1/users/keymaker/>   * compliant!    ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9       ]

If you're worried about keyloggers or remote control tools on admin's workstations, then you've lost the war already.

I'm afraid that approach is how the non-passphrased PEM files in $HOME/.chef or in NFS shares happened in the first place.

the signup process becomes "paste in your public ssh key" which people should be trained to do from interacting with AWS and other cloud services.

I'd actually discourage this, given a choice. Many people tend to re-use the same, unsigned key for many distinct applications, and this would permit multiple chef admin accounts to use the same private and public keys, which could get..... odd to clean up after and verify. If I had preferences, I'd use Kerberos credentials and avoid the whole "storing private keys" problem. But that rewrite might be even more painful than adding pass-phrased key support.

                    Nico Kadel-Garcia

From: Lamont Granquist [lamont@opscode.com]
Sent: Saturday, November 09, 2013 2:19 PM
To: chef@lists.opscode.com
Subject: [chef] Re: Re: RE: Re: RE: Re: Securing Knife

If you're worried about keyloggers or remote control tools on admin's workstations, then you've lost the war already.

There is a clear risk vector in stolen laptops and in drive-by hacks of laptops snarfing unencrypted credentials.

Making knife encrypt the existing user.pem file would be fairly easy.

Making knife, and the chef-server, use ssh identities and integrate with ssh-agent would be very cool, but obviously more work. Since Dan is doing work that will eliminate the need for validation keys and leverage the user creds for provisioning servers, if we could pick up existing ssh keys then that would make chef a lot easier to use -- the signup process becomes "paste in your public ssh key" which people should be trained to do from interacting with AWS and other cloud services. That key starts to be a lot of eggs in one basket, but for admins with root access, compromise of their ssh credentials is usually enough to own the entire shop anyway -- admin laptops should have encrypted drives and use ssh-agent at that point.

On 11/7/13 9:43 PM, Ranjib Dey wrote:
i like the idea. i dont think it will be lot of work to implement this. though i find the whole password/ssh-agent/gnome bit strange (keyloggers? each tool will add 1+ vector), but this will help in general 2 factor auth/ ldap backed etc.

On Thu, Nov 7, 2013 at 5:29 PM, Kemp, Joseph A. (JKEMP) <JKEMP@arinc.commailto:JKEMP@arinc.com> wrote:
Julian,

My understanding of the Chef infrastructure is that if someone gets a copy of the physical pem file and has access to your chef server they have the keys to your kingdom. This would be complete control over every managed device. It's just like keeping all of the root passwords in a text file on your laptop drive. I know I can use disk encryption to protect the pem but that seems like a pretty significant requirement to be able to use chef securely.

There are a handful of spots that load the key with a call like this: OpenSSL::PKey::RSA.new(rest.signing_key). Right now the code relies on the underlying openssl library to ask for the password but it can be easily passed in as a second parameter to this existing call. So in a sense knife already supports encrypted pems but it just isn't implemented very well.

  1. So the first option would be to allow the password to be passed on the command line to knife. Better would be to get the password from stdin. There are some significant risks to allowing the password on the command line but it is better than nothing.

  2. The second option is for knife to check to see if the pem is encrypted, if so ask for the password once and then pass it into all subsequent openssl calls for the user.

  3. Finally, ssh-agent has addressed this same type of issue for many years. It's a good balance between security and usability. Figuring out how to build on that model would most likely be the most elegant solution.

  4. Just thought of another angle. All of this "security" is built on top of openssl. Looking at the man page (/docs/manmaster/man1/openssl.html) it lists 5 different techniques to get the password into the openssl process. These look to be good options also.

-Joe

-----Original Message-----
From: Julian C. Dunn [mailto:jdunn@aquezada.commailto:jdunn@aquezada.com]
Sent: Thursday, November 07, 2013 6:40 PM
To: chef@lists.opscode.commailto:chef@lists.opscode.com
Subject: [chef] Re: RE: Re: Securing Knife

Joe,

It's the first time I've heard this raised as a concern, but that doesn't mean it's not valid. I think the use case so far has been that each Chef admin has the PEM on their local workstations as opposed to a shared workstation/jumpbox.

However, one has to balance usability versus security. Even if Knife only prompted once per command for the user's passphrase, that still seems like a PITA. Doesn't that get in the way of operations?

Again, I think it's a feature request that we would consider if you can define how you see PEM passphrases would work without being too intrusive.

  • Julian

On Thu, Nov 7, 2013 at 7:33 AM, Kadel-Garcia, Nico <NKadelGarcia-consultant@scholastic.commailto:NKadelGarcia-consultant@scholastic.com> wrote:

Such a private pem file is still stored locally, effectively in
plain-text, with no password protection. For home directories on
poorly secured NFS mounts it's even worse because any host connected
to the relevant network can NFS mount the directory, "sudo" to the
relevant uid, and gain access to the unencrypted keys. NFSv4 with
Kerberized authentication can help with that, as can proper CIFS
configurations for Windows based fileshares, but the key is still available on all backup media in plaintext.

I'd recommend using a highly secured local disk area, such as an
encrypted partition, and a symlink from the relevant workspace to the
locally encrypted partition. And I'd suggest running chef server
operations only from that secured workspace, especially for sensitive
environments and source code manipulation. Since the source code for
the cookbooks can often be used to manipulate or ruin deployed
systems, similar precautions should be used for SSH keys for any central source repository.

And as mentioned, don't forget to passphrase protect SSH keys? The old
"keychain" perl script works well for managing personal SSH keys in
command-line environments, and many modern window manager environments
like Gnome and KDE have built-in tools for SSH key management.


From: Mike [miketheman@gmail.commailto:miketheman@gmail.com]

Sent: Wednesday, November 06, 2013 5:45 PM
To: chef@lists.opscode.commailto:chef@lists.opscode.com
Subject: [chef] Re: Securing Knife

Have individual/personal admin-level pem files - don't share a
centralized one.

knife client create new_person --admin

Ref: http://docs.opscode.com/chef/knife.html#create

-M

On Wed, Nov 6, 2013 at 5:40 PM, Kemp, Joseph A. (JKEMP)
<JKEMP@arinc.commailto:JKEMP@arinc.com>
wrote:

I am puzzled how to secure the use of knife in open source chef. If
I add a password to the user PEM I am forced to enter the password
multiple times for each knife command. So that's not a very user
friendly option. Someone else suggested storing the pem on an
encrypted file system/device/etc. What is the best practice to provide controlled admin access to the chef server?
It's a little unnerving that someone with a copy of any admin PEM
file gains complete control over your infrastructure. I feel like I
must be missing something.

Thanks,

-Joe

--
[ Julian C. Dunn <jdunn@aquezada.commailto:jdunn@aquezada.com> * Sorry, I'm ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://<UrlBlockedError.aspx>sdf.org/1/users/keymaker/http://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]

And besides encrypting your drive, or just your /home dir, in Linux,
it's easy and can be done during the install. For OS X 10.9 they have
easy options to encrypt the drive before you install - which is what I
did; reinstalled but first wiped the disk and encrypted it before OS
install. Here's a simple HOWTO:
http://krypted.com/mac-security/encrypt-mavericks-and-mavericks-server/
-- Then as far as backing up sensitive files such as keys, I recommend
SpiderOak above all the other 'cloud' backup/sync options; when using
SpiderOak your data is encrypted on your system before it is sent to
their servers, so you hold the keys as it were and the data is useless
for anyone else.

Sorry, not Chef related, but I think this will alay some concerns the OP had.

On Sat, Nov 9, 2013 at 2:00 PM, Kadel-Garcia, Nico
NKadelGarcia-consultant@scholastic.com wrote:

If you're worried about keyloggers or remote control tools on admin's
workstations, then you've lost the war already.

I'm afraid that approach is how the non-passphrased PEM files in $HOME/.chef
or in NFS shares happened in the first place.

the signup process becomes "paste in your public ssh key" which people
should be trained to do from interacting with AWS and other cloud services.

I'd actually discourage this, given a choice. Many people tend to re-use the
same, unsigned key for many distinct applications, and this would permit
multiple chef admin accounts to use the same private and public keys, which
could get..... odd to clean up after and verify. If I had preferences, I'd
use Kerberos credentials and avoid the whole "storing private keys" problem.
But that rewrite might be even more painful than adding pass-phrased key
support.

                    Nico Kadel-Garcia

From: Lamont Granquist [lamont@opscode.com]
Sent: Saturday, November 09, 2013 2:19 PM
To: chef@lists.opscode.com
Subject: [chef] Re: Re: RE: Re: RE: Re: Securing Knife

If you're worried about keyloggers or remote control tools on admin's
workstations, then you've lost the war already.

There is a clear risk vector in stolen laptops and in drive-by hacks of
laptops snarfing unencrypted credentials.

Making knife encrypt the existing user.pem file would be fairly easy.

Making knife, and the chef-server, use ssh identities and integrate with
ssh-agent would be very cool, but obviously more work. Since Dan is doing
work that will eliminate the need for validation keys and leverage the user
creds for provisioning servers, if we could pick up existing ssh keys then
that would make chef a lot easier to use -- the signup process becomes
"paste in your public ssh key" which people should be trained to do from
interacting with AWS and other cloud services. That key starts to be a lot
of eggs in one basket, but for admins with root access, compromise of their
ssh credentials is usually enough to own the entire shop anyway -- admin
laptops should have encrypted drives and use ssh-agent at that point.

On 11/7/13 9:43 PM, Ranjib Dey wrote:

i like the idea. i dont think it will be lot of work to implement this.
though i find the whole password/ssh-agent/gnome bit strange (keyloggers?
each tool will add 1+ vector), but this will help in general 2 factor auth/
ldap backed etc.

On Thu, Nov 7, 2013 at 5:29 PM, Kemp, Joseph A. (JKEMP) JKEMP@arinc.com
wrote:

Julian,

My understanding of the Chef infrastructure is that if someone gets a copy
of the physical pem file and has access to your chef server they have the
keys to your kingdom. This would be complete control over every managed
device. It's just like keeping all of the root passwords in a text file on
your laptop drive. I know I can use disk encryption to protect the pem but
that seems like a pretty significant requirement to be able to use chef
securely.

There are a handful of spots that load the key with a call like this:
OpenSSL::PKey::RSA.new(rest.signing_key). Right now the code relies on the
underlying openssl library to ask for the password but it can be easily
passed in as a second parameter to this existing call. So in a sense knife
already supports encrypted pems but it just isn't implemented very well.

  1. So the first option would be to allow the password to be passed on the
    command line to knife. Better would be to get the password from stdin.
    There are some significant risks to allowing the password on the command
    line but it is better than nothing.

  2. The second option is for knife to check to see if the pem is
    encrypted, if so ask for the password once and then pass it into all
    subsequent openssl calls for the user.

  3. Finally, ssh-agent has addressed this same type of issue for many
    years. It's a good balance between security and usability. Figuring out how
    to build on that model would most likely be the most elegant solution.

  4. Just thought of another angle. All of this "security" is built on top
    of openssl. Looking at the man page
    (/docs/manmaster/man1/openssl.html) it
    lists 5 different techniques to get the password into the openssl process.
    These look to be good options also.

-Joe

-----Original Message-----
From: Julian C. Dunn [mailto:jdunn@aquezada.com]
Sent: Thursday, November 07, 2013 6:40 PM
To: chef@lists.opscode.com
Subject: [chef] Re: RE: Re: Securing Knife

Joe,

It's the first time I've heard this raised as a concern, but that doesn't
mean it's not valid. I think the use case so far has been that each Chef
admin has the PEM on their local workstations as opposed to a shared
workstation/jumpbox.

However, one has to balance usability versus security. Even if Knife only
prompted once per command for the user's passphrase, that still seems like a
PITA. Doesn't that get in the way of operations?

Again, I think it's a feature request that we would consider if you can
define how you see PEM passphrases would work without being too intrusive.

  • Julian

On Thu, Nov 7, 2013 at 7:33 AM, Kadel-Garcia, Nico
NKadelGarcia-consultant@scholastic.com wrote:

Such a private pem file is still stored locally, effectively in
plain-text, with no password protection. For home directories on
poorly secured NFS mounts it's even worse because any host connected
to the relevant network can NFS mount the directory, "sudo" to the
relevant uid, and gain access to the unencrypted keys. NFSv4 with
Kerberized authentication can help with that, as can proper CIFS
configurations for Windows based fileshares, but the key is still
available on all backup media in plaintext.

I'd recommend using a highly secured local disk area, such as an
encrypted partition, and a symlink from the relevant workspace to the
locally encrypted partition. And I'd suggest running chef server
operations only from that secured workspace, especially for sensitive
environments and source code manipulation. Since the source code for
the cookbooks can often be used to manipulate or ruin deployed
systems, similar precautions should be used for SSH keys for any central
source repository.

And as mentioned, don't forget to passphrase protect SSH keys? The old
"keychain" perl script works well for managing personal SSH keys in
command-line environments, and many modern window manager environments
like Gnome and KDE have built-in tools for SSH key management.


From: Mike [miketheman@gmail.com]

Sent: Wednesday, November 06, 2013 5:45 PM
To: chef@lists.opscode.com
Subject: [chef] Re: Securing Knife

Have individual/personal admin-level pem files - don't share a
centralized one.

knife client create new_person --admin

Ref: http://docs.opscode.com/chef/knife.html#create

-M

On Wed, Nov 6, 2013 at 5:40 PM, Kemp, Joseph A. (JKEMP)
JKEMP@arinc.com
wrote:

I am puzzled how to secure the use of knife in open source chef. If
I add a password to the user PEM I am forced to enter the password
multiple times for each knife command. So that's not a very user
friendly option. Someone else suggested storing the pem on an
encrypted file system/device/etc. What is the best practice to provide
controlled admin access to the chef server?
It's a little unnerving that someone with a copy of any admin PEM
file gains complete control over your infrastructure. I feel like I
must be missing something.

Thanks,

-Joe

--
[ Julian C. Dunn jdunn@aquezada.com * Sorry, I'm ]
[ WWW: Julian Dunn's Blog - Commentary on media, technology, and everything in between. * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]

On 11/9/13 12:00 PM, Kadel-Garcia, Nico wrote:

If you're worried about keyloggers or remote control tools on
admin's workstations, then you've lost the war already.

I'm afraid that approach is how the non-passphrased PEM files in
$HOME/.chef or in NFS shares happened in the first place.
I don't understand this. If you can control an admins workstation, you
can hijack existing ssh sessions, you can sniff passwords, you can steal
key material out of RAM, etc. If you're using kerberos, I can replace
the kinit binary to log your password when you authenticate (or if
you're using kerberized ssh or whatever, I can replace those binaries
and log your password, I always have an avenue to exploit you). Even
if you're using a two-factor, I may not be able to reauthenticate as
you, but when you authenticate I can backdoor whatever you're using to
communicate with and inject my own commands. That war isn't winnable.

the signup process becomes "paste in your public ssh key" which
people should be trained to do from interacting with AWS and other
cloud services.

I'd actually discourage this, given a choice. Many people tend to
re-use the same, unsigned key for many distinct applications, and
this would permit multiple chef admin accounts to use the same private
and public keys, which could get..... odd to clean up after and
verify. If I had preferences, I'd use Kerberos credentials and avoid
the whole "storing private keys" problem. But that rewrite might be
even more painful than adding pass-phrased key support.
You can have 10 different pieces of key material on your laptop and I
can still steal them all. I already presented how to easily backdoor
whatever client binary you're using to auth to kerberos. I can also
steal your credentials cache and reuse those credentials for their
lifetime (really trivially in the case of file credentials caches in
/tmp, but I still have access to them if you're using an in-memory
credentials cache -- I can impersonate you on your laptop so I have
access to everything that you do). By creating more and more keys, now
you have a lot of different things to manage with the same risk profile
and have to deal with rotating and managing them, and if you make any
mistakes that are created by having N different things to manage, then
you're less safe.

Once you're assuming active ongoing compromise of a laptop with
keylogger-levels of infiltration, then you're just rearranging deck
chairs on the titanic.

                    Nico Kadel-Garcia

From: Lamont Granquist [lamont@opscode.com]
Sent: Saturday, November 09, 2013 2:19 PM
To: chef@lists.opscode.com
Subject: [chef] Re: Re: RE: Re: RE: Re: Securing Knife

If you're worried about keyloggers or remote control tools on admin's
workstations, then you've lost the war already.

There is a clear risk vector in stolen laptops and in drive-by hacks
of laptops snarfing unencrypted credentials.

Making knife encrypt the existing user.pem file would be fairly easy.

Making knife, and the chef-server, use ssh identities and integrate
with ssh-agent would be very cool, but obviously more work. Since Dan
is doing work that will eliminate the need for validation keys and
leverage the user creds for provisioning servers, if we could pick up
existing ssh keys then that would make chef a lot easier to use -- the
signup process becomes "paste in your public ssh key" which people
should be trained to do from interacting with AWS and other cloud
services. That key starts to be a lot of eggs in one basket, but for
admins with root access, compromise of their ssh credentials is
usually enough to own the entire shop anyway -- admin laptops should
have encrypted drives and use ssh-agent at that point.

On 11/7/13 9:43 PM, Ranjib Dey wrote:

i like the idea. i dont think it will be lot of work to implement
this. though i find the whole password/ssh-agent/gnome bit strange
(keyloggers? each tool will add 1+ vector), but this will help in
general 2 factor auth/ ldap backed etc.

On Thu, Nov 7, 2013 at 5:29 PM, Kemp, Joseph A. (JKEMP)
<JKEMP@arinc.com mailto:JKEMP@arinc.com> wrote:

Julian,

My understanding of the Chef infrastructure is that if someone
gets a copy of the physical pem file and has access to your chef
server they have the keys to your kingdom.  This would be
complete control over every managed device.  It's just like
keeping all of the root passwords in a text file on your laptop
drive.  I know I can use disk encryption to protect the pem but
that seems like a pretty significant requirement to be able to
use chef securely.

There are a handful of spots that load the key with a call like
this: OpenSSL::PKey::RSA.new(rest.signing_key).  Right now the
code relies on the underlying openssl library to ask for the
password but it can be easily passed in as a second parameter to
this existing call.  So in a sense knife already supports
encrypted pems but it just isn't implemented very well.

1.  So the first option would be to allow the password to be
passed on the command line to knife.  Better would be to get the
password from stdin.  There are some significant risks to
allowing the password on the command line but it is better than
nothing.

2.  The second option is for knife to check to see if the pem is
encrypted, if so ask for the password once and then pass it into
all subsequent openssl calls for the user.

3. Finally, ssh-agent has addressed this same type of issue for
many years.  It's a good balance between security and usability.
Figuring out how to build on that model would most likely be the
most elegant solution.

4. Just thought of another angle.  All of this "security" is
built on top of openssl.  Looking at the man page
(http://www.openssl.org/docs/apps/openssl.html#PASS_PHRASE_ARGUMENTS)
it lists 5 different techniques to get the password into the
openssl process.  These look to be good options also.


-Joe


-----Original Message-----
From: Julian C. Dunn [mailto:jdunn@aquezada.com
<mailto:jdunn@aquezada.com>]
Sent: Thursday, November 07, 2013 6:40 PM
To: chef@lists.opscode.com <mailto:chef@lists.opscode.com>
Subject: [chef] Re: RE: Re: Securing Knife

Joe,

It's the first time I've heard this raised as a concern, but that
doesn't mean it's not valid. I think the use case so far has been
that each Chef admin has the PEM on their local workstations as
opposed to a shared workstation/jumpbox.

However, one has to balance usability versus security. Even if
Knife only prompted once per command for the user's passphrase,
that still seems like a PITA. Doesn't that get in the way of
operations?

Again, I think it's a feature request that we would consider if
you can define how you see PEM passphrases would work without
being too intrusive.

- Julian

On Thu, Nov 7, 2013 at 7:33 AM, Kadel-Garcia, Nico
<NKadelGarcia-consultant@scholastic.com
<mailto:NKadelGarcia-consultant@scholastic.com>> wrote:
> Such a private pem file is still stored locally, effectively in
> plain-text, with no password protection. For home directories on
> poorly secured NFS mounts it's even worse because any host
connected
> to the relevant network can NFS mount the directory, "sudo" to the
> relevant uid, and gain access to the unencrypted keys. NFSv4 with
> Kerberized authentication can help with that, as can proper CIFS
> configurations for Windows based fileshares, but the key is
still available on all backup media in plaintext.
>
> I'd recommend using a highly secured local disk area, such as an
> encrypted partition, and a symlink from the relevant workspace
to the
> locally encrypted partition. And I'd suggest running chef server
> operations only from that secured workspace, especially for
sensitive
> environments and source code manipulation. Since the source
code for
> the cookbooks can often be used to manipulate or ruin deployed
> systems, similar precautions should be used for SSH keys for
any central source repository.
>
> And as mentioned, don't forget to passphrase protect SSH keys?
The old
> "keychain" perl script works well for managing personal SSH keys in
> command-line environments, and many modern window manager
environments
> like Gnome and KDE have built-in tools for SSH key management.
>
> ________________________________
> From: Mike [miketheman@gmail.com <mailto:miketheman@gmail.com>]
>
> Sent: Wednesday, November 06, 2013 5:45 PM
> To: chef@lists.opscode.com <mailto:chef@lists.opscode.com>
> Subject: [chef] Re: Securing Knife
>
> Have individual/personal admin-level pem files - don't share a
> centralized one.
>
>   knife client create new_person --admin
>
> Ref: http://docs.opscode.com/chef/knife.html#create
>
> -M
>
>
> On Wed, Nov 6, 2013 at 5:40 PM, Kemp, Joseph A. (JKEMP)
> <JKEMP@arinc.com <mailto:JKEMP@arinc.com>>
> wrote:
>>
>> I am puzzled how to secure the use of knife in open source
chef.  If
>> I add a password to the user PEM I am forced to enter the password
>> multiple times for each knife command.  So that's not a very user
>> friendly option.  Someone else suggested storing the pem on an
>> encrypted file system/device/etc.  What is the best practice
to provide controlled admin access to the chef server?
>> It's a little unnerving that someone with a copy of any admin PEM
>> file gains complete control over your infrastructure.  I feel
like I
>> must be missing something.
>>
>>
>>
>> Thanks,
>>
>> -Joe
>
>



--
[ Julian C. Dunn <jdunn@aquezada.com <mailto:jdunn@aquezada.com>>
     * Sorry, I'm    ]
[ WWW: http://www.aquezada.com/staff/julian    * only Web 1.0  ]
[ gopher://sdf.org/1/users/keymaker/
<http://sdf.org/1/users/keymaker/>           * compliant!    ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9       ]

Here is my quick fix https://github.com/jkemp101/chef/compare/opscode:11.6-stable...Key-Password. It adds the concept of a private key password. So now I can start knife with the --key-pass command line option. This keeps openssl from asking for the password. Turns out the ${password_prompt} variable in Eclipse will cache the last password. So I only have to enter the password once per Eclipse session to fire off the external tool.

Note that this also allows the password to be set in the knife.rb config file. You can do something like client_key_password IO.read("#{current_dir }/password.txt") in the config file. This would allow the password to be read out of a text file that could be deleted at the end of the session. Thought I would pass it along in case someone else want to make use of it. I have only tested knife uploads and a couple other commands. Further testing is necessary…

-Joe

-----Original Message-----
From: Lamont Granquist [mailto:lamont@opscode.com]
Sent: Sunday, November 10, 2013 5:24 PM
To: chef@lists.opscode.com
Subject: [chef] Re: RE: Re: Re: RE: Re: RE: Re: Securing Knife

On 11/9/13 12:00 PM, Kadel-Garcia, Nico wrote:

> If you're worried about keyloggers or remote control tools on admin's workstations, then you've lost the war already.

I'm afraid that approach is how the non-passphrased PEM files in $HOME/.chef or in NFS shares happened in the first place.

I don’t understand this. If you can control an admins workstation, you can hijack existing ssh sessions, you can sniff passwords, you can steal key material out of RAM, etc. If you’re using kerberos, I can replace the kinit binary to log your password when you authenticate (or if you’re using kerberized ssh or whatever, I can replace those binaries and log your password, I always have an avenue to exploit you). Even if you’re using a two-factor, I may not be able to reauthenticate as you, but when you authenticate I can backdoor whatever you’re using to communicate with and inject my own commands. That war isn’t winnable.

> the signup process becomes "paste in your public ssh key" which people should be trained to do from interacting with AWS and other cloud services.

I'd actually discourage this, given a choice. Many people tend to re-use the same, unsigned key for many  distinct applications, and this would permit multiple chef admin accounts to use the same private and public keys, which could get..... odd to clean up after and verify. If I had preferences, I'd use Kerberos credentials and avoid the whole "storing private keys" problem. But that rewrite might be even more painful than adding pass-phrased key support.

You can have 10 different pieces of key material on your laptop and I can still steal them all. I already presented how to easily backdoor whatever client binary you’re using to auth to kerberos. I can also steal your credentials cache and reuse those credentials for their lifetime (really trivially in the case of file credentials caches in /tmp, but I still have access to them if you’re using an in-memory credentials cache – I can impersonate you on your laptop so I have access to everything that you do). By creating more and more keys, now you have a lot of different things to manage with the same risk profile and have to deal with rotating and managing them, and if you make any mistakes that are created by having N different things to manage, then you’re less safe.

Once you’re assuming active ongoing compromise of a laptop with keylogger-levels of infiltration, then you’re just rearranging deck chairs on the titanic.

                        Nico Kadel-Garcia

From: Lamont Granquist [lamont@opscode.com]
Sent: Saturday, November 09, 2013 2:19 PM
To: chef@lists.opscode.com
Subject: [chef] Re: Re: RE: Re: RE: Re: Securing Knife



If you're worried about keyloggers or remote control tools on admin's workstations, then you've lost the war already.

There is a clear risk vector in stolen laptops and in drive-by hacks of laptops snarfing unencrypted credentials.

Making knife encrypt the existing user.pem file would be fairly easy.

Making knife, and the chef-server, use ssh identities and integrate with ssh-agent would be very cool, but obviously more work.  Since Dan is doing work that will eliminate the need for validation keys and leverage the user creds for provisioning servers, if we could pick up existing ssh keys then that would make chef a lot easier to use -- the signup process becomes "paste in your public ssh key" which people should be trained to do from interacting with AWS and other cloud services.  That key starts to be a lot of eggs in one basket, but for admins with root access, compromise of their ssh credentials is usually enough to own the entire shop anyway -- admin laptops should have encrypted drives and use ssh-agent at that point.

On 11/7/13 9:43 PM, Ranjib Dey wrote:


	i like the idea. i dont think it will be lot of work to implement this. though i find the whole password/ssh-agent/gnome bit strange (keyloggers? each tool will add 1+ vector), but this will help in general 2 factor auth/ ldap backed etc.
	


	On Thu, Nov 7, 2013 at 5:29 PM, Kemp, Joseph A. (JKEMP) <JKEMP@arinc.com> wrote:
	

		Julian,
		
		My understanding of the Chef infrastructure is that if someone gets a copy of the physical pem file and has access to your chef server they have the keys to your kingdom.  This would be complete control over every managed device.  It's just like keeping all of the root passwords in a text file on your laptop drive.  I know I can use disk encryption to protect the pem but that seems like a pretty significant requirement to be able to use chef securely.
		
		There are a handful of spots that load the key with a call like this: OpenSSL::PKey::RSA.new(rest.signing_key).  Right now the code relies on the underlying openssl library to ask for the password but it can be easily passed in as a second parameter to this existing call.  So in a sense knife already supports encrypted pems but it just isn't implemented very well.
		
		1.  So the first option would be to allow the password to be passed on the command line to knife.  Better would be to get the password from stdin.  There are some significant risks to allowing the password on the command line but it is better than nothing.
		
		2.  The second option is for knife to check to see if the pem is encrypted, if so ask for the password once and then pass it into all subsequent openssl calls for the user.
		
		3. Finally, ssh-agent has addressed this same type of issue for many years.  It's a good balance between security and usability. Figuring out how to build on that model would most likely be the most elegant solution.
		
		4. Just thought of another angle.  All of this "security" is built on top of openssl.  Looking at the man page (http://www.openssl.org/docs/apps/openssl.html#PASS_PHRASE_ARGUMENTS) it lists 5 different techniques to get the password into the openssl process.  These look to be good options also.
		
		
		-Joe
		
		
		-----Original Message-----
		From: Julian C. Dunn [mailto:jdunn@aquezada.com]
		Sent: Thursday, November 07, 2013 6:40 PM
		To: chef@lists.opscode.com
		Subject: [chef] Re: RE: Re: Securing Knife
		
		Joe,
		
		It's the first time I've heard this raised as a concern, but that doesn't mean it's not valid. I think the use case so far has been that each Chef admin has the PEM on their local workstations as opposed to a shared workstation/jumpbox.
		
		However, one has to balance usability versus security. Even if Knife only prompted once per command for the user's passphrase, that still seems like a PITA. Doesn't that get in the way of operations?
		
		Again, I think it's a feature request that we would consider if you can define how you see PEM passphrases would work without being too intrusive.
		
		- Julian
		
		On Thu, Nov 7, 2013 at 7:33 AM, Kadel-Garcia, Nico <NKadelGarcia-consultant@scholastic.com> wrote:
		> Such a private pem file is still stored locally, effectively in
		> plain-text, with no password protection. For home directories on
		> poorly secured NFS mounts it's even worse because any host connected
		> to the relevant network can NFS mount the directory, "sudo" to the
		> relevant uid, and gain access to the unencrypted keys. NFSv4 with
		> Kerberized authentication can help with that, as can proper CIFS
		> configurations for Windows based fileshares, but the key is still available on all backup media in plaintext.
		>
		> I'd recommend using a highly secured local disk area, such as an
		> encrypted partition, and a symlink from the relevant workspace to the
		> locally encrypted partition. And I'd suggest running chef server
		> operations only from that secured workspace, especially for sensitive
		> environments and source code manipulation. Since the source code for
		> the cookbooks can often be used to manipulate or ruin deployed
		> systems, similar precautions should be used for SSH keys for any central source repository.
		>
		> And as mentioned, don't forget to passphrase protect SSH keys? The old
		> "keychain" perl script works well for managing personal SSH keys in
		> command-line environments, and many modern window manager environments
		> like Gnome and KDE have built-in tools for SSH key management.
		>
		> ________________________________
		> From: Mike [miketheman@gmail.com]
		>
		> Sent: Wednesday, November 06, 2013 5:45 PM
		> To: chef@lists.opscode.com
		> Subject: [chef] Re: Securing Knife
		>
		> Have individual/personal admin-level pem files - don't share a
		> centralized one.
		>
		>   knife client create new_person --admin
		>
		> Ref: http://docs.opscode.com/chef/knife.html#create
		>
		> -M
		>
		>
		> On Wed, Nov 6, 2013 at 5:40 PM, Kemp, Joseph A. (JKEMP)
		> <JKEMP@arinc.com>
		> wrote:
		>>
		>> I am puzzled how to secure the use of knife in open source chef.  If
		>> I add a password to the user PEM I am forced to enter the password
		>> multiple times for each knife command.  So that's not a very user
		>> friendly option.  Someone else suggested storing the pem on an
		>> encrypted file system/device/etc.  What is the best practice to provide controlled admin access to the chef server?
		>> It's a little unnerving that someone with a copy of any admin PEM
		>> file gains complete control over your infrastructure.  I feel like I
		>> must be missing something.
		>>
		>>
		>>
		>> Thanks,
		>>
		>> -Joe
		>
		>
		
		
		
		--
		[ Julian C. Dunn <jdunn@aquezada.com>          * Sorry, I'm    ]
		[ WWW: http://www.aquezada.com/staff/julian    * only Web 1.0  ]
		[ gopher://sdf.org/1/users/keymaker/           * compliant!    ]
		[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9       ]