Knife winrm browsing network shares


#1

Has anyone successfully used knife winrm for accessing remote shares?
I can successfully do a:
$ knife winrm -m server.mydomain.local -P ‘mypwd’ -p 5985 -x
’mydomain\Administrator’ dir

but get an “Access is denied” when doing dir \\filer\share

I have followed this doc and the knife winrm doc to setup the client
http://msdn.microsoft.com/en-us/library/windows/desktop/ee309365(v=vs.85).aspx
with no luck.

I see that the NTLM auth is successful on the windows host logs, but it seems
like it’s not using the CredSSP correctly. Toggling the CredSSP settings on the
windows server has no effect at all.

Any help is appreciated thanks!


#2

On Jun 6, 2013, at 3:00 PM, grepjuice@gmail.com wrote:

but get an “Access is denied” when doing dir \\filer\share

I’m curious – has anyone noticed what version of Chef they’re using with regards to these problems? At least some of the tickets we’ve seen appear to indicate that the problems with winrm are more prevalent with Chef 11.x, whereas Chef 10.18.2 seems to work fine.

But in our case there’s also a difference between our using Private Chef (based on Chef 11) versus the other site using the Open Source Chef, so we’re not sure if maybe this is a Private Chef vs. Open Source Chef problem, or maybe a Chef 11.x vs. Chef 10.18.x problem, or what.

I’m also curious if people have looked at what version of knife-essentials they’re using? Some other problems we’ve seen might be related to using older versions of knife-essentials versus newer ones, so this is a dimension to the issue that we’re also trying to track down.

If we can’t get this problem resolved quickly, we might be forced to give up on Private Chef and step back to Open Source Chef.

Yes, we are working the internal channels at Opscode, but getting more information from the community would also be very useful.


Brad Knowles brad@shub-internet.org
LinkedIn Profile: http://tinyurl.com/y8kpxu


#3

To narrow down the problem, it would be good to know the following:

  1. On what operating system / version are you running knife winrm?
  2. What OS version is the target OS?
  3. Is the target OS joined to a domain?
  4. What OS is running on the machine that is the target of the net use
    command (i.e. the “server” in “\server\share”)?
  5. Is the machine that is the target of the net use command in the same
    domain as the machine that is the target of the knife command?
  6. Did you set basic auth to true and disable encryption? I’m not sure I’d
    expect off-machine access to work in that case.

A short-term workaround is to use scheduled tasks – if you’re interested,
I can give you a specific example, though it would help to know what
you’re trying to accomplish with the net use (e.g. run a script, copy a
specific file, etc.).

Problems here are most likely due to the winrm gems used by knife winrm.
Those gems rely on libraries on the system running knife, which could be
Windows, Mac, Ubuntu, etc., and the presence of the right libraries and /
or bug fixes on those platforms varies. We are currently investigating
specific issues with NTLM for instance to see if unreleased versions of
the gems have the improvements needed to make the NTLM scenario work,
hopefully we’ll have some prescriptive statements about how to make this
scenario work.

A few more questions:

  1. Rather than use knife winrm to perform net use, is this operation
    something that you could do from within a Chef recipe that runs on the
    target system? That could free you from limitations of WinRM.
  2. You could probably use the recipe route via the RC version of
    mixlib-shellout, which supports running scripts with alternate
    credentials, or some upcoming changes to the Windows mount provider that
    will be merged to master this week.

Regarding private vs. open source chef, I don’t see anything here that
should make open source chef work differently than private chef. The
connectivity is really about winrm configuration on client and server, and
the winrm / ntlm gems and libraries on those systems, and finally knife
which ships with chef client and plugins like knife-windows that add the
winrm functionality to knife. The chef server code really is not involved
here.

Thanks.


Adam Edwards
Software Development Engineer, Opscode, Inc.

On 6/10/13 9:07 PM, “Brad Knowles” brad@shub-internet.org wrote:

On Jun 6, 2013, at 3:00 PM, grepjuice@gmail.com wrote:

but get an “Access is denied” when doing dir \\filer\share

I’m curious – has anyone noticed what version of Chef they’re using with
regards to these problems? At least some of the tickets we’ve seen
appear to indicate that the problems with winrm are more prevalent with
Chef 11.x, whereas Chef 10.18.2 seems to work fine.

But in our case there’s also a difference between our using Private Chef
(based on Chef 11) versus the other site using the Open Source Chef, so
we’re not sure if maybe this is a Private Chef vs. Open Source Chef
problem, or maybe a Chef 11.x vs. Chef 10.18.x problem, or what.

I’m also curious if people have looked at what version of
knife-essentials they’re using? Some other problems we’ve seen might be
related to using older versions of knife-essentials versus newer ones, so
this is a dimension to the issue that we’re also trying to track down.

If we can’t get this problem resolved quickly, we might be forced to give
up on Private Chef and step back to Open Source Chef.

Yes, we are working the internal channels at Opscode, but getting more
information from the community would also be very useful.


Brad Knowles brad@shub-internet.org
LinkedIn Profile: http://tinyurl.com/y8kpxu


#4

The root of this issue here is that access token created by winrm are marked “remote” by the server. This means that they cannot be used to access network resources. I am investigating adding credssp to the winrm library which would allow winrm to create non-remote tokens-but this is a ways out.

In the vagrant windows gem we use a powershell script to wrap the process in a new user session which has an access token that is not marked as remote.

https://github.com/WinRb/vagrant-windows/blob/master/lib/vagrant-windows/scripts/ps_runas.ps1.erb

On Jun 11, 2013, at 7:04 AM, “Adam Edwards” <adamed@opscode.commailto:adamed@opscode.com> wrote:

To narrow down the problem, it would be good to know the following:

  1. On what operating system / version are you running knife winrm?
  2. What OS version is the target OS?
  3. Is the target OS joined to a domain?
  4. What OS is running on the machine that is the target of the net use
    command (i.e. the “server” in “\server\share”)?
  5. Is the machine that is the target of the net use command in the same
    domain as the machine that is the target of the knife command?
  6. Did you set basic auth to true and disable encryption? I’m not sure I’d
    expect off-machine access to work in that case.

A short-term workaround is to use scheduled tasks – if you’re interested,
I can give you a specific example, though it would help to know what
you’re trying to accomplish with the net use (e.g. run a script, copy a
specific file, etc.).

Problems here are most likely due to the winrm gems used by knife winrm.
Those gems rely on libraries on the system running knife, which could be
Windows, Mac, Ubuntu, etc., and the presence of the right libraries and /
or bug fixes on those platforms varies. We are currently investigating
specific issues with NTLM for instance to see if unreleased versions of
the gems have the improvements needed to make the NTLM scenario work,
hopefully we’ll have some prescriptive statements about how to make this
scenario work.

A few more questions:

  1. Rather than use knife winrm to perform net use, is this operation
    something that you could do from within a Chef recipe that runs on the
    target system? That could free you from limitations of WinRM.
  2. You could probably use the recipe route via the RC version of
    mixlib-shellout, which supports running scripts with alternate
    credentials, or some upcoming changes to the Windows mount provider that
    will be merged to master this week.

Regarding private vs. open source chef, I don’t see anything here that
should make open source chef work differently than private chef. The
connectivity is really about winrm configuration on client and server, and
the winrm / ntlm gems and libraries on those systems, and finally knife
which ships with chef client and plugins like knife-windows that add the
winrm functionality to knife. The chef server code really is not involved
here.

Thanks.


Adam Edwards
Software Development Engineer, Opscode, Inc.

On 6/10/13 9:07 PM, “Brad Knowles” <brad@shub-internet.orgmailto:brad@shub-internet.org> wrote:

On Jun 6, 2013, at 3:00 PM, grepjuice@gmail.commailto:grepjuice@gmail.com wrote:

but get an “Access is denied” when doing dir \\filer\share

I’m curious – has anyone noticed what version of Chef they’re using with
regards to these problems? At least some of the tickets we’ve seen
appear to indicate that the problems with winrm are more prevalent with
Chef 11.x, whereas Chef 10.18.2 seems to work fine.

But in our case there’s also a difference between our using Private Chef
(based on Chef 11) versus the other site using the Open Source Chef, so
we’re not sure if maybe this is a Private Chef vs. Open Source Chef
problem, or maybe a Chef 11.x vs. Chef 10.18.x problem, or what.

I’m also curious if people have looked at what version of
knife-essentials they’re using? Some other problems we’ve seen might be
related to using older versions of knife-essentials versus newer ones, so
this is a dimension to the issue that we’re also trying to track down.

If we can’t get this problem resolved quickly, we might be forced to give
up on Private Chef and step back to Open Source Chef.

Yes, we are working the internal channels at Opscode, but getting more
information from the community would also be very useful.


Brad Knowles <brad@shub-internet.orgmailto:brad@shub-internet.org>
LinkedIn Profile: http://tinyurl.com/y8kpxu


#5

Thanks Paul. Brad, what do you think of trying Paul’s workaround below? I actually like this better than my scheduled tasks approach. Both do essentially the same thing — use the current token’s ability to launch a process with arbitrary capabilities that surpass the current token’s. In this way, the restrictions placed on tokens in winrm sessions are meaningless from a security standpoint (though still useful for preventing accidental DOS or other harmful side effects of remote execution).

Regardless of OSC vs.Private, you’ll need a workaround like this to get winrm to do what you want here until we solve the ntlm / kerberos / sspi issues.

-Adam

From: Paul Morton - BIA <pmorton@biaprotect.commailto:pmorton@biaprotect.com>
Reply-To: "chef@lists.opscode.commailto:chef@lists.opscode.com" <chef@lists.opscode.commailto:chef@lists.opscode.com>
Date: Tuesday, June 11, 2013 7:10 AM
To: "chef@lists.opscode.commailto:chef@lists.opscode.com" <chef@lists.opscode.commailto:chef@lists.opscode.com>
Subject: [chef] Re: Re: Re: knife winrm browsing network shares

The root of this issue here is that access token created by winrm are marked “remote” by the server. This means that they cannot be used to access network resources. I am investigating adding credssp to the winrm library which would allow winrm to create non-remote tokens-but this is a ways out.

In the vagrant windows gem we use a powershell script to wrap the process in a new user session which has an access token that is not marked as remote.

https://github.com/WinRb/vagrant-windows/blob/master/lib/vagrant-windows/scripts/ps_runas.ps1.erb

On Jun 11, 2013, at 7:04 AM, “Adam Edwards” <adamed@opscode.commailto:adamed@opscode.com> wrote:

To narrow down the problem, it would be good to know the following:

  1. On what operating system / version are you running knife winrm?
  2. What OS version is the target OS?
  3. Is the target OS joined to a domain?
  4. What OS is running on the machine that is the target of the net use
    command (i.e. the “server” in “\server\share”)?
  5. Is the machine that is the target of the net use command in the same
    domain as the machine that is the target of the knife command?
  6. Did you set basic auth to true and disable encryption? I’m not sure I’d
    expect off-machine access to work in that case.

A short-term workaround is to use scheduled tasks – if you’re interested,
I can give you a specific example, though it would help to know what
you’re trying to accomplish with the net use (e.g. run a script, copy a
specific file, etc.).

Problems here are most likely due to the winrm gems used by knife winrm.
Those gems rely on libraries on the system running knife, which could be
Windows, Mac, Ubuntu, etc., and the presence of the right libraries and /
or bug fixes on those platforms varies. We are currently investigating
specific issues with NTLM for instance to see if unreleased versions of
the gems have the improvements needed to make the NTLM scenario work,
hopefully we’ll have some prescriptive statements about how to make this
scenario work.

A few more questions:

  1. Rather than use knife winrm to perform net use, is this operation
    something that you could do from within a Chef recipe that runs on the
    target system? That could free you from limitations of WinRM.
  2. You could probably use the recipe route via the RC version of
    mixlib-shellout, which supports running scripts with alternate
    credentials, or some upcoming changes to the Windows mount provider that
    will be merged to master this week.

Regarding private vs. open source chef, I don’t see anything here that
should make open source chef work differently than private chef. The
connectivity is really about winrm configuration on client and server, and
the winrm / ntlm gems and libraries on those systems, and finally knife
which ships with chef client and plugins like knife-windows that add the
winrm functionality to knife. The chef server code really is not involved
here.

Thanks.


Adam Edwards
Software Development Engineer, Opscode, Inc.

On 6/10/13 9:07 PM, “Brad Knowles” <brad@shub-internet.orgmailto:brad@shub-internet.org> wrote:

On Jun 6, 2013, at 3:00 PM, grepjuice@gmail.commailto:grepjuice@gmail.com wrote:

but get an “Access is denied” when doing dir \\filer\share

I’m curious – has anyone noticed what version of Chef they’re using with
regards to these problems? At least some of the tickets we’ve seen
appear to indicate that the problems with winrm are more prevalent with
Chef 11.x, whereas Chef 10.18.2 seems to work fine.

But in our case there’s also a difference between our using Private Chef
(based on Chef 11) versus the other site using the Open Source Chef, so
we’re not sure if maybe this is a Private Chef vs. Open Source Chef
problem, or maybe a Chef 11.x vs. Chef 10.18.x problem, or what.

I’m also curious if people have looked at what version of
knife-essentials they’re using? Some other problems we’ve seen might be
related to using older versions of knife-essentials versus newer ones, so
this is a dimension to the issue that we’re also trying to track down.

If we can’t get this problem resolved quickly, we might be forced to give
up on Private Chef and step back to Open Source Chef.

Yes, we are working the internal channels at Opscode, but getting more
information from the community would also be very useful.


Brad Knowles <brad@shub-internet.orgmailto:brad@shub-internet.org>
LinkedIn Profile: http://tinyurl.com/y8kpxu


#6

Hi,

The easiest way I have found is to re-login as part of the winrm
session. So execute the command

net use \filer\share mypwd

before you access the dir in the same session. We also had some issues
around the authentication code and ended up having to add a patch
that looks like code in [1]. (We use AD for authentication but all our
deploy tools run on Linux).

HTH

[1] https://gist.github.com/realityforge/5725935

On Fri, Jun 7, 2013 at 7:00 AM, grepjuice@gmail.com wrote:

Has anyone successfully used knife winrm for accessing remote shares?
I can successfully do a:
$ knife winrm -m server.mydomain.local -P ‘mypwd’ -p 5985 -x
’mydomain\Administrator’ dir

but get an “Access is denied” when doing dir \\filer\share

I have followed this doc and the knife winrm doc to setup the client
http://msdn.microsoft.com/en-us/library/windows/desktop/ee309365(v=vs.85).aspx
with no luck.

I see that the NTLM auth is successful on the windows host logs, but it seems
like it’s not using the CredSSP correctly. Toggling the CredSSP settings on the
windows server has no effect at all.

Any help is appreciated thanks!


Cheers,

Peter Donald


#7

Works! Thank you.

On Thu, Jun 6, 2013 at 7:44 PM, Peter Donald peter@realityforge.org wrote:

Hi,

The easiest way I have found is to re-login as part of the winrm
session. So execute the command

net use \filer\share mypwd

before you access the dir in the same session. We also had some issues
around the authentication code and ended up having to add a patch
that looks like code in [1]. (We use AD for authentication but all our
deploy tools run on Linux).

HTH

[1] https://gist.github.com/realityforge/5725935

On Fri, Jun 7, 2013 at 7:00 AM, grepjuice@gmail.com wrote:

Has anyone successfully used knife winrm for accessing remote shares?
I can successfully do a:
$ knife winrm -m server.mydomain.local -P ‘mypwd’ -p 5985 -x
’mydomain\Administrator’ dir

but get an “Access is denied” when doing dir \\filer\share

I have followed this doc and the knife winrm doc to setup the client

http://msdn.microsoft.com/en-us/library/windows/desktop/ee309365(v=vs.85).aspx

with no luck.

I see that the NTLM auth is successful on the windows host logs, but it
seems
like it’s not using the CredSSP correctly. Toggling the CredSSP settings
on the
windows server has no effect at all.

Any help is appreciated thanks!


Cheers,

Peter Donald


#8

Looks like I spoke too soon, this hack seems to be unreliable, sometimes it
works other times I get a
System error 67 has occurred.
The network name cannot be found.
Access is denied.
It seems to be at random,and I’ve seen anecdotal evidence that “net use” is
not reliable enough to use in batch files. Have you found a way to make
this reliable?

Thanks.

On Fri, Jun 7, 2013 at 10:05 AM, Grep Juice grepjuice@gmail.com wrote:

Works! Thank you.

On Thu, Jun 6, 2013 at 7:44 PM, Peter Donald peter@realityforge.orgwrote:

Hi,

The easiest way I have found is to re-login as part of the winrm
session. So execute the command

net use \filer\share mypwd

before you access the dir in the same session. We also had some issues
around the authentication code and ended up having to add a patch
that looks like code in [1]. (We use AD for authentication but all our
deploy tools run on Linux).

HTH

[1] https://gist.github.com/realityforge/5725935

On Fri, Jun 7, 2013 at 7:00 AM, grepjuice@gmail.com wrote:

Has anyone successfully used knife winrm for accessing remote shares?
I can successfully do a:
$ knife winrm -m server.mydomain.local -P ‘mypwd’ -p 5985 -x
’mydomain\Administrator’ dir

but get an “Access is denied” when doing dir \\filer\share

I have followed this doc and the knife winrm doc to setup the client

http://msdn.microsoft.com/en-us/library/windows/desktop/ee309365(v=vs.85).aspx

with no luck.

I see that the NTLM auth is successful on the windows host logs, but it
seems
like it’s not using the CredSSP correctly. Toggling the CredSSP
settings on the
windows server has no effect at all.

Any help is appreciated thanks!


Cheers,

Peter Donald


#9

Hi,

I haven’t seen this behaviour as such. We use it for about ~20 VMs
that run chef-client every 30 minutes and ~30 or so nightly batch
jobs. While some intermittently fail, we have yet to triage the
failures to net use although it is possible. We are just at the start
of our windows automation journey. So we haven’t noticed it above the
noise.

However, we have noticed that sometimes if we rdp into the server that
winrm will sometimes turn out to be a bit flakey - unsure of the cause
at this stage.

On Tue, Jun 11, 2013 at 1:55 AM, Grep Juice grepjuice@gmail.com wrote:

Looks like I spoke too soon, this hack seems to be unreliable, sometimes it
works other times I get a
System error 67 has occurred.
The network name cannot be found.
Access is denied.
It seems to be at random,and I’ve seen anecdotal evidence that “net use” is
not reliable enough to use in batch files. Have you found a way to make this
reliable?

Thanks.

On Fri, Jun 7, 2013 at 10:05 AM, Grep Juice grepjuice@gmail.com wrote:

Works! Thank you.

On Thu, Jun 6, 2013 at 7:44 PM, Peter Donald peter@realityforge.org
wrote:

Hi,

The easiest way I have found is to re-login as part of the winrm
session. So execute the command

net use \filer\share mypwd

before you access the dir in the same session. We also had some issues
around the authentication code and ended up having to add a patch
that looks like code in [1]. (We use AD for authentication but all our
deploy tools run on Linux).

HTH

[1] https://gist.github.com/realityforge/5725935

On Fri, Jun 7, 2013 at 7:00 AM, grepjuice@gmail.com wrote:

Has anyone successfully used knife winrm for accessing remote shares?
I can successfully do a:
$ knife winrm -m server.mydomain.local -P ‘mypwd’ -p 5985 -x
’mydomain\Administrator’ dir

but get an “Access is denied” when doing dir \\filer\share

I have followed this doc and the knife winrm doc to setup the client

http://msdn.microsoft.com/en-us/library/windows/desktop/ee309365(v=vs.85).aspx
with no luck.

I see that the NTLM auth is successful on the windows host logs, but it
seems
like it’s not using the CredSSP correctly. Toggling the CredSSP
settings on the
windows server has no effect at all.

Any help is appreciated thanks!


Cheers,

Peter Donald


Cheers,

Peter Donald


#10

Hi,

I had a bit of a poke around and found that our scripts often do a bit
more. Typically our “net use” commands are not persistent and at the
start of the script we do something like

net use * /delete /y

followed by

net use \filer\share mypwd /persistent:no

Also - one thing that we seem to need to do but I am unsure why is
that we can not refer to sub-directories in the net use command. We
tend to interact with the top level shares ala

net use \filer\share mypwd /persistent:no
dir \filer\share\subdir

Rather than

net use \filer\share\subdir mypwd /persistent:no
dir \filer\share\subdir

Also noticed that we interact directly with the smb server rather than
using DFS … unclear on the reasoning there

Hope that helps.

On Tue, Jun 11, 2013 at 11:09 AM, Peter Donald peter@realityforge.org wrote:

Hi,

I haven’t seen this behaviour as such. We use it for about ~20 VMs
that run chef-client every 30 minutes and ~30 or so nightly batch
jobs. While some intermittently fail, we have yet to triage the
failures to net use although it is possible. We are just at the start
of our windows automation journey. So we haven’t noticed it above the
noise.

However, we have noticed that sometimes if we rdp into the server that
winrm will sometimes turn out to be a bit flakey - unsure of the cause
at this stage.

On Tue, Jun 11, 2013 at 1:55 AM, Grep Juice grepjuice@gmail.com wrote:

Looks like I spoke too soon, this hack seems to be unreliable, sometimes it
works other times I get a
System error 67 has occurred.
The network name cannot be found.
Access is denied.
It seems to be at random,and I’ve seen anecdotal evidence that “net use” is
not reliable enough to use in batch files. Have you found a way to make this
reliable?

Thanks.

On Fri, Jun 7, 2013 at 10:05 AM, Grep Juice grepjuice@gmail.com wrote:

Works! Thank you.

On Thu, Jun 6, 2013 at 7:44 PM, Peter Donald peter@realityforge.org
wrote:

Hi,

The easiest way I have found is to re-login as part of the winrm
session. So execute the command

net use \filer\share mypwd

before you access the dir in the same session. We also had some issues
around the authentication code and ended up having to add a patch
that looks like code in [1]. (We use AD for authentication but all our
deploy tools run on Linux).

HTH

[1] https://gist.github.com/realityforge/5725935

On Fri, Jun 7, 2013 at 7:00 AM, grepjuice@gmail.com wrote:

Has anyone successfully used knife winrm for accessing remote shares?
I can successfully do a:
$ knife winrm -m server.mydomain.local -P ‘mypwd’ -p 5985 -x
’mydomain\Administrator’ dir

but get an “Access is denied” when doing dir \\filer\share

I have followed this doc and the knife winrm doc to setup the client

http://msdn.microsoft.com/en-us/library/windows/desktop/ee309365(v=vs.85).aspx
with no luck.

I see that the NTLM auth is successful on the windows host logs, but it
seems
like it’s not using the CredSSP correctly. Toggling the CredSSP
settings on the
windows server has no effect at all.

Any help is appreciated thanks!


Cheers,

Peter Donald


Cheers,

Peter Donald


Cheers,

Peter Donald