Knife-vrealize always fails, seems to be in wrong path

Chef Infra (archive)
1

I posted this on github/chef, got some help but still having the issue.

When i have knife-vrealize installed, almost any call to knife fails. Uninstalling knife-vrealize fixes the issue, but then i can't spin machines from the cli.

FWIW, kitchen-vra works fine, I can still create and converge vms with Kitchen.

Fatal errors:

~\Documents\github> knife vra catalog list --entitled
Traceback (most recent call last):
        16: from C:/opscode/chef-workstation/bin/knife:370:in `<main>'
        15: from C:/opscode/chef-workstation/bin/knife:370:in `load'
        14: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/bin/knife:24:in `<top (required)>'
        13: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/application/knife.rb:163:in `run'
        12: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/knife.rb:224:in `run'
        11: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/knife.rb:161:in `subcommand_class_from'
        10: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/knife/core/subcommand_loader.rb:129:in `command_class_from'
         9: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/knife/core/subcommand_loader.rb:115:in `load_command'
         8: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/knife/core/subcommand_loader.rb:105:in `load_commands'
         7: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/knife/core/subcommand_loader.rb:105:in `each'
         6: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/knife/core/subcommand_loader.rb:105:in `block in load_commands'
         5: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/knife/core/subcommand_loader.rb:105:in `load'
         4: from C:/Users/mcascone1/AppData/Local/chefdk/gem/ruby/2.7.0/gems/knife-vrealize-6.0.0/lib/chef/knife/vra_server_create.rb:26:in `<top (required)>'
         3: from C:/Users/mcascone1/AppData/Local/chefdk/gem/ruby/2.7.0/gems/knife-vrealize-6.0.0/lib/chef/knife/vra_server_create.rb:27:in `<class:Chef>'
         2: from C:/Users/mcascone1/AppData/Local/chefdk/gem/ruby/2.7.0/gems/knife-vrealize-6.0.0/lib/chef/knife/vra_server_create.rb:28:in `<class:Knife>'
         1: from C:/Users/mcascone1/AppData/Local/chefdk/gem/ruby/2.7.0/gems/knife-vrealize-6.0.0/lib/chef/knife/vra_server_create.rb:29:in `<class:Cloud>'
C:/Users/mcascone1/AppData/Local/chefdk/gem/ruby/2.7.0/gems/knife-vrealize-6.0.0/lib/chef/knife/vra_server_create.rb:30:in

Even non-vra commands fail:

> knife cookbook list
Traceback (most recent call last):
        16: from C:/opscode/chef-workstation/bin/knife:370:in `<main>'
        15: from C:/opscode/chef-workstation/bin/knife:370:in `load'
        14: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/bin/knife:24:in `<top (required)>'
        13: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/application/knife.rb:163:in `run'
        12: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/knife.rb:224:in `run'
        11: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/knife.rb:161:in `subcommand_class_from'
        10: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/knife/core/subcommand_loader.rb:129:in `command_class_from'
         9: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/knife/core/subcommand_loader.rb:115:in `load_command'
         8: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/knife/core/subcommand_loader.rb:105:in `load_commands'
         7: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/knife/core/subcommand_loader.rb:105:in `each'
         6: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/knife/core/subcommand_loader.rb:105:in `block in load_commands'
         5: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/knife/core/subcommand_loader.rb:105:in `load'
         4: from C:/Users/mcascone1/AppData/Local/chefdk/gem/ruby/2.7.0/gems/knife-vrealize-6.0.0/lib/chef/knife/vra_server_create.rb:26:in `<top (required)>'
         3: from C:/Users/mcascone1/AppData/Local/chefdk/gem/ruby/2.7.0/gems/knife-vrealize-6.0.0/lib/chef/knife/vra_server_create.rb:27:in `<class:Chef>'
         2: from C:/Users/mcascone1/AppData/Local/chefdk/gem/ruby/2.7.0/gems/knife-vrealize-6.0.0/lib/chef/knife/vra_server_create.rb:28:in `<class:Knife>'
         1: from C:/Users/mcascone1/AppData/Local/chefdk/gem/ruby/2.7.0/gems/knife-vrealize-6.0.0/lib/chef/knife/vra_server_create.rb:29:in `<class:Cloud>'
C:/Users/mcascone1/AppData/Local/chefdk/gem/ruby/2.7.0/gems/knife-vrealize-6.0.0/lib/chef/knife/vra_server_create.rb:30:in `<class:VraServerCreate>': uninitialized constant Chef::Knife::Cloud::VraServerCreate::VraServiceHelpers (NameError)
Did you mean?  Chef::Knife::Cloud::VraServiceOptions

When I chef gem uninstall knife-vrealize , knife commands work again.

One thing i notice is that the gem itself appears to be coming from C:/Users/mcascone1/AppData/Local/chefdk/gem/ruby/2.7.0/gems/

On the github issue, it was suggested to update my GEM_PATH & GEM_HOME to C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.

After setting those vars and installing the gem, i still get the same errors. And it looks like it was still installed to the same place. I see the PATH warning there, but don't we not want the gems installed in our home dir? I am not as knowledgeable as I'd like to be in this area so please excuse if i am doing something dumb.

> $env:gem_path
C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0
> $env:gem_home
C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0
> chef gem install knife-vrealize
WARNING:  You don't have c:\users\mcascone1\appdata\local\chefdk\gem\ruby\2.7.0\bin in your PATH,
          gem executables will not run.
Successfully installed knife-vrealize-6.0.0
Parsing documentation for knife-vrealize-6.0.0
Installing ri documentation for knife-vrealize-6.0.0
Done installing documentation for knife-vrealize after 2 seconds
1 gem installed

> knife vra catalog list --entitled
Traceback (most recent call last):
        16: from C:/opscode/chef-workstation/bin/knife:370:in `<main>'
        15: from C:/opscode/chef-workstation/bin/knife:370:in `load'
        14: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/bin/knife:24:in `<top (required)>'
        13: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/application/knife.rb:163:in `run'
        12: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/knife.rb:224:in `run'
        11: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/knife.rb:161:in `subcommand_class_from'
        10: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/knife/core/subcommand_loader.rb:129:in `command_class_from'
         9: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/knife/core/subcommand_loader.rb:115:in `load_command'
         8: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/knife/core/subcommand_loader.rb:105:in `load_commands'
         7: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/knife/core/subcommand_loader.rb:105:in `each'
         6: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/knife/core/subcommand_loader.rb:105:in `block in load_commands'
         5: from C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0/gems/chef-16.4.41-universal-mingw32/lib/chef/knife/core/subcommand_loader.rb:105:in `load'
         4: from C:/Users/mcascone1/AppData/Local/chefdk/gem/ruby/2.7.0/gems/knife-vrealize-6.0.0/lib/chef/knife/vra_server_create.rb:26:in `<top (required)>'
         3: from C:/Users/mcascone1/AppData/Local/chefdk/gem/ruby/2.7.0/gems/knife-vrealize-6.0.0/lib/chef/knife/vra_server_create.rb:27:in `<class:Chef>'
         2: from C:/Users/mcascone1/AppData/Local/chefdk/gem/ruby/2.7.0/gems/knife-vrealize-6.0.0/lib/chef/knife/vra_server_create.rb:28:in `<class:Knife>'
         1: from C:/Users/mcascone1/AppData/Local/chefdk/gem/ruby/2.7.0/gems/knife-vrealize-6.0.0/lib/chef/knife/vra_server_create.rb:29:in `<class:Cloud>'
C:/Users/mcascone1/AppData/Local/chefdk/gem/ruby/2.7.0/gems/knife-vrealize-6.0.0/lib/chef/knife/vra_server_create.rb:30:in `<class:VraServerCreate>': uninitialized constant Chef::Knife::Cloud::VraServerCreate::VraServiceHelpers (NameError)
Did you mean?  Chef::Knife::Cloud::VraServiceOptions
~\Documents\github\Pipeline-ServicePasswordChange [develop ≡ +0 ~1 -0 !]
> $env:gem_path
C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0
~\Documents\github\Pipeline-ServicePasswordChange [develop ≡ +0 ~1 -0 !]
> $env:gem_root
C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0
~\Documents\github\Pipeline-ServicePasswordChange [develop ≡ +0 ~1 -0 !]
> $env:gem_home
C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0

Theres a bit more discussion on github, and i finally end here:

now i really don't understand.

I reset my path, set $env:gem_home and $env:gem_path to $env:gem_root which is C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0 .

I even renamed C:\Users\mcascone1\AppData\Local\chefdk to ...\chefdk-backup .

I put the gem in my gemfile and ran chef exec bundle install .

It still installs into my chefdk folder! It creates a new one and installs to it!

> chef exec bundle info knife-vrealize
  * knife-vrealize (6.0.0)
        Summary: Chef Infra Knife plugin to interact with VMware vRealize.
        Homepage: https://github.com/chef/knife-vrealize
        Path: C:/Users/mcascone1/AppData/Local/chefdk/gem/ruby/2.7.0/gems/knife-vrealize-6.0.0

> path
C:\opscode\chef-workstation\bin\
C:\Program Files\Microsoft VS Code\bin
C:\ProgramData\chocolatey\bin
C:\Program Files\PowerShell\7\
C:\Program Files\Git\cmd
C:\Program Files (x86)\Common Files\Oracle\Java\javapath
C:\windows\system32
C:\windows
C:\windows\System32\Wbem
C:\windows\System32\WindowsPowerShell\v1.0\
C:\windows\System32\OpenSSH\
C:\Program Files\Intel\WiFi\bin\
C:\Program Files\Common Files\Intel\WirelessCommon\
C:\Program Files\Docker\Docker\resources\bin
C:\ProgramData\DockerDesktop\version-bin
C:\HashiCorp\Vagrant\bin
C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Current\Bin\
C:\Program Files (x86)\Microsoft SQL Server\150\DTS\Binn\
C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\130\Tools\Binn\
C:\Program Files (x86)\Microsoft SQL Server\130\Tools\Binn\
C:\Program Files\Microsoft SQL Server\130\Tools\Binn\
C:\Program Files\Microsoft SQL Server\130\DTS\Binn\
C:\Program Files (x86)\Google\Chrome\Application
C:\Program Files\nodejs\
C:\Program Files\dotnet\
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\Common7\IDE\CommonExtensions\Microsoft\TestWindow

C:\Users\mcascone1\AppData\Local\Microsoft\WindowsApps
C:\Users\mcascone1\AppData\Roaming\npm
C:\Users\mcascone1\.dotnet\tools

> $env:gem_home
C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0

> $env:gem_path
C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0

And i get the same ol' error when running knife vra .

> knife vra catalog list --entitled
Traceback (most recent call last):
        16: from C:/opscode/chef-workstation/bin/knife:370:in `<main>'
        15: from C:/opscode/chef-workstation/bin/knife:370:in `load'
       ...
C:/Users/mcascone1/AppData/Local/chefdk/gem/ruby/2.7.0/gems/knife-vrealize-6.0.0/lib/chef/knife/vra_server_create.rb:30:in `<class:VraServerCreate>': uninitialized constant Chef::Knife::Cloud::VraServerCreate::VraServiceHelpers (NameError)
Did you mean?  Chef::Knife::Cloud::VraServiceOptions

Where is that path configured?

I'm at a loss. I don't have chefdk installed any more. I've uninstalled/re-installed chef-workstation.

What else to try?

2

Based on this github issue, i got some more insight but i still don't know how to fix it.

prior to running chef shell-init powershell | iex, I don't have $env:gem_home/path set. After running that, they are set to the "bad" path:

> $env:gem_home
> chef shell-init powershell | iex
> $env:gem_home
C:/Users/mcascone1/AppData/Local/chefdk/gem/ruby/2.7.0
> $env:gem_path
C:/Users/mcascone1/AppData/Local/chefdk/gem/ruby/2.7.0;C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0

But $env:gem_root is still set to the 'good' path:

> $env:gem_root
C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0

The 'bad' path is visible in my Ruby env vars:

> chef gem env | select-string -Context 2 chefdk

    - RUBY VERSION: 2.7.1 (2020-03-31 patchlevel 83) [x64-mingw32]
    - INSTALLATION DIRECTORY: C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0
>   - USER INSTALLATION DIRECTORY: C:/Users/mcascone1/AppData/Local/chefdk/gem/ruby/2.7.0
    - RUBY EXECUTABLE: C:/opscode/chef-workstation/embedded/bin/ruby.exe
    - GIT EXECUTABLE: C:/opscode/chef-workstation/embedded/bin/git.BAT
    - GEM PATHS:
       - C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0
>      - C:/Users/mcascone1/AppData/Local/chefdk/gem/ruby/2.7.0
    - GEM CONFIGURATION:
       - :update_sources => true
    - SHELL PATH:
       - C:/opscode/chef-workstation/bin
>      - C:/Users/mcascone1/AppData/Local/chefdk/gem/ruby/2.7.0/bin
       - C:/opscode/chef-workstation/embedded/bin
       - C:\Program Files\PowerShell\7

I tried setting all the $env:gem_x paths in my windows env vars, and opening a new shell, and those chefdk paths just won't go away.

I also found this - just the output of chef shell-init powershell without piping it to Invoke-Expression:

> chef shell-init powershell
$env:PATH="C:/opscode/chef-workstation/bin;C:/Users/mcascone1/AppData/Local/chefdk/gem/ruby/2.7.0/bin;C:/opscode/chef-workstation/embedded/bin;C:\Program Files\PowerShell\7;C:\Program Files\ConEmu\ConEmu\Scripts;C:\Program Files\ConEmu;C:\Program Files\ConEmu\ConEmu;C:\opscode\chef-workstation\bin\;C:\Program Files\Microsoft VS Code\bin;C:\ProgramData\chocolatey\bin;C:\Program Files\PowerShell\7\;C:\Program Files (x86)\Common Files\Oracle\Java\javapath;C:\windows\system32;C:\windows;C:\windows\System32\Wbem;C:\windows\System32\WindowsPowerShell\v1.0\;C:\windows\System32\OpenSSH\;C:\Program Files\Intel\WiFi\bin\;C:\Program Files\Common Files\Intel\WirelessCommon\;C:\Program Files\Docker\Docker\resources\bin;C:\ProgramData\DockerDesktop\version-bin;C:\HashiCorp\Vagrant\bin;C:\Program Files (x86)\Microsoft Visual Studio\2019\BuildTools\MSBuild\Current\Bin\;C:\Program Files (x86)\Microsoft SQL Server\150\DTS\Binn\;C:\Program Files\Microsoft SQL Server\Client SDK\ODBC\130\Tools\Binn\;C:\Program Files (x86)\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;C:\Program Files\Microsoft SQL Server\130\DTS\Binn\;C:\Program Files (x86)\Google\Chrome\Application;C:\Program Files\nodejs\;C:\Program Files\dotnet\;C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\Common7\IDE\CommonExtensions\Microsoft\TestWindow;C:\Program Files\Git\cmd;C:\Users\mcascone1\AppData\Local\Microsoft\WindowsApps;C:\Users\mcascone1\AppData\Roaming\npm;C:\Users\mcascone1\.dotnet\tools;C:/opscode/chef-workstation/embedded/git/usr/bin"
$env:GEM_ROOT="C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0"
$env:GEM_HOME="C:/Users/mcascone1/AppData/Local/chefdk/gem/ruby/2.7.0"
$env:GEM_PATH="C:/Users/mcascone1/AppData/Local/chefdk/gem/ruby/2.7.0;C:/opscode/chef-workstation/embedded/lib/ruby/gems/2.7.0"

Even when i edit the output of chef-shell init powershell to remove all of the references to chefdk, when i pipe that content into Invoke-Expression, i still get the chefdk path references in chef gem env as if i hadn't edited anything.

How can I get this chefdk path out of here, permanently?