Need help with Kitchen + EC2 + Windows

Chef Infra (archive)
1

Hello,

Tried to get Kitchen working with EC2 Windows instances all day yesterday and am close but just can't get it to converge, and I'm sure it's something simple, but I'm burnt out at this point so asking for help.

I'm using a really basic cookbook that just creates a local file on C:. It creates the EC2 instance, installs the Chef client, then it says it can't find the cookbook.

Here's my kitchen.yml:

driver:
name: ec2
aws_ssh_key_id: ssh_key
security_group_ids: ["sg-1", "sg-2", "sg-3"]
region: us-east-1
availability_zone: a
require_chef_omnibus: true
subnet_id: subnet-blah
iam_profile_name: bi-ops-windows-base-role
instance_type: t2.small
associate_public_ip: false
interface: private
tags:
'Name': 'test-kitchen-instance'
user_data: C:...path_to_userdata...\userdata.txt

provisioner:
name: chef_zero
always_update_cookbooks: true

verifier:
name: inspec

platforms:

  • name: windows-2012r2
    driver:
    image_id: ami-f6529b8c #plain jane AWS AMI
    transport:
    name: winrm
    ssh_key: C:...path_to_ssh_key...\ssh_key.pem
    username: administrator
    port: 5985

suites:

  • name: default
    run_list:
    • recipe[bababooey::default]
      verifier:
      inspec_tests:
      • test/smoke/default
        attributes:

Here's the output:
stractenberg@LFA-STRA MINGW64 ~/chef-repo/cookbooks/bababooey (master)
$ kitchen converge
-----> Starting Kitchen (v1.17.0)
-----> Creating ...
Detected platform: windows version 2012r2rtm on x86_64. Instance Type: t2.small. Default username: administrator (default).
If you are not using an account that qualifies under the AWS
free-tier, you may be charged to run these suites. The charge
should be minimal, but neither Test Kitchen nor its maintainers
are responsible for your incurred costs.

   Instance <i-05d95e4c5e09c742e> requested.
   Polling AWS for existence, attempt 0...
   Attempting to tag the instance, 0 retries
   EC2 instance <i-05d95e4c5e09c742e> created.
   Waited 0/300s for instance <i-05d95e4c5e09c742e> volumes to be ready.
   Waited 0/300s for instance <i-05d95e4c5e09c742e> to become ready.
   Waited 5/300s for instance <i-05d95e4c5e09c742e> to become ready.
   Waited 10/300s for instance <i-05d95e4c5e09c742e> to become ready.
   Waited 15/300s for instance <i-05d95e4c5e09c742e> to become ready.
   Waited 20/300s for instance <i-05d95e4c5e09c742e> to become ready.
   Waited 25/300s for instance <i-05d95e4c5e09c742e> to become ready.
   Waited 30/300s for instance <i-05d95e4c5e09c742e> to become ready.
   Waited 35/300s for instance <i-05d95e4c5e09c742e> to become ready.
   Waited 40/300s for instance <i-05d95e4c5e09c742e> to become ready.
   Waited 45/300s for instance <i-05d95e4c5e09c742e> to become ready.
   Waited 50/300s for instance <i-05d95e4c5e09c742e> to become ready.
   Waited 55/300s for instance <i-05d95e4c5e09c742e> to become ready.
   Waited 60/300s for instance <i-05d95e4c5e09c742e> to become ready.
   Waited 65/300s for instance <i-05d95e4c5e09c742e> to become ready.
   Waited 70/300s for instance <i-05d95e4c5e09c742e> to become ready.
   Waited 0/300s for instance <i-05d95e4c5e09c742e> to fetch windows admin password.
   Retrieved Windows password for instance <i-05d95e4c5e09c742e>.
   EC2 instance <i-05d95e4c5e09c742e> ready.
   [WinRM] Established



       Directory: C:\chef\ohai\hints


   Mode                LastWriteTime     Length Name
   ----                -------------     ------ ----
   -a---        10/26/2017   1:10 PM          0 ec2.json


   Finished creating <default-windows-2012r2> (1m46.00s).

-----> Converging ...
Preparing files for transfer
Preparing dna.json
Resolving cookbook dependencies with Berkshelf 6.3.1...
Removing non-cookbook files before transfer
Preparing validation.pem
Preparing client.rb
-----> Installing Chef Omnibus (install only if missing)
Downloading package from https://packages.chef.io/files/stable/chef/13.5.3/windows/2016/chef-client-13.5.3-1-x64.msi
Download complete.
Successfully verified C:\Users\ADMINI~1\AppData\Local\Temp\chef-client-13.5.3-1-x64.msi
Installing Chef Omnibus package C:\Users\ADMINI~1\AppData\Local\Temp\chef-client-13.5.3-1-x64.msi
Installation complete
Transferring files to
Starting Chef Client, version 13.5.3
Creating a new client identity for default-windows-2012r2 using the validator key.
resolving cookbooks for run list: ["bababooey::default"]

   ================================================================================
   Error Resolving Cookbooks for Run List:
   ================================================================================

   Missing Cookbooks:
   ------------------
   No such cookbook: bababooey

   Expanded Run List:
   ------------------
   * bababooey::default

   System Info:
   ------------
   chef_version=13.5.3
   platform=windows
   platform_version=6.3.9600
   ruby=ruby 2.4.2p198 (2017-09-14 revision 59899) [x64-mingw32]
   program_name=C:/opscode/chef/bin/chef-client
   executable=C:/opscode/chef/bin/chef-client


   Running handlers:
   [2017-10-26T13:11:16+00:00] ERROR: Running exception handlers
   Running handlers complete
   [2017-10-26T13:11:16+00:00] ERROR: Exception handlers complete
   Chef Client failed. 0 resources updated in 04 seconds
   [2017-10-26T13:11:16+00:00] FATAL: Stacktrace dumped to C:/Users/ADMINI~1/AppData/Local/Temp/kitchen/cache/chef-stacktrace.out
   [2017-10-26T13:11:16+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
   [2017-10-26T13:11:16+00:00] FATAL: Net::HTTPServerException: 412 "Precondition Failed"

------Exception-------
Class: Kitchen::ActionFailed
Message: 1 actions failed.
Converge failed on instance . Please see .kitchen/logs/default-windows-2012r2.log for more details

Please see .kitchen/logs/kitchen.log for more details
Also try running kitchen diagnose --all for configuration

Here's the kitchen.log
I, [2017-10-26T09:08:17.258564 #2440] INFO -- Kitchen: -----> Starting Kitchen (v1.17.0)
I, [2017-10-26T09:08:24.115435 #2440] INFO -- Kitchen: -----> Creating ...
I, [2017-10-26T09:10:10.117632 #2440] INFO -- Kitchen: -----> Converging ...
E, [2017-10-26T09:11:17.677141 #2440] ERROR -- Kitchen: ------Exception-------
E, [2017-10-26T09:11:17.677141 #2440] ERROR -- Kitchen: Class: Kitchen::ActionFailed
E, [2017-10-26T09:11:17.677141 #2440] ERROR -- Kitchen: Message: 1 actions failed.

Converge failed on instance <default-windows-2012r2>.  Please see .kitchen/logs/default-windows-2012r2.log for more details

E, [2017-10-26T09:11:17.677141 #2440] ERROR -- Kitchen: ----------------------
E, [2017-10-26T09:11:17.677141 #2440] ERROR -- Kitchen: ------Backtrace-------
E, [2017-10-26T09:11:17.677641 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/command.rb:183:in report_errors' E, [2017-10-26T09:11:17.677641 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/command.rb:174:in run_action'
E, [2017-10-26T09:11:17.677641 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/command/action.rb:36:in block in call' E, [2017-10-26T09:11:17.677641 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/2.4.0/benchmark.rb:293:in measure'
E, [2017-10-26T09:11:17.677641 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/command/action.rb:34:in call' E, [2017-10-26T09:11:17.677641 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/cli.rb:53:in perform'
E, [2017-10-26T09:11:17.678141 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/cli.rb:187:in block (2 levels) in <class:CLI>' E, [2017-10-26T09:11:17.678141 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/thor-0.19.1/lib/thor/command.rb:27:in run'
E, [2017-10-26T09:11:17.678141 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/thor-0.19.1/lib/thor/invocation.rb:126:in invoke_command' E, [2017-10-26T09:11:17.678141 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/cli.rb:334:in invoke_task'
E, [2017-10-26T09:11:17.678141 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/thor-0.19.1/lib/thor.rb:359:in dispatch' E, [2017-10-26T09:11:17.678141 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/thor-0.19.1/lib/thor/base.rb:440:in start'
E, [2017-10-26T09:11:17.678141 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/bin/kitchen:13:in block in <top (required)>' E, [2017-10-26T09:11:17.678641 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/errors.rb:171:in with_friendly_errors'
E, [2017-10-26T09:11:17.678641 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/bin/kitchen:13:in <top (required)>' E, [2017-10-26T09:11:17.678641 #2440] ERROR -- Kitchen: C:/opscode/chefdk/bin/kitchen:279:in load'
E, [2017-10-26T09:11:17.678641 #2440] ERROR -- Kitchen: C:/opscode/chefdk/bin/kitchen:279:in <main>' E, [2017-10-26T09:11:17.678641 #2440] ERROR -- Kitchen: ----End Backtrace----- E, [2017-10-26T09:11:17.678641 #2440] ERROR -- Kitchen: -Composite Exception-- E, [2017-10-26T09:11:17.678641 #2440] ERROR -- Kitchen: Class: Kitchen::InstanceFailure E, [2017-10-26T09:11:17.678641 #2440] ERROR -- Kitchen: Message: Converge failed on instance <default-windows-2012r2>. Please see .kitchen/logs/default-windows-2012r2.log for more details E, [2017-10-26T09:11:17.679141 #2440] ERROR -- Kitchen: ---------------------- E, [2017-10-26T09:11:17.679141 #2440] ERROR -- Kitchen: ------Backtrace------- E, [2017-10-26T09:11:17.679141 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/provisioner/base.rb:83:in rescue in call'
E, [2017-10-26T09:11:17.679141 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/provisioner/base.rb:86:in call' E, [2017-10-26T09:11:17.679141 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/instance.rb:384:in block in converge_action'
E, [2017-10-26T09:11:17.679141 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/instance.rb:527:in synchronize_or_call' E, [2017-10-26T09:11:17.679641 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/instance.rb:489:in block in action'
E, [2017-10-26T09:11:17.679641 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/2.4.0/benchmark.rb:293:in measure' E, [2017-10-26T09:11:17.679641 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/instance.rb:488:in action'
E, [2017-10-26T09:11:17.679641 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/instance.rb:380:in converge_action' E, [2017-10-26T09:11:17.679641 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/instance.rb:359:in block in transition_to'
E, [2017-10-26T09:11:17.679641 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/instance.rb:358:in each' E, [2017-10-26T09:11:17.679641 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/instance.rb:358:in transition_to'
E, [2017-10-26T09:11:17.680141 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/instance.rb:135:in converge' E, [2017-10-26T09:11:17.680141 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/command.rb:197:in public_send'
E, [2017-10-26T09:11:17.680141 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/command.rb:197:in run_action_in_thread' E, [2017-10-26T09:11:17.680141 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/command.rb:169:in block (2 levels) in run_action'
E, [2017-10-26T09:11:17.680141 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/logging-2.2.2/lib/logging/diagnostic_context.rb:474:in `block in create_with_logging_context'
E, [2017-10-26T09:11:17.680141 #2440] ERROR -- Kitchen: ----End Backtrace-----
E, [2017-10-26T09:11:17.680141 #2440] ERROR -- Kitchen: ---Nested Exception---
E, [2017-10-26T09:11:17.680141 #2440] ERROR -- Kitchen: Class: Kitchen::ActionFailed
E, [2017-10-26T09:11:17.680642 #2440] ERROR -- Kitchen: Message: WinRM exited (1) for command: [$env:TEST_KITCHEN = "1"
$env:PATH = try {
[System.Environment]::GetEnvironmentVariable('PATH','Machine')
} catch { $env:PATH }

& $env:systemdrive\opscode\chef\bin\chef-client.bat --local-mode --config $env:TEMP\kitchen\client.rb --log_level auto --force-formatter --no-color --json-attributes $env:TEMP\kitchen\dna.json --chef-zero-port 8889 ; exit $LastExitCode]
E, [2017-10-26T09:11:17.680642 #2440] ERROR -- Kitchen: ----------------------
E, [2017-10-26T09:11:17.680642 #2440] ERROR -- Kitchen: ------Backtrace-------
E, [2017-10-26T09:11:17.680642 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/provisioner/base.rb:83:in rescue in call' E, [2017-10-26T09:11:17.680642 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/provisioner/base.rb:86:in call'
E, [2017-10-26T09:11:17.680642 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/instance.rb:384:in block in converge_action' E, [2017-10-26T09:11:17.680642 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/instance.rb:527:in synchronize_or_call'
E, [2017-10-26T09:11:17.681142 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/instance.rb:489:in block in action' E, [2017-10-26T09:11:17.681142 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/2.4.0/benchmark.rb:293:in measure'
E, [2017-10-26T09:11:17.681142 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/instance.rb:488:in action' E, [2017-10-26T09:11:17.681142 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/instance.rb:380:in converge_action'
E, [2017-10-26T09:11:17.681142 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/instance.rb:359:in block in transition_to' E, [2017-10-26T09:11:17.681142 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/instance.rb:358:in each'
E, [2017-10-26T09:11:17.681142 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/instance.rb:358:in transition_to' E, [2017-10-26T09:11:17.681142 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/instance.rb:135:in converge'
E, [2017-10-26T09:11:17.681642 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/command.rb:197:in public_send' E, [2017-10-26T09:11:17.681642 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/command.rb:197:in run_action_in_thread'
E, [2017-10-26T09:11:17.681642 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/test-kitchen-1.17.0/lib/kitchen/command.rb:169:in block (2 levels) in run_action' E, [2017-10-26T09:11:17.681642 #2440] ERROR -- Kitchen: C:/opscode/chefdk/embedded/lib/ruby/gems/2.4.0/gems/logging-2.2.2/lib/logging/diagnostic_context.rb:474:in block in create_with_logging_context'
E, [2017-10-26T09:11:17.681642 #2440] ERROR -- Kitchen: ----End Backtrace-----

Thanks in advance. New to Kitchen that isn't local.

2

The remote test node has a kitchen folder here: C:\Users\Administrator\AppData\Local\Temp\kitchen\cookbooks

but there is nothing in the cookbooks folder.

Client.rb shows this:

node_name "default-windows-2012r2"
checksum_path "#{ENV[‘TEMP’]}\kitchen\checksums"
file_cache_path “#{ENV[‘TEMP’]}\kitchen\cache"
file_backup_path “#{ENV[‘TEMP’]}\kitchen\backup"
cookbook_path [”#{ENV[‘TEMP’]}\kitchen\cookbooks”, “#{ENV[‘TEMP’]}\kitchen\site-cookbooks”]
data_bag_path "#{ENV[‘TEMP’]}\kitchen\data_bags"
environment_path "#{ENV[‘TEMP’]}\kitchen\environments"
node_path "#{ENV[‘TEMP’]}\kitchen\nodes"
role_path "#{ENV[‘TEMP’]}\kitchen\roles"
client_path "#{ENV[‘TEMP’]}\kitchen\clients"
user_path "#{ENV[‘TEMP’]}\kitchen\users"
validation_key "#{ENV[‘TEMP’]}\kitchen\validation.pem"
client_key "#{ENV[‘TEMP’]}\kitchen\client.pem"
chef_server_url "http://127.0.0.1:8889"
encrypted_data_bag_secret "#{ENV[‘TEMP’]}\kitchen\encrypted_data_bag_secret"
treat_deprecation_warnings_as_errors false
named_run_list {}

dna.json contents:

{“run_list”:[“recipe[bababooey::default]”]}

Also, don’t know if this matters but it mentions –chef-zero-port 8889

& $env:systemdrive\opscode\chef\bin\chef-client.bat --local-mode --config $env:TEMP\kitchen\client.rb --log_level auto --force-formatter --no-color --json-attributes $env:TEMP\kitchen\dna.json --chef-zero-port 8889 ; exit $LastExitCode)
D [WinRM] Command created for $env:TEST_KITCHEN = “1”
$env:PATH = try {
[System.Environment]::GetEnvironmentVariable(‘PATH’,‘Machine’)
} catch { $env:PATH }

8889 is not open or listening. Windows firewall is disabled and this EC2 instance is in a security group that allows all traffic from the system I’m running kitchen on.