Chef Push: Invalid Key Errror


#1

Hey,

We currently have an issue with Chef Push when trying to run a job. When running knife job start … we get the following error:

error received ERROR: The data in your request was invalid Response: invalid key supplied -VV stack trace in a moment -bash-4.2$ knife job start chef-client --search 'role:jenkins-slave' -c $HOME/.chef/knife.rb -VV INFO: Using configuration from /mnt/resource/jenkins/.chef/knife.rb DEBUG: Chef::HTTP calling Chef::HTTP::JSONInput#handle_request DEBUG: Chef::HTTP calling Chef::HTTP::JSONToModelOutput#handle_request DEBUG: Chef::HTTP calling Chef::HTTP::CookieManager#handle_request DEBUG: Chef::HTTP calling Chef::HTTP::Decompressor#handle_request DEBUG: Chef::HTTP calling Chef::HTTP::Authenticator#handle_request DEBUG: Signing the request as jenkins DEBUG: Chef::HTTP calling Chef::HTTP::RemoteRequestID#handle_request DEBUG: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_request DEBUG: Initiating GET to https://OUR.CHEF.SERVER.COM/organizations/our-organization/search/node?q=role%253Adjenkins-slave&sort=X_CHEF_id_CHEF_X%20asc&start=0 DEBUG: ---- HTTP Request Header Data: ---- DEBUG: Accept: application/json DEBUG: Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3 DEBUG: X-OPS-SIGN: algorithm=sha1;version=1.0; DEBUG: X-OPS-USERID: jenkins DEBUG: X-OPS-TIMESTAMP: 2016-04-12T12:19:44Z DEBUG: X-OPS-CONTENT-HASH: 2jmj7l5rSw0yVb/vlWAYkK/YBwk= DEBUG: X-OPS-AUTHORIZATION-1: JNhkIKf54C9aqWP5TYKbnQ0aUOkTgDkpX+TU2QnYVW2JGp4nd2cAerhLFApN DEBUG: X-OPS-AUTHORIZATION-2: Lx+shlpzWjloYVyXZTnYsFMlxjXDhOOQ1OuA3/uZ8LDMy2+RB68HM+CSyh3E DEBUG: X-OPS-AUTHORIZATION-3: kPwNNZMTx2oNPSK/Rxv8tJXtMpekKE+KAU3tATaKXIPxiN9P0S7XkyuBr5Sm DEBUG: X-OPS-AUTHORIZATION-4: jSvk5wRjOAB2VHBSaROj2APBP8BxN7i+74zPrA4m7uW6nZJkCgZCXmUMRTj8 DEBUG: X-OPS-AUTHORIZATION-5: IMowmyJ/ip4z+4eCOEYdzDOOwHdqGR/HdCt7O8+uZkdGNRAmi0HHKyJTlHED DEBUG: X-OPS-AUTHORIZATION-6: L/xml7wTc2/vQcGCPVvVPP9818ZUk9A/zsGtZusTVQ== DEBUG: HOST: OUR.CHEF.SERVER.COM:443 DEBUG: X-Ops-Server-API-Version: 1 DEBUG: X-REMOTE-REQUEST-ID: 9b7c4a9b-305b-4da4-a58b-888ff904bccd DEBUG: ---- End HTTP Request Header Data ---- DEBUG: ---- HTTP Status and Header Data: ---- DEBUG: HTTP 1.1 200 OK DEBUG: date: Tue, 12 Apr 2016 12:19:44 GMT DEBUG: server: Apache DEBUG: content-type: application/json DEBUG: x-ops-api-info: flavor=cs;version=12.0.0;oc_erchef=1.6.4 DEBUG: content-encoding: gzip DEBUG: via: 1.1 OUR.CHEF.SERVER.COM DEBUG: cache-control: max-age=300, no-transform, public DEBUG: expires: Tue, 12 Apr 2016 12:24:44 GMT DEBUG: strict-transport-security: max-age=16070400; includeSubdomains DEBUG: x-xss-protection: 1; mode=block DEBUG: x-content-type-options: nosniff DEBUG: x-frame-options: SAMEORIGIN DEBUG: sslinsecurerenegotiation: off DEBUG: vary: User-Agent DEBUG: x-cache: MISS from OUR.CHEF.SERVER.COM DEBUG: connection: close DEBUG: transfer-encoding: chunked DEBUG: ---- End HTTP Status/Header Data ---- DEBUG: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_response DEBUG: HTTP server did not include a Content-Length header in response, cannot identify truncated downloads. DEBUG: Chef::HTTP calling Chef::HTTP::RemoteRequestID#handle_response DEBUG: Chef::HTTP calling Chef::HTTP::Authenticator#handle_response DEBUG: Chef::HTTP calling Chef::HTTP::Decompressor#handle_response DEBUG: decompressing gzip response DEBUG: Chef::HTTP calling Chef::HTTP::CookieManager#handle_response DEBUG: Chef::HTTP calling Chef::HTTP::JSONToModelOutput#handle_response DEBUG: Chef::HTTP calling Chef::HTTP::JSONInput#handle_response DEBUG: Chef::HTTP calling Chef::HTTP::JSONInput#handle_request DEBUG: Chef::HTTP calling Chef::HTTP::JSONToModelOutput#handle_request DEBUG: Chef::HTTP calling Chef::HTTP::CookieManager#handle_request DEBUG: Chef::HTTP calling Chef::HTTP::Decompressor#handle_request DEBUG: Chef::HTTP calling Chef::HTTP::Authenticator#handle_request DEBUG: Signing the request as jenkins DEBUG: Chef::HTTP calling Chef::HTTP::RemoteRequestID#handle_request DEBUG: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_request DEBUG: Initiating POST to https://OUR.CHEF.SERVER.COM/organizations/our-organization/pushy/jobs DEBUG: ---- HTTP Request Header Data: ---- DEBUG: Content-Type: application/json DEBUG: Accept: application/json DEBUG: Accept-Encoding: gzip;q=1.0,deflate;q=0.6,identity;q=0.3 DEBUG: X-OPS-SIGN: algorithm=sha1;version=1.0; DEBUG: X-OPS-USERID: jenkins DEBUG: X-OPS-TIMESTAMP: 2016-04-12T12:19:44Z DEBUG: X-OPS-CONTENT-HASH: TiYIiflFAANw1V6I+P5xtq2+E60= DEBUG: X-OPS-AUTHORIZATION-1: R+zeAFRnlvLtJ6S/lbE5Qf5OgAOgFNu+bheX6vEAT1UxmicDZkBV8Oewt6i4 DEBUG: X-OPS-AUTHORIZATION-2: /FU25gjEZcb4uHv4D66yQqwlaKoiYKrJB+s0T6kJqehxRfQlJLVntVN5f6Ud DEBUG: X-OPS-AUTHORIZATION-3: TPwWoxidqPQO6zGxx2Glis8p4mBo9cJx+P7aejbl12haA3HogK3AEiv/azY0 DEBUG: X-OPS-AUTHORIZATION-4: ZbEAh4Rq1ne7q5WW9eeP0gdIOatXH7jvOSGS/WjI2PqqDhlJP5JUPxOhCeUZ DEBUG: X-OPS-AUTHORIZATION-5: XDkvPibyexxB4KBjZHaF/gmIoHfHG4zuOm2nOysN8vRuGiVVo1pmUsLzQEjU DEBUG: X-OPS-AUTHORIZATION-6: Xdw98AfVbBwYrZdFbTjQhqoswpygJLJyPzA7ksj6Kg== DEBUG: HOST: OUR.CHEF.SERVER.COM:443 DEBUG: X-Ops-Server-API-Version: 1 DEBUG: X-REMOTE-REQUEST-ID: 9b7c4a9b-305b-4da4-a58b-888ff904bccd DEBUG: Content-Length: 279 DEBUG: ---- End HTTP Request Header Data ---- DEBUG: ---- HTTP Status and Header Data: ---- DEBUG: HTTP 1.1 400 Bad Request DEBUG: date: Tue, 12 Apr 2016 12:19:44 GMT DEBUG: server: Apache DEBUG: content-length: 32 DEBUG: via: 1.1 OUR.CHEF.SERVER.COM DEBUG: cache-control: max-age=300, no-transform, public DEBUG: expires: Tue, 12 Apr 2016 12:24:44 GMT DEBUG: strict-transport-security: max-age=16070400; includeSubdomains DEBUG: x-xss-protection: 1; mode=block DEBUG: x-content-type-options: nosniff DEBUG: x-frame-options: SAMEORIGIN DEBUG: sslinsecurerenegotiation: off DEBUG: vary: User-Agent DEBUG: connection: close DEBUG: ---- End HTTP Status/Header Data ---- DEBUG: ---- HTTP Response Body ---- DEBUG: {"error":"invalid key supplied"} DEBUG: ---- End HTTP Response Body ----- DEBUG: ---- HTTP Request Body ---- DEBUG: {"command":"chef-client","nodes":["NODE1","NODE2","NODE3"],"capture_output":false,"quorum":3,"env":{}} DEBUG: ---- End HTTP Request Body ---- DEBUG: Chef::HTTP calling Chef::HTTP::ValidateContentLength#handle_response DEBUG: Content-Length validated correctly. DEBUG: Chef::HTTP calling Chef::HTTP::RemoteRequestID#handle_response DEBUG: Chef::HTTP calling Chef::HTTP::Authenticator#handle_response DEBUG: Chef::HTTP calling Chef::HTTP::Decompressor#handle_response DEBUG: Chef::HTTP calling Chef::HTTP::CookieManager#handle_response DEBUG: Chef::HTTP calling Chef::HTTP::JSONToModelOutput#handle_response DEBUG: Expected JSON response, but got content-type '' DEBUG: Chef::HTTP calling Chef::HTTP::JSONInput#handle_response INFO: HTTP Request Returned 400 Bad Request: error /opt/chefdk/embedded/lib/ruby/2.1.0/net/http/response.rb:119:inerror!’: 400 “Bad Request” (Net::HTTPServerException)
from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/http.rb:145:in request' from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/http.rb:126:inpost’
from /mnt/resource/jenkins/.chefdk/gem/ruby/2.1.0/gems/knife-push-1.0.0/lib/chef/knife/job_helpers.rb:95:in run_helper' from /mnt/resource/jenkins/.chefdk/gem/ruby/2.1.0/gems/knife-push-1.0.0/lib/chef/knife/job_start.rb:110:inrun’
from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/knife.rb:405:in block in run_with_pretty_exceptions' from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/local_mode.rb:44:inwith_server_connectivity’
from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/knife.rb:404:in run_with_pretty_exceptions' from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/knife.rb:203:inrun’
from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/lib/chef/application/knife.rb:142:in run' from /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/chef-12.5.1/bin/knife:25:in<top (required)>'
from /opt/chefdk/bin/knife:54:in load' from /opt/chefdk/bin/knife:54:in'`

We’re unsure as to what the cause of the error may be, does anyone have any suggestions?


#2

Any luck on this issue? I’m getting the same error with push-jobs


#3

I was able to get around the issue by downgrading knife-push gem. Assuming you have the latest version 1.0.0, remove that Gem and install knife-push v 0.5.2. Then run knife rehash. With this setup knife job list and knife job status commands will error, but knife job start will work - this is the tradeoff till a fix arrives for this :).

Bug and solution was originally reported here:
https:/github.com/chef/knife-push/issues/32