Berks upload error

[Michael_Lindsay]

Hey All,

I’m just setting up a new chef server (inside a vpc) and I’m having trouble
uploading the cookbooks with berks from my workstation:

Banks-2:base mlindsay$ berks upload -c ~/.berkshelf/vpc.json

E, [2015-05-13T12:14:00.542111 #16190] ERROR – :
Ridley::Errors::HTTPInternalServerError: {“error”:[“internal service
error”]}

E, [2015-05-13T12:14:00.542241 #16190] ERROR – :
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/middleware/chef_response.rb:24:in
`on_complete’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:9:in
`block in call’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:57:in
`on_complete’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:8:in
`call’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:8:in
`call’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:8:in
`call’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/middleware/chef_auth.rb:74:in
`call’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/request/retry.rb:87:in
`call’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/rack_builder.rb:139:in
`build_response’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/connection.rb:377:in
`run_request’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/connection.rb:104:in
`run_request’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/connection.rb:177:in
`put’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`public_send’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`dispatch’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
`dispatch’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in
`block in invoke’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
`block in task’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in
`block in task’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
`block in initialize’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_thread.rb:21:in
`block in create’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/thread_handle.rb:13:in
`block in initialize’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor_system.rb:32:in
`block in get_thread’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in
`call’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in
`block in create’

(celluloid):0:in `remote procedure call’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in
`value’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
`method_missing’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/cell_proxy.rb:17:in
`send’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/pool_manager.rb:41:in
`send’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/pool_manager.rb:140:in
`method_missing’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`public_send’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`dispatch’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
`dispatch’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in
`block in invoke’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
`block in task’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in
`block in task’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
`block in initialize’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:15:in
`block in create’

(celluloid):0:in `remote procedure call’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in
`value’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
`method_missing’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resource.rb:132:in
`raw_request’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resource.rb:123:in
`request’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resources/sandbox_resource.rb:46:in
`commit’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`public_send’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`dispatch’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
`dispatch’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in
`block in invoke’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
`block in task’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in
`block in task’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
`block in initialize’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:15:in
`block in create’

(celluloid):0:in `remote procedure call’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in
`value’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
`method_missing’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/chef_objects/sandbox_object.rb:53:in
`commit’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resources/cookbook_resource.rb:227:in
`upload’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`public_send’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`dispatch’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
`dispatch’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in
`block in invoke’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
`block in task’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in
`block in task’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
`block in initialize’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_thread.rb:21:in
`block in create’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/thread_handle.rb:13:in
`block in initialize’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor_system.rb:32:in
`block in get_thread’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in
`call’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in
`block in create’

(celluloid):0:in `remote procedure call’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in
`value’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
`method_missing’

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:54:in `block
(2 levels) in upload’

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:50:in `each’

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:50:in `block
in upload’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/client.rb:36:in
`open’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley.rb:51:in
`open’

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf.rb:157:in
`ridley_connection’

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:49:in `upload’

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:36:in `run’

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/berksfile.rb:542:in
`upload’

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cli.rb:208:in `upload’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/command.rb:27:in
`run’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/invocation.rb:126:in
`invoke_command’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor.rb:359:in
`dispatch’

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cli.rb:52:in `dispatch’

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/base.rb:440:in
`start’

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cli.rb:27:in `execute!’

/opt/chefdk/embedded/apps/berkshelf/bin/berks:5:in `<top (required)>’

/opt/chefdk/bin/berks:37:in `load’

/opt/chefdk/bin/berks:37:in `’

I haven’t had any luck googling the error for a solution.

Any thoughts?

• Michael

[sdelano]

Hi Michael,

The error {“error”:[“internal service error”]} is coming from the Chef Server, so the rest of that output is just Berks sending you a stacktrace.

In order to get closer to the root of the issue, you’ll need to provide some logs from the Chef Server. From the stacktrace, this appears to be happening when making a cookbook upload request: https://github.com/berkshelf/berkshelf/blob/master/lib/berkshelf/uploader.rb#L55-L60

You can find the relevant logs from the Chef Server in /var/log/opscode/opscode-erchef/requests.log.*

To filter them down, you should be able to grep for status=500, which is the corresponding error code. A message about what failed should be in the erchef log and should help us track down the issue.

Thanks!

—
Stephen Delano - Engineering Lead, Chef

On Wed, May 13, 2015 at 12:19 PM, Michael Lindsay mlindsay@metrodigi.com
wrote:

Hey All,
I'm just setting up a new chef server (inside a vpc) and I'm having trouble
uploading the cookbooks with berks from my workstation:
Banks-2:base mlindsay$ berks upload -c ~/.berkshelf/vpc.json
E, [2015-05-13T12:14:00.542111 #16190] ERROR -- :
Ridley::Errors::HTTPInternalServerError: {"error":["internal service
error"]}
E, [2015-05-13T12:14:00.542241 #16190] ERROR -- :
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/middleware/chef_response.rb:24:in
on_complete' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:9:in block in call'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:57:in
on_complete' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:8:in call'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:8:in
call' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:8:in call'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/middleware/chef_auth.rb:74:in
call' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/request/retry.rb:87:in call'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/rack_builder.rb:139:in
build_response' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/connection.rb:377:in run_request'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/connection.rb:104:in
run_request' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/connection.rb:177:in put'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
public_send' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in dispatch'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
dispatch' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in block in invoke'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
block in task' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in block in task'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
block in initialize' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_thread.rb:21:in block in create'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/thread_handle.rb:13:in
block in initialize' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor_system.rb:32:in block in get_thread'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in
call' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in block in create'
(celluloid):0:in remote procedure call' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in value'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
method_missing' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/cell_proxy.rb:17:in send'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/pool_manager.rb:41:in
_send_' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/pool_manager.rb:140:in method_missing'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
public_send' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in dispatch'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
dispatch' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in block in invoke'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
block in task' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in block in task'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
block in initialize' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:15:in block in create'
(celluloid):0:in remote procedure call' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in value'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
method_missing' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resource.rb:132:in raw_request'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resource.rb:123:in
request' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resources/sandbox_resource.rb:46:in commit'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
public_send' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in dispatch'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
dispatch' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in block in invoke'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
block in task' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in block in task'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
block in initialize' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:15:in block in create'
(celluloid):0:in remote procedure call' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in value'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
method_missing' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/chef_objects/sandbox_object.rb:53:in commit'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resources/cookbook_resource.rb:227:in
upload' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in public_send'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
dispatch' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in dispatch'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in
block in invoke' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in block in task'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in
block in task' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in block in initialize'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_thread.rb:21:in
block in create' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/thread_handle.rb:13:in block in initialize'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor_system.rb:32:in
block in get_thread' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in call'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in
block in create' (celluloid):0:in remote procedure call'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in
value' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in method_missing'
/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:54:in block (2 levels) in upload' /opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:50:in each'
/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:50:in block in upload' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/client.rb:36:in open'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley.rb:51:in
open' /opt/chefdk/embedded/apps/berkshelf/lib/berkshelf.rb:157:in ridley_connection'
/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:49:in upload' /opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:36:in run'
/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/berksfile.rb:542:in
upload' /opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cli.rb:208:in upload'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/command.rb:27:in
run' /opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/invocation.rb:126:in invoke_command'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor.rb:359:in
dispatch' /opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cli.rb:52:in dispatch'
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/base.rb:440:in
start' /opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cli.rb:27:in execute!'
/opt/chefdk/embedded/apps/berkshelf/bin/berks:5:in <top (required)>' /opt/chefdk/bin/berks:37:in load'
/opt/chefdk/bin/berks:37:in `'
I haven't had any luck googling the error for a solution.
Any thoughts?

• Michael

[Michael_Lindsay]

Hey Stephen,

Thanks for getting back to me... Is this the kind of thing you are looking
for?

==> /var/log/opscode/opscode-erchef/crash.log <==

2015-05-13 20:17:17 =ERROR REPORT====

Checking presence of checksum: <<"5b259a23bcccc6be76de01d811162d36">> for
org <<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has
taken longer than 5000 ms

2015-05-13 20:17:17 =ERROR REPORT====

Checking presence of checksum: <<"668e14353b42023b737efa15b3c675b9">> for
org <<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has
taken longer than 5000 ms

==> /var/log/opscode/opscode-erchef/current <==

2015-05-13_20:17:17.33628 [error] Checking presence of checksum:
<<"5b259a23bcccc6be76de01d811162d36">> for org
<<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has taken
longer than 5000 ms

2015-05-13_20:17:17.33629

==> /var/log/opscode/opscode-erchef/erchef.log <==

2015-05-13 20:17:17.330 [error] Checking presence of checksum:
<<"668e14353b42023b737efa15b3c675b9">> for org
<<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has taken
longer than 5000 ms

followed by this

==> /var/log/opscode/opscode-erchef/crash.log <==

2015-05-13 20:17:22 =ERROR REPORT====

{<<"method=PUT;
path=/organizations/metrodigi/sandboxes/d81dca36ec74c00114c5f3438e323f7d;
status=500;
">>,{error,{throw,{checksum_check_error,23},[{chef_wm_named_sandbox,validate_checksums_uploaded,2,[{file,"src/chef_wm_named_sandbox.erl"},{line,142}]},{chef_wm_named_sandbox,from_json,2,[{file,"src/chef_wm_named_sandbox.erl"},{line,97}]},{webmachine_resource,resource_call,3,[{file,"src/webmachine_resource.erl"},{line,186}]},{webmachine_resource,do,3,[{file,"src/webmachine_resource.erl"},{line,142}]},{webmachine_decision_core,resource_call,1,[{file,"src/webmachine_decision_core.erl"},{line,48}]},{webmachine_decision_core,accept_helper,1,[{file,"src/webmachine_decision_core.erl"},{line,616}]},{webmachine_decision_core,decision,1,[{file,"src/webmachine_decision_core.erl"},{line,521}]},{webmachine_decision_core,handle_request,2,[{file,"src/webmachine_decision_core.erl"},{line,33}]}]}}}

Here's a note, during my install, in order to get chef-server-ctl to work I
had to:

sudo chmod 0644 /etc/opscode/pivotal.pem

should I have not used sudo when I: chef-solo -o
'recipe[chef-server::default]' -j dna.json ?

Thanks,

• Michael

On Wed, May 13, 2015 at 1:00 PM, Stephen Delano stephen@opscode.com wrote:

Hi Michael,

The error {“error”:[“internal service error”]} is coming from the Chef
Server, so the rest of that output is just Berks sending you a stacktrace.

In order to get closer to the root of the issue, you’ll need to provide
some logs from the Chef Server. From the stacktrace, this appears to be
happening when making a cookbook upload request:
https://github.com/berkshelf/berkshelf/blob/master/lib/berkshelf/uploader.rb#L55-L60

You can find the relevant logs from the Chef Server in
/var/log/opscode/opscode-erchef/requests.log.*

To filter them down, you should be able to grep for status=500, which is
the corresponding error code. A message about what failed should be in the
erchef log and should help us track down the issue.

Thanks!

—
Stephen Delano - Engineering Lead, Chef

On Wed, May 13, 2015 at 12:19 PM, Michael Lindsay mlindsay@metrodigi.com
wrote:

Hey All,

I'm just setting up a new chef server (inside a vpc) and I'm having
trouble uploading the cookbooks with berks from my workstation:

Banks-2:base mlindsay$ berks upload -c ~/.berkshelf/vpc.json

E, [2015-05-13T12:14:00.542111 #16190] ERROR -- :
Ridley::Errors::HTTPInternalServerError: {"error":["internal service
error"]}

E, [2015-05-13T12:14:00.542241 #16190] ERROR -- :
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/middleware/chef_response.rb:24:in
`on_complete'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:9:in
`block in call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:57:in
`on_complete'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:8:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:8:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:8:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/middleware/chef_auth.rb:74:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/request/retry.rb:87:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/rack_builder.rb:139:in
`build_response'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/connection.rb:377:in
`run_request'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/connection.rb:104:in
`run_request'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/connection.rb:177:in
`put'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`public_send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in
`block in invoke'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_thread.rb:21:in
`block in create'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/thread_handle.rb:13:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor_system.rb:32:in
`block in get_thread'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in
`block in create'

(celluloid):0:in `remote procedure call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in
`value'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
`method_missing'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/cell_proxy.rb:17:in
`send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/pool_manager.rb:41:in
`send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/pool_manager.rb:140:in
`method_missing'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`public_send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in
`block in invoke'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:15:in
`block in create'

(celluloid):0:in `remote procedure call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in
`value'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
`method_missing'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resource.rb:132:in
`raw_request'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resource.rb:123:in
`request'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resources/sandbox_resource.rb:46:in
`commit'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`public_send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in
`block in invoke'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:15:in
`block in create'

(celluloid):0:in `remote procedure call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in
`value'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
`method_missing'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/chef_objects/sandbox_object.rb:53:in
`commit'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resources/cookbook_resource.rb:227:in
`upload'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`public_send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in
`block in invoke'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_thread.rb:21:in
`block in create'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/thread_handle.rb:13:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor_system.rb:32:in
`block in get_thread'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in
`block in create'

(celluloid):0:in `remote procedure call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in
`value'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
`method_missing'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:54:in
`block (2 levels) in upload'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:50:in `each'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:50:in
`block in upload'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/client.rb:36:in
`open'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley.rb:51:in
`open'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf.rb:157:in
`ridley_connection'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:49:in
`upload'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:36:in `run'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/berksfile.rb:542:in
`upload'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cli.rb:208:in `upload'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/command.rb:27:in
`run'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/invocation.rb:126:in
`invoke_command'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor.rb:359:in
`dispatch'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cli.rb:52:in `dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/base.rb:440:in
`start'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cli.rb:27:in `execute!'

/opt/chefdk/embedded/apps/berkshelf/bin/berks:5:in `<top (required)>'

/opt/chefdk/bin/berks:37:in `load'

/opt/chefdk/bin/berks:37:in `'

I haven't had any luck googling the error for a solution.

Any thoughts?

• Michael

[sdelano]

Thanks Michael,

A few more things:

• Does the same error (or at least a similar failure) occur when you run a knife cookbook upload?

• From the Chef Server, what is the output of the chef-server-ctl status command? Is the bookshelf service running? I expect it is if you’ve gotten this far.

• What are the contents of your /etc/opscode/chef-server.rb configuration file (on the Chef Server)?

Thanks!

—
Stephen Delano - Engineering Lead, Chef

On Wed, May 13, 2015 at 1:27 PM, Michael Lindsay mlindsay@metrodigi.com
wrote:

Hey Stephen,
Thanks for getting back to me... Is this the kind of thing you are looking
for?
==> /var/log/opscode/opscode-erchef/crash.log <==
2015-05-13 20:17:17 =ERROR REPORT====
Checking presence of checksum: <<"5b259a23bcccc6be76de01d811162d36">> for
org <<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has
taken longer than 5000 ms
2015-05-13 20:17:17 =ERROR REPORT====
Checking presence of checksum: <<"668e14353b42023b737efa15b3c675b9">> for
org <<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has
taken longer than 5000 ms
==> /var/log/opscode/opscode-erchef/current <==
2015-05-13_20:17:17.33628 [error] Checking presence of checksum:
<<"5b259a23bcccc6be76de01d811162d36">> for org
<<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has taken
longer than 5000 ms
2015-05-13_20:17:17.33629
==> /var/log/opscode/opscode-erchef/erchef.log <==
2015-05-13 20:17:17.330 [error] Checking presence of checksum:
<<"668e14353b42023b737efa15b3c675b9">> for org
<<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has taken
longer than 5000 ms
followed by this
==> /var/log/opscode/opscode-erchef/crash.log <==
2015-05-13 20:17:22 =ERROR REPORT====
{<<"method=PUT;
path=/organizations/metrodigi/sandboxes/d81dca36ec74c00114c5f3438e323f7d;
status=500;
">>,{error,{throw,{checksum_check_error,23},[{chef_wm_named_sandbox,validate_checksums_uploaded,2,[{file,"src/chef_wm_named_sandbox.erl"},{line,142}]},{chef_wm_named_sandbox,from_json,2,[{file,"src/chef_wm_named_sandbox.erl"},{line,97}]},{webmachine_resource,resource_call,3,[{file,"src/webmachine_resource.erl"},{line,186}]},{webmachine_resource,do,3,[{file,"src/webmachine_resource.erl"},{line,142}]},{webmachine_decision_core,resource_call,1,[{file,"src/webmachine_decision_core.erl"},{line,48}]},{webmachine_decision_core,accept_helper,1,[{file,"src/webmachine_decision_core.erl"},{line,616}]},{webmachine_decision_core,decision,1,[{file,"src/webmachine_decision_core.erl"},{line,521}]},{webmachine_decision_core,handle_request,2,[{file,"src/webmachine_decision_core.erl"},{line,33}]}]}}}
Here's a note, during my install, in order to get chef-server-ctl to work I
had to:
sudo chmod 0644 /etc/opscode/pivotal.pem
should I have not used sudo when I: chef-solo -o
'recipe[chef-server::default]' -j dna.json ?
Thanks,

• Michael
  On Wed, May 13, 2015 at 1:00 PM, Stephen Delano stephen@opscode.com wrote:

Hi Michael,

The error {“error”:[“internal service error”]} is coming from the Chef
Server, so the rest of that output is just Berks sending you a stacktrace.

In order to get closer to the root of the issue, you’ll need to provide
some logs from the Chef Server. From the stacktrace, this appears to be
happening when making a cookbook upload request:
https://github.com/berkshelf/berkshelf/blob/master/lib/berkshelf/uploader.rb#L55-L60

You can find the relevant logs from the Chef Server in
/var/log/opscode/opscode-erchef/requests.log.*

To filter them down, you should be able to grep for status=500, which is
the corresponding error code. A message about what failed should be in the
erchef log and should help us track down the issue.

Thanks!

—
Stephen Delano - Engineering Lead, Chef

On Wed, May 13, 2015 at 12:19 PM, Michael Lindsay mlindsay@metrodigi.com
wrote:

Hey All,

I'm just setting up a new chef server (inside a vpc) and I'm having
trouble uploading the cookbooks with berks from my workstation:

Banks-2:base mlindsay$ berks upload -c ~/.berkshelf/vpc.json

E, [2015-05-13T12:14:00.542111 #16190] ERROR -- :
Ridley::Errors::HTTPInternalServerError: {"error":["internal service
error"]}

E, [2015-05-13T12:14:00.542241 #16190] ERROR -- :
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/middleware/chef_response.rb:24:in
`on_complete'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:9:in
`block in call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:57:in
`on_complete'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:8:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:8:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:8:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/middleware/chef_auth.rb:74:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/request/retry.rb:87:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/rack_builder.rb:139:in
`build_response'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/connection.rb:377:in
`run_request'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/connection.rb:104:in
`run_request'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/connection.rb:177:in
`put'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`public_send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in
`block in invoke'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_thread.rb:21:in
`block in create'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/thread_handle.rb:13:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor_system.rb:32:in
`block in get_thread'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in
`block in create'

(celluloid):0:in `remote procedure call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in
`value'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
`method_missing'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/cell_proxy.rb:17:in
`send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/pool_manager.rb:41:in
`send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/pool_manager.rb:140:in
`method_missing'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`public_send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in
`block in invoke'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:15:in
`block in create'

(celluloid):0:in `remote procedure call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in
`value'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
`method_missing'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resource.rb:132:in
`raw_request'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resource.rb:123:in
`request'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resources/sandbox_resource.rb:46:in
`commit'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`public_send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in
`block in invoke'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:15:in
`block in create'

(celluloid):0:in `remote procedure call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in
`value'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
`method_missing'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/chef_objects/sandbox_object.rb:53:in
`commit'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resources/cookbook_resource.rb:227:in
`upload'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`public_send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in
`block in invoke'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_thread.rb:21:in
`block in create'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/thread_handle.rb:13:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor_system.rb:32:in
`block in get_thread'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in
`block in create'

(celluloid):0:in `remote procedure call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in
`value'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
`method_missing'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:54:in
`block (2 levels) in upload'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:50:in `each'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:50:in
`block in upload'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/client.rb:36:in
`open'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley.rb:51:in
`open'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf.rb:157:in
`ridley_connection'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:49:in
`upload'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:36:in `run'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/berksfile.rb:542:in
`upload'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cli.rb:208:in `upload'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/command.rb:27:in
`run'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/invocation.rb:126:in
`invoke_command'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor.rb:359:in
`dispatch'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cli.rb:52:in `dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/base.rb:440:in
`start'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cli.rb:27:in `execute!'

/opt/chefdk/embedded/apps/berkshelf/bin/berks:5:in `<top (required)>'

/opt/chefdk/bin/berks:37:in `load'

/opt/chefdk/bin/berks:37:in `'

I haven't had any luck googling the error for a solution.

Any thoughts?

• Michael

[Michael_Lindsay]

So, because all of my cookbooks depend on other cookbooks had to find one
that I depended on that nothing else did so...

1. knife cookbook upload

Banks-2:md-chef mlindsay$ git clone git@github.com:
opscode-cookbooks/users.git

Cloning into 'users'...

remote: Counting objects: 468, done.

remote: Total 468 (delta 0), reused 0 (delta 0), pack-reused 468

Receiving objects: 100% (468/468), 94.43 KiB | 0 bytes/s, done.

Resolving deltas: 100% (176/176), done.

Checking connectivity... done.

Banks-2:md-chef mlindsay$ knife cookbook upload users -c ~/.chef/knife.rb

ERROR: knife encountered an unexpected error

This may be a bug in the 'cookbook upload' knife command or plugin

Please collect the output of this command with the -VV option before
filing a bug report.

Exception: NoMethodError: undefined method `source_url' for
#Chef::Cookbook::Metadata:0x007fd59a168ee8

Banks-2:md-chef mlindsay$ knife cookbook upload users -c ~/.chef/knife.rb
-VV

DEBUG: No chefignore file found at /Users/mlindsay/chefignore no files will
be ignored

/Users/mlindsay/md-chef/users/metadata.rb:8:in from_file': undefined method source_url' for #Chef::Cookbook::Metadata:0x007f9751ace960
(NoMethodError)

from /opt/chefdk/embedded/apps/chef/lib/chef/mixin/from_file.rb:30:in
`instance_eval'

from /opt/chefdk/embedded/apps/chef/lib/chef/mixin/from_file.rb:30:in
`from_file'

from
/opt/chefdk/embedded/apps/chef/lib/chef/cookbook/cookbook_version_loader.rb:172:in
`apply_ruby_metadata'

from
/opt/chefdk/embedded/apps/chef/lib/chef/cookbook/cookbook_version_loader.rb:110:in
`block in metadata'

from
/opt/chefdk/embedded/apps/chef/lib/chef/cookbook/cookbook_version_loader.rb:107:in
`each'

from
/opt/chefdk/embedded/apps/chef/lib/chef/cookbook/cookbook_version_loader.rb:107:in
`metadata'

from
/opt/chefdk/embedded/apps/chef/lib/chef/cookbook/cookbook_version_loader.rb:99:in
`block in cookbook_version'

from
/opt/chefdk/embedded/apps/chef/lib/chef/cookbook/cookbook_version_loader.rb:88:in
`tap'

from
/opt/chefdk/embedded/apps/chef/lib/chef/cookbook/cookbook_version_loader.rb:88:in
`cookbook_version'

from /opt/chefdk/embedded/apps/chef/lib/chef/cookbook_loader.rb:95:in
`load_cookbook'

from /opt/chefdk/embedded/apps/chef/lib/chef/cookbook_loader.rb:103:in `'

from
/opt/chefdk/embedded/apps/chef/lib/chef/knife/cookbook_upload.rb:171:in
`block in cookbooks_to_upload'

from
/opt/chefdk/embedded/apps/chef/lib/chef/knife/cookbook_upload.rb:168:in
`each'

from
/opt/chefdk/embedded/apps/chef/lib/chef/knife/cookbook_upload.rb:168:in
`cookbooks_to_upload'

from
/opt/chefdk/embedded/apps/chef/lib/chef/knife/cookbook_upload.rb:206:in
`warn_about_cookbook_shadowing'

from /opt/chefdk/embedded/apps/chef/lib/chef/knife/cookbook_upload.rb:96:in
`run'

from /opt/chefdk/embedded/apps/chef/lib/chef/knife.rb:493:in
`run_with_pretty_exceptions'

from /opt/chefdk/embedded/apps/chef/lib/chef/knife.rb:174:in `run'

from /opt/chefdk/embedded/apps/chef/lib/chef/application/knife.rb:139:in
`run'

from /opt/chefdk/embedded/apps/chef/bin/knife:25:in `<top (required)>'

from /opt/chefdk/bin/knife:34:in `load'

from /opt/chefdk/bin/knife:34:in `'

root@chef:/home/ubuntu# chef-server-ctl status

run: bookshelf: (pid 6463) 14882s; run: log: (pid 5272) 15207s

AUTO-GENERATED BY CHEF - DO NOT MODIFY

topology "standalone"

api_fqdn "chef.DOMAIN.com"

On Wed, May 13, 2015 at 1:48 PM, Stephen Delano stephen@opscode.com wrote:

Thanks Michael,

A few more things:

• Does the same error (or at least a similar failure) occur when you run a
  knife cookbook upload?
• From the Chef Server, what is the output of the chef-server-ctl status
  command? Is the bookshelf service running? I expect it is if you’ve gotten
  this far.
• What are the contents of your /etc/opscode/chef-server.rb configuration
  file (on the Chef Server)?

Thanks!

—
Stephen Delano - Engineering Lead, Chef

On Wed, May 13, 2015 at 1:27 PM, Michael Lindsay mlindsay@metrodigi.com
wrote:

Hey Stephen,

Thanks for getting back to me... Is this the kind of thing you are
looking for?

==> /var/log/opscode/opscode-erchef/crash.log <==

2015-05-13 20:17:17 =ERROR REPORT====

Checking presence of checksum: <<"5b259a23bcccc6be76de01d811162d36">> for
org <<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has
taken longer than 5000 ms

2015-05-13 20:17:17 =ERROR REPORT====

Checking presence of checksum: <<"668e14353b42023b737efa15b3c675b9">> for
org <<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has
taken longer than 5000 ms

==> /var/log/opscode/opscode-erchef/current <==

2015-05-13_20:17:17.33628 [error] Checking presence of checksum:
<<"5b259a23bcccc6be76de01d811162d36">> for org
<<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has taken
longer than 5000 ms

2015-05-13_20:17:17.33629

==> /var/log/opscode/opscode-erchef/erchef.log <==

2015-05-13 20:17:17.330 [error] Checking presence of checksum:
<<"668e14353b42023b737efa15b3c675b9">> for org
<<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has taken
longer than 5000 ms

followed by this

==> /var/log/opscode/opscode-erchef/crash.log <==

2015-05-13 20:17:22 =ERROR REPORT====

{<<"method=PUT;
path=/organizations/metrodigi/sandboxes/d81dca36ec74c00114c5f3438e323f7d;
status=500;
">>,{error,{throw,{checksum_check_error,23},[{chef_wm_named_sandbox,validate_checksums_uploaded,2,[{file,"src/chef_wm_named_sandbox.erl"},{line,142}]},{chef_wm_named_sandbox,from_json,2,[{file,"src/chef_wm_named_sandbox.erl"},{line,97}]},{webmachine_resource,resource_call,3,[{file,"src/webmachine_resource.erl"},{line,186}]},{webmachine_resource,do,3,[{file,"src/webmachine_resource.erl"},{line,142}]},{webmachine_decision_core,resource_call,1,[{file,"src/webmachine_decision_core.erl"},{line,48}]},{webmachine_decision_core,accept_helper,1,[{file,"src/webmachine_decision_core.erl"},{line,616}]},{webmachine_decision_core,decision,1,[{file,"src/webmachine_decision_core.erl"},{line,521}]},{webmachine_decision_core,handle_request,2,[{file,"src/webmachine_decision_core.erl"},{line,33}]}]}}}

Here's a note, during my install, in order to get chef-server-ctl to work
I had to:

sudo chmod 0644 /etc/opscode/pivotal.pem

should I have not used sudo when I: chef-solo -o
'recipe[chef-server::default]' -j dna.json ?

Thanks,

• Michael

On Wed, May 13, 2015 at 1:00 PM, Stephen Delano stephen@opscode.com
wrote:

Hi Michael,

The error {“error”:[“internal service error”]} is coming from the Chef
Server, so the rest of that output is just Berks sending you a stacktrace.

In order to get closer to the root of the issue, you’ll need to provide
some logs from the Chef Server. From the stacktrace, this appears to be
happening when making a cookbook upload request:
https://github.com/berkshelf/berkshelf/blob/master/lib/berkshelf/uploader.rb#L55-L60

You can find the relevant logs from the Chef Server in
/var/log/opscode/opscode-erchef/requests.log.*

To filter them down, you should be able to grep for status=500, which is
the corresponding error code. A message about what failed should be in the
erchef log and should help us track down the issue.

Thanks!

—
Stephen Delano - Engineering Lead, Chef

On Wed, May 13, 2015 at 12:19 PM, Michael Lindsay <
mlindsay@metrodigi.com> wrote:

Hey All,

I'm just setting up a new chef server (inside a vpc) and I'm having
trouble uploading the cookbooks with berks from my workstation:

Banks-2:base mlindsay$ berks upload -c ~/.berkshelf/vpc.json

E, [2015-05-13T12:14:00.542111 #16190] ERROR -- :
Ridley::Errors::HTTPInternalServerError: {"error":["internal service
error"]}

E, [2015-05-13T12:14:00.542241 #16190] ERROR -- :
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/middleware/chef_response.rb:24:in
`on_complete'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:9:in
`block in call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:57:in
`on_complete'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:8:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:8:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:8:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/middleware/chef_auth.rb:74:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/request/retry.rb:87:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/rack_builder.rb:139:in
`build_response'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/connection.rb:377:in
`run_request'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/connection.rb:104:in
`run_request'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/connection.rb:177:in
`put'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`public_send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in
`block in invoke'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_thread.rb:21:in
`block in create'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/thread_handle.rb:13:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor_system.rb:32:in
`block in get_thread'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in
`block in create'

(celluloid):0:in `remote procedure call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in
`value'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
`method_missing'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/cell_proxy.rb:17:in
`send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/pool_manager.rb:41:in
`send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/pool_manager.rb:140:in
`method_missing'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`public_send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in
`block in invoke'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:15:in
`block in create'

(celluloid):0:in `remote procedure call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in
`value'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
`method_missing'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resource.rb:132:in
`raw_request'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resource.rb:123:in
`request'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resources/sandbox_resource.rb:46:in
`commit'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`public_send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in
`block in invoke'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:15:in
`block in create'

(celluloid):0:in `remote procedure call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in
`value'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
`method_missing'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/chef_objects/sandbox_object.rb:53:in
`commit'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resources/cookbook_resource.rb:227:in
`upload'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`public_send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in
`block in invoke'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_thread.rb:21:in
`block in create'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/thread_handle.rb:13:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor_system.rb:32:in
`block in get_thread'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in
`block in create'

(celluloid):0:in `remote procedure call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in
`value'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
`method_missing'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:54:in
`block (2 levels) in upload'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:50:in
`each'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:50:in
`block in upload'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/client.rb:36:in
`open'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley.rb:51:in
`open'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf.rb:157:in
`ridley_connection'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:49:in
`upload'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:36:in
`run'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/berksfile.rb:542:in
`upload'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cli.rb:208:in `upload'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/command.rb:27:in
`run'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/invocation.rb:126:in
`invoke_command'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor.rb:359:in
`dispatch'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cli.rb:52:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/base.rb:440:in
`start'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cli.rb:27:in
`execute!'

/opt/chefdk/embedded/apps/berkshelf/bin/berks:5:in `<top (required)>'

/opt/chefdk/bin/berks:37:in `load'

/opt/chefdk/bin/berks:37:in `'

I haven't had any luck googling the error for a solution.

Any thoughts?

• Michael

[Michael_Lindsay]

A little more about my set up.

The chef server installed by solo is 12.0.8 running on Ubuntu 14.04. As I
mentioned previously, the only thing special that I did was sudo chmod 0644 /etc/opscode/pivotal.pem in order to get chef-server-ctl to run.

I'm running The client I am using was previously pointing chef server
11.1.3. And in fact, I'm using the -c option to load a vpc config file
(for both knife and berks). I know that one difference now (could just be
how it was set up the first time) is that I need to add
'/organizations/ORGNAME' to my vpc_knife.rb file. With this in place I can
knife ssl check -c ~/.chef/vpc_knife.rb and get a client list. I'm still
trying to tweak the berkshelf settings.

On Wed, May 13, 2015 at 3:25 PM, Michael Lindsay mlindsay@metrodigi.com
wrote:

So, because all of my cookbooks depend on other cookbooks had to find one
that I depended on that nothing else did so...

1. knife cookbook upload

Banks-2:md-chef mlindsay$ git clone git@github.com:
opscode-cookbooks/users.git

Cloning into 'users'...

remote: Counting objects: 468, done.

remote: Total 468 (delta 0), reused 0 (delta 0), pack-reused 468

Receiving objects: 100% (468/468), 94.43 KiB | 0 bytes/s, done.

Resolving deltas: 100% (176/176), done.

Checking connectivity... done.

Banks-2:md-chef mlindsay$ knife cookbook upload users -c ~/.chef/knife.rb

ERROR: knife encountered an unexpected error

This may be a bug in the 'cookbook upload' knife command or plugin

Please collect the output of this command with the -VV option before
filing a bug report.

Exception: NoMethodError: undefined method `source_url' for
#Chef::Cookbook::Metadata:0x007fd59a168ee8

Banks-2:md-chef mlindsay$ knife cookbook upload users -c ~/.chef/knife.rb
-VV

DEBUG: No chefignore file found at /Users/mlindsay/chefignore no files
will be ignored

/Users/mlindsay/md-chef/users/metadata.rb:8:in from_file': undefined method source_url' for #Chef::Cookbook::Metadata:0x007f9751ace960
(NoMethodError)

from /opt/chefdk/embedded/apps/chef/lib/chef/mixin/from_file.rb:30:in
`instance_eval'

from /opt/chefdk/embedded/apps/chef/lib/chef/mixin/from_file.rb:30:in
`from_file'

from
/opt/chefdk/embedded/apps/chef/lib/chef/cookbook/cookbook_version_loader.rb:172:in
`apply_ruby_metadata'

from
/opt/chefdk/embedded/apps/chef/lib/chef/cookbook/cookbook_version_loader.rb:110:in
`block in metadata'

from
/opt/chefdk/embedded/apps/chef/lib/chef/cookbook/cookbook_version_loader.rb:107:in
`each'

from
/opt/chefdk/embedded/apps/chef/lib/chef/cookbook/cookbook_version_loader.rb:107:in
`metadata'

from
/opt/chefdk/embedded/apps/chef/lib/chef/cookbook/cookbook_version_loader.rb:99:in
`block in cookbook_version'

from
/opt/chefdk/embedded/apps/chef/lib/chef/cookbook/cookbook_version_loader.rb:88:in
`tap'

from
/opt/chefdk/embedded/apps/chef/lib/chef/cookbook/cookbook_version_loader.rb:88:in
`cookbook_version'

from /opt/chefdk/embedded/apps/chef/lib/chef/cookbook_loader.rb:95:in
`load_cookbook'

from /opt/chefdk/embedded/apps/chef/lib/chef/cookbook_loader.rb:103:in `'

from
/opt/chefdk/embedded/apps/chef/lib/chef/knife/cookbook_upload.rb:171:in
`block in cookbooks_to_upload'

from
/opt/chefdk/embedded/apps/chef/lib/chef/knife/cookbook_upload.rb:168:in
`each'

from
/opt/chefdk/embedded/apps/chef/lib/chef/knife/cookbook_upload.rb:168:in
`cookbooks_to_upload'

from
/opt/chefdk/embedded/apps/chef/lib/chef/knife/cookbook_upload.rb:206:in
`warn_about_cookbook_shadowing'

from
/opt/chefdk/embedded/apps/chef/lib/chef/knife/cookbook_upload.rb:96:in `run'

from /opt/chefdk/embedded/apps/chef/lib/chef/knife.rb:493:in
`run_with_pretty_exceptions'

from /opt/chefdk/embedded/apps/chef/lib/chef/knife.rb:174:in `run'

from /opt/chefdk/embedded/apps/chef/lib/chef/application/knife.rb:139:in
`run'

from /opt/chefdk/embedded/apps/chef/bin/knife:25:in `<top (required)>'

from /opt/chefdk/bin/knife:34:in `load'

from /opt/chefdk/bin/knife:34:in `'

root@chef:/home/ubuntu# chef-server-ctl status

run: bookshelf: (pid 6463) 14882s; run: log: (pid 5272) 15207s

AUTO-GENERATED BY CHEF - DO NOT MODIFY

topology "standalone"

api_fqdn "chef.DOMAIN.com"

On Wed, May 13, 2015 at 1:48 PM, Stephen Delano stephen@opscode.com
wrote:

Thanks Michael,

A few more things:

• Does the same error (or at least a similar failure) occur when you run
  a knife cookbook upload?
• From the Chef Server, what is the output of the chef-server-ctl status command? Is the bookshelf service running? I expect it is if you’ve
  gotten this far.
• What are the contents of your /etc/opscode/chef-server.rb configuration
  file (on the Chef Server)?

Thanks!

—
Stephen Delano - Engineering Lead, Chef

On Wed, May 13, 2015 at 1:27 PM, Michael Lindsay mlindsay@metrodigi.com
wrote:

Hey Stephen,

Thanks for getting back to me... Is this the kind of thing you are
looking for?

==> /var/log/opscode/opscode-erchef/crash.log <==

2015-05-13 20:17:17 =ERROR REPORT====

Checking presence of checksum: <<"5b259a23bcccc6be76de01d811162d36">>
for org <<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has
taken longer than 5000 ms

2015-05-13 20:17:17 =ERROR REPORT====

Checking presence of checksum: <<"668e14353b42023b737efa15b3c675b9">>
for org <<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has
taken longer than 5000 ms

==> /var/log/opscode/opscode-erchef/current <==

2015-05-13_20:17:17.33628 [error] Checking presence of checksum:
<<"5b259a23bcccc6be76de01d811162d36">> for org
<<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has taken
longer than 5000 ms

2015-05-13_20:17:17.33629

==> /var/log/opscode/opscode-erchef/erchef.log <==

2015-05-13 20:17:17.330 [error] Checking presence of checksum:
<<"668e14353b42023b737efa15b3c675b9">> for org
<<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has taken
longer than 5000 ms

followed by this

==> /var/log/opscode/opscode-erchef/crash.log <==

2015-05-13 20:17:22 =ERROR REPORT====

{<<"method=PUT;
path=/organizations/metrodigi/sandboxes/d81dca36ec74c00114c5f3438e323f7d;
status=500;
">>,{error,{throw,{checksum_check_error,23},[{chef_wm_named_sandbox,validate_checksums_uploaded,2,[{file,"src/chef_wm_named_sandbox.erl"},{line,142}]},{chef_wm_named_sandbox,from_json,2,[{file,"src/chef_wm_named_sandbox.erl"},{line,97}]},{webmachine_resource,resource_call,3,[{file,"src/webmachine_resource.erl"},{line,186}]},{webmachine_resource,do,3,[{file,"src/webmachine_resource.erl"},{line,142}]},{webmachine_decision_core,resource_call,1,[{file,"src/webmachine_decision_core.erl"},{line,48}]},{webmachine_decision_core,accept_helper,1,[{file,"src/webmachine_decision_core.erl"},{line,616}]},{webmachine_decision_core,decision,1,[{file,"src/webmachine_decision_core.erl"},{line,521}]},{webmachine_decision_core,handle_request,2,[{file,"src/webmachine_decision_core.erl"},{line,33}]}]}}}

Here's a note, during my install, in order to get chef-server-ctl to
work I had to:

sudo chmod 0644 /etc/opscode/pivotal.pem

should I have not used sudo when I: chef-solo -o
'recipe[chef-server::default]' -j dna.json ?

Thanks,

• Michael

On Wed, May 13, 2015 at 1:00 PM, Stephen Delano stephen@opscode.com
wrote:

Hi Michael,

The error {“error”:[“internal service error”]} is coming from the
Chef Server, so the rest of that output is just Berks sending you a
stacktrace.

In order to get closer to the root of the issue, you’ll need to provide
some logs from the Chef Server. From the stacktrace, this appears to be
happening when making a cookbook upload request:
https://github.com/berkshelf/berkshelf/blob/master/lib/berkshelf/uploader.rb#L55-L60

You can find the relevant logs from the Chef Server in
/var/log/opscode/opscode-erchef/requests.log.*

To filter them down, you should be able to grep for status=500, which
is the corresponding error code. A message about what failed should be in
the erchef log and should help us track down the issue.

Thanks!

—
Stephen Delano - Engineering Lead, Chef

On Wed, May 13, 2015 at 12:19 PM, Michael Lindsay <
mlindsay@metrodigi.com> wrote:

Hey All,

I'm just setting up a new chef server (inside a vpc) and I'm having
trouble uploading the cookbooks with berks from my workstation:

Banks-2:base mlindsay$ berks upload -c ~/.berkshelf/vpc.json

E, [2015-05-13T12:14:00.542111 #16190] ERROR -- :
Ridley::Errors::HTTPInternalServerError: {"error":["internal service
error"]}

E, [2015-05-13T12:14:00.542241 #16190] ERROR -- :
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/middleware/chef_response.rb:24:in
`on_complete'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:9:in
`block in call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:57:in
`on_complete'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:8:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:8:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:8:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/middleware/chef_auth.rb:74:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/request/retry.rb:87:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/rack_builder.rb:139:in
`build_response'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/connection.rb:377:in
`run_request'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/connection.rb:104:in
`run_request'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/connection.rb:177:in
`put'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`public_send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in
`block in invoke'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_thread.rb:21:in
`block in create'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/thread_handle.rb:13:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor_system.rb:32:in
`block in get_thread'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in
`block in create'

(celluloid):0:in `remote procedure call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in
`value'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
`method_missing'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/cell_proxy.rb:17:in
`send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/pool_manager.rb:41:in
`send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/pool_manager.rb:140:in
`method_missing'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`public_send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in
`block in invoke'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:15:in
`block in create'

(celluloid):0:in `remote procedure call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in
`value'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
`method_missing'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resource.rb:132:in
`raw_request'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resource.rb:123:in
`request'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resources/sandbox_resource.rb:46:in
`commit'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`public_send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in
`block in invoke'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:15:in
`block in create'

(celluloid):0:in `remote procedure call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in
`value'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
`method_missing'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/chef_objects/sandbox_object.rb:53:in
`commit'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resources/cookbook_resource.rb:227:in
`upload'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`public_send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in
`block in invoke'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_thread.rb:21:in
`block in create'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/thread_handle.rb:13:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor_system.rb:32:in
`block in get_thread'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in
`block in create'

(celluloid):0:in `remote procedure call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in
`value'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
`method_missing'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:54:in
`block (2 levels) in upload'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:50:in
`each'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:50:in
`block in upload'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/client.rb:36:in
`open'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley.rb:51:in
`open'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf.rb:157:in
`ridley_connection'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:49:in
`upload'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:36:in
`run'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/berksfile.rb:542:in
`upload'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cli.rb:208:in
`upload'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/command.rb:27:in
`run'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/invocation.rb:126:in
`invoke_command'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor.rb:359:in
`dispatch'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cli.rb:52:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/base.rb:440:in
`start'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cli.rb:27:in
`execute!'

/opt/chefdk/embedded/apps/berkshelf/bin/berks:5:in `<top (required)>'

/opt/chefdk/bin/berks:37:in `load'

/opt/chefdk/bin/berks:37:in `'

I haven't had any luck googling the error for a solution.

Any thoughts?

• Michael

[Michael_Lindsay]

I have also tried setting my new configuration to be the default knife.rb
and renamed config.json in my .berkshelf directory. Same error:

==> /var/log/opscode/opscode-erchef/current <==

2015-05-14_17:12:38.73280 [error] Checking presence of checksum:
<<"5b259a23bcccc6be76de01d811162d36">> for org
<<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has taken
longer than 5000 ms

...

2015-05-14_17:12:43.74133 [error] Checking presence of checksum:
<<"471f01912a5109386225389563db4159">> for org
<<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has taken
longer than 5000 ms

2015-05-14_17:12:43.74134

2015-05-14_17:12:43.74351 [error] {<<"method=PUT;
path=/organizations/metrodigi/sandboxes/d81dca36ec749108cdfdcab1941a562e;
status=500;
">>,{error,{throw,{checksum_check_error,23},[{chef_wm_named_sandbox,validate_checksums_uploaded,2,[{file,"src/chef_wm_named_sandbox.erl"},{line,142}]},{chef_wm_named_sandbox,from_json,2,[{file,"src/chef_wm_named_sandbox.erl"},{line,97}]},{webmachine_resource,resource_call,3,[{file,"src/webmachine_resource.erl"},{line,186}]},{webmachine_resource,do,3,[{file,"src/webmachine_resource.erl"},{line,142}]},{webmachine_decision_core,resource_call,1,[{file,"src/webmachine_decision_core.erl"},{line,48}]},{webmachine_decision_core,accept_helper,1,[{file,"src/webmachine_decision_core.erl"},{line,616}]},{webmachine_decision_core,decision,1,[{file,"src/webmachine_decision_core.erl"},{line,521}]},{webmachine_decision_core,handle_request,2,[{file,"src/webmachine_decision_core.erl"},{line,33}]}]}}}

==> /var/log/opscode/opscode-erchef/requests.log.1 <==

2015-05-14T17:12:43Z erchef@127.0.0.1 method=PUT;
path=/organizations/metrodigi/sandboxes/d81dca36ec749108cdfdcab1941a562e;
status=500; req_id=g3IAA2QAEGVyY2hlZkAxMjcuMC4wLjEDAAJViQAAAAMAAAAA;
org_name=metrodigi; policies=false; couchdb_groups=false;
couchdb_organizations=false; couchdb_containers=false; couchdb_acls=false;
503_mode=false; couchdb_associations=false;
couchdb_association_requests=false; req_time=10034; rdbms_time=17;
rdbms_count=5; authz_time=6; authz_count=1; s3_time=10002; s3_count=1;
user=mlindsay; req_api_version=0;

On Thu, May 14, 2015 at 10:11 AM, Michael Lindsay mlindsay@metrodigi.com
wrote:

A little more about my set up.

The chef server installed by solo is 12.0.8 running on Ubuntu 14.04. As I
mentioned previously, the only thing special that I did was sudo chmod 0644 /etc/opscode/pivotal.pem in order to get chef-server-ctl to run.

I'm running The client I am using was previously pointing chef server
11.1.3. And in fact, I'm using the -c option to load a vpc config file
(for both knife and berks). I know that one difference now (could just be
how it was set up the first time) is that I need to add
'/organizations/ORGNAME' to my vpc_knife.rb file. With this in place I can
knife ssl check -c ~/.chef/vpc_knife.rb and get a client list. I'm still
trying to tweak the berkshelf settings.

On Wed, May 13, 2015 at 3:25 PM, Michael Lindsay mlindsay@metrodigi.com
wrote:

So, because all of my cookbooks depend on other cookbooks had to find one
that I depended on that nothing else did so...

1. knife cookbook upload

Banks-2:md-chef mlindsay$ git clone git@github.com:
opscode-cookbooks/users.git

Cloning into 'users'...

remote: Counting objects: 468, done.

remote: Total 468 (delta 0), reused 0 (delta 0), pack-reused 468

Receiving objects: 100% (468/468), 94.43 KiB | 0 bytes/s, done.

Resolving deltas: 100% (176/176), done.

Checking connectivity... done.

Banks-2:md-chef mlindsay$ knife cookbook upload users -c ~/.chef/knife.rb

ERROR: knife encountered an unexpected error

This may be a bug in the 'cookbook upload' knife command or plugin

Please collect the output of this command with the -VV option before
filing a bug report.

Exception: NoMethodError: undefined method `source_url' for
#Chef::Cookbook::Metadata:0x007fd59a168ee8

Banks-2:md-chef mlindsay$ knife cookbook upload users -c ~/.chef/knife.rb
-VV

DEBUG: No chefignore file found at /Users/mlindsay/chefignore no files
will be ignored

/Users/mlindsay/md-chef/users/metadata.rb:8:in from_file': undefined method source_url' for #Chef::Cookbook::Metadata:0x007f9751ace960
(NoMethodError)

from /opt/chefdk/embedded/apps/chef/lib/chef/mixin/from_file.rb:30:in
`instance_eval'

from /opt/chefdk/embedded/apps/chef/lib/chef/mixin/from_file.rb:30:in
`from_file'

from
/opt/chefdk/embedded/apps/chef/lib/chef/cookbook/cookbook_version_loader.rb:172:in
`apply_ruby_metadata'

from
/opt/chefdk/embedded/apps/chef/lib/chef/cookbook/cookbook_version_loader.rb:110:in
`block in metadata'

from
/opt/chefdk/embedded/apps/chef/lib/chef/cookbook/cookbook_version_loader.rb:107:in
`each'

from
/opt/chefdk/embedded/apps/chef/lib/chef/cookbook/cookbook_version_loader.rb:107:in
`metadata'

from
/opt/chefdk/embedded/apps/chef/lib/chef/cookbook/cookbook_version_loader.rb:99:in
`block in cookbook_version'

from
/opt/chefdk/embedded/apps/chef/lib/chef/cookbook/cookbook_version_loader.rb:88:in
`tap'

from
/opt/chefdk/embedded/apps/chef/lib/chef/cookbook/cookbook_version_loader.rb:88:in
`cookbook_version'

from /opt/chefdk/embedded/apps/chef/lib/chef/cookbook_loader.rb:95:in
`load_cookbook'

from /opt/chefdk/embedded/apps/chef/lib/chef/cookbook_loader.rb:103:in
`'

from
/opt/chefdk/embedded/apps/chef/lib/chef/knife/cookbook_upload.rb:171:in
`block in cookbooks_to_upload'

from
/opt/chefdk/embedded/apps/chef/lib/chef/knife/cookbook_upload.rb:168:in
`each'

from
/opt/chefdk/embedded/apps/chef/lib/chef/knife/cookbook_upload.rb:168:in
`cookbooks_to_upload'

from
/opt/chefdk/embedded/apps/chef/lib/chef/knife/cookbook_upload.rb:206:in
`warn_about_cookbook_shadowing'

from
/opt/chefdk/embedded/apps/chef/lib/chef/knife/cookbook_upload.rb:96:in `run'

from /opt/chefdk/embedded/apps/chef/lib/chef/knife.rb:493:in
`run_with_pretty_exceptions'

from /opt/chefdk/embedded/apps/chef/lib/chef/knife.rb:174:in `run'

from /opt/chefdk/embedded/apps/chef/lib/chef/application/knife.rb:139:in
`run'

from /opt/chefdk/embedded/apps/chef/bin/knife:25:in `<top (required)>'

from /opt/chefdk/bin/knife:34:in `load'

from /opt/chefdk/bin/knife:34:in `'

root@chef:/home/ubuntu# chef-server-ctl status

run: bookshelf: (pid 6463) 14882s; run: log: (pid 5272) 15207s

AUTO-GENERATED BY CHEF - DO NOT MODIFY

topology "standalone"

api_fqdn "chef.DOMAIN.com"

On Wed, May 13, 2015 at 1:48 PM, Stephen Delano stephen@opscode.com
wrote:

Thanks Michael,

A few more things:

• Does the same error (or at least a similar failure) occur when you run
  a knife cookbook upload?
• From the Chef Server, what is the output of the chef-server-ctl status command? Is the bookshelf service running? I expect it is if you’ve
  gotten this far.
• What are the contents of your /etc/opscode/chef-server.rb
  configuration file (on the Chef Server)?

Thanks!

—
Stephen Delano - Engineering Lead, Chef

On Wed, May 13, 2015 at 1:27 PM, Michael Lindsay <mlindsay@metrodigi.com

wrote:

Hey Stephen,

Thanks for getting back to me... Is this the kind of thing you are
looking for?

==> /var/log/opscode/opscode-erchef/crash.log <==

2015-05-13 20:17:17 =ERROR REPORT====

Checking presence of checksum: <<"5b259a23bcccc6be76de01d811162d36">>
for org <<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has
taken longer than 5000 ms

2015-05-13 20:17:17 =ERROR REPORT====

Checking presence of checksum: <<"668e14353b42023b737efa15b3c675b9">>
for org <<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has
taken longer than 5000 ms

==> /var/log/opscode/opscode-erchef/current <==

2015-05-13_20:17:17.33628 [error] Checking presence of checksum:
<<"5b259a23bcccc6be76de01d811162d36">> for org
<<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has taken
longer than 5000 ms

2015-05-13_20:17:17.33629

==> /var/log/opscode/opscode-erchef/erchef.log <==

2015-05-13 20:17:17.330 [error] Checking presence of checksum:
<<"668e14353b42023b737efa15b3c675b9">> for org
<<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has taken
longer than 5000 ms

followed by this

==> /var/log/opscode/opscode-erchef/crash.log <==

2015-05-13 20:17:22 =ERROR REPORT====

{<<"method=PUT;
path=/organizations/metrodigi/sandboxes/d81dca36ec74c00114c5f3438e323f7d;
status=500;
">>,{error,{throw,{checksum_check_error,23},[{chef_wm_named_sandbox,validate_checksums_uploaded,2,[{file,"src/chef_wm_named_sandbox.erl"},{line,142}]},{chef_wm_named_sandbox,from_json,2,[{file,"src/chef_wm_named_sandbox.erl"},{line,97}]},{webmachine_resource,resource_call,3,[{file,"src/webmachine_resource.erl"},{line,186}]},{webmachine_resource,do,3,[{file,"src/webmachine_resource.erl"},{line,142}]},{webmachine_decision_core,resource_call,1,[{file,"src/webmachine_decision_core.erl"},{line,48}]},{webmachine_decision_core,accept_helper,1,[{file,"src/webmachine_decision_core.erl"},{line,616}]},{webmachine_decision_core,decision,1,[{file,"src/webmachine_decision_core.erl"},{line,521}]},{webmachine_decision_core,handle_request,2,[{file,"src/webmachine_decision_core.erl"},{line,33}]}]}}}

Here's a note, during my install, in order to get chef-server-ctl to
work I had to:

sudo chmod 0644 /etc/opscode/pivotal.pem

should I have not used sudo when I: chef-solo -o
'recipe[chef-server::default]' -j dna.json ?

Thanks,

• Michael

On Wed, May 13, 2015 at 1:00 PM, Stephen Delano stephen@opscode.com
wrote:

Hi Michael,

The error {“error”:[“internal service error”]} is coming from the
Chef Server, so the rest of that output is just Berks sending you a
stacktrace.

In order to get closer to the root of the issue, you’ll need to
provide some logs from the Chef Server. From the stacktrace, this appears
to be happening when making a cookbook upload request:
https://github.com/berkshelf/berkshelf/blob/master/lib/berkshelf/uploader.rb#L55-L60

You can find the relevant logs from the Chef Server in
/var/log/opscode/opscode-erchef/requests.log.*

To filter them down, you should be able to grep for status=500, which
is the corresponding error code. A message about what failed should be in
the erchef log and should help us track down the issue.

Thanks!

—
Stephen Delano - Engineering Lead, Chef

On Wed, May 13, 2015 at 12:19 PM, Michael Lindsay <
mlindsay@metrodigi.com> wrote:

Hey All,

I'm just setting up a new chef server (inside a vpc) and I'm having
trouble uploading the cookbooks with berks from my workstation:

Banks-2:base mlindsay$ berks upload -c ~/.berkshelf/vpc.json

E, [2015-05-13T12:14:00.542111 #16190] ERROR -- :
Ridley::Errors::HTTPInternalServerError: {"error":["internal service
error"]}

E, [2015-05-13T12:14:00.542241 #16190] ERROR -- :
/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/middleware/chef_response.rb:24:in
`on_complete'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:9:in
`block in call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:57:in
`on_complete'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:8:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:8:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/response.rb:8:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/middleware/chef_auth.rb:74:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/request/retry.rb:87:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/rack_builder.rb:139:in
`build_response'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/connection.rb:377:in
`run_request'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/connection.rb:104:in
`run_request'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/faraday-0.9.0/lib/faraday/connection.rb:177:in
`put'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`public_send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in
`block in invoke'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_thread.rb:21:in
`block in create'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/thread_handle.rb:13:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor_system.rb:32:in
`block in get_thread'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in
`block in create'

(celluloid):0:in `remote procedure call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in
`value'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
`method_missing'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/cell_proxy.rb:17:in
`send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/pool_manager.rb:41:in
`send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/pool_manager.rb:140:in
`method_missing'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`public_send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in
`block in invoke'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:15:in
`block in create'

(celluloid):0:in `remote procedure call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in
`value'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
`method_missing'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resource.rb:132:in
`raw_request'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resource.rb:123:in
`request'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resources/sandbox_resource.rb:46:in
`commit'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`public_send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in
`block in invoke'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_fiber.rb:15:in
`block in create'

(celluloid):0:in `remote procedure call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in
`value'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
`method_missing'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/chef_objects/sandbox_object.rb:53:in
`commit'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/resources/cookbook_resource.rb:227:in
`upload'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`public_send'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:26:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:63:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:60:in
`block in invoke'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/cell.rb:71:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor.rb:357:in
`block in task'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks.rb:57:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/tasks/task_thread.rb:21:in
`block in create'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/thread_handle.rb:13:in
`block in initialize'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/actor_system.rb:32:in
`block in get_thread'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in
`call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/internal_pool.rb:130:in
`block in create'

(celluloid):0:in `remote procedure call'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/calls.rb:92:in
`value'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/celluloid-0.16.0/lib/celluloid/proxies/sync_proxy.rb:33:in
`method_missing'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:54:in
`block (2 levels) in upload'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:50:in
`each'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:50:in
`block in upload'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley/client.rb:36:in
`open'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/ridley-4.0.0/lib/ridley.rb:51:in
`open'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf.rb:157:in
`ridley_connection'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:49:in
`upload'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/uploader.rb:36:in
`run'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/berksfile.rb:542:in
`upload'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cli.rb:208:in
`upload'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/command.rb:27:in
`run'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/invocation.rb:126:in
`invoke_command'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor.rb:359:in
`dispatch'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cli.rb:52:in
`dispatch'

/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/base.rb:440:in
`start'

/opt/chefdk/embedded/apps/berkshelf/lib/berkshelf/cli.rb:27:in
`execute!'

/opt/chefdk/embedded/apps/berkshelf/bin/berks:5:in `<top (required)>'

/opt/chefdk/bin/berks:37:in `load'

/opt/chefdk/bin/berks:37:in `'

I haven't had any luck googling the error for a solution.

Any thoughts?

• Michael

[kallistec]

On Thursday, May 14, 2015 at 10:19 AM, Michael Lindsay wrote:

I have also tried setting my new configuration to be the default knife.rb and renamed config.json in my .berkshelf directory. Same error:

==> /var/log/opscode/opscode-erchef/current <==
2015-05-14_17:12:38.73280 [error] Checking presence of checksum: <<"5b259a23bcccc6be76de01d811162d36">> for org <<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has taken longer than 5000 ms
...
2015-05-14_17:12:43.74133 [error] Checking presence of checksum: <<"471f01912a5109386225389563db4159">> for org <<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has taken longer than 5000 ms
2015-05-14_17:12:43.74134

This is definitely your problem, it could be any number of things. If you run chef-server-ctl tail, it will tail the logs of every component. You should see a request to the bookshelf application, which is one possible source of the issue. If you have some super aggressive firewall or something, it’s possible that the same behavior could be caused by the TCP packets from the erchef service to the bookshelf service simply getting dropped, which would hang for a long time attempting (and failing) to set up the TCP connection. Your normal network debugging techniques could help determine if this is your issue.

--
Daniel DeLeo

[Michael_Lindsay]

Hey Daniel,

Thanks for your input...
With chef-server-ctl i'm now seeing entries for nginx and oc_bifrost as
well, but still challenging to find the root cause of the problem. As far
as I can tell, it looks like chef is sending back adequate responses to the
client.

==> /var/log/opscode/nginx/access.log <==
50.193.20.201 - - [14/May/2015:17:42:27 +0000] "GET
/organizations/metrodigi/cookbooks/apt/2.6.1 HTTP/1.1" 404 "0.019" 65 "-"
"Ridley v4.0.0" "127.0.0.1:8000" "404" "0.019" "11.4.0"
"algorithm=sha1;version=1.0;" "mlindsay" "2015-05-14T17:42:28Z"
"2jmj7l5rSw0yVb/vlWAYkK/YBwk=" 949
50.193.20.201 - - [14/May/2015:17:42:29 +0000] "POST
/organizations/metrodigi/sandboxes HTTP/1.1" 201 "0.199" 7391 "-" "Ridley
v4.0.0" "127.0.0.1:8000" "201" "0.199" "11.4.0"
"algorithm=sha1;version=1.0;" "mlindsay" "2015-05-14T17:42:30Z"
"UiN/Daaqc12o4YUmmZuhSKOSn60=" 1896
50.193.20.201 - - [14/May/2015:17:42:29 +0000] "PUT
/bookshelf/organization-f03b7e8dadaf3bb09705d81dca36ec74/checksum-c4dea3dbb103ce5203726dbf60289da4?AWSAccessKeyId=8c0a17c966f9c0df8783049623b56f39cfcfbdc4&Expires=1431626249&Signature=%2FHIhr0cDXbNH00NS9RtNcaP1ygM%3D
HTTP/1.1" 204 "0.041" 0 "-" "Faraday v0.9.0" "127.0.0.1:4321" "204" "0.006"
"11.4.0" "algorithm=sha1;version=1.0;" "mlindsay" "2015-05-14T17:42:30Z"
"zJPbL3MjEPno/dkNXQ9xNIXGWwk=" 2275

As far as the network is concerned, We're moving into and Amazon VPC so, it
is the thing most likely to be broken, Aside from allowing inbound access
to 443, which is clearly happening and outbound access to everything, is
there anything else I should look for specifically? And by 'normal network
debugging techniques' are you talking about something like, wireshark?

TIA,

• Michael

On Thu, May 14, 2015 at 10:24 AM, Daniel DeLeo dan@kallistec.com wrote:

On Thursday, May 14, 2015 at 10:19 AM, Michael Lindsay wrote:

I have also tried setting my new configuration to be the default
knife.rb and renamed config.json in my .berkshelf directory. Same error:

==> /var/log/opscode/opscode-erchef/current <==
2015-05-14_17:12:38.73280 [error] Checking presence of checksum:
<<"5b259a23bcccc6be76de01d811162d36">> for org
<<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has taken
longer than 5000 ms
...
2015-05-14_17:12:43.74133 [error] Checking presence of checksum:
<<"471f01912a5109386225389563db4159">> for org
<<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has taken
longer than 5000 ms
2015-05-14_17:12:43.74134

This is definitely your problem, it could be any number of things. If you
run chef-server-ctl tail, it will tail the logs of every component. You
should see a request to the bookshelf application, which is one possible
source of the issue. If you have some super aggressive firewall or
something, it’s possible that the same behavior could be caused by the TCP
packets from the erchef service to the bookshelf service simply getting
dropped, which would hang for a long time attempting (and failing) to set
up the TCP connection. Your normal network debugging techniques could help
determine if this is your issue.

--
Daniel DeLeo

[Michael_Lindsay]

So I'm tailing the logs during restart. Could this be anything:
==> /var/log/opscode/opscode-expander/current <==
2015-05-14_18:21:22.65634 log writing failed. can't be called from trap
context
2015-05-14_18:21:22.97614 [Thu, 14 May 2015 18:21:22 +0000] INFO: Stopping
worker 1 (PID: 9232)
2015-05-14_18:21:22.97617 [Thu, 14 May 2015 18:21:22 +0000] INFO: Stopping
worker 2 (PID: 9235)
2015-05-14_18:21:22.97617 log writing failed. can't be called from trap
context
2015-05-14_18:21:22.97617
/opt/opscode/embedded/service/gem/ruby/2.1.0/gems/amqp-0.6.7/lib/mq.rb:246:in
`synchronize': can't be called from trap context (ThreadError)

Looking through the logs at anything I might not be able to access locally
doesn't seem to be yielding any fruit either:

$ curl http://0.0.0.0:9090

You are being redirected.

$ telnet 127.0.0.1 4321
Trying 127.0.0.1...
Connected to 127.0.0.1.

$ telnet 127.0.0.1 16379
Trying 127.0.0.1...
Connected to 127.0.0.1.

$ telnet 127.0.0.1 5672
Trying 127.0.0.1...
Connected to 127.0.0.1.

Does anything external have to talk to any of these ports?

My best guess about where things might be starting to go wrong is this
request, or the one after it:

2015-05-14T18:41:52Z oc_bifrost@127.0.0.1 method=GET;
path=/containers/0e8a6fe4d394239f11394f32cd67065e/acl/create/actors/29b471de57bab8c221569918c41056e9;
status=200; requestor_id=29b471de57bab8c221569918c41056e9; req_time=4;
rdbms.bifrost_db.has_permission_time=2;
rdbms.bifrost_db.has_permission_count=1; rdbms.bifrost_db.exists_time=0;
rdbms.bifrost_db.exists_count=1;

or this request:

PUT /organizations/metrodigi/sandboxes/d81dca36ec7445c7773a76ce4362f2a6
HTTP/1.1" 500

Is it failing to create an organization sandbox? If so, any ideas what
might cause that?

On Thu, May 14, 2015 at 11:05 AM, Michael Lindsay mlindsay@metrodigi.com
wrote:

Hey Daniel,

Thanks for your input...
With chef-server-ctl i'm now seeing entries for nginx and oc_bifrost as
well, but still challenging to find the root cause of the problem. As far
as I can tell, it looks like chef is sending back adequate responses to the
client.

==> /var/log/opscode/nginx/access.log <==
50.193.20.201 - - [14/May/2015:17:42:27 +0000] "GET
/organizations/metrodigi/cookbooks/apt/2.6.1 HTTP/1.1" 404 "0.019" 65 "-"
"Ridley v4.0.0" "127.0.0.1:8000" "404" "0.019" "11.4.0"
"algorithm=sha1;version=1.0;" "mlindsay" "2015-05-14T17:42:28Z"
"2jmj7l5rSw0yVb/vlWAYkK/YBwk=" 949
50.193.20.201 - - [14/May/2015:17:42:29 +0000] "POST
/organizations/metrodigi/sandboxes HTTP/1.1" 201 "0.199" 7391 "-" "Ridley
v4.0.0" "127.0.0.1:8000" "201" "0.199" "11.4.0"
"algorithm=sha1;version=1.0;" "mlindsay" "2015-05-14T17:42:30Z"
"UiN/Daaqc12o4YUmmZuhSKOSn60=" 1896
50.193.20.201 - - [14/May/2015:17:42:29 +0000] "PUT
/bookshelf/organization-f03b7e8dadaf3bb09705d81dca36ec74/checksum-c4dea3dbb103ce5203726dbf60289da4?AWSAccessKeyId=8c0a17c966f9c0df8783049623b56f39cfcfbdc4&Expires=1431626249&Signature=%2FHIhr0cDXbNH00NS9RtNcaP1ygM%3D
HTTP/1.1" 204 "0.041" 0 "-" "Faraday v0.9.0" "127.0.0.1:4321" "204"
"0.006" "11.4.0" "algorithm=sha1;version=1.0;" "mlindsay"
"2015-05-14T17:42:30Z" "zJPbL3MjEPno/dkNXQ9xNIXGWwk=" 2275

As far as the network is concerned, We're moving into and Amazon VPC so,
it is the thing most likely to be broken, Aside from allowing inbound
access to 443, which is clearly happening and outbound access to
everything, is there anything else I should look for specifically? And by
'normal network debugging techniques' are you talking about something like,
wireshark?

TIA,

• Michael

On Thu, May 14, 2015 at 10:24 AM, Daniel DeLeo dan@kallistec.com wrote:

On Thursday, May 14, 2015 at 10:19 AM, Michael Lindsay wrote:

I have also tried setting my new configuration to be the default
knife.rb and renamed config.json in my .berkshelf directory. Same error:

==> /var/log/opscode/opscode-erchef/current <==
2015-05-14_17:12:38.73280 [error] Checking presence of checksum:
<<"5b259a23bcccc6be76de01d811162d36">> for org
<<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has taken
longer than 5000 ms
...
2015-05-14_17:12:43.74133 [error] Checking presence of checksum:
<<"471f01912a5109386225389563db4159">> for org
<<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has taken
longer than 5000 ms
2015-05-14_17:12:43.74134

This is definitely your problem, it could be any number of things. If you
run chef-server-ctl tail, it will tail the logs of every component. You
should see a request to the bookshelf application, which is one possible
source of the issue. If you have some super aggressive firewall or
something, it’s possible that the same behavior could be caused by the TCP
packets from the erchef service to the bookshelf service simply getting
dropped, which would hang for a long time attempting (and failing) to set
up the TCP connection. Your normal network debugging techniques could help
determine if this is your issue.

--
Daniel DeLeo

[Michael_Lindsay]

I seem to have now disabled my ability to upload cookbooks to either of my
chef servers with berkshelf...

I've put the backup of my old knife.rb file back, moved everything out of
.chef that was intended for the configuration of the new server, copied my
.berkshelf file out of my home directory to re-initalize. I can do berks
update, but berks install gives me the following error:

Ridley::Errors::FromFileParserError Could not parse
/var/folders/w5/1k5n_41n3fg_ph5nc4k3xwn80000gp/T/d20150514-31290-1w5vk7n/metadata.rb': undefined method source_url' for
#Ridley::Chef::Cookbook::Metadata:0x007fca2d03bf70

I've just officially screwed myself, and I don't know how, and I don't know
how to fix it. Please Help!

TIA,

• Michael

On Thu, May 14, 2015 at 11:46 AM, Michael Lindsay mlindsay@metrodigi.com
wrote:

So I'm tailing the logs during restart. Could this be anything:
==> /var/log/opscode/opscode-expander/current <==
2015-05-14_18:21:22.65634 log writing failed. can't be called from trap
context
2015-05-14_18:21:22.97614 [Thu, 14 May 2015 18:21:22 +0000] INFO: Stopping
worker 1 (PID: 9232)
2015-05-14_18:21:22.97617 [Thu, 14 May 2015 18:21:22 +0000] INFO: Stopping
worker 2 (PID: 9235)
2015-05-14_18:21:22.97617 log writing failed. can't be called from trap
context
2015-05-14_18:21:22.97617
/opt/opscode/embedded/service/gem/ruby/2.1.0/gems/amqp-0.6.7/lib/mq.rb:246:in
`synchronize': can't be called from trap context (ThreadError)

Looking through the logs at anything I might not be able to access locally
doesn't seem to be yielding any fruit either:

$ curl http://0.0.0.0:9090

You are being redirected.

$ telnet 127.0.0.1 4321
Trying 127.0.0.1...
Connected to 127.0.0.1.

$ telnet 127.0.0.1 16379
Trying 127.0.0.1...
Connected to 127.0.0.1.

$ telnet 127.0.0.1 5672
Trying 127.0.0.1...
Connected to 127.0.0.1.

Does anything external have to talk to any of these ports?

My best guess about where things might be starting to go wrong is this
request, or the one after it:

2015-05-14T18:41:52Z oc_bifrost@127.0.0.1 method=GET;
path=/containers/0e8a6fe4d394239f11394f32cd67065e/acl/create/actors/29b471de57bab8c221569918c41056e9;
status=200; requestor_id=29b471de57bab8c221569918c41056e9; req_time=4;
rdbms.bifrost_db.has_permission_time=2;
rdbms.bifrost_db.has_permission_count=1; rdbms.bifrost_db.exists_time=0;
rdbms.bifrost_db.exists_count=1;

or this request:

PUT /organizations/metrodigi/sandboxes/d81dca36ec7445c7773a76ce4362f2a6
HTTP/1.1" 500

Is it failing to create an organization sandbox? If so, any ideas what
might cause that?

On Thu, May 14, 2015 at 11:05 AM, Michael Lindsay mlindsay@metrodigi.com
wrote:

Hey Daniel,

Thanks for your input...
With chef-server-ctl i'm now seeing entries for nginx and oc_bifrost as
well, but still challenging to find the root cause of the problem. As far
as I can tell, it looks like chef is sending back adequate responses to the
client.

==> /var/log/opscode/nginx/access.log <==
50.193.20.201 - - [14/May/2015:17:42:27 +0000] "GET
/organizations/metrodigi/cookbooks/apt/2.6.1 HTTP/1.1" 404 "0.019" 65 "-"
"Ridley v4.0.0" "127.0.0.1:8000" "404" "0.019" "11.4.0"
"algorithm=sha1;version=1.0;" "mlindsay" "2015-05-14T17:42:28Z"
"2jmj7l5rSw0yVb/vlWAYkK/YBwk=" 949
50.193.20.201 - - [14/May/2015:17:42:29 +0000] "POST
/organizations/metrodigi/sandboxes HTTP/1.1" 201 "0.199" 7391 "-" "Ridley
v4.0.0" "127.0.0.1:8000" "201" "0.199" "11.4.0"
"algorithm=sha1;version=1.0;" "mlindsay" "2015-05-14T17:42:30Z"
"UiN/Daaqc12o4YUmmZuhSKOSn60=" 1896
50.193.20.201 - - [14/May/2015:17:42:29 +0000] "PUT
/bookshelf/organization-f03b7e8dadaf3bb09705d81dca36ec74/checksum-c4dea3dbb103ce5203726dbf60289da4?AWSAccessKeyId=8c0a17c966f9c0df8783049623b56f39cfcfbdc4&Expires=1431626249&Signature=%2FHIhr0cDXbNH00NS9RtNcaP1ygM%3D
HTTP/1.1" 204 "0.041" 0 "-" "Faraday v0.9.0" "127.0.0.1:4321" "204"
"0.006" "11.4.0" "algorithm=sha1;version=1.0;" "mlindsay"
"2015-05-14T17:42:30Z" "zJPbL3MjEPno/dkNXQ9xNIXGWwk=" 2275

As far as the network is concerned, We're moving into and Amazon VPC so,
it is the thing most likely to be broken, Aside from allowing inbound
access to 443, which is clearly happening and outbound access to
everything, is there anything else I should look for specifically? And by
'normal network debugging techniques' are you talking about something like,
wireshark?

TIA,

• Michael

On Thu, May 14, 2015 at 10:24 AM, Daniel DeLeo dan@kallistec.com wrote:

On Thursday, May 14, 2015 at 10:19 AM, Michael Lindsay wrote:

I have also tried setting my new configuration to be the default
knife.rb and renamed config.json in my .berkshelf directory. Same error:

==> /var/log/opscode/opscode-erchef/current <==
2015-05-14_17:12:38.73280 [error] Checking presence of checksum:
<<"5b259a23bcccc6be76de01d811162d36">> for org
<<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has taken
longer than 5000 ms
...
2015-05-14_17:12:43.74133 [error] Checking presence of checksum:
<<"471f01912a5109386225389563db4159">> for org
<<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has taken
longer than 5000 ms
2015-05-14_17:12:43.74134

This is definitely your problem, it could be any number of things. If
you run chef-server-ctl tail, it will tail the logs of every component.
You should see a request to the bookshelf application, which is one
possible source of the issue. If you have some super aggressive firewall or
something, it’s possible that the same behavior could be caused by the TCP
packets from the erchef service to the bookshelf service simply getting
dropped, which would hang for a long time attempting (and failing) to set
up the TCP connection. Your normal network debugging techniques could help
determine if this is your issue.

--
Daniel DeLeo

[Michael_Lindsay]

Sorry, more spamming my own thread....

When I do berks update, I've started getting this error:

Fetching 'nodejs' from git@github.com:redguide/nodejs.git (at master)
Ridley::Errors::FromFileParserError Could not parse
/var/folders/w5/1k5n_41n3fg_ph5nc4k3xwn80000gp/T/d20150514-32474-17tpmlb/metadata.rb': undefined method source_url' for
#Ridley::Chef::Cookbook::Metadata:0x007f915c907de8

My colleague, who's knife/berks configuration do not appear to be totally
broken is getting this:

Fetching 'nodejs' from git@github.com:redguide/nodejs.git (at master)
The cookbook downloaded for nodejs (~> 2.2.0) did not satisfy the
constraint.

Not clear why this problem is cropping up now.

On Thu, May 14, 2015 at 1:54 PM, Michael Lindsay mlindsay@metrodigi.com
wrote:

I seem to have now disabled my ability to upload cookbooks to either of my
chef servers with berkshelf...

I've put the backup of my old knife.rb file back, moved everything out of
.chef that was intended for the configuration of the new server, copied my
.berkshelf file out of my home directory to re-initalize. I can do berks
update, but berks install gives me the following error:

Ridley::Errors::FromFileParserError Could not parse
/var/folders/w5/1k5n_41n3fg_ph5nc4k3xwn80000gp/T/d20150514-31290-1w5vk7n/metadata.rb': undefined method source_url' for
#Ridley::Chef::Cookbook::Metadata:0x007fca2d03bf70

I've just officially screwed myself, and I don't know how, and I don't
know how to fix it. Please Help!

TIA,

• Michael

On Thu, May 14, 2015 at 11:46 AM, Michael Lindsay mlindsay@metrodigi.com
wrote:

So I'm tailing the logs during restart. Could this be anything:
==> /var/log/opscode/opscode-expander/current <==
2015-05-14_18:21:22.65634 log writing failed. can't be called from trap
context
2015-05-14_18:21:22.97614 [Thu, 14 May 2015 18:21:22 +0000] INFO:
Stopping worker 1 (PID: 9232)
2015-05-14_18:21:22.97617 [Thu, 14 May 2015 18:21:22 +0000] INFO:
Stopping worker 2 (PID: 9235)
2015-05-14_18:21:22.97617 log writing failed. can't be called from trap
context
2015-05-14_18:21:22.97617
/opt/opscode/embedded/service/gem/ruby/2.1.0/gems/amqp-0.6.7/lib/mq.rb:246:in
`synchronize': can't be called from trap context (ThreadError)

Looking through the logs at anything I might not be able to access
locally doesn't seem to be yielding any fruit either:

$ curl http://0.0.0.0:9090

You are being redirected.

$ telnet 127.0.0.1 4321
Trying 127.0.0.1...
Connected to 127.0.0.1.

$ telnet 127.0.0.1 16379
Trying 127.0.0.1...
Connected to 127.0.0.1.

$ telnet 127.0.0.1 5672
Trying 127.0.0.1...
Connected to 127.0.0.1.

Does anything external have to talk to any of these ports?

My best guess about where things might be starting to go wrong is this
request, or the one after it:

2015-05-14T18:41:52Z oc_bifrost@127.0.0.1 method=GET;
path=/containers/0e8a6fe4d394239f11394f32cd67065e/acl/create/actors/29b471de57bab8c221569918c41056e9;
status=200; requestor_id=29b471de57bab8c221569918c41056e9; req_time=4;
rdbms.bifrost_db.has_permission_time=2;
rdbms.bifrost_db.has_permission_count=1; rdbms.bifrost_db.exists_time=0;
rdbms.bifrost_db.exists_count=1;

or this request:

PUT /organizations/metrodigi/sandboxes/d81dca36ec7445c7773a76ce4362f2a6
HTTP/1.1" 500

Is it failing to create an organization sandbox? If so, any ideas what
might cause that?

On Thu, May 14, 2015 at 11:05 AM, Michael Lindsay <mlindsay@metrodigi.com

wrote:

Hey Daniel,

Thanks for your input...
With chef-server-ctl i'm now seeing entries for nginx and oc_bifrost as
well, but still challenging to find the root cause of the problem. As far
as I can tell, it looks like chef is sending back adequate responses to the
client.

==> /var/log/opscode/nginx/access.log <==
50.193.20.201 - - [14/May/2015:17:42:27 +0000] "GET
/organizations/metrodigi/cookbooks/apt/2.6.1 HTTP/1.1" 404 "0.019" 65 "-"
"Ridley v4.0.0" "127.0.0.1:8000" "404" "0.019" "11.4.0"
"algorithm=sha1;version=1.0;" "mlindsay" "2015-05-14T17:42:28Z"
"2jmj7l5rSw0yVb/vlWAYkK/YBwk=" 949
50.193.20.201 - - [14/May/2015:17:42:29 +0000] "POST
/organizations/metrodigi/sandboxes HTTP/1.1" 201 "0.199" 7391 "-" "Ridley
v4.0.0" "127.0.0.1:8000" "201" "0.199" "11.4.0"
"algorithm=sha1;version=1.0;" "mlindsay" "2015-05-14T17:42:30Z"
"UiN/Daaqc12o4YUmmZuhSKOSn60=" 1896
50.193.20.201 - - [14/May/2015:17:42:29 +0000] "PUT
/bookshelf/organization-f03b7e8dadaf3bb09705d81dca36ec74/checksum-c4dea3dbb103ce5203726dbf60289da4?AWSAccessKeyId=8c0a17c966f9c0df8783049623b56f39cfcfbdc4&Expires=1431626249&Signature=%2FHIhr0cDXbNH00NS9RtNcaP1ygM%3D
HTTP/1.1" 204 "0.041" 0 "-" "Faraday v0.9.0" "127.0.0.1:4321" "204"
"0.006" "11.4.0" "algorithm=sha1;version=1.0;" "mlindsay"
"2015-05-14T17:42:30Z" "zJPbL3MjEPno/dkNXQ9xNIXGWwk=" 2275

As far as the network is concerned, We're moving into and Amazon VPC so,
it is the thing most likely to be broken, Aside from allowing inbound
access to 443, which is clearly happening and outbound access to
everything, is there anything else I should look for specifically? And by
'normal network debugging techniques' are you talking about something like,
wireshark?

TIA,

• Michael

On Thu, May 14, 2015 at 10:24 AM, Daniel DeLeo dan@kallistec.com
wrote:

On Thursday, May 14, 2015 at 10:19 AM, Michael Lindsay wrote:

I have also tried setting my new configuration to be the default
knife.rb and renamed config.json in my .berkshelf directory. Same error:

==> /var/log/opscode/opscode-erchef/current <==
2015-05-14_17:12:38.73280 [error] Checking presence of checksum:
<<"5b259a23bcccc6be76de01d811162d36">> for org
<<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has taken
longer than 5000 ms
...
2015-05-14_17:12:43.74133 [error] Checking presence of checksum:
<<"471f01912a5109386225389563db4159">> for org
<<"f03b7e8dadaf3bb09705d81dca36ec74">> from bucket "bookshelf" has taken
longer than 5000 ms
2015-05-14_17:12:43.74134

This is definitely your problem, it could be any number of things. If
you run chef-server-ctl tail, it will tail the logs of every component.
You should see a request to the bookshelf application, which is one
possible source of the issue. If you have some super aggressive firewall or
something, it’s possible that the same behavior could be caused by the TCP
packets from the erchef service to the bookshelf service simply getting
dropped, which would hang for a long time attempting (and failing) to set
up the TCP connection. Your normal network debugging techniques could help
determine if this is your issue.

--
Daniel DeLeo

[kallistec]

On Thursday, May 14, 2015 at 2:28 PM, Michael Lindsay wrote:

Sorry, more spamming my own thread....

When I do berks update, I've started getting this error:
Fetching 'nodejs' from git@github.com (mailto:git@github.com):redguide/nodejs.git (at master)
Ridley::Errors::FromFileParserError Could not parse /var/folders/w5/1k5n_41n3fg_ph5nc4k3xwn80000gp/T/d20150514-32474-17tpmlb/metadata.rb': undefined method source_url' for #Ridley::Chef::Cookbook::Metadata:0x007f915c907de8

I think source_url is a new addition to cookbook metadata that isn’t yet supported by the version of berkshelf you have.

My colleague, who's knife/berks configuration do not appear to be totally broken is getting this:
Fetching 'nodejs' from git@github.com (mailto:git@github.com):redguide/nodejs.git (at master)
The cookbook downloaded for nodejs (~> 2.2.0) did not satisfy the constraint.

This happens when you have a version constraint somewhere that says the nodejs cookbook has to be version ~> 2.2.0, but you’re pulling it from github and the version there doesn’t match that constraint.

Not clear why this problem is cropping up now.

Neither of these seem related to your bookshelf problems, FWIW.

--
Daniel DeLeo

[Michael_Lindsay]

Hey Daniel,

Indeed, I had a number of things going on, among them an unpinned nodejs
dependency. I'm pretty well sorted now with our existing environment and
I'm going to set up another client machine from scratch to be the one I
talk to my new vpn chef server (until we cut over). Will update on success
or failure.

Thanks again!

• Michael

On Fri, May 15, 2015 at 10:27 AM, Daniel DeLeo dan@kallistec.com wrote:

On Thursday, May 14, 2015 at 2:28 PM, Michael Lindsay wrote:

Sorry, more spamming my own thread....

When I do berks update, I've started getting this error:
Fetching 'nodejs' from git@github.com (mailto:git@github.com):redguide/nodejs.git
(at master)
Ridley::Errors::FromFileParserError Could not parse
/var/folders/w5/1k5n_41n3fg_ph5nc4k3xwn80000gp/T/d20150514-32474-17tpmlb/metadata.rb': undefined method source_url' for
#Ridley::Chef::Cookbook::Metadata:0x007f915c907de8

I think source_url is a new addition to cookbook metadata that isn’t yet
supported by the version of berkshelf you have.

My colleague, who's knife/berks configuration do not appear to be
totally broken is getting this:
Fetching 'nodejs' from git@github.com (mailto:git@github.com):redguide/nodejs.git
(at master)
The cookbook downloaded for nodejs (~> 2.2.0) did not satisfy the
constraint.

This happens when you have a version constraint somewhere that says the
nodejs cookbook has to be version ~> 2.2.0, but you’re pulling it from
github and the version there doesn’t match that constraint.

Not clear why this problem is cropping up now.

Neither of these seem related to your bookshelf problems, FWIW.

--
Daniel DeLeo

[Michael_Lindsay]

Hey all,

So I now have a fresh install of chefDK (knife -v Chef: 12.3.0, berks
-v 3.2.4) and a brand new chef server (12.0.8). berks update is still
returning the same error. When I knife cookbook upload chef_handler -o ~/.berkshelf/cookbooks/, I get this:

ERROR: Server returned error 500 for
https://chef.DOMAIN.com/organizations/ORG/sandboxes/d9dcaaebaee9c6d69d55963db5afb9e4
Is this closer to the root of the problem?

On the server, I see the same erros as I do with berks:

2015-05-18_17:22:31.31151 [error] Checking presence of checksum:
<<"5c059b160b3d288c0796ee1c964e0ca6">> for org
<<"d324ac69246364aa17fdd9dcaaebaee9">> from bucket "bookshelf" has taken
longer than 5000 ms

Followed by this:

2015-05-18_16:51:45.70811 [error] {<<"method=PUT;
path=/organizations/ORG/sandboxes/d9dcaaebaee9e4daa96ffd80cb8245b5;
status=500;
">>,{error,{throw,{checksum_check_error,23},[{chef_wm_named_sandbox,validate_checksums_uploaded,2,[{file,"src/chef_wm_named_sandbox.erl"},{line,142}]},{chef_wm_named_sandbox,from_json,2,[{file,"src/chef_wm_named_sandbox.erl"},{line,97}]},{webmachine_resource,resource_call,3,[{file,"src/webmachine_resource.erl"},{line,186}]},{webmachine_resource,do,3,[{file,"src/webmachine_resource.erl"},{line,142}]},{webmachine_decision_core,resource_call,1,[{file,"src/webmachine_decision_core.erl"},{line,48}]},{webmachine_decision_core,accept_helper,1,[{file,"src/webmachine_decision_core.erl"},{line,616}]},{webmachine_decision_core,decision,1,[{file,"src/webmachine_decision_core.erl"},{line,521}]},{webmachine_decision_core,handle_request,2,[{file,"src/webmachine_decision_core.erl"},{line,33}]}]}}}

I can get to the chef manage site and knife ssl check is still happy. Any
thoughts on where I should look next? Is there any additional information
I can provide that will help?

• Michael

On Fri, May 15, 2015 at 11:32 AM, Michael Lindsay mlindsay@metrodigi.com
wrote:

Hey Daniel,

Indeed, I had a number of things going on, among them an unpinned nodejs
dependency. I'm pretty well sorted now with our existing environment and
I'm going to set up another client machine from scratch to be the one I
talk to my new vpn chef server (until we cut over). Will update on success
or failure.

Thanks again!

• Michael

On Fri, May 15, 2015 at 10:27 AM, Daniel DeLeo dan@kallistec.com wrote:

On Thursday, May 14, 2015 at 2:28 PM, Michael Lindsay wrote:

Sorry, more spamming my own thread....

When I do berks update, I've started getting this error:
Fetching 'nodejs' from git@github.com (mailto:git@github.com):redguide/nodejs.git
(at master)
Ridley::Errors::FromFileParserError Could not parse
/var/folders/w5/1k5n_41n3fg_ph5nc4k3xwn80000gp/T/d20150514-32474-17tpmlb/metadata.rb': undefined method source_url' for
#Ridley::Chef::Cookbook::Metadata:0x007f915c907de8

I think source_url is a new addition to cookbook metadata that isn’t
yet supported by the version of berkshelf you have.

My colleague, who's knife/berks configuration do not appear to be
totally broken is getting this:
Fetching 'nodejs' from git@github.com (mailto:git@github.com):redguide/nodejs.git
(at master)
The cookbook downloaded for nodejs (~> 2.2.0) did not satisfy the
constraint.

This happens when you have a version constraint somewhere that says the
nodejs cookbook has to be version ~> 2.2.0, but you’re pulling it from
github and the version there doesn’t match that constraint.

Not clear why this problem is cropping up now.

Neither of these seem related to your bookshelf problems, FWIW.

--
Daniel DeLeo

[Michael_Lindsay]

Found and fixed!

I’d limited my security group to only accept inbound from office IP for
ports 80,443 and 9683 from our office IP adddress. Opening it up to all IP
addresses has resolved the problem. Would like to tighten this up at some
point (should I add it’s own internal ip address? 127.0.0.1/32?)

Thanks helping out Stephan and Daniel!

• Michael

[kallistec]

On Monday, May 18, 2015 at 12:57 PM, Michael Lindsay wrote:

Found and fixed!

I'd limited my security group to only accept inbound from office IP for ports 80,443 and 9683 from our office IP adddress. Opening it up to all IP addresses has resolved the problem. Would like to tighten this up at some point (should I add it's own internal ip address? 127.0.0.1/32 (http://127.0.0.1/32)?)

Thanks helping out Stephan and Daniel!

• Michael
  You can see the active settings for the Chef Server in /etc/opscode/chef-server-running.json. On my system, bookshelf is configured for 127.0.0.1, port 4321. If that is blocked, the server won’t work. But you shouldn’t need to open up that port to the world.

--
Daniel DeLeo

[Michael_Lindsay]

4321 is closed to the world (but I can telnet to localhost over it) on that
instance and everything is still working like a charm. The only publicly
available ports are 80,443 & 9683. Thanks for the info!

On Tue, May 19, 2015 at 11:17 AM, Daniel DeLeo dan@kallistec.com wrote:

On Monday, May 18, 2015 at 12:57 PM, Michael Lindsay wrote:

Found and fixed!

I'd limited my security group to only accept inbound from office IP for
ports 80,443 and 9683 from our office IP adddress. Opening it up to all IP
addresses has resolved the problem. Would like to tighten this up at some
point (should I add it's own internal ip address? 127.0.0.1/32 (
http://127.0.0.1/32)?)

Thanks helping out Stephan and Daniel!

• Michael
  You can see the active settings for the Chef Server in
  /etc/opscode/chef-server-running.json. On my system, bookshelf is
  configured for 127.0.0.1, port 4321. If that is blocked, the server won’t
  work. But you shouldn’t need to open up that port to the world.

--
Daniel DeLeo