Hi All,
We are running chef server on a Centos 5.5 machine.
We ran into a disk space issue yesterday and then what we did was to run
compaction . But we did take the backup of chef.couch .(7.6G) . Now we
restarted the chef server and the webui.pem was regenerated. After deleting
the current webui.pem we got knife to work and to our dismay, we found only
those nodes that we had when we had done the compaction last time , which
was well over a month back.
Now we wanted to restore it back . We copied the chef.couch back to its
place . Now when I login to the GUI of couchdb - I can see all the
environments , nodes and cookbooks . But can’t access it through knife . All
the processes seem to have started correctly but webui gives a HTTP 500.
Can’t get the chef-client to run on any box.
Even deleted webui.pem and after restarting, it didn’t regenerate the
webui.pem (In an earlier instance this had worked)
When I try knife configure --i and enter parameters, at the end I get this:
.
ERROR: Failed to authenticate to http://chefservercom:4000 as chef-webui
with key /etc/chef/webui.pem
Response: Failed to authenticate. Ensure that your client key is valid.
Logs on couchdb - I keep seeing “nil” for various documents.
[Tue, 30 Aug 2011 17:26:56 GMT] [info] [<0.761.2>] 127.0.0.1 - - ‘GET’
/chef/_design/id_map/_view/name_to_id?include_docs=true&key=[%22environment%22,%22qa%22]
200
[Tue, 30 Aug 2011 17:26:56 GMT] [debug] [<0.761.2>] Include Doc:
<<“16d8c737-df88-41a3-9030-6112ae0842b2”>> nil
[Tue, 30 Aug 2011 17:26:56 GMT] [debug] [<0.761.2>] Include Doc:
<<“c44bf7e2-62ae-41e5-89b8-b6f81f2b71c4”>> nil
Logs on chef-server -
merb : chef-server (api) : worker (port 4000) ~ undefined method
cookbook_versions' for nil:NilClass - (NoMethodError) --- We have multiple cookbook versions for certain cookbooks, so maybe this is not picking it up for some reason. /usr/lib/ruby/gems/1.8/gems/chef-0.10.4/lib/chef/environment.rb:377:incdb_minimal_filtered_versions’
/usr/lib/ruby/gems/1.8/gems/chef-0.10.4/lib/chef/cookbook_version_selector.rb:159:in
expand_to_cookbook_versions' /usr/lib/ruby/gems/1.8/gems/chef-server-api-0.10.4/app/controllers/environments.rb:188:incookbook_versions_for_run_list’
/usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/controller/abstract_controller.rb:321:in
send' /usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/controller/abstract_controller.rb:321:in_call_action’
/usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/controller/abstract_controller.rb:289:in
_dispatch' /usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/controller/abstract_controller.rb:284:incatch’
/usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/controller/abstract_controller.rb:284:in
_dispatch' /usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/controller/merb_controller.rb:285:in_dispatch’
/usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/controller/merb_controller.rb:262:in
_call' /usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/controller/merb_controller.rb:252:incall’
/usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/dispatch/dispatcher.rb:91:in
dispatch_action' /usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/dispatch/dispatcher.rb:69:inhandle’
/usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/dispatch/dispatcher.rb:29:in
handle' /usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/rack/application.rb:17:incall’
/usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/rack/middleware/static.rb:28:in
call' /usr/lib/ruby/gems/1.8/gems/rack-1.1.0/lib/rack/content_length.rb:13:incall’
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/connection.rb:84:in
pre_process' /usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/connection.rb:82:incatch’
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/connection.rb:82:in
pre_process' /usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/connection.rb:57:inprocess’
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/connection.rb:42:in
receive_data' /usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:inrun_machine’
/usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in
run' /usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/backends/base.rb:61:instart’
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/server.rb:159:in start' /usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/rack/adapter/thin.rb:30:instart_server’
/usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/rack/adapter/abstract.rb:305:in
start_at_port' /usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/rack/adapter/abstract.rb:138:instart’
/usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/server.rb:174:in
bootup' /usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/server.rb:159:indaemonize’
/usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/server.rb:143:in
fork' /usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/server.rb:143:indaemonize’
/usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/server.rb:35:in
start' /usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core.rb:165:instart’
/usr/lib/ruby/gems/1.8/gems/chef-server-api-0.10.4/bin/chef-server:84
/usr/bin/chef-server:19:in `load’
/usr/bin/chef-server:19
chef-server-webui :
merb : chef-server-webui : worker (port 4040) ~ Started request handling:
Tue Aug 30 22:42:15 +0530 2011
merb : chef-server-webui : worker (port 4040) ~ Params: {“action”=>“index”,
“controller”=>“nodes”}
merb : chef-server-webui : worker (port 4040) ~ I cannot read
/etc/chef/webui.pem, which you told me to use to sign requests! -
(Chef::Exceptions::PrivateKeyMissing)
/usr/lib/ruby/gems/1.8/gems/chef-0.10.4/lib/chef/rest/auth_credentials.rb:62:in
load_signing_key' /usr/lib/ruby/gems/1.8/gems/chef-0.10.4/lib/chef/rest/auth_credentials.rb:33:ininitialize’
/usr/lib/ruby/gems/1.8/gems/chef-0.10.4/lib/chef/rest.rb:47:in new' /usr/lib/ruby/gems/1.8/gems/chef-0.10.4/lib/chef/rest.rb:47:ininitialize’
/usr/lib/ruby/gems/1.8/gems/chef-0.10.4/lib/chef/environment.rb:89:in new' /usr/lib/ruby/gems/1.8/gems/chef-0.10.4/lib/chef/environment.rb:89:inchef_server_rest’
/usr/lib/ruby/gems/1.8/gems/chef-0.10.4/lib/chef/environment.rb:271:in
list' /usr/lib/ruby/gems/1.8/gems/chef-server-webui-0.10.4/app/controllers/application.rb:117:inload_environments’
/usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/controller/abstract_controller.rb:352:in
send' /usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/controller/abstract_controller.rb:352:in_call_filters’
/usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/controller/abstract_controller.rb:344:in
each' /usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/controller/abstract_controller.rb:344:in_call_filters’
/usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/controller/abstract_controller.rb:286:in
_dispatch' /usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/controller/abstract_controller.rb:284:incatch’
/usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/controller/abstract_controller.rb:284:in
_dispatch' /usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/controller/merb_controller.rb:285:in_dispatch’
/usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/controller/merb_controller.rb:262:in
_call' /usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/controller/merb_controller.rb:252:incall’
/usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/dispatch/dispatcher.rb:91:in
dispatch_action' /usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/dispatch/dispatcher.rb:69:inhandle’
/usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/dispatch/dispatcher.rb:29:in
handle' /usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/rack/application.rb:17:incall’
/usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/rack/middleware/static.rb:28:in
call' /usr/lib/ruby/gems/1.8/gems/rack-1.1.0/lib/rack/content_length.rb:13:incall’
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/connection.rb:84:in
pre_process' /usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/connection.rb:82:incatch’
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/connection.rb:82:in
pre_process' /usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/connection.rb:57:inprocess’
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/connection.rb:42:in
receive_data' /usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:inrun_machine’
/usr/lib/ruby/gems/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in
run' /usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/backends/base.rb:61:instart’
/usr/lib/ruby/gems/1.8/gems/thin-1.2.11/lib/thin/server.rb:159:in start' /usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/rack/adapter/thin.rb:30:instart_server’
/usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/rack/adapter/abstract.rb:305:in
start_at_port' /usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/rack/adapter/abstract.rb:138:instart’
/usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/server.rb:174:in
bootup' /usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/server.rb:159:indaemonize’
/usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/server.rb:143:in
fork' /usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/server.rb:143:indaemonize’
/usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core/server.rb:35:in
start' /usr/lib/ruby/gems/1.8/gems/merb-core-1.1.3/lib/merb-core.rb:165:instart’
/usr/lib/ruby/gems/1.8/gems/chef-server-webui-0.10.4/bin/chef-server-webui:71
/usr/bin/chef-server-webui:19:in `load’
/usr/bin/chef-server-webui:19
Can you please suggest some ways to get this to work again ?
Thanks
Madhurranjan