Do you have the steps you used to backup and then restore the couchdb to
a newly built server? If you could post them, I would be extremely
From: email@example.com [mailto:firstname.lastname@example.org]
Sent: Tuesday, February 28, 2012 2:09 PM
Subject: [chef] Re: Re: Re: Re: Re: odd errors after restoring couchdb
onto new chef server
just to close out this thread:
to solve this on the newly built server, i purged all cookbooks, then
reuploaded all from my chef-repo in git. i didn’t attempt to backup and
restore the checksum_path dir.
the new restored server seems to be working ok.
thanks for the help!
On Fri, 24 Feb 2012, Daniel DeLeo wrote:
On Friday, February 24, 2012 at 6:18 PM, email@example.com wrote:
On Sat, 25 Feb 2012, AJ Christensen wrote:
- wierd errors running a chef-client against new server,
- ERROR: Server returned error for
- cookbook_file EOFError: end of file reached
did you copy the upload/copy the cookbooks to the new server?
are they in the file cache? Kind of looks like couchdb has
information about the files, but the files aren’t on the
chef-server file system (hazard
i hadn’t done an upload/copy of the cookbooks to the new server
because i thought the cookbooks were loaded in the couchdb-load.
when i run knife cookbook list, it looks like all my cookbooks are
regardless, i just reuploaded all cookbooks from my chef-repo to
the new server. i saw no upload errors. now, running chef-client
on a non-chef-server node, pointing chef_server_url directly at
port 4000 rather than hitting nginx proxy over https, is erroring
out on “EOFError - end of file reached”, which is similar to error
shown in gist.
[Fri, 24 Feb 2012 23:13:08 +0000] DEBUG: Sending HTTP Request via
[Fri, 24 Feb 2012 23:13:08 +0000] ERROR: Running exception
handlers [Fri, 24 Feb 2012 23:13:08 +0000] FATAL: Saving node
information to /var/cache/chef/failed-run-data.json
[Fri, 24 Feb 2012 23:13:08 +0000] ERROR: Exception handlers
complete [Fri, 24 Feb 2012 23:13:08 +0000] DEBUG: Re-raising
exception: EOFError - end of file reached
i’m looking at chef-server debug log and couchdb log. but nothing
is jumping out at me as relevant. but i may not recognize
last 3 lines of couchdb log fwiw:
[Fri, 24 Feb 2012 23:18:13 GMT] [info] [<0.17029.0>] 127.0.0.1 - -
‘POST’ /chef/_all_docs?include_docs=true 200 [Fri, 24 Feb 2012
23:18:14 GMT] [info] [<0.17030.0>] 127.0.0.1 - - ‘GET’
[Fri, 24 Feb 2012 23:18:14 GMT] [info] [<0.17031.0>] 127.0.0.1 - -
trying various things, flailing. for the heck of it:
- deleted possible problematic cookbook (mrepo) and reuploaded. same
- deleted client node which i’m testing on (admin2), reran, same
- reloaded couchdb dump again
and now a different cookbook download claims “EOFError - end of file
[Sat, 25 Feb 2012 02:14:22 +0000] DEBUG: Sending HTTP Request via
[Sat, 25 Feb 2012 02:14:22 +0000] ERROR: Running exception handlers
[Sat, 25 Feb 2012 02:14:22 +0000] FATAL: Saving node information to
[Sat, 25 Feb 2012 02:14:22 +0000] ERROR: Exception handlers complete
[Sat, 25 Feb 2012 02:14:22 +0000] DEBUG: Re-raising exception:
EOFError - end of file reached
dunno what to do here…
The files that comprise your cookbooks aren’t stored in couch, they’re
stored according to their MD5 under whatever you’ve configured for
checksum_pathin your server.rb. When you backup/restore your chef
server, you need to restore these files also.
The error you’re seeing occurs because the file transfer happens
outside of the normal request cycle, so server side errors aren’t
converted to 500s.
To fix it, you can either bulk delete all cookbooks using the
--purgeoption, or copy the files.
- Merb is trying to hand off the work to another thread (in the case
of threaded servers) or the event loop (in case of evented servers like
thin) so that the transfer can happen while other requests are
processed. I’m not sure if this even works any more. In any case, I
believe a user has contributed a fix for this problem.