I was having a problem where I had updated a template file (I verified
through the web interface that the template file was updated) but when the
client was run on a client it would not get the new version. Trying to force
it to I ended up delete /var/cache/chef/checksums/ on the server… This
seems to have been very bad.
When I realized this had borked the cookbook I did a bulk delete and purge
of that cookbook. I then ran the upload command on it again. This doesn’t
seem to really be doing anything. It doesn’t output checksums of all files
that its uploading like usual, it just displays the messages of the steps
and acts like it ran fine. I delete the checksums in ~/.chef/checksums just
in case but still nothing.
So now when I run the client on a node with -l debug I get lots of:
[Wed, 05 Jan 2011 11:44:53 -0600] DEBUG:
/cookbooks/ecdmarket/0.0.1/files/40e78bbf6e012c82eaefebd7292faeb3 (zero
content length or no Content-Length header)
And then a 500 error.
Which I assume is due to the upload not working correctly.
I was having a problem where I had updated a template file (I verified
through the web interface that the template file was updated) but when the
client was run on a client it would not get the new version. Trying to force
it to I ended up delete /var/cache/chef/checksums/ on the server... This
seems to have been very bad.
When I realized this had borked the cookbook I did a bulk delete and purge
of that cookbook. I then ran the upload command on it again. This doesn't
seem to really be doing anything. It doesn't output checksums of all files
that its uploading like usual, it just displays the messages of the steps
and acts like it ran fine. I delete the checksums in ~/.chef/checksums just
in case but still nothing.
What version of Chef do you have on the server? The --purge option
was not implemented until 0.9.10 and, unfortunately, I made a mistake
when I implemented it so it's not quite usable yet (the fix is merged
in master, but not yet in a released version). These bugs are tracked
as CHEF-1364[0] and CHEF-1836[1].
What you're seeing is that the database has checksum objects that are
supposed to correspond to files on disk (in that checksums directory)
but the files have been deleted. The easiest way to get back to a good
state is to delete the checksum documents from the database. To do
that, you need to log in to your chef-server box, then run shef.
From shef, run the following code to delete the checksum documents:
require 'chef/checksum'
r = Chef::REST.new('http://localhost:5984/chef/_design/checksums/_view/',
So now when I run the client on a node with -l debug I get lots of:
[Wed, 05 Jan 2011 11:44:53 -0600] DEBUG:
/cookbooks/ecdmarket/0.0.1/files/40e78bbf6e012c82eaefebd7292faeb3 (zero
content length or no Content-Length header)
And then a 500 error.
Which I assume is due to the upload not working correctly.
Does anyone have any ideas?
Thanks,
Tristan
Ah, yeah, I'm running 0.9.8. I should probably update, I think the problem I
had with the template not updating on the client is fixed in a later version
as well.
Great, that worked! Everything is working again.
But... the template still isn't being used on the client. This is really
pretty nuts, hehe. I just added '-smp enable' to the template and I see
clearly its in the template by going to the web interface and viewing it in
the cookbook but when I look at the file it is saved as on the clients that
piece isn't there...
I'd rather not have to get things updated to a newer Chef to fix this unless
its the quickest/easier way.
Tristan
On Wed, Jan 5, 2011 at 12:13 PM, Daniel DeLeo dan@kallistec.com wrote:
I was having a problem where I had updated a template file (I verified
through the web interface that the template file was updated) but when
the
client was run on a client it would not get the new version. Trying to
force
it to I ended up delete /var/cache/chef/checksums/ on the server... This
seems to have been very bad.
When I realized this had borked the cookbook I did a bulk delete and
purge
of that cookbook. I then ran the upload command on it again. This doesn't
seem to really be doing anything. It doesn't output checksums of all
files
that its uploading like usual, it just displays the messages of the steps
and acts like it ran fine. I delete the checksums in ~/.chef/checksums
just
in case but still nothing.
What version of Chef do you have on the server? The --purge option
was not implemented until 0.9.10 and, unfortunately, I made a mistake
when I implemented it so it's not quite usable yet (the fix is merged
in master, but not yet in a released version). These bugs are tracked
as CHEF-1364[0] and CHEF-1836[1].
What you're seeing is that the database has checksum objects that are
supposed to correspond to files on disk (in that checksums directory)
but the files have been deleted. The easiest way to get back to a good
state is to delete the checksum documents from the database. To do
that, you need to log in to your chef-server box, then run shef.
From shef, run the following code to delete the checksum documents:
So now when I run the client on a node with -l debug I get lots of:
[Wed, 05 Jan 2011 11:44:53 -0600] DEBUG:
/cookbooks/ecdmarket/0.0.1/files/40e78bbf6e012c82eaefebd7292faeb3 (zero
content length or no Content-Length header)
And then a 500 error.
Which I assume is due to the upload not working correctly.
Does anyone have any ideas?
Thanks,
Tristan
Ah, yeah, I'm running 0.9.8. I should probably update, I think the problem I
had with the template not updating on the client is fixed in a later version
as well.
There were some issues with early cookbook version implementations in
0.9.8 [1][2][3]. Reading through those bugs may help you figure out
how to get that cookbook to load.
Thanks. It seems the latest version of Chef in the EPEL repos is 0.9.8, is
this true? Is there another repo to use for centos that is updated more
often?
On Wed, Jan 5, 2011 at 12:29 PM, Bryan McLellan btm@loftninjas.org wrote:
Ah, yeah, I'm running 0.9.8. I should probably update, I think the
problem I
had with the template not updating on the client is fixed in a later
version
as well.
There were some issues with early cookbook version implementations in
0.9.8 [1][2][3]. Reading through those bugs may help you figure out
how to get that cookbook to load.
Thanks. It seems the latest version of Chef in the EPEL repos is 0.9.8, is
this true? Is there another repo to use for centos that is updated more
often?
On Wed, Jan 5, 2011 at 12:29 PM, Bryan McLellan btm@loftninjas.org wrote:
Ah, yeah, I'm running 0.9.8. I should probably update, I think the
problem I
had with the template not updating on the client is fixed in a later
version
as well.
There were some issues with early cookbook version implementations in
0.9.8 [1][2][3]. Reading through those bugs may help you figure out
how to get that cookbook to load.
Man, this is not a good day for me and Chef. Now I'm suddenly getting this
new 500 error when I run the chef client as well as if I do a knife cookbook
upload when it gets to the INFO: Uploading files. Very weird is I see the
files (recipes and templates) when I go to the web interface.
[Wed, 05 Jan 2011 13:31:15 -0600] DEBUG: Sending HTTP Request via PUT to
10.36.34.41:4000/nodes/ecduat02vm01
[Wed, 05 Jan 2011 13:31:18 -0600] WARN: HTTP Request Returned 500 Internal
Server Error: Connection failed - user: chef
/usr/lib/ruby/1.8/net/http.rb:2097:in error!': 500 "Internal Server Error" (Net::HTTPFatalError) from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/rest.rb:229:in api_request'
from
/usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/rest.rb:280:in retriable_rest_request' from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/rest.rb:210:in api_request'
from
/usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/rest.rb:126:in put_rest' from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/node.rb:502:in save'
from
/usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/client.rb:175:in build_node' from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/client.rb:75:in run'
from
/usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/application/client.rb:212:in run_application' from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/application/client.rb:202:in loop'
from
/usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/application/client.rb:202:in run_application' from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/application.rb:62:in run'
from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/chef-client:26
from /usr/bin/chef-client:19:in `load'
from /usr/bin/chef-client:19
On Wed, Jan 5, 2011 at 12:43 PM, Sean OMeara someara@gmail.com wrote:
Unfortunately no, not currently.
Centos5 has a lot of old libraries that make it a real pain to keep up
with.
I plan to provide bundler-ized RPMs for Centos6 (once it comes out)
Thanks. It seems the latest version of Chef in the EPEL repos is 0.9.8,
is
this true? Is there another repo to use for centos that is updated more
often?
On Wed, Jan 5, 2011 at 12:29 PM, Bryan McLellan btm@loftninjas.org
wrote:
Ah, yeah, I'm running 0.9.8. I should probably update, I think the
problem I
had with the template not updating on the client is fixed in a later
version
as well.
There were some issues with early cookbook version implementations in
0.9.8 [1][2][3]. Reading through those bugs may help you figure out
how to get that cookbook to load.
Thanks, Sean.
Man, this is not a good day for me and Chef. Now I'm suddenly getting this
new 500 error when I run the chef client as well as if I do a knife cookbook
upload when it gets to the INFO: Uploading files. Very weird is I see the
files (recipes and templates) when I go to the web interface.
[Wed, 05 Jan 2011 13:31:15 -0600] DEBUG: Sending HTTP Request via PUT to
10.36.34.41:4000/nodes/ecduat02vm01
[Wed, 05 Jan 2011 13:31:18 -0600] WARN: HTTP Request Returned 500 Internal
Server Error: Connection failed - user: chef
/usr/lib/ruby/1.8/net/http.rb:2097:in error!': 500 "Internal Server Error" (Net::HTTPFatalError) from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/rest.rb:229:in api_request'
from
/usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/rest.rb:280:in retriable_rest_request' from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/rest.rb:210:in api_request'
from
/usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/rest.rb:126:in put_rest' from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/node.rb:502:in save'
from
/usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/client.rb:175:in build_node' from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/client.rb:75:in run'
from
/usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/application/client.rb:212:in run_application' from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/application/client.rb:202:in loop'
from
/usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/application/client.rb:202:in run_application' from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/application.rb:62:in run'
from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/chef-client:26
from /usr/bin/chef-client:19:in `load'
from /usr/bin/chef-client:19
On Wed, Jan 5, 2011 at 12:43 PM, Sean OMeara someara@gmail.com wrote:
Unfortunately no, not currently.
Centos5 has a lot of old libraries that make it a real pain to keep up
with.
I plan to provide bundler-ized RPMs for Centos6 (once it comes out)
Thanks. It seems the latest version of Chef in the EPEL repos is 0.9.8,
is
this true? Is there another repo to use for centos that is updated more
often?
On Wed, Jan 5, 2011 at 12:29 PM, Bryan McLellan btm@loftninjas.org
wrote:
Ah, yeah, I'm running 0.9.8. I should probably update, I think the
problem I
had with the template not updating on the client is fixed in a later
version
as well.
There were some issues with early cookbook version implementations in
0.9.8 [1][2][3]. Reading through those bugs may help you figure out
how to get that cookbook to load.
Thanks, Sean.
Man, this is not a good day for me and Chef. Now I'm suddenly getting
this
new 500 error when I run the chef client as well as if I do a knife
cookbook
upload when it gets to the INFO: Uploading files. Very weird is I see the
files (recipes and templates) when I go to the web interface.
[Wed, 05 Jan 2011 13:31:15 -0600] DEBUG: Sending HTTP Request via PUT to
10.36.34.41:4000/nodes/ecduat02vm01
[Wed, 05 Jan 2011 13:31:18 -0600] WARN: HTTP Request Returned 500
Internal
Server Error: Connection failed - user: chef
/usr/lib/ruby/1.8/net/http.rb:2097:in error!': 500 "Internal Server Error" (Net::HTTPFatalError) from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/rest.rb:229:in api_request'
from
/usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/rest.rb:280:in retriable_rest_request' from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/rest.rb:210:in api_request'
from
/usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/rest.rb:126:in put_rest' from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/node.rb:502:in save'
from
/usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/client.rb:175:in build_node' from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/client.rb:75:in run'
from
run' from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/chef-client:26 from /usr/bin/chef-client:19:in load'
from /usr/bin/chef-client:19
On Wed, Jan 5, 2011 at 12:43 PM, Sean OMeara someara@gmail.com wrote:
Unfortunately no, not currently.
Centos5 has a lot of old libraries that make it a real pain to keep up
with.
I plan to provide bundler-ized RPMs for Centos6 (once it comes out)
Thanks. It seems the latest version of Chef in the EPEL repos is
0.9.8,
is
this true? Is there another repo to use for centos that is updated
more
often?
On Wed, Jan 5, 2011 at 12:29 PM, Bryan McLellan btm@loftninjas.org
wrote:
Ah, yeah, I'm running 0.9.8. I should probably update, I think the
problem I
had with the template not updating on the client is fixed in a
later
version
as well.
There were some issues with early cookbook version implementations in
0.9.8 [1][2][3]. Reading through those bugs may help you figure out
how to get that cookbook to load.
If you used EPEL and ELFF repos to update your Chef-related RPMs, updating RabbitMQ will wipe your existing rabbitmq configs. It's not too bad… Just need to add_vhost /chef, add_user, set permissions on it.
Initially I thought it happened because RabbitMQ was updated from v1 to v2.0, but it's also happening when you upgrade from one version of RabbitMQ v2 to the next.
Thanks, Sean.
Man, this is not a good day for me and Chef. Now I'm suddenly getting this
new 500 error when I run the chef client as well as if I do a knife cookbook
upload when it gets to the INFO: Uploading files. Very weird is I see the
files (recipes and templates) when I go to the web interface.
[Wed, 05 Jan 2011 13:31:15 -0600] DEBUG: Sending HTTP Request via PUT to
10.36.34.41:4000/nodes/ecduat02vm01http://10.36.34.41:4000/nodes/ecduat02vm01
[Wed, 05 Jan 2011 13:31:18 -0600] WARN: HTTP Request Returned 500 Internal
Server Error: Connection failed - user: chef
/usr/lib/ruby/1.8/net/http.rb:2097:in error!': 500 "Internal Server Error" (Net::HTTPFatalError) from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/rest.rb:229:in api_request'
from
/usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/rest.rb:280:in retriable_rest_request' from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/rest.rb:210:in api_request'
from
/usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/rest.rb:126:in put_rest' from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/node.rb:502:in save'
from
/usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/client.rb:175:in build_node' from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/client.rb:75:in run'
from
/usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/application/client.rb:212:in run_application' from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/application/client.rb:202:in loop'
from
/usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/application/client.rb:202:in run_application' from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/../lib/chef/application.rb:62:in run'
from /usr/lib/ruby/gems/1.8/gems/chef-0.9.8/bin/chef-client:26
from /usr/bin/chef-client:19:in `load'
from /usr/bin/chef-client:19
On Wed, Jan 5, 2011 at 12:43 PM, Sean OMeara <someara@gmail.commailto:someara@gmail.com> wrote:
Unfortunately no, not currently.
Centos5 has a lot of old libraries that make it a real pain to keep up
with.
I plan to provide bundler-ized RPMs for Centos6 (once it comes out)
Thanks. It seems the latest version of Chef in the EPEL repos is 0.9.8,
is
this true? Is there another repo to use for centos that is updated more
often?
On Wed, Jan 5, 2011 at 12:29 PM, Bryan McLellan <btm@loftninjas.orgmailto:btm@loftninjas.org>
wrote:
Ah, yeah, I'm running 0.9.8. I should probably update, I think the
problem I
had with the template not updating on the client is fixed in a later
version
as well.
There were some issues with early cookbook version implementations in
0.9.8 [1][2][3]. Reading through those bugs may help you figure out
how to get that cookbook to load.