# Cookbook files/ directory not syncd, chef-client a noop but shef -z "run\_chef" works

**URL:** <https://discourse.chef.io/t/cookbook-files-directory-not-syncd-chef-client-a-noop-but-shef-z-run-chef-works/1634>\
**Category:** Chef Infra (archive)\
**Created:** [December 28, 2010, 11:24pm UTC](https://discourse.chef.io/t/cookbook-files-directory-not-syncd-chef-client-a-noop-but-shef-z-run-chef-works/1634 "2010-12-28T23:24:21Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Charles\_Duffy](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/charles_duffy/32/793_2.png) [@Charles\_Duffy](https://discourse.chef.io/u/Charles_Duffy)\
**Post date:** [December 28, 2010, 11:24pm UTC](https://discourse.chef.io/t/cookbook-files-directory-not-syncd-chef-client-a-noop-but-shef-z-run-chef-works/1634/1 "2010-12-28T23:24:21Z")

</div>

I’ve got several very odd things, likely unrelated, happening in Chef 0.9.12  
– listed here in order of importance:

- “chef-client” exits without invoking any recipes, whereas “run\_chef” in  
shef -z actually tries to run the cookbooks applied to this node.
- the files/ directory in one of my cookbooks isn’t being pushed to the  
cache on my client
- the shef command cookbooks.all appears to display “nil” for each cookbook

Pointers on any of these would be appreciated.

cookbooks$ find yumrepo/  
…  
yumrepo/files/default/RPM-GPG-KEY-EPEL  
…  
cookbooks$ knife cookbook upload -o . yumrepo  
INFO: Saving yumrepo  
INFO: Validating ruby files  
INFO: Validating templates  
INFO: Syntax OK  
INFO: Generating Metadata  
INFO: Uploading files  
INFO: Upload complete!

…and over on the client…

# chef-client

[Tue, 28 Dec 2010 23:05:51 +0000] INFO: Starting Chef Run (Version 0.9.12)  
[Tue, 28 Dec 2010 23:06:09 +0000] INFO: Chef Run complete in 17.457648  
seconds  
[Tue, 28 Dec 2010 23:06:09 +0000] INFO: cleaning the checksum cache  
[Tue, 28 Dec 2010 23:06:09 +0000] INFO: Running report handlers  
[Tue, 28 Dec 2010 23:06:09 +0000] INFO: Report handlers complete

# shef -z

loading configuration: /etc/chef/client.rb  
Session type: solo  
Loading…done.

This is shef, the Chef shell.  
Chef Version: 0.9.12  
[http://www.opscode.com/chef](http://www.opscode.com/chef)  
[http://wiki.opscode.com/display/chef/Home](http://wiki.opscode.com/display/chef/Home)

run `help' for help,`exit’ or ^D to quit.

Ohai2u [root@staging.example.com](mailto:root@staging.example.com)!  
chef \> cookbooks.all  
=\> [nil, nil, nil, nil, nil]  
chef \> run\_chef  
…and things actually run as they should, until…  
[Tue, 28 Dec 2010 23:09:22 +0000] ERROR:  
cookbook\_file[/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL]  
(/var/chef/cache/cookbooks/yumrepo/definitions/yumkey.rb:47:in `from\_file’)  
had an error:  
cookbook yumrepo does not contain file files/RPM-GPG-KEY-EPEL

and indeed, /var/chef/cache/cookbooks/yumrepo/ has no files/ directory  
whatsoever.

---

<div class="post-metadata">

**Author:** ![Charles\_Duffy](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/charles_duffy/32/793_2.png) [@Charles\_Duffy](https://discourse.chef.io/u/Charles_Duffy)\
**Post date:** [December 29, 2010, 3:26pm UTC](https://discourse.chef.io/t/cookbook-files-directory-not-syncd-chef-client-a-noop-but-shef-z-run-chef-works/1634/2 "2010-12-29T15:26:01Z")

</div>

Doing a bit more research --

At least one item was simply a matter of me confusing myself -- my  
expectation that cookbook files would be stored in the cache on the client  
node appears to have been simply inaccurate. Watching the HTTP PUT by knife  
cookbook upload clearly shows the files in question being put, and their  
names and signatures are included when the node retrieves its list of  
cookbooks. I simply wasn't seeing anything being done with run\_client on  
account of it being less verbose than shef's default logging level (and  
because my run\_list was inappropriately constructed, not including the  
recipe in which I was inserting extra logging).

It looks like there's only one real issue here, and I'm wondering if it's  
simply a limitation of the "solo" mode shef -z runs in:

[Wed, 29 Dec 2010 15:16:33 +0000] ERROR:  
template[/etc/yum.repos.d/epel.repo]  
(/var/chef/cache/cookbooks/yumrepo/definitions/yumrepo.rb:45:in `from_file') had an error: Must call Chef::Cookbook::FileVendor.on_create before calling create_from_manifest factory /usr/lib64/ruby/gems/1.8/gems/chef-0.9.12/lib/chef/cookbook/file_vendor.rb:34:in `create\_from\_manifest'  
/usr/lib64/ruby/gems/1.8/gems/chef-0.9.12/lib/chef/cookbook\_version.rb:902:in  
`file_vendor' /usr/lib64/ruby/gems/1.8/gems/chef-0.9.12/lib/chef/cookbook_version.rb:536:in `preferred\_filename\_on\_disk\_location'  
/usr/lib64/ruby/gems/1.8/gems/chef-0.9.12/lib/chef/provider/template.rb:69:in  
`template_location' /usr/lib64/ruby/gems/1.8/gems/chef-0.9.12/lib/chef/provider/template.rb:39:in `action\_create'  
/usr/lib64/ruby/gems/1.8/gems/chef-0.9.12/lib/chef/resource.rb:395:in `send' /usr/lib64/ruby/gems/1.8/gems/chef-0.9.12/lib/chef/resource.rb:395:in `run\_action'  
/usr/lib64/ruby/gems/1.8/gems/chef-0.9.12/lib/chef/runner.rb:53:in  
`run_action' /usr/lib64/ruby/gems/1.8/gems/chef-0.9.12/lib/chef/runner.rb:89:in `converge'  
/usr/lib64/ruby/gems/1.8/gems/chef-0.9.12/lib/chef/runner.rb:89:in `each' /usr/lib64/ruby/gems/1.8/gems/chef-0.9.12/lib/chef/runner.rb:89:in `converge'  
/usr/lib64/ruby/gems/1.8/gems/chef-0.9.12/lib/chef/resource\_collection.rb:94:in  
`execute_each_resource' /usr/lib64/ruby/gems/1.8/gems/chef-0.9.12/lib/chef/resource_collection/stepable_iterator.rb:116:in `call'  
/usr/lib64/ruby/gems/1.8/gems/chef-0.9.12/lib/chef/resource\_collection/stepable\_iterator.rb:116:in  
`call_iterator_block' /usr/lib64/ruby/gems/1.8/gems/chef-0.9.12/lib/chef/resource_collection/stepable_iterator.rb:85:in `step'  
/usr/lib64/ruby/gems/1.8/gems/chef-0.9.12/lib/chef/resource\_collection/stepable\_iterator.rb:104:in  
`iterate' /usr/lib64/ruby/gems/1.8/gems/chef-0.9.12/lib/chef/resource_collection/stepable_iterator.rb:55:in `each\_with\_index'  
/usr/lib64/ruby/gems/1.8/gems/chef-0.9.12/lib/chef/resource\_collection.rb:92:in  
`execute_each_resource' /usr/lib64/ruby/gems/1.8/gems/chef-0.9.12/lib/chef/runner.rb:84:in `converge'  
/usr/lib64/ruby/gems/1.8/gems/chef-0.9.12/lib/chef/shef/ext.rb:233:in  
`run_chef' (irb):1:in `irb\_binding'

Is cookbook\_file expected to work in the context of shef -z?

Beyond that, please accept my apologies for the noise.

On Tue, Dec 28, 2010 at 5:24 PM, Charles Duffy [charles@dyfis.net](mailto:charles@dyfis.net) wrote:

> I've got several very odd things, likely unrelated, happening in Chef  
> 0.9.12 -- listed here in order of importance:
> 
> - "chef-client" exits without invoking any recipes, whereas "run\_chef" in  
> shef -z actually tries to run the cookbooks applied to this node.
> - the files/ directory in one of my cookbooks isn't being pushed to the  
> cache on my client
> - the shef command cookbooks.all appears to display "nil" for each cookbook
> 
> Pointers on any of these would be appreciated.
> 
> cookbooks$ find yumrepo/  
> ...  
> yumrepo/files/default/RPM-GPG-KEY-EPEL  
> ...  
> cookbooks$ knife cookbook upload -o . yumrepo  
> INFO: Saving yumrepo  
> INFO: Validating ruby files  
> INFO: Validating templates  
> INFO: Syntax OK  
> INFO: Generating Metadata  
> INFO: Uploading files  
> INFO: Upload complete!
> 
> ...and over on the client...
> 
> # chef-client
> 
> [Tue, 28 Dec 2010 23:05:51 +0000] INFO: Starting Chef Run (Version 0.9.12)  
> [Tue, 28 Dec 2010 23:06:09 +0000] INFO: Chef Run complete in 17.457648  
> seconds  
> [Tue, 28 Dec 2010 23:06:09 +0000] INFO: cleaning the checksum cache  
> [Tue, 28 Dec 2010 23:06:09 +0000] INFO: Running report handlers  
> [Tue, 28 Dec 2010 23:06:09 +0000] INFO: Report handlers complete
> 
> # shef -z
> 
> loading configuration: /etc/chef/client.rb  
> Session type: solo  
> Loading........done.
> 
> This is shef, the Chef shell.  
> Chef Version: 0.9.12  
> [Configuration Management System Software - Chef Infra | Chef](http://www.opscode.com/chef)  
> [http://wiki.opscode.com/display/chef/Home](http://wiki.opscode.com/display/chef/Home)
> 
> run `help' for help, `exit' or ^D to quit.
> 
> Ohai2u [root@staging.example.com](mailto:root@staging.example.com)!  
> chef \> cookbooks.all  
> =\> [nil, nil, nil, nil, nil]  
> chef \> run\_chef  
> ...and things actually run as they should, until...  
> [Tue, 28 Dec 2010 23:09:22 +0000] ERROR:  
> cookbook\_file[/etc/pki/rpm-gpg/RPM-GPG-KEY-EPEL]  
> (/var/chef/cache/cookbooks/yumrepo/definitions/yumkey.rb:47:in `from\_file')  
> had an error:  
> cookbook yumrepo does not contain file files/RPM-GPG-KEY-EPEL
> 
> and indeed, /var/chef/cache/cookbooks/yumrepo/ has no files/ directory  
> whatsoever.

---

<div class="post-metadata">

**Author:** ![kallistec](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/kallistec/32/23_2.png) [@kallistec](https://discourse.chef.io/u/kallistec)\
**Post date:** [December 29, 2010, 11:37pm UTC](https://discourse.chef.io/t/cookbook-files-directory-not-syncd-chef-client-a-noop-but-shef-z-run-chef-works/1634/3 "2010-12-29T23:37:57Z")

</div>

Ohai!

On Wed, Dec 29, 2010 at 8:26 AM, Charles Duffy [charles@dyfis.net](mailto:charles@dyfis.net) wrote:

> Doing a bit more research --  
> At least one item was simply a matter of me confusing myself -- my  
> expectation that cookbook files would be stored in the cache on the client  
> node appears to have been simply inaccurate. Watching the HTTP PUT by knife  
> cookbook upload clearly shows the files in question being put, and their  
> names and signatures are included when the node retrieves its list of  
> cookbooks. I simply wasn't seeing anything being done with run\_client on  
> account of it being less verbose than shef's default logging level (and  
> because my run\_list was inappropriately constructed, not including the  
> recipe in which I was inserting extra logging).  
> It looks like there's only one real issue here, and I'm wondering if it's  
> simply a limitation of the "solo" mode shef -z runs in:

I think there's possibly two bugs here. One is that even though you  
ran shef in "chef-client" mode, it says it's in solo mode. The  
offending code is here:

[https://github.com/opscode/chef/blob/master/chef/lib/chef/shef/shef\_session.rb#L177](https://github.com/opscode/chef/blob/master/chef/lib/chef/shef/shef_session.rb#L177)

This is annoying, but probably unrelated to being unable to use  
cookbook files. The root cause of the cookbook file bug is that shef  
is not configuring the cookbook fetching system to work as a  
chef-client so it doesn't know if it should be getting cookbook files  
and templates from disk (like chef-solo) or from the remote server  
(like chef-client).

I'll create tickets for these.

Dan

---

<div class="post-metadata">

**Author:** ![Charles\_Duffy](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/charles_duffy/32/793_2.png) [@Charles\_Duffy](https://discourse.chef.io/u/Charles_Duffy)\
**Post date:** [December 30, 2010, 12:33am UTC](https://discourse.chef.io/t/cookbook-files-directory-not-syncd-chef-client-a-noop-but-shef-z-run-chef-works/1634/4 "2010-12-30T00:33:04Z")

</div>

Dan --

Thank you for those extraordinarily detailed bug reports, which were  
sufficient for me (as a non-rubyist) to patch my shef into a functional  
state (in one case with a minor tweak, "@rest" rather than "rest").

For anyone else interested --  
[http://tickets.opscode.com/browse/CHEF-1954](http://tickets.opscode.com/browse/CHEF-1954)  
[http://tickets.opscode.com/browse/CHEF-1955](http://tickets.opscode.com/browse/CHEF-1955)  
[http://tickets.opscode.com/browse/CHEF-1955](http://tickets.opscode.com/browse/CHEF-1955)  
On Wed, Dec 29, 2010 at 5:37 PM, Daniel DeLeo [dan@kallistec.com](mailto:dan@kallistec.com) wrote:

> Ohai!
> 
> On Wed, Dec 29, 2010 at 8:26 AM, Charles Duffy [charles@dyfis.net](mailto:charles@dyfis.net) wrote:
> 
> > Doing a bit more research --  
> > At least one item was simply a matter of me confusing myself -- my  
> > expectation that cookbook files would be stored in the cache on the  
> > client  
> > node appears to have been simply inaccurate. Watching the HTTP PUT by  
> > knife  
> > cookbook upload clearly shows the files in question being put, and their  
> > names and signatures are included when the node retrieves its list of  
> > cookbooks. I simply wasn't seeing anything being done with run\_client on  
> > account of it being less verbose than shef's default logging level (and  
> > because my run\_list was inappropriately constructed, not including the  
> > recipe in which I was inserting extra logging).  
> > It looks like there's only one real issue here, and I'm wondering if it's  
> > simply a limitation of the "solo" mode shef -z runs in:
> 
> I think there's possibly two bugs here. One is that even though you  
> ran shef in "chef-client" mode, it says it's in solo mode. The  
> offending code is here:
> 
> [https://github.com/opscode/chef/blob/master/chef/lib/chef/shef/shef\_session.rb#L177](https://github.com/opscode/chef/blob/master/chef/lib/chef/shef/shef_session.rb#L177)
> 
> This is annoying, but probably unrelated to being unable to use  
> cookbook files. The root cause of the cookbook file bug is that shef  
> is not configuring the cookbook fetching system to work as a  
> chef-client so it doesn't know if it should be getting cookbook files  
> and templates from disk (like chef-solo) or from the remote server  
> (like chef-client).
> 
> I'll create tickets for these.
> 
> Dan

---

<div class="post-metadata">

**Author:** ![Charles\_Duffy](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/charles_duffy/32/793_2.png) [@Charles\_Duffy](https://discourse.chef.io/u/Charles_Duffy)\
**Post date:** [January 19, 2011, 3:46am UTC](https://discourse.chef.io/t/cookbook-files-directory-not-syncd-chef-client-a-noop-but-shef-z-run-chef-works/1634/5 "2011-01-19T03:46:44Z")

</div>

...well, damnit --

I went back and tried to reproduce this fix, but it looks like I'm hitting a  
different failure mode now -- I'm getting a FileNotFound thrown from  
preferred\_manifest\_record, which is called shortly _before_ file\_vendor is  
referenced.

Like the prior failure, this happens only with 'shef -z' but not using  
run\_client:

Chef::Exceptions::FileNotFound: cookbook loggly does not contain file  
files/loggly.com.crt  
from  
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/lib/chef/cookbook\_version.rb:527:in  
`preferred_manifest_record' from /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/lib/chef/cookbook_version.rb:532:in `preferred\_filename\_on\_disk\_location'  
from  
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/lib/chef/provider/cookbook\_file.rb:63:in  
`file_cache_location' from /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/lib/chef/provider/cookbook_file.rb:36:in `action\_create'

On Wed, Dec 29, 2010 at 6:33 PM, Charles Duffy [charles@dyfis.net](mailto:charles@dyfis.net) wrote:

> Dan --
> 
> Thank you for those extraordinarily detailed bug reports, which were  
> sufficient for me (as a non-rubyist) to patch my shef into a functional  
> state (in one case with a minor tweak, "@rest" rather than "rest").
> 
> For anyone else interested --  
> [http://tickets.opscode.com/browse/CHEF-1954](http://tickets.opscode.com/browse/CHEF-1954)  
> [http://tickets.opscode.com/browse/CHEF-1955](http://tickets.opscode.com/browse/CHEF-1955)  
> [http://tickets.opscode.com/browse/CHEF-1955](http://tickets.opscode.com/browse/CHEF-1955)  
> On Wed, Dec 29, 2010 at 5:37 PM, Daniel DeLeo [dan@kallistec.com](mailto:dan@kallistec.com) wrote:
> 
> > Ohai!
> > 
> > On Wed, Dec 29, 2010 at 8:26 AM, Charles Duffy [charles@dyfis.net](mailto:charles@dyfis.net) wrote:
> > 
> > > Doing a bit more research --  
> > > At least one item was simply a matter of me confusing myself -- my  
> > > expectation that cookbook files would be stored in the cache on the  
> > > client  
> > > node appears to have been simply inaccurate. Watching the HTTP PUT by  
> > > knife  
> > > cookbook upload clearly shows the files in question being put, and their  
> > > names and signatures are included when the node retrieves its list of  
> > > cookbooks. I simply wasn't seeing anything being done with run\_client on  
> > > account of it being less verbose than shef's default logging level (and  
> > > because my run\_list was inappropriately constructed, not including the  
> > > recipe in which I was inserting extra logging).  
> > > It looks like there's only one real issue here, and I'm wondering if  
> > > it's  
> > > simply a limitation of the "solo" mode shef -z runs in:
> > 
> > I think there's possibly two bugs here. One is that even though you  
> > ran shef in "chef-client" mode, it says it's in solo mode. The  
> > offending code is here:
> > 
> > [https://github.com/opscode/chef/blob/master/chef/lib/chef/shef/shef\_session.rb#L177](https://github.com/opscode/chef/blob/master/chef/lib/chef/shef/shef_session.rb#L177)
> > 
> > This is annoying, but probably unrelated to being unable to use  
> > cookbook files. The root cause of the cookbook file bug is that shef  
> > is not configuring the cookbook fetching system to work as a  
> > chef-client so it doesn't know if it should be getting cookbook files  
> > and templates from disk (like chef-solo) or from the remote server  
> > (like chef-client).
> > 
> > I'll create tickets for these.
> > 
> > Dan