# Cookbook upload: HTTP 500: Too many links

**URL:** <https://discourse.chef.io/t/cookbook-upload-http-500-too-many-links/2521>\
**Category:** Chef Infra (archive)\
**Created:** [February 8, 2012, 6:06am UTC](https://discourse.chef.io/t/cookbook-upload-http-500-too-many-links/2521 "2012-02-08T06:06:11Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kallen](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/kallen/32/1049_2.png) [@Kallen](https://discourse.chef.io/u/Kallen)\
**Post date:** [February 8, 2012, 6:06am UTC](https://discourse.chef.io/t/cookbook-upload-http-500-too-many-links/2521/1 "2012-02-08T06:06:11Z")

</div>

hiya. seeing this error on one of my chef servers. anyone run into this?  
know what to do?

chef:~/ops/chef-repo$ knife cookbook upload -a -o ./cookbooks/ -V  
Uploading activemq [1.0.0]  
INFO: Saving activemq  
INFO: Validating ruby files  
INFO: Validating templates  
INFO: Syntax OK  
INFO: HTTP Request Returned 500 Internal Server Error: Too many links - /var/cache/chef/sandboxes/26f2b236b39b403ca272034f5fa95599  
ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 1/5 in 4s  
INFO: HTTP Request Returned 500 Internal Server Error: Too many links - /var/cache/chef/sandboxes/242d0dbbe1a84a98b11a6ab776fb5ddd  
ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 2/5 in 7s  
INFO: HTTP Request Returned 500 Internal Server Error: Too many links - /var/cache/chef/sandboxes/848dbce5df314dfcaaed178c1e50856f  
ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 3/5 in 11s  
INFO: HTTP Request Returned 500 Internal Server Error: Too many links - /var/cache/chef/sandboxes/5e975b05a06347c0bb949cbe1c6f549e

i try uploading just one cookbook, and the error is different:

chef:~/ops/chef-repo$ knife cookbook upload s3cmd -o ./cookbooks/  
Uploading s3cmd [0.0.1]  
ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 1/5 in 3s  
ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 2/5 in 7s  
ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 3/5 in 10s

i tried restarting couchdb and chef-server, still no joy.

chef-server is 0.10.8-1, ubuntu natty.  
this is a secondary chef-server in my infrastructure serving a DR  
datacenter. it loads the same sets of cookbooks, data bags, etc from  
chef-repo in git.

fwiw, i realized i hadn't been compacting couchdb here. the chef couchdb  
on the fs was 1.9G. i ran compact and it's now 45Mb. but i'm still seeing  
the above error.

also fwiw, above compact was done like so, not via chef-server cookbook:

/usr/bin/curl -H "Content-Type: application/json" -X POST [http://localhost:5984/chef/\_compact](http://localhost:5984/chef/_compact)  
/usr/bin/curl -H "Content-Type: application/json" -X POST [http://localhost:5984/chef/\_view\_cleanup](http://localhost:5984/chef/_view_cleanup)  
for view in registrations sandboxes id\_map clients checksums cookbooks data\_bags nodes roles users; do  
/usr/bin/curl -H "Content-Type: application/json" -X POST [http://localhost:5984/chef/\_compact/$view](http://localhost:5984/chef/_compact/$view)  
done

thanks  
kallen

and one more bit of log in case it's helpful:

merb : chef-server (api) : worker (port 4000) ~ Started request handling: Wed Feb 08 05:47:27 +0000 2012  
merb : chef-server (api) : worker (port 4000) ~ Params: {"action"=\>"create", "checksums"=\>{"83580e30b45b9a8e6b3359d47  
d14208e"=\>nil, "d217102ad8b64da1aa17218621820d49"=\>nil, "099b134c847994fc237be5593604fad7"=\>nil, "cd33cd00aad2e9f89a6  
74baa9c49b24c"=\>nil, "cd357558feae778ce719e03dedd4f074"=\>nil, "bfccb2c807ce7d9fdb9c6a2cf6f49133"=\>nil}, "controller"=

> "sandboxes"}  
> merb : chef-server (api) : worker (port 4000) ~ Too many links - /var/cache/chef/sandboxes/26f2b236b39b403ca272034f5fa95599 - (Errno::EMLINK)  
> /usr/lib/ruby/1.8/fileutils.rb:243:in `mkdir' /usr/lib/ruby/1.8/fileutils.rb:243:in `fu\_mkdir'  
> /usr/lib/ruby/1.8/fileutils.rb:217:in `mkdir_p' /usr/lib/ruby/1.8/fileutils.rb:215:in `reverse\_each'  
> /usr/lib/ruby/1.8/fileutils.rb:215:in `mkdir_p' /usr/lib/ruby/1.8/fileutils.rb:201:in `each'  
> /usr/lib/ruby/1.8/fileutils.rb:201:in `mkdir_p' /usr/share/chef-server-api/app/controllers/sandboxes.rb:76:in `create'  
> /usr/lib/ruby/1.8/merb-core/controller/abstract\_controller.rb:315:in `send' /usr/lib/ruby/1.8/merb-core/controller/abstract_controller.rb:315:in `\_call\_action'  
> /usr/lib/ruby/1.8/merb-core/controller/abstract\_controller.rb:289:in `_dispatch' /usr/lib/ruby/1.8/merb-core/controller/merb_controller.rb:252:in `\_dispatch'  
> /usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:102:in `dispatch_action' /usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:74:in `handle'  
> /usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:36:in `handle' /usr/lib/ruby/1.8/merb-core/rack/application.rb:17:in `call'  
> /usr/lib/ruby/1.8/merb-core/rack/middleware/static.rb:28:in `call' /usr/lib/ruby/1.8/rack/content_length.rb:13:in `call'  
> /usr/lib/ruby/1.8/thin/connection.rb:76:in `pre_process' /usr/lib/ruby/1.8/thin/connection.rb:74:in `catch'  
> /usr/lib/ruby/1.8/thin/connection.rb:74:in `pre_process' /usr/lib/ruby/1.8/thin/connection.rb:57:in `process'  
> /usr/lib/ruby/1.8/thin/connection.rb:42:in `receive_data' /usr/lib/ruby/1.8/eventmachine.rb:257:in `run\_machine'  
> /usr/lib/ruby/1.8/eventmachine.rb:257:in `run' /usr/lib/ruby/1.8/thin/backends/base.rb:57:in `start'  
> /usr/lib/ruby/1.8/thin/server.rb:156:in `start' /usr/lib/ruby/1.8/merb-core/rack/adapter/thin.rb:30:in `start\_server'  
> /usr/lib/ruby/1.8/merb-core/rack/adapter/abstract.rb:298:in `start_at_port' /usr/lib/ruby/1.8/merb-core/rack/adapter/abstract.rb:128:in `start'  
> /usr/lib/ruby/1.8/merb-core/server.rb:174:in `bootup' /usr/lib/ruby/1.8/merb-core/server.rb:159:in `daemonize'  
> /usr/lib/ruby/1.8/merb-core/server.rb:143:in `fork' /usr/lib/ruby/1.8/merb-core/server.rb:143:in `daemonize'  
> /usr/lib/ruby/1.8/merb-core/server.rb:35:in `start' /usr/lib/ruby/1.8/merb-core.rb:170:in `start'  
> /usr/sbin/chef-server:86

---

<div class="post-metadata">

**Author:** ![AJ\_Christensen](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/aj_christensen/32/525_2.png) [@AJ\_Christensen](https://discourse.chef.io/u/AJ_Christensen)\
**Post date:** [February 8, 2012, 6:08am UTC](https://discourse.chef.io/t/cookbook-upload-http-500-too-many-links/2521/2 "2012-02-08T06:08:18Z")

</div>

what file system are you using on the chef server? what is the  
contents of the directory the error comes from? have you got some  
crazy symlink stuff going on in a cookbook?

--AJ

On 8 February 2012 19:06, [kallen@groknaut.net](mailto:kallen@groknaut.net) wrote:

> hiya. seeing this error on one of my chef servers. anyone run into this?  
> know what to do?
> 
> chef:~/ops/chef-repo$ knife cookbook upload -a -o ./cookbooks/ -V  
> Uploading activemq [1.0.0]  
> INFO: Saving activemq  
> INFO: Validating ruby files  
> INFO: Validating templates  
> INFO: Syntax OK  
> INFO: HTTP Request Returned 500 Internal Server Error: Too many links - /var/cache/chef/sandboxes/26f2b236b39b403ca272034f5fa95599  
> ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 1/5 in 4s  
> INFO: HTTP Request Returned 500 Internal Server Error: Too many links - /var/cache/chef/sandboxes/242d0dbbe1a84a98b11a6ab776fb5ddd  
> ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 2/5 in 7s  
> INFO: HTTP Request Returned 500 Internal Server Error: Too many links - /var/cache/chef/sandboxes/848dbce5df314dfcaaed178c1e50856f  
> ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 3/5 in 11s  
> INFO: HTTP Request Returned 500 Internal Server Error: Too many links - /var/cache/chef/sandboxes/5e975b05a06347c0bb949cbe1c6f549e
> 
> i try uploading just one cookbook, and the error is different:
> 
> chef:~/ops/chef-repo$ knife cookbook upload s3cmd -o ./cookbooks/  
> Uploading s3cmd [0.0.1]  
> ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 1/5 in 3s  
> ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 2/5 in 7s  
> ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 3/5 in 10s
> 
> i tried restarting couchdb and chef-server, still no joy.
> 
> chef-server is 0.10.8-1, ubuntu natty.  
> this is a secondary chef-server in my infrastructure serving a DR  
> datacenter. it loads the same sets of cookbooks, data bags, etc from  
> chef-repo in git.
> 
> fwiw, i realized i hadn't been compacting couchdb here. the chef couchdb  
> on the fs was 1.9G. i ran compact and it's now 45Mb. but i'm still seeing  
> the above error.
> 
> also fwiw, above compact was done like so, not via chef-server cookbook:
> 
> /usr/bin/curl -H "Content-Type: application/json" -X POST [http://localhost:5984/chef/\_compact](http://localhost:5984/chef/_compact)  
> /usr/bin/curl -H "Content-Type: application/json" -X POST [http://localhost:5984/chef/\_view\_cleanup](http://localhost:5984/chef/_view_cleanup)  
> for view in registrations sandboxes id\_map clients checksums cookbooks data\_bags nodes roles users; do  
> /usr/bin/curl -H "Content-Type: application/json" -X POST [http://localhost:5984/chef/\_compact/$view](http://localhost:5984/chef/_compact/$view)  
> done
> 
> thanks  
> kallen
> 
> and one more bit of log in case it's helpful:
> 
> merb : chef-server (api) : worker (port 4000) ~ Started request handling: Wed Feb 08 05:47:27 +0000 2012  
> merb : chef-server (api) : worker (port 4000) ~ Params: {"action"=\>"create", "checksums"=\>{"83580e30b45b9a8e6b3359d47  
> d14208e"=\>nil, "d217102ad8b64da1aa17218621820d49"=\>nil, "099b134c847994fc237be5593604fad7"=\>nil, "cd33cd00aad2e9f89a6  
> 74baa9c49b24c"=\>nil, "cd357558feae778ce719e03dedd4f074"=\>nil, "bfccb2c807ce7d9fdb9c6a2cf6f49133"=\>nil}, "controller"=
> 
> > "sandboxes"}  
> > merb : chef-server (api) : worker (port 4000) ~ Too many links - /var/cache/chef/sandboxes/26f2b236b39b403ca272034f5fa95599 - (Errno::EMLINK)  
> > /usr/lib/ruby/1.8/fileutils.rb:243:in `mkdir' /usr/lib/ruby/1.8/fileutils.rb:243:in `fu\_mkdir'  
> > /usr/lib/ruby/1.8/fileutils.rb:217:in `mkdir_p' /usr/lib/ruby/1.8/fileutils.rb:215:in `reverse\_each'  
> > /usr/lib/ruby/1.8/fileutils.rb:215:in `mkdir_p' /usr/lib/ruby/1.8/fileutils.rb:201:in `each'  
> > /usr/lib/ruby/1.8/fileutils.rb:201:in `mkdir_p' /usr/share/chef-server-api/app/controllers/sandboxes.rb:76:in `create'  
> > /usr/lib/ruby/1.8/merb-core/controller/abstract\_controller.rb:315:in `send' /usr/lib/ruby/1.8/merb-core/controller/abstract_controller.rb:315:in `\_call\_action'  
> > /usr/lib/ruby/1.8/merb-core/controller/abstract\_controller.rb:289:in `_dispatch' /usr/lib/ruby/1.8/merb-core/controller/merb_controller.rb:252:in `\_dispatch'  
> > /usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:102:in `dispatch_action' /usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:74:in `handle'  
> > /usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:36:in `handle' /usr/lib/ruby/1.8/merb-core/rack/application.rb:17:in `call'  
> > /usr/lib/ruby/1.8/merb-core/rack/middleware/static.rb:28:in `call' /usr/lib/ruby/1.8/rack/content_length.rb:13:in `call'  
> > /usr/lib/ruby/1.8/thin/connection.rb:76:in `pre_process' /usr/lib/ruby/1.8/thin/connection.rb:74:in `catch'  
> > /usr/lib/ruby/1.8/thin/connection.rb:74:in `pre_process' /usr/lib/ruby/1.8/thin/connection.rb:57:in `process'  
> > /usr/lib/ruby/1.8/thin/connection.rb:42:in `receive_data' /usr/lib/ruby/1.8/eventmachine.rb:257:in `run\_machine'  
> > /usr/lib/ruby/1.8/eventmachine.rb:257:in `run' /usr/lib/ruby/1.8/thin/backends/base.rb:57:in `start'  
> > /usr/lib/ruby/1.8/thin/server.rb:156:in `start' /usr/lib/ruby/1.8/merb-core/rack/adapter/thin.rb:30:in `start\_server'  
> > /usr/lib/ruby/1.8/merb-core/rack/adapter/abstract.rb:298:in `start_at_port' /usr/lib/ruby/1.8/merb-core/rack/adapter/abstract.rb:128:in `start'  
> > /usr/lib/ruby/1.8/merb-core/server.rb:174:in `bootup' /usr/lib/ruby/1.8/merb-core/server.rb:159:in `daemonize'  
> > /usr/lib/ruby/1.8/merb-core/server.rb:143:in `fork' /usr/lib/ruby/1.8/merb-core/server.rb:143:in `daemonize'  
> > /usr/lib/ruby/1.8/merb-core/server.rb:35:in `start' /usr/lib/ruby/1.8/merb-core.rb:170:in `start'  
> > /usr/sbin/chef-server:86

---

<div class="post-metadata">

**Author:** ![Kallen](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/kallen/32/1049_2.png) [@Kallen](https://discourse.chef.io/u/Kallen)\
**Post date:** [February 8, 2012, 6:13am UTC](https://discourse.chef.io/t/cookbook-upload-http-500-too-many-links/2521/3 "2012-02-08T06:13:50Z")

</div>

On Wed, 08 Feb 2012, AJ Christensen wrote:

> what file system are you using on the chef server? what is the  
> contents of the directory the error comes from? have you got some  
> crazy symlink stuff going on in a cookbook?

chef server filesystem is ext3.

no crazy symlink stuff is going on in any cookbooks (that i know of).  
nothing written by me anyway.

contents of the directory the error comes from .. you mean cwd?

chef:~/ops/chef-repo$ ls -l  
total 48  
drwxr-xr-x 2 wopr wopr 4096 2012-01-13 00:32 certificates  
-rw-r--r-- 1 wopr wopr 156 2012-01-13 00:32 chefignore  
drwxr-xr-x 2 wopr wopr 4096 2012-01-13 00:32 config  
drwxr-xr-x 49 wopr wopr 4096 2012-02-03 23:55 cookbooks  
drwxr-xr-x 11 wopr wopr 4096 2012-02-07 08:20 data\_bags  
drwxr-xr-x 2 wopr wopr 4096 2012-01-13 00:32 environments  
drwxr-xr-x 2 wopr wopr 4096 2012-02-07 08:20 nodes  
-rw-r--r-- 1 wopr wopr 2484 2012-01-13 00:32 Rakefile  
-rw-r--r-- 1 wopr wopr 3521 2012-01-13 00:32 README.md  
drwxr-xr-x 2 wopr wopr 4096 2012-02-07 08:20 roles  
drwxr-xr-x 23 wopr wopr 4096 2012-01-13 00:32 site-cookbooks  
drwxr-xr-x 2 wopr wopr 4096 2012-01-13 00:32 tasks

chef:~/ops/chef-repo$ ls -1 cookbooks/ | wc -l  
48

> On 8 February 2012 19:06, [kallen@groknaut.net](mailto:kallen@groknaut.net) wrote:
> 
> > hiya. seeing this error on one of my chef servers. anyone run into this?  
> > know what to do?
> > 
> > chef:~/ops/chef-repo$ knife cookbook upload -a -o ./cookbooks/ -V  
> > Uploading activemq [1.0.0]  
> > INFO: Saving activemq  
> > INFO: Validating ruby files  
> > INFO: Validating templates  
> > INFO: Syntax OK  
> > INFO: HTTP Request Returned 500 Internal Server Error: Too many links - /var/cache/chef/sandboxes/26f2b236b39b403ca272034f5fa95599  
> > ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 1/5 in 4s  
> > INFO: HTTP Request Returned 500 Internal Server Error: Too many links - /var/cache/chef/sandboxes/242d0dbbe1a84a98b11a6ab776fb5ddd  
> > ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 2/5 in 7s  
> > INFO: HTTP Request Returned 500 Internal Server Error: Too many links - /var/cache/chef/sandboxes/848dbce5df314dfcaaed178c1e50856f  
> > ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 3/5 in 11s  
> > INFO: HTTP Request Returned 500 Internal Server Error: Too many links - /var/cache/chef/sandboxes/5e975b05a06347c0bb949cbe1c6f549e
> > 
> > i try uploading just one cookbook, and the error is different:
> > 
> > chef:~/ops/chef-repo$ knife cookbook upload s3cmd -o ./cookbooks/  
> > Uploading s3cmd [0.0.1]  
> > ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 1/5 in 3s  
> > ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 2/5 in 7s  
> > ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 3/5 in 10s
> > 
> > i tried restarting couchdb and chef-server, still no joy.
> > 
> > chef-server is 0.10.8-1, ubuntu natty.  
> > this is a secondary chef-server in my infrastructure serving a DR  
> > datacenter. it loads the same sets of cookbooks, data bags, etc from  
> > chef-repo in git.
> > 
> > fwiw, i realized i hadn't been compacting couchdb here. the chef couchdb  
> > on the fs was 1.9G. i ran compact and it's now 45Mb. but i'm still seeing  
> > the above error.
> > 
> > also fwiw, above compact was done like so, not via chef-server cookbook:
> > 
> > /usr/bin/curl -H "Content-Type: application/json" -X POST [http://localhost:5984/chef/\_compact](http://localhost:5984/chef/_compact)  
> > /usr/bin/curl -H "Content-Type: application/json" -X POST [http://localhost:5984/chef/\_view\_cleanup](http://localhost:5984/chef/_view_cleanup)  
> > for view in registrations sandboxes id\_map clients checksums cookbooks data\_bags nodes roles users; do  
> > /usr/bin/curl -H "Content-Type: application/json" -X POST [http://localhost:5984/chef/\_compact/$view](http://localhost:5984/chef/_compact/$view)  
> > done
> > 
> > thanks  
> > kallen
> > 
> > and one more bit of log in case it's helpful:
> > 
> > merb : chef-server (api) : worker (port 4000) ~ Started request handling: Wed Feb 08 05:47:27 +0000 2012  
> > merb : chef-server (api) : worker (port 4000) ~ Params: {"action"=\>"create", "checksums"=\>{"83580e30b45b9a8e6b3359d47  
> > d14208e"=\>nil, "d217102ad8b64da1aa17218621820d49"=\>nil, "099b134c847994fc237be5593604fad7"=\>nil, "cd33cd00aad2e9f89a6  
> > 74baa9c49b24c"=\>nil, "cd357558feae778ce719e03dedd4f074"=\>nil, "bfccb2c807ce7d9fdb9c6a2cf6f49133"=\>nil}, "controller"=
> > 
> > > "sandboxes"}  
> > > merb : chef-server (api) : worker (port 4000) ~ Too many links - /var/cache/chef/sandboxes/26f2b236b39b403ca272034f5fa95599 - (Errno::EMLINK)  
> > > /usr/lib/ruby/1.8/fileutils.rb:243:in `mkdir' /usr/lib/ruby/1.8/fileutils.rb:243:in `fu\_mkdir'  
> > > /usr/lib/ruby/1.8/fileutils.rb:217:in `mkdir_p' /usr/lib/ruby/1.8/fileutils.rb:215:in `reverse\_each'  
> > > /usr/lib/ruby/1.8/fileutils.rb:215:in `mkdir_p' /usr/lib/ruby/1.8/fileutils.rb:201:in `each'  
> > > /usr/lib/ruby/1.8/fileutils.rb:201:in `mkdir_p' /usr/share/chef-server-api/app/controllers/sandboxes.rb:76:in `create'  
> > > /usr/lib/ruby/1.8/merb-core/controller/abstract\_controller.rb:315:in `send' /usr/lib/ruby/1.8/merb-core/controller/abstract_controller.rb:315:in `\_call\_action'  
> > > /usr/lib/ruby/1.8/merb-core/controller/abstract\_controller.rb:289:in `_dispatch' /usr/lib/ruby/1.8/merb-core/controller/merb_controller.rb:252:in `\_dispatch'  
> > > /usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:102:in `dispatch_action' /usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:74:in `handle'  
> > > /usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:36:in `handle' /usr/lib/ruby/1.8/merb-core/rack/application.rb:17:in `call'  
> > > /usr/lib/ruby/1.8/merb-core/rack/middleware/static.rb:28:in `call' /usr/lib/ruby/1.8/rack/content_length.rb:13:in `call'  
> > > /usr/lib/ruby/1.8/thin/connection.rb:76:in `pre_process' /usr/lib/ruby/1.8/thin/connection.rb:74:in `catch'  
> > > /usr/lib/ruby/1.8/thin/connection.rb:74:in `pre_process' /usr/lib/ruby/1.8/thin/connection.rb:57:in `process'  
> > > /usr/lib/ruby/1.8/thin/connection.rb:42:in `receive_data' /usr/lib/ruby/1.8/eventmachine.rb:257:in `run\_machine'  
> > > /usr/lib/ruby/1.8/eventmachine.rb:257:in `run' /usr/lib/ruby/1.8/thin/backends/base.rb:57:in `start'  
> > > /usr/lib/ruby/1.8/thin/server.rb:156:in `start' /usr/lib/ruby/1.8/merb-core/rack/adapter/thin.rb:30:in `start\_server'  
> > > /usr/lib/ruby/1.8/merb-core/rack/adapter/abstract.rb:298:in `start_at_port' /usr/lib/ruby/1.8/merb-core/rack/adapter/abstract.rb:128:in `start'  
> > > /usr/lib/ruby/1.8/merb-core/server.rb:174:in `bootup' /usr/lib/ruby/1.8/merb-core/server.rb:159:in `daemonize'  
> > > /usr/lib/ruby/1.8/merb-core/server.rb:143:in `fork' /usr/lib/ruby/1.8/merb-core/server.rb:143:in `daemonize'  
> > > /usr/lib/ruby/1.8/merb-core/server.rb:35:in `start' /usr/lib/ruby/1.8/merb-core.rb:170:in `start'  
> > > /usr/sbin/chef-server:86

---

<div class="post-metadata">

**Author:** ![AJ\_Christensen](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/aj_christensen/32/525_2.png) [@AJ\_Christensen](https://discourse.chef.io/u/AJ_Christensen)\
**Post date:** [February 8, 2012, 6:25am UTC](https://discourse.chef.io/t/cookbook-upload-http-500-too-many-links/2521/4 "2012-02-08T06:25:00Z")

</div>

No, I meant the directories such as:

/var/cache/chef/sandboxes/26f2b236b39b403ca272034f5fa95599

Cheers,

--AJ

On 8 February 2012 19:13, [kallen@groknaut.net](mailto:kallen@groknaut.net) wrote:

> On Wed, 08 Feb 2012, AJ Christensen wrote:
> 
> > what file system are you using on the chef server? what is the  
> > contents of the directory the error comes from? have you got some  
> > crazy symlink stuff going on in a cookbook?
> 
> chef server filesystem is ext3.
> 
> no crazy symlink stuff is going on in any cookbooks (that i know of).  
> nothing written by me anyway.
> 
> contents of the directory the error comes from .. you mean cwd?
> 
> chef:~/ops/chef-repo$ ls -l  
> total 48  
> drwxr-xr-x 2 wopr wopr 4096 2012-01-13 00:32 certificates  
> -rw-r--r-- 1 wopr wopr 156 2012-01-13 00:32 chefignore  
> drwxr-xr-x 2 wopr wopr 4096 2012-01-13 00:32 config  
> drwxr-xr-x 49 wopr wopr 4096 2012-02-03 23:55 cookbooks  
> drwxr-xr-x 11 wopr wopr 4096 2012-02-07 08:20 data\_bags  
> drwxr-xr-x 2 wopr wopr 4096 2012-01-13 00:32 environments  
> drwxr-xr-x 2 wopr wopr 4096 2012-02-07 08:20 nodes  
> -rw-r--r-- 1 wopr wopr 2484 2012-01-13 00:32 Rakefile  
> -rw-r--r-- 1 wopr wopr 3521 2012-01-13 00:32 README.md  
> drwxr-xr-x 2 wopr wopr 4096 2012-02-07 08:20 roles  
> drwxr-xr-x 23 wopr wopr 4096 2012-01-13 00:32 site-cookbooks  
> drwxr-xr-x 2 wopr wopr 4096 2012-01-13 00:32 tasks
> 
> chef:~/ops/chef-repo$ ls -1 cookbooks/ | wc -l  
> 48
> 
> > On 8 February 2012 19:06, [kallen@groknaut.net](mailto:kallen@groknaut.net) wrote:
> > 
> > > hiya. seeing this error on one of my chef servers. anyone run into this?  
> > > know what to do?
> > > 
> > > chef:~/ops/chef-repo$ knife cookbook upload -a -o ./cookbooks/ -V  
> > > Uploading activemq [1.0.0]  
> > > INFO: Saving activemq  
> > > INFO: Validating ruby files  
> > > INFO: Validating templates  
> > > INFO: Syntax OK  
> > > INFO: HTTP Request Returned 500 Internal Server Error: Too many links - /var/cache/chef/sandboxes/26f2b236b39b403ca272034f5fa95599  
> > > ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 1/5 in 4s  
> > > INFO: HTTP Request Returned 500 Internal Server Error: Too many links - /var/cache/chef/sandboxes/242d0dbbe1a84a98b11a6ab776fb5ddd  
> > > ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 2/5 in 7s  
> > > INFO: HTTP Request Returned 500 Internal Server Error: Too many links - /var/cache/chef/sandboxes/848dbce5df314dfcaaed178c1e50856f  
> > > ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 3/5 in 11s  
> > > INFO: HTTP Request Returned 500 Internal Server Error: Too many links - /var/cache/chef/sandboxes/5e975b05a06347c0bb949cbe1c6f549e
> > > 
> > > i try uploading just one cookbook, and the error is different:
> > > 
> > > chef:~/ops/chef-repo$ knife cookbook upload s3cmd -o ./cookbooks/  
> > > Uploading s3cmd [0.0.1]  
> > > ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 1/5 in 3s  
> > > ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 2/5 in 7s  
> > > ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 3/5 in 10s
> > > 
> > > i tried restarting couchdb and chef-server, still no joy.
> > > 
> > > chef-server is 0.10.8-1, ubuntu natty.  
> > > this is a secondary chef-server in my infrastructure serving a DR  
> > > datacenter. it loads the same sets of cookbooks, data bags, etc from  
> > > chef-repo in git.
> > > 
> > > fwiw, i realized i hadn't been compacting couchdb here. the chef couchdb  
> > > on the fs was 1.9G. i ran compact and it's now 45Mb. but i'm still seeing  
> > > the above error.
> > > 
> > > also fwiw, above compact was done like so, not via chef-server cookbook:
> > > 
> > > /usr/bin/curl -H "Content-Type: application/json" -X POST [http://localhost:5984/chef/\_compact](http://localhost:5984/chef/_compact)  
> > > /usr/bin/curl -H "Content-Type: application/json" -X POST [http://localhost:5984/chef/\_view\_cleanup](http://localhost:5984/chef/_view_cleanup)  
> > > for view in registrations sandboxes id\_map clients checksums cookbooks data\_bags nodes roles users; do  
> > > /usr/bin/curl -H "Content-Type: application/json" -X POST [http://localhost:5984/chef/\_compact/$view](http://localhost:5984/chef/_compact/$view)  
> > > done
> > > 
> > > thanks  
> > > kallen
> > > 
> > > and one more bit of log in case it's helpful:
> > > 
> > > merb : chef-server (api) : worker (port 4000) ~ Started request handling: Wed Feb 08 05:47:27 +0000 2012  
> > > merb : chef-server (api) : worker (port 4000) ~ Params: {"action"=\>"create", "checksums"=\>{"83580e30b45b9a8e6b3359d47  
> > > d14208e"=\>nil, "d217102ad8b64da1aa17218621820d49"=\>nil, "099b134c847994fc237be5593604fad7"=\>nil, "cd33cd00aad2e9f89a6  
> > > 74baa9c49b24c"=\>nil, "cd357558feae778ce719e03dedd4f074"=\>nil, "bfccb2c807ce7d9fdb9c6a2cf6f49133"=\>nil}, "controller"=
> > > 
> > > > "sandboxes"}  
> > > > merb : chef-server (api) : worker (port 4000) ~ Too many links - /var/cache/chef/sandboxes/26f2b236b39b403ca272034f5fa95599 - (Errno::EMLINK)  
> > > > /usr/lib/ruby/1.8/fileutils.rb:243:in `mkdir' /usr/lib/ruby/1.8/fileutils.rb:243:in `fu\_mkdir'  
> > > > /usr/lib/ruby/1.8/fileutils.rb:217:in `mkdir_p' /usr/lib/ruby/1.8/fileutils.rb:215:in `reverse\_each'  
> > > > /usr/lib/ruby/1.8/fileutils.rb:215:in `mkdir_p' /usr/lib/ruby/1.8/fileutils.rb:201:in `each'  
> > > > /usr/lib/ruby/1.8/fileutils.rb:201:in `mkdir_p' /usr/share/chef-server-api/app/controllers/sandboxes.rb:76:in `create'  
> > > > /usr/lib/ruby/1.8/merb-core/controller/abstract\_controller.rb:315:in `send' /usr/lib/ruby/1.8/merb-core/controller/abstract_controller.rb:315:in `\_call\_action'  
> > > > /usr/lib/ruby/1.8/merb-core/controller/abstract\_controller.rb:289:in `_dispatch' /usr/lib/ruby/1.8/merb-core/controller/merb_controller.rb:252:in `\_dispatch'  
> > > > /usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:102:in `dispatch_action' /usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:74:in `handle'  
> > > > /usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:36:in `handle' /usr/lib/ruby/1.8/merb-core/rack/application.rb:17:in `call'  
> > > > /usr/lib/ruby/1.8/merb-core/rack/middleware/static.rb:28:in `call' /usr/lib/ruby/1.8/rack/content_length.rb:13:in `call'  
> > > > /usr/lib/ruby/1.8/thin/connection.rb:76:in `pre_process' /usr/lib/ruby/1.8/thin/connection.rb:74:in `catch'  
> > > > /usr/lib/ruby/1.8/thin/connection.rb:74:in `pre_process' /usr/lib/ruby/1.8/thin/connection.rb:57:in `process'  
> > > > /usr/lib/ruby/1.8/thin/connection.rb:42:in `receive_data' /usr/lib/ruby/1.8/eventmachine.rb:257:in `run\_machine'  
> > > > /usr/lib/ruby/1.8/eventmachine.rb:257:in `run' /usr/lib/ruby/1.8/thin/backends/base.rb:57:in `start'  
> > > > /usr/lib/ruby/1.8/thin/server.rb:156:in `start' /usr/lib/ruby/1.8/merb-core/rack/adapter/thin.rb:30:in `start\_server'  
> > > > /usr/lib/ruby/1.8/merb-core/rack/adapter/abstract.rb:298:in `start_at_port' /usr/lib/ruby/1.8/merb-core/rack/adapter/abstract.rb:128:in `start'  
> > > > /usr/lib/ruby/1.8/merb-core/server.rb:174:in `bootup' /usr/lib/ruby/1.8/merb-core/server.rb:159:in `daemonize'  
> > > > /usr/lib/ruby/1.8/merb-core/server.rb:143:in `fork' /usr/lib/ruby/1.8/merb-core/server.rb:143:in `daemonize'  
> > > > /usr/lib/ruby/1.8/merb-core/server.rb:35:in `start' /usr/lib/ruby/1.8/merb-core.rb:170:in `start'  
> > > > /usr/sbin/chef-server:86

---

<div class="post-metadata">

**Author:** ![Kallen](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/kallen/32/1049_2.png) [@Kallen](https://discourse.chef.io/u/Kallen)\
**Post date:** [February 8, 2012, 6:33am UTC](https://discourse.chef.io/t/cookbook-upload-http-500-too-many-links/2521/5 "2012-02-08T06:33:10Z")

</div>

On Wed, 08 Feb 2012, AJ Christensen wrote:

> No, I meant the directories such as:
> 
> /var/cache/chef/sandboxes/26f2b236b39b403ca272034f5fa95599

ahh. well...

getting no such file or dir for the paths mentioned in the errors below.  
but:

DR chef server having the problem:  
find /var/cache/chef/sandboxes -maxdepth 1 -type d | wc -l  
31999

prod chef server that's thankfully not having problem:  
find /var/cache/chef/sandboxes -maxdepth 1 -type d | wc -l  
1162

yeah that's a lotta dirs in that dir.

how'd that happen? i wonder what dirs therein i can prune, how to  
prevent it from happening again. or if i'll need to take a heavy hammer  
and wipe and recreate this DR chef server.

> On 8 February 2012 19:13, [kallen@groknaut.net](mailto:kallen@groknaut.net) wrote:
> 
> > On Wed, 08 Feb 2012, AJ Christensen wrote:
> > 
> > > what file system are you using on the chef server? what is the  
> > > contents of the directory the error comes from? have you got some  
> > > crazy symlink stuff going on in a cookbook?
> > 
> > chef server filesystem is ext3.
> > 
> > no crazy symlink stuff is going on in any cookbooks (that i know of).  
> > nothing written by me anyway.
> > 
> > contents of the directory the error comes from .. you mean cwd?
> > 
> > chef:~/ops/chef-repo$ ls -l  
> > total 48  
> > drwxr-xr-x 2 wopr wopr 4096 2012-01-13 00:32 certificates  
> > -rw-r--r-- 1 wopr wopr 156 2012-01-13 00:32 chefignore  
> > drwxr-xr-x 2 wopr wopr 4096 2012-01-13 00:32 config  
> > drwxr-xr-x 49 wopr wopr 4096 2012-02-03 23:55 cookbooks  
> > drwxr-xr-x 11 wopr wopr 4096 2012-02-07 08:20 data\_bags  
> > drwxr-xr-x 2 wopr wopr 4096 2012-01-13 00:32 environments  
> > drwxr-xr-x 2 wopr wopr 4096 2012-02-07 08:20 nodes  
> > -rw-r--r-- 1 wopr wopr 2484 2012-01-13 00:32 Rakefile  
> > -rw-r--r-- 1 wopr wopr 3521 2012-01-13 00:32 README.md  
> > drwxr-xr-x 2 wopr wopr 4096 2012-02-07 08:20 roles  
> > drwxr-xr-x 23 wopr wopr 4096 2012-01-13 00:32 site-cookbooks  
> > drwxr-xr-x 2 wopr wopr 4096 2012-01-13 00:32 tasks
> > 
> > chef:~/ops/chef-repo$ ls -1 cookbooks/ | wc -l  
> > 48
> > 
> > > On 8 February 2012 19:06, [kallen@groknaut.net](mailto:kallen@groknaut.net) wrote:
> > > 
> > > > hiya. seeing this error on one of my chef servers. anyone run into this?  
> > > > know what to do?
> > > > 
> > > > chef:~/ops/chef-repo$ knife cookbook upload -a -o ./cookbooks/ -V  
> > > > Uploading activemq [1.0.0]  
> > > > INFO: Saving activemq  
> > > > INFO: Validating ruby files  
> > > > INFO: Validating templates  
> > > > INFO: Syntax OK  
> > > > INFO: HTTP Request Returned 500 Internal Server Error: Too many links - /var/cache/chef/sandboxes/26f2b236b39b403ca272034f5fa95599  
> > > > ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 1/5 in 4s  
> > > > INFO: HTTP Request Returned 500 Internal Server Error: Too many links - /var/cache/chef/sandboxes/242d0dbbe1a84a98b11a6ab776fb5ddd  
> > > > ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 2/5 in 7s  
> > > > INFO: HTTP Request Returned 500 Internal Server Error: Too many links - /var/cache/chef/sandboxes/848dbce5df314dfcaaed178c1e50856f  
> > > > ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 3/5 in 11s  
> > > > INFO: HTTP Request Returned 500 Internal Server Error: Too many links - /var/cache/chef/sandboxes/5e975b05a06347c0bb949cbe1c6f549e
> > > > 
> > > > i try uploading just one cookbook, and the error is different:
> > > > 
> > > > chef:~/ops/chef-repo$ knife cookbook upload s3cmd -o ./cookbooks/  
> > > > Uploading s3cmd [0.0.1]  
> > > > ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 1/5 in 3s  
> > > > ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 2/5 in 7s  
> > > > ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 3/5 in 10s
> > > > 
> > > > i tried restarting couchdb and chef-server, still no joy.
> > > > 
> > > > chef-server is 0.10.8-1, ubuntu natty.  
> > > > this is a secondary chef-server in my infrastructure serving a DR  
> > > > datacenter. it loads the same sets of cookbooks, data bags, etc from  
> > > > chef-repo in git.
> > > > 
> > > > fwiw, i realized i hadn't been compacting couchdb here. the chef couchdb  
> > > > on the fs was 1.9G. i ran compact and it's now 45Mb. but i'm still seeing  
> > > > the above error.
> > > > 
> > > > also fwiw, above compact was done like so, not via chef-server cookbook:
> > > > 
> > > > /usr/bin/curl -H "Content-Type: application/json" -X POST [http://localhost:5984/chef/\_compact](http://localhost:5984/chef/_compact)  
> > > > /usr/bin/curl -H "Content-Type: application/json" -X POST [http://localhost:5984/chef/\_view\_cleanup](http://localhost:5984/chef/_view_cleanup)  
> > > > for view in registrations sandboxes id\_map clients checksums cookbooks data\_bags nodes roles users; do  
> > > > /usr/bin/curl -H "Content-Type: application/json" -X POST [http://localhost:5984/chef/\_compact/$view](http://localhost:5984/chef/_compact/$view)  
> > > > done
> > > > 
> > > > thanks  
> > > > kallen
> > > > 
> > > > and one more bit of log in case it's helpful:
> > > > 
> > > > merb : chef-server (api) : worker (port 4000) ~ Started request handling: Wed Feb 08 05:47:27 +0000 2012  
> > > > merb : chef-server (api) : worker (port 4000) ~ Params: {"action"=\>"create", "checksums"=\>{"83580e30b45b9a8e6b3359d47  
> > > > d14208e"=\>nil, "d217102ad8b64da1aa17218621820d49"=\>nil, "099b134c847994fc237be5593604fad7"=\>nil, "cd33cd00aad2e9f89a6  
> > > > 74baa9c49b24c"=\>nil, "cd357558feae778ce719e03dedd4f074"=\>nil, "bfccb2c807ce7d9fdb9c6a2cf6f49133"=\>nil}, "controller"=
> > > > 
> > > > > "sandboxes"}  
> > > > > merb : chef-server (api) : worker (port 4000) ~ Too many links - /var/cache/chef/sandboxes/26f2b236b39b403ca272034f5fa95599 - (Errno::EMLINK)  
> > > > > /usr/lib/ruby/1.8/fileutils.rb:243:in `mkdir' /usr/lib/ruby/1.8/fileutils.rb:243:in `fu\_mkdir'  
> > > > > /usr/lib/ruby/1.8/fileutils.rb:217:in `mkdir_p' /usr/lib/ruby/1.8/fileutils.rb:215:in `reverse\_each'  
> > > > > /usr/lib/ruby/1.8/fileutils.rb:215:in `mkdir_p' /usr/lib/ruby/1.8/fileutils.rb:201:in `each'  
> > > > > /usr/lib/ruby/1.8/fileutils.rb:201:in `mkdir_p' /usr/share/chef-server-api/app/controllers/sandboxes.rb:76:in `create'  
> > > > > /usr/lib/ruby/1.8/merb-core/controller/abstract\_controller.rb:315:in `send' /usr/lib/ruby/1.8/merb-core/controller/abstract_controller.rb:315:in `\_call\_action'  
> > > > > /usr/lib/ruby/1.8/merb-core/controller/abstract\_controller.rb:289:in `_dispatch' /usr/lib/ruby/1.8/merb-core/controller/merb_controller.rb:252:in `\_dispatch'  
> > > > > /usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:102:in `dispatch_action' /usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:74:in `handle'  
> > > > > /usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:36:in `handle' /usr/lib/ruby/1.8/merb-core/rack/application.rb:17:in `call'  
> > > > > /usr/lib/ruby/1.8/merb-core/rack/middleware/static.rb:28:in `call' /usr/lib/ruby/1.8/rack/content_length.rb:13:in `call'  
> > > > > /usr/lib/ruby/1.8/thin/connection.rb:76:in `pre_process' /usr/lib/ruby/1.8/thin/connection.rb:74:in `catch'  
> > > > > /usr/lib/ruby/1.8/thin/connection.rb:74:in `pre_process' /usr/lib/ruby/1.8/thin/connection.rb:57:in `process'  
> > > > > /usr/lib/ruby/1.8/thin/connection.rb:42:in `receive_data' /usr/lib/ruby/1.8/eventmachine.rb:257:in `run\_machine'  
> > > > > /usr/lib/ruby/1.8/eventmachine.rb:257:in `run' /usr/lib/ruby/1.8/thin/backends/base.rb:57:in `start'  
> > > > > /usr/lib/ruby/1.8/thin/server.rb:156:in `start' /usr/lib/ruby/1.8/merb-core/rack/adapter/thin.rb:30:in `start\_server'  
> > > > > /usr/lib/ruby/1.8/merb-core/rack/adapter/abstract.rb:298:in `start_at_port' /usr/lib/ruby/1.8/merb-core/rack/adapter/abstract.rb:128:in `start'  
> > > > > /usr/lib/ruby/1.8/merb-core/server.rb:174:in `bootup' /usr/lib/ruby/1.8/merb-core/server.rb:159:in `daemonize'  
> > > > > /usr/lib/ruby/1.8/merb-core/server.rb:143:in `fork' /usr/lib/ruby/1.8/merb-core/server.rb:143:in `daemonize'  
> > > > > /usr/lib/ruby/1.8/merb-core/server.rb:35:in `start' /usr/lib/ruby/1.8/merb-core.rb:170:in `start'  
> > > > > /usr/sbin/chef-server:86

---

<div class="post-metadata">

**Author:** ![AJ\_Christensen](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/aj_christensen/32/525_2.png) [@AJ\_Christensen](https://discourse.chef.io/u/AJ_Christensen)\
**Post date:** [February 8, 2012, 6:43am UTC](https://discourse.chef.io/t/cookbook-upload-http-500-too-many-links/2521/6 "2012-02-08T06:43:18Z")

</div>

You can probably smash the sandboxes directory and re-upload all of  
the cookbooks, do backups or whatever.

What are all the files? some bad symlinking? you might actually have  
to look at them, not just count them 🙂

--AJ

On 8 February 2012 19:33, [kallen@groknaut.net](mailto:kallen@groknaut.net) wrote:

> On Wed, 08 Feb 2012, AJ Christensen wrote:
> 
> > No, I meant the directories such as:
> > 
> > /var/cache/chef/sandboxes/26f2b236b39b403ca272034f5fa95599
> 
> ahh. well...
> 
> getting no such file or dir for the paths mentioned in the errors below.  
> but:
> 
> DR chef server having the problem:  
> find /var/cache/chef/sandboxes -maxdepth 1 -type d | wc -l  
> 31999
> 
> prod chef server that's thankfully not having problem:  
> find /var/cache/chef/sandboxes -maxdepth 1 -type d | wc -l  
> 1162
> 
> yeah that's a lotta dirs in that dir.
> 
> how'd that happen? i wonder what dirs therein i can prune, how to  
> prevent it from happening again. or if i'll need to take a heavy hammer  
> and wipe and recreate this DR chef server.
> 
> > On 8 February 2012 19:13, [kallen@groknaut.net](mailto:kallen@groknaut.net) wrote:
> > 
> > > On Wed, 08 Feb 2012, AJ Christensen wrote:
> > > 
> > > > what file system are you using on the chef server? what is the  
> > > > contents of the directory the error comes from? have you got some  
> > > > crazy symlink stuff going on in a cookbook?
> > > 
> > > chef server filesystem is ext3.
> > > 
> > > no crazy symlink stuff is going on in any cookbooks (that i know of).  
> > > nothing written by me anyway.
> > > 
> > > contents of the directory the error comes from .. you mean cwd?
> > > 
> > > chef:~/ops/chef-repo$ ls -l  
> > > total 48  
> > > drwxr-xr-x 2 wopr wopr 4096 2012-01-13 00:32 certificates  
> > > -rw-r--r-- 1 wopr wopr 156 2012-01-13 00:32 chefignore  
> > > drwxr-xr-x 2 wopr wopr 4096 2012-01-13 00:32 config  
> > > drwxr-xr-x 49 wopr wopr 4096 2012-02-03 23:55 cookbooks  
> > > drwxr-xr-x 11 wopr wopr 4096 2012-02-07 08:20 data\_bags  
> > > drwxr-xr-x 2 wopr wopr 4096 2012-01-13 00:32 environments  
> > > drwxr-xr-x 2 wopr wopr 4096 2012-02-07 08:20 nodes  
> > > -rw-r--r-- 1 wopr wopr 2484 2012-01-13 00:32 Rakefile  
> > > -rw-r--r-- 1 wopr wopr 3521 2012-01-13 00:32 README.md  
> > > drwxr-xr-x 2 wopr wopr 4096 2012-02-07 08:20 roles  
> > > drwxr-xr-x 23 wopr wopr 4096 2012-01-13 00:32 site-cookbooks  
> > > drwxr-xr-x 2 wopr wopr 4096 2012-01-13 00:32 tasks
> > > 
> > > chef:~/ops/chef-repo$ ls -1 cookbooks/ | wc -l  
> > > 48
> > > 
> > > > On 8 February 2012 19:06, [kallen@groknaut.net](mailto:kallen@groknaut.net) wrote:
> > > > 
> > > > > hiya. seeing this error on one of my chef servers. anyone run into this?  
> > > > > know what to do?
> > > > > 
> > > > > chef:~/ops/chef-repo$ knife cookbook upload -a -o ./cookbooks/ -V  
> > > > > Uploading activemq [1.0.0]  
> > > > > INFO: Saving activemq  
> > > > > INFO: Validating ruby files  
> > > > > INFO: Validating templates  
> > > > > INFO: Syntax OK  
> > > > > INFO: HTTP Request Returned 500 Internal Server Error: Too many links - /var/cache/chef/sandboxes/26f2b236b39b403ca272034f5fa95599  
> > > > > ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 1/5 in 4s  
> > > > > INFO: HTTP Request Returned 500 Internal Server Error: Too many links - /var/cache/chef/sandboxes/242d0dbbe1a84a98b11a6ab776fb5ddd  
> > > > > ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 2/5 in 7s  
> > > > > INFO: HTTP Request Returned 500 Internal Server Error: Too many links - /var/cache/chef/sandboxes/848dbce5df314dfcaaed178c1e50856f  
> > > > > ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 3/5 in 11s  
> > > > > INFO: HTTP Request Returned 500 Internal Server Error: Too many links - /var/cache/chef/sandboxes/5e975b05a06347c0bb949cbe1c6f549e
> > > > > 
> > > > > i try uploading just one cookbook, and the error is different:
> > > > > 
> > > > > chef:~/ops/chef-repo$ knife cookbook upload s3cmd -o ./cookbooks/  
> > > > > Uploading s3cmd [0.0.1]  
> > > > > ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 1/5 in 3s  
> > > > > ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 2/5 in 7s  
> > > > > ERROR: Server returned error for [https://chef.dr.spergtastic.com/sandboxes](https://chef.dr.spergtastic.com/sandboxes), retrying 3/5 in 10s
> > > > > 
> > > > > i tried restarting couchdb and chef-server, still no joy.
> > > > > 
> > > > > chef-server is 0.10.8-1, ubuntu natty.  
> > > > > this is a secondary chef-server in my infrastructure serving a DR  
> > > > > datacenter. it loads the same sets of cookbooks, data bags, etc from  
> > > > > chef-repo in git.
> > > > > 
> > > > > fwiw, i realized i hadn't been compacting couchdb here. the chef couchdb  
> > > > > on the fs was 1.9G. i ran compact and it's now 45Mb. but i'm still seeing  
> > > > > the above error.
> > > > > 
> > > > > also fwiw, above compact was done like so, not via chef-server cookbook:
> > > > > 
> > > > > /usr/bin/curl -H "Content-Type: application/json" -X POST [http://localhost:5984/chef/\_compact](http://localhost:5984/chef/_compact)  
> > > > > /usr/bin/curl -H "Content-Type: application/json" -X POST [http://localhost:5984/chef/\_view\_cleanup](http://localhost:5984/chef/_view_cleanup)  
> > > > > for view in registrations sandboxes id\_map clients checksums cookbooks data\_bags nodes roles users; do  
> > > > > /usr/bin/curl -H "Content-Type: application/json" -X POST [http://localhost:5984/chef/\_compact/$view](http://localhost:5984/chef/_compact/$view)  
> > > > > done
> > > > > 
> > > > > thanks  
> > > > > kallen
> > > > > 
> > > > > and one more bit of log in case it's helpful:
> > > > > 
> > > > > merb : chef-server (api) : worker (port 4000) ~ Started request handling: Wed Feb 08 05:47:27 +0000 2012  
> > > > > merb : chef-server (api) : worker (port 4000) ~ Params: {"action"=\>"create", "checksums"=\>{"83580e30b45b9a8e6b3359d47  
> > > > > d14208e"=\>nil, "d217102ad8b64da1aa17218621820d49"=\>nil, "099b134c847994fc237be5593604fad7"=\>nil, "cd33cd00aad2e9f89a6  
> > > > > 74baa9c49b24c"=\>nil, "cd357558feae778ce719e03dedd4f074"=\>nil, "bfccb2c807ce7d9fdb9c6a2cf6f49133"=\>nil}, "controller"=
> > > > > 
> > > > > > "sandboxes"}  
> > > > > > merb : chef-server (api) : worker (port 4000) ~ Too many links - /var/cache/chef/sandboxes/26f2b236b39b403ca272034f5fa95599 - (Errno::EMLINK)  
> > > > > > /usr/lib/ruby/1.8/fileutils.rb:243:in `mkdir' /usr/lib/ruby/1.8/fileutils.rb:243:in `fu\_mkdir'  
> > > > > > /usr/lib/ruby/1.8/fileutils.rb:217:in `mkdir_p' /usr/lib/ruby/1.8/fileutils.rb:215:in `reverse\_each'  
> > > > > > /usr/lib/ruby/1.8/fileutils.rb:215:in `mkdir_p' /usr/lib/ruby/1.8/fileutils.rb:201:in `each'  
> > > > > > /usr/lib/ruby/1.8/fileutils.rb:201:in `mkdir_p' /usr/share/chef-server-api/app/controllers/sandboxes.rb:76:in `create'  
> > > > > > /usr/lib/ruby/1.8/merb-core/controller/abstract\_controller.rb:315:in `send' /usr/lib/ruby/1.8/merb-core/controller/abstract_controller.rb:315:in `\_call\_action'  
> > > > > > /usr/lib/ruby/1.8/merb-core/controller/abstract\_controller.rb:289:in `_dispatch' /usr/lib/ruby/1.8/merb-core/controller/merb_controller.rb:252:in `\_dispatch'  
> > > > > > /usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:102:in `dispatch_action' /usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:74:in `handle'  
> > > > > > /usr/lib/ruby/1.8/merb-core/dispatch/dispatcher.rb:36:in `handle' /usr/lib/ruby/1.8/merb-core/rack/application.rb:17:in `call'  
> > > > > > /usr/lib/ruby/1.8/merb-core/rack/middleware/static.rb:28:in `call' /usr/lib/ruby/1.8/rack/content_length.rb:13:in `call'  
> > > > > > /usr/lib/ruby/1.8/thin/connection.rb:76:in `pre_process' /usr/lib/ruby/1.8/thin/connection.rb:74:in `catch'  
> > > > > > /usr/lib/ruby/1.8/thin/connection.rb:74:in `pre_process' /usr/lib/ruby/1.8/thin/connection.rb:57:in `process'  
> > > > > > /usr/lib/ruby/1.8/thin/connection.rb:42:in `receive_data' /usr/lib/ruby/1.8/eventmachine.rb:257:in `run\_machine'  
> > > > > > /usr/lib/ruby/1.8/eventmachine.rb:257:in `run' /usr/lib/ruby/1.8/thin/backends/base.rb:57:in `start'  
> > > > > > /usr/lib/ruby/1.8/thin/server.rb:156:in `start' /usr/lib/ruby/1.8/merb-core/rack/adapter/thin.rb:30:in `start\_server'  
> > > > > > /usr/lib/ruby/1.8/merb-core/rack/adapter/abstract.rb:298:in `start_at_port' /usr/lib/ruby/1.8/merb-core/rack/adapter/abstract.rb:128:in `start'  
> > > > > > /usr/lib/ruby/1.8/merb-core/server.rb:174:in `bootup' /usr/lib/ruby/1.8/merb-core/server.rb:159:in `daemonize'  
> > > > > > /usr/lib/ruby/1.8/merb-core/server.rb:143:in `fork' /usr/lib/ruby/1.8/merb-core/server.rb:143:in `daemonize'  
> > > > > > /usr/lib/ruby/1.8/merb-core/server.rb:35:in `start' /usr/lib/ruby/1.8/merb-core.rb:170:in `start'  
> > > > > > /usr/sbin/chef-server:86

---

<div class="post-metadata">

**Author:** ![Kallen](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/kallen/32/1049_2.png) [@Kallen](https://discourse.chef.io/u/Kallen)\
**Post date:** [February 8, 2012, 7:17am UTC](https://discourse.chef.io/t/cookbook-upload-http-500-too-many-links/2521/7 "2012-02-08T07:17:27Z")

</div>

On Wed, 08 Feb 2012, AJ Christensen wrote:

> You can probably smash the sandboxes directory and re-upload all of  
> the cookbooks, do backups or whatever.
> 
> What are all the files? some bad symlinking? you might actually have  
> to look at them, not just count them 🙂

i find nothing but directories under /var/cache/chef/sandboxes/; there  
are no symlinks. with ext3 on this fs, i've hit the max number of  
subdirectories in one directory: 32000.

i'll smash away and re-upload.

thanks for helping me find wtf was goin' on

k

> On 8 February 2012 19:33, [kallen@groknaut.net](mailto:kallen@groknaut.net) wrote:
> 
> > On Wed, 08 Feb 2012, AJ Christensen wrote:
> > 
> > > No, I meant the directories such as:
> > > 
> > > /var/cache/chef/sandboxes/26f2b236b39b403ca272034f5fa95599
> > 
> > ahh. well...
> > 
> > getting no such file or dir for the paths mentioned in the errors below.  
> > but:
> > 
> > DR chef server having the problem:  
> > find /var/cache/chef/sandboxes -maxdepth 1 -type d | wc -l  
> > 31999
> > 
> > prod chef server that's thankfully not having problem:  
> > find /var/cache/chef/sandboxes -maxdepth 1 -type d | wc -l  
> > 1162
> > 
> > yeah that's a lotta dirs in that dir.
> > 
> > how'd that happen? i wonder what dirs therein i can prune, how to  
> > prevent it from happening again. or if i'll need to take a heavy hammer  
> > and wipe and recreate this DR chef server.