$ berks upload keeps failing since yesterday


#1

Hello,
I’m working on a custom cookbook; since yesterday, whenever I run $ berks
upload to ship the latest version of the cookbook to my Chef server,
Berkshelf goes through all the cookbook dependencies alphabetically, as
normal, skips all of them since they don’t require an update, again as
normal, then it gets to my cookbook (it’s called “srv01”) and exits
unexpectedly with this output http://pastebin.com/3GZg4SMA (Pastebin
link).

Any cookbook that would come after “srv01” alphabetically is obviously
skipped since the program exits at that point. That’s not a huge deal for
me since they don’t require any updates, but could be problematic,
generally speaking.

But really my main problem is that the cookbook I’m actually working on
doesn’t get updated on the Chef server.
If I just do a $ knife cookbook upload srv01, it uploads fine.

I’m using the Hosted Chef service and I know from looking at status.chef.io
that they’ve been having issues lately, but unfortunately I have no idea
what the error I’m getting means, and while I can work around it, I’d like
to see if I can fix it, assuming it’s something I’m doing wrong.

What do you think? Please let me know if I need to provide additional
information.
Thanks!


#2

For what it’s worth, I actually just had to add a cookbook to my Berksfile,
fortunately it comes before “srv01” alphabetically, and $ berks upload
uploads that cookbook fine. Of course I still get the failure when it
reaches my own cookbook. But that suggests that the issue is with my
cookbook, somehow. I just don’t know what it could be.

On Wed, Apr 22, 2015 at 12:59 PM, Fabien Delpierre <
fabien.delpierre@gmail.com> wrote:

Hello,
I’m working on a custom cookbook; since yesterday, whenever I run $ berks
upload to ship the latest version of the cookbook to my Chef server,
Berkshelf goes through all the cookbook dependencies alphabetically, as
normal, skips all of them since they don’t require an update, again as
normal, then it gets to my cookbook (it’s called “srv01”) and exits
unexpectedly with this output http://pastebin.com/3GZg4SMA (Pastebin
link).

Any cookbook that would come after “srv01” alphabetically is obviously
skipped since the program exits at that point. That’s not a huge deal for
me since they don’t require any updates, but could be problematic,
generally speaking.

But really my main problem is that the cookbook I’m actually working on
doesn’t get updated on the Chef server.
If I just do a $ knife cookbook upload srv01, it uploads fine.

I’m using the Hosted Chef service and I know from looking at
status.chef.io that they’ve been having issues lately, but unfortunately
I have no idea what the error I’m getting means, and while I can work
around it, I’d like to see if I can fix it, assuming it’s something I’m
doing wrong.

What do you think? Please let me know if I need to provide additional
information.
Thanks!


#3

On Wednesday, April 22, 2015 at 10:24 AM, Fabien Delpierre wrote:

For what it’s worth, I actually just had to add a cookbook to my Berksfile, fortunately it comes before “srv01” alphabetically, and $ berks upload uploads that cookbook fine. Of course I still get the failure when it reaches my own cookbook. But that suggests that the issue is with my cookbook, somehow. I just don’t know what it could be.

On Wed, Apr 22, 2015 at 12:59 PM, Fabien Delpierre <fabien.delpierre@gmail.com (mailto:fabien.delpierre@gmail.com)> wrote:

Hello,
I’m working on a custom cookbook; since yesterday, whenever I run $ berks upload to ship the latest version of the cookbook to my Chef server, Berkshelf goes through all the cookbook dependencies alphabetically, as normal, skips all of them since they don’t require an update, again as normal, then it gets to my cookbook (it’s called “srv01”) and exits unexpectedly with this output (http://pastebin.com/3GZg4SMA) (Pastebin link).

Unfortunately, this error from the Chef Server doesn’t say exactly what the problematic data is, but the error message:

{“error”:["Invalid element in array value of ‘templates’.”]}

indicates that in the JSON representation of your cookbook content, one of the entries in templates has a value the server considers invalid. If if uploads fine with knife then this is probably a bug in berks. You can file a much more delightful bug report if you can narrow the problem down to a specific template file name (e.g., try deleting the templates one-by-one until you find out which file name is the problem). Going through that will probably give you an idea of how to work around the issue as well.


Daniel DeLeo


#4

Daniel,
Fortunately I’m only working with one template file here, so it’s easy to
narrow down, but I don’t see what could possibly be wrong with the
template: http://pastebin.com/TTLTg3Yz
You don’t have to read through it unless you feel like it, but I’m sharing
it to show that that this is very, very standard/trivial stuff.
That being said, even after moving this lone template file out of the
templates/default directory of my cookbook so that it wouldn’t be uploaded
by Berkshelf, $ berks upload still fails in the same fashion, so that
stinks.

On Wed, Apr 22, 2015 at 1:31 PM, Daniel DeLeo dan@kallistec.com wrote:

On Wednesday, April 22, 2015 at 10:24 AM, Fabien Delpierre wrote:

For what it’s worth, I actually just had to add a cookbook to my
Berksfile, fortunately it comes before “srv01” alphabetically, and $ berks
upload uploads that cookbook fine. Of course I still get the failure when
it reaches my own cookbook. But that suggests that the issue is with my
cookbook, somehow. I just don’t know what it could be.

On Wed, Apr 22, 2015 at 12:59 PM, Fabien Delpierre <
fabien.delpierre@gmail.com (mailto:fabien.delpierre@gmail.com)> wrote:

Hello,
I’m working on a custom cookbook; since yesterday, whenever I run $
berks upload to ship the latest version of the cookbook to my Chef server,
Berkshelf goes through all the cookbook dependencies alphabetically, as
normal, skips all of them since they don’t require an update, again as
normal, then it gets to my cookbook (it’s called “srv01”) and exits
unexpectedly with this output (http://pastebin.com/3GZg4SMA) (Pastebin
link).

Unfortunately, this error from the Chef Server doesn’t say exactly what
the problematic data is, but the error message:

{“error”:["Invalid element in array value of ‘templates’.”]}

indicates that in the JSON representation of your cookbook content, one of
the entries in templates has a value the server considers invalid. If if
uploads fine with knife then this is probably a bug in berks. You can
file a much more delightful bug report if you can narrow the problem down
to a specific template file name (e.g., try deleting the templates
one-by-one until you find out which file name is the problem). Going
through that will probably give you an idea of how to work around the issue
as well.


Daniel DeLeo


#5

On Wednesday, April 22, 2015 at 11:18 AM, Fabien Delpierre wrote:

Daniel,
Fortunately I’m only working with one template file here, so it’s easy to narrow down, but I don’t see what could possibly be wrong with the template: http://pastebin.com/TTLTg3Yz
You don’t have to read through it unless you feel like it, but I’m sharing it to show that that this is very, very standard/trivial stuff.
That being said, even after moving this lone template file out of the templates/default directory of my cookbook so that it wouldn’t be uploaded by Berkshelf, $ berks upload still fails in the same fashion, so that stinks.

It wouldn’t be the content in the template, but the filename that would be the issue. When you upload a cookbook, the client (knife or berks) sends the server a JSON data structure as described here: https://github.com/chef/chef/blob/44d70ba0f1b38bc3e1170ae978ad5e9a11679df3/lib/chef/cookbook_manifest.rb#L78-L112

From a quick reading of the server code https://github.com/chef/oc_erchef/blob/8a122c269746070352814fa498c57ff75eb60121/apps/chef_objects/src/chef_cookbook_version.erl#L274-L279 it is checking that each item in the templates array is a JSON object with name, path, checksum, and specificity keys (as strings) and String values.

I don’t know what the debugging options are for berks here, it would be really helpful if you could see what it was attempting to upload. Worst case scenario you can use socat as a debugging proxy. Filing a bug with berks is probably your best path forward at this point, though.


Daniel DeLeo


#6

The file name is nothing special (something.rc.erb), but that’s probably
irrelevant since the problem happens even when I move the template
completely outside of the cookbook’s directory structure.
So anyway, earlier I had a vague suspicion that maybe OS X’s ubiquitous
.DS_Store file might be the culprit but I didn’t put the theory to the
test. This time I did and deleted both .DS_Store files from the templates
and templates/default directories… and lo! $ berks upload worked.
Pretty stupid.

On Wed, Apr 22, 2015 at 2:57 PM, Daniel DeLeo dan@kallistec.com wrote:

On Wednesday, April 22, 2015 at 11:18 AM, Fabien Delpierre wrote:

Daniel,
Fortunately I’m only working with one template file here, so it’s easy
to narrow down, but I don’t see what could possibly be wrong with the
template: http://pastebin.com/TTLTg3Yz
You don’t have to read through it unless you feel like it, but I’m
sharing it to show that that this is very, very standard/trivial stuff.
That being said, even after moving this lone template file out of the
templates/default directory of my cookbook so that it wouldn’t be uploaded
by Berkshelf, $ berks upload still fails in the same fashion, so that
stinks.

It wouldn’t be the content in the template, but the filename that would be
the issue. When you upload a cookbook, the client (knife or berks) sends
the server a JSON data structure as described here:
https://github.com/chef/chef/blob/44d70ba0f1b38bc3e1170ae978ad5e9a11679df3/lib/chef/cookbook_manifest.rb#L78-L112

From a quick reading of the server code
https://github.com/chef/oc_erchef/blob/8a122c269746070352814fa498c57ff75eb60121/apps/chef_objects/src/chef_cookbook_version.erl#L274-L279
it is checking that each item in the templates array is a JSON object with
name, path, checksum, and specificity keys (as strings) and String values.

I don’t know what the debugging options are for berks here, it would be
really helpful if you could see what it was attempting to upload. Worst
case scenario you can use socat as a debugging proxy. Filing a bug with
berks is probably your best path forward at this point, though.


Daniel DeLeo