Sanity checker for cookbooks? HTTP error 400 Field metadata.name invalid

Chef Infra (archive)
1

While using “knife upload” to upload data into my new chef 12 server, I’m suffering this error during the cookbook upload

ERROR: cookbooks failed to write: HTTP error writing: 400 “Bad Request” cause: {“error”:[“Field ‘metadata.name’ invalid”]}

This seems similar to https://github.com/chef/chef-server/issues/120.

I cannot tell which cookbook is causing this error. 2 questions:

How can I figure out which cookbook this is? using -VV for knife debug output is not telling me what I need. I’m trying to sprinkle debug lines into the chef code, but it’s very slow going and frustrating as I am not familiar with the codebase.

Is there a tool out there that will sanity check a cookbook prior to trying to upload it that would catch an error like this?

Thanks,
kallen

2

Hey there! Have you tried foodcritic? It’s got a fair amount of linting rules for cookbooks, many of which you might find useful. It has a few specific metadata-related rules.

Have you tried uploading a specific cookbook? e.g. knife cookbook upload...

Perhaps you’re using knife upload ... and not passing the right object for the server. I’d recommend knife cookbook upload and do each cookbook separately. That way you’ll know.

3

Yep, we’ve linted all of our cookbooks with foodcritic. They all pass.

The precise knife upload command invoked is:

/opt/opscode/bin/knife upload /cookbooks -c /path/to/knife.rb -VV

where the working directory contains these dirs:

clients
cookbooks
data_bags
environments

The knife version is 12.5.0.current.0.

The knife config is

chef_server_url "https://localhost/organizations/org02"
node_name "pivotal"
client_key "/etc/opscode/pivotal.pem"
versioned_cookbooks true
chef_repo_path "/ebs/chef/restore/org02/transfer"
cookbook_path "/ebs/chef/restore/org02/transfer/cookbooks"
verify_api_cert false
ssl_verify_mode :verify_none

And, well, we have 440 cookbooks to upload. I’m hoping to not have to upload each one of those one at a time in order to get to the bottom of this error.

But if that’s the best technique we’ve got… I can try it and go have dinner while it runs.

Any better techniques or tools out there?

Thanks!
kallen

4

In the debug logs, there should be some stuff from the HTTP request indicating the request path, which should include the cookbook name.

knife cookbook upload requires you to set cookbook_path in your configuration file, but it may have some nicer error handling for this case, which would be worth trying. You can use the -a flag to bulk upload everything.

5

I notice that you are specifying /cookbooks as the directory. You might want to try it without the / .

Personally, I switched from knife to berkshelf - in part exactly to better manage this type of problem. It does take a bit of getting used to, because it actually does away with the concept of a chef repository.

Kevin Keane
The NetTech
http://www.4nettech.com
Our values: Privacy, Liberty, Justice
See https://www.4nettech.com/corp/the-nettech-values.html

6

A couple questions …

First, you mention there’s some nicer error handling that might be worth trying, but it’s not clear to me from what you said what it is. Can you expand upon that?

Second, I wonder if I’m reading the debug output correctly. I’ve thrown up a gist, which I hope has the appropriate amount of “snips for brevity”: https://gist.github.com/groknaut/db2c2fefa79f6cc638ad

The last cookbook mentioned in the output is mycompany-snort 0.1.1. In the output, it does say that it successfully uploaded that cookbook. But since that is the last one mentioned, I wonder if that’s the one I should look at, and not necessarily believe lines mean success: https://gist.github.com/groknaut/db2c2fefa79f6cc638ad#file-knife-upload-output-L197-L198

Assuming the error in question, “Field metadata.name invalid” is related to the cookbook’s metadata.rb file, here’s that file. Looks ok to me … ?

name 'mycompany-snort’
maintainer 'mycompany, Inc.'
maintainer_email 'ops@mycompany.com’
license 'All rights reserved’
description ‘Installs/Configures Snort network instrusion detection/prevention system’ long_description IO.read(File.join(File.dirname(FILE), ‘README.md’))
version ‘0.1.1’

Thanks,
kallen

7

In case it could be of some help here’s what I had to do on my old cookbooks during migration from chef11 to chef12:

I hope I documented enough, feel free to ask if needed.

8

knife upload PATH and knife cookbook upload -a have different code paths which may have different error handling.

Second, I wonder if I'm reading the debug output correctly. I've thrown up a gist, which I hope has the appropriate amount of "snips for brevity": https://gist.github.com/groknaut/db2c2fefa79f6cc638ad2

Looks like you snipped the most important part. You should see a line like:

DEBUG: Initiating PUT to https://localhost/organizations/org02/cookbooks/mycompany-snort/0.1.1?force=true

Right before the error. From that URL, you should be able to figure out the cookbook name and version number.

Assuming the error in question, "Field metadata.name invalid" is related to the cookbook's metadata.rb file, here's that file.

That cookbook uploaded fine. From the debug logs, you can see that you got a 200 OK back from the server for the PUT that I quoted above.

9

Thanks for the reply. I’ll look at the different knife upload calls in the future.

I ended up finding it. Somehow we had a cookbook mycompany v0.0.0 uploaded to the chef server at some point which had no metadata file. No idea how that happened.

In the 77075 lines of debug output of that knife upload command, during which the error was thrown to STDERR at the end of the run, on line 7688 I now realize the precise error message appears for that bad cookbook. Next time I’ll know to search more thru the entirety of the debug output for the error, (and look for status=400 in the opscode-erchef requests.log)

Thanks!
kallen