A few weeks ago, I finally updated from chef 11 to chef 12 (I believe it was 12.0.3 at the time). The upgrade went smoothly and without a hitch that I could tell.
A few days ago, I updated to chef-server-core 12.1.0, and all hell broke loose. Well, not all hell, but some critical parts of chef aren’t working.
The RPM told me that it detected chef 11, and to run chef-server-ctl update. I wasn’t sure why it still saw chef 11 since I already upgraded to 12 a while ago, but I followed the instruction.
Now my chef client runs are failing with an “internal service error”. knife node show works without a problem, so he node data actually exists.
I haven’t found anything that seemed relevant in any log files, but then I’m ot sure what I’d be looking for. I did see error 500s in the nginx logs for the URL: POST /organizations/myorganization/environments/_default/cookbook_versions HTTP/1.1
How do I go about troubleshooting and fixing this?
Here is what happens on one of my clients:
Starting Chef Client, version 12.3.0
resolving cookbooks for run list: [ <redacted - the run list is correct> ]
Error Resolving Cookbooks for Run List:
Unknown Server Error:
The server had a fatal error attempting to load the node data.
internal service error
Running handlers complete
Chef Client failed. 0 resources updated in 108.207621603 seconds
Our values: Privacy, Liberty, Justice