Quorum Failure In Chef Delivery Push Jobs


I am suddenly getting quorum failure on any run from converge from Delivery… I had been running fine up until last Friday.

I am able to log in to the nodes in any of the environments and can run chef-client locally.

Running a knife node list lists all of my nodes, however when running a knife node status, none of my Acceptance, Union, Rehearsal, or Delivered nodes are showing up.

Running knife node status returns the node as “unavailable”

I have tried removing a node and client from Chef, deleting and re-creating the VM and re-bootstrapping the node into Chef, and while the node is created in Chef, I still am getting the “unavailable” status on a Knife node status

Any Ideas of where I went pear shaped???


I have been reviewing a few other online resources and it seems that my push-jobs clients are failing to start… when I log into a node and attempt to start the pushy-client I get the following error upon attempting to retrieve the configuration from the Chef server:
ERROR: [] Exiting: No key returned from server; server may be using 1.x protocol. The config flag ‘allow_unencrypted’ disables encryption and allows use of 1.x server. Use with caution!

Anyone have any idea which configuration is causing the error???


All, just wanted to post that I figured this out. The issue began a few days ago when the push-jobs client updated because I had not pinned a version. That led to an issue with the clients not being able to converge. I was able to get around that problem by moving to the 2.7.0 version of the push-jobs client, until last Friday, when I started getting the “quorum failure” notices on the attempt to converge from Delivery, yet I was able to converge manually from a chef-client run on the node. After doing further research, I found that the push-jobs client had been again updated to solve this issue, with a default attribute to set a new config option. Updating to the new 2.8.1 client and writing a wrapper cookbook to override this new default attribute has resolved the issue.

Just in case anyone else has this issue and stumbles upon the original post…


Thank you so much for this post/feedback! We really appreciate sharing the knowledge … If there is anything else you find please let us know!

Have a great week.
Salim Afiune