I’m looking for suggestions on how to perform a canary deployment within an environment. As a concrete example, suppose I have:
- a java role
- a production environment
- multiple nodes with the java role in the production environment
I want to be able to upgrade a single node (role[java] env[production]) to java 8 and if, after upgrading, the java processes on that node pass validation checks, the rollout should continue for the remaining nodes.
Any advice is greatly appreciated!
A quick simple answer that I can elaborate on if needed:
Use role cookbooks and environments with version pins for those cookbooks.
Push your changes as a new cookbook version, update the pin for your “canary” environment, test, and move to your other environments.
Delivery basically operates in this manner. I highly suggest it if you don’t already have a ci/cd suite.
We’ve successfully used this method for ~3 years here at EverTrue, with two environments (“stage” and “prod”). Works really well, but depends on the non-canary env having things pinned very tightly, and your top-level cookbooks themselves being relatively tightly pinned, so cookbooks not pinned at the top level don’t end up shipping straight to both envs.
Lead DevOps Engineer