Advice for canary-style deployments

Chef Infra (archive)
1

Hi,

I’m looking for suggestions on how to perform a canary deployment within an environment. As a concrete example, suppose I have:

  • a java role
  • a production environment
  • multiple nodes with the java role in the production environment

I want to be able to upgrade a single node (role[java] env[production]) to java 8 and if, after upgrading, the java processes on that node pass validation checks, the rollout should continue for the remaining nodes.

Any advice is greatly appreciated!

2

A quick simple answer that I can elaborate on if needed:
Use role cookbooks and environments with version pins for those cookbooks.
Push your changes as a new cookbook version, update the pin for your “canary” environment, test, and move to your other environments.

Delivery basically operates in this manner. I highly suggest it if you don’t already have a ci/cd suite.
–Jp Robinson

3

We’ve successfully used this method for ~3 years here at EverTrue, with two environments (“stage” and “prod”). Works really well, but depends on the non-canary env having things pinned very tightly, and your top-level cookbooks themselves being relatively tightly pinned, so cookbooks not pinned at the top level don’t end up shipping straight to both envs.

Jeff Byrnes
Lead DevOps Engineer
EverTrue
@thejeffbyrnes