# Chef Orchestration

**URL:** <https://discourse.chef.io/t/chef-orchestration/3217>\
**Category:** Chef Infra (archive)\
**Created:** [October 22, 2012, 5:52pm UTC](https://discourse.chef.io/t/chef-orchestration/3217 "2012-10-22T17:52:22Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Mark\_Pimentel](https://avatars.discourse-cdn.com/v4/letter/m/9f8e36/32.png) [@Mark\_Pimentel](https://discourse.chef.io/u/Mark_Pimentel)\
**Post date:** [October 22, 2012, 5:52pm UTC](https://discourse.chef.io/t/chef-orchestration/3217/1 "2012-10-22T17:52:22Z")

</div>

Just curious to know if anyone is using chef to do multi-node deployment  
orchestration.

If so, how are you conducting the orchestration?

What other non-chef tools are you using for the orchestration?

–  
Thanks,

Mark

---

<div class="post-metadata">

**Author:** ![Bryan\_Berry](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/bryan_berry/32/213_2.png) [@Bryan\_Berry](https://discourse.chef.io/u/Bryan_Berry)\
**Post date:** [October 22, 2012, 6:37pm UTC](https://discourse.chef.io/t/chef-orchestration/3217/2 "2012-10-22T18:37:21Z")

</div>

knife can do some basic orchestration tasks but it can't provide any  
transaction support. If you want to set up 3 servers, you can just  
wrap around the knife api using rake or thor tasks. However, if you  
actually want to have any real coordination you will need a  
distributed lock manager, like Apache ZooKeeper. However, though I  
have no experience w/ it. I know that heroku created doozerd, which  
may be lighter weight, but I have no idea how well-maintained or  
complete it is.

On Mon, Oct 22, 2012 at 7:52 PM, Mark Pimentel [markpimentel22@gmail.com](mailto:markpimentel22@gmail.com) wrote:

> Just curious to know if anyone is using chef to do multi-node deployment  
> orchestration.
> 
> If so, how are you conducting the orchestration?
> 
> What other non-chef tools are you using for the orchestration?
> 
> --  
> Thanks,
> 
> Mark

---

<div class="post-metadata">

**Author:** ![realityforge](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/realityforge/32/395_2.png) [@realityforge](https://discourse.chef.io/u/realityforge)\
**Post date:** [October 22, 2012, 10:25pm UTC](https://discourse.chef.io/t/chef-orchestration/3217/3 "2012-10-22T22:25:14Z")

</div>

Hi,

On Tue, Oct 23, 2012 at 4:52 AM, Mark Pimentel [markpimentel22@gmail.com](mailto:markpimentel22@gmail.com)wrote:

> Just curious to know if anyone is using chef to do multi-node deployment  
> orchestration.
> 
> If so, how are you conducting the orchestration?
> 
> What other non-chef tools are you using for the orchestration?

When we need to do a multi-node release we typically;

- flip the release flag wherever that may be (typically this means updating  
a data bag item with a new version)
- each release is broken into several stages, for each stage we
  - perform a search to return a list of nodes to converge
  - converge a batch of nodes from the list of nodes until there are none  
left to converge

if any converge fails we have a semi-adhoc process semi automated process  
of roll back and correct which usually just means reverting the data bag  
item and re running the release.

We currently drive this from jenkins and had planned to use rundeck but  
never quite got around to it.

--  
Cheers,

Peter Donald

---

<div class="post-metadata">

**Author:** ![Mark\_Pimentel](https://avatars.discourse-cdn.com/v4/letter/m/9f8e36/32.png) [@Mark\_Pimentel](https://discourse.chef.io/u/Mark_Pimentel)\
**Post date:** [October 23, 2012, 12:54pm UTC](https://discourse.chef.io/t/chef-orchestration/3217/4 "2012-10-23T12:54:48Z")

</div>

I get the release flag part as I have considered doing that as a trigger  
point to trip the deployment.  
I was looking for some way to manage an upgrade of an agent on 100+ nodes  
that depend on whether the agent is running a job or not. So if running a  
job, wait, otherwise shutdown, upgrade, start.

As for zookeeper, while I have taken a look, is there anything lighter  
weight? Zookeeper seems like quite the application.

Any suggestions would be greatly appreciated.

Thanks.

On Mon, Oct 22, 2012 at 6:25 PM, Peter Donald [peter@realityforge.org](mailto:peter@realityforge.org)wrote:

> Hi,
> 
> On Tue, Oct 23, 2012 at 4:52 AM, Mark Pimentel [markpimentel22@gmail.com](mailto:markpimentel22@gmail.com)wrote:
> 
> > Just curious to know if anyone is using chef to do multi-node deployment  
> > orchestration.
> > 
> > If so, how are you conducting the orchestration?
> > 
> > What other non-chef tools are you using for the orchestration?
> 
> When we need to do a multi-node release we typically;
> 
> - flip the release flag wherever that may be (typically this means  
> updating a data bag item with a new version)
> - each release is broken into several stages, for each stage we
> - perform a search to return a list of nodes to converge
> - converge a batch of nodes from the list of nodes until there are none  
> left to converge
> 
> if any converge fails we have a semi-adhoc process semi automated process  
> of roll back and correct which usually just means reverting the data bag  
> item and re running the release.
> 
> We currently drive this from jenkins and had planned to use rundeck but  
> never quite got around to it.
> 
> --  
> Cheers,
> 
> Peter Donald

--  
Thanks,

Mark

---

<div class="post-metadata">

**Author:** ![Bryan\_Berry](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/bryan_berry/32/213_2.png) [@Bryan\_Berry](https://discourse.chef.io/u/Bryan_Berry)\
**Post date:** [October 23, 2012, 12:58pm UTC](https://discourse.chef.io/t/chef-orchestration/3217/5 "2012-10-23T12:58:11Z")

</div>

you could do that with knife exec

You don't really need orchestration as the operation completed on each  
node is not dependent on the state of other nodes

[http://wiki.opscode.com/display/chef/Knife+Exec](http://wiki.opscode.com/display/chef/Knife+Exec)

On Tue, Oct 23, 2012 at 2:54 PM, Mark Pimentel [markpimentel22@gmail.com](mailto:markpimentel22@gmail.com) wrote:

> I get the release flag part as I have considered doing that as a trigger  
> point to trip the deployment.  
> I was looking for some way to manage an upgrade of an agent on 100+ nodes  
> that depend on whether the agent is running a job or not. So if running a  
> job, wait, otherwise shutdown, upgrade, start.
> 
> As for zookeeper, while I have taken a look, is there anything lighter  
> weight? Zookeeper seems like quite the application.
> 
> Any suggestions would be greatly appreciated.
> 
> Thanks.
> 
> On Mon, Oct 22, 2012 at 6:25 PM, Peter Donald [peter@realityforge.org](mailto:peter@realityforge.org)  
> wrote:
> 
> > Hi,
> > 
> > On Tue, Oct 23, 2012 at 4:52 AM, Mark Pimentel [markpimentel22@gmail.com](mailto:markpimentel22@gmail.com)  
> > wrote:
> > 
> > > Just curious to know if anyone is using chef to do multi-node deployment  
> > > orchestration.
> > > 
> > > If so, how are you conducting the orchestration?
> > > 
> > > What other non-chef tools are you using for the orchestration?
> > 
> > When we need to do a multi-node release we typically;
> > 
> > - flip the release flag wherever that may be (typically this means  
> > updating a data bag item with a new version)
> > - each release is broken into several stages, for each stage we
> > - perform a search to return a list of nodes to converge
> > - converge a batch of nodes from the list of nodes until there are none  
> > left to converge
> > 
> > if any converge fails we have a semi-adhoc process semi automated process  
> > of roll back and correct which usually just means reverting the data bag  
> > item and re running the release.
> > 
> > We currently drive this from jenkins and had planned to use rundeck but  
> > never quite got around to it.
> > 
> > --  
> > Cheers,
> > 
> > Peter Donald
> 
> --  
> Thanks,
> 
> Mark
