Triggering chef runs/deploys:

Chef Infra (archive)
1

So I’m at the point in my chef buildout that I need to start working on application deploys. The deploy resource is clear enough to work with. My question is, how do I trigger a deployment?

I understand that if I use the ‘revision’ deploy strategy, I can set up a branch for the production environment and trigger deploys by promoting to that branch and waiting for the next chef run, but the guys I’m working with aren’t crazy about that (and I have my hesitations as well). Right now, the only alternative I can see is to keep the deploy action disabled normally and when we want to deploy: enable deploy, ssh to all the servers and run the client, disable deploy.

So how does everyone else do this? The part of the above I really dont like is the “ssh to all the servers” part. I dont like using ssh for command and control - shell is too fragile and generic (try managing 120+ applications on 1500+ servers with nothing but ssh + scripts… bad experience), ssh key management is a pain, etc. This leads into my side question: is there some other way to trigger (push) a chef run?

FWIW the apps I’m working with right now are jruby on rails apps deployed in glassfish (using the glassfish gem) and some miscellaneous jruby daemons started with hashdot.

As usual, any insight or discussion is appreciated
michael

2

On 26 August 2010 06:25, Leinartas, Michael
MICHAEL.LEINARTAS@orbitz.com wrote:

So I'm at the point in my chef buildout that I need to start working on
application deploys. The deploy resource is clear enough to work with. My
question is, how do I trigger a deployment?

One option would be to create a data bag for applications, and trigger
deployments based on changes to those (possibly put the revision you
want deployed into the data bag so that the right version hits the
production servers). Alternatively, you might be better off using a
different tool for application deployments if continuous deployment
isn't something you want to do.

I'm curious as to what the problem you see with branching is though.
As well as providing an easy place for Chef to hook into, it provides
you with one place that everybody knows is the production code, so
there's never any doubt as to which version is on the production
servers.

Jon

3

Hello!

We wrote the 'application' cookbook to do application deployments via Chef-runs. It was originally written for a Rails application infrastructure, but it can be extended with new recipes to handle other kinds of application stacks (like jruby/glassfish, or php/drupal, python/django).

Our blog post introducing it:

A recent blog post of a Chef user implementing the cookbook for his application:

And the README documentation for the cookbook itself:

http://github.com/opscode/cookbooks/blob/master/application/README.md

(though you should download it from the cookbooks site with knife to put it in your local chef-repo.)

That said, we don't currently have a framework in place for triggering Chef runs outside of using ssh (i.e., knife ssh). That is something we'd like to implement, and if you have ideas about it please let us know!

On Aug 25, 2010, at 11:25 PM, Leinartas, Michael wrote:

So I'm at the point in my chef buildout that I need to start working on application deploys. The deploy resource is clear enough to work with. My question is, how do I trigger a deployment?

I understand that if I use the 'revision' deploy strategy, I can set up a branch for the production environment and trigger deploys by promoting to that branch and waiting for the next chef run, but the guys I'm working with aren't crazy about that (and I have my hesitations as well). Right now, the only alternative I can see is to keep the deploy action disabled normally and when we want to deploy: enable deploy, ssh to all the servers and run the client, disable deploy.

So how does everyone else do this? The part of the above I really dont like is the "ssh to all the servers" part. I dont like using ssh for command and control - shell is too fragile and generic (try managing 120+ applications on 1500+ servers with nothing but ssh + scripts.. bad experience), ssh key management is a pain, etc. This leads into my side question: is there some other way to trigger (push) a chef run?

FWIW the apps I'm working with right now are jruby on rails apps deployed in glassfish (using the glassfish gem) and some miscellaneous jruby daemons started with hashdot.

As usual, any insight or discussion is appreciated
michael

--
Opscode, Inc
Joshua Timberman, Technical Evangelist
C: 720.334.RUBY E: joshua@opscode.com

4

On 26.08.2010, at 16:14, Joshua Timberman wrote:

That said, we don't currently have a framework in place for triggering Chef runs outside of using ssh (i.e., knife ssh). That is something we'd like to implement, and if you have ideas about it please let us know!

isn't rightscale using nanite GitHub - ezmobius/nanite: self assembling fabric of ruby daemons for command and control?

5

On 26.08.2010, at 16:14, Joshua Timberman wrote:

That said, we don't currently have a framework in place for triggering Chef runs outside of using ssh (i.e., knife ssh). That is something we'd like to implement, and if you have ideas about it please let us know!

isn't rightscale using nanite GitHub - ezmobius/nanite: self assembling fabric of ruby daemons for command and control?

At Scalarium we are using Nanite to push Chef-Solo configurations to
every server, depending on the role and the task (deploy, first setup,
etc.). If works fine for us while managing several hundred servers but
has its gotchas.

Jonathan

--
Jonathan Weiss
http://blog.innerewut.de
http://twitter.com/jweiss

6

On 26.08.2010, at 18:50, Jonathan Weiss wrote:

On 26.08.2010, at 16:14, Joshua Timberman wrote:

That said, we don't currently have a framework in place for triggering Chef runs outside of using ssh (i.e., knife ssh). That is something we'd like to implement, and if you have ideas about it please let us know!

isn't rightscale using nanite GitHub - ezmobius/nanite: self assembling fabric of ruby daemons for command and control?

At Scalarium we are using Nanite to push Chef-Solo configurations to
every server, depending on the role and the task (deploy, first setup,
etc.). If works fine for us while managing several hundred servers but
has its gotchas.

is that any better than rsync and ssh? which i do with your webistrano :wink:

cu edi

--
DI Edmund Haselwanter, edmund@haselwanter.com, http://edmund.haselwanter.com/
http://www.iteh.at | Facebook | http://at.linkedin.com/in/haselwanteredmund

7

is that any better than rsync and ssh? which i do with your webistrano :wink:

Yes as it uses RabbitMQ for communication which means that the "push"
is async and you can react to different responses differently.
Our backend is also all Nanite agents, so newly booted servers get
their Chef configuration upon demand and "configure themselves"
according to the state of the rest of the cluster and to the recipes
that apply to their role.

Jonathan

--
Jonathan Weiss
http://blog.innerewut.de
http://twitter.com/jweiss

8

Branching makes sense to me and I can live with it, but I come from an ops background managing java apps where we have discrete build, packaging, installation, and deploy steps so it's unfamiliar - I dont know where the traps lay and honestly, I haven't sat down and thought long and hard about our deployment cases yet. In any case, the guy's I'm working with (who have final say) want the deployment trigger to be separate, even if there is a production branch. I mean if this is the standard way of approaching the problem and I can't find any reason not to, I'll try and convince them to go this route.

Mostly though at this point I'm trying to survey what options are feasible and in current use by others in the community.

michael

On 8/26/10 4:52 AM, "Jon Wood" jon@blankpad.net wrote:

On 26 August 2010 06:25, Leinartas, Michael
MICHAEL.LEINARTAS@orbitz.com wrote:

So I'm at the point in my chef buildout that I need to start working on
application deploys. The deploy resource is clear enough to work with. My
question is, how do I trigger a deployment?

One option would be to create a data bag for applications, and trigger
deployments based on changes to those (possibly put the revision you
want deployed into the data bag so that the right version hits the
production servers). Alternatively, you might be better off using a
different tool for application deployments if continuous deployment
isn't something you want to do.

I'm curious as to what the problem you see with branching is though.
As well as providing an easy place for Chef to hook into, it provides
you with one place that everybody knows is the production code, so
there's never any doubt as to which version is on the production
servers.

Jon

9

I saw mention of nanite in a presentation I watched when I was first checking out chef a couple months back but hadn't seen any mention since then. The project looks neat and I'd like to check it out, though with my current timeline I may end up opting for an ssh trigger and changing it out later.

Are you aware of anything out there that builds on nanite? That is, existing chef integration or something that'd give me a head start for some simple remote executions. The examples included in the project look like they illustrate a lot of the potential, but the reason I shied away from it in the first place is because it looks like a decent investment in time and thinking and I'm currently putting all that bandwidth to use getting everything added to Chef.

Thanks, good to know this is a potential option
michael

On 8/26/10 1:54 PM, "Jonathan Weiss" jw@innerewut.de wrote:

is that any better than rsync and ssh? which i do with your webistrano :wink:

Yes as it uses RabbitMQ for communication which means that the "push"
is async and you can react to different responses differently.
Our backend is also all Nanite agents, so newly booted servers get
their Chef configuration upon demand and "configure themselves"
according to the state of the rest of the cluster and to the recipes
that apply to their role.

Jonathan

--
Jonathan Weiss
http://blog.innerewut.de
http://twitter.com/jweiss