# Newbie on autoscaling with Chef

**URL:** <https://discourse.chef.io/t/newbie-on-autoscaling-with-chef/4742>\
**Category:** Chef Infra (archive)\
**Created:** [November 26, 2013, 10:25am UTC](https://discourse.chef.io/t/newbie-on-autoscaling-with-chef/4742 "2013-11-26T10:25:34Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![James\_Crosswell](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/james_crosswell/32/425_2.png) [@James\_Crosswell](https://discourse.chef.io/u/James_Crosswell)\
**Post date:** [November 26, 2013, 10:25am UTC](https://discourse.chef.io/t/newbie-on-autoscaling-with-chef/4742/1 "2013-11-26T10:25:34Z")

</div>

I’d like to use Chef to manage an application hosted on Amazon’s EC2  
platform. There are three “components” to the appliction:

1. Web application
2. Database server
3. Background worker/application

I’d like to have the 3rd component (background worker) automatically scale  
based on CPU usage. So there might be one permanent background worker, but  
if CPU levels on that worker remained above x% for over y seconds then a  
couple of extra background workers would automatically be powered up.  
Similarly, if CPU levels dropped below z% for y seconds, a couple of  
background workers would automatically be terminated.

I think a major piece of the puzzle that is missing for me is how to  
register new instances as nodes with the Chef server along with the  
necessary cookbooks. All of the stuff I’ve read on using chef so far  
appears to assume a static list of nodes that are being managed manually  
from a workstation. What I’d like to do is have new nodes (not necessarily  
workstations) register themselves to be configured by Chef… and also drop  
off the list of managed nodes when they are terminated.

Is this possible using Chef?

Kind Regards,

James Crosswell  
Founder | Tea Boy  
[www.mentaldesk.com](http://www.mentaldesk.com)

---

<div class="post-metadata">

**Author:** ![dcondomitti](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/dcondomitti/32/104_2.png) [@dcondomitti](https://discourse.chef.io/u/dcondomitti)\
**Post date:** [November 26, 2013, 10:50am UTC](https://discourse.chef.io/t/newbie-on-autoscaling-with-chef/4742/2 "2013-11-26T10:50:24Z")

</div>

All of this is absolutely possible. It sounds complicated at first but when it’s broken down into the three different tasks it’s really not that bad:

Automatic scaling with ASGs is most effectively accomplished using custom metrics[1]. You can do this based off of the built in metrics for EC2 nodes for memory or CPU usage but normally queue depth is a more accurate measurement of when you need to bring up more workers. If you have a fixed number of workers running jobs they normally won’t dramatically increase CPU load on their own.

Registering the workers can be handled by either including your chef validator and client.rb (with the chef client preinstalled) in a custom AMI[2] or by using instance metadata to handle installing it[3]. Creating an AMI with your validator and chef-client already preinstalled would be the easiest to accomplish as the only metadata that needs to be passed in is a role or runlist.

Cleanup can be handled with shutdown hooks depending on the permissions on your chef server. At my org we use knife delete\_[client,node] in orchestration scripts for cleaning up production/staging nodes but allow vagrant nodes to clean themselves up. There’s a good example for self-cleanup of nodes on cotap’s engineering blog [4] though if you’re using Ubutntu (we’re not) but a similar process can be built for any other distro.

[1] [Autoscaling with custom metrics – That's Geeky](http://www.thatsgeeky.com/2012/01/autoscaling-with-custom-metrics/)  
[2] [Account Suspended](http://marksdevserver.com/2013/06/19/chef-bootstrap-autoscaled-ec2-instance/)  
[3] [aws advent](http://awsadvent.tumblr.com/post/37773106407/bootstrap-cfg-mgmt-aws)  
[4] [Cotap Engineering — Instance Deregistration with Chef and Sensu](http://engineering.cotap.com/post/66396370532/instance-deregistration-with-chef-and-sensu)

On Tuesday, November 26, 2013 at 2:25 AM, James Crosswell wrote:

> I'd like to use Chef to manage an application hosted on Amazon's EC2 platform. There are three "components" to the appliction:  
> Web application  
> Database server  
> Background worker/application
> 
> I'd like to have the 3rd component (background worker) automatically scale based on CPU usage. So there might be one permanent background worker, but if CPU levels on that worker remained above x% for over y seconds then a couple of extra background workers would automatically be powered up. Similarly, if CPU levels dropped below z% for y seconds, a couple of background workers would automatically be terminated.
> 
> I think a major piece of the puzzle that is missing for me is how to register new instances as nodes with the Chef server along with the necessary cookbooks. All of the stuff I've read on using chef so far appears to assume a static list of nodes that are being managed manually from a workstation. What I'd like to do is have new nodes (not necessarily workstations) register themselves to be configured by Chef... and also drop off the list of managed nodes when they are terminated.
> 
> Is this possible using Chef?  
> Kind Regards,
> 
> James Crosswell  
> Founder | Tea Boy  
> [www.mentaldesk.com](http://www.mentaldesk.com) ([http://www.mentaldesk.com](http://www.mentaldesk.com))

---

<div class="post-metadata">

**Author:** ![Ryutaro\_YOSHIBA](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/ryutaro_yoshiba/32/439_2.png) [@Ryutaro\_YOSHIBA](https://discourse.chef.io/u/Ryutaro_YOSHIBA)\
**Post date:** [November 26, 2013, 11:03am UTC](https://discourse.chef.io/t/newbie-on-autoscaling-with-chef/4742/3 "2013-11-26T11:03:08Z")

</div>

Hi,

I've just wrote sample code which let new servers register automatically to chef server and deregister when instances are terminated.

> <https://github.com/ryuzee/sandbox-devops/blob/master/ec2-chef-server-integration-sample/init.sh>

When creating autoscaling, you can set user\_data at launch configuration and if user\_data is set and user\_data starts with #!, cloud-init will execute this user\_data as executable script.

Another point is that it is not good to store validation key in the AMI. I recommend you to set IAM role to instances(which can be set at launch configuration) and you can get validation key from S3 bucket.

--  
Ryutaro YOSHIBA

> On 2013/11/26, at 19:25, James Crosswell [james@mentaldesk.com](mailto:james@mentaldesk.com) wrote:
> 
> I'd like to use Chef to manage an application hosted on Amazon's EC2 platform. There are three "components" to the appliction:  
> Web application  
> Database server  
> Background worker/application  
> I'd like to have the 3rd component (background worker) automatically scale based on CPU usage. So there might be one permanent background worker, but if CPU levels on that worker remained above x% for over y seconds then a couple of extra background workers would automatically be powered up. Similarly, if CPU levels dropped below z% for y seconds, a couple of background workers would automatically be terminated.
> 
> I think a major piece of the puzzle that is missing for me is how to register new instances as nodes with the Chef server along with the necessary cookbooks. All of the stuff I've read on using chef so far appears to assume a static list of nodes that are being managed manually from a workstation. What I'd like to do is have new nodes (not necessarily workstations) register themselves to be configured by Chef... and also drop off the list of managed nodes when they are terminated.
> 
> Is this possible using Chef?
> 
> Kind Regards,
> 
> James Crosswell  
> Founder | Tea Boy  
> [www.mentaldesk.com](http://www.mentaldesk.com)

---

<div class="post-metadata">

**Author:** ![James\_Crosswell](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/james_crosswell/32/425_2.png) [@James\_Crosswell](https://discourse.chef.io/u/James_Crosswell)\
**Post date:** [November 26, 2013, 11:25am UTC](https://discourse.chef.io/t/newbie-on-autoscaling-with-chef/4742/4 "2013-11-26T11:25:24Z")

</div>

Thanks guys. The nodes in my case are Windows nodes but I'm sure I can do  
the same with Windows startup/shutdown scripts, until such a time as the  
code can be ported to Linux. It looks like a fair amount of work will be  
involved in setting this all up, but at least I know I'm not chasing wild  
geese now 🙂

Kind Regards,

James Crosswell  
Founder | Tea Boy

> **[Mental Desk](https://mentaldesk.com/)**
>
> Software Engineering & Product Management consulting. Websites | Apps | APIs | AI.

On 26 November 2013 11:50, Daniel Condomitti [daniel@condomitti.com](mailto:daniel@condomitti.com) wrote:

> All of this is absolutely possible. It sounds complicated at first but  
> when it’s broken down into the three different tasks it’s really not that  
> bad:
> 
> Automatic scaling with ASGs is most effectively accomplished using custom  
> metrics[1]. You can do this based off of the built in metrics for EC2 nodes  
> for memory or CPU usage but normally queue depth is a more accurate  
> measurement of when you need to bring up more workers. If you have a fixed  
> number of workers running jobs they normally won’t dramatically increase  
> CPU load on their own.
> 
> Registering the workers can be handled by either including your chef  
> validator and client.rb (with the chef client preinstalled) in a custom  
> AMI[2] or by using instance metadata to handle installing it[3]. Creating  
> an AMI with your validator and chef-client already preinstalled would  
> be the easiest to accomplish as the only metadata that needs to be passed  
> in is a role or runlist.
> 
> Cleanup can be handled with shutdown hooks depending on the permissions on  
> your chef server. At my org we use knife delete\_[client,node] in  
> orchestration scripts for cleaning up production/staging nodes but allow  
> vagrant nodes to clean themselves up. There’s a good example for  
> self-cleanup of nodes on cotap’s engineering blog [4] though if you’re  
> using Ubutntu (we’re not) but a similar process can be built for any other  
> distro.
> 
> [1] [http://www.thatsgeeky.com/2012/01/autoscaling-with-custom-metrics/](http://www.thatsgeeky.com/2012/01/autoscaling-with-custom-metrics/)  
> [2]  
> [Account Suspended](http://marksdevserver.com/2013/06/19/chef-bootstrap-autoscaled-ec2-instance/)  
> [3] [aws advent](http://awsadvent.tumblr.com/post/37773106407/bootstrap-cfg-mgmt-aws)  
> [4]  
> [Cotap Engineering — Instance Deregistration with Chef and Sensu](http://engineering.cotap.com/post/66396370532/instance-deregistration-with-chef-and-sensu)
> 
> On Tuesday, November 26, 2013 at 2:25 AM, James Crosswell wrote:
> 
> I'd like to use Chef to manage an application hosted on Amazon's EC2  
> platform. There are three "components" to the appliction:
> 
> 1. Web application
> 2. Database server
> 3. Background worker/application
> 
> I'd like to have the 3rd component (background worker) automatically scale  
> based on CPU usage. So there might be one permanent background worker, but  
> if CPU levels on that worker remained above x% for over y seconds then a  
> couple of extra background workers would automatically be powered up.  
> Similarly, if CPU levels dropped below z% for y seconds, a couple of  
> background workers would automatically be terminated.
> 
> I think a major piece of the puzzle that is missing for me is how to  
> register new instances as nodes with the Chef server along with the  
> necessary cookbooks. All of the stuff I've read on using chef so far  
> appears to assume a static list of nodes that are being managed manually  
> from a workstation. What I'd like to do is have new nodes (not necessarily  
> workstations) register themselves to be configured by Chef... and also drop  
> off the list of managed nodes when they are terminated.
> 
> Is this possible using Chef?
> 
> Kind Regards,
> 
> James Crosswell  
> Founder | Tea Boy  
> [www.mentaldesk.com](http://www.mentaldesk.com)