We manage several environments that use lots of auto-scaling.
We’ve iterated on this a little.
At first, we were using user-data and this just ends up being bad. There
is no way of changing this unless you have the ability to stop the
instance. To me, this is unacceptable. We ended up getting trapped when
we wanted to change some of the user-data.
We then ran JEOS with chef-client and put the role in client-config.json.
When chef would run it would configure the server for the role that it is
supposed to be. This can be easily changed or managed with chef.
The problem with all of this bootstrapping and configuration on the fly is
that when you want to auto-scale you’re triggering off of an event. You
want the machine to become available as soon as it can boot. This lead us
to baking/bundling AMI’s with the full chef run. We still assign the role
with client-config.json. Chef was used to build/configure and will then
maintain state. If it’s a minor change, we don’t care about we won’t
bundle/bake a new AMI. If we care about it, we bake/bundle.
We’ll never be impacted from scaling by waiting for a machine to configure
or if the chef server is unavailable.
With user-data or managing the role on the machine you can always just
change what roles/recipes are assigned to the role.
On Thu, Nov 10, 2011 at 12:08 PM, Aaron Abramson firstname.lastname@example.org:
I’ve thought about this, but haven’t put much work into implementing it
yet. So this is more of a concept than anything.
Set up a script listening on a server. It could be a simple php script,
Set up the user-data with a bash script that hits the listener. (wget -O
The script receives the request, and it knows 2 things, the public
hostname of the server making the request, and the “role” it is requesting.
The script makes certain assumptions (username, ssh-key, etc) and then
executes a system command, calling “knife bootstrap server -r
So, if you auto-scale official cannonical ubuntu AMI’s, simply have the
user-data “ping” your knife listener, and knife then bootstraps the server.
Perhaps such a bootstrap request feature could be built into chef-server?
On Nov 10, 2011, at 11:55 AM, John Martinez wrote:
I agree with you, using knife to provision EC2 instances is not the best
way to do this in autoscaled environments. As a temporary solution, I’ve
done exactly what you suggest. I pass along a lot of things into user-data,
and specifically, a node’s role is one of those. I read user-data and tell
Chef on the node what its role is (amongst other things). It’s a kludge.
I’m going to be looking into integrating Chef with CloudFormation, which
plays nice® with ASGs.
On Nov 10, 2011, at 9:44 AM, email@example.com firstname.lastname@example.org
Hi all, I’d like to get some feedback on how people are doing this now.
seems that since .10, the preferred way of starting Amazon EC2
instances is via
the ‘knife ec2 server create’ command, while the ‘knife ec2 instance
chef preinstalled AMIs are deprecated. That’s all fine and good for
launching instances, but what about autoscaling? When a node comes up
autoscalling, I want it to update and configure itself according to its
and environment. With the instance data method of doing thing
what’s the alternative (aside from having a custom baked AMI that I
manually update as is the normal way of doing this without chef)? If
no alternative, and I have to use the deprecated method which would
‘knife ec2 instance data’ to wire into the autoscaling group as
using this method how would I specify the environment that the server
run as too? I don’t see anyway to do this using the instance data.
command outside the server after the server it is started to set the
environment is not an option as we won’t know when a new server has
up (controlled by Amazon).
I’m interested in what other people have found to be best practice for
scenario and any advice, tips or tricks that have been learned while
work with chef and autoscaled EC2 instances.
Thanks for your thoughts!