# Knife ec2 plugin

**URL:** <https://discourse.chef.io/t/knife-ec2-plugin/3106>\
**Category:** Chef Infra (archive)\
**Created:** [September 13, 2012, 2:46pm UTC](https://discourse.chef.io/t/knife-ec2-plugin/3106 "2012-09-13T14:46:42Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![Maven\_User](https://avatars.discourse-cdn.com/v4/letter/m/c57346/32.png) [@Maven\_User](https://discourse.chef.io/u/Maven_User)\
**Post date:** [September 13, 2012, 2:46pm UTC](https://discourse.chef.io/t/knife-ec2-plugin/3106/1 "2012-09-13T14:46:42Z")

</div>

Hi all -

With knife’s ec2 plugin (which is awesome), I can create and instance from  
an ami, but I cannot go the other way.

The use case is this:

virgin ubuntu 12.X ami -\> knife clone/bootstrap with our product -\> create  
ami from this instance (for consumption elsewhere)

Before I start diving down this rabbit hole, is/was there a reason NOT to  
do this? Time? Or was there a technical reason? Does this feature exist  
and I don’t see it?

Just curious, seems like a good feature to have but don’t want to waste a  
ton of time if there was a technical reason NOT to do this.

---

<div class="post-metadata">

**Author:** ![kallistec](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/kallistec/32/23_2.png) [@kallistec](https://discourse.chef.io/u/kallistec)\
**Post date:** [September 13, 2012, 4:01pm UTC](https://discourse.chef.io/t/knife-ec2-plugin/3106/2 "2012-09-13T16:01:20Z")

</div>

On Thursday, September 13, 2012 at 7:46 AM, Maven User wrote:

> Hi all -
> 
> With knife's ec2 plugin (which is awesome), I can create and instance from an ami, but I cannot go the other way.
> 
> The use case is this:
> 
> virgin ubuntu 12.X ami -\> knife clone/bootstrap with our product -\> create ami from this instance (for consumption elsewhere)
> 
> Before I start diving down this rabbit hole, is/was there a reason NOT to do this? Time? Or was there a technical reason? Does this feature exist and I don't see it?
> 
> Just curious, seems like a good feature to have but don't want to waste a ton of time if there was a technical reason NOT to do this.  
> Creating an AMI is an involved process. Most people only follow the pattern you describe when their initial chef-client runs take a prohibitive amount of time and they need really fast instance creation for things like autoscaling.

--  
Daniel DeLeo

---

<div class="post-metadata">

**Author:** ![johnmartinez](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/johnmartinez/32/558_2.png) [@johnmartinez](https://discourse.chef.io/u/johnmartinez)\
**Post date:** [September 13, 2012, 4:15pm UTC](https://discourse.chef.io/t/knife-ec2-plugin/3106/3 "2012-09-13T16:15:13Z")

</div>

On Sep 13, 2012, at 9:01 AM, Daniel DeLeo [dan@kallistec.com](mailto:dan@kallistec.com) wrote:

> On Thursday, September 13, 2012 at 7:46 AM, Maven User wrote:
> 
> > Hi all -
> > 
> > With knife's ec2 plugin (which is awesome), I can create and instance from an ami, but I cannot go the other way.
> > 
> > The use case is this:
> > 
> > virgin ubuntu 12.X ami -\> knife clone/bootstrap with our product -\> create ami from this instance (for consumption elsewhere)
> > 
> > Before I start diving down this rabbit hole, is/was there a reason NOT to do this? Time? Or was there a technical reason? Does this feature exist and I don't see it?
> > 
> > Just curious, seems like a good feature to have but don't want to waste a ton of time if there was a technical reason NOT to do this.  
> > Creating an AMI is an involved process. Most people only follow the pattern you describe when their initial chef-client runs take a prohibitive amount of time and they need really fast instance creation for things like autoscaling.

Concur. From my experience, the two main reasons I use to create custom AMIs in AWS to use in conjunction with Chef:

To cut-down on instance boot up time (as described by Daniel)  
You are using Autoscaling or CloudFormation where the knife API command can't be used

-john

---

<div class="post-metadata">

**Author:** ![Peter\_Norton](https://avatars.discourse-cdn.com/v4/letter/p/dec6dc/32.png) [@Peter\_Norton](https://discourse.chef.io/u/Peter_Norton)\
**Post date:** [September 13, 2012, 4:22pm UTC](https://discourse.chef.io/t/knife-ec2-plugin/3106/4 "2012-09-13T16:22:29Z")

</div>

Creating an AMI is tricky because there are 2 varieties of AMIs: 1 is the  
instance-store, the other is the ebs-backed. They each require a different  
set of actions. The instance-store requires that you place the amazon ssl  
key and cert on the instance to be stamped, while the ebs method requires  
that to get a consistent image you need to reboot the instance. Both  
require that you then do some stuff after the snapshot is taken etc.

So it's not trivially simple, but the above will help you get started. I  
have some ironfan-based clusters and recipes to make this work, but ironfan  
isn't required to do this yourself. If you ping me I can probably provide  
some guidance, but I'm short of bandwidth at the moment.

-Peter

On Thu, Sep 13, 2012 at 10:46 AM, Maven User [maven.2.user@gmail.com](mailto:maven.2.user@gmail.com) wrote:

> Hi all -
> 
> With knife's ec2 plugin (which is awesome), I can create and instance from  
> an ami, but I cannot go the other way.
> 
> The use case is this:
> 
> virgin ubuntu 12.X ami -\> knife clone/bootstrap with our product -\> create  
> ami from this instance (for consumption elsewhere)
> 
> Before I start diving down this rabbit hole, is/was there a reason NOT to  
> do this? Time? Or was there a technical reason? Does this feature exist  
> and I don't see it?
> 
> Just curious, seems like a good feature to have but don't want to waste a  
> ton of time if there was a technical reason NOT to do this.

---

<div class="post-metadata">

**Author:** ![Maven\_User](https://avatars.discourse-cdn.com/v4/letter/m/c57346/32.png) [@Maven\_User](https://discourse.chef.io/u/Maven_User)\
**Post date:** [September 13, 2012, 5:19pm UTC](https://discourse.chef.io/t/knife-ec2-plugin/3106/5 "2012-09-13T17:19:32Z")

</div>

Cool - thanks all for the feedback!

I’m new to most of this tech, so forgive my naivete.

In the AWS Console, I can right click on a running instance and create an  
ami.

Essentially, what we’re doing with knife is perfecting a virginal Ubuntu  
instance (apps/config/users/etc) then (currently) calling the AWS tools  
(installed on build machines) to turn that instance into an AMI which in  
turn we’re using Asgard to roll out to different ASGs.

If/when we want to make changes, we’ll roll out new AMI’s (the result of  
our build process). The Instance that we create is transient (we’ll throw  
it away when the AMI has been vetted).

We for sure want EBS backed for now.

Please keep the feedback coming!

---

<div class="post-metadata">

**Author:** ![johnmartinez](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/johnmartinez/32/558_2.png) [@johnmartinez](https://discourse.chef.io/u/johnmartinez)\
**Post date:** [September 13, 2012, 5:36pm UTC](https://discourse.chef.io/t/knife-ec2-plugin/3106/6 "2012-09-13T17:36:10Z")

</div>

One more recommendation: get familiar with the ec2-api tools.

> **[Developer Tools - SDKs and Programming Toolkits for Building on AWS](https://aws.amazon.com/developer/tools/)**
>
> Find the latest SDKs, AWS CLI, and programming toolkits for use with Amazon Web Services.

On Sep 13, 2012, at 10:19 AM, Maven User [maven.2.user@gmail.com](mailto:maven.2.user@gmail.com) wrote:

> Cool - thanks all for the feedback!
> 
> I'm new to most of this tech, so forgive my naivete.
> 
> In the AWS Console, I can right click on a running instance and create an ami.
> 
> Essentially, what we're doing with knife is perfecting a virginal Ubuntu instance (apps/config/users/etc) then (currently) calling the AWS tools (installed on build machines) to turn that instance into an AMI which in turn we're using Asgard to roll out to different ASGs.
> 
> If/when we want to make changes, we'll roll out new AMI's (the result of our build process). The Instance that we create is transient (we'll throw it away when the AMI has been vetted).
> 
> We for sure want EBS backed for now.
> 
> Please keep the feedback coming!

---

<div class="post-metadata">

**Author:** ![Edward\_Sargisson](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/edward_sargisson/32/570_2.png) [@Edward\_Sargisson](https://discourse.chef.io/u/Edward_Sargisson)\
**Post date:** [September 14, 2012, 2:04pm UTC](https://discourse.chef.io/t/knife-ec2-plugin/3106/7 "2012-09-14T14:04:10Z")

</div>

Point 2 doesn't require a custom AMI if you don't want it to. You can do  
autoscaling with a virgin AMI and use user-data bootstrapping to get  
chef-client on there and running.

Details on my site here:  
[http://www.trailhunger.com/blog/technical/2011/05/28/keeping-an-amazon-elastic-compute-cloud-ec2-instance-up-with-chef-and-auto-scaling/](http://www.trailhunger.com/blog/technical/2011/05/28/keeping-an-amazon-elastic-compute-cloud-ec2-instance-up-with-chef-and-auto-scaling/)

Cheers,  
Edward

On Thu, Sep 13, 2012 at 9:15 AM, John Martinez [john@johnmartinez.com](mailto:john@johnmartinez.com)wrote:

> On Sep 13, 2012, at 9:01 AM, Daniel DeLeo [dan@kallistec.com](mailto:dan@kallistec.com) wrote:
> 
> On Thursday, September 13, 2012 at 7:46 AM, Maven User wrote:
> 
> Hi all -
> 
> With knife's ec2 plugin (which is awesome), I can create and instance from  
> an ami, but I cannot go the other way.
> 
> The use case is this:
> 
> virgin ubuntu 12.X ami -\> knife clone/bootstrap with our product -\> create  
> ami from this instance (for consumption elsewhere)
> 
> Before I start diving down this rabbit hole, is/was there a reason NOT to  
> do this? Time? Or was there a technical reason? Does this feature exist  
> and I don't see it?
> 
> Just curious, seems like a good feature to have but don't want to waste a  
> ton of time if there was a technical reason NOT to do this.
> 
> Creating an AMI is an involved process. Most people only follow the  
> pattern you describe when their initial chef-client runs take a prohibitive  
> amount of time and they need really fast instance creation for things like  
> autoscaling.
> 
> Concur. From my experience, the two main reasons I use to create custom  
> AMIs in AWS to use in conjunction with Chef:
> 
> 1. To cut-down on instance boot up time (as described by Daniel)
> 2. You are using Autoscaling or CloudFormation where the knife API  
> command can't be used
> 
> -john

---

<div class="post-metadata">

**Author:** ![Jim\_Croft](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/jim_croft/32/495_2.png) [@Jim\_Croft](https://discourse.chef.io/u/Jim_Croft)\
**Post date:** [September 14, 2012, 2:33pm UTC](https://discourse.chef.io/t/knife-ec2-plugin/3106/8 "2012-09-14T14:33:22Z")

</div>

I've posted a similar method here: [https://gist.github.com/3722252](https://gist.github.com/3722252)

This uses cloud-init rather than a custom script (pros and cons there) in  
the user-data and works on recent Ubuntu AMIs. I believe Amazon have also  
forked cloud-init to make it work on Amazon Linux, CentOS, etc. I've only  
ever used it with Ubuntu though.

Cheers

Jim

On 14 September 2012 15:04, Edward Sargisson [esarge@pobox.com](mailto:esarge@pobox.com) wrote:

> Point 2 doesn't require a custom AMI if you don't want it to. You can do  
> autoscaling with a virgin AMI and use user-data bootstrapping to get  
> chef-client on there and running.
> 
> Details on my site here:  
> [http://www.trailhunger.com/blog/technical/2011/05/28/keeping-an-amazon-elastic-compute-cloud-ec2-instance-up-with-chef-and-auto-scaling/](http://www.trailhunger.com/blog/technical/2011/05/28/keeping-an-amazon-elastic-compute-cloud-ec2-instance-up-with-chef-and-auto-scaling/)
> 
> Cheers,  
> Edward
> 
> On Thu, Sep 13, 2012 at 9:15 AM, John Martinez [john@johnmartinez.com](mailto:john@johnmartinez.com)wrote:
> 
> > On Sep 13, 2012, at 9:01 AM, Daniel DeLeo [dan@kallistec.com](mailto:dan@kallistec.com) wrote:
> > 
> > On Thursday, September 13, 2012 at 7:46 AM, Maven User wrote:
> > 
> > Hi all -
> > 
> > With knife's ec2 plugin (which is awesome), I can create and instance  
> > from an ami, but I cannot go the other way.
> > 
> > The use case is this:
> > 
> > virgin ubuntu 12.X ami -\> knife clone/bootstrap with our product -\>  
> > create ami from this instance (for consumption elsewhere)
> > 
> > Before I start diving down this rabbit hole, is/was there a reason NOT to  
> > do this? Time? Or was there a technical reason? Does this feature exist  
> > and I don't see it?
> > 
> > Just curious, seems like a good feature to have but don't want to waste a  
> > ton of time if there was a technical reason NOT to do this.
> > 
> > Creating an AMI is an involved process. Most people only follow the  
> > pattern you describe when their initial chef-client runs take a prohibitive  
> > amount of time and they need really fast instance creation for things like  
> > autoscaling.
> > 
> > Concur. From my experience, the two main reasons I use to create custom  
> > AMIs in AWS to use in conjunction with Chef:
> > 
> > 1. To cut-down on instance boot up time (as described by Daniel)
> > 2. You are using Autoscaling or CloudFormation where the knife API  
> > command can't be used
> > 
> > -john

--

_Jim Croft_  
Systems Developer, Cloudreach Limited

Support Cloudreach at ByteNite [http://goo.gl/dWWyT](http://goo.gl/dWWyT)  
Case Study: Transforming business at BRI [http://goo.gl/ng4LJ](http://goo.gl/ng4LJ)

Cloudreach Limited is a limited company registered in England with registered number 06975407

The above terms reflect a potential business arrangement, are provided solely as a basis for further discussion,  
and are not intended to be and do not constitute a legally binding obligation. No legally binding obligations  
will be created, implied, or inferred until an agreement in final form is executed in writing by all parties involved.

This email may be confidential or privileged. If you received this communication by mistake, please don't forward  
it to anyone else, please erase all copies and attachments, and please let us know that it has gone to the wrong person.