# Launching spot instances?

**URL:** <https://discourse.chef.io/t/launching-spot-instances/3228>\
**Category:** Chef Infra (archive)\
**Created:** [October 24, 2012, 8:44pm UTC](https://discourse.chef.io/t/launching-spot-instances/3228 "2012-10-24T20:44:52Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![cjbottaro](https://avatars.discourse-cdn.com/v4/letter/c/a8b319/32.png) [@cjbottaro](https://discourse.chef.io/u/cjbottaro)\
**Post date:** [October 24, 2012, 8:44pm UTC](https://discourse.chef.io/t/launching-spot-instances/3228/1 "2012-10-24T20:44:52Z")

</div>

How can I launch spot instances using “knife ec2 server create”? I don’t  
see any relevant command line switches.

Thanks for the help.

---

<div class="post-metadata">

**Author:** ![Mike](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/mike/32/18_2.png) [@Mike](https://discourse.chef.io/u/Mike)\
**Post date:** [October 25, 2012, 1:56pm UTC](https://discourse.chef.io/t/launching-spot-instances/3228/2 "2012-10-25T13:56:07Z")

</div>

While the underlying library fog may provide this functionality, this  
actually seems less useful to be part of the knife-ec2 command set.

`knife ec2 server create` attempts to create an instance, wait for it  
to come up, bootstrap it via ssh, and start up chef-client (usually).

From what I garner, placing a spot request with a bid provide is not  
ensured to be delivered at the time pf placing the request, and  
therefore the flow of what "ec2 server create" does today would not be  
the same, rather it seems like it would be: "If I can get a spot  
request now, please give it to me, otherwise...?" Fail? The bid  
request was probably placed, so this isn't a failure per se.

The other part of why I think this might not be a great idea is: Spot  
instances come up and down whenever their bid price is met or outbid  
respectively. This can cause easy buildup of cruft in your node/client  
list if you are allowing these to auto-register/validate themselves,  
and then have to clean up after them once they are gone.

All in all, I think spot instances have a use case, but I don't know  
if during ec2 server create (and subsequently bootstrap phase) makes  
sense to support this.

On Wed, Oct 24, 2012 at 1:44 PM, Christopher J. Bottaro  
[cjbottaro@academicworks.com](mailto:cjbottaro@academicworks.com) wrote:

> How can I launch spot instances using "knife ec2 server create"? I don't  
> see any relevant command line switches.
> 
> Thanks for the help.

---

<div class="post-metadata">

**Author:** ![James](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/james/32/232_2.png) [@James](https://discourse.chef.io/u/James)\
**Post date:** [October 26, 2012, 8:41am UTC](https://discourse.chef.io/t/launching-spot-instances/3228/3 "2012-10-26T08:41:20Z")

</div>

I do not think it is a bad idea to add spot instance functionality to knife  
ec2. Spot instances are great, not only because they are very inexpensive,  
but because you can set an end date for your reservation. Load testing,  
staging, queue workers, batch jobs, and map reduce are examples of  
potential use cases. In cases where it would be ideal, but is not necessary  
for a spot instance to stay up, you can bid higher than the on-demand  
price. It is pretty rare for spot prices to exceed on-demand, so you will  
most likely still save money with this tactic.

I think it is possible to add this functionality to knife ec2. You would  
monitor the status of the bid request until it is fulfilled. It would  
probably be ideal to write a basic cookbook to manage cleaning up the  
nodes/clients when the spot instances are terminated.

Here is a blog on the new Spot Instance Status API:  
[http://aws.typepad.com/aws/2012/10/amazon-ec2-spot-instance-bid-status.html](http://aws.typepad.com/aws/2012/10/amazon-ec2-spot-instance-bid-status.html)

James

On Thu, Oct 25, 2012 at 6:56 AM, Mike [miketheman@gmail.com](mailto:miketheman@gmail.com) wrote:

> While the underlying library fog may provide this functionality, this  
> actually seems less useful to be part of the knife-ec2 command set.
> 
> `knife ec2 server create` attempts to create an instance, wait for it  
> to come up, bootstrap it via ssh, and start up chef-client (usually).
> 
> From what I garner, placing a spot request with a bid provide is not  
> ensured to be delivered at the time pf placing the request, and  
> therefore the flow of what "ec2 server create" does today would not be  
> the same, rather it seems like it would be: "If I can get a spot  
> request now, please give it to me, otherwise...?" Fail? The bid  
> request was probably placed, so this isn't a failure per se.
> 
> The other part of why I think this might not be a great idea is: Spot  
> instances come up and down whenever their bid price is met or outbid  
> respectively. This can cause easy buildup of cruft in your node/client  
> list if you are allowing these to auto-register/validate themselves,  
> and then have to clean up after them once they are gone.
> 
> All in all, I think spot instances have a use case, but I don't know  
> if during ec2 server create (and subsequently bootstrap phase) makes  
> sense to support this.
> 
> On Wed, Oct 24, 2012 at 1:44 PM, Christopher J. Bottaro  
> [cjbottaro@academicworks.com](mailto:cjbottaro@academicworks.com) wrote:
> 
> > How can I launch spot instances using "knife ec2 server create"? I don't  
> > see any relevant command line switches.
> > 
> > Thanks for the help.

---

<div class="post-metadata">

**Author:** ![Jeffrey\_Hulten1](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/jeffrey_hulten1/32/585_2.png) [@Jeffrey\_Hulten1](https://discourse.chef.io/u/Jeffrey_Hulten1)\
**Post date:** [October 26, 2012, 7:09pm UTC](https://discourse.chef.io/t/launching-spot-instances/3228/4 "2012-10-26T19:09:27Z")

</div>

I could see a "knife ec2 stack create" that used a default cloud formation stack json and passed in the parameters. This could be customized then to run spot instances.

The issue with knife and spot instances is that they can take up to 15 minutes to start up. Basically (as I understand it) there is a batch job that takes the pool of bids and the available capacity and determines what to start and what to kill on a scheduled basis. That means that your request for spot instances cannot be instantaneous. This is not a model that the knife ec2 plugin can support.

--  
Jeffrey Hulten  
Principal Consultant at Automated Labs, LLC  
[jeffh@automatedlabs.com](mailto:jeffh@automatedlabs.com) 206-853-5216  
Skype: jeffhulten

On Oct 25, 2012, at 6:56 AM, Mike wrote:

> While the underlying library fog may provide this functionality, this  
> actually seems less useful to be part of the knife-ec2 command set.
> 
> `knife ec2 server create` attempts to create an instance, wait for it  
> to come up, bootstrap it via ssh, and start up chef-client (usually).
> 
> From what I garner, placing a spot request with a bid provide is not  
> ensured to be delivered at the time pf placing the request, and  
> therefore the flow of what "ec2 server create" does today would not be  
> the same, rather it seems like it would be: "If I can get a spot  
> request now, please give it to me, otherwise...?" Fail? The bid  
> request was probably placed, so this isn't a failure per se.
> 
> The other part of why I think this might not be a great idea is: Spot  
> instances come up and down whenever their bid price is met or outbid  
> respectively. This can cause easy buildup of cruft in your node/client  
> list if you are allowing these to auto-register/validate themselves,  
> and then have to clean up after them once they are gone.
> 
> All in all, I think spot instances have a use case, but I don't know  
> if during ec2 server create (and subsequently bootstrap phase) makes  
> sense to support this.
> 
> On Wed, Oct 24, 2012 at 1:44 PM, Christopher J. Bottaro  
> [cjbottaro@academicworks.com](mailto:cjbottaro@academicworks.com) wrote:
> 
> > How can I launch spot instances using "knife ec2 server create"? I don't  
> > see any relevant command line switches.
> > 
> > Thanks for the help.

---

<div class="post-metadata">

**Author:** ![Mike](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/mike/32/18_2.png) [@Mike](https://discourse.chef.io/u/Mike)\
**Post date:** [October 26, 2012, 9:07pm UTC](https://discourse.chef.io/t/launching-spot-instances/3228/5 "2012-10-26T21:07:16Z")

</div>

James,

All the points about spot instances you reference are correct.

What I think I was trying to convey is that the behavior currently of  
knife-ec2 is to perform the requested steps (provision, ssh,  
bootstrap) in sequence, and exit upon completion.

How would you propose to do that with spot instances that are  
\_not\_granted immediately? The request at the price is still made, and  
will open up when available, but may not launch right now. There also  
may be more than 1, and knife

Would the operator expect the shell session to remain open until _one_  
spot instance is launched, so it can then perform bootstrapping, role  
assignment etc? That is not a very tempting scenario to me.

I think a better use case for the _utterly specific_ use case of spot  
instances would be to leverage AWS's api and model, potentially  
pre-bake your own AMI with a default chef client.rb and validation.pem  
for your organization (and roles?) , so that when a spot request  
launches, it can boot, register with chef, get the latest cookbook  
code and run until it's terminated.

That's why I don't think this is a good fit for `knife ec2 server create`.

On Fri, Oct 26, 2012 at 3:09 PM, Jeffrey Hulten [jeffh@automatedlabs.com](mailto:jeffh@automatedlabs.com) wrote:

> I could see a "knife ec2 stack create" that used a default cloud formation stack json and passed in the parameters. This could be customized then to run spot instances.
> 
> The issue with knife and spot instances is that they can take up to 15 minutes to start up. Basically (as I understand it) there is a batch job that takes the pool of bids and the available capacity and determines what to start and what to kill on a scheduled basis. That means that your request for spot instances cannot be instantaneous. This is not a model that the knife ec2 plugin can support.
> 
> --  
> Jeffrey Hulten  
> Principal Consultant at Automated Labs, LLC  
> [jeffh@automatedlabs.com](mailto:jeffh@automatedlabs.com) 206-853-5216  
> Skype: jeffhulten
> 
> On Oct 25, 2012, at 6:56 AM, Mike wrote:
> 
> > While the underlying library fog may provide this functionality, this  
> > actually seems less useful to be part of the knife-ec2 command set.
> > 
> > `knife ec2 server create` attempts to create an instance, wait for it  
> > to come up, bootstrap it via ssh, and start up chef-client (usually).
> > 
> > From what I garner, placing a spot request with a bid provide is not  
> > ensured to be delivered at the time pf placing the request, and  
> > therefore the flow of what "ec2 server create" does today would not be  
> > the same, rather it seems like it would be: "If I can get a spot  
> > request now, please give it to me, otherwise...?" Fail? The bid  
> > request was probably placed, so this isn't a failure per se.
> > 
> > The other part of why I think this might not be a great idea is: Spot  
> > instances come up and down whenever their bid price is met or outbid  
> > respectively. This can cause easy buildup of cruft in your node/client  
> > list if you are allowing these to auto-register/validate themselves,  
> > and then have to clean up after them once they are gone.
> > 
> > All in all, I think spot instances have a use case, but I don't know  
> > if during ec2 server create (and subsequently bootstrap phase) makes  
> > sense to support this.
> > 
> > On Wed, Oct 24, 2012 at 1:44 PM, Christopher J. Bottaro  
> > [cjbottaro@academicworks.com](mailto:cjbottaro@academicworks.com) wrote:
> > 
> > > How can I launch spot instances using "knife ec2 server create"? I don't  
> > > see any relevant command line switches.
> > > 
> > > Thanks for the help.

---

<div class="post-metadata">

**Author:** ![James](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/james/32/232_2.png) [@James](https://discourse.chef.io/u/James)\
**Post date:** [October 27, 2012, 7:47pm UTC](https://discourse.chef.io/t/launching-spot-instances/3228/6 "2012-10-27T19:47:51Z")

</div>

Mike,

There are a number of ways to implement this, but it is not currently on my  
or our roadmap, so I haven't put much thought into how I would design it. I  
can say that this would benefit us, but not enough to warrant the effort  
currently. We don't generally use knife ec2 to provision on EC2, we're  
almost eliminating usage of EC2 org-wide, and in the use-case that is most  
important for us to use spot instances (map reduce), we do so using boto +  
custom stuff.

Most single instance reservations I make get filled quickly, but you could  
definitely add some logic around this process in a number of different  
ways. You could allow a timeout to define when to stop waiting and cancel  
the bid. You could allow a price range as a parameter. I don't think any of  
the other knife compute gems support creating multiple instances  
simultaneously, so multiple instance reservations are probably not a  
necessary feature.

Your implementation could work for some use cases.

Spot instances are a really great way to drop the cost of EC2, just have to  
be aware of, and work around, limitations.

James

On Fri, Oct 26, 2012 at 2:07 PM, Mike [miketheman@gmail.com](mailto:miketheman@gmail.com) wrote:

> James,
> 
> All the points about spot instances you reference are correct.
> 
> What I think I was trying to convey is that the behavior currently of  
> knife-ec2 is to perform the requested steps (provision, ssh,  
> bootstrap) in sequence, and exit upon completion.
> 
> How would you propose to do that with spot instances that are  
> \_not\_granted immediately? The request at the price is still made, and  
> will open up when available, but may not launch right now. There also  
> may be more than 1, and knife
> 
> Would the operator expect the shell session to remain open until _one_  
> spot instance is launched, so it can then perform bootstrapping, role  
> assignment etc? That is not a very tempting scenario to me.
> 
> I think a better use case for the _utterly specific_ use case of spot  
> instances would be to leverage AWS's api and model, potentially  
> pre-bake your own AMI with a default chef client.rb and validation.pem  
> for your organization (and roles?) , so that when a spot request  
> launches, it can boot, register with chef, get the latest cookbook  
> code and run until it's terminated.
> 
> That's why I don't think this is a good fit for `knife ec2 server create`.
> 
> On Fri, Oct 26, 2012 at 3:09 PM, Jeffrey Hulten [jeffh@automatedlabs.com](mailto:jeffh@automatedlabs.com)  
> wrote:
> 
> > I could see a "knife ec2 stack create" that used a default cloud  
> > formation stack json and passed in the parameters. This could be customized  
> > then to run spot instances.
> > 
> > The issue with knife and spot instances is that they can take up to 15  
> > minutes to start up. Basically (as I understand it) there is a batch job  
> > that takes the pool of bids and the available capacity and determines what  
> > to start and what to kill on a scheduled basis. That means that your  
> > request for spot instances cannot be instantaneous. This is not a model  
> > that the knife ec2 plugin can support.
> > 
> > --  
> > Jeffrey Hulten  
> > Principal Consultant at Automated Labs, LLC  
> > [jeffh@automatedlabs.com](mailto:jeffh@automatedlabs.com) 206-853-5216  
> > Skype: jeffhulten
> > 
> > On Oct 25, 2012, at 6:56 AM, Mike wrote:
> > 
> > > While the underlying library fog may provide this functionality, this  
> > > actually seems less useful to be part of the knife-ec2 command set.
> > > 
> > > `knife ec2 server create` attempts to create an instance, wait for it  
> > > to come up, bootstrap it via ssh, and start up chef-client (usually).
> > > 
> > > From what I garner, placing a spot request with a bid provide is not  
> > > ensured to be delivered at the time pf placing the request, and  
> > > therefore the flow of what "ec2 server create" does today would not be  
> > > the same, rather it seems like it would be: "If I can get a spot  
> > > request now, please give it to me, otherwise...?" Fail? The bid  
> > > request was probably placed, so this isn't a failure per se.
> > > 
> > > The other part of why I think this might not be a great idea is: Spot  
> > > instances come up and down whenever their bid price is met or outbid  
> > > respectively. This can cause easy buildup of cruft in your node/client  
> > > list if you are allowing these to auto-register/validate themselves,  
> > > and then have to clean up after them once they are gone.
> > > 
> > > All in all, I think spot instances have a use case, but I don't know  
> > > if during ec2 server create (and subsequently bootstrap phase) makes  
> > > sense to support this.
> > > 
> > > On Wed, Oct 24, 2012 at 1:44 PM, Christopher J. Bottaro  
> > > [cjbottaro@academicworks.com](mailto:cjbottaro@academicworks.com) wrote:
> > > 
> > > > How can I launch spot instances using "knife ec2 server create"? I  
> > > > don't  
> > > > see any relevant command line switches.
> > > > 
> > > > Thanks for the help.

---

<div class="post-metadata">

**Author:** ![Mike](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/mike/32/18_2.png) [@Mike](https://discourse.chef.io/u/Mike)\
**Post date:** [October 27, 2012, 8:02pm UTC](https://discourse.chef.io/t/launching-spot-instances/3228/7 "2012-10-27T20:02:57Z")

</div>

James,

As I mentioned, it seems like using the spot instance market via  
direct api tools such as boto makes more sense than an interactive  
knife session.  
We also use spot instances for MapReduce, and use API tools for that -  
not knife-ec2.

If you'd like to design and implement it, I'm sure it would make  
Christopher (original thread starter) happy. 🙂

Again, I'll reiterate - knife-ec2 plugin seems to provide the "I need  
an instance now, please assign resources and bootstrap it" resource.  
For multiple ec2 instances on the regular market, I've used  
spiceweasel by Matt Ray ([GitHub - mattray/spiceweasel: Generates Chef knife commands from a simple JSON or YAML file.](https://github.com/mattray/spiceweasel)) and  
it's been a charm.

-Mike

On Sat, Oct 27, 2012 at 7:47 PM, James [js@aegisco.com](mailto:js@aegisco.com) wrote:

> Mike,
> 
> There are a number of ways to implement this, but it is not currently on my  
> or our roadmap, so I haven't put much thought into how I would design it. I  
> can say that this would benefit us, but not enough to warrant the effort  
> currently. We don't generally use knife ec2 to provision on EC2, we're  
> almost eliminating usage of EC2 org-wide, and in the use-case that is most  
> important for us to use spot instances (map reduce), we do so using boto +  
> custom stuff.
> 
> Most single instance reservations I make get filled quickly, but you could  
> definitely add some logic around this process in a number of different ways.  
> You could allow a timeout to define when to stop waiting and cancel the bid.  
> You could allow a price range as a parameter. I don't think any of the other  
> knife compute gems support creating multiple instances simultaneously, so  
> multiple instance reservations are probably not a necessary feature.
> 
> Your implementation could work for some use cases.
> 
> Spot instances are a really great way to drop the cost of EC2, just have to  
> be aware of, and work around, limitations.
> 
> James
> 
> On Fri, Oct 26, 2012 at 2:07 PM, Mike [miketheman@gmail.com](mailto:miketheman@gmail.com) wrote:
> 
> > James,
> > 
> > All the points about spot instances you reference are correct.
> > 
> > What I think I was trying to convey is that the behavior currently of  
> > knife-ec2 is to perform the requested steps (provision, ssh,  
> > bootstrap) in sequence, and exit upon completion.
> > 
> > How would you propose to do that with spot instances that are  
> > \_not\_granted immediately? The request at the price is still made, and  
> > will open up when available, but may not launch right now. There also  
> > may be more than 1, and knife
> > 
> > Would the operator expect the shell session to remain open until _one_  
> > spot instance is launched, so it can then perform bootstrapping, role  
> > assignment etc? That is not a very tempting scenario to me.
> > 
> > I think a better use case for the _utterly specific_ use case of spot  
> > instances would be to leverage AWS's api and model, potentially  
> > pre-bake your own AMI with a default chef client.rb and validation.pem  
> > for your organization (and roles?) , so that when a spot request  
> > launches, it can boot, register with chef, get the latest cookbook  
> > code and run until it's terminated.
> > 
> > That's why I don't think this is a good fit for `knife ec2 server create`.
> > 
> > On Fri, Oct 26, 2012 at 3:09 PM, Jeffrey Hulten [jeffh@automatedlabs.com](mailto:jeffh@automatedlabs.com)  
> > wrote:
> > 
> > > I could see a "knife ec2 stack create" that used a default cloud  
> > > formation stack json and passed in the parameters. This could be customized  
> > > then to run spot instances.
> > > 
> > > The issue with knife and spot instances is that they can take up to 15  
> > > minutes to start up. Basically (as I understand it) there is a batch job  
> > > that takes the pool of bids and the available capacity and determines what  
> > > to start and what to kill on a scheduled basis. That means that your request  
> > > for spot instances cannot be instantaneous. This is not a model that the  
> > > knife ec2 plugin can support.
> > > 
> > > --  
> > > Jeffrey Hulten  
> > > Principal Consultant at Automated Labs, LLC  
> > > [jeffh@automatedlabs.com](mailto:jeffh@automatedlabs.com) 206-853-5216  
> > > Skype: jeffhulten
> > > 
> > > On Oct 25, 2012, at 6:56 AM, Mike wrote:
> > > 
> > > > While the underlying library fog may provide this functionality, this  
> > > > actually seems less useful to be part of the knife-ec2 command set.
> > > > 
> > > > `knife ec2 server create` attempts to create an instance, wait for it  
> > > > to come up, bootstrap it via ssh, and start up chef-client (usually).
> > > > 
> > > > From what I garner, placing a spot request with a bid provide is not  
> > > > ensured to be delivered at the time pf placing the request, and  
> > > > therefore the flow of what "ec2 server create" does today would not be  
> > > > the same, rather it seems like it would be: "If I can get a spot  
> > > > request now, please give it to me, otherwise...?" Fail? The bid  
> > > > request was probably placed, so this isn't a failure per se.
> > > > 
> > > > The other part of why I think this might not be a great idea is: Spot  
> > > > instances come up and down whenever their bid price is met or outbid  
> > > > respectively. This can cause easy buildup of cruft in your node/client  
> > > > list if you are allowing these to auto-register/validate themselves,  
> > > > and then have to clean up after them once they are gone.
> > > > 
> > > > All in all, I think spot instances have a use case, but I don't know  
> > > > if during ec2 server create (and subsequently bootstrap phase) makes  
> > > > sense to support this.
> > > > 
> > > > On Wed, Oct 24, 2012 at 1:44 PM, Christopher J. Bottaro  
> > > > [cjbottaro@academicworks.com](mailto:cjbottaro@academicworks.com) wrote:
> > > > 
> > > > > How can I launch spot instances using "knife ec2 server create"? I  
> > > > > don't  
> > > > > see any relevant command line switches.
> > > > > 
> > > > > Thanks for the help.