AWS Spot instances for Integration-Testing Cookbooks on Github

Chef Infra (archive)
1

Hi everybody,

having some cookbooks hosted on Github and using kitchen-ci for
integration-testing them, I would like to have the kitchen-ci tests
run on every commit / push.

With travis-ci (which I’m using for foodcritic / chefspec) this does
not work unfortunately, since it prohibits creation of nested
containers.

So I’m now considering to spin up an AWS spot instance during the
travis-ci build to run the integration tests on.

Anyone doing this already?

Any pointers or suggestions to get me kick-started are heartily welcome :slight_smile:

Cheers,
Torben

2

Weeeee, just found there is a kitchen-ec2 driver, and it has support for
spot instances... neat! :slight_smile:
Am 28.10.2014 08:24 schrieb "Torben Knerr" mail@tknerr.de:

Hi everybody,

having some cookbooks hosted on Github and using kitchen-ci for
integration-testing them, I would like to have the kitchen-ci tests
run on every commit / push.

With travis-ci (which I'm using for foodcritic / chefspec) this does
not work unfortunately, since it prohibits creation of nested
containers.

So I'm now considering to spin up an AWS spot instance during the
travis-ci build to run the integration tests on.

Anyone doing this already?

Any pointers or suggestions to get me kick-started are heartily welcome :slight_smile:

Cheers,
Torben

3

Torben,

We've been doing this with some of our cookbooks for the last few months, and in fact, have a talk on it at the Chef Boston meetup group in Sept.

Just getting going with my day right now; I'll follow-up this with more details & examples.

-- Jeff Byrnes

Lead DevOps Engineer

@thejeffbyrnes

On Tue, Oct 28, 2014 at 3:25 AM, Torben Knerr mail@tknerr.de wrote:

Hi everybody,
having some cookbooks hosted on Github and using kitchen-ci for
integration-testing them, I would like to have the kitchen-ci tests
run on every commit / push.
With travis-ci (which I'm using for foodcritic / chefspec) this does
not work unfortunately, since it prohibits creation of nested
containers.
So I'm now considering to spin up an AWS spot instance during the
travis-ci build to run the integration tests on.
Anyone doing this already?
Any pointers or suggestions to get me kick-started are heartily welcome :slight_smile:
Cheers,
Torben

4

Hi Jeff,

cool stuff, looking forward!

Cheers, Torben

On Tue, Oct 28, 2014 at 11:35 AM, Jeff Byrnes jeff@evertrue.com wrote:

Torben,

We've been doing this with some of our cookbooks for the last few months,
and in fact, have a talk on it at the Chef Boston meetup group in Sept.

Just getting going with my day right now; I'll follow-up this with more
details & examples.

--
Jeff Byrnes
Lead DevOps Engineer
@thejeffbyrnes
http://www.evertrue.com

On Tue, Oct 28, 2014 at 3:25 AM, Torben Knerr mail@tknerr.de wrote:

Hi everybody,

having some cookbooks hosted on Github and using kitchen-ci for
integration-testing them, I would like to have the kitchen-ci tests
run on every commit / push.

With travis-ci (which I'm using for foodcritic / chefspec) this does
not work unfortunately, since it prohibits creation of nested
containers.

So I'm now considering to spin up an AWS spot instance during the
travis-ci build to run the integration tests on.

Anyone doing this already?

Any pointers or suggestions to get me kick-started are heartily welcome
:slight_smile:

Cheers,
Torben

5

Here’s a sample of kitchen files we use for ec2 provisioning, now that you’ve found that driver (we have wrapper automation that fills in those env vars, obviously)

driver:

name: ec2

ssh_key: <%= ENV[‘SSH_KEY_FILE’] %>

aws_ssh_key_id: <%= ENV[‘AWS_SSH_KEY_ID’] %>

region: us-west-1

availability_zone: <%= ENV[‘AWS_ZONE’] %>

require_chef_omnibus: true

subnet_id: <%= ENV[‘AWS_SUBNET’] %>

security_group_ids: ‘<%= ENV[‘AWS_SEC_GROUP’] %>’

flavor_id: ‘t1.micro’

provisioner:

name: chef_solo

platforms:

  • name: centos-6.4

suites:

  • name: default

    run_list:

    • recipe[ci-jenkins::default]

    attributes: {

    “ci-jenkins”: {

    "enable_cookbook_testing": true

    }

    }

From: Torben Knerr <mail@tknerr.demailto:mail@tknerr.de>
Reply-To: "chef@lists.opscode.commailto:chef@lists.opscode.com" <chef@lists.opscode.commailto:chef@lists.opscode.com>
Date: Tuesday, October 28, 2014 at 12:32 AM
To: "chef@lists.opscode.commailto:chef@lists.opscode.com" <chef@lists.opscode.commailto:chef@lists.opscode.com>
Subject: [chef] Re: AWS Spot instances for Integration-Testing Cookbooks on Github

Weeeee, just found there is a kitchen-ec2 driver, and it has support for spot instances… neat! :slight_smile:

Am 28.10.2014 08:24 schrieb “Torben Knerr” <mail@tknerr.demailto:mail@tknerr.de>:
Hi everybody,

having some cookbooks hosted on Github and using kitchen-ci for
integration-testing them, I would like to have the kitchen-ci tests
run on every commit / push.

With travis-ci (which I’m using for foodcritic / chefspec) this does
not work unfortunately, since it prohibits creation of nested
containers.

So I’m now considering to spin up an AWS spot instance during the
travis-ci build to run the integration tests on.

Anyone doing this already?

Any pointers or suggestions to get me kick-started are heartily welcome :slight_smile:

Cheers,
Torben

6

Ok! Sorry this took so long to get back to. Here’s the slides from our presentation back in Sept: http://evertrue.github.io/test-kitchen-travis-slides/

You can refer to these two cookbooks (one was done live during the preso) to guide you on how we set this up using EC2 & Travis:



Most pertinent is the .travis.yml & .kitchen.cloud.yml (which we force Test Kitchen to use by way of a Rake task).

Tara’s config, below, is very similar to ours, though clearly Jenkins-centric.

Bear in mind that kitchen-ec2, you’ll need to use the current HEAD of the Git repo (see this changeset for details on what’s missing from the current published release, v0.8.0). Hopefully they’ll cut a new version soon, but we use the bleeding edge version right now & it’s solid.

Tara, I’m curious, how did you get around the need for the AWS API keypair in your Test Kitchen config?

–
Jeff Byrnes
@berkleebassist
Lead DevOps Engineer
EverTrue
704.516.4628

On October 28, 2014 at 12:28:13 PM, Tara Hernandez (tara.hernandez@lithium.com) wrote:

Here’s a sample of kitchen files we use for ec2 provisioning, now that you’ve found that driver (we have wrapper automation that fills in those env vars, obviously)

driver:
name: ec2
ssh_key: <%= ENV[‘SSH_KEY_FILE’] %>
aws_ssh_key_id: <%= ENV[‘AWS_SSH_KEY_ID’] %>
region: us-west-1
availability_zone: <%= ENV[‘AWS_ZONE’] %>
require_chef_omnibus: true
subnet_id: <%= ENV[‘AWS_SUBNET’] %>
security_group_ids: ‘<%= ENV[‘AWS_SEC_GROUP’] %>'
flavor_id: ‘t1.micro’

provisioner:
name: chef_solo

platforms:

  • name: centos-6.4

suites:

  • name: default
    run_list:
    • recipe[ci-jenkins::default]
      attributes: {
      “ci-jenkins”: {
      “enable_cookbook_testing”: true
      }
      }

From: Torben Knerr mail@tknerr.de
Reply-To: "chef@lists.opscode.com" chef@lists.opscode.com
Date: Tuesday, October 28, 2014 at 12:32 AM
To: "chef@lists.opscode.com" chef@lists.opscode.com
Subject: [chef] Re: AWS Spot instances for Integration-Testing Cookbooks on Github

Weeeee, just found there is a kitchen-ec2 driver, and it has support for spot instances… neat! :slight_smile:

Am 28.10.2014 08:24 schrieb “Torben Knerr” mail@tknerr.de:
Hi everybody,

having some cookbooks hosted on Github and using kitchen-ci for
integration-testing them, I would like to have the kitchen-ci tests
run on every commit / push.

With travis-ci (which I’m using for foodcritic / chefspec) this does
not work unfortunately, since it prohibits creation of nested
containers.

So I’m now considering to spin up an AWS spot instance during the
travis-ci build to run the integration tests on.

Anyone doing this already?

Any pointers or suggestions to get me kick-started are heartily welcome :slight_smile:

Cheers,
Torben

7

Oh, sorry — the wrapper automation makes use of environment variables for that stuff (e.g. AWS_SECRET_KEY etc.) — it’s not needed in the driver definition and we try and keep that stuff at least slightly secure…

From: Jeff Byrnes <jeff@evertrue.commailto:jeff@evertrue.com>
Reply-To: "chef@lists.opscode.commailto:chef@lists.opscode.com" <chef@lists.opscode.commailto:chef@lists.opscode.com>
Date: Tuesday, October 28, 2014 at 10:12 AM
To: "chef@lists.opscode.commailto:chef@lists.opscode.com" <chef@lists.opscode.commailto:chef@lists.opscode.com>
Subject: [chef] Re: Re: Re: AWS Spot instances for Integration-Testing Cookbooks on Github

Ok! Sorry this took so long to get back to. Here’s the slides from our presentation back in Sept: http://evertrue.github.io/test-kitchen-travis-slides/

You can refer to these two cookbooks (one was done live during the preso) to guide you on how we set this up using EC2 & Travis:

Most pertinent is the .travis.yml & .kitchen.cloud.yml (which we force Test Kitchen to use by way of a Rake task).

Tara’s config, below, is very similar to ours, though clearly Jenkins-centric.

Bear in mind that kitchen-ec2, you’ll need to use the current HEAD of the Git repo (see this changesethttps://github.com/test-kitchen/kitchen-ec2/compare/v0.8.0...master for details on what’s missing from the current published release, v0.8.0). Hopefully they’ll cut a new version soon, but we use the bleeding edge version right now & it’s solid.

Tara, I’m curious, how did you get around the need for the AWS API keypair in your Test Kitchen config?

–
Jeff Byrnes
@berkleebassisthttp://twitter.com/berkleebassist
Lead DevOps Engineer
EverTruehttp://www.evertrue.com/
704.516.4628

On October 28, 2014 at 12:28:13 PM, Tara Hernandez (tara.hernandez@lithium.commailto:tara.hernandez@lithium.com) wrote:

Here’s a sample of kitchen files we use for ec2 provisioning, now that you’ve found that driver (we have wrapper automation that fills in those env vars, obviously)

driver:

name: ec2

ssh_key: <%= ENV[‘SSH_KEY_FILE’] %>

aws_ssh_key_id: <%= ENV[‘AWS_SSH_KEY_ID’] %>

region: us-west-1

availability_zone: <%= ENV[‘AWS_ZONE’] %>

require_chef_omnibus: true

subnet_id: <%= ENV[‘AWS_SUBNET’] %>

security_group_ids: ‘<%= ENV[‘AWS_SEC_GROUP’] %>’

flavor_id: ‘t1.micro’

provisioner:

name: chef_solo

platforms:

  • name: centos-6.4

suites:

  • name: default

    run_list:

    • recipe[ci-jenkins::default]

    attributes: {

    “ci-jenkins”: {

    "enable_cookbook_testing": true

    }

    }

From: Torben Knerr <mail@tknerr.demailto:mail@tknerr.de>
Reply-To: "chef@lists.opscode.commailto:chef@lists.opscode.com" <chef@lists.opscode.commailto:chef@lists.opscode.com>
Date: Tuesday, October 28, 2014 at 12:32 AM
To: "chef@lists.opscode.commailto:chef@lists.opscode.com" <chef@lists.opscode.commailto:chef@lists.opscode.com>
Subject: [chef] Re: AWS Spot instances for Integration-Testing Cookbooks on Github

Weeeee, just found there is a kitchen-ec2 driver, and it has support for spot instances… neat! :slight_smile:

Am 28.10.2014 08:24 schrieb “Torben Knerr” <mail@tknerr.demailto:mail@tknerr.de>:
Hi everybody,

having some cookbooks hosted on Github and using kitchen-ci for
integration-testing them, I would like to have the kitchen-ci tests
run on every commit / push.

With travis-ci (which I’m using for foodcritic / chefspec) this does
not work unfortunately, since it prohibits creation of nested
containers.

So I’m now considering to spin up an AWS spot instance during the
travis-ci build to run the integration tests on.

Anyone doing this already?

Any pointers or suggestions to get me kick-started are heartily welcome :slight_smile:

Cheers,
Torben

8

Thanks Jeff and Tara, exactly what I was looking for!

Cheers, Torben

On Tue, Oct 28, 2014 at 6:38 PM, Tara Hernandez
tara.hernandez@lithium.com wrote:

Oh, sorry — the wrapper automation makes use of environment variables for
that stuff (e.g. AWS_SECRET_KEY etc.) — it’s not needed in the driver
definition and we try and keep that stuff at least slightly secure…

From: Jeff Byrnes jeff@evertrue.com
Reply-To: "chef@lists.opscode.com" chef@lists.opscode.com
Date: Tuesday, October 28, 2014 at 10:12 AM
To: "chef@lists.opscode.com" chef@lists.opscode.com
Subject: [chef] Re: Re: Re: AWS Spot instances for Integration-Testing
Cookbooks on Github

Ok! Sorry this took so long to get back to. Here’s the slides from our
presentation back in Sept:
Test Kitchen & Travis CI

You can refer to these two cookbooks (one was done live during the preso) to
guide you on how we set this up using EC2 & Travis:

GitHub - evertrue/et_travis_demo-cookbook: A cookbook to demonstrade how easy it is to set up Travis!
GitHub - evertrue/et_travis_live_demo-cookbook: Don't Break

Most pertinent is the .travis.yml & .kitchen.cloud.yml (which we force Test
Kitchen to use by way of a Rake task).

Tara’s config, below, is very similar to ours, though clearly
Jenkins-centric.

Bear in mind that kitchen-ec2, you’ll need to use the current HEAD of the
Git repo (see this changeset for details on what’s missing from the current
published release, v0.8.0). Hopefully they’ll cut a new version soon, but we
use the bleeding edge version right now & it’s solid.

Tara, I’m curious, how did you get around the need for the AWS API keypair
in your Test Kitchen config?

--
Jeff Byrnes
@berkleebassist
Lead DevOps Engineer
EverTrue
704.516.4628

On October 28, 2014 at 12:28:13 PM, Tara Hernandez
(tara.hernandez@lithium.com) wrote:

Here’s a sample of kitchen files we use for ec2 provisioning, now that
you’ve found that driver (we have wrapper automation that fills in those env
vars, obviously)

driver:

name: ec2

ssh_key: <%= ENV['SSH_KEY_FILE'] %>

aws_ssh_key_id: <%= ENV['AWS_SSH_KEY_ID'] %>

region: us-west-1

availability_zone: <%= ENV['AWS_ZONE'] %>

require_chef_omnibus: true

subnet_id: <%= ENV['AWS_SUBNET'] %>

security_group_ids: ‘<%= ENV[‘AWS_SEC_GROUP'] %>'

flavor_id: 't1.micro'

provisioner:

name: chef_solo

platforms:

  • name: centos-6.4

suites:

  • name: default

    run_list:

    • recipe[ci-jenkins::default]

    attributes: {

    "ci-jenkins": {

    "enable_cookbook_testing": true

    }

    }

From: Torben Knerr mail@tknerr.de
Reply-To: "chef@lists.opscode.com" chef@lists.opscode.com
Date: Tuesday, October 28, 2014 at 12:32 AM
To: "chef@lists.opscode.com" chef@lists.opscode.com
Subject: [chef] Re: AWS Spot instances for Integration-Testing Cookbooks on
Github

Weeeee, just found there is a kitchen-ec2 driver, and it has support for
spot instances... neat! :slight_smile:

Am 28.10.2014 08:24 schrieb "Torben Knerr" mail@tknerr.de:

Hi everybody,

having some cookbooks hosted on Github and using kitchen-ci for
integration-testing them, I would like to have the kitchen-ci tests
run on every commit / push.

With travis-ci (which I'm using for foodcritic / chefspec) this does
not work unfortunately, since it prohibits creation of nested
containers.

So I'm now considering to spin up an AWS spot instance during the
travis-ci build to run the integration tests on.

Anyone doing this already?

Any pointers or suggestions to get me kick-started are heartily welcome
:slight_smile:

Cheers,
Torben