Using kitchen override/stub a cookbook

Chef Infra (archive)
1

I am trying to setup testing with kitchen and docker, some cookbooks like
the supermarket “hostname” cookbook are not compatible with docker but they
are used in my final deployments. I’d like to stub these out just for when
kitchen docker is used to speed up early testing but not when vagrant or
doing a berks install.

I thought I’d use groups in my Berksfile, but that would still try to
upload/install the stub cookbook to my chef server while doing a berks
install unless i specify a -e docker_integration, and Berksfile doesn’t
seem to be processed by erb.

I was unable to find anywhere in the .kitchen.yml file where I could
override a single cookbook, or a way to have it not process a cookbook.

–

Bill Warner

2

On Aug 6, 2015, at 1:50 PM, Bill Warner bill.warner@gmail.com wrote:

I am trying to setup testing with kitchen and docker, some cookbooks like the supermarket "hostname" cookbook are not compatible with docker but they are used in my final deployments. I'd like to stub these out just for when kitchen docker is used to speed up early testing but not when vagrant or doing a berks install.

I thought I'd use groups in my Berksfile, but that would still try to upload/install the stub cookbook to my chef server while doing a berks install unless i specify a -e docker_integration, and Berksfile doesn't seem to be processed by erb.

I was unable to find anywhere in the .kitchen.yml file where I could override a single cookbook, or a way to have it not process a cookbook.

Use a wrapper cookbook that only include_recipe-s the real one if you aren't on Docker.

--Noah

3

+1 Noah
Using wrapper for test always is really good idea. Here is example

I'm using noe template tool for creating cookbook skeletons, which are by
default use wrapper cookbook: GitHub - jsirex/chef-cookbook.noe: Noe skeleton for Chef Cookbook

2015-08-06 23:54 GMT+03:00 Noah Kantrowitz noah@coderanger.net:

On Aug 6, 2015, at 1:50 PM, Bill Warner bill.warner@gmail.com wrote:

I am trying to setup testing with kitchen and docker, some cookbooks
like the supermarket "hostname" cookbook are not compatible with docker but
they are used in my final deployments. I'd like to stub these out just for
when kitchen docker is used to speed up early testing but not when vagrant
or doing a berks install.

I thought I'd use groups in my Berksfile, but that would still try to
upload/install the stub cookbook to my chef server while doing a berks
install unless i specify a -e docker_integration, and Berksfile doesn't
seem to be processed by erb.

I was unable to find anywhere in the .kitchen.yml file where I could
override a single cookbook, or a way to have it not process a cookbook.

Use a wrapper cookbook that only include_recipe-s the real one if you
aren't on Docker.

--Noah

4

It's actually a wrapper cookbook that I'm testing. the cookbook includes
my base cookbook for internal standards configuration which has a few
community cookbooks as well as a few in house ones to get the base configs,
users, monitoring, dns, host etc all set up. then it includes some
community and in house cookbooks to personalize the target application such
as kafka, cassandra or our in house apps.

I have all the tests running with Vagrant now but due to some minor
incompatibilities with docker (mainly not being able to write to /etc/hosts
or /etc/resolv.conf, some of the cookbooks in the runlist always fail. if
there was a way I could blacklist or stub those cookbooks I'd still be able
to get a nearly full test much faster than I can currently with Vagrant.

I may be able to do a custom rake task that would rewrite the Berksfile and
point to empty cookbooks for those...not sure how else to test it.

On Fri, Aug 7, 2015 at 5:44 AM, Yauhen Artsiukhou jsirex@gmail.com wrote:

+1 Noah
Using wrapper for test always is really good idea. Here is example
GitHub - jsirex/collectd-lib-cookbook: Chef Cookbook for Collectd

I'm using noe template tool for creating cookbook skeletons, which are by
default use wrapper cookbook: GitHub - jsirex/chef-cookbook.noe: Noe skeleton for Chef Cookbook

2015-08-06 23:54 GMT+03:00 Noah Kantrowitz noah@coderanger.net:

On Aug 6, 2015, at 1:50 PM, Bill Warner bill.warner@gmail.com wrote:

I am trying to setup testing with kitchen and docker, some cookbooks
like the supermarket "hostname" cookbook are not compatible with docker but
they are used in my final deployments. I'd like to stub these out just for
when kitchen docker is used to speed up early testing but not when vagrant
or doing a berks install.

I thought I'd use groups in my Berksfile, but that would still try to
upload/install the stub cookbook to my chef server while doing a berks
install unless i specify a -e docker_integration, and Berksfile doesn't
seem to be processed by erb.

I was unable to find anywhere in the .kitchen.yml file where I could
override a single cookbook, or a way to have it not process a cookbook.

Use a wrapper cookbook that only include_recipe-s the real one if you
aren't on Docker.

--Noah

--

Bill Warner

5

On Friday, August 7, 2015 at 8:55 AM, Bill Warner wrote:

It's actually a wrapper cookbook that I'm testing. the cookbook includes my base cookbook for internal standards configuration which has a few community cookbooks as well as a few in house ones to get the base configs, users, monitoring, dns, host etc all set up. then it includes some community and in house cookbooks to personalize the target application such as kafka, cassandra or our in house apps.

I have all the tests running with Vagrant now but due to some minor incompatibilities with docker (mainly not being able to write to /etc/hosts or /etc/resolv.conf, some of the cookbooks in the runlist always fail. if there was a way I could blacklist or stub those cookbooks I'd still be able to get a nearly full test much faster than I can currently with Vagrant.

I may be able to do a custom rake task that would rewrite the Berksfile and point to empty cookbooks for those...not sure how else to test it.
You can detect that you’re on Docker with ohai (you’ll have to poke around for these attributes, I don’t recall what they are off-hand) and then put the logic into your wrapper cookbook. You could also do this via attributes. It’s maybe a tiny bit ugly to have that logic in non-test code, but you lose the ability to manage the run_list when you go to role cookbooks, so it’s your best option.

--
Daniel DeLeo

6

that's not a bad idea at all, and if we ever move to more docker related
stuff this would likely be necessary anyway so not just test code.

Looks like the ohai related node items:
node['virtualization']['system'] = 'docker'
node[virtualization']['systems']['docker'] = 'guest'

I'll give this a try.

-Bill

On Fri, Aug 7, 2015 at 9:11 AM, Daniel DeLeo dan@kallistec.com wrote:

On Friday, August 7, 2015 at 8:55 AM, Bill Warner wrote:

It's actually a wrapper cookbook that I'm testing. the cookbook includes
my base cookbook for internal standards configuration which has a few
community cookbooks as well as a few in house ones to get the base configs,
users, monitoring, dns, host etc all set up. then it includes some
community and in house cookbooks to personalize the target application such
as kafka, cassandra or our in house apps.

I have all the tests running with Vagrant now but due to some minor
incompatibilities with docker (mainly not being able to write to /etc/hosts
or /etc/resolv.conf, some of the cookbooks in the runlist always fail. if
there was a way I could blacklist or stub those cookbooks I'd still be able
to get a nearly full test much faster than I can currently with Vagrant.

I may be able to do a custom rake task that would rewrite the Berksfile
and point to empty cookbooks for those...not sure how else to test it.
You can detect that you’re on Docker with ohai (you’ll have to poke around
for these attributes, I don’t recall what they are off-hand) and then put
the logic into your wrapper cookbook. You could also do this via
attributes. It’s maybe a tiny bit ugly to have that logic in non-test code,
but you lose the ability to manage the run_list when you go to role
cookbooks, so it’s your best option.

--
Daniel DeLeo

--

Bill Warner

7

Putting the container in privileged mode you can copy to a temp location
unmount and replace the hosts file and resolv.conf etc..

This will allow the other cookbooks function as expected.

You can encapsulate this fix within a cookbook for use in your suites.

Using berks groups include exclude it during as needed.
On 7 Aug 2015 22:50, "Bill Warner" bill.warner@gmail.com wrote:

that's not a bad idea at all, and if we ever move to more docker related
stuff this would likely be necessary anyway so not just test code.

Looks like the ohai related node items:
node['virtualization']['system'] = 'docker'
node[virtualization']['systems']['docker'] = 'guest'

I'll give this a try.

-Bill

On Fri, Aug 7, 2015 at 9:11 AM, Daniel DeLeo dan@kallistec.com wrote:

On Friday, August 7, 2015 at 8:55 AM, Bill Warner wrote:

It's actually a wrapper cookbook that I'm testing. the cookbook
includes my base cookbook for internal standards configuration which has a
few community cookbooks as well as a few in house ones to get the base
configs, users, monitoring, dns, host etc all set up. then it includes some
community and in house cookbooks to personalize the target application such
as kafka, cassandra or our in house apps.

I have all the tests running with Vagrant now but due to some minor
incompatibilities with docker (mainly not being able to write to /etc/hosts
or /etc/resolv.conf, some of the cookbooks in the runlist always fail. if
there was a way I could blacklist or stub those cookbooks I'd still be able
to get a nearly full test much faster than I can currently with Vagrant.

I may be able to do a custom rake task that would rewrite the Berksfile
and point to empty cookbooks for those...not sure how else to test it.
You can detect that you’re on Docker with ohai (you’ll have to poke
around for these attributes, I don’t recall what they are off-hand) and
then put the logic into your wrapper cookbook. You could also do this via
attributes. It’s maybe a tiny bit ugly to have that logic in non-test code,
but you lose the ability to manage the run_list when you go to role
cookbooks, so it’s your best option.

--
Daniel DeLeo

--

Bill Warner

8

Hello,

I am new to chef, and have gone the effort to port a few cookbook recipes
to the AIX operating system. In one of my recipes, i update kernel
parameters. However, on AIX, the parameters
require a reboot of the OS for them to come into effect. Does CHEF have
multi-boot support? I suspect not. Ive found recipe’s for shutting down
the os, but nothing for support the
chef kitchen for ‘waiting for the os to come back up’ and continue
installation.

Do i have it correctly?

Jubal Kohlmeier jubal@us.ibm.com
503-747-1333 503-502-7733 cell
IT Specialist Oracle Applications Benchmark

9

You are correct in that Chef doesn't have a "resume after reboot" option.
However, Chef is both idempotent and convergent. Rerunning chef after
reboot will result in NOT configuring the system resources that are in the
state declared by the recipes. For example, if your recipes manage a config
file and install a package, those won't change when chef reruns. However if
the reboot made a change to a file chef manages then it will be updated by
chef after the reboot.

Make sense?

Cheers,
Joshua

On Sunday, August 9, 2015, Jubal Kohlmeier jubal@us.ibm.com wrote:

Hello,

I am new to chef, and have gone the effort to port a few cookbook recipes
to the AIX operating system. In one of my recipes, i update kernel
parameters. However, on AIX, the parameters
require a reboot of the OS for them to come into effect. Does CHEF have
multi-boot support? I suspect not. Ive found recipe's for shutting down the
os, but nothing for support the
chef kitchen for 'waiting for the os to come back up' and continue
installation.

Do i have it correctly?

Jubal Kohlmeier jubal@us.ibm.com
<javascript:_e(%7B%7D,'cvml','jubal@us.ibm.com');>
503-747-1333 503-502-7733 cell
IT Specialist Oracle Applications Benchmark

10

Is the reboot resource idempotent? To extend Joshua’s answer one can run chef-client as a daemon on a node and run this recipe.

Put resources that need to converge before a reboot here

resource1 “Some resource”
resource2 “Some resource”

Reboot the machine

reboot “Need to reboot for kernel changes to take effect”

Put resources that need to converge after a reboot here

Resource3 “Some resource”
Resource4 “Some resource”
What will this recipe do after the reboot? I know it will not converge resource1 and resource2, but will it converge the reboot resource? You can put guards against it but just wondering what the vanilla behavior will be.

Chris

From: Joshua Timberman [mailto:joshua@chef.io]
Sent: Monday, August 10, 2015 9:56 AM
To: chef@lists.opscode.com
Subject: [chef] Re: Does Chef have support for multi-boot install

You are correct in that Chef doesn’t have a “resume after reboot” option. However, Chef is both idempotent and convergent. Rerunning chef after reboot will result in NOT configuring the system resources that are in the state declared by the recipes. For example, if your recipes manage a config file and install a package, those won’t change when chef reruns. However if the reboot made a change to a file chef manages then it will be updated by chef after the reboot.

Make sense?

Cheers,
Joshua

On Sunday, August 9, 2015, Jubal Kohlmeier <jubal@us.ibm.commailto:jubal@us.ibm.com> wrote:
Hello,

I am new to chef, and have gone the effort to port a few cookbook recipes to the AIX operating system. In one of my recipes, i update kernel parameters. However, on AIX, the parameters
require a reboot of the OS for them to come into effect. Does CHEF have multi-boot support? I suspect not. Ive found recipe’s for shutting down the os, but nothing for support the
chef kitchen for ‘waiting for the os to come back up’ and continue installation.

Do i have it correctly?

Jubal Kohlmeier jubal@us.ibm.com<javascript:_e(%7B%7D,‘cvml’,‘jubal@us.ibm.com’);>
503-747-1333 503-502-7733 cell
IT Specialist Oracle Applications Benchmark

11

you'd want to have the reboot resource have a default "action :nothing",
and then have the resources that require a reboot notify the reboot
resource to run. the concept of idempotency doesn't really apply to an
"action", which is why execute resources aren't inherently idempotent
(though the result may be).

On Wed, Aug 12, 2015 at 12:49 PM Fouts, Chris Chris.Fouts@sensus.com
wrote:

Is the reboot resource idempotent? To extend Joshua’s answer one can run
chef-client as a daemon on a node and run this recipe.

Put resources that need to converge before a reboot here

resource1 “Some resource”

resource2 “Some resource”

Reboot the machine

reboot “Need to reboot for kernel changes to take effect”

Put resources that need to converge after a reboot here

Resource3 “Some resource”

Resource4 “Some resource”

What will this recipe do after the reboot? I know it will not converge
resource1 and resource2, but will it converge the reboot resource? You can
put guards against it but just wondering what the vanilla behavior will be.

Chris

From: Joshua Timberman [mailto:joshua@chef.io]
Sent: Monday, August 10, 2015 9:56 AM
To: chef@lists.opscode.com
Subject: [chef] Re: Does Chef have support for multi-boot install

You are correct in that Chef doesn't have a "resume after reboot" option.
However, Chef is both idempotent and convergent. Rerunning chef after
reboot will result in NOT configuring the system resources that are in the
state declared by the recipes. For example, if your recipes manage a config
file and install a package, those won't change when chef reruns. However if
the reboot made a change to a file chef manages then it will be updated by
chef after the reboot.

Make sense?

Cheers,

Joshua

On Sunday, August 9, 2015, Jubal Kohlmeier jubal@us.ibm.com wrote:

Hello,

I am new to chef, and have gone the effort to port a few cookbook recipes
to the AIX operating system. In one of my recipes, i update kernel
parameters. However, on AIX, the parameters
require a reboot of the OS for them to come into effect. Does CHEF have
multi-boot support? I suspect not. Ive found recipe's for shutting down the
os, but nothing for support the
chef kitchen for 'waiting for the os to come back up' and continue
installation.

Do i have it correctly?

Jubal Kohlmeier jubal@us.ibm.com
503-747-1333 503-502-7733 cell
IT Specialist Oracle Applications Benchmark

12

That makes a lot of sense!

Chris

From: Nathan Williams [mailto:nath.e.will@gmail.com]
Sent: Wednesday, August 12, 2015 4:01 PM
To: chef@lists.opscode.com
Subject: [chef] Re: RE: Re: Does Chef have support for multi-boot install

you’d want to have the reboot resource have a default “action :nothing”, and then have the resources that require a reboot notify the reboot resource to run. the concept of idempotency doesn’t really apply to an “action”, which is why execute resources aren’t inherently idempotent (though the result may be).

On Wed, Aug 12, 2015 at 12:49 PM Fouts, Chris <Chris.Fouts@sensus.commailto:Chris.Fouts@sensus.com> wrote:
Is the reboot resource idempotent? To extend Joshua’s answer one can run chef-client as a daemon on a node and run this recipe.

Put resources that need to converge before a reboot here

resource1 “Some resource”
resource2 “Some resource”

Reboot the machine

reboot “Need to reboot for kernel changes to take effect”

Put resources that need to converge after a reboot here

Resource3 “Some resource”
Resource4 “Some resource”
What will this recipe do after the reboot? I know it will not converge resource1 and resource2, but will it converge the reboot resource? You can put guards against it but just wondering what the vanilla behavior will be.

Chris

From: Joshua Timberman [mailto:joshua@chef.iomailto:joshua@chef.io]
Sent: Monday, August 10, 2015 9:56 AM
To: chef@lists.opscode.commailto:chef@lists.opscode.com
Subject: [chef] Re: Does Chef have support for multi-boot install

You are correct in that Chef doesn’t have a “resume after reboot” option. However, Chef is both idempotent and convergent. Rerunning chef after reboot will result in NOT configuring the system resources that are in the state declared by the recipes. For example, if your recipes manage a config file and install a package, those won’t change when chef reruns. However if the reboot made a change to a file chef manages then it will be updated by chef after the reboot.

Make sense?

Cheers,
Joshua

On Sunday, August 9, 2015, Jubal Kohlmeier <jubal@us.ibm.commailto:jubal@us.ibm.com> wrote:
Hello,

I am new to chef, and have gone the effort to port a few cookbook recipes to the AIX operating system. In one of my recipes, i update kernel parameters. However, on AIX, the parameters
require a reboot of the OS for them to come into effect. Does CHEF have multi-boot support? I suspect not. Ive found recipe’s for shutting down the os, but nothing for support the
chef kitchen for ‘waiting for the os to come back up’ and continue installation.

Do i have it correctly?

Jubal Kohlmeier jubal@us.ibm.commailto:jubal@us.ibm.com
503-747-1333 503-502-7733 cell
IT Specialist Oracle Applications Benchmark