Call for Testing: significant changes made to minitest-handler-cookbook

Chef Infra (archive)
1

Dear Chefs,

I have significantly refactored the minitest-handler-cookbook and
added tests using test-kitchen 1.0 + kitchen-lxc driver.

https://github.com/btm/minitest-handler-cookbook/tree/refactor

In this refactoring i have done my done my best to clean up the code
and simplify its use. My changes are backwards compatible so current
use should not break. I wrote parts of the minitest-handler cookbook
before knowing much about existing ruby testing practices. For this
reason the cookbook was unnecessarily complicated.

Old Convention:

place your tests in files/default/tests/minitest/_test.rb
and support files in files/default/tests/minitest/support/

About 70% of the time I place my tests in the wrong directory and
misname them. My fingers also tire from typing these long paths. For
this reason I have changed the convention.

New Convention:

Place your tests in files/default/_test.rb and support
files in files/default/
For example, put tests for the default recipe in
files/default/default_test.rb and your helper file in
files/default/spec_helper.rb

In the new branch, minitest-handler will still copy files located in
files/default/tests/minitest/ and files/default/tests/minitest/support

I would really prefer to put such tests in test/ or test/minitest/
but then these files would not be uploaded to chef-server. I have
opened this ticket http://tickets.opscode.com/browse/CHEF-4044 to
change this but I haven’t seen any progress on it yet.

New Feature:

By default, minitest-handler will load all test files for each recipe
loaded during the chef run, whether via the run_list, through a role,
or include_recipe. You can tell minitest-handler to only load tests
for certain recipes using the node[:minitest][:recipes] attribute. If
that attribute is non-empty, minitest-handler will only execute the
tests specified in that list.

Call for Testing:

I have only tested this new branch w/ ubuntu linux. I could really use
testing help to ensure that recipes loaded through a role are tested
and to test this cookbook on Windows.

If I don’t hear of any major issues on this branch, I will merge it
into master early next week and push it to community.opscode.com

2

I gave this a quick run on Windows this AM, using the older path location
for tests and support files, and I didn't run into any issues.

What do you think about moving the file path out to an attribute as opposed
to deprecating the current location. With deprecation comes the assumption
that at some point the existing path will no longer be support. That has
the potential for a decent amount of work for folks. I for one would have
to incur a decent amount of work to move all our test files.

We've wrapped a simple "cookbook create" task so it creates a skeleton test
file in the current location, so we don't suffer placing them in the wrong
directory too much, albeit it does still happen at times.

It feels like using an attribute to control this would allow you achieve
your goal of flattening the path, while not incurring a lot of work
consumers of the cookbook.

Thoughts?

On Thu, Apr 11, 2013 at 6:19 AM, Bryan Berry bryan.berry@gmail.com wrote:

Dear Chefs,

I have significantly refactored the minitest-handler-cookbook and
added tests using test-kitchen 1.0 + kitchen-lxc driver.

https://github.com/btm/minitest-handler-cookbook/tree/refactor

In this refactoring i have done my done my best to clean up the code
and simplify its use. My changes are backwards compatible so current
use should not break. I wrote parts of the minitest-handler cookbook
before knowing much about existing ruby testing practices. For this
reason the cookbook was unnecessarily complicated.

Old Convention:

place your tests in files/default/tests/minitest/_test.rb
and support files in files/default/tests/minitest/support/

About 70% of the time I place my tests in the wrong directory and
misname them. My fingers also tire from typing these long paths. For
this reason I have changed the convention.

New Convention:

Place your tests in files/default/_test.rb and support
files in files/default/
For example, put tests for the default recipe in
files/default/default_test.rb and your helper file in
files/default/spec_helper.rb

In the new branch, minitest-handler will still copy files located in
files/default/tests/minitest/ and files/default/tests/minitest/support

I would really prefer to put such tests in test/ or test/minitest/
but then these files would not be uploaded to chef-server. I have
opened this ticket http://tickets.opscode.com/browse/CHEF-4044 to
change this but I haven't seen any progress on it yet.

New Feature:

By default, minitest-handler will load all test files for each recipe
loaded during the chef run, whether via the run_list, through a role,
or include_recipe. You can tell minitest-handler to only load tests
for certain recipes using the node[:minitest][:recipes] attribute. If
that attribute is non-empty, minitest-handler will only execute the
tests specified in that list.

Call for Testing:

I have only tested this new branch w/ ubuntu linux. I could really use
testing help to ensure that recipes loaded through a role are tested
and to test this cookbook on Windows.

If I don't hear of any major issues on this branch, I will merge it
into master early next week and push it to community.opscode.com

3

hmm, i don't know, i feel like setting this w/ an attribute will just
make the issue more complicated

i could be swayed to make it files/default/test rather than just files/default/

On Thu, Apr 11, 2013 at 5:02 PM, David Petzel davidpetzel@gmail.com wrote:

I gave this a quick run on Windows this AM, using the older path location
for tests and support files, and I didn't run into any issues.

What do you think about moving the file path out to an attribute as opposed
to deprecating the current location. With deprecation comes the assumption
that at some point the existing path will no longer be support. That has the
potential for a decent amount of work for folks. I for one would have to
incur a decent amount of work to move all our test files.

We've wrapped a simple "cookbook create" task so it creates a skeleton test
file in the current location, so we don't suffer placing them in the wrong
directory too much, albeit it does still happen at times.

It feels like using an attribute to control this would allow you achieve
your goal of flattening the path, while not incurring a lot of work
consumers of the cookbook.

Thoughts?

On Thu, Apr 11, 2013 at 6:19 AM, Bryan Berry bryan.berry@gmail.com wrote:

Dear Chefs,

I have significantly refactored the minitest-handler-cookbook and
added tests using test-kitchen 1.0 + kitchen-lxc driver.

https://github.com/btm/minitest-handler-cookbook/tree/refactor

In this refactoring i have done my done my best to clean up the code
and simplify its use. My changes are backwards compatible so current
use should not break. I wrote parts of the minitest-handler cookbook
before knowing much about existing ruby testing practices. For this
reason the cookbook was unnecessarily complicated.

Old Convention:

place your tests in files/default/tests/minitest/_test.rb
and support files in files/default/tests/minitest/support/

About 70% of the time I place my tests in the wrong directory and
misname them. My fingers also tire from typing these long paths. For
this reason I have changed the convention.

New Convention:

Place your tests in files/default/_test.rb and support
files in files/default/
For example, put tests for the default recipe in
files/default/default_test.rb and your helper file in
files/default/spec_helper.rb

In the new branch, minitest-handler will still copy files located in
files/default/tests/minitest/ and files/default/tests/minitest/support

I would really prefer to put such tests in test/ or test/minitest/
but then these files would not be uploaded to chef-server. I have
opened this ticket http://tickets.opscode.com/browse/CHEF-4044 to
change this but I haven't seen any progress on it yet.

New Feature:

By default, minitest-handler will load all test files for each recipe
loaded during the chef run, whether via the run_list, through a role,
or include_recipe. You can tell minitest-handler to only load tests
for certain recipes using the node[:minitest][:recipes] attribute. If
that attribute is non-empty, minitest-handler will only execute the
tests specified in that list.

Call for Testing:

I have only tested this new branch w/ ubuntu linux. I could really use
testing help to ensure that recipes loaded through a role are tested
and to test this cookbook on Windows.

If I don't hear of any major issues on this branch, I will merge it
into master early next week and push it to community.opscode.com

4

If the path is anything other than what it is now, it will eventually
create work to update existing cookbooks. I'm not suggesting its a huge
undertaking, but at the same time is some level of effort.

What's the case where it becomes more complicated? The code, or the user
experience? I can see it complicating the code path slightly, but since we
are already using attributes to control the file pattern to search for as
well as where to copy them too it feels like a natural extension to what
has already been configured via attributes.

Again, its not a huge deal, but I did want to pose the question while the
thought was fresh in my mind.

BTW, I really like the new functionality that picks up tests for recipes
that were run via include_recipe.

On Thu, Apr 11, 2013 at 11:14 AM, Bryan Berry bryan.berry@gmail.com wrote:

hmm, i don't know, i feel like setting this w/ an attribute will just
make the issue more complicated

i could be swayed to make it files/default/test rather than just
files/default/

On Thu, Apr 11, 2013 at 5:02 PM, David Petzel davidpetzel@gmail.com
wrote:

I gave this a quick run on Windows this AM, using the older path location
for tests and support files, and I didn't run into any issues.

What do you think about moving the file path out to an attribute as
opposed
to deprecating the current location. With deprecation comes the
assumption
that at some point the existing path will no longer be support. That has
the
potential for a decent amount of work for folks. I for one would have to
incur a decent amount of work to move all our test files.

We've wrapped a simple "cookbook create" task so it creates a skeleton
test
file in the current location, so we don't suffer placing them in the
wrong
directory too much, albeit it does still happen at times.

It feels like using an attribute to control this would allow you achieve
your goal of flattening the path, while not incurring a lot of work
consumers of the cookbook.

Thoughts?

On Thu, Apr 11, 2013 at 6:19 AM, Bryan Berry bryan.berry@gmail.com
wrote:

Dear Chefs,

I have significantly refactored the minitest-handler-cookbook and
added tests using test-kitchen 1.0 + kitchen-lxc driver.

https://github.com/btm/minitest-handler-cookbook/tree/refactor

In this refactoring i have done my done my best to clean up the code
and simplify its use. My changes are backwards compatible so current
use should not break. I wrote parts of the minitest-handler cookbook
before knowing much about existing ruby testing practices. For this
reason the cookbook was unnecessarily complicated.

Old Convention:

place your tests in files/default/tests/minitest/_test.rb
and support files in files/default/tests/minitest/support/

About 70% of the time I place my tests in the wrong directory and
misname them. My fingers also tire from typing these long paths. For
this reason I have changed the convention.

New Convention:

Place your tests in files/default/_test.rb and support
files in files/default/
For example, put tests for the default recipe in
files/default/default_test.rb and your helper file in
files/default/spec_helper.rb

In the new branch, minitest-handler will still copy files located in
files/default/tests/minitest/ and files/default/tests/minitest/support

I would really prefer to put such tests in test/ or test/minitest/
but then these files would not be uploaded to chef-server. I have
opened this ticket http://tickets.opscode.com/browse/CHEF-4044 to
change this but I haven't seen any progress on it yet.

New Feature:

By default, minitest-handler will load all test files for each recipe
loaded during the chef run, whether via the run_list, through a role,
or include_recipe. You can tell minitest-handler to only load tests
for certain recipes using the node[:minitest][:recipes] attribute. If
that attribute is non-empty, minitest-handler will only execute the
tests specified in that list.

Call for Testing:

I have only tested this new branch w/ ubuntu linux. I could really use
testing help to ensure that recipes loaded through a role are tested
and to test this cookbook on Windows.

If I don't hear of any major issues on this branch, I will merge it
into master early next week and push it to community.opscode.com

5

while i say "deprecated" i probably will never actually break the
backwards compatibility

What's the case where it becomes more complicated? The code, or the user
experience? I can see it complicating the code path slightly, but since we
are already using attributes to control the file pattern to search for as
well as where to copy them too it feels like a natural extension to what has
already been configured via attributes.

I don't know. i am still not convinced it is a good idea. will have to
think about it

On Thu, Apr 11, 2013 at 5:20 PM, David Petzel davidpetzel@gmail.com wrote:

If the path is anything other than what it is now, it will eventually create
work to update existing cookbooks. I'm not suggesting its a huge
undertaking, but at the same time is some level of effort.

What's the case where it becomes more complicated? The code, or the user
experience? I can see it complicating the code path slightly, but since we
are already using attributes to control the file pattern to search for as
well as where to copy them too it feels like a natural extension to what has
already been configured via attributes.

Again, its not a huge deal, but I did want to pose the question while the
thought was fresh in my mind.

BTW, I really like the new functionality that picks up tests for recipes
that were run via include_recipe.

On Thu, Apr 11, 2013 at 11:14 AM, Bryan Berry bryan.berry@gmail.com wrote:

hmm, i don't know, i feel like setting this w/ an attribute will just
make the issue more complicated

i could be swayed to make it files/default/test rather than just
files/default/

On Thu, Apr 11, 2013 at 5:02 PM, David Petzel davidpetzel@gmail.com
wrote:

I gave this a quick run on Windows this AM, using the older path
location
for tests and support files, and I didn't run into any issues.

What do you think about moving the file path out to an attribute as
opposed
to deprecating the current location. With deprecation comes the
assumption
that at some point the existing path will no longer be support. That has
the
potential for a decent amount of work for folks. I for one would have to
incur a decent amount of work to move all our test files.

We've wrapped a simple "cookbook create" task so it creates a skeleton
test
file in the current location, so we don't suffer placing them in the
wrong
directory too much, albeit it does still happen at times.

It feels like using an attribute to control this would allow you achieve
your goal of flattening the path, while not incurring a lot of work
consumers of the cookbook.

Thoughts?

On Thu, Apr 11, 2013 at 6:19 AM, Bryan Berry bryan.berry@gmail.com
wrote:

Dear Chefs,

I have significantly refactored the minitest-handler-cookbook and
added tests using test-kitchen 1.0 + kitchen-lxc driver.

https://github.com/btm/minitest-handler-cookbook/tree/refactor

In this refactoring i have done my done my best to clean up the code
and simplify its use. My changes are backwards compatible so current
use should not break. I wrote parts of the minitest-handler cookbook
before knowing much about existing ruby testing practices. For this
reason the cookbook was unnecessarily complicated.

Old Convention:

place your tests in files/default/tests/minitest/_test.rb
and support files in files/default/tests/minitest/support/

About 70% of the time I place my tests in the wrong directory and
misname them. My fingers also tire from typing these long paths. For
this reason I have changed the convention.

New Convention:

Place your tests in files/default/_test.rb and support
files in files/default/
For example, put tests for the default recipe in
files/default/default_test.rb and your helper file in
files/default/spec_helper.rb

In the new branch, minitest-handler will still copy files located in
files/default/tests/minitest/ and files/default/tests/minitest/support

I would really prefer to put such tests in test/ or test/minitest/
but then these files would not be uploaded to chef-server. I have
opened this ticket http://tickets.opscode.com/browse/CHEF-4044 to
change this but I haven't seen any progress on it yet.

New Feature:

By default, minitest-handler will load all test files for each recipe
loaded during the chef run, whether via the run_list, through a role,
or include_recipe. You can tell minitest-handler to only load tests
for certain recipes using the node[:minitest][:recipes] attribute. If
that attribute is non-empty, minitest-handler will only execute the
tests specified in that list.

Call for Testing:

I have only tested this new branch w/ ubuntu linux. I could really use
testing help to ensure that recipes loaded through a role are tested
and to test this cookbook on Windows.

If I don't hear of any major issues on this branch, I will merge it
into master early next week and push it to community.opscode.com

6

On Thu, Apr 11, 2013 at 12:19 PM, Bryan Berry bryan.berry@gmail.com wrote:

Dear Chefs,

I have significantly refactored the minitest-handler-cookbook and
added tests using test-kitchen 1.0 + kitchen-lxc driver.

Nice! Can these be run on travis-ci as well? It's not in the .travis.yml
file yet at least...

Btw: just curious - where does the ubuntu-cycle base lxc script [1] come
from? Did you have to write this from scratch?

Cheers, Torben

[1] lxc-ubuntu-cycle · GitHub

7

I don't believe that you can run this on travis-ci as you are already
running w/in an lxc container

ubuntu-cycle script is some minor additions to the default ubuntu lxc
script that i have made to play better w/ kitchen-lxc

On Sat, Apr 13, 2013 at 2:50 PM, Torben Knerr ukio@gmx.de wrote:

On Thu, Apr 11, 2013 at 12:19 PM, Bryan Berry bryan.berry@gmail.com wrote:

Dear Chefs,

I have significantly refactored the minitest-handler-cookbook and
added tests using test-kitchen 1.0 + kitchen-lxc driver.

Nice! Can these be run on travis-ci as well? It's not in the .travis.yml
file yet at least...

Btw: just curious - where does the ubuntu-cycle base lxc script [1] come
from? Did you have to write this from scratch?

Cheers, Torben

[1] lxc-ubuntu-cycle · GitHub

8

On Sat, Apr 13, 2013 at 2:54 PM, Bryan Berry bryan.berry@gmail.com wrote:

I don't believe that you can run this on travis-ci as you are already
running w/in an lxc container

Ah, I thought they were running in Vagrant boxes and thus you can't use the
kitchen-vagrant driver on travis-ci, can you?

ubuntu-cycle script is some minor additions to the default ubuntu lxc
script that i have made to play better w/ kitchen-lxc

Cool, thanks!

On Sat, Apr 13, 2013 at 2:50 PM, Torben Knerr ukio@gmx.de wrote:

On Thu, Apr 11, 2013 at 12:19 PM, Bryan Berry bryan.berry@gmail.com
wrote:

Dear Chefs,

I have significantly refactored the minitest-handler-cookbook and
added tests using test-kitchen 1.0 + kitchen-lxc driver.

Nice! Can these be run on travis-ci as well? It's not in the .travis.yml
file yet at least...

Btw: just curious - where does the ubuntu-cycle base lxc script [1] come
from? Did you have to write this from scratch?

Cheers, Torben

[1] lxc-ubuntu-cycle · GitHub

9

On Thu, Apr 11, 2013 at 11:14 AM, Bryan Berry bryan.berry@gmail.com wrote:

i could be swayed to make it files/default/test rather than just
files/default/

I like using the test subdirectory as it keeps the tests and the "normal"
cookbook files separated We support a lot of "new" chef users, so
anything to lessen the confusion is appreciated.

10

ok, the ppl have spoken

files/default/test it is

thus

example:
files/default/test/default_test.rb
files/default/test/spec_helper.rb

i will still support the old paths

On Mon, Apr 15, 2013 at 1:08 PM, Brian Akins brian@akins.org wrote:

On Thu, Apr 11, 2013 at 11:14 AM, Bryan Berry bryan.berry@gmail.com wrote:

i could be swayed to make it files/default/test rather than just
files/default/

I like using the test subdirectory as it keeps the tests and the "normal"
cookbook files separated We support a lot of "new" chef users, so anything
to lessen the confusion is appreciated.

11

Would be sweet to get test/* to be part of the cookbook in chef server. So
we can put tests in their logical place.

On Mon, Apr 15, 2013 at 5:09 AM, Bryan Berry bryan.berry@gmail.com wrote:

ok, the ppl have spoken

files/default/test it is

thus

example:
files/default/test/default_test.rb
files/default/test/spec_helper.rb

i will still support the old paths

On Mon, Apr 15, 2013 at 1:08 PM, Brian Akins brian@akins.org wrote:

On Thu, Apr 11, 2013 at 11:14 AM, Bryan Berry bryan.berry@gmail.com
wrote:

i could be swayed to make it files/default/test rather than just
files/default/

I like using the test subdirectory as it keeps the tests and the "normal"
cookbook files separated We support a lot of "new" chef users, so
anything
to lessen the confusion is appreciated.