# Status of test-kitchen?

**URL:** <https://discourse.chef.io/t/status-of-test-kitchen/3136>\
**Category:** Chef Infra (archive)\
**Created:** [September 26, 2012, 2:13pm UTC](https://discourse.chef.io/t/status-of-test-kitchen/3136 "2012-09-26T14:13:27Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Bryan\_Berry](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/bryan_berry/32/213_2.png) [@Bryan\_Berry](https://discourse.chef.io/u/Bryan_Berry)\
**Post date:** [September 26, 2012, 2:13pm UTC](https://discourse.chef.io/t/status-of-test-kitchen/3136/1 "2012-09-26T14:13:27Z")

</div>

I really like test-kitchen and I think it has tremendous potential however  
I find that in its current state it is not a productive tool for me. There  
are many outstanding bugs, as one can see here:  
[http://tickets.opscode.com/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+KITCHEN+AND+fixVersion+%3D+EMPTY](http://tickets.opscode.com/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+KITCHEN+AND+fixVersion+%3D+EMPTY)  
but  
the one that I find particularly challenging is that I cannot get  
test-kitchen to test only one virtualbox image rather than all 4 (centos  
5,6, ubuntu 10,11). Further, I can’t figure out how to run `librarian-chef update` if I make any changes to my Cheffile.

I have gotten the distinct impression that test-kitchen is the preferred  
testing tool for the opscode-cookbooks/ set of cookbooks. However, at this  
stage I find myself just spending to much time fighting w/ test-kitchen. I  
would like to know if anyone plans to put in the time and effort to bring  
test-kitchen to a more stable and usable form. I would love to pitch in,  
but right now my priority is to improve the state of the  
database/postgresql and java-related cookbooks.

In the meantime I will refocus my workflow, which I call "The Riot Way"  
based on Jamie Winsor’s cookbook authoring guide  
[http://vialstudios.com/guide-authoring-cookbooks.html](http://vialstudios.com/guide-authoring-cookbooks.html)

BryanWB

---

<div class="post-metadata">

**Author:** ![Mathias\_Lafeldt](https://avatars.discourse-cdn.com/v4/letter/m/aeb1de/32.png) [@Mathias\_Lafeldt](https://discourse.chef.io/u/Mathias_Lafeldt)\
**Post date:** [September 26, 2012, 9:38pm UTC](https://discourse.chef.io/t/status-of-test-kitchen/3136/2 "2012-09-26T21:38:52Z")

</div>

Last time I checked, test-kitchen required some custom Vagrant boxes  
to work. It wasn't even possible to use the boxes built with Bento,  
not to speak of [http://files.vagrantup.com/lucid32.box](http://files.vagrantup.com/lucid32.box) and friends.

I'd like to know the status as well.

-Mathias

On Wed, Sep 26, 2012 at 4:13 PM, Bryan Berry [bryan.berry@gmail.com](mailto:bryan.berry@gmail.com) wrote:

> I really like test-kitchen and I think it has tremendous potential however I  
> find that in its current state it is not a productive tool for me. There  
> are many outstanding bugs, as one can see here:  
> [http://tickets.opscode.com/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+KITCHEN+AND+fixVersion+%3D+EMPTY](http://tickets.opscode.com/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+KITCHEN+AND+fixVersion+%3D+EMPTY)  
> but the one that I find particularly challenging is that I cannot get  
> test-kitchen to test only one virtualbox image rather than all 4 (centos  
> 5,6, ubuntu 10,11). Further, I can't figure out how to run `librarian-chef update` if I make any changes to my Cheffile.
> 
> I have gotten the distinct impression that test-kitchen is the preferred  
> testing tool for the opscode-cookbooks/ set of cookbooks. However, at this  
> stage I find myself just spending to much time fighting w/ test-kitchen. I  
> would like to know if anyone plans to put in the time and effort to bring  
> test-kitchen to a more stable and usable form. I would love to pitch in, but  
> right now my priority is to improve the state of the database/postgresql and  
> java-related cookbooks.
> 
> In the meantime I will refocus my workflow, which I call "The Riot Way"  
> based on Jamie Winsor's cookbook authoring guide  
> [http://vialstudios.com/guide-authoring-cookbooks.html](http://vialstudios.com/guide-authoring-cookbooks.html)
> 
> BryanWB

---

<div class="post-metadata">

**Author:** ![Joshua\_Timberman](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/joshua_timberman/32/424_2.png) [@Joshua\_Timberman](https://discourse.chef.io/u/Joshua_Timberman)\
**Post date:** [September 27, 2012, 11:36pm UTC](https://discourse.chef.io/t/status-of-test-kitchen/3136/3 "2012-09-27T23:36:32Z")

</div>

Hi Bryan,

On Sep 26, 2012, at 8:13 AM, Bryan Berry [bryan.berry@gmail.com](mailto:bryan.berry@gmail.com) wrote:

> I really like test-kitchen and I think it has tremendous potential however I find that in its current state it is not a productive tool for me. There are many outstanding bugs, as one can see here: [http://tickets.opscode.com/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+KITCHEN+AND+fixVersion+%3D+EMPTY](http://tickets.opscode.com/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+KITCHEN+AND+fixVersion+%3D+EMPTY) but the one that I find particularly challenging is that I cannot get test-kitchen to test only one virtualbox image rather than all 4 (centos 5,6, ubuntu 10,11). Further, I can't figure out how to run `librarian-chef update` if I make any changes to my Cheffile.

Our current code review meetings cover tickets that are resolved. That is, we review pull requests/patches. Most of the tickets in Test Kitchen aren't resolved, so there isn't much to review (yet).

That said, I see four tickets in resolved state. They're all things I'd like to get into the next release (0.5.6), but I don't have an ETA.

- KITCHEN-4
- KITCHEN-22
- KITCHEN-35
- KITCHEN-37

Much of our general code review effort recently has been focused on getting 10.14 regressions fixed (mainly in the deploy resource)[0]. This week we had a couple people out of the office, too.

Bringing up the topic on the mailing list(s) is good though, and this is exactly the kind of topic that the chef-dev list and #chef-hacking IRC channel are intended to facilitate.

> I have gotten the distinct impression that test-kitchen is the preferred testing tool for the opscode-cookbooks/ set of cookbooks. However, at this stage I find myself just spending to much time fighting w/ test-kitchen. I would like to know if anyone plans to put in the time and effort to bring test-kitchen to a more stable and usable form. I would love to pitch in, but right now my priority is to improve the state of the database/postgresql and java-related cookbooks.

Your contributions to the cookbooks are awesome, thank you. The 1.6.0 release of the Java cookbook passed a test-kitchen run, too :-).

As you've noticed, one of the things we're doing is add Test Kitchen support to our cookbooks, and that is revealing issues that need to be fixed. For example, we can't really do much useful testing of heavily search based cookbooks (like nagios or munin) in test kitchen because it only supports Chef Solo[1]. Our internal CI system uses OpenStack, so a runner for that in Test Kitchen would be awesome[2].

> In the meantime I will refocus my workflow, which I call "The Riot Way" based on Jamie Winsor's cookbook authoring guide [http://vialstudios.com/guide-authoring-cookbooks.html](http://vialstudios.com/guide-authoring-cookbooks.html)

I've only skimmed the guide, but it looks great. Jamie and Riot are awesome.

## [0]: 10.14.4 was just tagged for release while I wrote this post. [1]: [https://tickets.opscode.com/browse/KITCHEN-31](https://tickets.opscode.com/browse/KITCHEN-31) [2]: [http://tickets.opscode.com/browse/KITCHEN-5](http://tickets.opscode.com/browse/KITCHEN-5)

Opscode, Inc  
Joshua Timberman, Technical Community Manager  
IRC, Skype, Twitter, Github: jtimberman

---

<div class="post-metadata">

**Author:** ![Jeremiah\_Snapp](https://avatars.discourse-cdn.com/v4/letter/j/3bc359/32.png) [@Jeremiah\_Snapp](https://discourse.chef.io/u/Jeremiah_Snapp)\
**Post date:** [September 28, 2012, 12:15am UTC](https://discourse.chef.io/t/status-of-test-kitchen/3136/4 "2012-09-28T00:15:13Z")

</div>

I know I talked about this before but this seems like a great time to bring  
it up again. What about chef-solo-search[0](https://github.com/edelight/chef-solo-search)? It gives chef solo the ability  
to use many search features. Obviously it can’t completely replace  
searching against a chef server. At least I don’t think it can. But can it  
help with some of the issues you are running into? If so is there any  
technical or philosophical reason preventing it from being rolled into the  
chef client code?

---

<div class="post-metadata">

**Author:** ![Bryan\_Berry](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/bryan_berry/32/213_2.png) [@Bryan\_Berry](https://discourse.chef.io/u/Bryan_Berry)\
**Post date:** [September 28, 2012, 9:00am UTC](https://discourse.chef.io/t/status-of-test-kitchen/3136/5 "2012-09-28T09:00:04Z")

</div>

+1, I would love to know the same

On Fri, Sep 28, 2012 at 2:15 AM, Jeremiah Snapp  
[jeremiah.snapp@gmail.com](mailto:jeremiah.snapp@gmail.com) wrote:

> I know I talked about this before but this seems like a great time to bring  
> it up again. What about chef-solo-search[0](https://github.com/edelight/chef-solo-search)? It gives chef solo the ability  
> to use many search features. Obviously it can't completely replace searching  
> against a chef server. At least I don't think it can. But can it help with  
> some of the issues you are running into? If so is there any technical or  
> philosophical reason preventing it from being rolled into the chef client  
> code?

---

<div class="post-metadata">

**Author:** ![Bryan\_Berry](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/bryan_berry/32/213_2.png) [@Bryan\_Berry](https://discourse.chef.io/u/Bryan_Berry)\
**Post date:** [September 28, 2012, 9:08am UTC](https://discourse.chef.io/t/status-of-test-kitchen/3136/6 "2012-09-28T09:08:26Z")

</div>

On Fri, Sep 28, 2012 at 1:36 AM, Joshua Timberman [joshua@opscode.com](mailto:joshua@opscode.com) wrote:

> Hi Bryan,
> 
> On Sep 26, 2012, at 8:13 AM, Bryan Berry [bryan.berry@gmail.com](mailto:bryan.berry@gmail.com) wrote:  
> Bringing up the topic on the mailing list(s) is good though, and this is exactly the kind of topic that the chef-dev list and #chef-hacking IRC channel are intended to facilitate.
> 
> > I have gotten the distinct impression that test-kitchen is the preferred testing tool for the opscode-cookbooks/ set of cookbooks. However, at this stage I find myself just spending to much time fighting w/ test-kitchen. I would like to know if anyone plans to put in the time and effort to bring test-kitchen to a more stable and usable form. I would love to pitch in, but right now my priority is to improve the state of the database/postgresql and java-related cookbooks.
> 
> Your contributions to the cookbooks are awesome, thank you. The 1.6.0 release of the Java cookbook passed a test-kitchen run, too :-).

Tks, I used test-kitchen for my changes to the java cookbok but I have  
to say I found it quite frustrating and time-consuming. I hope the  
process gets the smoother in the coming weeks.

Good to hear that you guys are still focused on improving test-kitchen

---

<div class="post-metadata">

**Author:** ![Haselwanter\_Edmund](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/haselwanter_edmund/32/635_2.png) [@Haselwanter\_Edmund](https://discourse.chef.io/u/Haselwanter_Edmund)\
**Post date:** [September 28, 2012, 9:11am UTC](https://discourse.chef.io/t/status-of-test-kitchen/3136/7 "2012-09-28T09:11:47Z")

</div>

+1. would love to see solo and client parity

--  
Edmund Haselwanter  
[edmund@haselwanter.com](mailto:edmund@haselwanter.com)

On 28.09.2012, at 11:00, Bryan Berry [bryan.berry@gmail.com](mailto:bryan.berry@gmail.com) wrote:

> +1, I would love to know the same
> 
> On Fri, Sep 28, 2012 at 2:15 AM, Jeremiah Snapp  
> [jeremiah.snapp@gmail.com](mailto:jeremiah.snapp@gmail.com) wrote:
> 
> > I know I talked about this before but this seems like a great time to bring  
> > it up again. What about chef-solo-search[0](https://github.com/edelight/chef-solo-search)? It gives chef solo the ability  
> > to use many search features. Obviously it can't completely replace searching  
> > against a chef server. At least I don't think it can. But can it help with  
> > some of the issues you are running into? If so is there any technical or  
> > philosophical reason preventing it from being rolled into the chef client  
> > code?