Hi Bryan,
On Sep 26, 2012, at 8:13 AM, Bryan Berry bryan.berry@gmail.com wrote:
I really like test-kitchen and I think it has tremendous potential however I find that in its current state it is not a productive tool for me. There are many outstanding bugs, as one can see here: http://tickets.opscode.com/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+KITCHEN+AND+fixVersion+%3D+EMPTY but the one that I find particularly challenging is that I cannot get test-kitchen to test only one virtualbox image rather than all 4 (centos 5,6, ubuntu 10,11). Further, I can't figure out how to run librarian-chef update if I make any changes to my Cheffile.
Our current code review meetings cover tickets that are resolved. That is, we review pull requests/patches. Most of the tickets in Test Kitchen aren't resolved, so there isn't much to review (yet).
That said, I see four tickets in resolved state. They're all things I'd like to get into the next release (0.5.6), but I don't have an ETA.
- KITCHEN-4
- KITCHEN-22
- KITCHEN-35
- KITCHEN-37
Much of our general code review effort recently has been focused on getting 10.14 regressions fixed (mainly in the deploy resource)[0]. This week we had a couple people out of the office, too.
Bringing up the topic on the mailing list(s) is good though, and this is exactly the kind of topic that the chef-dev list and #chef-hacking IRC channel are intended to facilitate.
I have gotten the distinct impression that test-kitchen is the preferred testing tool for the opscode-cookbooks/ set of cookbooks. However, at this stage I find myself just spending to much time fighting w/ test-kitchen. I would like to know if anyone plans to put in the time and effort to bring test-kitchen to a more stable and usable form. I would love to pitch in, but right now my priority is to improve the state of the database/postgresql and java-related cookbooks.
Your contributions to the cookbooks are awesome, thank you. The 1.6.0 release of the Java cookbook passed a test-kitchen run, too :-).
As you've noticed, one of the things we're doing is add Test Kitchen support to our cookbooks, and that is revealing issues that need to be fixed. For example, we can't really do much useful testing of heavily search based cookbooks (like nagios or munin) in test kitchen because it only supports Chef Solo[1]. Our internal CI system uses OpenStack, so a runner for that in Test Kitchen would be awesome[2].
In the meantime I will refocus my workflow, which I call "The Riot Way" based on Jamie Winsor's cookbook authoring guide http://vialstudios.com/guide-authoring-cookbooks.html
I've only skimmed the guide, but it looks great. Jamie and Riot are awesome.
Opscode, Inc
Joshua Timberman, Technical Community Manager
IRC, Skype, Twitter, Github: jtimberman