On Aug 18, 2012, at 8:45 AM, Torben Knerr firstname.lastname@example.org wrote:
Thanks! We’re all pretty excited about this. Andrew did a great job with it.
- Concerning cookbooks with dependencies to other cookbooks:
Having noticed the librarian Cheffile in the
apache2/test/kitchen/cookbooks directory, I wonder whether the
cookbook dependencies therein are automatically resolved by
test-kitchen, are they? Would a Berksfile be supported as well?
Yes - test-kitchen wraps up librarian-chef in the base runner:
The reason Berkshelf isn’t supported is simply because it didn’t exist (publicly available) when test-kitchen was created and developed.
If these dependencies are indeed automatically resolved then that
would be worth mentioning in the README.md
Would you please open a ticket in the KITCHEN project for that?
- Do you have an example with test-kitchen tests running on Travis
CI? Btw: can you actually spin up a vagrant VM on Travis?
I’m not sure if Travis would support that. I haven’t done much with Travis besides wire it up with foodcritic and knife cookbook test. And by that I mean I’ve merged pull requests that do that :).
We have Jenkins internally for other projects, like building our omnibus packages, and the services that run Hosted/Private Chef. We also have the cookbooks tested under test-kitchen, though it is very simplistic right now.
I’d like to see Travis.CI support in each of our cookbooks, whether they’re just doing the foodcritic/cookbook test, or running test-kitchen, that’d be great.
- Concerning fast running tests:
It would be great to have fast running unit-like tests (which don’t
require to spin up a vagrant VM first) covered by test-kitchen as
well. Chefspec comes to my mind, but I haven’t found it referenced in
Oh wait, isn’t that already supported by the cookbook / script
parameter which defaults to ‘rspec spec’? Or would I do that within
the cookbook / preflight command to make sure this runs before the
vagrant VM is spun up?
Yes, that should be done by the “script” parameter. You should be able to just run that in your cookbook directory without test-kitchen if you don’t want the full provisioning cycle.
- Concerning integration_test:
The README “Kitchenfile DSL” section talks about a features dir
underneath the /test/kitchen dir, but in the examples I see
Also, is the “integration_test” block in the Kitchenfile really
necessary to run the features? From the “Cucumber” section above,
which says “Test Kitchen will run any features in the test/features
subdirectory tagged with the configuration name in order to check that
the service is working as expected.”, it sounds like you wouldn’t need
to do anything extra to make the cucumber features run. In general,
the “integration_test” section really confused me…
The cookbook features will automatically get run by test-kitchen if the directory exists.
The integration_test block is for non-cookbook projects (like mixlib-shellout) that have unit or integration testing. While mixlib-shellout is kind of a contrived example, a better one would be a Rails application that has its own test suite that should be run in a fully deployed scenario with its database, etc.
I don’t get what the cookbook and integration_test runtimes parameter
is good for. Why does RVM need to be installed on the base box? Are
you running the features locally on the target box? I would have
expected that the features are run from the outside (e.g. on my
workstation) against the provisioned target vm (the “black box”), so
why would I care that RVM is installed inside that black box?!?
Rvm was used as a relatively simple way to get specific versions of Ruby installed for performing the tests. Think of a non-cookbook project, like a Ruby library that you want to test on different versions of Ruby (like 1.8.7 vs 1.9.3, or even JRuby). My personal preference for cookbook testing is that we make the RVM part optional, and use the Ruby that is included in Chef on the base boxes already.
I created a ticket to track this:
Joshua Timberman, Technical Program Manager
IRC, Skype, Twitter, Github: jtimberman