I've been running a few of our cookbooks through Jenkins with Kitchen
and Docker, since first of January. I had to use either Docker, or LXC,
because my Jenkins build host was already a VMWare VM and getting nested
virtualization working with something like Vagrant/Virtualbox wasn't all
that straightforward.
I broke down the build jobs up into 3 stages, quick tests, integration
tests, and deploy.
Stage 1: Unit and linting tests, with Chefspec, Rubocop, and Foodcritic,
which all takes about 1 minute or less. First stage is triggered by our
Gitlab server, or upstream dependencies.
Stage 2: Use kitchen.ci and kitchen-docker to spin up baseboxes
generated with rinse https://index.docker.io/u/atomicpenguin/centos/
and then run BATS integration tests against those machines. Some of the
cookbooks are EL5 and EL6 which take about 6-8 minutes to test both.
Some just need to be tested with EL6 which take about half the time, 3-4
minutes.
Stage 3: do a berks upload, and berks apply production. If there are
data bags in the cookbook, then I test with chef-zero in Stage 2, and do
a rake databag:upload_all on Stage 3.
If there are downstream dependencies like a role cookbook which depends
on an upstream baseline role cookbook. Then the upstream cookbook
triggers a 3-stage build on downstream cookbooks.
If you want to validate branching behaviour such as in your org-java
cookbook, then a good way to test a lot of corner cases is probably
Chefspec. Kitchen works great doing just a few branching platforms and
automatically integration testing those cases. As you add more
platforms to Kitchen, two things tend to happen. Your integration tests
become more complex, and your build time grows. If your integration
tests for all your combined platforms for a given cookbook don't fit in
a ten to fifteen minute window, then unit tests with Chefspec is
probably going to provide more value and quicker feedback.
Eric G. Wolfe
Senior Linux Administrator,
IT Infrastructure Systems
Marshall University Computing Services
Drinko Library 428-K
One John Marshall Dr.
Huntington, WV 25755
Phone: 304.942.3970
Email: eric.wolfe@marshall.edu
The worst sin towards our fellow creatures is not to hate them,
but to be indifferent to them; that's the essence of inhumanity.
-- G.B. Shaw
On 03/27/2014 07:18 PM, Christopher Armstrong wrote:
Ohai Chefs,
We have rubocop and foodcritic configured to run on Travis CI to
automatically check our pull requests, which is awesome. I verify each
build with test-kitchen before I cut a new cookbook release, but this
is getting very time consuming to run manually each time. For the java
cookbook, testing different java versions on various platforms results
in a ton of individual runs. I'd love to set up a system to
automatically run test-kitchen and then give me the red/green light.
Has anyone tried doing this and could recommend some tools/workflows?
Thanks,
Chris