thanks, that clarifies a lot!
I don’t quite get what you mean with environment though (maybe I’m
confused with Chef environments :-)) but I guess you mean the Chef
Server and which Recipes it knows, right?
For me there is value in both, testing individual cookbooks as well as
their composition (roles, node’s run_list). For individual cookbooks,
I’d test on all levels like foodcritic, chefspec,
minitest-chef-handler and feature tests. For any composition of
cookbooks I’d be more interested in the net outcome, i.e. cucumber
features tests against a machine (remote). Don’t know if spec tests
would make sense at that level.
Btw: I was wondering why cucumber features should be run on a
machine (locally), but it seems to be quite often used in that way
(e.g. test-kitchen does that as well). I always thought of
cucumber/gherkin as a tool that is especially well suited for
acceptance-level testing, which IMHO should always be outside-in.
Would be interested to hear the community’s thoughts about that…
Anyways, Cuken vs. cucumber-nagios is now much clearer to me. It’s
interesting to see that current master of cucumber-nagios is now
re-using the Cuken steps, so with the next gem release you will
probably get Cuken as well when you use cucumber-nagios
Thanks a lot,
On Thu, Aug 30, 2012 at 1:56 AM, Hedge Hog email@example.com wrote:
On Wed, Aug 29, 2012 at 12:41 AM, Torben Knerr firstname.lastname@example.org wrote:
I’m currently struggling with writing cucumber features for my
recipes, and I’m not sure whether Cuken or cucumber-nagios would
be the “better” library.
Cuken grew out of cucumber-nagios, and was inspired by Aruba.
Cucken aims to cover more than nagios use cases.
While I have contributed to both, I wouldn’t be willing to opine on
which is better.
We currently use cuken.
Can you recommend either of them, or do you have examples of cookbooks
being tested with them?
Cucken is not aimed at the test/spec or TDD/BDD of individual cookbooks.
To my mind testing/describing specific cookbooks starts to resemble
Rather, Cuken is intended to help with describing what you have/want
after Chef has done its thing.
This can mean features being run on the machine (local), but
generally, this means features being run against a machine (remote).
This approach also means you will generally spec/test the net result
of more than one cookbook/recipe.
To my mind this makes sense because there is no (practical/real)
concept in chef of having the result of exactly one cookbook - all
chef runs tend to be the net result of having run several cookbooks.
Of course you are not limited to using cuken in this way.
In fact the zenoss example (see the relishapp representation)
illustrates a description of a local development environment, as well
as some VMs.
The proof-of-concept was to see if we could get to the point of
editing one source (the feature files) to describe the dev (or prod)
environment as well as the actual servers.
This seems to work for us. We now:
- build a couple of AMIs from base images (client/server kernel specifics)
- rebundle these AMIs as our base images
- build custom OS specific images from the rebaundled sys-base images
- rebundle these AMIs as our server/client images
- test the Chef configuration results.
All done automatically, with the code described under our features folder.
Note that we use Librarian-Chef and our cookbook recipes are not
currently ‘described’ using cuken, although we could do this just as
easily as we did with Vangrant’s files.
The reason for this is that recipe changes can be rapid and, as
mentioned, the results are rarely atomic.
We may yet expand cuken’s scope to our production recipes - which are
However, I’m currently of the view that cuken’s sweetspot is (yes, I
should update the project description):
- pre system-convergence (describing the environment of a chef run: dev/prod)
- post system-convergence (describing the results of a chef run: web
page availability, security tests)
This means we don’t spec cookbooks per se.
a) the environment in which cookbooks run and
b) the results of running those cookboooks.
Hope that helps clarify?
πόλλ’ οἶδ ἀλώπηξ, ἀλλ’ ἐχῖνος ἓν μέγα
[The fox knows many things, but the hedgehog knows one big thing.]
Archilochus, Greek poet (c. 680 BC – c. 645 BC)