I noticed that Bryan Berry suggested “cookbook testing” as a topic for the
community summit.
"Cookbook testing and how to automate it, esp. for opscode/cookbooks"
http://wiki.opscode.com/pages/diffpagesbyversion.action?pageId=20414482&selectedPageVersions=3&selectedPageVersions=2
I’m also interested in that topic. As someone responsible for a
relatively small number of nodes, testability (and the prerequisite
repeatability) have been my biggest benefits from Chef.
So far, what we’ve done for testing is to use rspec for implementing
tests. Here’s an example:
it "should respond on port 80" do
lambda {
TCPSocket.open(@server, 'http')
}.should_not raise_error
end
Before running the tests, I have to manually bootstrap a node using knife.
If my instance is the only one in its environment, the spec can find it
using knife’s search feature. The bootstrap takes a few minutes, and the
20 or so tests take about half a minute to run.
While I’m iteratively developing a recipe, my work cycle is to edit
source, upload a cookbook, and rerun chef-client (usually by rerunning
knife boostrap, because the execution environment is different from
invoking chef-client directly). This feels a bit slower than the cycle
I’m used to when coding in Ruby because of the upload and bootstrap steps.
I like rspec over other testing tools because of how it generates handy
reports, such as this one, which displays an English list of covered test
cases:
$ rspec spec/ -f doc
Foo role
should respond on port 80
should run memcached
should accept memcached connections
should have mysql account
should allow passwordless sudo to user foo as user bar
should allow passwordless sudo to root as a member of sysadmin
should allow key login as user bar
should mount homedirs on ext4, not NFS
should rotate production.log
should have baz as default vhost
...
That sample report also gives a feel for sort of things we check. So far,
nearly all of our checks are non-intrusive enough to run on a production
system. (The exception is testing of local email delivery
configurations.)
Areas I’d love to see improvement:
* Shortening the edit-upload-bootstrap-test cycle
* Automating the bootstrap in the context of testing
* Adding rspec primitives for Chef-related testing, which might
include support for multiple platforms
As an example of rspec primitives, instead of:
it "should respond on port 80" do
lambda {
TCPSocket.open(@server, 'http')
}.should_not raise_error
end
I’d like to write:
it { should respond_on_port(80) }
Rspec supports the the syntactic sugar; it’s just a matter of adding some
"matcher" plugins.
How do other chef users verify that recipes work as expected?
I’m not sure how applicable my approach is to opscode/cookbooks because
it relies on having a specific server configuration and can only test a
cookbook in the context of that single server. If we automated the
boostrap step so it could be embedded into the rspec setup blocks, it
would be possible to test a cookbook in several sample contexts, but the
time required to setup each server instance might be prohibitive.
Marcel