I’m new to Chef and trying to grasp the best practices of kitchen. I’ve been observing the community cookbooks practice where in majority it maintains:
My assumption is, kitchen.yml is maintained for workstation development, while kitchen.docker.yml is maintained for Travis CI.
My question is, why don’t we use docker in workstation development as well and just maintain one kitchen configuration?
Getting Docker set up on Mac and Windows machines has historically been more difficult than VirtualBox, whereas most CI infrastructure is already running on Linux. This has improved in the last year or so, but boilerplate changes slowly. Personally I run kitchen-docker for everything but I’m the maintainer so probably a tad biased.
This brings up a related question, on the subject of kitchen-docker… I know that kitchen-docker isn’t included in ChefDK, but I noticed the other day that kitchen-dokken is. I read up a bit about Dokken and it seems like it’s a more Docker focused solution to test-kitchen, where kitchen-docker basically operates the same way that other plugins like EC2 do. I briefly tried converting over to kitchen-dokken, but it requires a bit more in the way of configuration than kitchen-docker does, as it has its own busser and such. Does anyone know of a good conversion guide between the two setups?
Nathan Clemons
DevOps Engineer
Moxie Cloud Services (MCS)
O +1.425.467.5075
M +1.360.861.6291
E nclemons@gomoxie.com
W www.gomoxie.comhttp://www.gomoxie.com/
Historically docker didn’t support centos-5 well either, and probably still doesn’t support freebsd while we’ve got vagrant boxes for that? We also need to test some cookbooks against Windows and Solaris and have private vagrant boxes in vagrant cloud for testing those.
Docker on mac native was also busted with kitchen-dokken up until very recently (there had been a workaround but then they nuked all the tools that let you apply the workaround and simplified the interface in the beta so much that it was just totally broken for me), I see that issue got closed in the past week or two though so I’ve been planning on giving it another shot.
Anyway, maybe we’re getting there, for some cookbooks, but literally up until a week or two ago we had a huge showstopper on macs (which most of Chef, Inc uses internally as workstations, except for some crazy windows peeps).
There’s a few reasons for the separate files:
Most importantly is the fact that people have VirtualBox and Vagrant already installed and it provides a great test environment for local workstation testing. We didn’t want to remove this workflow by switching over to kitchen-dokken entirely. It gives people simple testing of Linux distros and as Lamont mentioned we can also test Windows/FreeBSD/OS X/Solaris with Vagrant, which we can’t do it in Docker.
The second reason is that testing in Docker is difficult and not entirely accurate. Docker images for Ubuntu are just a few hundred megabytes and for platforms like openSUSE they’re barely over 100 megabytes. They achieve this space savings by stripping nearly everything out you wouldn’t need for running a basic app. When testing cookbooks we need a lot of this stripped out functionality. For instance we can’t see if a process is running with inspec if we don’t have procps installed and we can’t see if a port is open without net-tools. That’s just for kitchen verify. For the actual converge we can’t get services to start on openSUSE without first installing perl-Getopt-Long-Descriptive and systemd-sysv / initscripts on Ubuntu. I’ve spent a lot of time carefully crafting kitchen config files for Docker that will actually converge. It’s been worth it for us since we can test our cookbooks with kitchen-dokken in Travis CI. I wouldn’t call it general purpose replacement for Virtualbox / Vagrant though since the time you save on converges is completely eaten up tracking down missing packages.
-Tim
Any info you have on the dokken config you’re using that you can share would be appreciated.
Nathan Clemons
DevOps Engineer
Moxie Cloud Services (MCS)
O +1.425.467.5075
M +1.360.861.6291
E nclemons@gomoxie.com
W www.gomoxie.comhttp://www.gomoxie.com/
Thanks for all of the insights! I’m more enlightened now.
@tas50 Probably this is off topic. On the second reason you gave, the missing packages required for convergence/inspec in particular, is there any plan to consolidate these in chef/bento style? Currently bento only supports Vagrant boxes, probably it is time now to support docker images as well?
I’m going to spend some time to see if I can possibly make a “fat” docker image of each of the platforms we test. It would basically be our existing bento images, but in docker form. Those will obviously be pretty large in comparison to the traditional docker images. If these work as expected, they’ll give us everything we need for run tests in a CI systems like Travis.
As for sharing what I’ve learned, I started writing a blog post that I need to finish up, which documented what I’m doing in the community cookbooks with kitchen-dokken. It will include a breakdown of the packages I had to add to get testing working and what things you can’t test (like iptables). Once I have some free time I’ll try to wrap that up since I think it would be pretty beneficial to those writing community cookbooks.
-Tim
Definitely. And useful for those of us who simply want better testing in our CI toolchain, period.
Nathan Clemons
DevOps Engineer
Moxie Cloud Services (MCS)
O +1.425.467.5075
M +1.360.861.6291
E nclemons@gomoxie.com
W www.gomoxie.comhttp://www.gomoxie.com/