+1 for using Chef Solo
I guess we have the same motivation: avoid the additional complexity of
Chef Server
unless you really need it!
For myself I have found a sweet spot using Chef Solo, Vagrant and
Berkshelf, which,
apart from the Chef Solo bit, is quite different from your approach, but
probably worth
sharing as well.
The key points to make it play nicely together are:
- use single-repo-per-cookbook rather than a monolithic chef-repo [0]
- pull in cookbook dependencies via Berkshelf / Berksfile [1]
- use application cookbooks [2] rather than roles
- differentiate between cookbook repositories [3] vs. infrastructure
repositories [4]
A Vagrant-based infrastructure repository yields some really nice
properties:
- different providers: deploy your infrastructure on virtualbox, lxc, aws,
rackspace, anywhere...
- use the same configuration mechanism (Vagrantfile) no matter where you
deploy
- use the same CLI (vagrant up) no matter where you deploy
- use the same plugin ecosystem (vagrant-omnibus, vagrant-berkshelf,
vagrant-cachier, etc..) no matter where you deploy
- define the infrastructure-specific environments, data_bags and node
configuration where they belong to
This approach works really well if you have a static environment where you
can still name
all of your machines (i.e. no elasticity / autoscaling). In this case Chef
Solo + Vagrant
seem to be a perfect fit.
If you have something more dynamic with autoscaling etc. you probably need
Chef Server
and search. There's some overhead involved in running an additional piece
of infrastructure,
but also the workflow gets slightly more complicated when using Chef Server.
Happy to hear what others think about it!
Refs:
[0]
[1] Redirecting…
[2]
devopsanywhere: How to Write Reusable Chef Cookbooks, Gangnam Style
[3] GitHub - tknerr/sample-toplevel-cookbook: A sample "top-level" cookbook hosting a website
[4]
sample-infrastructure-repo/Vagrantfile at master · tknerr/sample-infrastructure-repo · GitHub
On Wed, Oct 16, 2013 at 10:33 PM, Igor Serebryany <
igor.serebryany@airbnb.com> wrote:
Sander, at the time, we googled around and learned about additional
tooling which might have been able to help. but both figuring out the
tooling AND getting it going on CI AND teaching it to the rest of our team
was yak-shaving. our task was to get our chaotic legacy infrastructure on
chef, as quickly as possible, and what we ended up doing was extremely
light-weight.
even today, i'm not convinced that the berkshelf way has any advantages
over our approach. i think that what we're doing is still much easier to
operate, and much easier to teach. for developing and testing cookbooks,
our approach involves no additional tooling beyond just git. and as far as
ops work,stemcell https://github.com/airbnb/stemcell and opticahttps://github.com/airbnb/optica neatly
replace knife and chef-server, and arguably do a better job. (stemcell,
because of how it keeps instance metadata in the role files, and optica
because of how flexible and easy to use it is).
actually, i think the biggest drawback to what we're doing is in community
integration; i have a few cookbooks i'd like to share, but (a) i'd have to
break them out into separate repos to do so and (b) i'd have to start doing
all that manual versioning i've been lucky to get to ignore. also, our
approach invites people to just edit upstream cookbooks because they're in
our repo. once or twice so far, we've had to back-port our changes to newer
versions of, say, the runit cookbookhttps://github.com/opscode-cookbooks/runit (this
was complicated when that cookbook was basically completely rewritten as
resource/provider libraries).
--igor
On Wed, Oct 16, 2013 at 11:31 AM, Sander van Zoest sander@vanzoest.comwrote:
Hi Igor,
When you say that the chef server approach was not scaling, did you
consider using "the berkshelf way" approach and having the cookbook build
and uploaded via CI server? If so, what issue did you run into that the
berkshelf way did not solve?
Best,
-- Sander
On Tue, Oct 15, 2013 at 3:55 PM, Igor Serebryany <
igor.serebryany@airbnb.com> wrote:
Hi Chefs!
We just put up a blog post which details our tooling and workflow around
Chef Solo at Airbnb. I thought I'd let the list know, it's here:
Making Breakfast: Chef at Airbnb | by AirbnbEng | The Airbnb Tech Blog | Medium
Along with the post, we open-sourced several tools, including stemcell (
GitHub - airbnb/stemcell: Airbnb's EC2 instance creation and bootstrapping tool) which we use to launch and
bootstrap instances and optica (GitHub - airbnb/optica: A tool for keeping track of nodes in your infrastructure) which
we use to keep track of our instances and their last converge state.
I'm happy to answer questions about our approach here if anyone is
interested.
--Igor
--
Sander van Zoest
sander@vanzoest.com
San Diego, CA, USA
http://Sander.vanZoest.com/