Chef Server

Chef Infra (archive)
1

Hey guys.

As you may have guessed I am a noob trying to get chef up and going
for my infrastructure. I have some puppet experience but I wanted a
more “just ruby” type of system so I am giving chef a go. I also used
automateit briefly and liked it quite a bit but it’s abandoned so I
decided against it.

I just wanted to relay some thought from a beginner in hope that this
can give some insight into what the experience is like for a person
starting out. I hope you don’t take any of this as criticism, just
some musings that I hope might trigger discussion.

I tried making things work just chef solo for about a week but I think
I have come to the conclusion that I need a chef server. A lot of the
recipes presume a server and some things don’t work with out it.

There are so many moving parts in chef that it’s kind of mind
boggling. Runit, solr, rabbitmq, merb (merb!!??), couchdb, chef,
ohai, knife, and other addons like knife-solo (which needs python),
librarian etc. I am sure there is a good reason for all this but it’s
a bit intimidating to have to manage and learn all of these things.

The bootstrap commands seem to ignore the fact you may have rubygems
already installed and want to download one anyway. I wasn’t expecting
that and ended up writing my own bootstrap script.

Knife, chef-client, chef-solo all need different config files and some
items are repeated. I would prefer just one config file with all the
options in it.

It would be nice if I could define my nodes in ruby like I define all
my recipes and roles. Same goes for databags. I also prefer yaml to
json but that’s just me.

The wiki is pretty nice, thanks to whoever is maintaining that.

You guys have been most helpful on the mailing list. Thanks for your patience.

2

Hi Tim,

what you describe seems to be LittleChef's way :wink:

There are basically no moving parts: you have your project's directory
(kitchen) with your cookbooks, roles and even nodes and databags,
which as you put it are just (JSON) files.

Have a look if you are interested: GitHub - tobami/littlechef: Cook with Chef without a Chef Server

Cheers,
Miquel

2012/1/2 Tim Uckun timuckun@gmail.com:

Hey guys.

As you may have guessed I am a noob trying to get chef up and going
for my infrastructure. I have some puppet experience but I wanted a
more "just ruby" type of system so I am giving chef a go. I also used
automateit briefly and liked it quite a bit but it's abandoned so I
decided against it.

I just wanted to relay some thought from a beginner in hope that this
can give some insight into what the experience is like for a person
starting out. I hope you don't take any of this as criticism, just
some musings that I hope might trigger discussion.

I tried making things work just chef solo for about a week but I think
I have come to the conclusion that I need a chef server. A lot of the
recipes presume a server and some things don't work with out it.

There are so many moving parts in chef that it's kind of mind
boggling. Runit, solr, rabbitmq, merb (merb!!??), couchdb, chef,
ohai, knife, and other addons like knife-solo (which needs python),
librarian etc. I am sure there is a good reason for all this but it's
a bit intimidating to have to manage and learn all of these things.

The bootstrap commands seem to ignore the fact you may have rubygems
already installed and want to download one anyway. I wasn't expecting
that and ended up writing my own bootstrap script.

Knife, chef-client, chef-solo all need different config files and some
items are repeated. I would prefer just one config file with all the
options in it.

It would be nice if I could define my nodes in ruby like I define all
my recipes and roles. Same goes for databags. I also prefer yaml to
json but that's just me.

The wiki is pretty nice, thanks to whoever is maintaining that.

You guys have been most helpful on the mailing list. Thanks for your patience.

3

what you describe seems to be LittleChef's way :wink:

There are basically no moving parts: you have your project's directory
(kitchen) with your cookbooks, roles and even nodes and databags,
which as you put it are just (JSON) files.

Have a look if you are interested: GitHub - tobami/littlechef: Cook with Chef without a Chef Server

I have set up something similar myself. I have a repo and a boot
script which puts the proper keys in proper places and pulls the repo
down, then I run chef-solo against that directory structure (kitchen).
I got the idea from pocketknife which from what I can gather is
inspired by littlechef but is written in ruby. What I don't have is
the search and databag functionality as they look for a server.

Is there a way I can just add the search and databag functionality
into my own project without adopting littlechef?

4

Hello!

On Mon, Jan 2, 2012 at 3:49 PM, Tim Uckun timuckun@gmail.com wrote:

There are so many moving parts in chef that it's kind of mind
boggling. Runit, solr, rabbitmq, merb (merb!!??), couchdb, chef,
ohai, knife, and other addons like knife-solo (which needs python),
librarian etc. I am sure there is a good reason for all this but it's
a bit intimidating to have to manage and learn all of these things.

Just to clarify a couple things about the choices in this stack:

  1. Runit is not required, it is only used by default on Debian/Ubuntu
    because it is more robust for service management than plain ol' init
    scripts. After learning more about it, I find it far more simple than
    any other service/daemon management tool.

http://smarden.org/runit

  1. Merb was used because the Chef Server API and WebUI were written
    well before Rails 3 or Sinatra were available. However, on Opscode
    Hosted Chef, we're moving API endpoints over to Erlang based services,
    and this will come to the Open Source Chef Server at some point
    (mentioned in our community summit in November, but release date is
    not set - we're still moving internal services first0).

  2. There is definitely a learning curve when considering additional
    tools and plugins from the community. Chef is intended to provide you
    with primitives you can use to build the configuration management and
    system integration system that fits your needs best, and comes with a
    number of tools and plugins that you can use right away.

The bootstrap commands seem to ignore the fact you may have rubygems
already installed and want to download one anyway. I wasn't expecting
that and ended up writing my own bootstrap script.

Indeed. If you're doing Ruby/RubyGems installation in your base OS
installation or image, you'll probably need to account for that by a
custom bootstrap template.

Knife, chef-client, chef-solo all need different config files and some
items are repeated. I would prefer just one config file with all the
options in it.

You can specify the configuration file to use with any of the commands
with the -c option.

knife -c /etc/chef/client.rb

For example. You should read the configuration settings page for more
information about how the various configuration options matter by
context of the tool they're used in.

http://wiki.opscode.com/display/chef/Chef+Configuration+Settings

It would be nice if I could define my nodes in ruby like I define all
my recipes and roles. Same goes for databags. I also prefer yaml to
json but that's just me.

I believe you can, actually, use Ruby rather than JSON for node
documents in the Chef Repository.

The class used is Chef::knife::Core::ObjectLoader

https://github.com/opscode/chef/blob/master/chef/lib/chef/knife/core/object_loader.rb

The wiki is pretty nice, thanks to whoever is maintaining that.

Our support engineering team is responsible for the updates and
content. We have some plans for further awesomeness coming soon.

--
Opscode, Inc
Joshua Timberman, Technical Program Manager
IRC, Skype, Twitter, Github: jtimberman

5

To be clear, the Web UI won't be re-built in Erlang (that would.. um.. hurt) but likely will move to Rails.

Adam

Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com

On Jan 3, 2012, at 7:21 PM, Joshua Timberman wrote:

Hello!

On Mon, Jan 2, 2012 at 3:49 PM, Tim Uckun timuckun@gmail.com wrote:

There are so many moving parts in chef that it's kind of mind
boggling. Runit, solr, rabbitmq, merb (merb!!??), couchdb, chef,
ohai, knife, and other addons like knife-solo (which needs python),
librarian etc. I am sure there is a good reason for all this but it's
a bit intimidating to have to manage and learn all of these things.

Just to clarify a couple things about the choices in this stack:

  1. Runit is not required, it is only used by default on Debian/Ubuntu
    because it is more robust for service management than plain ol' init
    scripts. After learning more about it, I find it far more simple than
    any other service/daemon management tool.

http://smarden.org/runit

  1. Merb was used because the Chef Server API and WebUI were written
    well before Rails 3 or Sinatra were available. However, on Opscode
    Hosted Chef, we're moving API endpoints over to Erlang based services,
    and this will come to the Open Source Chef Server at some point
    (mentioned in our community summit in November, but release date is
    not set - we're still moving internal services first0).

  2. There is definitely a learning curve when considering additional
    tools and plugins from the community. Chef is intended to provide you
    with primitives you can use to build the configuration management and
    system integration system that fits your needs best, and comes with a
    number of tools and plugins that you can use right away.

The bootstrap commands seem to ignore the fact you may have rubygems
already installed and want to download one anyway. I wasn't expecting
that and ended up writing my own bootstrap script.

Indeed. If you're doing Ruby/RubyGems installation in your base OS
installation or image, you'll probably need to account for that by a
custom bootstrap template.

Knife, chef-client, chef-solo all need different config files and some
items are repeated. I would prefer just one config file with all the
options in it.

You can specify the configuration file to use with any of the commands
with the -c option.

knife -c /etc/chef/client.rb

For example. You should read the configuration settings page for more
information about how the various configuration options matter by
context of the tool they're used in.

http://wiki.opscode.com/display/chef/Chef+Configuration+Settings

It would be nice if I could define my nodes in ruby like I define all
my recipes and roles. Same goes for databags. I also prefer yaml to
json but that's just me.

I believe you can, actually, use Ruby rather than JSON for node
documents in the Chef Repository.

The class used is Chef::knife::Core::ObjectLoader

https://github.com/opscode/chef/blob/master/chef/lib/chef/knife/core/object_loader.rb

The wiki is pretty nice, thanks to whoever is maintaining that.

Our support engineering team is responsible for the updates and
content. We have some plans for further awesomeness coming soon.

--
Opscode, Inc
Joshua Timberman, Technical Program Manager
IRC, Skype, Twitter, Github: jtimberman

6

Hi Tim,

there databag functionality for Chef Solo since Chef 0.10.4
As to databag search, we developed that at edelight as a separate
cookbook library:

You just need to add the chef-solo-search library to your cookbooks
repository, and the functionality will be available to Chef Solo.
We should eventually integrate the library with Chef Solo proper anyway.

Miquel

2012/1/4 Adam Jacob adam@opscode.com:

To be clear, the Web UI won't be re-built in Erlang (that would.. um.. hurt) but likely will move to Rails.

Adam

Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com

On Jan 3, 2012, at 7:21 PM, Joshua Timberman wrote:

Hello!

On Mon, Jan 2, 2012 at 3:49 PM, Tim Uckun timuckun@gmail.com wrote:

There are so many moving parts in chef that it's kind of mind
boggling. Runit, solr, rabbitmq, merb (merb!!??), couchdb, chef,
ohai, knife, and other addons like knife-solo (which needs python),
librarian etc. I am sure there is a good reason for all this but it's
a bit intimidating to have to manage and learn all of these things.

Just to clarify a couple things about the choices in this stack:

  1. Runit is not required, it is only used by default on Debian/Ubuntu
    because it is more robust for service management than plain ol' init
    scripts. After learning more about it, I find it far more simple than
    any other service/daemon management tool.

http://smarden.org/runit

  1. Merb was used because the Chef Server API and WebUI were written
    well before Rails 3 or Sinatra were available. However, on Opscode
    Hosted Chef, we're moving API endpoints over to Erlang based services,
    and this will come to the Open Source Chef Server at some point
    (mentioned in our community summit in November, but release date is
    not set - we're still moving internal services first0).

  2. There is definitely a learning curve when considering additional
    tools and plugins from the community. Chef is intended to provide you
    with primitives you can use to build the configuration management and
    system integration system that fits your needs best, and comes with a
    number of tools and plugins that you can use right away.

The bootstrap commands seem to ignore the fact you may have rubygems
already installed and want to download one anyway. I wasn't expecting
that and ended up writing my own bootstrap script.

Indeed. If you're doing Ruby/RubyGems installation in your base OS
installation or image, you'll probably need to account for that by a
custom bootstrap template.

Knife, chef-client, chef-solo all need different config files and some
items are repeated. I would prefer just one config file with all the
options in it.

You can specify the configuration file to use with any of the commands
with the -c option.

knife -c /etc/chef/client.rb

For example. You should read the configuration settings page for more
information about how the various configuration options matter by
context of the tool they're used in.

http://wiki.opscode.com/display/chef/Chef+Configuration+Settings

It would be nice if I could define my nodes in ruby like I define all
my recipes and roles. Same goes for databags. I also prefer yaml to
json but that's just me.

I believe you can, actually, use Ruby rather than JSON for node
documents in the Chef Repository.

The class used is Chef::knife::Core::ObjectLoader

https://github.com/opscode/chef/blob/master/chef/lib/chef/knife/core/object_loader.rb

The wiki is pretty nice, thanks to whoever is maintaining that.

Our support engineering team is responsible for the updates and
content. We have some plans for further awesomeness coming soon.

--
Opscode, Inc
Joshua Timberman, Technical Program Manager
IRC, Skype, Twitter, Github: jtimberman

7

On Wed, Jan 4, 2012 at 5:30 PM, Adam Jacob adam@opscode.com wrote:

To be clear, the Web UI won't be re-built in Erlang (that would.. um.. hurt) but likely will move to Rails.

That all sounds pretty awesome. I was thinking the whole thing could
be done with something like torquebox which gives you clustering,
background tasks, message queues etc but erlang sounds pretty great
too.

Looking forward it.

8

You just need to add the chef-solo-search library to your cookbooks
repository, and the functionality will be available to Chef Solo.
We should eventually integrate the library with Chef Solo proper anyway.

Thanks Miguel. That's very handy.

Cheers.