Dynamically include recipes?

Chef Infra (archive)
1

Hi there, list –

I’m trying to figure out a way to do a (relatively) dynamic include_recipe
command. Specifically, the context of this question is a Nagios server, and
building the service definition files.

I would like to be able to do the following:
- Get a list of all cookbooks defined in the chef server
- For each cookbook C, see if that cookbook includes a recipe called
"nagios"
- If the cookbook does include a recipe called “nagios”, then do
include_recipe “#{C}::nagios”

The idea is that when someone within my organization wants to add a chef
cookbook to install/configure a new service or a new application, they can
add the monitoring for that service/application as well – without having to
alter the nagios cookbook.

I had sort of assumed that I could do this using search(), but it looks like
cookbooks/recipes aren’t searchable. Sadness. So I’m wondering if anyone
has suggestions for other ways to make this happen.

(I know that one potential issue is the requirement that metadata.rb also be
updated. I’m not entirely sure how to deal with that, either. I figured I
would worry about one problem at a time…)

Thanks!

  • Ian
2

Ian,

You can set arbitrary attributes on nodes within recipes.

so, you can do something like this:

name "appserver-config-a"
description "appserver-config-a"
run_list [
"recipe[fooservice::client]",
]

Then, in at the top of your cookbooks/fooservice/client.rb :

node.set[:fooservice][:client].

You can use that attribute from your nagios::server recipe to search for:

fooservice_clients = search(:node, "fooservice_client:true")

You have now effectively searched for nodes with the fooservice
cookbook installed.

-s

On Mon, Jan 10, 2011 at 5:58 PM, Ian Marlier imarlier@brightcove.com wrote:

Hi there, list --

I'm trying to figure out a way to do a (relatively) dynamic include_recipe
command. Specifically, the context of this question is a Nagios server, and
building the service definition files.

I would like to be able to do the following:
- Get a list of all cookbooks defined in the chef server
- For each cookbook C, see if that cookbook includes a recipe called
"nagios"
- If the cookbook does include a recipe called "nagios", then do
include_recipe "#{C}::nagios"

The idea is that when someone within my organization wants to add a chef
cookbook to install/configure a new service or a new application, they can
add the monitoring for that service/application as well -- without having to
alter the nagios cookbook.

I had sort of assumed that I could do this using search(), but it looks like
cookbooks/recipes aren't searchable. Sadness. So I'm wondering if anyone
has suggestions for other ways to make this happen.

(I know that one potential issue is the requirement that metadata.rb also be
updated. I'm not entirely sure how to deal with that, either. I figured I
would worry about one problem at a time...)

Thanks!

  • Ian
3

Sean --

Thanks for the reply, but that's a little bit different than what I'm trying
to do. I'm not actually interested (in this case) in a list of nodes with a
given cookbook installed.

Instead, what I'm interested in is a list of cookbooks that contain a
recipe with a given name, so that I can include all of those recipes. In
my use case, those recipes would define a nagios checkcommand and a nagios
service.

What I had hoped to do was this:

cookbooks = search(:cookbooks, "recipe:nagios")
cookbooks.each do |c|
include_recipe "#{c}::nagios"
end

However, the search function doesn't appear to work on cookbooks. I'm
wondering if there's another way that I can do this.

If there's no way to do this within Chef, I can work around it, for example
by doing some bash scripting at cookbook upload time -- though that would be
disappointing, as it would effectively make this recipe only operable in
certain chef server environments. If there's a "real" way, that would be
most excellent.

  • Ian

On Mon, Jan 10, 2011 at 7:48 PM, Sean OMeara someara@gmail.com wrote:

Ian,

You can set arbitrary attributes on nodes within recipes.

so, you can do something like this:

name "appserver-config-a"
description "appserver-config-a"
run_list [
"recipe[fooservice::client]",
]

Then, in at the top of your cookbooks/fooservice/client.rb :

node.set[:fooservice][:client].

You can use that attribute from your nagios::server recipe to search for:

fooservice_clients = search(:node, "fooservice_client:true")

You have now effectively searched for nodes with the fooservice
cookbook installed.

-s

On Mon, Jan 10, 2011 at 5:58 PM, Ian Marlier imarlier@brightcove.com
wrote:

Hi there, list --

I'm trying to figure out a way to do a (relatively) dynamic
include_recipe
command. Specifically, the context of this question is a Nagios server,
and
building the service definition files.

I would like to be able to do the following:
- Get a list of all cookbooks defined in the chef server
- For each cookbook C, see if that cookbook includes a recipe called
"nagios"
- If the cookbook does include a recipe called "nagios", then do
include_recipe "#{C}::nagios"

The idea is that when someone within my organization wants to add a chef
cookbook to install/configure a new service or a new application, they
can
add the monitoring for that service/application as well -- without having
to
alter the nagios cookbook.

I had sort of assumed that I could do this using search(), but it looks
like
cookbooks/recipes aren't searchable. Sadness. So I'm wondering if
anyone
has suggestions for other ways to make this happen.

(I know that one potential issue is the requirement that metadata.rb also
be
updated. I'm not entirely sure how to deal with that, either. I figured
I
would worry about one problem at a time...)

Thanks!

  • Ian
4

On Mon, Jan 10, 2011 at 6:29 PM, Ian Marlier imarlier@brightcove.com wrote:

What I had hoped to do was this:
cookbooks = search(:cookbooks, "recipe:nagios")
cookbooks.each do |c|
include_recipe "#{c}::nagios"
end
However, the search function doesn't appear to work on cookbooks. I'm
wondering if there's another way that I can do this.

You can't do this with search, but you can use the REST API. For
example, knife exec 'pp api.get "cookbooks/_latest" will print a
hash of COOKBOOK_NAME => CONONICAL_URL which you can iterate over to
get the contents of each cookbook. This will pretty print a hash of
COOKBOOK_NAME => [RECIPE] for each cookbook:

knife exec -E '(api.get "cookbooks/_latest").each {|c, url| pp c =>
api.get(url).recipe_filenames}'

You could then filter that for the desired recipes. Note that outside
of shef/knife exec you need to use get_rest instead of just
get--shef aliases that method for brevity.

If there's no way to do this within Chef, I can work around it, for example
by doing some bash scripting at cookbook upload time -- though that would be
disappointing, as it would effectively make this recipe only operable in
certain chef server environments. If there's a "real" way, that would be
most excellent.

Despite the above, if you want to pursue the design you described
previously (having nagios recipes "sprinkled" throughout your cookbook
repo), your best bet is to do some analysis up front and modify your
nagios cookbook accordingly. Even if you can get the list of recipes
you want from the API, you still need their cookbooks to be listed in
the metadata for your nagios cookbook in order to have them synced to
the edge. If you try to do this from within a recipe, you'll need 2
chef runs to properly converge everything.

  • Ian

HTH,
Dan DeLeo