Resource groups

Chef Infra (archive)
1

Hi everybody,

I’m pretty new to Chef and would like assistance for the following problem.
I have a service and several configuration files for it. My other recipes communicate with that service, so it should be configured and ready to use before running them. I have following structure:

service ‘myservice’ do
action :nothing
end

template ‘/path/to/service/config1’ do
…
notifies :restart, resources(:service => ‘myservice’), :immediate
end

template ‘/path/to/service/config2’ do
…
notifies :restart, resource(:service => ‘myservice’), :immediate
end

service ‘myservice’ do
action [:start, :enable]
end

So, I want that if any of two configs change, the service is restarted not later than the second “service” resource. But in configuration that shown above, if both configs change, the service will be restarted two times.

How would you solve this “the Chef-way” ?

After reading through Chef sources I’ve developed following solution. I have a special resource – “resource_group”:

service ‘myservice’ do
action :nothing
end

resource_group ‘myservice_configs’ do
template ‘/path/to/service/config1’ do
…
end

template '/path/to/service/config2' do
  ...
end

notifies :restart, resource(:service => 'myservice'), :immediate

end

service ‘myservice’ do
action [:start, :enable]
end

So, when it is defined, it records all resources (through “method_missing” magic) that are created inside of it. The resource_group resource itself is created after resources in it (that’s how Chef works). One resource_group’s action triggers, it checks embedded resources’ #updated? method and if any resource was updated, it set’s it’s own #updated_by_last_action to true (thus triggering notifications).
What do you think about this solution? Maybe it could make it into Chef core?

Thanks,
Max

2

Why not get rid of immediate suffix ?
In that case all :restart attempts will be grouped and service will be
restarted only once. But not immediate, some where near to the end of
recipe.

On Mon, 7 Nov 2011 21:58:11 +0400, Maxim Kulkin wrote:

Hi everybody,

I'm pretty new to Chef and would like assistance for the following
problem.
I have a service and several configuration files for it. My other
recipes communicate with that service, so it should be configured and
ready to use before running them. I have following structure:

service 'myservice' do
action :nothing
end

template '/path/to/service/config1' do
...
notifies :restart, resources(:service => 'myservice'), :immediate
end

template '/path/to/service/config2' do
...
notifies :restart, resource(:service => 'myservice'), :immediate
end

service 'myservice' do
action [:start, :enable]
end

So, I want that if any of two configs change, the service is
restarted not later than the second "service" resource. But in
configuration that shown above, if both configs change, the service
will be restarted two times.

How would you solve this "the Chef-way" ?

After reading through Chef sources I've developed following solution.
I have a special resource -- "resource_group":

service 'myservice' do
action :nothing
end

resource_group 'myservice_configs' do
template '/path/to/service/config1' do
...
end

template '/path/to/service/config2' do
  ...
end

notifies :restart, resource(:service => 'myservice'), :immediate

end

service 'myservice' do
action [:start, :enable]
end

So, when it is defined, it records all resources (through
"method_missing" magic) that are created inside of it. The
resource_group resource itself is created after resources in it
(that's how Chef works). One resource_group's action triggers, it
checks embedded resources' #updated? method and if any resource was
updated, it set's it's own #updated_by_last_action to true (thus
triggering notifications).
What do you think about this solution? Maybe it could make it into
Chef core?

Thanks,
Max

3

Good example in Apache2 recipe:
https://github.com/opscode/cookbooks/blob/master/apache2/recipes/default.rb

On Mon, 07 Nov 2011 20:01:56 +0200, Dmitry Zamaruev wrote:

Why not get rid of immediate suffix ?
In that case all :restart attempts will be grouped and service will
be restarted only once. But not immediate, some where near to the end
of recipe.

On Mon, 7 Nov 2011 21:58:11 +0400, Maxim Kulkin wrote:

Hi everybody,

I'm pretty new to Chef and would like assistance for the following
problem.
I have a service and several configuration files for it. My other
recipes communicate with that service, so it should be configured
and
ready to use before running them. I have following structure:

service 'myservice' do
action :nothing
end

template '/path/to/service/config1' do
...
notifies :restart, resources(:service => 'myservice'),
:immediate
end

template '/path/to/service/config2' do
...
notifies :restart, resource(:service => 'myservice'), :immediate
end

service 'myservice' do
action [:start, :enable]
end

So, I want that if any of two configs change, the service is
restarted not later than the second "service" resource. But in
configuration that shown above, if both configs change, the service
will be restarted two times.

How would you solve this "the Chef-way" ?

After reading through Chef sources I've developed following
solution.
I have a special resource -- "resource_group":

service 'myservice' do
action :nothing
end

resource_group 'myservice_configs' do
template '/path/to/service/config1' do
...
end

template '/path/to/service/config2' do
  ...
end

notifies :restart, resource(:service => 'myservice'), :immediate

end

service 'myservice' do
action [:start, :enable]
end

So, when it is defined, it records all resources (through
"method_missing" magic) that are created inside of it. The
resource_group resource itself is created after resources in it
(that's how Chef works). One resource_group's action triggers, it
checks embedded resources' #updated? method and if any resource was
updated, it set's it's own #updated_by_last_action to true (thus
triggering notifications).
What do you think about this solution? Maybe it could make it into
Chef core?

Thanks,
Max

4

As I said, other resources use that service, so it needs to be configured immediately. If I make notifications delayed, the restart will be after other recipes get executed and thus resources in those other recipes won't be able to communicate with that service because of it's misconfiguration.
Say, it's a web service and configuration files determine port and credentials to communicate with that service. If those values will change, but the service is not restarted, other recipes won't be able to access it.

On Nov 7, 2011, at 22:01, Dmitry Zamaruev wrote:

Why not get rid of immediate suffix ?
In that case all :restart attempts will be grouped and service will be restarted only once. But not immediate, some where near to the end of recipe.

On Mon, 7 Nov 2011 21:58:11 +0400, Maxim Kulkin wrote:

Hi everybody,

I'm pretty new to Chef and would like assistance for the following problem.
I have a service and several configuration files for it. My other
recipes communicate with that service, so it should be configured and
ready to use before running them. I have following structure:

service 'myservice' do
action :nothing
end

template '/path/to/service/config1' do
...
notifies :restart, resources(:service => 'myservice'), :immediate
end

template '/path/to/service/config2' do
...
notifies :restart, resource(:service => 'myservice'), :immediate
end

service 'myservice' do
action [:start, :enable]
end

So, I want that if any of two configs change, the service is
restarted not later than the second "service" resource. But in
configuration that shown above, if both configs change, the service
will be restarted two times.

How would you solve this "the Chef-way" ?

After reading through Chef sources I've developed following solution.
I have a special resource -- "resource_group":

service 'myservice' do
action :nothing
end

resource_group 'myservice_configs' do
template '/path/to/service/config1' do
...
end

template '/path/to/service/config2' do
...
end

notifies :restart, resource(:service => 'myservice'), :immediate
end

service 'myservice' do
action [:start, :enable]
end

So, when it is defined, it records all resources (through
"method_missing" magic) that are created inside of it. The
resource_group resource itself is created after resources in it
(that's how Chef works). One resource_group's action triggers, it
checks embedded resources' #updated? method and if any resource was
updated, it set's it's own #updated_by_last_action to true (thus
triggering notifications).
What do you think about this solution? Maybe it could make it into Chef core?

Thanks,
Max

5

I use following workaround:

ruby_block "service_changed" do
block
node["service_changed"] = true
node.save
end
end

ruby_block "service_restarted" do
block
node.delete("service_changed")
node.save
end
end

template "1" do
notifies :create, "ruby_block[service_changed]", :immediately
end

template "2" do
notifies :create, "ruby_block[service_changed]", :immediately
end

service "my_service" do
action :restart
only_if { node.attribute?("service_changed") }
notifies :create, "ruby_block[service_restarted]"
end

As soon as control got to last service entry - it is restarted only if
templates changed.
If there would be native approach in Chef - it would be very nice.

On Mon, 7 Nov 2011 22:18:30 +0400, Maxim Kulkin wrote:

As I said, other resources use that service, so it needs to be
configured immediately. If I make notifications delayed, the restart
will be after other recipes get executed and thus resources in those
other recipes won't be able to communicate with that service because
of it's misconfiguration.
Say, it's a web service and configuration files determine port and
credentials to communicate with that service. If those values will
change, but the service is not restarted, other recipes won't be able
to access it.

On Nov 7, 2011, at 22:01, Dmitry Zamaruev wrote:

Why not get rid of immediate suffix ?
In that case all :restart attempts will be grouped and service will
be restarted only once. But not immediate, some where near to the end
of recipe.

On Mon, 7 Nov 2011 21:58:11 +0400, Maxim Kulkin wrote:

Hi everybody,

I'm pretty new to Chef and would like assistance for the following
problem.
I have a service and several configuration files for it. My other
recipes communicate with that service, so it should be configured
and
ready to use before running them. I have following structure:

service 'myservice' do
action :nothing
end

template '/path/to/service/config1' do
...
notifies :restart, resources(:service => 'myservice'),
:immediate
end

template '/path/to/service/config2' do
...
notifies :restart, resource(:service => 'myservice'), :immediate
end

service 'myservice' do
action [:start, :enable]
end

So, I want that if any of two configs change, the service is
restarted not later than the second "service" resource. But in
configuration that shown above, if both configs change, the service
will be restarted two times.

How would you solve this "the Chef-way" ?

After reading through Chef sources I've developed following
solution.
I have a special resource -- "resource_group":

service 'myservice' do
action :nothing
end

resource_group 'myservice_configs' do
template '/path/to/service/config1' do
...
end

template '/path/to/service/config2' do
...
end

notifies :restart, resource(:service => 'myservice'), :immediate
end

service 'myservice' do
action [:start, :enable]
end

So, when it is defined, it records all resources (through
"method_missing" magic) that are created inside of it. The
resource_group resource itself is created after resources in it
(that's how Chef works). One resource_group's action triggers, it
checks embedded resources' #updated? method and if any resource
was
updated, it set's it's own #updated_by_last_action to true (thus
triggering notifications).
What do you think about this solution? Maybe it could make it into
Chef core?

Thanks,
Max

6

Yes, I'm also aware of ability to pass data through node attributes, but this approach seems too heavy weight to me. I was experimenting with it and here is another solution that I've found:

service 'myservice' do
action :nothing
end

template '/path/to/service/config1' do
...
end

template '/path/to/service/config2' do
...
end

service 'myservice' do
action [:start, :enable]
end

service 'myservice' do
action :restart
only_if {
resources(:template => ['/path/to/service/config1', '/path/to/service/config2']).any?(&:updated?)
}
end

Alternatively, you can collect resources into an array during their creation:

myservice_configs =
myservice_configs << template('/path/to/service/config1') do
...
end
myservice_configs << template('/path/to/service/config2') do
...
end

service 'myservice' do
action :restart
only_if { myservice_configs.any?(&:updated?) }
end

And here is my solution with resource groups that I've mentioned earlier. You can put it into some cookbook's "libraries/resource_group.rb" file:

https://gist.github.com/1347238

On Nov 7, 2011, at 22:29, Dmitry Zamaruev wrote:

I use following workaround:

ruby_block "service_changed" do
block
node["service_changed"] = true
node.save
end
end

ruby_block "service_restarted" do
block
node.delete("service_changed")
node.save
end
end

template "1" do
notifies :create, "ruby_block[service_changed]", :immediately
end

template "2" do
notifies :create, "ruby_block[service_changed]", :immediately
end

service "my_service" do
action :restart
only_if { node.attribute?("service_changed") }
notifies :create, "ruby_block[service_restarted]"
end

As soon as control got to last service entry - it is restarted only if templates changed.
If there would be native approach in Chef - it would be very nice.

On Mon, 7 Nov 2011 22:18:30 +0400, Maxim Kulkin wrote:

As I said, other resources use that service, so it needs to be
configured immediately. If I make notifications delayed, the restart
will be after other recipes get executed and thus resources in those
other recipes won't be able to communicate with that service because
of it's misconfiguration.
Say, it's a web service and configuration files determine port and
credentials to communicate with that service. If those values will
change, but the service is not restarted, other recipes won't be able
to access it.

On Nov 7, 2011, at 22:01, Dmitry Zamaruev wrote:

Why not get rid of immediate suffix ?
In that case all :restart attempts will be grouped and service will be restarted only once. But not immediate, some where near to the end of recipe.

On Mon, 7 Nov 2011 21:58:11 +0400, Maxim Kulkin wrote:

Hi everybody,

I'm pretty new to Chef and would like assistance for the following problem.
I have a service and several configuration files for it. My other
recipes communicate with that service, so it should be configured and
ready to use before running them. I have following structure:

service 'myservice' do
action :nothing
end

template '/path/to/service/config1' do
...
notifies :restart, resources(:service => 'myservice'), :immediate
end

template '/path/to/service/config2' do
...
notifies :restart, resource(:service => 'myservice'), :immediate
end

service 'myservice' do
action [:start, :enable]
end

So, I want that if any of two configs change, the service is
restarted not later than the second "service" resource. But in
configuration that shown above, if both configs change, the service
will be restarted two times.

How would you solve this "the Chef-way" ?

After reading through Chef sources I've developed following solution.
I have a special resource -- "resource_group":

service 'myservice' do
action :nothing
end

resource_group 'myservice_configs' do
template '/path/to/service/config1' do
...
end

template '/path/to/service/config2' do
...
end

notifies :restart, resource(:service => 'myservice'), :immediate
end

service 'myservice' do
action [:start, :enable]
end

So, when it is defined, it records all resources (through
"method_missing" magic) that are created inside of it. The
resource_group resource itself is created after resources in it
(that's how Chef works). One resource_group's action triggers, it
checks embedded resources' #updated? method and if any resource was
updated, it set's it's own #updated_by_last_action to true (thus
triggering notifications).
What do you think about this solution? Maybe it could make it into Chef core?

Thanks,
Max

7

On Monday, November 7, 2011 at 9:58 AM, Maxim Kulkin wrote:

Hi everybody,

After reading through Chef sources I've developed following solution. I have a special resource -- "resource_group":

What do you think about this solution? Maybe it could make it into Chef core?

Thanks,
Max

I replied over on the chef-dev list.

--
Dan DeLeo

8

On Monday, November 7, 2011 at 9:58 AM, Maxim Kulkin wrote:

Hi everybody,

I'm pretty new to Chef and would like assistance for the following problem.
I have a service and several configuration files for it. My other recipes communicate with that service, so it should be configured and ready to use before running them. I have following structure:

service 'myservice' do
action :nothing
end

template '/path/to/service/config1' do
...
notifies :restart, resources(:service => 'myservice'), :immediate
end

template '/path/to/service/config2' do
...
notifies :restart, resource(:service => 'myservice'), :immediate
end

service 'myservice' do
action [:start, :enable]
end

So, I want that if any of two configs change, the service is restarted not later than the second "service" resource. But in configuration that shown above, if both configs change, the service will be restarted two times.

How would you solve this "the Chef-way" ?

After reading through Chef sources I've developed following solution. I have a special resource -- "resource_group":

service 'myservice' do
action :nothing
end

resource_group 'myservice_configs' do
template '/path/to/service/config1' do
...
end

template '/path/to/service/config2' do
...
end

notifies :restart, resource(:service => 'myservice'), :immediate
end

service 'myservice' do
action [:start, :enable]
end

So, when it is defined, it records all resources (through "method_missing" magic) that are created inside of it. The resource_group resource itself is created after resources in it (that's how Chef works). One resource_group's action triggers, it checks embedded resources' #updated? method and if any resource was updated, it set's it's own #updated_by_last_action to true (thus triggering notifications).
What do you think about this solution? Maybe it could make it into Chef core?

Thanks,
Max

This is the same solution I came up with to the problem: http://tickets.opscode.com/browse/CHEF-2452

I think resource groups are actually useful for several things:

  • better control over notifications
  • failure zones: allow a group to fail without stopping the Chef run
  • customizing recipes: include_recipe could be extended so that only certain groups from a recipe are included. Also, groups could be overridden by a user of the recipe. Think of replacing the "install" group with resources to install a package from source.

In order to accomplish all of these, resource groups would have to be integrated pretty deeply into Chef's resource collection. But there's no reason we can't add groups now and iterate. We do need to be careful not to accidentally leak implementation that then becomes an API that users want to rely on.

--
Dan DeLeo