Sending notifications to resources that don't exist yet

Chef Infra (archive)
1

Is it possible to send a notification to a resource if the resource hasn’t
been created yet? When I try this it blows up complaining that the
resource is unknown. But I don’t see any other way to do what I need to.

My application needs a server-specific configuration file, which I create
with a template. The app needs this when it deploys, so I create the
template (in /srv/app/shared/) before the app resource, and then symlink it
in before deploy. This basically works.

But the config file itself is templated with attributes. When those
attributes change, I want to restart the application. But I can’t send a
notification because the config file needs to be defined before the app.

I tried defining the template inside the application’s before_deploy
callback, but that doesn’t seem to do anything. This kind of circular
dependency problem has to have a standard solution, right?

2

Hi,

One way of doing this is to define one resource first but with an "action
:nothing", define the second resource that notifies the first and then
reopen the first resource but apply actions. i.e. Something like the
following (ignore syntax errors as away from editor)

service "foo" do
action :nothing
end

template "/etc/foo.conf" do
notifies :restart, resource(:service => "foo"), :immediate
end

service "foo" do
action [:enable, :start]
end

HTH

On Wed, Oct 24, 2012 at 10:57 AM, Leo Dirac (SR)
leo@scaledrecognition.comwrote:

Is it possible to send a notification to a resource if the resource hasn't
been created yet? When I try this it blows up complaining that the
resource is unknown. But I don't see any other way to do what I need to.

My application needs a server-specific configuration file, which I create
with a template. The app needs this when it deploys, so I create the
template (in /srv/app/shared/) before the app resource, and then symlink it
in before deploy. This basically works.

But the config file itself is templated with attributes. When those
attributes change, I want to restart the application. But I can't send a
notification because the config file needs to be defined before the app.

I tried defining the template inside the application's before_deploy
callback, but that doesn't seem to do anything. This kind of circular
dependency problem has to have a standard solution, right?

--
Cheers,

Peter Donald

3

That's exactly how we do it. A base application provider defines an empty service and we delay all notifications until after the rest of the app has been configured.

On Tuesday, October 23, 2012 at 5:19 PM, Peter Donald wrote:

Hi,

One way of doing this is to define one resource first but with an "action :nothing", define the second resource that notifies the first and then reopen the first resource but apply actions. i.e. Something like the following (ignore syntax errors as away from editor)

service "foo" do
action :nothing
end

template "/etc/foo.conf" do
notifies :restart, resource(:service => "foo"), :immediate
end

service "foo" do
action [:enable, :start]
end

HTH

On Wed, Oct 24, 2012 at 10:57 AM, Leo Dirac (SR) <leo@scaledrecognition.com (mailto:leo@scaledrecognition.com)> wrote:

Is it possible to send a notification to a resource if the resource hasn't been created yet? When I try this it blows up complaining that the resource is unknown. But I don't see any other way to do what I need to.

My application needs a server-specific configuration file, which I create with a template. The app needs this when it deploys, so I create the template (in /srv/app/shared/) before the app resource, and then symlink it in before deploy. This basically works.

But the config file itself is templated with attributes. When those attributes change, I want to restart the application. But I can't send a notification because the config file needs to be defined before the app.

I tried defining the template inside the application's before_deploy callback, but that doesn't seem to do anything. This kind of circular dependency problem has to have a standard solution, right?

--
Cheers,

Peter Donald

4

On Wed, Oct 24, 2012 at 1:57 AM, Leo Dirac (SR)
leo@scaledrecognition.comwrote:

I tried defining the template inside the application's before_deploy
callback, but that doesn't seem to do anything. This kind of circular
dependency problem has to have a standard solution, right?

You got other answers that are very good when you're using basic Chef
resources.

Since you're using the application cookbook, you have other options.
The before_deploy callback is the best pick if you want your template to
change at any time, even if you are not deploying a new version of the app.

However it's broken at the moment due to
http://tickets.opscode.com/browse/CHEF-3493
I am looking for a workaround I can apply in the cookbook while the
underlying Chef bug is addressed.

Could you possibly revert to 10.12 while we work on this? I should have an
update in the next week or so.

Andrea

5

Andrea, I deeply appreciate the work you put into maintaining the critical
application cookbook. But I'm hesitant to add another dependency if I can
avoid it. Especially one that makes my stack traces harder to read.

Peter's suggestion worked for me, but I'm confused by it. If I don't use
the :immediately timing option on notifies, it doesn't work. I see in the
log "Processing supervisor_service[srapiworker] action restart" followed by
a debug log line from the provider, but nothing happens. If I use
:immediately the same log line is followed by the service actually
restarting through the provider's execute command. This doesn't make any
sense to me.

Why is the :immediately needed to prevent the messages becoming no-ops?

On Wed, Oct 24, 2012 at 12:09 AM, Andrea Campi <andrea.campi@zephirworks.com

wrote:

On Wed, Oct 24, 2012 at 1:57 AM, Leo Dirac (SR) <leo@scaledrecognition.com

wrote:

I tried defining the template inside the application's before_deploy
callback, but that doesn't seem to do anything. This kind of circular
dependency problem has to have a standard solution, right?

You got other answers that are very good when you're using basic Chef
resources.

Since you're using the application cookbook, you have other options.
The before_deploy callback is the best pick if you want your template to
change at any time, even if you are not deploying a new version of the app.

However it's broken at the moment due to
http://tickets.opscode.com/browse/CHEF-3493
I am looking for a workaround I can apply in the cookbook while the
underlying Chef bug is addressed.

Could you possibly revert to 10.12 while we work on this? I should have an
update in the next week or so.

Andrea

6

Could you gist a recipe to represent your issue?

Since 0.9.10 (CHEF-899 [1]) you've been able to notify a resource that
hasn't been created yet because the notifications get resolved just
before convergence. Similar support was added in 10.14.0 for
subscribes (CHEF-1994 [2]). The most common situation where this is
still a problem is when a resource is created during convergence (like
in a ruby_block) that needs to be notified by another resource. When
you're looking at a recipe, their visual proximity makes it feel like
it should work but we're talking about pretty disparate points in the
run.

On Wed, Oct 24, 2012 at 4:32 PM, Leo Dirac (SR)
leo@scaledrecognition.com wrote:

Peter's suggestion worked for me, but I'm confused by it. If I don't use
the :immediately timing option on notifies, it doesn't work. I see in the
log "Processing supervisor_service[srapiworker] action restart" followed by
a debug log line from the provider, but nothing happens. If I use
:immediately the same log line is followed by the service actually
restarting through the provider's execute command. This doesn't make any
sense to me.

Why is the :immediately needed to prevent the messages becoming no-ops?

Having dug into CHEF-3493 a lot myself, I suspect you're working
inside another resource and it's possible that is putting you in a
recipe_eval which puts you in another run context and the queued
notifications don't get run because that run_context isn't a complete
chef run. Dan mentioned recipe_eval was a hack for something a bit
ago.

Seeing your recipe would help a lot though.

Bryan

[1] http://tickets.opscode.com/browse/CHEF-899
[2] http://tickets.opscode.com/browse/CHEF-1994

7

Hi Bryan -- here's a gist with the recipe as I finally got it working.

I think you're probably right about why the template couldn't notify the
supervisor_service -- the application recipe is complex and probably not
creating the supervisor_service until convergence time.

Two parts I still really don't understand. Why the before_restart and
after_restart callbacks didn't seem to ever do anything -- never fired as
far as I could see. I even tried various chef versions from 10.12 to 10.16
on Andrea's hint.

Also, it makes no sense to me why the notifies commands need the :immediate
modifier to work. Without it, the notification got processed (I saw
Chef::Log messages within them), but the execute command which did the
real work just got skipped.

I think I understand why the top-level notifies within the application
didn't do anything -- that would only fire if the application resource
itself changed, which basically never happens. It's the sub-resources that
are getting modified during code deployments. Is that right?

Thanks for your help!

On Wed, Oct 24, 2012 at 6:36 PM, Bryan McLellan btm@loftninjas.org wrote:

Could you gist a recipe to represent your issue?

Since 0.9.10 (CHEF-899 [1]) you've been able to notify a resource that
hasn't been created yet because the notifications get resolved just
before convergence. Similar support was added in 10.14.0 for
subscribes (CHEF-1994 [2]). The most common situation where this is
still a problem is when a resource is created during convergence (like
in a ruby_block) that needs to be notified by another resource. When
you're looking at a recipe, their visual proximity makes it feel like
it should work but we're talking about pretty disparate points in the
run.

On Wed, Oct 24, 2012 at 4:32 PM, Leo Dirac (SR)
leo@scaledrecognition.com wrote:

Peter's suggestion worked for me, but I'm confused by it. If I don't use
the :immediately timing option on notifies, it doesn't work. I see in
the
log "Processing supervisor_service[srapiworker] action restart" followed
by
a debug log line from the provider, but nothing happens. If I use
:immediately the same log line is followed by the service actually
restarting through the provider's execute command. This doesn't make any
sense to me.

Why is the :immediately needed to prevent the messages becoming no-ops?

Having dug into CHEF-3493 a lot myself, I suspect you're working
inside another resource and it's possible that is putting you in a
recipe_eval which puts you in another run context and the queued
notifications don't get run because that run_context isn't a complete
chef run. Dan mentioned recipe_eval was a hack for something a bit
ago.

Seeing your recipe would help a lot though.

Bryan

[1] http://tickets.opscode.com/browse/CHEF-899
[2] http://tickets.opscode.com/browse/CHEF-1994

8

On Thu, Oct 25, 2012 at 7:22 PM, Leo Dirac (SR)
leo@scaledrecognition.comwrote:

Hi Bryan -- here's a gist with the recipe as I finally got it working.
Chef recipe to run gunicorn application and gearman workers. Posted for discussion about notification problems. This is probably not a good example to follow. :) · GitHub

I think you're probably right about why the template couldn't notify the
supervisor_service -- the application recipe is complex and probably not
creating the supervisor_service until convergence time.

…
I think I understand why the top-level notifies within the application
didn't do anything -- that would only fire if the application resource
itself changed, which basically never happens. It's the sub-resources that
are getting modified during code deployments. Is that right?

Kinda, yeah.

I appreciate you want to keep things, the application cookbook contains a
lot of magic.
However looking at your gist, it may actually be easier if you implemented
an LWRP :slight_smile:

You would end up with:

application "imgmuck" do
… SCM stuff
gunicorn do
…
end

srapiworker do
.. maybe some options
end
end

Your hypothetical srapiworker LWRP would by kind of similar to the gunicorn
or celery LWRPs in application_python—just shuffling around things from
your gist.

That's the theory anyway, because you would will have this problem:

Two parts I still really don't understand. Why the before_restart and
after_restart callbacks didn't seem to ever do anything -- never fired as
far as I could see. I even tried various chef versions from 10.12 to 10.16
on Andrea's hint.

Again that's CHEF-3493
I just pushed a workaround to my repo [1] and sent a pull request.
Fair warning: I've only proved that it works in test-kitchen, I don't have
it running on real nodes yet :slight_smile:
Still, you want to give it a whirl.

Andrea

[1] GitHub - andreacampi/application at COOK-1673