Ohai Chefs
I was wondering if someone had more elegant ideas for making resources notify another resource before running itself.
I’d want something like this:
file ‘foo’ do
content ‘bar’
notifies :stop, ‘service[foobar]’, :before
notifies :start, ‘service[foobar]’, :immediately
end
I realise this is possible with chaining resources. But I was wondering if there were more elegant solutions.
Kr,
Steven
check Chef webinar 1-4 (one of them has to have it)
you define what service does in one snippet and you set notify stuff in
other snippet
On Thu, Nov 21, 2013 at 1:39 PM, Steven De Coeyer steven@banteng.be wrote:
Ohai Chefs
I was wondering if someone had more elegant ideas for making resources
notify another resource before running itself.I’d want something like this:
file 'foo’ do
content ‘bar'
notifies :stop, ‘service[foobar]', :before
notifies :start, ‘service[foobar]’, :immediately
endI realise this is possible with chaining resources. But I was wondering if
there were more elegant solutions.Kr,
Steven
Thank you for your reply. I understand how notifies work, I was just wondering if there was an elegant way to notify resources before running current resource.
I know you could do it with chaining resources through notify, but that smells a bit.
I found an issue asking the same question:
https://tickets.opscode.com/browse/CHEF-2421
On 21 Nov 2013, at 14:40, Jasna Benčić jasna.bencic@teamsnap.com wrote:
check Chef webinar 1-4 (one of them has to have it) https://learnchef.opscode.com/
you define what service does in one snippet and you set notify stuff in other snippet
On Thu, Nov 21, 2013 at 1:39 PM, Steven De Coeyer steven@banteng.be wrote:
Ohai ChefsI was wondering if someone had more elegant ideas for making resources notify another resource before running itself.
I’d want something like this:
file 'foo’ do
content ‘bar'
notifies :stop, ‘service[foobar]', :before
notifies :start, ‘service[foobar]’, :immediately
endI realise this is possible with chaining resources. But I was wondering if there were more elegant solutions.
Kr,
Steven
On Thursday, November 21, 2013 at 6:20 AM, Steven De Coeyer wrote:
Thank you for your reply. I understand how notifies work, I was just wondering if there was an elegant way to notify resources before running current resource.
I know you could do it with chaining resources through notify, but that smells a bit.
I found an issue asking the same question:
https://tickets.opscode.com/browse/CHEF-2421
This would certainly be useful. The canonical use case is if you’re managing a service that needs to be stopped before it’s upgraded, which is common for some kinds of web app.
Unfortunately, the way most Chef providers are written now, there’s no reasonable way to implement this as a before notification. One possible avenue we haven’t explored yet is to run an individual resource in why-run mode to see if it would make a change, which might look like this in the DSL:
service “my-web-app” do
action :stop
# means “only if the resource deploy[my-web-app] would make changes when running the :deploy action"
test_resource “deploy[my-web-app]”, :deploy
end
The implementation would still be pretty complicated, but much less so than the things we’ve tried before. Someone outside of opscode would need to take this on because I don’t think we can fit it in our priorities for quite a while.
--
Daniel DeLeo
Sounds like a nice challenge to get more familiar with the internals of Chef.
Might actually look into this.
Thank you.
Kr,
Steven
On 21 Nov 2013, at 18:22, Daniel DeLeo dan@kallistec.com wrote:
On Thursday, November 21, 2013 at 6:20 AM, Steven De Coeyer wrote:
Thank you for your reply. I understand how notifies work, I was just wondering if there was an elegant way to notify resources before running current resource.
I know you could do it with chaining resources through notify, but that smells a bit.
I found an issue asking the same question:
https://tickets.opscode.com/browse/CHEF-2421
This would certainly be useful. The canonical use case is if you’re managing a service that needs to be stopped before it’s upgraded, which is common for some kinds of web app.Unfortunately, the way most Chef providers are written now, there’s no reasonable way to implement this as a before notification. One possible avenue we haven’t explored yet is to run an individual resource in why-run mode to see if it would make a change, which might look like this in the DSL:
service “my-web-app” do action :stop # means “only if the resource deploy[my-web-app] would make changes when running the :deploy action" test_resource “deploy[my-web-app]”, :deploy endThe implementation would still be pretty complicated, but much less so than the things we’ve tried before. Someone outside of opscode would need to take this on because I don’t think we can fit it in our priorities for quite a while.
--
Daniel DeLeo
Check out chef-rewind[1] if you want to see some code that mucks around
with the ordering of a run_context. Be forewarned, it can get a little
hairy if it is your first dive in.
[1] GitHub - bryanwb/chef-rewind: monkeypatch chef to edit existing resources in place
On Thu, Nov 21, 2013 at 5:25 PM, Steven De Coeyer steven@banteng.be wrote:
Sounds like a nice challenge to get more familiar with the internals of
Chef.
Might actually look into this.Thank you.
Kr,
StevenOn 21 Nov 2013, at 18:22, Daniel DeLeo dan@kallistec.com wrote:
On Thursday, November 21, 2013 at 6:20 AM, Steven De Coeyer wrote:
Thank you for your reply. I understand how notifies work, I was just
wondering if there was an elegant way to notify resources before running
current resource.I know you could do it with chaining resources through notify, but that
smells a bit.I found an issue asking the same question:
https://tickets.opscode.com/browse/CHEF-2421This would certainly be useful. The canonical use case is if you’re
managing a service that needs to be stopped before it’s upgraded, which is
common for some kinds of web app.Unfortunately, the way most Chef providers are written now, there’s no
reasonable way to implement this as a before notification. One possible
avenue we haven’t explored yet is to run an individual resource in why-run
mode to see if it would make a change, which might look like this in the
DSL:service “my-web-app” do action :stop # means “only if the resource deploy[my-web-app] would make changeswhen running the :deploy action"
test_resource “deploy[my-web-app]”, :deploy
endThe implementation would still be pretty complicated, but much less so
than the things we’ve tried before. Someone outside of opscode would need
to take this on because I don’t think we can fit it in our priorities for
quite a while.--
Daniel DeLeo
Hello guys,
On Thu, 21 Nov 2013 09:22:33 -0800
Daniel DeLeo dan@kallistec.com wrote:
Thank you for your reply. I understand how notifies work, I was
just wondering if there was an elegant way to notify resources
before running current resource. I know you could do it with
chaining resources through notify, but that smells a bit.
I found an issue asking the same question:
https://tickets.opscode.com/browse/CHEF-2421This would certainly be useful. The canonical use case is if you’re
managing a service that needs to be stopped before it’s upgraded,
which is common for some kinds of web app.Unfortunately, the way most Chef providers are written now, there’s
no reasonable way to implement this as a before notification. One
possible avenue we haven’t explored yet is to run an individual
resource in why-run mode to see if it would make a change, which
might look like this in the DSL:service “my-web-app” do action :stop # means “only if the resource deploy[my-web-app] would makechanges when running the :deploy action" test_resource
“deploy[my-web-app]”, :deploy endThe implementation would still be pretty complicated, but much less
so than the things we’ve tried before. Someone outside of opscode
would need to take this on because I don’t think we can fit it in our
priorities for quite a while.
I'm the one who tried to implement this a year ago. I proposed a
solution for the :before in this comment:
This solution included why-run support. In fact, it was easier for me
to implement this with why-run than without it.
After this solution was proposed, I know you talked a lot about it and
whether to add this feature to Chef or not. Even you asked on this
list. Much time was spent on this.
Time passed and the issue was finally closed putting it to "Won't
fix" state. But I didn't received any feedback about my code and I
didn't know how to improve it or what was the way forward.
The point is: Do you think it is a good idea to attempt to rebase this
solution and make a new PR or that would not help?
Bye,
Chef rewind is actually just a wrapper around something that’s already easy to do:
https://gist.github.com/sethvargo/6443603
I wrote a nice Recipe DSL-like syntax for doing this in chef-sugar:
before ‘service[apache2]’ do
log 'I am before the apache 2 service fires!'
end
There’s also an after and compile_time filter.
Thanks,
Seth Vargo
Solutions Engineer, Opscode
@sethvargo
From: Andrew Gross Andrew Gross
Reply: chef@lists.opscode.com chef@lists.opscode.com
Date: November 22, 2013 at 8:18:43 AM
To: chef chef@lists.opscode.com
Subject: [chef] Re: Re: notifies :before
Check out chef-rewind[1] if you want to see some code that mucks around with the ordering of a run_context. Be forewarned, it can get a little hairy if it is your first dive in.
[1] https://github.com/bryanwb/chef-rewind
On Thu, Nov 21, 2013 at 5:25 PM, Steven De Coeyer steven@banteng.be wrote:
Sounds like a nice challenge to get more familiar with the internals of Chef.
Might actually look into this.
Thank you.
Kr,
Steven
On 21 Nov 2013, at 18:22, Daniel DeLeo dan@kallistec.com wrote:
On Thursday, November 21, 2013 at 6:20 AM, Steven De Coeyer wrote:
Thank you for your reply. I understand how notifies work, I was just wondering if there was an elegant way to notify resources before running current resource.
I know you could do it with chaining resources through notify, but that smells a bit.
I found an issue asking the same question:
https://tickets.opscode.com/browse/CHEF-2421
This would certainly be useful. The canonical use case is if you’re managing a service that needs to be stopped before it’s upgraded, which is common for some kinds of web app.
Unfortunately, the way most Chef providers are written now, there’s no reasonable way to implement this as a before notification. One possible avenue we haven’t explored yet is to run an individual resource in why-run mode to see if it would make a change, which might look like this in the DSL:
service “my-web-app” do
action :stop
# means “only if the resource deploy[my-web-app] would make changes when running the :deploy action"
test_resource “deploy[my-web-app]”, :deploy
end
The implementation would still be pretty complicated, but much less so than the things we’ve tried before. Someone outside of opscode would need to take this on because I don’t think we can fit it in our priorities for quite a while.
–
Daniel DeLeo
On Friday, November 22, 2013 at 11:22 AM, Xabier de Zuazo wrote:
Hello guys,
On Thu, 21 Nov 2013 09:22:33 -0800
Daniel DeLeo <dan@kallistec.com (mailto:dan@kallistec.com)> wrote:Thank you for your reply. I understand how notifies work, I was
just wondering if there was an elegant way to notify resources
before running current resource. I know you could do it with
chaining resources through notify, but that smells a bit.
I found an issue asking the same question:
https://tickets.opscode.com/browse/CHEF-2421This would certainly be useful. The canonical use case is if you’re
managing a service that needs to be stopped before it’s upgraded,
which is common for some kinds of web app.Unfortunately, the way most Chef providers are written now, there’s
no reasonable way to implement this as a before notification. One
possible avenue we haven’t explored yet is to run an individual
resource in why-run mode to see if it would make a change, which
might look like this in the DSL:service “my-web-app” do
action :stopmeans “only if the resource deploy[my-web-app] would make
changes when running the :deploy action" test_resource
“deploy[my-web-app]”, :deploy endThe implementation would still be pretty complicated, but much less
so than the things we’ve tried before. Someone outside of opscode
would need to take this on because I don’t think we can fit it in our
priorities for quite a while.I'm the one who tried to implement this a year ago. I proposed a
solution for the :before in this comment:This solution included why-run support. In fact, it was easier for me
to implement this with why-run than without it.After this solution was proposed, I know you talked a lot about it and
whether to add this feature to Chef or not. Even you asked on this
list. Much time was spent on this.Time passed and the issue was finally closed putting it to "Won't
fix" state. But I didn't received any feedback about my code and I
didn't know how to improve it or what was the way forward.The point is: Do you think it is a good idea to attempt to rebase this
solution and make a new PR or that would not help?Bye,
--
Xabier de Zuazo Oteiza
IT System Administrator - Onddo Labs S.L.
www.onddo.com (http://www.onddo.com)Public Key = http://www.onddo.com/xabier_zuazo.pub
Key Fingerprint = 8EFA 5B17 7275 5F1F 42B2 26B4 8E18 8B67 9DE1 9468
What happened here is, when we first implemented why run mode, we thought that we could restructure the way that providers work to do the following:
This was right around the time you submitted your patch, so it seemed sensible that we could just do:
2a. Run before notifications
At the same time that you were updating your patch, we learned that what we’d done by restructuring the providers to work that way was based on faulty assumptions and made the system more complicated and difficult to debug. In particular, whether or not a provider needs to take some action may depend on what the provider did in a previous step (this was most apparent for complicated providers like deploy). So we changed it to work more like it did previously:
Unfortunately, this meant that we fundamentally broke the things your patch needed to work, after we suggested you could take advantage of them. I personally feel quite terrible that this happened. Sorry.
I know this is a valuable feature for some people and implementing the same behavior on your own is a PITA, so I’ve been trying to dream up ways that it could work without a scary level of implementation complexity. What I’m proposing is a little different than your patch:
Example:
service “my-web-app” do
action :nothing
end
deploy “my-web-app” do
notify_before “service[my-web-app]”, :stop
end
service “my-web-app” do
action :stop
only_if { test_action “deploy[my-web-app]”, :deploy }
only_if_would_update “deploy[my-web-app]”, :deploy
end
deploy “my-web-app” do
end
I hope that explains the difference. Apologies again for sending you down the wrong path with your patch.
--
Daniel DeLeo
[Thread moved to chef-dev]
Hello Daniel,
On Fri, 22 Nov 2013 12:07:49 -0800
Daniel DeLeo dan@kallistec.com wrote:
[...]
I'm the one who tried to implement this a year ago. I proposed a
solution for the :before in this comment:This solution included why-run support. In fact, it was easier for
me to implement this with why-run than without it.After this solution was proposed, I know you talked a lot about it
and whether to add this feature to Chef or not. Even you asked on
this list. Much time was spent on this.Time passed and the issue was finally closed putting it to "Won't
fix" state. But I didn't received any feedback about my code and I
didn't know how to improve it or what was the way forward.
[...]
What happened here is, when we first implemented why run mode, we
thought that we could restructure the way that providers work to do
the following:
- Examine the system
- Compile a list of actions (as ruby Proc objects) that would
converge the system to the desired state 3. Execute each action
generated in step 2.This was right around the time you submitted your patch, so it seemed
sensible that we could just do:
- Examine the system
- Compile a list of actions (as ruby Proc objects) that would
converge the system to the desired state2a. Run before notifications
- Execute each action generated in step 2.
At the same time that you were updating your patch, we learned that
what we’d done by restructuring the providers to work that way was
based on faulty assumptions and made the system more complicated and
difficult to debug. In particular, whether or not a provider needs to
take some action may depend on what the provider did in a previous
step (this was most apparent for complicated providers like deploy).
So we changed it to work more like it did previously:
- Examine some aspect of the system
- Run some code that converges that aspect of the system to the
desired state 3. Repeat 1-2 until all aspects of the system described
by a resource are in the desired state.Unfortunately, this meant that we fundamentally broke the things your
patch needed to work, after we suggested you could take advantage of
them. I personally feel quite terrible that this happened. Sorry.I know this is a valuable feature for some people and implementing
the same behavior on your own is a PITA, so I’ve been trying to dream
up ways that it could work without a scary level of implementation
complexity. [...]
Thanks for your detailed reply. Don't worry. I know it's a long ticket
and this makes it really difficult to review 
I reviewed my patches and, in short, the events of the ticket were:
Since then, it seems that you have not made big changes to why-run
logic. I mean, this code seems to work in current master. I rebased it
here (fixing some small conflicts):
I tested it with some simple recipes and seems to work.
I know my proposal is far from being perfect. But I find it strange
that it was rejected without giving me a chance to improve it. Maybe I
was misguided, but AFAIK nobody wrote me about it :-S
Anyway, based on this code, do you think it appropriate for me to open
a new ticket and propose as a solution? (I can not open the previous
ticket) Or you just don't like my solution and you are looking for
something else?
Cheers,
On 11/24/13 7:57 PM, Xabier de Zuazo wrote:
Thanks for your detailed reply. Don't worry. I know it's a long ticket
and this makes it really difficult to reviewI reviewed my patches and, in short, the events of the ticket were:
- My first PR.
- You recommended me to use why-run to predict updated_by_last_action.
- Updated to use why-run.
- You told me that CHEF-3589 broke my path.
- Rebased and logic changed to work with CHEF-3589.
- Won't fix.
Since then, it seems that you have not made big changes to why-run
logic. I mean, this code seems to work in current master. I rebased it
here (fixing some small conflicts):Comparing chef:main...zuazo-forks:CHEF-2421-3 · chef/chef · GitHub
I tested it with some simple recipes and seems to work.
I know my proposal is far from being perfect. But I find it strange
that it was rejected without giving me a chance to improve it. Maybe I
was misguided, but AFAIK nobody wrote me about it :-SAnyway, based on this code, do you think it appropriate for me to open
a new ticket and propose as a solution? (I can not open the previous
ticket) Or you just don't like my solution and you are looking for
something else?Cheers,
What is your feature supposed to do now? The thing that you wanted to
do was to be able to use the old behavior of why-run mode in order to be
able to figure out if there were any queue'd up converge_by blocks, but
before the blocks were executed, in order to get a kind of
"would_modify_by_action?" check. We completely broke the ability to do
that, and as its noted in the ticket its difficult to even predict in
the general case that a resource will be updated before you call the
action. So what are the conditions under which you expect these before
notifications to fire? And why can't you restructure your run_list so
that those resources simply come first and are idempotent? Why can't
you restructure your run_list to use an after notification?
This smells like its become a feature-for-the-sake-of-a-feature where
there's other, better, clearer ways to get the job done.
Hello Lamont Granquist,
I will try to answer all your questions. Please, let me know if
anything is not clear or I'm wrong. I'm far from being a Chef
expert
Before we begin: I know my solution may not be perfect and certainly
can be improved. But I also think it's not a bad starting point.
On Tue, 26 Nov 2013 11:49:59 -0800
Lamont Granquist lamont@opscode.com wrote:
[...]
Comparing chef:main...zuazo-forks:CHEF-2421-3 · chef/chef · GitHub
[...]
What is your feature supposed to do now? The thing that you wanted
to do was to be able to use the old behavior of why-run mode in order
to be able to figure out if there were any queue'd up converge_by
blocks, but before the blocks were executed, in order to get a kind
of "would_modify_by_action?" check.
My patch does not work exactly like that.
I'll try to explain more or less how it works (linking some code
snippets).
In summary:
If resource supports why-run, updated_by_last_action will be
predicted and :befores executed.
If resource does not support why-run, it will try to use not_if &
only_if conditionals to know whether to run :before notifications,
printing a warning about it, because may not be the expected
behavior.
I think this sounds simple enough.
To predict updated_by_last_action with why-run (1.), Chef will
run :before notifications inside the #converge_by method, but before
running the converge_by &block.
WhyRun::ConvergeActions#add_action patch: [CHEF-2421(3)] execute other resources BEFORE main resource by zuazo · Pull Request #1145 · chef/chef · GitHub
This works because whenever we call #converge_by,
updated_by_last_action will become true, so, the resource will be run
and all notifications called.
Provider methods that establish this behavior: http://git.io/KFih4w
My solution predicts updated_by_last_action because the logic that
determines its value is outside #converge_by. It is inside each
provider logic, before calling #converge_by. So, if #converge_by is
called, updated_by_last_action will always be true.
You can check the code of any provider to verify this behavior.
If why-run is not supported by the resource (2.), it will run
the :before notifications only if it has only_if/not_if conditions and
they are met.
Provider#run_action patch: [CHEF-2421(3)] execute other resources BEFORE main resource by zuazo · Pull Request #1145 · chef/chef · GitHub
Provider#run_before_notifications: [CHEF-2421(3)] execute other resources BEFORE main resource by zuazo · Pull Request #1145 · chef/chef · GitHub
The latter can be considered as an incomplete imitation of the :before
notifications. But without why-run, I think it's impossible to predict
updated_by_last_action without changing all the providers in the
space and part of the void.
We completely broke the ability
to do that, and as its noted in the ticket its difficult to even
predict in the general case that a resource will be updated before
you call the action.
I know you advised me to use why-run to predict updated_by_last_action,
insinuating that I could use something like the
Provider#resource_updated? or the ConvergeActions#empty? method,
and I started to implement this way. But at the time, I
could not find a clean way to implement that without adding tricky
code, so I finally used a similar but slightly different approach (it
was long ago, so I do not remember in detail).
I think it was a mistake on my part not clarify this. I hope it is
already clear how it works.
So what are the conditions under which you
expect these before notifications to fire?
When why-run is supported, before notifications are fired when the
resource will be updated, and just before calling the converge_by
block.
And why can't you
restructure your run_list so that those resources simply come first
and are idempotent?
Because sometimes you require to change the status of the same
resource multiple times during a chef run.
Some simple examples are mainly related with stopping services, perhaps
before updates, and then starting them again. Something like:
execute "install/update database" do
command " ... "
notifies :stop, "service[database]", :before
notifies :start, "service[database]", :immediately
only_if { ... }
end
execute "install my-webapp" do
command " ... "
notifies :stop, "service[tomcat]", :before
notifies :start, "service[tomcat]", :immediately
only_if { ... }
end
There are some more examples in the ticket.
Why can't you restructure your run_list to use
an after notification?
I think the following comment in the ticket answers this question:
https://tickets.opscode.com/browse/CHEF-2421?focusedCommentId=27014&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-27014
Implementing some :before notifications using :after surely is not
impossible. But IMHO the code can become weird and difficult to
understand in some cases.
This smells like its become a feature-for-the-sake-of-a-feature where
there's other, better, clearer ways to get the job done.
Maybe, but this seems to be a requested feature by some people. This is
not the first time the topic comes out on the list/IRC.
From Sign in to GitHub · GitHub
Generally the code seems overly complicated and i think it still has
vestigial code that was meant to deal with the old approach of trying
to only trigger when the resource would be updated, but before the
converge blocks were invoked. since you simply cannot do that any
more, the replacement feature must be to just hit the before notify
every time the provider is invoked, which is a simpler problem, and i
don't understand the complexity here.
Some of these things are already answered above.
AFAIK, using this solution the provider does NOT call the before
notification every time the provider is invoked. It calls it whenever
the provider will be run, i.e. whenever updated_by_last_action will be
true.
i'm also less sure of the value
of this feature, since you can just structure your run list so that
the resource you are 'notifying' comes first.
You maybe right, but I think it's not so obvious how to implement this
without :before.
is there a compelling
use case where you want to have a recipe follow in the run list but
have a prior recipe's resource notify a resource that comes later?
I think this feature is useful for calling both resources that comes
later and resources that has come before. Some examples are in this
email and in the ticket.
Either way, and above all, thanks for taking time to review this
ticket. The decision on the importance of this feature and whether to
do it my way or not is in your hands. My opinions are clearly
debatable
Regards,