Chefspec with guard/notifications


#1

Ohai Chefs!
I am having a problem getting Chefspec to work with guarded resources and notifications. It seems that Chefspec thinks that notifications are always written even if the guard blocks that resource from executing.

Given a recipe like this:

guardfile = '/tmp/guardfile'

# Testing guards and chained notifications.
template 'guardedtemplate' do
  not_if { File.exist?(guardfile) }
  notifies :restart, 'service[guardedservice]'
end

service 'guardedservice' do
  action :nothing
  notifies :write, 'log[guardedlog]'
end

log 'guardedlog' do
  action :nothing
end

and a spec file like this:

  # Testing guards and notifications
  it 'does not send a notification to the first service if guardfile exists' do
    allow(File).to receive(:exist?).and_call_original
    allow(File).to receive(:exist?).with('/tmp/guardfile').and_return(true)
    expect(guardedtemplate).to_not notify('service[guardedservice]')
    expect(guardedtemplate).to_not notify('service[not_service]')
  end

  it 'does not send a notification to the second service through the first if guardfile exists' do
    expect(guardedservice).to_not notify('log[guardedlog]')
  end

The rspec output says that the resource was notified.

  does not send a notification to the first service if guardfile exists (FAILED - 1)
  does not send a notification to the second service through the first if guardfile exists (FAILED - 2)

Failures:

  1) cme_test_e22225::chained does not send a notification to the first service if guardfile exists
     Failure/Error: expect(guardedtemplate).to_not notify('service[guardedservice]')
       expected "template[guardedtemplate]" to not notify "service[guardedservice]", but it did.
     # ./spec/unit/recipes/chained_spec.rb:51:in `block (2 levels) in <top (required)>'

  2) cme_test_e22225::chained does not send a notification to the second service through the first if guardfile exists
     Failure/Error: expect(guardedservice).to_not notify('log[guardedlog]')
       expected "service[guardedservice]" to not notify "log[guardedlog]", but it did.
     # ./spec/unit/recipes/chained_spec.rb:56:in `block (2 levels) in <top (required)>'

I searched around and found this: https://github.com/sethvargo/chefspec/issues/541 and https://github.com/sethvargo/chefspec/issues/613 and
Where sethvargo says:

In ChefSpec, you are asserting that the notification was written, not that action was taken. There’s no way to actually test that a notification fired (there’s no hook in Chef that ChefSpec can use). So you’re testing “did I tell the resource to notify on change” not “did the resource notify”.
….
ChefSpec writes all actions to noop. ChefSpec is really just a giant registry of the before + after Chef run. When you give a resource an action of “nothing”, it will always have the action of “nothing”, regardless of the notifications because the notifications won’t fire (because Chef doesn’t actually run).

And

Chef doesn’t really provide a way to fire those notifications, so no :frowning:. I would consider refactoring your code to rely less on notifications (I personally find notifications are unintuitive and difficult to follow). Usually you can refactor the code to be more declarative without notifications.

Also this https://github.com/sethvargo/chefspec/issues/546 / https://github.com/sethvargo/chefspec/pull/547#issuecomment-68649062

At the same time, ChefSpec is a unit testing framework. I realize it may be misleading, but when you test a notification in ChefSpec, you are not testing if a notification fired, but rather whether a notification was written. Chef does not provide a mechanism for deciding if X caused Y. It is theoretically possible for ChefSpec to assert if a notification fired, but it is impossible to know which resource caused the notification to fire.

I think this issue is a slightly different. In my case the notification is always written regardless if the guard is in place.
Any thoughts?


#2

Did you ever find a solution to this? I am getting started with chefspec, and having trouble figuring out the best way to test that a recipe runs correctly when there are conditional operations like this in place.