Notifies on resource not notifying

Chef Infra (archive)
1

Hello,

I am writing a recipe that calls 3 resources, the first resource has
conditional execution and a notifies, goal being that the other 2 dont run
unless this one passed the condition.

I can see the first resource being run, but it doesn’t seem to be
notifying, can someone help figure out what I am messing up? is there a
better way to go about this?

Here is a gist of the recipe, and the debug output from Chef 10.16.4:

Any help appreciated.

Thanks

2

Hi Seif!

This is a bug in the LWRP DSL.

You can read about it here:
http://tickets.opscode.com/browse/CHEF-3681

There is a fix coming in Chef 11 which will require a small changes to
the all LWRP code including teamcity and windows cookbooks to make
notifications work.

In the meantime, you modify the teamcity_build LWRP and cargo-cult the
pattern referenced in the above ticket.

Good luck and let me know if you have any questions!

-s

On Thu, Dec 27, 2012 at 10:43 AM, Seif Attar iam@seifattar.net wrote:

Hello,

I am writing a recipe that calls 3 resources, the first resource has
conditional execution and a notifies, goal being that the other 2 dont run
unless this one passed the condition.

I can see the first resource being run, but it doesn't seem to be notifying,
can someone help figure out what I am messing up? is there a better way to
go about this?

Here is a gist of the recipe, and the debug output from Chef 10.16.4:

default.rb · GitHub

Any help appreciated.

Thanks

3

Thanks Sean,

My ruby skills are limited, and I didn't quite get that gist or what it was
doing, but I did manage to get something working:

I am not sure what

if sub_run_context.resource_collection.any?(&:updated?)

Is supposed to do, as it was always returning false for me. What
effects will removing it do? does the provider look sane to you now?

Much appreciated,
Seif

On 27 December 2012 15:52, Sean OMeara someara@gmail.com wrote:

Hi Seif!

This is a bug in the LWRP DSL.

You can read about it here:
http://tickets.opscode.com/browse/CHEF-3681

There is a fix coming in Chef 11 which will require a small changes to
the all LWRP code including teamcity and windows cookbooks to make
notifications work.

In the meantime, you modify the teamcity_build LWRP and cargo-cult the
pattern referenced in the above ticket.

Intelligent Provider Action · GitHub

Good luck and let me know if you have any questions!

-s

On Thu, Dec 27, 2012 at 10:43 AM, Seif Attar iam@seifattar.net wrote:

Hello,

I am writing a recipe that calls 3 resources, the first resource has
conditional execution and a notifies, goal being that the other 2 dont
run
unless this one passed the condition.

I can see the first resource being run, but it doesn't seem to be
notifying,
can someone help figure out what I am messing up? is there a better way
to
go about this?

Here is a gist of the recipe, and the debug output from Chef 10.16.4:

default.rb · GitHub

Any help appreciated.

Thanks

4

On Thursday, December 27, 2012 at 10:32 AM, Seif Attar wrote:

Thanks Sean,

My ruby skills are limited, and I didn't quite get that gist or what it was doing, but I did manage to get something working:

Workaround for LWRP DSL bug · seif/teamcity@125633d · GitHub

I am not sure what
if sub_run_context.resource_collection.any?(&:updated?)

Is supposed to do, as it was always returning false for me. What effects will removing it do? does the provider look sane to you now?

I think Sean was assuming your LWRP was built upon a collection of core chef resources, in which case the bug he described would likely be your issue. Looking at your provider code, I see that it's all plain ruby code, so it's not relevant to you. If you'd created resources in your provider's action block, then the line above would iterate through them and see if any had been marked updated. I wrote an explanation of how the two approaches differ on our page about changes in Chef 11 (near the bottom of the page in the LWRP section): http://wiki.opscode.com/display/chef/Breaking+Changes+in+Chef+11

Much appreciated,
Seif
Backing up a bit, the thing that's making this work now is the line that does @new_resource.updated_by_last_action(true). This sets a flag on the resource that indicates that the resource was updated and its notifications should be fired.

In your provider code (as now written), your resource will always be marked updated. If you expect your LWRP to make changes to the system every time it runs, then this is the right thing to do.

In general, providers are expected to be idempotent, that is, they will not take any action if the system is already in the desired state. When this happens, the provider should not mark the resource as updated. Looks to me like you'd want to do this based on whether the artifacts you've downloaded previously match the one that you're fetching from the server. The remote_file resource currently does this: chef/lib/chef/provider/remote_file.rb at main · chef/chef · GitHub

It looks like you need HTTP basic auth for your LWRP, and I'm not sure if you can use a hack (like putting auth info in the URL) to make remote_file to use basic auth or not. If you can make it work, using remote_file would provide idempotent behavior for you. If not, I'd encourage you to submit a patch for remote_file to add a basic auth option.

HTH,

--
Daniel DeLeo

5

Hi Seif.

It turns out that I gave you the wrong advice, since I didn't actually
dig into the teamcity cookbook code (like I am now.)

The gist I sent you was a workaround for when you're using nested Chef
resources in LWRP code. It would help in a situation like this:

cookbooks/custom/providers/foo.rb

action :download do

directory "/some/dir" do
action :create
owner "fonzi"

remote_file "somefile" do
source "http://www.example.com/somefile.tgz"
checksum "123456"
end

end

^ In that example, "directory" and "remote_file" are Chef resources.
One of the contracts that a Chef provider has to fulfill is that it
will set the new_resource.updated_by_last_action(true) it they make a
change to the system.

If 'directory[/some/dir]' had to be repaired in any way, the
directory provider setting that flag is what allows notifications to
happen.

if 'remote_file[somefile]' had to be repaired, the same idea applies.

The bug the gist worked around was that the custom_foo resource
defined in the LWRP code could not inherit the updated status of the
nested 'directory' or 'remote_file' resources.

While good to know, this is now the problem you have.

The work you're doing is implemented in two pure-ruby library
functions that ship with the cookbook.

initialize_connection
download_all

Since these are not idempotent Chef resources, the work-around does not apply.
Please feel free to un-cargo-cult your code. My bad.

What you really need to do is setup tests to determine if you need to
"fix" a teamcity_build resource on disk.

If the resource needs to be repaired, THEN do the work:
initialize_connection
download_all
new_resource.updated_by_last_action(true)

That will allow notifications to work in the default run_context

This deck does a really good job at explaining how it all works.

-s

On Thu, Dec 27, 2012 at 12:32 PM, Seif Attar iam@seifattar.net wrote:

Thanks Sean,

My ruby skills are limited, and I didn't quite get that gist or what it was
doing, but I did manage to get something working:

Workaround for LWRP DSL bug · seif/teamcity@125633d · GitHub

I am not sure what

if sub_run_context.resource_collection.any?(&:updated?)

Is supposed to do, as it was always returning false for me. What effects
will removing it do? does the provider look sane to you now?

Much appreciated,

Seif

On 27 December 2012 15:52, Sean OMeara someara@gmail.com wrote:

Hi Seif!

This is a bug in the LWRP DSL.

You can read about it here:
http://tickets.opscode.com/browse/CHEF-3681

There is a fix coming in Chef 11 which will require a small changes to
the all LWRP code including teamcity and windows cookbooks to make
notifications work.

In the meantime, you modify the teamcity_build LWRP and cargo-cult the
pattern referenced in the above ticket.

Intelligent Provider Action · GitHub

Good luck and let me know if you have any questions!

-s

On Thu, Dec 27, 2012 at 10:43 AM, Seif Attar iam@seifattar.net wrote:

Hello,

I am writing a recipe that calls 3 resources, the first resource has
conditional execution and a notifies, goal being that the other 2 dont
run
unless this one passed the condition.

I can see the first resource being run, but it doesn't seem to be
notifying,
can someone help figure out what I am messing up? is there a better way
to
go about this?

Here is a gist of the recipe, and the debug output from Chef 10.16.4:

default.rb · GitHub

Any help appreciated.

Thanks

6

Thank you both for taking the time to explain this, I get it now.

To get the teamcity_build resource to notify, all I need to do is call
@new_resource.updated_by_last_action(true)

Thanks for the advice on the idempotency, I think I will need to take it
with the original cookbook author as he had provided a parameter
'overwrite' on the resource, which currently throws an exception if
overwrite is false and the file already existed. I guess to keep with
current behaviour:

If the files doesn't exist -> download and set updated
files exists and override is false -> throw
file exists, override is true and checksums differ -> download and set
updated
file exists, override is true and checksums differ -> do nothing and don't
set updated.

Had a quick look at adding basic auth to remote_file and it seems it would
require changes to Chef::Rest as well, too risky for my basic ruby skills
:slight_smile: copying the checksum code into those recipes would be easier.

Thanks again,
Seif

On 27 December 2012 18:12, Sean OMeara someara@gmail.com wrote:

Hi Seif.

It turns out that I gave you the wrong advice, since I didn't actually
dig into the teamcity cookbook code (like I am now.)

The gist I sent you was a workaround for when you're using nested Chef
resources in LWRP code. It would help in a situation like this:

cookbooks/custom/providers/foo.rb

action :download do

directory "/some/dir" do
action :create
owner "fonzi"

remote_file "somefile" do
source "http://www.example.com/somefile.tgz"
checksum "123456"
end

end

^ In that example, "directory" and "remote_file" are Chef resources.
One of the contracts that a Chef provider has to fulfill is that it
will set the new_resource.updated_by_last_action(true) it they make a
change to the system.

If 'directory[/some/dir]' had to be repaired in any way, the
directory provider setting that flag is what allows notifications to
happen.

if 'remote_file[somefile]' had to be repaired, the same idea applies.

The bug the gist worked around was that the custom_foo resource
defined in the LWRP code could not inherit the updated status of the
nested 'directory' or 'remote_file' resources.

While good to know, this is now the problem you have.

The work you're doing is implemented in two pure-ruby library
functions that ship with the cookbook.

initialize_connection
download_all

Since these are not idempotent Chef resources, the work-around does not
apply.
Please feel free to un-cargo-cult your code. My bad.

What you really need to do is setup tests to determine if you need to
"fix" a teamcity_build resource on disk.

If the resource needs to be repaired, THEN do the work:
initialize_connection
download_all
new_resource.updated_by_last_action(true)

That will allow notifications to work in the default run_context

This deck does a really good job at explaining how it all works.
http://www.slideshare.net/geekbri/lwrp-presentation#btnNext

-s

On Thu, Dec 27, 2012 at 12:32 PM, Seif Attar iam@seifattar.net wrote:

Thanks Sean,

My ruby skills are limited, and I didn't quite get that gist or what it
was
doing, but I did manage to get something working:

Workaround for LWRP DSL bug · seif/teamcity@125633d · GitHub

I am not sure what

if sub_run_context.resource_collection.any?(&:updated?)

Is supposed to do, as it was always returning false for me. What effects
will removing it do? does the provider look sane to you now?

Much appreciated,

Seif

On 27 December 2012 15:52, Sean OMeara someara@gmail.com wrote:

Hi Seif!

This is a bug in the LWRP DSL.

You can read about it here:
http://tickets.opscode.com/browse/CHEF-3681

There is a fix coming in Chef 11 which will require a small changes to
the all LWRP code including teamcity and windows cookbooks to make
notifications work.

In the meantime, you modify the teamcity_build LWRP and cargo-cult the
pattern referenced in the above ticket.

Intelligent Provider Action · GitHub

Good luck and let me know if you have any questions!

-s

On Thu, Dec 27, 2012 at 10:43 AM, Seif Attar iam@seifattar.net wrote:

Hello,

I am writing a recipe that calls 3 resources, the first resource has
conditional execution and a notifies, goal being that the other 2 dont
run
unless this one passed the condition.

I can see the first resource being run, but it doesn't seem to be
notifying,
can someone help figure out what I am messing up? is there a better
way
to
go about this?

Here is a gist of the recipe, and the debug output from Chef 10.16.4:

default.rb · GitHub

Any help appreciated.

Thanks

7

I seem to be getting the same problem with the template resource, the
resource runs, I can see in the log output that the content has been
updated, but the notifies don't trigger.

Is it possible that I am suffering from the same bug? I can't imagine that
a basic resource like template would have a problem with notifications,
there must be something with my setup.

I had a look at the source code of the template and file providers, but
couldn't see anywhere that it calls updated_by_last_action(true) :S

My gut feeling is that it is something that has to do with the windows file
provider.

On 27 December 2012 18:46, Seif Attar iam@seifattar.net wrote:

Thank you both for taking the time to explain this, I get it now.

To get the teamcity_build resource to notify, all I need to do is call
@new_resource.updated_by_last_action(true)

Thanks for the advice on the idempotency, I think I will need to take it
with the original cookbook author as he had provided a parameter
'overwrite' on the resource, which currently throws an exception if
overwrite is false and the file already existed. I guess to keep with
current behaviour:

If the files doesn't exist -> download and set updated
files exists and override is false -> throw
file exists, override is true and checksums differ -> download and set
updated
file exists, override is true and checksums differ -> do nothing and don't
set updated.

Had a quick look at adding basic auth to remote_file and it seems it would
require changes to Chef::Rest as well, too risky for my basic ruby skills
:slight_smile: copying the checksum code into those recipes would be easier.

Thanks again,
Seif

On 27 December 2012 18:12, Sean OMeara someara@gmail.com wrote:

Hi Seif.

It turns out that I gave you the wrong advice, since I didn't actually
dig into the teamcity cookbook code (like I am now.)

The gist I sent you was a workaround for when you're using nested Chef
resources in LWRP code. It would help in a situation like this:

cookbooks/custom/providers/foo.rb

action :download do

directory "/some/dir" do
action :create
owner "fonzi"

remote_file "somefile" do
source "http://www.example.com/somefile.tgz"
checksum "123456"
end

end

^ In that example, "directory" and "remote_file" are Chef resources.
One of the contracts that a Chef provider has to fulfill is that it
will set the new_resource.updated_by_last_action(true) it they make a
change to the system.

If 'directory[/some/dir]' had to be repaired in any way, the
directory provider setting that flag is what allows notifications to
happen.

if 'remote_file[somefile]' had to be repaired, the same idea applies.

The bug the gist worked around was that the custom_foo resource
defined in the LWRP code could not inherit the updated status of the
nested 'directory' or 'remote_file' resources.

While good to know, this is now the problem you have.

The work you're doing is implemented in two pure-ruby library
functions that ship with the cookbook.

initialize_connection
download_all

Since these are not idempotent Chef resources, the work-around does not
apply.
Please feel free to un-cargo-cult your code. My bad.

What you really need to do is setup tests to determine if you need to
"fix" a teamcity_build resource on disk.

If the resource needs to be repaired, THEN do the work:
initialize_connection
download_all
new_resource.updated_by_last_action(true)

That will allow notifications to work in the default run_context

This deck does a really good job at explaining how it all works.
Lwrp presentation | PPT

-s

On Thu, Dec 27, 2012 at 12:32 PM, Seif Attar iam@seifattar.net wrote:

Thanks Sean,

My ruby skills are limited, and I didn't quite get that gist or what it
was
doing, but I did manage to get something working:

Workaround for LWRP DSL bug · seif/teamcity@125633d · GitHub

I am not sure what

if sub_run_context.resource_collection.any?(&:updated?)

Is supposed to do, as it was always returning false for me. What effects
will removing it do? does the provider look sane to you now?

Much appreciated,

Seif

On 27 December 2012 15:52, Sean OMeara someara@gmail.com wrote:

Hi Seif!

This is a bug in the LWRP DSL.

You can read about it here:
http://tickets.opscode.com/browse/CHEF-3681

There is a fix coming in Chef 11 which will require a small changes to
the all LWRP code including teamcity and windows cookbooks to make
notifications work.

In the meantime, you modify the teamcity_build LWRP and cargo-cult the
pattern referenced in the above ticket.

Intelligent Provider Action · GitHub

Good luck and let me know if you have any questions!

-s

On Thu, Dec 27, 2012 at 10:43 AM, Seif Attar iam@seifattar.net
wrote:

Hello,

I am writing a recipe that calls 3 resources, the first resource has
conditional execution and a notifies, goal being that the other 2
dont
run
unless this one passed the condition.

I can see the first resource being run, but it doesn't seem to be
notifying,
can someone help figure out what I am messing up? is there a better
way
to
go about this?

Here is a gist of the recipe, and the debug output from Chef 10.16.4:

default.rb · GitHub

Any help appreciated.

Thanks

8

Can you gist a code snippet?
-s

On Fri, Dec 28, 2012 at 12:47 PM, Seif Attar iam@seifattar.net wrote:

I seem to be getting the same problem with the template resource, the
resource runs, I can see in the log output that the content has been
updated, but the notifies don't trigger.

Is it possible that I am suffering from the same bug? I can't imagine that a
basic resource like template would have a problem with notifications, there
must be something with my setup.

I had a look at the source code of the template and file providers, but
couldn't see anywhere that it calls updated_by_last_action(true) :S

My gut feeling is that it is something that has to do with the windows file
provider.

On 27 December 2012 18:46, Seif Attar iam@seifattar.net wrote:

Thank you both for taking the time to explain this, I get it now.

To get the teamcity_build resource to notify, all I need to do is call
@new_resource.updated_by_last_action(true)

Thanks for the advice on the idempotency, I think I will need to take it
with the original cookbook author as he had provided a parameter 'overwrite'
on the resource, which currently throws an exception if overwrite is false
and the file already existed. I guess to keep with current behaviour:

If the files doesn't exist -> download and set updated
files exists and override is false -> throw
file exists, override is true and checksums differ -> download and set
updated
file exists, override is true and checksums differ -> do nothing and don't
set updated.

Had a quick look at adding basic auth to remote_file and it seems it would
require changes to Chef::Rest as well, too risky for my basic ruby skills :slight_smile:
copying the checksum code into those recipes would be easier.

Thanks again,
Seif

On 27 December 2012 18:12, Sean OMeara someara@gmail.com wrote:

Hi Seif.

It turns out that I gave you the wrong advice, since I didn't actually
dig into the teamcity cookbook code (like I am now.)

The gist I sent you was a workaround for when you're using nested Chef
resources in LWRP code. It would help in a situation like this:

cookbooks/custom/providers/foo.rb

action :download do

directory "/some/dir" do
action :create
owner "fonzi"

remote_file "somefile" do
source "http://www.example.com/somefile.tgz"
checksum "123456"
end

end

^ In that example, "directory" and "remote_file" are Chef resources.
One of the contracts that a Chef provider has to fulfill is that it
will set the new_resource.updated_by_last_action(true) it they make a
change to the system.

If 'directory[/some/dir]' had to be repaired in any way, the
directory provider setting that flag is what allows notifications to
happen.

if 'remote_file[somefile]' had to be repaired, the same idea applies.

The bug the gist worked around was that the custom_foo resource
defined in the LWRP code could not inherit the updated status of the
nested 'directory' or 'remote_file' resources.

While good to know, this is now the problem you have.

The work you're doing is implemented in two pure-ruby library
functions that ship with the cookbook.

initialize_connection
download_all

Since these are not idempotent Chef resources, the work-around does not
apply.
Please feel free to un-cargo-cult your code. My bad.

What you really need to do is setup tests to determine if you need to
"fix" a teamcity_build resource on disk.

If the resource needs to be repaired, THEN do the work:
initialize_connection
download_all
new_resource.updated_by_last_action(true)

That will allow notifications to work in the default run_context

This deck does a really good job at explaining how it all works.
Lwrp presentation | PPT

-s

On Thu, Dec 27, 2012 at 12:32 PM, Seif Attar iam@seifattar.net wrote:

Thanks Sean,

My ruby skills are limited, and I didn't quite get that gist or what it
was
doing, but I did manage to get something working:

Workaround for LWRP DSL bug · seif/teamcity@125633d · GitHub

I am not sure what

if sub_run_context.resource_collection.any?(&:updated?)

Is supposed to do, as it was always returning false for me. What
effects
will removing it do? does the provider look sane to you now?

Much appreciated,

Seif

On 27 December 2012 15:52, Sean OMeara someara@gmail.com wrote:

Hi Seif!

This is a bug in the LWRP DSL.

You can read about it here:
http://tickets.opscode.com/browse/CHEF-3681

There is a fix coming in Chef 11 which will require a small changes to
the all LWRP code including teamcity and windows cookbooks to make
notifications work.

In the meantime, you modify the teamcity_build LWRP and cargo-cult the
pattern referenced in the above ticket.

Intelligent Provider Action · GitHub

Good luck and let me know if you have any questions!

-s

On Thu, Dec 27, 2012 at 10:43 AM, Seif Attar iam@seifattar.net
wrote:

Hello,

I am writing a recipe that calls 3 resources, the first resource has
conditional execution and a notifies, goal being that the other 2
dont
run
unless this one passed the condition.

I can see the first resource being run, but it doesn't seem to be
notifying,
can someone help figure out what I am messing up? is there a better
way
to
go about this?

Here is a gist of the recipe, and the debug output from Chef
10.16.4:

default.rb · GitHub

Any help appreciated.

Thanks

9

Sure.

Thanks Sean. I managed to find the code that calls updated_by_last_action
and I don't see any code around win32 in there, my theory failed, I think.

On 28 December 2012 17:49, Sean OMeara someara@gmail.com wrote:

Can you gist a code snippet?
-s

On Fri, Dec 28, 2012 at 12:47 PM, Seif Attar iam@seifattar.net wrote:

I seem to be getting the same problem with the template resource, the
resource runs, I can see in the log output that the content has been
updated, but the notifies don't trigger.

Is it possible that I am suffering from the same bug? I can't imagine
that a
basic resource like template would have a problem with notifications,
there
must be something with my setup.

I had a look at the source code of the template and file providers, but
couldn't see anywhere that it calls updated_by_last_action(true) :S

My gut feeling is that it is something that has to do with the windows
file
provider.

On 27 December 2012 18:46, Seif Attar iam@seifattar.net wrote:

Thank you both for taking the time to explain this, I get it now.

To get the teamcity_build resource to notify, all I need to do is call
@new_resource.updated_by_last_action(true)

Thanks for the advice on the idempotency, I think I will need to take it
with the original cookbook author as he had provided a parameter
'overwrite'
on the resource, which currently throws an exception if overwrite is
false
and the file already existed. I guess to keep with current behaviour:

If the files doesn't exist -> download and set updated
files exists and override is false -> throw
file exists, override is true and checksums differ -> download and set
updated
file exists, override is true and checksums differ -> do nothing and
don't
set updated.

Had a quick look at adding basic auth to remote_file and it seems it
would
require changes to Chef::Rest as well, too risky for my basic ruby
skills :slight_smile:
copying the checksum code into those recipes would be easier.

Thanks again,
Seif

On 27 December 2012 18:12, Sean OMeara someara@gmail.com wrote:

Hi Seif.

It turns out that I gave you the wrong advice, since I didn't actually
dig into the teamcity cookbook code (like I am now.)

The gist I sent you was a workaround for when you're using nested Chef
resources in LWRP code. It would help in a situation like this:

cookbooks/custom/providers/foo.rb

action :download do

directory "/some/dir" do
action :create
owner "fonzi"

remote_file "somefile" do
source "http://www.example.com/somefile.tgz"
checksum "123456"
end

end

^ In that example, "directory" and "remote_file" are Chef resources.
One of the contracts that a Chef provider has to fulfill is that it
will set the new_resource.updated_by_last_action(true) it they make a
change to the system.

If 'directory[/some/dir]' had to be repaired in any way, the
directory provider setting that flag is what allows notifications to
happen.

if 'remote_file[somefile]' had to be repaired, the same idea applies.

The bug the gist worked around was that the custom_foo resource
defined in the LWRP code could not inherit the updated status of the
nested 'directory' or 'remote_file' resources.

While good to know, this is now the problem you have.

The work you're doing is implemented in two pure-ruby library
functions that ship with the cookbook.

initialize_connection
download_all

Since these are not idempotent Chef resources, the work-around does not
apply.
Please feel free to un-cargo-cult your code. My bad.

What you really need to do is setup tests to determine if you need to
"fix" a teamcity_build resource on disk.

If the resource needs to be repaired, THEN do the work:
initialize_connection
download_all
new_resource.updated_by_last_action(true)

That will allow notifications to work in the default run_context

This deck does a really good job at explaining how it all works.
Lwrp presentation | PPT

-s

On Thu, Dec 27, 2012 at 12:32 PM, Seif Attar iam@seifattar.net
wrote:

Thanks Sean,

My ruby skills are limited, and I didn't quite get that gist or what
it
was
doing, but I did manage to get something working:

Workaround for LWRP DSL bug · seif/teamcity@125633d · GitHub

I am not sure what

if sub_run_context.resource_collection.any?(&:updated?)

Is supposed to do, as it was always returning false for me. What
effects
will removing it do? does the provider look sane to you now?

Much appreciated,

Seif

On 27 December 2012 15:52, Sean OMeara someara@gmail.com wrote:

Hi Seif!

This is a bug in the LWRP DSL.

You can read about it here:
http://tickets.opscode.com/browse/CHEF-3681

There is a fix coming in Chef 11 which will require a small changes
to
the all LWRP code including teamcity and windows cookbooks to make
notifications work.

In the meantime, you modify the teamcity_build LWRP and cargo-cult
the
pattern referenced in the above ticket.

Intelligent Provider Action · GitHub

Good luck and let me know if you have any questions!

-s

On Thu, Dec 27, 2012 at 10:43 AM, Seif Attar iam@seifattar.net
wrote:

Hello,

I am writing a recipe that calls 3 resources, the first resource
has
conditional execution and a notifies, goal being that the other 2
dont
run
unless this one passed the condition.

I can see the first resource being run, but it doesn't seem to be
notifying,
can someone help figure out what I am messing up? is there a
better
way
to
go about this?

Here is a gist of the recipe, and the debug output from Chef
10.16.4:

default.rb · GitHub

Any help appreciated.

Thanks

10

On Friday, December 28, 2012 at 9:57 AM, Seif Attar wrote:

Sure.

template resource not notifying · GitHub

Thanks Sean. I managed to find the code that calls updated_by_last_action and I don't see any code around win32 in there, my theory failed, I think.

What version of Chef is this?

--
Daniel DeLeo

11

On 28 December 2012 18:45, Daniel DeLeo dan@kallistec.com wrote:

What version of Chef is this?

10.16.4

12

Okay so If I'm reading this right -

'teamcity_build[download-artefact]' notifies ->
'windows_zipfile[unzip_artefacts]' notifies ->
'template[CHEF.js]' notifies ->
'windows_batch[run_dropkick]'

... everything is being notified correctly except the windows_batch resource...

Quickly experimenting with chained notifications...

I can get the chaining to execute when modifying content in
template[file1], but not on perms.
So... there's a bug. (I'll file it later).

Is your template resource actually fixing itself?
It has a default action of :nothing, which you are notifying to
:create... but the :create action is still convergent (idempotent),
and the windows_batch resource will only get notified if it has to
repair a file on disk.

You may look at notifying windows_batch from either windows_zipfile or
teamcity_build directly instead of trying to chain them like that. You
can put multiple notifies and subscribe statements on a resource, in
case you didn't know =)

-s

On Fri, Dec 28, 2012 at 12:57 PM, Seif Attar iam@seifattar.net wrote:

Sure.

template resource not notifying · GitHub

Thanks Sean. I managed to find the code that calls updated_by_last_action
and I don't see any code around win32 in there, my theory failed, I think.

On 28 December 2012 17:49, Sean OMeara someara@gmail.com wrote:

Can you gist a code snippet?
-s

On Fri, Dec 28, 2012 at 12:47 PM, Seif Attar iam@seifattar.net wrote:

I seem to be getting the same problem with the template resource, the
resource runs, I can see in the log output that the content has been
updated, but the notifies don't trigger.

Is it possible that I am suffering from the same bug? I can't imagine
that a
basic resource like template would have a problem with notifications,
there
must be something with my setup.

I had a look at the source code of the template and file providers, but
couldn't see anywhere that it calls updated_by_last_action(true) :S

My gut feeling is that it is something that has to do with the windows
file
provider.

On 27 December 2012 18:46, Seif Attar iam@seifattar.net wrote:

Thank you both for taking the time to explain this, I get it now.

To get the teamcity_build resource to notify, all I need to do is call
@new_resource.updated_by_last_action(true)

Thanks for the advice on the idempotency, I think I will need to take
it
with the original cookbook author as he had provided a parameter
'overwrite'
on the resource, which currently throws an exception if overwrite is
false
and the file already existed. I guess to keep with current behaviour:

If the files doesn't exist -> download and set updated
files exists and override is false -> throw
file exists, override is true and checksums differ -> download and set
updated
file exists, override is true and checksums differ -> do nothing and
don't
set updated.

Had a quick look at adding basic auth to remote_file and it seems it
would
require changes to Chef::Rest as well, too risky for my basic ruby
skills :slight_smile:
copying the checksum code into those recipes would be easier.

Thanks again,
Seif

On 27 December 2012 18:12, Sean OMeara someara@gmail.com wrote:

Hi Seif.

It turns out that I gave you the wrong advice, since I didn't actually
dig into the teamcity cookbook code (like I am now.)

The gist I sent you was a workaround for when you're using nested Chef
resources in LWRP code. It would help in a situation like this:

cookbooks/custom/providers/foo.rb

action :download do

directory "/some/dir" do
action :create
owner "fonzi"

remote_file "somefile" do
source "http://www.example.com/somefile.tgz"
checksum "123456"
end

end

^ In that example, "directory" and "remote_file" are Chef resources.
One of the contracts that a Chef provider has to fulfill is that it
will set the new_resource.updated_by_last_action(true) it they make a
change to the system.

If 'directory[/some/dir]' had to be repaired in any way, the
directory provider setting that flag is what allows notifications to
happen.

if 'remote_file[somefile]' had to be repaired, the same idea applies.

The bug the gist worked around was that the custom_foo resource
defined in the LWRP code could not inherit the updated status of the
nested 'directory' or 'remote_file' resources.

While good to know, this is now the problem you have.

The work you're doing is implemented in two pure-ruby library
functions that ship with the cookbook.

initialize_connection
download_all

Since these are not idempotent Chef resources, the work-around does
not
apply.
Please feel free to un-cargo-cult your code. My bad.

What you really need to do is setup tests to determine if you need to
"fix" a teamcity_build resource on disk.

If the resource needs to be repaired, THEN do the work:
initialize_connection
download_all
new_resource.updated_by_last_action(true)

That will allow notifications to work in the default run_context

This deck does a really good job at explaining how it all works.
Lwrp presentation | PPT

-s

On Thu, Dec 27, 2012 at 12:32 PM, Seif Attar iam@seifattar.net
wrote:

Thanks Sean,

My ruby skills are limited, and I didn't quite get that gist or what
it
was
doing, but I did manage to get something working:

Workaround for LWRP DSL bug · seif/teamcity@125633d · GitHub

I am not sure what

if sub_run_context.resource_collection.any?(&:updated?)

Is supposed to do, as it was always returning false for me. What
effects
will removing it do? does the provider look sane to you now?

Much appreciated,

Seif

On 27 December 2012 15:52, Sean OMeara someara@gmail.com wrote:

Hi Seif!

This is a bug in the LWRP DSL.

You can read about it here:
http://tickets.opscode.com/browse/CHEF-3681

There is a fix coming in Chef 11 which will require a small changes
to
the all LWRP code including teamcity and windows cookbooks to make
notifications work.

In the meantime, you modify the teamcity_build LWRP and cargo-cult
the
pattern referenced in the above ticket.

Intelligent Provider Action · GitHub

Good luck and let me know if you have any questions!

-s

On Thu, Dec 27, 2012 at 10:43 AM, Seif Attar iam@seifattar.net
wrote:

Hello,

I am writing a recipe that calls 3 resources, the first resource
has
conditional execution and a notifies, goal being that the other 2
dont
run
unless this one passed the condition.

I can see the first resource being run, but it doesn't seem to be
notifying,
can someone help figure out what I am messing up? is there a
better
way
to
go about this?

Here is a gist of the recipe, and the debug output from Chef
10.16.4:

default.rb · GitHub

Any help appreciated.

Thanks

13

On Friday, December 28, 2012 at 10:49 AM, Seif Attar wrote:

On 28 December 2012 18:45, Daniel DeLeo <dan@kallistec.com (mailto:dan@kallistec.com)> wrote:

What version of Chef is this?

10.16.4

Hrm, I wonder if it has something to do with the notifications being chained?

Can you assign your template resource to a variable, e.g.:

template_resource = template "path/to/file" do # etc.

And then, at the end of the recipe, add a ruby_block resource to print out template_resource.updated_by_last_action? and template_resource.immediate_notifications?

--
Daniel DeLeo

14

On Fri, 2012-12-28 at 14:26 -0500, Sean OMeara wrote:
Okay so If I'm reading this right -

'teamcity_build[download-artefact]' notifies ->
'windows_zipfile[unzip_artefacts]' notifies ->
'template[CHEF.js]' notifies ->
'windows_batch[run_dropkick]'

... everything is being notified correctly except the windows_batch
resource...

Quickly experimenting with chained notifications...

gist:4400651 · GitHub

I can get the chaining to execute when modifying content in
template[file1], but not on perms.
So... there's a bug. (I'll file it later).

Is your template resource actually fixing itself?
It has a default action of :nothing, which you are notifying to
:create... but the :create action is still convergent (idempotent),
and the windows_batch resource will only get notified if it has to
repair a file on disk.

Sad to hear about the perms bug, but I am not asking for any permissions to
be changed. The file is being updated with content from the template, and
the log file says that content was updated:

The file that I am fixing already exists in the extracted zip with
different content to what the template generates, and it does get updated.
When the file is fixed, the permissions do change on it from what it was
originally, doesn't bother me, but maybe that is causing the problem since
you found a bug with perms?

You may look at notifying windows_batch from either windows_zipfile or

teamcity_build directly instead of trying to chain them like that. You
can put multiple notifies and subscribe statements on a resource, in
case you didn't know =)

Thanks, I didn't know you can have multiple notifies, but I don't think
that would work in my situation, the teamcity resource downloads build
artefacts, when then get upzipped, the template generates CHEF.js which
contains data that is needed in the execute. So each one of those resources
does have a dependency on the one before.

15

Thanks Daniel, I’ll try this out soon and see what the output is.

BTW, what is it with all the Re: Re: Res? Will the mailing list software
break thread ordering if they are removed? Can’t it be fixed

16

On Friday, December 28, 2012 at 12:37 PM, Seif Attar wrote:

Thanks Daniel, I'll try this out soon and see what the output is.
Looking forward to it. This behavior is pretty odd considering that we have a test for just the behavior you're using:

BTW, what is it with all the Re: Re: Res? Will the mailing list software break thread ordering if they are removed? Can't it be fixed

Not sure. My mail client adds them by default, but they started showing up before I joined the conversation. I don't know anything about our list software, so I'm not sure if it's the culprit.

--
Daniel DeLeo

17

On 28 December 2012 21:26, Daniel DeLeo dan@kallistec.com wrote:

Looking forward to it. This behavior is pretty odd considering that we
have a test for just the behavior you're using:

chef/spec/unit/runner_spec.rb at main · chef/chef · GitHub

Yeah, I think it must be something to do with windows, I have a lot of
failing tests on windows.

I have updated the gist with the output of those values:

Thanks btw, I didn't you can specify a line range in github, very handy.

I forgot how I got there, but I think this is the cause of the problem:

The file resource suffers from the same bug (tried replacing the template
with a file :delete and that didn't trigger notify either)

Not sure. My mail client adds them by default, but they started showing up

before I joined the conversation. I don't know anything about our list
software, so I'm not sure if it's the culprit.

I use the gmail web ui on windows and I never saw it do such a thing, but
it adds the Re: on this mailing list even when I reply to emails. The
option to change the subject in the gmail client says that will start a new
conversation. Weird, as I haven't seen such behaviour with other mailing
lists.

Again, thanks a lot for you help.

18

So is this a bug I should I raise a ticket for?

I am going to start working on creating a custom resource for the recipe I
was working on, since it kind of works in principle, but instead of
chaining notifies I am going to just have the calls into other resources
inside an if statement to ensure idempotency.

something like

if(!file.Exists(teamcity_download_target_path) and !checksumsMatch)
teacity…
unzip…
template…
execute…
end

unless there is a better way to work around this?

19

On Wednesday, January 2, 2013 at 6:55 AM, Seif Attar wrote:

So is this a bug I should I raise a ticket for?
This is a bug, but still not sure exactly what's causing it.

In your previous email, you linked This gist: template resource not notifying · GitHub

This shows that by the time you get to notifying the template, it doesn't have any notifications set.

For sanity's sake can you copy the debug code you have in the ruby block to outside of the ruby block (so it runs during compile time)?

Thanks,

Dan DeLeo

20

On 2 January 2013 16:43, Daniel DeLeo dan@kallistec.com wrote:

For sanity's sake can you copy the debug code you have in the ruby block
to outside of the ruby block (so it runs during compile time)?

Sure, updated gist:

So you don't think it has anything to do with

the comment says that reporting won't work.