Variable scope in LWRPs


#1

Hi all,

I have the following problem and I think its because I do not understand the variable scopes in Chef LWRPs.

You can look at the cookbook at https://github.com/jkleinlercher/chef_problems and simply test the behaviour with kitchen.

Even though I define the following very similar resources, I recognized, that the attribute ‘colors’ inherits from the previous defined resources. The attribute ‘colors’ is defaulted to “[‘blue’]” and in the provider the element ‘magenta’ is added to the array. However, the second and third resource inherit the whole array from the previous resource. That is quite weird to me …

Resource definition in recipes/default.rb:

chef_problems_problem1 “test1” do
address "myaddress1"
action :install
end

chef_problems_problem1 “test2” do
address "myaddress2"
action :install
end

chef_problems_problem1 “test3” do
address "myaddress3"
action :install
end

In the output of the kitchen converge you see that the variable new_resource.colors inherits the values of the previous resources:

 * chef_problems_problem1[test1] action install

new_resource.colors at the beginning: [“blue”]

Values of local variables:
address: myaddress1
colors: [“blue”]

adding magenta to local variable colors

colors after adding magenta: [“blue”, “magenta”]
new_resource.colors at the end: [“blue”, “magenta”]

 * chef_problems_problem1[test2] action install

new_resource.colors at the beginning: [“blue”, “magenta”]

Values of local variables:
address: myaddress2
colors: [“blue”, “magenta”]

adding magenta to local variable colors

colors after adding magenta: [“blue”, “magenta”, “magenta”]
new_resource.colors at the end: [“blue”, “magenta”, “magenta”]

 * chef_problems_problem1[test3] action install

new_resource.colors at the beginning: [“blue”, “magenta”, “magenta”]

Values of local variables:
address: myaddress3
colors: [“blue”, “magenta”, “magenta”]

adding magenta to local variable colors

colors after adding magenta: [“blue”, “magenta”, “magenta”, “magenta”]
new_resource.colors at the end: [“blue”, “magenta”, “magenta”, “magenta”]

Maybe you can help me to find where the problem is here.

best regards,
Johannes


#2

On Sunday, May 10, 2015 at 12:07 PM, Johannes Kleinlercher wrote:

Hi all,
I have the following problem and I think its because I do not understand the variable scopes in Chef LWRPs.
You can look at the cookbook at https://github.com/jkleinlercher/chef_problems and simply test the behaviour with kitchen.
Even though I define the following very similar resources, I recognized, that the attribute ‘colors’ inherits from the previous defined resources. The attribute ‘colors’ is defaulted to “[‘blue’]” and in the provider the element ‘magenta’ is added to the array. However, the second and third resource inherit the whole array from the previous resource. That is quite weird to me …
Resource definition in recipes/default.rb:
chef_problems_problem1 “test1” do address “myaddress1” action :install end chef_problems_problem1 “test2” do address “myaddress2” action :install end chef_problems_problem1 “test3” do address “myaddress3” action :install end
In the output of the kitchen converge you see that the variable new_resource.colors inherits the values of the previous resources:

  • chef_problems_problem1[test1] action install new_resource.colors at the beginning: [“blue”] Values of local variables: address: myaddress1 colors: [“blue”] adding magenta to local variable colors colors after adding magenta: [“blue”, “magenta”] new_resource.colors at the end: [“blue”, “magenta”] * chef_problems_problem1[test2] action install new_resource.colors at the beginning: [“blue”, “magenta”] Values of local variables: address: myaddress2 colors: [“blue”, “magenta”] adding magenta to local variable colors colors after adding magenta: [“blue”, “magenta”, “magenta”] new_resource.colors at the end: [“blue”, “magenta”, “magenta”] * chef_problems_problem1[test3] action install new_resource.colors at the beginning: [“blue”, “magenta”, “magenta”] Values of local variables: address: myaddress3 colors: [“blue”, “magenta”, “magenta”] adding magenta to local variable colors colors after adding magenta: [“blue”, “magenta”, “magenta”, “magenta”] new_resource.colors at the end: [“blue”, “magenta”, “magenta”, “magenta”]
    Maybe you can help me to find where the problem is here.

best regards,
Johannes

I think this is a bug. I think default value in this line: https://github.com/jkleinlercher/chef_problems/blob/fe42785a400cc154b75112b2d92f6136c60c0f08/resources/problem1.rb#L7 is reused for each resource. Then when you mutate it in this line: https://github.com/jkleinlercher/chef_problems/blob/fe42785a400cc154b75112b2d92f6136c60c0f08/providers/problem1.rb#L12 you’re updating the one array object.

You can confirm this by printing the object_id of the array in your debug logging. You should be able to work around it by calling dup on the array, or setting the default value in an initialize method instead of via the DSL.


Daniel DeLeo


#3

Oh wow, that looks super familiar, we had the same problem in
Chef::Config. I wonder if the same solution is warranted in Chef?

On Tue, May 12, 2015 at 7:24 PM, Daniel DeLeo dan@kallistec.com wrote:

On Sunday, May 10, 2015 at 12:07 PM, Johannes Kleinlercher wrote:

Hi all,
I have the following problem and I think its because I do not understand
the variable scopes in Chef LWRPs.
You can look at the cookbook at
https://github.com/jkleinlercher/chef_problems and simply test the
behaviour with kitchen.
Even though I define the following very similar resources, I recognized,
that the attribute ‘colors’ inherits from the previous defined resources.
The attribute ‘colors’ is defaulted to “[‘blue’]” and in the provider the
element ‘magenta’ is added to the array. However, the second and third
resource inherit the whole array from the previous resource. That is quite
weird to me …
Resource definition in recipes/default.rb:
chef_problems_problem1 “test1” do address “myaddress1” action :install
end chef_problems_problem1 “test2” do address “myaddress2” action :install
end chef_problems_problem1 “test3” do address “myaddress3” action :install
end
In the output of the kitchen converge you see that the variable
new_resource.colors inherits the values of the previous resources:

  • chef_problems_problem1[test1] action install new_resource.colors at
    the beginning: [“blue”] Values of local variables: address: myaddress1
    colors: [“blue”] adding magenta to local variable colors colors after
    adding magenta: [“blue”, “magenta”] new_resource.colors at the end:
    [“blue”, “magenta”] * chef_problems_problem1[test2] action install
    new_resource.colors at the beginning: [“blue”, “magenta”] Values of local
    variables: address: myaddress2 colors: [“blue”, “magenta”] adding magenta
    to local variable colors colors after adding magenta: [“blue”, “magenta”,
    “magenta”] new_resource.colors at the end: [“blue”, “magenta”, “magenta”] *
    chef_problems_problem1[test3] action install new_resource.colors at the
    beginning: [“blue”, “magenta”, “magenta”] Values of local variables:
    address: myaddress3 colors: [“blue”, “magenta”, “magenta”] adding magenta
    to local variable colors colors after adding magenta: [“blue”, “magenta”,
    “magenta”, “magenta”] new_resource.colors at the end: [“blue”, “magenta”,
    “magenta”, “magenta”]
    Maybe you can help me to find where the problem is here.

best regards,
Johannes

I think this is a bug. I think default value in this line:
https://github.com/jkleinlercher/chef_problems/blob/fe42785a400cc154b75112b2d92f6136c60c0f08/resources/problem1.rb#L7
is reused for each resource. Then when you mutate it in this line:
https://github.com/jkleinlercher/chef_problems/blob/fe42785a400cc154b75112b2d92f6136c60c0f08/providers/problem1.rb#L12
you’re updating the one array object.

You can confirm this by printing the object_id of the array in your debug
logging. You should be able to work around it by calling dup on the array,
or setting the default value in an initialize method instead of via the DSL.


Daniel DeLeo


#4

I already mentioned this very similiar problem https://tickets.opscode.com/browse/CHEF-4608 at
http://stackoverflow.com/questions/30150766/variable-scope-in-chef-lwrps

However, I wonder if it gets fixed? I think normal chef users do not expect this behaviour if they are not into ruby and there is also no documentation anywhere according to this fact …

Von meinem iPhone gesendet

Am 13.05.2015 um 06:14 schrieb John Keiser <jkeiser@chef.iomailto:jkeiser@chef.io>:

Oh wow, that looks super familiar, we had the same problem in Chef::Config. I wonder if the same solution is warranted in Chef?

On Tue, May 12, 2015 at 7:24 PM, Daniel DeLeo <dan@kallistec.commailto:dan@kallistec.com> wrote:

On Sunday, May 10, 2015 at 12:07 PM, Johannes Kleinlercher wrote:

Hi all,
I have the following problem and I think its because I do not understand the variable scopes in Chef LWRPs.
You can look at the cookbook at https://github.com/jkleinlercher/chef_problems and simply test the behaviour with kitchen.
Even though I define the following very similar resources, I recognized, that the attribute ‘colors’ inherits from the previous defined resources. The attribute ‘colors’ is defaulted to “[‘blue’]” and in the provider the element ‘magenta’ is added to the array. However, the second and third resource inherit the whole array from the previous resource. That is quite weird to me …
Resource definition in recipes/default.rb:
chef_problems_problem1 “test1” do address “myaddress1” action :install end chef_problems_problem1 “test2” do address “myaddress2” action :install end chef_problems_problem1 “test3” do address “myaddress3” action :install end
In the output of the kitchen converge you see that the variable new_resource.colors inherits the values of the previous resources:

  • chef_problems_problem1[test1] action install new_resource.colors at the beginning: [“blue”] Values of local variables: address: myaddress1 colors: [“blue”] adding magenta to local variable colors colors after adding magenta: [“blue”, “magenta”] new_resource.colors at the end: [“blue”, “magenta”] * chef_problems_problem1[test2] action install new_resource.colors at the beginning: [“blue”, “magenta”] Values of local variables: address: myaddress2 colors: [“blue”, “magenta”] adding magenta to local variable colors colors after adding magenta: [“blue”, “magenta”, “magenta”] new_resource.colors at the end: [“blue”, “magenta”, “magenta”] * chef_problems_problem1[test3] action install new_resource.colors at the beginning: [“blue”, “magenta”, “magenta”] Values of local variables: address: myaddress3 colors: [“blue”, “magenta”, “magenta”] adding magenta to local variable colors colors after adding magenta: [“blue”, “magenta”, “magenta”, “magenta”] new_resource.colors at the end: [“blue”, “magenta”, “magenta”, “magenta”]
    Maybe you can help me to find where the problem is here.

best regards,
Johannes

I think this is a bug. I think default value in this line: https://github.com/jkleinlercher/chef_problems/blob/fe42785a400cc154b75112b2d92f6136c60c0f08/resources/problem1.rb#L7 is reused for each resource. Then when you mutate it in this line: https://github.com/jkleinlercher/chef_problems/blob/fe42785a400cc154b75112b2d92f6136c60c0f08/providers/problem1.rb#L12 you’re updating the one array object.

You can confirm this by printing the object_id of the array in your debug logging. You should be able to work around it by calling dup on the array, or setting the default value in an initialize method instead of via the DSL.


Daniel DeLeo


#5

You can use the Poise helpers to fix this too, https://github.com/poise/poise#lazy-initializers. Possibly a decent generic solution that could get back ported to Chef itself.

–Noah

On May 13, 2015, at 8:50 AM, Johannes Kleinlercher johannes@kleinlercher.at wrote:

I already mentioned this very similiar problem https://tickets.opscode.com/browse/CHEF-4608 at
http://stackoverflow.com/questions/30150766/variable-scope-in-chef-lwrps

However, I wonder if it gets fixed? I think normal chef users do not expect this behaviour if they are not into ruby and there is also no documentation anywhere according to this fact …

Von meinem iPhone gesendet

Am 13.05.2015 um 06:14 schrieb John Keiser jkeiser@chef.io:

Oh wow, that looks super familiar, we had the same problem in Chef::Config. I wonder if the same solution is warranted in Chef?

On Tue, May 12, 2015 at 7:24 PM, Daniel DeLeo dan@kallistec.com wrote:

On Sunday, May 10, 2015 at 12:07 PM, Johannes Kleinlercher wrote:

Hi all,
I have the following problem and I think its because I do not understand the variable scopes in Chef LWRPs.
You can look at the cookbook at https://github.com/jkleinlercher/chef_problems and simply test the behaviour with kitchen.
Even though I define the following very similar resources, I recognized, that the attribute ‘colors’ inherits from the previous defined resources. The attribute ‘colors’ is defaulted to “[‘blue’]” and in the provider the element ‘magenta’ is added to the array. However, the second and third resource inherit the whole array from the previous resource. That is quite weird to me …
Resource definition in recipes/default.rb:
chef_problems_problem1 “test1” do address “myaddress1” action :install end chef_problems_problem1 “test2” do address “myaddress2” action :install end chef_problems_problem1 “test3” do address “myaddress3” action :install end
In the output of the kitchen converge you see that the variable new_resource.colors inherits the values of the previous resources:

  • chef_problems_problem1[test1] action install new_resource.colors at the beginning: [“blue”] Values of local variables: address: myaddress1 colors: [“blue”] adding magenta to local variable colors colors after adding magenta: [“blue”, “magenta”] new_resource.colors at the end: [“blue”, “magenta”] * chef_problems_problem1[test2] action install new_resource.colors at the beginning: [“blue”, “magenta”] Values of local variables: address: myaddress2 colors: [“blue”, “magenta”] adding magenta to local variable colors colors after adding magenta: [“blue”, “magenta”, “magenta”] new_resource.colors at the end: [“blue”, “magenta”, “magenta”] * chef_problems_problem1[test3] action install new_resource.colors at the beginning: [“blue”, “magenta”, “magenta”] Values of local variables: address: myaddress3 colors: [“blue”, “magenta”, “magenta”] adding magenta to local variable colors colors after adding magenta: [“blue”, “magenta”, “magenta”, “magenta”] new_resource.colors at the end: [“blue”, “magenta”, “magenta”, “magenta”]
    Maybe you can help me to find where the problem is here.

best regards,
Johannes

I think this is a bug. I think default value in this line: https://github.com/jkleinlercher/chef_problems/blob/fe42785a400cc154b75112b2d92f6136c60c0f08/resources/problem1.rb#L7 is reused for each resource. Then when you mutate it in this line: https://github.com/jkleinlercher/chef_problems/blob/fe42785a400cc154b75112b2d92f6136c60c0f08/providers/problem1.rb#L12 you’re updating the one array object.

You can confirm this by printing the object_id of the array in your debug logging. You should be able to work around it by calling dup on the array, or setting the default value in an initialize method instead of via the DSL.


Daniel DeLeo


#6

If defaults aren’t already lazy, they absolutely should be, yeah.

On Wed, May 13, 2015 at 3:59 AM, Noah Kantrowitz noah@coderanger.net
wrote:

You can use the Poise helpers to fix this too,
https://github.com/poise/poise#lazy-initializers. Possibly a decent
generic solution that could get back ported to Chef itself.

–Noah

On May 13, 2015, at 8:50 AM, Johannes Kleinlercher <
johannes@kleinlercher.at> wrote:

I already mentioned this very similiar problem
https://tickets.opscode.com/browse/CHEF-4608 at
http://stackoverflow.com/questions/30150766/variable-scope-in-chef-lwrps

However, I wonder if it gets fixed? I think normal chef users do not
expect this behaviour if they are not into ruby and there is also no
documentation anywhere according to this fact …

Von meinem iPhone gesendet

Am 13.05.2015 um 06:14 schrieb John Keiser jkeiser@chef.io:

Oh wow, that looks super familiar, we had the same problem in
Chef::Config. I wonder if the same solution is warranted in Chef?

On Tue, May 12, 2015 at 7:24 PM, Daniel DeLeo dan@kallistec.com
wrote:

On Sunday, May 10, 2015 at 12:07 PM, Johannes Kleinlercher wrote:

Hi all,
I have the following problem and I think its because I do not
understand the variable scopes in Chef LWRPs.

You can look at the cookbook at
https://github.com/jkleinlercher/chef_problems and simply test the
behaviour with kitchen.

Even though I define the following very similar resources, I
recognized, that the attribute ‘colors’ inherits from the previous defined
resources. The attribute ‘colors’ is defaulted to “[‘blue’]” and in the
provider the element ‘magenta’ is added to the array. However, the second
and third resource inherit the whole array from the previous resource. That
is quite weird to me …

Resource definition in recipes/default.rb:
chef_problems_problem1 “test1” do address “myaddress1” action
:install end chef_problems_problem1 “test2” do address “myaddress2” action
:install end chef_problems_problem1 “test3” do address “myaddress3” action
:install end

In the output of the kitchen converge you see that the variable
new_resource.colors inherits the values of the previous resources:

  • chef_problems_problem1[test1] action install new_resource.colors at
    the beginning: [“blue”] Values of local variables: address: myaddress1
    colors: [“blue”] adding magenta to local variable colors colors after
    adding magenta: [“blue”, “magenta”] new_resource.colors at the end:
    [“blue”, “magenta”] * chef_problems_problem1[test2] action install
    new_resource.colors at the beginning: [“blue”, “magenta”] Values of local
    variables: address: myaddress2 colors: [“blue”, “magenta”] adding magenta
    to local variable colors colors after adding magenta: [“blue”, “magenta”,
    “magenta”] new_resource.colors at the end: [“blue”, “magenta”, “magenta”] *
    chef_problems_problem1[test3] action install new_resource.colors at the
    beginning: [“blue”, “magenta”, “magenta”] Values of local variables:
    address: myaddress3 colors: [“blue”, “magenta”, “magenta”] adding magenta
    to local variable colors colors after adding magenta: [“blue”, “magenta”,
    “magenta”, “magenta”] new_resource.colors at the end: [“blue”, “magenta”,
    “magenta”, “magenta”]

Maybe you can help me to find where the problem is here.

best regards,
Johannes

I think this is a bug. I think default value in this line:
https://github.com/jkleinlercher/chef_problems/blob/fe42785a400cc154b75112b2d92f6136c60c0f08/resources/problem1.rb#L7
is reused for each resource. Then when you mutate it in this line:
https://github.com/jkleinlercher/chef_problems/blob/fe42785a400cc154b75112b2d92f6136c60c0f08/providers/problem1.rb#L12
you’re updating the one array object.

You can confirm this by printing the object_id of the array in your
debug logging. You should be able to work around it by calling dup on the
array, or setting the default value in an initialize method instead of via
the DSL.


Daniel DeLeo