This seems like such a simple problem and something that everyone should
come across at one point or another.
Here’s a short snippet of code that illustrates the issue I’m having:
(1…2).each do |count|
ruby_block “ls /tmp” do
block do
node.set[‘test1’][‘lsdir’] = %x[ls /tmp]
end
end
file “/tmp/file#{count}.txt” do
content node[‘test1’][‘lsdir’]
end
end
So, at the end of the run, both /tmp/file1.txt and /tmp/file2.txt are
empty. I had hoped for /tmp/file2.txt to show that '/tmp/file1.txt now
exists.
I know this is because the value of attributes are evaluated during the
compile phase at the beginning of the chef run and at that time, the value
of node[‘test1’][‘lsdir’] is nil. Also, this same issue arises when trying
to use a variable instead of an attribute.
How can I get something like this to execute as I would have hoped? This
has to be a common pattern; set an attribute, use it, modify the attribute,
use the new value. I know there is the new ‘lazy’ delayed evaluator, but I
can only get that to work when used in a value for a resource attribute.
value of content is getting evaluated at compile time. its an attributed
value and determined soon as the resource is created. while you are setting
the node attribute via a ruby block, which gets evaluated at the compile
time. some of the options. Im not sure if both assigning, mutating and then
using the attribute is common pattern (or a good pattern at all).
i think if you move the file resource inside a lwrp and then notify that
resource, that might work. because in that case the file resource will be
created inside the provider's action block (which is evaluated only during
the execution phase), hence u'll have the node.foo.bar value as you
expected. there are several other ways to do the same trick, but this is
the cleanest i can think of ..
cheers
ranjib
This seems like such a simple problem and something that everyone should
come across at one point or another.
Here's a short snippet of code that illustrates the issue I'm having:
(1..2).each do |count|
ruby_block "ls /tmp" do
block do
node.set['test1']['lsdir'] = %x[ls /tmp]
end
end
file "/tmp/file#{count}.txt" do
content node['test1']['lsdir']
end
end
So, at the end of the run, both /tmp/file1.txt and /tmp/file2.txt are
empty. I had hoped for /tmp/file2.txt to show that '/tmp/file1.txt now
exists.
I know this is because the value of attributes are evaluated during the
compile phase at the beginning of the chef run and at that time, the value
of node['test1']['lsdir'] is nil. Also, this same issue arises when trying
to use a variable instead of an attribute.
How can I get something like this to execute as I would have hoped? This
has to be a common pattern; set an attribute, use it, modify the attribute,
use the new value. I know there is the new 'lazy' delayed evaluator, but I
can only get that to work when used in a value for a resource attribute.
This seems like such a simple problem and something that everyone should come across at one point or another.
Here's a short snippet of code that illustrates the issue I'm having:
(1..2).each do |count|
ruby_block "ls /tmp" do
block do
node.set['test1']['lsdir'] = %x[ls /tmp]
end
end
file "/tmp/file#{count}.txt" do
content node['test1']['lsdir']
end
end
So, at the end of the run, both /tmp/file1.txt and /tmp/file2.txt are empty. I had hoped for /tmp/file2.txt to show that '/tmp/file1.txt now exists.
I know this is because the value of attributes are evaluated during the compile phase at the beginning of the chef run and at that time, the value of node['test1']['lsdir'] is nil. Also, this same issue arises when trying to use a variable instead of an attribute.
How can I get something like this to execute as I would have hoped? This has to be a common pattern; set an attribute, use it, modify the attribute, use the new value. I know there is the new 'lazy' delayed evaluator, but I can only get that to work when used in a value for a resource attribute.
This seems like such a simple problem and something that everyone should
come across at one point or another.
Here's a short snippet of code that illustrates the issue I'm having:
(1..2).each do |count|
ruby_block "ls /tmp" do
block do
node.set['test1']['lsdir'] = %x[ls /tmp]
end
end
file "/tmp/file#{count}.txt" do
content node['test1']['lsdir']
end
end
So, at the end of the run, both /tmp/file1.txt and /tmp/file2.txt are
empty. I had hoped for /tmp/file2.txt to show that '/tmp/file1.txt now
exists.
I know this is because the value of attributes are evaluated during the
compile phase at the beginning of the chef run and at that time, the value
of node['test1']['lsdir'] is nil. Also, this same issue arises when trying
to use a variable instead of an attribute.
How can I get something like this to execute as I would have hoped?
This has to be a common pattern; set an attribute, use it, modify the
attribute, use the new value. I know there is the new 'lazy' delayed
evaluator, but I can only get that to work when used in a value for a
resource attribute.
Thanks everyone for the responses. So... it seems the consensus is that
this is best done in a LWRP. I had considered doing that, it just seemed
like overkill for what should be a simple operation; that being, change the
value of a variable(or attribute) and use that variable again later in the
code.
@Torben, Yes, I would probably get warnings for reusing a resource if I
didn't give it a unique name in the loop, but that would be the extent of
the impact.
John
On Tue, Dec 17, 2013 at 12:21 AM, Torben Knerr ukio@gmx.de wrote:
Another possible caveat:
Shouldn't it be 'ruby_block "ls /tmp #{count}" do' ?
-Torben
On Dec 17, 2013 6:39 AM, "Noah Kantrowitz" noah@coderanger.net wrote:
This seems like such a simple problem and something that everyone
should come across at one point or another.
Here's a short snippet of code that illustrates the issue I'm having:
(1..2).each do |count|
ruby_block "ls /tmp" do
block do
node.set['test1']['lsdir'] = %x[ls /tmp]
end
end
file "/tmp/file#{count}.txt" do
content node['test1']['lsdir']
end
end
So, at the end of the run, both /tmp/file1.txt and /tmp/file2.txt are
empty. I had hoped for /tmp/file2.txt to show that '/tmp/file1.txt now
exists.
I know this is because the value of attributes are evaluated during the
compile phase at the beginning of the chef run and at that time, the value
of node['test1']['lsdir'] is nil. Also, this same issue arises when trying
to use a variable instead of an attribute.
How can I get something like this to execute as I would have hoped?
This has to be a common pattern; set an attribute, use it, modify the
attribute, use the new value. I know there is the new 'lazy' delayed
evaluator, but I can only get that to work when used in a value for a
resource attribute.
Thanks everyone for the responses. So… it seems the consensus is that this is best done in a LWRP. I had considered doing that, it just seemed like overkill for what should be a simple operation; that being, change the value of a variable(or attribute) and use that variable again later in the code.
Slightly off-topic, but where are you modifying It may be a style or philosophy thing, but I personally wouldn’t try to modify an attribute once I declare it. I rather that I set my conditions up front in attributes file and maybe override them as necessary within the environment role. Changing them within the recipe strikes me as adding a significant amount of complexity to the code maintenance, as you now have to look at the recipes AND attributes when you modify the cookbook.
Slightly off-topic, but where are you modifying It may be a style or
philosophy thing, but I personally wouldn't try to modify an attribute once
I declare it. I rather that I set my conditions up front in attributes file
and maybe override them as necessary within the environment role. Changing
them within the recipe strikes me as adding a significant amount of
complexity to the code maintenance, as you now have to look at the recipes
AND attributes when you modify the cookbook.
Yes, I agree. Here's the actual scenario that I'm dealing with. I'm
creating some new EBS volumes, once they are created, I store those new
device entries in an attribute. Later in the code, I need to use those new
devices to either format, create a raid, etc. So, I need to save those
device values somewhere during the Chef run and be able to use them later
in the code.
I wonder if there is a chef_sugar-ish solution we could build for
this--it's awfully common. I suspect it is possible to write a
"all_together_resource" resource that let you write:
all_together do
lsdir = 0
(1..2).each do |count|
ruby_block "ls /tmp" do
block do
lsdir = %x[ls /tmp]
end
end
file "/tmp/file#{count}.txt" do
content lsdir
end
end
end
This would create an all_together_resource resource which will execute the
entire block in its action :run, with each resource running immediately as
soon as the resource is declared.
Slightly off-topic, but where are you modifying It may be a style or
philosophy thing, but I personally wouldn't try to modify an attribute once
I declare it. I rather that I set my conditions up front in attributes file
and maybe override them as necessary within the environment role. Changing
them within the recipe strikes me as adding a significant amount of
complexity to the code maintenance, as you now have to look at the recipes
AND attributes when you modify the cookbook.
Yes, I agree. Here's the actual scenario that I'm dealing with. I'm
creating some new EBS volumes, once they are created, I store those new
device entries in an attribute. Later in the code, I need to use those new
devices to either format, create a raid, etc. So, I need to save those
device values somewhere during the Chef run and be able to use them later
in the code.
I wonder if there is a chef_sugar-ish solution we could build for
this--it's awfully common. I suspect it is possible to write a
"all_together_resource" resource that let you write:
all_together do
lsdir = 0
(1..2).each do |count|
ruby_block "ls /tmp" do
block do
lsdir = %x[ls /tmp]
end
end
file "/tmp/file#{count}.txt" do
content lsdir
end
end
end
This would create an all_together_resource resource which will execute the
entire block in its action :run, with each resource running immediately as
soon as the resource is declared.
Slightly off-topic, but where are you modifying It may be a style or
philosophy thing, but I personally wouldn't try to modify an attribute once
I declare it. I rather that I set my conditions up front in attributes file
and maybe override them as necessary within the environment role. Changing
them within the recipe strikes me as adding a significant amount of
complexity to the code maintenance, as you now have to look at the recipes
AND attributes when you modify the cookbook.
Yes, I agree. Here's the actual scenario that I'm dealing with. I'm
creating some new EBS volumes, once they are created, I store those new
device entries in an attribute. Later in the code, I need to use those new
devices to either format, create a raid, etc. So, I need to save those
device values somewhere during the Chef run and be able to use them later
in the code.
I see a couple of more "flexible" options. I think node.run_state would
work here but I'm not positive without testing it.
Another option that I've always found interesting is creating/using a
databag as part of a recipe. It's a bit risky in that the databag is just
out there without a commensurate copy in your repo though.
Really this feels like the job of an LWRP. There's a lot of logic you're
describing that would really muddy a recipe.
Slightly off-topic, but where are you modifying It may be a style or
philosophy thing, but I personally wouldn't try to modify an attribute once
I declare it. I rather that I set my conditions up front in attributes file
and maybe override them as necessary within the environment role. Changing
them within the recipe strikes me as adding a significant amount of
complexity to the code maintenance, as you now have to look at the recipes
AND attributes when you modify the cookbook.
Yes, I agree. Here's the actual scenario that I'm dealing with. I'm
creating some new EBS volumes, once they are created, I store those new
device entries in an attribute. Later in the code, I need to use those new
devices to either format, create a raid, etc. So, I need to save those
device values somewhere during the Chef run and be able to use them later
in the code.
Isn't the issue only that %x[ls /tmp] is being evaluated at compile time?
Avoid that notation and you'll be able to do what you intend. Use "system"
instead and it will be evaluated as part of the ruby block.
Slightly off-topic, but where are you modifying It may be a style or
philosophy thing, but I personally wouldn't try to modify an attribute once
I declare it. I rather that I set my conditions up front in attributes file
and maybe override them as necessary within the environment role. Changing
them within the recipe strikes me as adding a significant amount of
complexity to the code maintenance, as you now have to look at the recipes
AND attributes when you modify the cookbook.
Yes, I agree. Here's the actual scenario that I'm dealing with. I'm
creating some new EBS volumes, once they are created, I store those new
device entries in an attribute. Later in the code, I need to use those new
devices to either format, create a raid, etc. So, I need to save those
device values somewhere during the Chef run and be able to use them later
in the code.
There's also a fundamental misunderstanding about how to use Chef going on
here...
A Chef resource is a test-and-repair datastructure. It lives in a compiled
resource_collection and is meant to describe some policy point about the
system.
This example is trying to use one Chef resource to harvest information from
the system, use the node objetc as global state tracking mechanism, then
use another resource one to ensure its contents of a file.
The best thing to do would be to restate the problem. Start every system
with "I declare".
"I declare that the file /tmp/file-1 should have the output of ls /tmp as
its content.
There are a couple different ways to solve this, but the easiest would be
to stash the contents of your ls in a compile phase variable and pass it as
a parameters to your file resources.
this is not a resource. It will be discarded after the compile phase
Isn't the issue only that %x[ls /tmp] is being evaluated at compile time?
Avoid that notation and you'll be able to do what you intend. Use "system"
instead and it will be evaluated as part of the ruby block.
Slightly off-topic, but where are you modifying It may be a style or
philosophy thing, but I personally wouldn't try to modify an attribute once
I declare it. I rather that I set my conditions up front in attributes file
and maybe override them as necessary within the environment role. Changing
them within the recipe strikes me as adding a significant amount of
complexity to the code maintenance, as you now have to look at the recipes
AND attributes when you modify the cookbook.
Yes, I agree. Here's the actual scenario that I'm dealing with. I'm
creating some new EBS volumes, once they are created, I store those new
device entries in an attribute. Later in the code, I need to use those new
devices to either format, create a raid, etc. So, I need to save those
device values somewhere during the Chef run and be able to use them later
in the code.
There's also a fundamental misunderstanding about how to use Chef going on
here...
I completely understand what you're saying and I agree that my original
example is a bad example of what I'm trying to accomplish which is creating
a raid volume from 2 newly created ebs volumes.
So... to use your suggestion of declaration: 'I declare this system should
have a raid 0 volume created from 2 new EBS volumes.'
Isn't the issue only that %x[ls /tmp] is being evaluated at compile time?
Avoid that notation and you'll be able to do what you intend. Use "system"
instead and it will be evaluated as part of the ruby block.
I thought any ruby code inside a ruby block will not be evaluated during
compile time and instead during the normal order of the execution phase. I
haven't tested this yet, but does it really matter if inside that ruby
block I've used, %x[ls /tmp] or ls /tmp or system("ls /tmp") ?
you are right , when you pass a block inside a ruby_block resource's block
attribute, its not invoked during the compile phase (unless you do
run_action(:run) hacks). The challenge is how you capture this computed
value and pass it back to another resource. node.run_state is one way of
doing it. but again this is hacky. Also when reusing the coputed value you
have to ensure the consumer ruby code is also evaluated in the execution
phase, not it compile phase. i.e. another ruby_block resource where the
block parameter is accessing node.run_state will work, but file resource's
content attribute is evaluated during compile time , even if you do content
node.run_state[:value_from_other_ruby_blockl], you are accessing a value
that is not computed yet.
look this is hacky, but this will work:
ruby_block "foo" do
block do
node.run_state[:troll] = ls /tmp
end
end.run_action(:run)
file "/etc/bar" do
content node.run_state[:troll]
end
i think use cases like this shows that we really dont have tow independent
resources, rather they are coupled where one of them is dependent on
other's attribute value. hence it might be worth to cleanly implement it in
an lwrp where.
Isn't the issue only that %x[ls /tmp] is being evaluated at compile
time? Avoid that notation and you'll be able to do what you intend. Use
"system" instead and it will be evaluated as part of the ruby block.
I thought any ruby code inside a ruby block will not be evaluated during
compile time and instead during the normal order of the execution phase. I
haven't tested this yet, but does it really matter if inside that ruby
block I've used, %x[ls /tmp] or ls /tmp or system("ls /tmp") ?
Isn't the issue only that %x[ls /tmp] is being evaluated at compile
time? Avoid that notation and you'll be able to do what you intend. Use
"system" instead and it will be evaluated as part of the ruby block.
I thought any ruby code inside a ruby block will not be evaluated during
compile time and instead during the normal order of the execution phase. I
haven't tested this yet, but does it really matter if inside that ruby
block I've used, %x[ls /tmp] or ls /tmp or system("ls /tmp") ?
Thanks again to everyone for their input. I think it's been decided beyond
a shadow of a doubt that I should implement this in a custom provider.
A question to Ranjib's example; where can I find more information about
node.run_state? I only see mention of it regarding a windows provider and
really no where else.
Isn't the issue only that %x[ls /tmp] is being evaluated at compile
time? Avoid that notation and you'll be able to do what you intend. Use
"system" instead and it will be evaluated as part of the ruby block.
I thought any ruby code inside a ruby block will not be evaluated during
compile time and instead during the normal order of the execution phase. I
haven't tested this yet, but does it really matter if inside that ruby
block I've used, %x[ls /tmp] or ls /tmp or system("ls /tmp") ?
i cant think of any standard docs, except i some blogs out there that
recommended using it for temporary storage (the dude from nordstorm?) . I
knew its there, but never used it. When i first encountered it in our code
base (it was sumologic cookbook's sumo_source definition), i had changed it
to an hwrp.
iirc node.run_state was used to detect loaded recipes (remember currently
only recipes that are assigned via run list are present inside node.recipes
not the ones which applied via included_recipe), there was some breaking
changes during chef 10-> 11 migration too,
Thanks again to everyone for their input. I think it's been decided
beyond a shadow of a doubt that I should implement this in a custom
provider.
A question to Ranjib's example; where can I find more information about
node.run_state? I only see mention of it regarding a windows provider and
really no where else.
Isn't the issue only that %x[ls /tmp] is being evaluated at compile
time? Avoid that notation and you'll be able to do what you intend. Use
"system" instead and it will be evaluated as part of the ruby block.
I thought any ruby code inside a ruby block will not be evaluated during
compile time and instead during the normal order of the execution phase. I
haven't tested this yet, but does it really matter if inside that ruby
block I've used, %x[ls /tmp] or ls /tmp or system("ls /tmp") ?
There's also a fundamental misunderstanding about how to use Chef going on
here...
I completely understand what you're saying and I agree that my original
example is a bad example of what I'm trying to accomplish which is creating
a raid volume from 2 newly created ebs volumes.
So... to use your suggestion of declaration: 'I declare this system should
have a raid 0 volume created from 2 new EBS volumes.'
