Hi,
Is there any way to run the chef-client and have it only report what it
thinks it’s going to do rather than just doing it? This would save a lot of
debugging time!
Mark
chef 11 supposedly will support a noop mode, it doesn't exist yet (last I
checked).
On Feb 17, 2012 7:05 AM, "mark bradley" gopearls42@gmail.com wrote:
Hi,
Is there any way to run the chef-client and have it only report what it
thinks it's going to do rather than just doing it? This would save a lot of
debugging time!Mark
Thanks! Can't wait ..
On Fri, Feb 17, 2012 at 10:11 AM, Kyle Bader kyle.bader@gmail.com wrote:
chef 11 supposedly will support a noop mode, it doesn't exist yet (last I
checked).
On Feb 17, 2012 7:05 AM, "mark bradley" gopearls42@gmail.com wrote:Hi,
Is there any way to run the chef-client and have it only report what it
thinks it's going to do rather than just doing it? This would save a lot of
debugging time!Mark
On Fri, Feb 17, 2012 at 7:05 AM, mark bradley gopearls42@gmail.com wrote:
Is there any way to run the chef-client and have it only report what it
thinks it's going to do rather than just doing it? This would save a lot of
debugging time!
This has been in the Opscode backlog for some time, which is to say we
plan to implement it, but there isn't a set timeline.
There is, we believe, a good reason for it not being given a high
priority. In short, it is because we believe that noop is a great in
theory, but in practice it has to lie to you. There are previous
conversations on this subject which you can find linked to from this
ticket: http://tickets.opscode.com/browse/CHEF-13
Generally, the time when noop is useful is the time when your system
is simple enough that you can deduce everything quite easily. The time
when you cross over to noop lying to you is unfortunately quite subtle
and can be dauntingly early.
Example #1:
package "apache2"
service "apache2" do
action [ :enable, :start ]
end
template "/etc/apache2/httpd.conf" do
source "http.conf.erb"
notifies :restart, "service[apache2]"
end
Now, assuming that the apache2 package is already installed but not
running, we would expect an output like this:
INFO: [noop] Would [ :enable, :start ] service[apache2]
But what if the apache2 package is not already installed?
INFO: [noop] Would :install package[apache2]
Chef::Exceptions::Exec: /usr/sbin/update-rc.d apache2 defaults
returned 1, expected 0
We can't check to see what action we would take against the service
because the service does not exist yet. Sure, we could skip the check
and claim that it would start the service, but that is a lie if the
service is already running. This becomes incredibly important when you
consider notifications. If the template doesn't exist, or is
different, we would change it and say so, but what would happen with
the service? Are we still checking if it is running or pretending that
it is?
Example #2:
execute "/tmp/some-installer.sh" do
not_if { File.exists?("/opt/some-software") }
notifies :run, "execute[setup-software]", :immediately
end
execute "setup-software" do
action :nothing
command "/opt/some-software/setup.sh --unique-setting special_kittens"
end
Obviously we wouldn't run the first execute resource in any case,
because we don't know what it actually does. It may not be idempotent,
and usually is not. It may or may not actually change the system. But
what about the not_if and only_if conditionals? Should we evaluate
those or execute them (Ruby blocks get evaluated, strings are executed
as system commands)? In many cases these blocks don't change the
system, they just inspect them, but Chef can't guess your intentions.
Example #3:
execute "first-thing"
execute "second-thing" do
only_if { File.exists?("/tmp/created-by-first-thing") }
end
continue this pattern a couple times.
When your cookbooks get complex and each resource builds on a prior
resource, you will soon hit a circumstance where noop would say it
would run a resource, but then claim that all later resources would
not run. This would be a lie, because the first resource causes the
later resources to be triggered.
Bryan
On Fri, Feb 17, 2012 at 11:29 AM, Bryan McLellan btm@loftninjas.org wrote:
On Fri, Feb 17, 2012 at 7:05 AM, mark bradley gopearls42@gmail.com
wrote:Is there any way to run the chef-client and have it only report what it
thinks it's going to do rather than just doing it? This would save a lot
of
debugging time!This has been in the Opscode backlog for some time, which is to say we
plan to implement it, but there isn't a set timeline.There is, we believe, a good reason for it not being given a high
priority. In short, it is because we believe that noop is a great in
theory, but in practice it has to lie to you. There are previous
conversations on this subject which you can find linked to from this
ticket: http://tickets.opscode.com/browse/CHEF-13Generally, the time when noop is useful is the time when your system
is simple enough that you can deduce everything quite easily. The time
when you cross over to noop lying to you is unfortunately quite subtle
and can be dauntingly early.Example #1:
package "apache2"
service "apache2" do
action [ :enable, :start ]
end
template "/etc/apache2/httpd.conf" do
source "http.conf.erb"
notifies :restart, "service[apache2]"
endNow, assuming that the apache2 package is already installed but not
running, we would expect an output like this:INFO: [noop] Would [ :enable, :start ] service[apache2]
But what if the apache2 package is not already installed?
INFO: [noop] Would :install package[apache2]
Chef::Exceptions::Exec: /usr/sbin/update-rc.d apache2 defaults
returned 1, expected 0We can't check to see what action we would take against the service
because the service does not exist yet. Sure, we could skip the check
and claim that it would start the service, but that is a lie if the
service is already running. This becomes incredibly important when you
consider notifications. If the template doesn't exist, or is
different, we would change it and say so, but what would happen with
the service? Are we still checking if it is running or pretending that
it is?Example #2:
execute "/tmp/some-installer.sh" do
not_if { File.exists?("/opt/some-software") }
notifies :run, "execute[setup-software]", :immediately
end
execute "setup-software" do
action :nothing
command "/opt/some-software/setup.sh --unique-setting special_kittens"
endObviously we wouldn't run the first execute resource in any case,
because we don't know what it actually does. It may not be idempotent,
and usually is not. It may or may not actually change the system. But
what about the not_if and only_if conditionals? Should we evaluate
those or execute them (Ruby blocks get evaluated, strings are executed
as system commands)? In many cases these blocks don't change the
system, they just inspect them, but Chef can't guess your intentions.Example #3:
execute "first-thing"
execute "second-thing" do
only_if { File.exists?("/tmp/created-by-first-thing") }
endcontinue this pattern a couple times.
When your cookbooks get complex and each resource builds on a prior
resource, you will soon hit a circumstance where noop would say it
would run a resource, but then claim that all later resources would
not run. This would be a lie, because the first resource causes the
later resources to be triggered.Bryan
Yes, I can see the paradox, but perhaps you're over thinking this? I'd
rather have some reporting and know that it might be lying than botch up
my environment because of a mistake. At least I'll know that my scripts
have a likelihood of being right.
Puppet seems to have solved this problem, somehow, and of course make -n
from years gone by.
Not that I'm an expert on Chef, by any means. All I was getting at was that
there is utility in this feature even though it may not be perfect. You
guys are the experts, I leave it in your capable hands. Just think of this
as a +1 vote for the feature. Maybe one day I'll know enough to contribute
code too 
Mark
Mark,
You would get much better results running your recipes on non-production
nodes than running in noop for the reasons Bryan outlined. How much
security would this feature really give you?
James
On Fri, Feb 17, 2012 at 10:55 AM, mark bradley gopearls42@gmail.com wrote:
On Fri, Feb 17, 2012 at 11:29 AM, Bryan McLellan btm@loftninjas.orgwrote:
On Fri, Feb 17, 2012 at 7:05 AM, mark bradley gopearls42@gmail.com
wrote:Is there any way to run the chef-client and have it only report what it
thinks it's going to do rather than just doing it? This would save a
lot of
debugging time!This has been in the Opscode backlog for some time, which is to say we
plan to implement it, but there isn't a set timeline.There is, we believe, a good reason for it not being given a high
priority. In short, it is because we believe that noop is a great in
theory, but in practice it has to lie to you. There are previous
conversations on this subject which you can find linked to from this
ticket: http://tickets.opscode.com/browse/CHEF-13Generally, the time when noop is useful is the time when your system
is simple enough that you can deduce everything quite easily. The time
when you cross over to noop lying to you is unfortunately quite subtle
and can be dauntingly early.Example #1:
package "apache2"
service "apache2" do
action [ :enable, :start ]
end
template "/etc/apache2/httpd.conf" do
source "http.conf.erb"
notifies :restart, "service[apache2]"
endNow, assuming that the apache2 package is already installed but not
running, we would expect an output like this:INFO: [noop] Would [ :enable, :start ] service[apache2]
But what if the apache2 package is not already installed?
INFO: [noop] Would :install package[apache2]
Chef::Exceptions::Exec: /usr/sbin/update-rc.d apache2 defaults
returned 1, expected 0We can't check to see what action we would take against the service
because the service does not exist yet. Sure, we could skip the check
and claim that it would start the service, but that is a lie if the
service is already running. This becomes incredibly important when you
consider notifications. If the template doesn't exist, or is
different, we would change it and say so, but what would happen with
the service? Are we still checking if it is running or pretending that
it is?Example #2:
execute "/tmp/some-installer.sh" do
not_if { File.exists?("/opt/some-software") }
notifies :run, "execute[setup-software]", :immediately
end
execute "setup-software" do
action :nothing
command "/opt/some-software/setup.sh --unique-setting special_kittens"
endObviously we wouldn't run the first execute resource in any case,
because we don't know what it actually does. It may not be idempotent,
and usually is not. It may or may not actually change the system. But
what about the not_if and only_if conditionals? Should we evaluate
those or execute them (Ruby blocks get evaluated, strings are executed
as system commands)? In many cases these blocks don't change the
system, they just inspect them, but Chef can't guess your intentions.Example #3:
execute "first-thing"
execute "second-thing" do
only_if { File.exists?("/tmp/created-by-first-thing") }
endcontinue this pattern a couple times.
When your cookbooks get complex and each resource builds on a prior
resource, you will soon hit a circumstance where noop would say it
would run a resource, but then claim that all later resources would
not run. This would be a lie, because the first resource causes the
later resources to be triggered.Bryan
Yes, I can see the paradox, but perhaps you're over thinking this? I'd
rather have some reporting and know that it might be lying than botch up
my environment because of a mistake. At least I'll know that my scripts
have a likelihood of being right.Puppet seems to have solved this problem, somehow, and of course make -n
from years gone by.Not that I'm an expert on Chef, by any means. All I was getting at was
that there is utility in this feature even though it may not be perfect.
You guys are the experts, I leave it in your capable hands. Just think of
this as a +1 vote for the feature. Maybe one day I'll know enough to
contribute code tooMark
On Fri, Feb 17, 2012 at 10:55 AM, mark bradley gopearls42@gmail.com wrote:
Yes, I can see the paradox, but perhaps you're over thinking this? I'd
rather have some reporting and know that it might be lying than botch up
my environment because of a mistake. At least I'll know that my scripts have
a likelihood of being right.
That is the exactly the problem though, you want noop ensure that you
do not "botch up my environment because of a mistake" but because of
the reasons that I outlined, it won't. Thus when you use it and it
tells you everything will be fine, but then you run it in production
and your mysql server is unexpectedly restarted, what value did you
get? Not only did it lie to you, it lied about the very thing you were
trusting it to do.
I may be rambling here, but we run into a similar problem when writing
unit tests for Ohai. Let us say we're testing a plugin that reads the
current version of FoobarOS from /etc/foobar-release which contains
"Foobar 1.3". We want to ensure our code does the right thing, but
unless you run it on FoobarOS, there isn't going to be a
/etc/foobar-release file, so for the test you create a fake version of
the file. This is all well and good until FoobarOS 2.0 comes out and
suddenly /etc/foobar-release contains "FoobarOS version 2.0" The
solution (which I'm frustrated because I haven't gotten time to do
this yet) is that you need a library of virtual machines where you can
run the tests automatically without mocking the files.
As James chimed in, the real test for your cookbooks is running them
in a real environment. As long as I've used Chef, we've had multiple
environments not only for testing our chef recipes, but for the
software itself. Most commonly, a multiple development environments, a
testing/pre-prod environment, and a production environment.
In summary, we're writing recipes that say:
set A to true if it is false
set B to true only if A is true
Noop would say:
I would set A to true because it is false
I would not set B to true because A is false
Then when you run this:
A is set to true
B is set to true because A is true
But wait, that is different than what noop
Puppet seems to have solved this problem, somehow,
Puppet is not up front with you about the lie. (Or maybe they are
somewhere in their docs, I don't spend much time there.)
Here is a basic example:
node 'localhost' {
package {'apache2':
ensure => present
}
service {'apache2':
ensure => stopped,
enable => false,
require => Package['apache2']
}
file {'/etc/apache2/fake_config.conf':
ensure => present,
content => "This could do something. Maybe",
backup => false,
notify => Service['apache2']
}
}
I've stripped out some of the noise from verbose, and added line numbers:
~$ sudo puppet agent -t --noop
1: info: Caching catalog for localhost
2: info: Applying configuration version '1329519751'
3: notice: /Stage[main]//Node[localhost]/File[/etc/apache2/fake_config.conf]/ensure:
current_value absent, should be present (noop)
4: info: /Stage[main]//Node[localhost]/File[/etc/apache2/fake_config.conf]:
Scheduling refresh of Service[apache2]
5: notice: /Stage[main]//Node[localhost]/Package[apache2]/ensure:
current_value purged, should be present (noop)
6: err: /Stage[main]//Node[localhost]/Service[apache2]: Could not
evaluate: Could not find init script for 'apache2'
So it would have restarted apache2 on line 4 because it changed a file
on line 3, and it would have installed the service on line 5, but why
the error on line 6? Clearly the package is not installed yet, so we
cannot check its status. But we were just told that it would have
restarted the service, but it can't tell us if could start the
service? There is a lie in there somewhere. Fortunately this example
is simple and we can deduce what would have happened anyway. But if we
can do that, why do we need noop?
~$ sudo puppet agent -t
1: info: Caching catalog for localhost
2: info: Applying configuration version '1329519751'
3: err: /Stage[main]//Node[localhost]/File[/etc/apache2/fake_config.conf]/ensure:
change from absent to present failed: Could not set 'present on
ensure: No such file or directory -
/etc/apache2/fake_config.conf.puppettmp_9021 at
/etc/puppet/manifests/site.pp:18
4: notice: /Stage[main]//Node[localhost]/Package[apache2]/ensure:
ensure changed 'purged' to 'present'
5: notice: /Stage[main]//Node[localhost]/Service[apache2]: Dependency
File[/etc/apache2/fake_config.conf] has failures: true
6: warning: /Stage[main]//Node[localhost]/Service[apache2]: Skipping
because of failed dependencies
Wait, didn't noop tell us that it would create a file and restart
apache2? It didn't even create the file! Clearly I've forgotten how
Puppet dependencies work, because it created an automatic dependency
between the service and the configuration file in the wrong direction,
but noop wasn't honest with me about what was going to do here either.
Further, the Puppet DSL that most people use is quite limited. The
notify syntax only has a single action, in the above case notifying a
service resource means restarting it. You still can't notify a service
to reload [1] without a workaround like declaring an execute resource
to run the reload and notifying that resource instead. The onlyif
attribute in Puppet can only be used with the exec (and augeas)
resource and can only be string containing a command to run, where
with Chef only_if and not_if are meta-attributes; can be used with all
resources, and can take a Ruby block as well as a string to run.
For instance this code:
exec { "ls /tmp":
path => ["/bin", "/usr/bin", "/usr/bin", "/usr/sbin"],
onlyif => "true"
}
Under noop returns: notice: /Stage[main]//Node[localhost]/Exec[ls
/tmp]/returns: current_value notrun, should be 0 (noop)
But if you change the onlyif from true to false, you don't see
anything. This is because even under noop Puppet still runs the
commands in onlyif. This is a compromise of course, as without running
the commands you can't tell what you would do, but if the user expects
the run to have no side-affects they may be surprised. I think it is
also a bug that it says nothing at all.
and of course make -n from years gone by.
Having put a bunch of time into testing out the above example to show
you, I'll leave the exercise of make to the reader, but I think this
comparison is unreasonable anyway as long as make is a build system
and not a configuration management system.
Not that I'm an expert on Chef, by any means. All I was getting at was that
there is utility in this feature even though it may not be perfect. You guys
are the experts, I leave it in your capable hands. Just think of this as a
+1 vote for the feature. Maybe one day I'll know enough to contribute code
too
We look forward to you becoming an expert and contributing.
Bryan
Bryan, thanks very much for the detailed answer. I understand the issue
much better now
Mark
On Fri, Feb 17, 2012 at 6:29 PM, Bryan McLellan btm@loftninjas.org wrote:
On Fri, Feb 17, 2012 at 10:55 AM, mark bradley gopearls42@gmail.com
wrote:Yes, I can see the paradox, but perhaps you're over thinking this? I'd
rather have some reporting and know that it might be lying than botch
up
my environment because of a mistake. At least I'll know that my scripts
have
a likelihood of being right.That is the exactly the problem though, you want noop ensure that you
do not "botch up my environment because of a mistake" but because of
the reasons that I outlined, it won't. Thus when you use it and it
tells you everything will be fine, but then you run it in production
and your mysql server is unexpectedly restarted, what value did you
get? Not only did it lie to you, it lied about the very thing you were
trusting it to do.I may be rambling here, but we run into a similar problem when writing
unit tests for Ohai. Let us say we're testing a plugin that reads the
current version of FoobarOS from /etc/foobar-release which contains
"Foobar 1.3". We want to ensure our code does the right thing, but
unless you run it on FoobarOS, there isn't going to be a
/etc/foobar-release file, so for the test you create a fake version of
the file. This is all well and good until FoobarOS 2.0 comes out and
suddenly /etc/foobar-release contains "FoobarOS version 2.0" The
solution (which I'm frustrated because I haven't gotten time to do
this yet) is that you need a library of virtual machines where you can
run the tests automatically without mocking the files.As James chimed in, the real test for your cookbooks is running them
in a real environment. As long as I've used Chef, we've had multiple
environments not only for testing our chef recipes, but for the
software itself. Most commonly, a multiple development environments, a
testing/pre-prod environment, and a production environment.In summary, we're writing recipes that say:
set A to true if it is false
set B to true only if A is trueNoop would say:
I would set A to true because it is false
I would not set B to true because A is falseThen when you run this:
A is set to true
B is set to true because A is trueBut wait, that is different than what noop
Puppet seems to have solved this problem, somehow,
Puppet is not up front with you about the lie. (Or maybe they are
somewhere in their docs, I don't spend much time there.)Here is a basic example:
node 'localhost' {
package {'apache2':
ensure => present
}service {'apache2':
ensure => stopped,
enable => false,
require => Package['apache2']
}file {'/etc/apache2/fake_config.conf':
ensure => present,
content => "This could do something. Maybe",
backup => false,
notify => Service['apache2']
}
}I've stripped out some of the noise from verbose, and added line numbers:
~$ sudo puppet agent -t --noop
1: info: Caching catalog for localhost
2: info: Applying configuration version '1329519751'
3: notice:
/Stage[main]//Node[localhost]/File[/etc/apache2/fake_config.conf]/ensure:
current_value absent, should be present (noop)
4: info: /Stage[main]//Node[localhost]/File[/etc/apache2/fake_config.conf]:
Scheduling refresh of Service[apache2]
5: notice: /Stage[main]//Node[localhost]/Package[apache2]/ensure:
current_value purged, should be present (noop)
6: err: /Stage[main]//Node[localhost]/Service[apache2]: Could not
evaluate: Could not find init script for 'apache2'So it would have restarted apache2 on line 4 because it changed a file
on line 3, and it would have installed the service on line 5, but why
the error on line 6? Clearly the package is not installed yet, so we
cannot check its status. But we were just told that it would have
restarted the service, but it can't tell us if could start the
service? There is a lie in there somewhere. Fortunately this example
is simple and we can deduce what would have happened anyway. But if we
can do that, why do we need noop?~$ sudo puppet agent -t
1: info: Caching catalog for localhost
2: info: Applying configuration version '1329519751'
3: err:
/Stage[main]//Node[localhost]/File[/etc/apache2/fake_config.conf]/ensure:
change from absent to present failed: Could not set 'present on
ensure: No such file or directory -
/etc/apache2/fake_config.conf.puppettmp_9021 at
/etc/puppet/manifests/site.pp:18
4: notice: /Stage[main]//Node[localhost]/Package[apache2]/ensure:
ensure changed 'purged' to 'present'
5: notice: /Stage[main]//Node[localhost]/Service[apache2]: Dependency
File[/etc/apache2/fake_config.conf] has failures: true
6: warning: /Stage[main]//Node[localhost]/Service[apache2]: Skipping
because of failed dependenciesWait, didn't noop tell us that it would create a file and restart
apache2? It didn't even create the file! Clearly I've forgotten how
Puppet dependencies work, because it created an automatic dependency
between the service and the configuration file in the wrong direction,
but noop wasn't honest with me about what was going to do here either.Further, the Puppet DSL that most people use is quite limited. The
notify syntax only has a single action, in the above case notifying a
service resource means restarting it. You still can't notify a service
to reload [1] without a workaround like declaring an execute resource
to run the reload and notifying that resource instead. The onlyif
attribute in Puppet can only be used with the exec (and augeas)
resource and can only be string containing a command to run, where
with Chef only_if and not_if are meta-attributes; can be used with all
resources, and can take a Ruby block as well as a string to run.For instance this code:
exec { "ls /tmp":
path => ["/bin", "/usr/bin", "/usr/bin", "/usr/sbin"],
onlyif => "true"
}Under noop returns: notice: /Stage[main]//Node[localhost]/Exec[ls
/tmp]/returns: current_value notrun, should be 0 (noop)But if you change the onlyif from true to false, you don't see
anything. This is because even under noop Puppet still runs the
commands in onlyif. This is a compromise of course, as without running
the commands you can't tell what you would do, but if the user expects
the run to have no side-affects they may be surprised. I think it is
also a bug that it says nothing at all.and of course make -n from years gone by.
Having put a bunch of time into testing out the above example to show
you, I'll leave the exercise of make to the reader, but I think this
comparison is unreasonable anyway as long as make is a build system
and not a configuration management system.Not that I'm an expert on Chef, by any means. All I was getting at was
that
there is utility in this feature even though it may not be perfect. You
guys
are the experts, I leave it in your capable hands. Just think of this as
a
+1 vote for the feature. Maybe one day I'll know enough to contribute
code
tooWe look forward to you becoming an expert and contributing.
Bryan