CHEF-2880 debian policy and service provider


#1

A chef user recently ran into a problem with the service resource
where a service was disabled on their system by debian policy
unbeknownst to them. The underlying script that the resource asked to
start the service returned successfully, however it did not actually
start the service because of the system policy.

There are varying opinions as to what should be done here.

  1. Nothing. Chef shouldn’t fix your system for you.
  2. Warn. Chef should tell you if it thinks you are doing it wrong.
  3. Fail. Chef should throw an exception if you asked it to do
    something it couldn’t (by checking policy first).

CHEF-2880 [1] proposes:

  1. Always due #2 from above.
  2. Add an option to the resource to run "invoke-rc.d --disclose-deny"
    which will cause #3 above to happen.

We’re not crazy about adding resource method solely for this. The
simplest solution is to just run “invoke-rc.d --disclose-deny” all the
time. The big question here, is there a use case where you would have
the service disabled by policy but still want Chef to keep running if
you ask it to start it? Laurent? Thom? Tollef? (CHEF-597 [2])

The argument is that, if you ask Chef to do something and it cannot,
it should fail. Or you shouldn’t ask it to try. But we usually trust
the underlying system, if there is a bug, perhaps it is in invoke-rc.d
lying to us unless we use --disclose-deny. In any case, it isn’t as if
we’re going to start running ps after a service resource action to
verify if it worked or not.


Bryan McLellan | opscode | technical program manager
© 206.607.7108 | (t) @btmspox | (b) http://blog.loftninjas.org

[1] http://tickets.opscode.com/browse/CHEF-2880
[2] http://tickets.opscode.com/browse/CHEF-597


#2

On Wed, Feb 8, 2012 at 10:12 PM, Bryan McLellan btm@opscode.com wrote:

The argument is that, if you ask Chef to do something and it cannot,
it should fail. Or you shouldn’t ask it to try. But we usually trust
the underlying system, if there is a bug, perhaps it is in invoke-rc.d
lying to us unless we use --disclose-deny. In any case, it isn’t as if
we’re going to start running ps after a service resource action to
verify if it worked or not.

Yes :slight_smile:

Beyond this specific instance, I think the principle you outlined is
sane: Chef shouldn’t try to do what you mean, as more often than not
it would violate POLA.
A warning can and will be ignored, and may lead to unexpected consequences.
A failure on the other hand requires the user to pay attention and
either fix the underlying systems, or adapt the cookbook.

Andrea


#3

On Thursday, February 9, 2012 at 12:11 AM, Andrea Campi wrote:

On Wed, Feb 8, 2012 at 10:12 PM, Bryan McLellan <btm@opscode.com (mailto:btm@opscode.com)> wrote:

The argument is that, if you ask Chef to do something and it cannot,
it should fail. Or you shouldn’t ask it to try. But we usually trust
the underlying system, if there is a bug, perhaps it is in invoke-rc.d
lying to us unless we use --disclose-deny. In any case, it isn’t as if
we’re going to start running ps after a service resource action to
verify if it worked or not.

Yes :slight_smile:

Beyond this specific instance, I think the principle you outlined is
sane: Chef shouldn’t try to do what you mean, as more often than not
it would violate POLA.
A warning can and will be ignored, and may lead to unexpected consequences.
A failure on the other hand requires the user to pay attention and
either fix the underlying systems, or adapt the cookbook.

Andrea
I’m of this opinion as well. I think that in this case you’ve created an inconsistency between how the system is configured and what you’re asking chef to do, so the best option is to fail and let the admin either fix the policy or the recipe.


Dan DeLeo


#4

Bryan McLellan btm@opscode.com writes:

Bonjour,

A chef user recently ran into a problem with the service resource
where a service was disabled on their system by debian policy
unbeknownst to them. The underlying script that the resource asked to
start the service returned successfully, however it did not actually
start the service because of the system policy.

There are varying opinions as to what should be done here.

  1. Nothing. Chef shouldn’t fix your system for you.
  2. Warn. Chef should tell you if it thinks you are doing it wrong.
  3. Fail. Chef should throw an exception if you asked it to do
    something it couldn’t (by checking policy first).

CHEF-2880 [1] proposes:

  1. Always due #2 from above.
  2. Add an option to the resource to run "invoke-rc.d --disclose-deny"
    which will cause #3 above to happen.

We’re not crazy about adding resource method solely for this. The
simplest solution is to just run “invoke-rc.d --disclose-deny” all the
time. The big question here, is there a use case where you would have
the service disabled by policy but still want Chef to keep running if
you ask it to start it? Laurent? Thom? Tollef? (CHEF-597 [2])

I use chef to bootstrap debian systems from scratch.
I use cdebootstrap, I install a basic chef setup, then I run chef from
the chrooted system to continue with its setup (fstab, network,
iscsi, multipath, swap, etc…)

As cdeboostrap does, i’m using policy-rc.d to prevent invoke-rc.d to
start any service from the chroot.
In that case i don’t expect my cookbooks to fail.

Then when the debian os is run for real, i expect the same cookbooks
to start services.

So I’m definitly in favor of what CHEF-2880 proposes:

  1. warn by default, in that case chef doesn’t change the default
    behavior of invoke-rc.d. This is the only added value I would
    expect from chef
  2. add the option to make it fail when the script action has been
    denied


Laurent


#5

That’s a slick use case, but I think you’re taking advantage of a bug. :slight_smile:

2012/2/9 Laurent Désarmes laurent+opscode@u-picardie.fr:

So I’m definitly in favor of what CHEF-2880 proposes:

  1. warn by default, in that case chef doesn’t change the default
    behavior of invoke-rc.d. This is the only added value I would
    expect from chef
  2. add the option to make it fail when the script action has been
    denied

Thus far, it seems like if there is going to be a switch, it should
default in the other direction and fail by default. It would be nice
if we could find a tricker way to build said switch without clouding
up the resource with another method.

Bryan


#6

]] Bryan McLellan

We’re not crazy about adding resource method solely for this. The
simplest solution is to just run “invoke-rc.d --disclose-deny” all the
time. The big question here, is there a use case where you would have
the service disabled by policy but still want Chef to keep running if
you ask it to start it? Laurent? Thom? Tollef? (CHEF-597 [2])

Well, invoke-rc.d is not what is used by init, invoke-rc.d is used by
maintainer scripts to decide what, if anything, should be done on
upgrades.

The symlinks in /etc/rc$runlevel.d/ are usually what controls if a
service should start.

I think we shouldn’t use invoke-rc.d, but rather use the service
command.

A use case for having policy-rc.d decline anything invoke-rc.d asks it
would be to not restart services on upgrade, but rather handle that
through chef or similar mechanisms.

To solve Laurent’s use case, I’d say just diverting the service command
at the start and undiverting it afterwards would be just as sane as
using policy-rc.d.

Regards,

Tollef Fog Heen
UNIX is user friendly, it’s just picky about who its friends are


#7

On Fri, Feb 10, 2012 at 1:42 PM, Tollef Fog Heen tfheen@err.no wrote:

I think we shouldn’t use invoke-rc.d, but rather use the service
command.

This sounds like reverting CHEF-597 [1], yes? Other than Laurent’s use
case which we’ve come up with workarounds for, is there any reason not
to? This would fix CHEF-2880 simply by ignoring the Debian service
policy.

Bryan

[1] http://tickets.opscode.com/browse/CHEF-597


#8

]] Bryan McLellan

On Fri, Feb 10, 2012 at 1:42 PM, Tollef Fog Heen tfheen@err.no wrote:

I think we shouldn’t use invoke-rc.d, but rather use the service
command.

This sounds like reverting CHEF-597 [1], yes?

Yup.

Other than Laurent’s use case which we’ve come up with workarounds
for, is there any reason not to? This would fix CHEF-2880 simply by
ignoring the Debian service policy.

Well, I guess Thom (who filed 597) could argue that if you’ve configured
your system to not start $service, finding it running would be a
surprise. To that, I’d respond that manually calling /etc/init.d/foo or
service foo start does not check run levels etc, and to me, doing
explicit management in chef recipes is mentally much closer to calling
service or the init script directly than it is to ask init to put the
service in the state it should be in according to rcN.d.

The latter might be useful, something like

service «foo» do
action :default
end

which would start or stop the service as appropriate.


Tollef Fog Heen
UNIX is user friendly, it’s just picky about who its friends are


#9

On Fri, Feb 10, 2012 at 20:06, Tollef Fog Heen tfheen@err.no wrote:

]] Bryan McLellan

On Fri, Feb 10, 2012 at 1:42 PM, Tollef Fog Heen tfheen@err.no wrote:

I think we shouldn’t use invoke-rc.d, but rather use the service
command.

This sounds like reverting CHEF-597 [1], yes?

Yup.

Other than Laurent’s use case which we’ve come up with workarounds
for, is there any reason not to? This would fix CHEF-2880 simply by
ignoring the Debian service policy.

Well, I guess Thom (who filed 597) could argue that if you’ve configured
your system to not start $service, finding it running would be a
surprise. To that, I’d respond that manually calling /etc/init.d/foo or
service foo start does not check run levels etc, and to me, doing
explicit management in chef recipes is mentally much closer to calling
service or the init script directly than it is to ask init to put the
service in the state it should be in according to rcN.d.

Yeah, I was certainly thinking that if you had policy configured it
would be a surprise to find chef overriding it,
but i equally take your point that chef ought to be the definitive
source of truth for the system.
So I’m fine with reverting 597.
-Thom


#10

Bryan McLellan btm@loftninjas.org writes:

On Fri, Feb 10, 2012 at 1:42 PM, Tollef Fog Heen tfheen@err.no wrote:

I think we shouldn’t use invoke-rc.d, but rather use the service
command.

This sounds like reverting CHEF-597 [1], yes? Other than Laurent’s use
case which we’ve come up with workarounds for, is there any reason not
to? This would fix CHEF-2880 simply by ignoring the Debian service
policy.

is there a need for removing the invoke-rc.d provider ?
what about keeping the provider and reverting the default to the old
one ?


#11

On Wed, Feb 15, 2012 at 11:47 AM, laurent+opscode@u-picardie.fr wrote:

is there a need for removing the invoke-rc.d provider ?
what about keeping the provider and reverting the default to the old
one ?

I thought about that as I just did the revert this morning. The risk
is that it is cruft that only Laurent and Thom use, but we love you
guys so that’s okay. :slight_smile:

If you’d like that, let me know and I can fix that later today.

Bryan


#12

Bryan McLellan btm@loftninjas.org writes:

On Wed, Feb 15, 2012 at 11:47 AM, laurent+opscode@u-picardie.fr wrote:

is there a need for removing the invoke-rc.d provider ?
what about keeping the provider and reverting the default to the old
one ?

I thought about that as I just did the revert this morning. The risk
is that it is cruft that only Laurent and Thom use, but we love you
guys so that’s okay. :slight_smile:

If you’d like that, let me know and I can fix that later today.

Oh yes pretty please :wink:
TIA


Laurent