New package resource action :reconfig


#1

Hi,

I have written a patch which introduces a new action :reconfig to the package resource. Because it is a (non-breaking) change to the dsl api I think it should be discussed first before I file a new ticket.

Background: The package resource already has a :response_file attribute, which can be used to preseed configuration values for a package. In the case of the apt provider this preseeding only happens when the :install action is invoked, for reconfiguration of already installed packages its not usable at the moment. For apt, this would mean to invoke dpkg-reconfigure. I don’t think it is elegant implementing it in the :install action - therefore I invented the :reconfig action.

I intend to also resolve http://tickets.opscode.com/browse/CHEF-294 which would be even more powerful together with a reconfiguration mechanism.

What do you think? Do you like the introduction of the :reconfig action or would you implement this in a less invasive way? We could also discuss the naming of the action, I obviously inherited :reconfig from the debian world (see dpkg-reconfigure).

Best regards,
Dennis


Percona XtraDB cluster build failing
#2

Dennis Klein d.klein@gsi.de writes:

Hi,

I have written a patch which introduces a new action :reconfig to the package resource. Because it is a (non-breaking) change to the dsl api I think it should be discussed first before I file a new ticket.

Background: The package resource already has a :response_file attribute, which can be used to preseed configuration values for a package. In the case of the apt provider this preseeding only happens when the :install action is invoked, for reconfiguration of already installed packages its not usable at the moment. For apt, this would mean to invoke dpkg-reconfigure. I don’t think it is elegant implementing it in the :install action - therefore I invented the :reconfig action.

I intend to also resolve http://tickets.opscode.com/browse/CHEF-294 which would be even more powerful together with a reconfiguration mechanism.

What do you think? Do you like the introduction of the :reconfig action or would you implement this in a less invasive way? We could also discuss the naming of the action, I obviously inherited :reconfig from the debian world (see dpkg-reconfigure).

Yes, i think it’s a good idea, how do you plan on making it idempotent ?
Have you written a new debconf resource and provider that the apt
package resource would require ?


Laurent


#3

Exactly same question i have; how it will be idempotent ? to me this is
equivalent to yum reinstall <package>. It cant be idempotent by
definition. Generally i do this kinds of one time task using knife ssh.
On Sat, 2011-08-06 at 19:43 +0200, laurent+opscode@u-picardie.fr wrote:

Dennis Klein d.klein@gsi.de writes:

Hi,

I have written a patch which introduces a new action :reconfig to the package resource. Because it is a (non-breaking) change to the dsl api I think it should be discussed first before I file a new ticket.

Background: The package resource already has a :response_file attribute, which can be used to preseed configuration values for a package. In the case of the apt provider this preseeding only happens when the :install action is invoked, for reconfiguration of already installed packages its not usable at the moment. For apt, this would mean to invoke dpkg-reconfigure. I don’t think it is elegant implementing it in the :install action - therefore I invented the :reconfig action.

I intend to also resolve http://tickets.opscode.com/browse/CHEF-294 which would be even more powerful together with a reconfiguration mechanism.

What do you think? Do you like the introduction of the :reconfig action or would you implement this in a less invasive way? We could also discuss the naming of the action, I obviously inherited :reconfig from the debian world (see dpkg-reconfigure).

Yes, i think it’s a good idea, how do you plan on making it idempotent ?
Have you written a new debconf resource and provider that the apt
package resource would require ?


#4

RanjibDey ranjibd@thoughtworks.com writes:

Exactly same question i have; how it will be idempotent ? to me this is
equivalent to yum reinstall <package>. It cant be idempotent by
definition. Generally i do this kinds of one time task using knife ssh.

I don’t know about yum, but for apt we could make it idempotent with a
debconf provider that would use debconf-show and
debconf-set-selections


Laurent


#5

On Aug 6, 2011, at 7:43 PM, laurent+opscode@u-picardie.fr wrote:

Dennis Klein d.klein@gsi.de writes:

Hi,

I have written a patch which introduces a new action :reconfig to the package resource. Because it is a (non-breaking) change to the dsl api I think it should be discussed first before I file a new ticket.

Background: The package resource already has a :response_file attribute, which can be used to preseed configuration values for a package. In the case of the apt provider this preseeding only happens when the :install action is invoked, for reconfiguration of already installed packages its not usable at the moment. For apt, this would mean to invoke dpkg-reconfigure. I don’t think it is elegant implementing it in the :install action - therefore I invented the :reconfig action.

I intend to also resolve http://tickets.opscode.com/browse/CHEF-294 which would be even more powerful together with a reconfiguration mechanism.

What do you think? Do you like the introduction of the :reconfig action or would you implement this in a less invasive way? We could also discuss the naming of the action, I obviously inherited :reconfig from the debian world (see dpkg-reconfigure).

Yes, i think it’s a good idea, how do you plan on making it idempotent ?
Have you written a new debconf resource and provider that the apt
package resource would require ?


Laurent

I see 3 options for idempotency:

  1. invoke the reconfiguration only if the :response_file has changed
  • disadvantage: non-chef misconfigurations of a package will go unnoticed.
  1. rely on idempotency of the reconfiguration facility on the specific platforms
  • disadvantage: if the platform implementation changes, idempotency may not be given any more.
  1. implement logic which retrievs the actual configuration values from the node and decide, if a reconfiguration is needed. (maybe even put the retrieval logic to ohai)
  • disadvantage: intense coding necessary, in the case of apt, extra packages need to be installed to be able to retrieve config values or you have to reimplement a database client as part of the provider …

So far I only implemented the :reconfig action for the apt provider. Therefore I chose option 2, because dpkg-reconfigure is already idempotent.


Dennis


#6

On Aug 6, 2011, at 8:25 PM, Dennis Klein wrote:

On Aug 6, 2011, at 7:43 PM, laurent+opscode@u-picardie.fr wrote:

Yes, i think it’s a good idea, how do you plan on making it idempotent ?
Have you written a new debconf resource and provider that the apt
package resource would require ?


Laurent

I see 3 options for idempotency:

  1. invoke the reconfiguration only if the :response_file has changed
  • disadvantage: non-chef misconfigurations of a package will go unnoticed.
  1. rely on idempotency of the reconfiguration facility on the specific platforms
  • disadvantage: if the platform implementation changes, idempotency may not be given any more.
  1. implement logic which retrievs the actual configuration values from the node and decide, if a reconfiguration is needed. (maybe even put the retrieval logic to ohai)
  • disadvantage: intense coding necessary, in the case of apt, extra packages need to be installed to be able to retrieve config values or you have to reimplement a database client as part of the provider …

So far I only implemented the :reconfig action for the apt provider. Therefore I chose option 2, because dpkg-reconfigure is already idempotent.


Dennis

https://gist.github.com/1129625 (this is only a hack, but to get an idea about how it could look like)


Dennis


#7

Hi,

I see 3 options for idempotency:

  1. invoke the reconfiguration only if the :response_file has changed
  • disadvantage: non-chef misconfigurations of a package will go unnoticed.
  1. rely on idempotency of the reconfiguration facility on the specific platforms
  • disadvantage: if the platform implementation changes, idempotency may not be given any more.
  1. implement logic which retrievs the actual configuration values from the node and decide, if a reconfiguration is needed. (maybe even put the retrieval logic to ohai)
  • disadvantage: intense coding necessary, in the case of apt, extra packages need to be installed to be able to retrieve config values or you have to reimplement a database client as part of the provider …

So far I only implemented the :reconfig action for the apt provider. Therefore I chose option 2, because dpkg-reconfigure is already idempotent.

https://gist.github.com/1129625 (this is only a hack, but to get an idea about how it could look like)

Ok.
dpkg-reconfigure is idempotent but from what i read the resource will
be set updated_by_last_action on every run. (dpkg-reconfigure always
returns 0 whether changes have been made or not)

I’d say option 3 is the only way to go. What extra packages are you
thinking of ?


Laurent


#8

I see 3 options for idempotency:

  1. invoke the reconfiguration only if the :response_file has changed
  • disadvantage: non-chef misconfigurations of a package will go unnoticed.
  1. rely on idempotency of the reconfiguration facility on the specific platforms
  • disadvantage: if the platform implementation changes, idempotency may not be given any more.
  1. implement logic which retrievs the actual configuration values from the node and decide, if a reconfiguration is needed. (maybe even put the retrieval logic to ohai)
  • disadvantage: intense coding necessary, in the case of apt, extra packages need to be installed to be able to retrieve config values or you have to reimplement a database client as part of the provider …

So far I only implemented the :reconfig action for the apt provider. Therefore I chose option 2, because dpkg-reconfigure is already idempotent.

https://gist.github.com/1129625 (this is only a hack, but to get an idea about how it could look like)

Ok.
dpkg-reconfigure is idempotent but from what i read the resource will
be set updated_by_last_action on every run. (dpkg-reconfigure always
returns 0 whether changes have been made or not)

I’d say option 3 is the only way to go. What extra packages are you
thinking of ?

I was thinking of the package debconf-utils which is not installed by default and which comes along with a command debconf-get-selections. But one could also work with debconf-show I guess.

I wonder what level of idempotency we need to achiev because when I look at the various file and template resources chef is shipping they basically dont care about the actual state of the live file they write. They just determine the state change depending on the chef-client cache (afaik, pls correct me, if I am wrong here). So, I feel to go along with the common “chef sense” of idempotency even option 1 would be sufficient ?!

If we would go with option 3, what do you think about putting the config value retrieval logic into ohai and the comparison logic only into the package providers (Could this work with chef-solo too?!).


Dennis


#9

Dennis Klein d.klein@gsi.de writes:

https://gist.github.com/1129625 (this is only a hack, but to get an idea about how it could look like)

Ok.
dpkg-reconfigure is idempotent but from what i read the resource will
be set updated_by_last_action on every run. (dpkg-reconfigure always
returns 0 whether changes have been made or not)

I’d say option 3 is the only way to go. What extra packages are you
thinking of ?

I was thinking of the package debconf-utils which is not installed by default and which comes along with a command debconf-get-selections. But one could also work with debconf-show I guess.

Ok, yes i was thinking of parsing the result of debconf-show.

I wonder what level of idempotency we need to achiev because when I look at the various file and template resources chef is shipping they basically dont care about the actual state of the live file they write. They just determine the state change depending on the chef-client cache (afaik, pls correct me, if I am wrong here). So, I feel to go along with the common “chef sense” of idempotency even option 1 would be sufficient ?!

I can’t really say.

If we would go with option 3, what do you think about putting the config value retrieval logic into ohai and the comparison logic only into the package providers (Could this work with chef-solo too?!).

Same here, I must admit I had the same question when i started writing
a lrwp. In the end i chose the load_current_resource way. If i could
require ohai plugins in my cookbook or lwrp i might do it with an ohai
plugin. (in that case I guess that would be easier to have the
attributes indexed)

Concerning the response file, at the moment i’m using the logic you
described in point 1, debconf-set-selection and dpkg-reconfigure are
run from an execute resource when a response_file template changes.
(guess i can make a definition out of that)

For point 2, you just have to find a way to detect if dpkg-reconfigure
actually did something to set updated_by_last_action correctly.
in the end i guess you’ll have to compare the result of debconf-show
before and after “preseed_package”. if it’s different then run
dpkg-reconfigure and set updated_by_last_action to true.


Laurent


#10

On Sun, Aug 7, 2011 at 07:10, Dennis Klein d.klein@gsi.de wrote:

I wonder what level of idempotency we need to achiev because when I look at
the various file and template resources chef is shipping they basically dont
care about the actual state of the live file they write. They just determine
the state change depending on the chef-client cache (afaik, pls correct me,
if I am wrong here). So, I feel to go along with the common “chef sense” of
idempotency even option 1 would be sufficient ?!

There is a checksum compare of the live file as the test just before writing
it, a good guarantee of idempotency. See for example
https://github.com/opscode/chef/blob/master/chef/lib/chef/provider/file.rb#L65

You’ll want an idempotent test as tight as is reasonable - you don’t want to
be fooled in trivial cases, but truly confirming the existing state can be
expensive or difficult, as you’ve discussed. So we compromise: Chef trusts
what package management says about the state of a package, which will not
discover or fix a package if a file it delivered is somehow clobbered.

The problem here is reconfigure does a number of things and hides the
outcome of “did anything change?” which it may not actually know depending
on implementation. So you can have it :

  • have the resource run every time - system inefficient, admin efficient,
    “probably idempotent enough”
  • (from your 1&2 above) be satisfied with a looser, less accurate sense of
    idempotency - maybe okay, but situation dependent
  • (from 3) essentially re-implement components of dpkg using concepts of
    idempotency…but that sounds awfully familiar to and a lot harder than
    this:
  • manage the package components after their initial installation with chef

Of course that last one means giving up some benefits of being able to do
dpkg-managed reconfigures, but trades that for explicitly managing
and understanding the state of the contents of a given package in the same
way as rest of the configuration management you’re doing with Chef.

So for the purposes of having reconfig in there at all, I’d keep it pretty
simple by having the provider not have a built-in concept of idempotency,
like execute or script, and default to using it with a notifies/subscribes
action :reconfig attached to the template that delivers the response_file.


Aaron Peterson aaron@opscode.com
Opscode Technical Evangelist


Aaron Peterson aaron@opscode.com
Opscode Technical Evangelist