CHEF-3930: Run apt-get update automagically if apt-get install fails


#1

Should the apt provider try to run apt-get update if :install fails?

We talked about this a bunch at code review [1] today, starting at
22:45. http://www.youtube.com/watch?v=8pOd8NzmKIY

  • This happens all the time, thanks man.
  • I told Chef to install a package, not update my sources! Get off my lawn!

In all seriousness, it’s a pretty interesting boundary between Chef
being pretty helpful versus Chef being strict about its role. There
isn’t a solution here that pleases everyone, but I’m curious about any
other opinions.

Bryan

[1] http://wiki.opscode.com/display/chef/Code+Review


#2

What about making it opt-in / opt-out with an attribute?
I don’t much care which way the default is, but it would sure be nice to
have the option.

On Thu, Feb 28, 2013 at 12:01 AM, Bryan McLellan btm@loftninjas.org wrote:

Should the apt provider try to run apt-get update if :install fails?

We talked about this a bunch at code review [1] today, starting at
22:45. http://www.youtube.com/watch?v=8pOd8NzmKIY

  • This happens all the time, thanks man.
  • I told Chef to install a package, not update my sources! Get off my lawn!

In all seriousness, it’s a pretty interesting boundary between Chef
being pretty helpful versus Chef being strict about its role. There
isn’t a solution here that pleases everyone, but I’m curious about any
other opinions.

Bryan

[1] http://wiki.opscode.com/display/chef/Code+Review


#3

On Wed, Feb 27, 2013 at 6:05 PM, Andrea Campi
andrea.campi@zephirworks.com wrote:

What about making it opt-in / opt-out with an attribute?
I don’t much care which way the default is, but it would sure be nice to
have the option.

It would have to be a global attribute and the only established
pattern we have for that is Chef::Config options. These can kind of be
managed by templatizing the client.rb (e.g. chef-client cookbook) but
it’s less of a “flag” than would be convenient for this sort of thing.

The reason it can’t be a resource attribute is if you want it the
other way around, you’re populating “auto_update true/false” in every
package resource in every cookbook if you want it the other way
around. Thus if you default it to false, you’re not doing yourself
much of a favor over the existing alternatives of doing it well
(apt-update on mtime at the start of run in base, apt-get update on
apt_repository LWRP, etc).

But yeah, it feels awesome. Of course the apt package provider should
help me with my apt-cache! We also felt it violates a Chef Law, but
none of us could name it.

Bryan


#4

On Thu, Feb 28, 2013 at 12:19 AM, Bryan McLellan btm@loftninjas.org wrote:

On Wed, Feb 27, 2013 at 6:05 PM, Andrea Campi
andrea.campi@zephirworks.com wrote:

What about making it opt-in / opt-out with an attribute?
I don’t much care which way the default is, but it would sure be nice to
have the option.

It would have to be a global attribute and the only established
pattern we have for that is Chef::Config options. These can kind of be
managed by templatizing the client.rb (e.g. chef-client cookbook) but
it’s less of a “flag” than would be convenient for this sort of thing.

The reason it can’t be a resource attribute is if you want it the
other way around, you’re populating “auto_update true/false” in every
package resource in every cookbook if you want it the other way
around.

D’oh, for some reason I was thinking “cookbook”.

But yeah, it feels awesome. Of course the apt package provider should
help me with my apt-cache! We also felt it violates a Chef Law, but
none of us could name it.

Yeah, it feels a bit “do what I mean”, but in this case I think that’s
quite reasonable.

Playing devil’s advocate, how will that work with cookbooks that use
apt_repository, which runs apt-get update ?
Worst case there will be more than one apt-get update in a single run (and
that probably only if the package is not there for real).
Can you think of other edge cases?


#5

On Wed, Feb 27, 2013 at 6:44 PM, Andrea Campi
andrea.campi@zephirworks.com wrote:

On Thu, Feb 28, 2013 at 12:19 AM, Bryan McLellan btm@loftninjas.org wrote:

Is anyone opposed to the idea of it simply being a flag, that in the
case of a missing package, apt-get update or yum update is called and
a single retry is performed? That would likely solve 95% of the
problems I have when this happens.

On Wed, Feb 27, 2013 at 6:05 PM, Andrea Campi
andrea.campi@zephirworks.com wrote:

What about making it opt-in / opt-out with an attribute?
I don’t much care which way the default is, but it would sure be nice to
have the option.

It would have to be a global attribute and the only established
pattern we have for that is Chef::Config options. These can kind of be
managed by templatizing the client.rb (e.g. chef-client cookbook) but
it’s less of a “flag” than would be convenient for this sort of thing.

The reason it can’t be a resource attribute is if you want it the
other way around, you’re populating “auto_update true/false” in every
package resource in every cookbook if you want it the other way
around.

D’oh, for some reason I was thinking “cookbook”.

But yeah, it feels awesome. Of course the apt package provider should
help me with my apt-cache! We also felt it violates a Chef Law, but
none of us could name it.

Yeah, it feels a bit “do what I mean”, but in this case I think that’s quite
reasonable.

Playing devil’s advocate, how will that work with cookbooks that use
apt_repository, which runs apt-get update ?
Worst case there will be more than one apt-get update in a single run (and
that probably only if the package is not there for real).
Can you think of other edge cases?


#6

I would be +1 for this - in a previous life we had to often run apt-get
update two or three times in a run just to cover edge cases when a solution
like this could have helped us or made things as least less repetitive
during a chef run.

-Pete

On Wed, Feb 27, 2013 at 7:20 PM, John E. Vincent (lusis) <
lusis.org+chef-list@gmail.com> wrote:

On Wed, Feb 27, 2013 at 6:44 PM, Andrea Campi
andrea.campi@zephirworks.com wrote:

On Thu, Feb 28, 2013 at 12:19 AM, Bryan McLellan btm@loftninjas.org
wrote:

Is anyone opposed to the idea of it simply being a flag, that in the
case of a missing package, apt-get update or yum update is called and
a single retry is performed? That would likely solve 95% of the
problems I have when this happens.

On Wed, Feb 27, 2013 at 6:05 PM, Andrea Campi
andrea.campi@zephirworks.com wrote:

What about making it opt-in / opt-out with an attribute?
I don’t much care which way the default is, but it would sure be nice
to

have the option.

It would have to be a global attribute and the only established
pattern we have for that is Chef::Config options. These can kind of be
managed by templatizing the client.rb (e.g. chef-client cookbook) but
it’s less of a “flag” than would be convenient for this sort of thing.

The reason it can’t be a resource attribute is if you want it the
other way around, you’re populating “auto_update true/false” in every
package resource in every cookbook if you want it the other way
around.

D’oh, for some reason I was thinking “cookbook”.

But yeah, it feels awesome. Of course the apt package provider should
help me with my apt-cache! We also felt it violates a Chef Law, but
none of us could name it.

Yeah, it feels a bit “do what I mean”, but in this case I think that’s
quite
reasonable.

Playing devil’s advocate, how will that work with cookbooks that use
apt_repository, which runs apt-get update ?
Worst case there will be more than one apt-get update in a single run
(and
that probably only if the package is not there for real).
Can you think of other edge cases?


#7

I also feel like this is crossing the line. Where now we are doing
something that we weren’t asked to do simply because we think we know what
you want. What happens if that is exactly what a user doesn’t want. Tho I
can’t see why an apt-get update would be a bad thing or break anything on
the surface. It just smells bad. If it is to be implemented it has to be an
optional disable.

On the Side note I like the idea of error callbacks for providers or retry
hooks. Something we could code at a high level for some providers to do
when failure happens to try to recover. This was brought up and then
dismissed, but I could see there being some interesting uses. This being
one of them.

On Wed, Feb 27, 2013 at 4:24 PM, Pete Cheslock petecheslock@gmail.comwrote:

I would be +1 for this - in a previous life we had to often run apt-get
update two or three times in a run just to cover edge cases when a solution
like this could have helped us or made things as least less repetitive
during a chef run.

-Pete

On Wed, Feb 27, 2013 at 7:20 PM, John E. Vincent (lusis) <
lusis.org+chef-list@gmail.com> wrote:

On Wed, Feb 27, 2013 at 6:44 PM, Andrea Campi
andrea.campi@zephirworks.com wrote:

On Thu, Feb 28, 2013 at 12:19 AM, Bryan McLellan btm@loftninjas.org
wrote:

Is anyone opposed to the idea of it simply being a flag, that in the
case of a missing package, apt-get update or yum update is called and
a single retry is performed? That would likely solve 95% of the
problems I have when this happens.

On Wed, Feb 27, 2013 at 6:05 PM, Andrea Campi
andrea.campi@zephirworks.com wrote:

What about making it opt-in / opt-out with an attribute?
I don’t much care which way the default is, but it would sure be
nice to

have the option.

It would have to be a global attribute and the only established
pattern we have for that is Chef::Config options. These can kind of be
managed by templatizing the client.rb (e.g. chef-client cookbook) but
it’s less of a “flag” than would be convenient for this sort of thing.

The reason it can’t be a resource attribute is if you want it the
other way around, you’re populating “auto_update true/false” in every
package resource in every cookbook if you want it the other way
around.

D’oh, for some reason I was thinking “cookbook”.

But yeah, it feels awesome. Of course the apt package provider should
help me with my apt-cache! We also felt it violates a Chef Law, but
none of us could name it.

Yeah, it feels a bit “do what I mean”, but in this case I think that’s
quite
reasonable.

Playing devil’s advocate, how will that work with cookbooks that use
apt_repository, which runs apt-get update ?
Worst case there will be more than one apt-get update in a single run
(and
that probably only if the package is not there for real).
Can you think of other edge cases?


#8

Yeah, I’m not a huge fan, for the same reason – we move from you being explicit, to our making decisions about behavior.

Adam

From: Jesse Nelson <spheromak@gmail.commailto:spheromak@gmail.com>
Reply-To: "chef@lists.opscode.commailto:chef@lists.opscode.com" <chef@lists.opscode.commailto:chef@lists.opscode.com>
Date: Wednesday, February 27, 2013 8:07 PM
To: chef <chef@lists.opscode.commailto:chef@lists.opscode.com>
Subject: [chef] Re: Re: Re: Re: Re: Re: CHEF-3930: Run apt-get update automagically if apt-get install fails

I also feel like this is crossing the line. Where now we are doing something that we weren’t asked to do simply because we think we know what you want. What happens if that is exactly what a user doesn’t want. Tho I can’t see why an apt-get update would be a bad thing or break anything on the surface. It just smells bad. If it is to be implemented it has to be an optional disable.

On the Side note I like the idea of error callbacks for providers or retry hooks. Something we could code at a high level for some providers to do when failure happens to try to recover. This was brought up and then dismissed, but I could see there being some interesting uses. This being one of them.

On Wed, Feb 27, 2013 at 4:24 PM, Pete Cheslock <petecheslock@gmail.commailto:petecheslock@gmail.com> wrote:
I would be +1 for this - in a previous life we had to often run apt-get update two or three times in a run just to cover edge cases when a solution like this could have helped us or made things as least less repetitive during a chef run.

-Pete

On Wed, Feb 27, 2013 at 7:20 PM, John E. Vincent (lusis) <lusis.org+chef-list@gmail.commailto:lusis.org+chef-list@gmail.com> wrote:
On Wed, Feb 27, 2013 at 6:44 PM, Andrea Campi
<andrea.campi@zephirworks.commailto:andrea.campi@zephirworks.com> wrote:

On Thu, Feb 28, 2013 at 12:19 AM, Bryan McLellan <btm@loftninjas.orgmailto:btm@loftninjas.org> wrote:

Is anyone opposed to the idea of it simply being a flag, that in the
case of a missing package, apt-get update or yum update is called and
a single retry is performed? That would likely solve 95% of the
problems I have when this happens.

On Wed, Feb 27, 2013 at 6:05 PM, Andrea Campi
<andrea.campi@zephirworks.commailto:andrea.campi@zephirworks.com> wrote:

What about making it opt-in / opt-out with an attribute?
I don’t much care which way the default is, but it would sure be nice to
have the option.

It would have to be a global attribute and the only established
pattern we have for that is Chef::Config options. These can kind of be
managed by templatizing the client.rb (e.g. chef-client cookbook) but
it’s less of a “flag” than would be convenient for this sort of thing.

The reason it can’t be a resource attribute is if you want it the
other way around, you’re populating “auto_update true/false” in every
package resource in every cookbook if you want it the other way
around.

D’oh, for some reason I was thinking “cookbook”.

But yeah, it feels awesome. Of course the apt package provider should
help me with my apt-cache! We also felt it violates a Chef Law, but
none of us could name it.

Yeah, it feels a bit “do what I mean”, but in this case I think that’s quite
reasonable.

Playing devil’s advocate, how will that work with cookbooks that use
apt_repository, which runs apt-get update ?
Worst case there will be more than one apt-get update in a single run (and
that probably only if the package is not there for real).
Can you think of other edge cases?


#9

The issue here seems to be a conflict between the general Chef philosophy
and what a generally sane default. I noticed similar behavior in

http://tickets.opscode.com/browse/CHEF-3734

Where the git resource was behaving in a similar way to the deploy
resource, even though that was not what was explicitly asked for. In this
case it looks like the change is leaning towards making git do explicitly
what was asked and no more.

It feels similar to the conflict when writing an API, you want to have a
layer where you have definitions that provide explicit behavior so you can
manipulate the system. At the same time though you want to provide a good
interface to the end user and give them a smooth interface that avoids many
of the pitfalls associated with inexperience. Normally this is solved by
just writing two libraries and having the user facing library wrap the
explicit one, however Chef doesn’t seem to quite fit in as either choice
(You could also argue that Chef is the wrapper library and the OS is the
lower level API).

On a side note:

I like the idea of having a generic system for handling recovery from
simple issues. I can see the logic about not having Chef do implicit
changes that may be unexpected. A downside to using an explicit retry hook
for something like packages is that you end up having to write it in to
every block, making your recipes seem very cluttered. It would be nice if
along with a retry system there was a simpler way to modify defaults for a
provider without needing to monkey patch the resource definition or
wrapping the call with your own LWRP.

Andrew

On Thu, Feb 28, 2013 at 12:43 AM, Adam Jacob adam@opscode.com wrote:

Yeah, I’m not a huge fan, for the same reason – we move from you being
explicit, to our making decisions about behavior.

Adam

From: Jesse Nelson spheromak@gmail.com
Reply-To: "chef@lists.opscode.com" chef@lists.opscode.com
Date: Wednesday, February 27, 2013 8:07 PM
To: chef chef@lists.opscode.com
Subject: [chef] Re: Re: Re: Re: Re: Re: CHEF-3930: Run apt-get update
automagically if apt-get install fails

I also feel like this is crossing the line. Where now we are doing
something that we weren’t asked to do simply because we think we know what
you want. What happens if that is exactly what a user doesn’t want. Tho I
can’t see why an apt-get update would be a bad thing or break anything on
the surface. It just smells bad. If it is to be implemented it has to be an
optional disable.

On the Side note I like the idea of error callbacks for providers or
retry hooks. Something we could code at a high level for some providers to
do when failure happens to try to recover. This was brought up and then
dismissed, but I could see there being some interesting uses. This being
one of them.

On Wed, Feb 27, 2013 at 4:24 PM, Pete Cheslock petecheslock@gmail.comwrote:

I would be +1 for this - in a previous life we had to often run apt-get
update two or three times in a run just to cover edge cases when a solution
like this could have helped us or made things as least less repetitive
during a chef run.

-Pete

On Wed, Feb 27, 2013 at 7:20 PM, John E. Vincent (lusis) <
lusis.org+chef-list@gmail.com> wrote:

On Wed, Feb 27, 2013 at 6:44 PM, Andrea Campi
andrea.campi@zephirworks.com wrote:

On Thu, Feb 28, 2013 at 12:19 AM, Bryan McLellan btm@loftninjas.org
wrote:

Is anyone opposed to the idea of it simply being a flag, that in the
case of a missing package, apt-get update or yum update is called and
a single retry is performed? That would likely solve 95% of the
problems I have when this happens.

On Wed, Feb 27, 2013 at 6:05 PM, Andrea Campi
andrea.campi@zephirworks.com wrote:

What about making it opt-in / opt-out with an attribute?
I don’t much care which way the default is, but it would sure be
nice to

have the option.

It would have to be a global attribute and the only established
pattern we have for that is Chef::Config options. These can kind of be
managed by templatizing the client.rb (e.g. chef-client cookbook) but
it’s less of a “flag” than would be convenient for this sort of thing.

The reason it can’t be a resource attribute is if you want it the
other way around, you’re populating “auto_update true/false” in every
package resource in every cookbook if you want it the other way
around.

D’oh, for some reason I was thinking “cookbook”.

But yeah, it feels awesome. Of course the apt package provider should
help me with my apt-cache! We also felt it violates a Chef Law, but
none of us could name it.

Yeah, it feels a bit “do what I mean”, but in this case I think that’s
quite
reasonable.

Playing devil’s advocate, how will that work with cookbooks that use
apt_repository, which runs apt-get update ?
Worst case there will be more than one apt-get update in a single run
(and
that probably only if the package is not there for real).
Can you think of other edge cases?


#10

Perhaps being able to specify, for any given resource

on_failure_notifies :action, “resource_type[name]”), :immediately_and_retry

would allow both problems to be solved?

Jeffrey Hulten
Principal Consultant at Automated Labs
jeffh@automatedlabs.com 206-853-5216
Skype: jeffhulten

On Feb 28, 2013, at 7:10 AM, Andrew Gross andrew@yipit.com wrote:

The issue here seems to be a conflict between the general Chef philosophy and what a generally sane default. I noticed similar behavior in

http://tickets.opscode.com/browse/CHEF-3734

Where the git resource was behaving in a similar way to the deploy resource, even though that was not what was explicitly asked for. In this case it looks like the change is leaning towards making git do explicitly what was asked and no more.

It feels similar to the conflict when writing an API, you want to have a layer where you have definitions that provide explicit behavior so you can manipulate the system. At the same time though you want to provide a good interface to the end user and give them a smooth interface that avoids many of the pitfalls associated with inexperience. Normally this is solved by just writing two libraries and having the user facing library wrap the explicit one, however Chef doesn’t seem to quite fit in as either choice (You could also argue that Chef is the wrapper library and the OS is the lower level API).

On a side note:

I like the idea of having a generic system for handling recovery from simple issues. I can see the logic about not having Chef do implicit changes that may be unexpected. A downside to using an explicit retry hook for something like packages is that you end up having to write it in to every block, making your recipes seem very cluttered. It would be nice if along with a retry system there was a simpler way to modify defaults for a provider without needing to monkey patch the resource definition or wrapping the call with your own LWRP.

Andrew

On Thu, Feb 28, 2013 at 12:43 AM, Adam Jacob adam@opscode.com wrote:
Yeah, I’m not a huge fan, for the same reason – we move from you being explicit, to our making decisions about behavior.

Adam

From: Jesse Nelson spheromak@gmail.com
Reply-To: "chef@lists.opscode.com" chef@lists.opscode.com
Date: Wednesday, February 27, 2013 8:07 PM
To: chef chef@lists.opscode.com
Subject: [chef] Re: Re: Re: Re: Re: Re: CHEF-3930: Run apt-get update automagically if apt-get install fails

I also feel like this is crossing the line. Where now we are doing something that we weren’t asked to do simply because we think we know what you want. What happens if that is exactly what a user doesn’t want. Tho I can’t see why an apt-get update would be a bad thing or break anything on the surface. It just smells bad. If it is to be implemented it has to be an optional disable.

On the Side note I like the idea of error callbacks for providers or retry hooks. Something we could code at a high level for some providers to do when failure happens to try to recover. This was brought up and then dismissed, but I could see there being some interesting uses. This being one of them.

On Wed, Feb 27, 2013 at 4:24 PM, Pete Cheslock petecheslock@gmail.com wrote:
I would be +1 for this - in a previous life we had to often run apt-get update two or three times in a run just to cover edge cases when a solution like this could have helped us or made things as least less repetitive during a chef run.

-Pete

On Wed, Feb 27, 2013 at 7:20 PM, John E. Vincent (lusis) lusis.org+chef-list@gmail.com wrote:
On Wed, Feb 27, 2013 at 6:44 PM, Andrea Campi
andrea.campi@zephirworks.com wrote:

On Thu, Feb 28, 2013 at 12:19 AM, Bryan McLellan btm@loftninjas.org wrote:

Is anyone opposed to the idea of it simply being a flag, that in the
case of a missing package, apt-get update or yum update is called and
a single retry is performed? That would likely solve 95% of the
problems I have when this happens.

On Wed, Feb 27, 2013 at 6:05 PM, Andrea Campi
andrea.campi@zephirworks.com wrote:

What about making it opt-in / opt-out with an attribute?
I don’t much care which way the default is, but it would sure be nice to
have the option.

It would have to be a global attribute and the only established
pattern we have for that is Chef::Config options. These can kind of be
managed by templatizing the client.rb (e.g. chef-client cookbook) but
it’s less of a “flag” than would be convenient for this sort of thing.

The reason it can’t be a resource attribute is if you want it the
other way around, you’re populating “auto_update true/false” in every
package resource in every cookbook if you want it the other way
around.

D’oh, for some reason I was thinking “cookbook”.

But yeah, it feels awesome. Of course the apt package provider should
help me with my apt-cache! We also felt it violates a Chef Law, but
none of us could name it.

Yeah, it feels a bit “do what I mean”, but in this case I think that’s quite
reasonable.

Playing devil’s advocate, how will that work with cookbooks that use
apt_repository, which runs apt-get update ?
Worst case there will be more than one apt-get update in a single run (and
that probably only if the package is not there for real).
Can you think of other edge cases?


#11

It seems like that would still require updating every resource with that
bit of code.

I prefer to have a central location for me to explicitly define this
behavior and allow it to “do what I mean”. Having the default be to have
Chef refuse to cross the line and implicitly do what I expect it to do is
fine, but I do believe there should be an option I can turn on that will
tell Chef “Yeah, I know you’re doing more than I’m explicitly telling you
to do each and every time I configure individual resources, but I’m okay
with that because this is the behavior I expect and I’d prefer to to the
equivalent of going to my coffee vendor and ordering ‘The usual’ instead of
having to say every time ‘Large coffee, 3 creams, 1 sugar’ because that’s
always what I’m going to want unless I explicitly tell you differently.”

On Thu, Feb 28, 2013 at 1:50 PM, Jeffrey Hulten jeffh@automatedlabs.comwrote:

Perhaps being able to specify, for any given resource

on_failure_notifies :action, “resource_type[name]”), :immediately_and_retry

would allow both problems to be solved?

Jeffrey Hulten
Principal Consultant at Automated Labs
jeffh@automatedlabs.com 206-853-5216
Skype: jeffhulten

On Feb 28, 2013, at 7:10 AM, Andrew Gross andrew@yipit.com wrote:

The issue here seems to be a conflict between the general Chef
philosophy and what a generally sane default. I noticed similar behavior in

http://tickets.opscode.com/browse/CHEF-3734

Where the git resource was behaving in a similar way to the deploy
resource, even though that was not what was explicitly asked for. In this
case it looks like the change is leaning towards making git do explicitly
what was asked and no more.

It feels similar to the conflict when writing an API, you want to have a
layer where you have definitions that provide explicit behavior so you can
manipulate the system. At the same time though you want to provide a good
interface to the end user and give them a smooth interface that avoids many
of the pitfalls associated with inexperience. Normally this is solved by
just writing two libraries and having the user facing library wrap the
explicit one, however Chef doesn’t seem to quite fit in as either choice
(You could also argue that Chef is the wrapper library and the OS is the
lower level API).

On a side note:

I like the idea of having a generic system for handling recovery from
simple issues. I can see the logic about not having Chef do implicit
changes that may be unexpected. A downside to using an explicit retry hook
for something like packages is that you end up having to write it in to
every block, making your recipes seem very cluttered. It would be nice if
along with a retry system there was a simpler way to modify defaults for a
provider without needing to monkey patch the resource definition or
wrapping the call with your own LWRP.

Andrew

On Thu, Feb 28, 2013 at 12:43 AM, Adam Jacob adam@opscode.com wrote:
Yeah, I’m not a huge fan, for the same reason – we move from you being
explicit, to our making decisions about behavior.

Adam

From: Jesse Nelson spheromak@gmail.com
Reply-To: "chef@lists.opscode.com" chef@lists.opscode.com
Date: Wednesday, February 27, 2013 8:07 PM
To: chef chef@lists.opscode.com
Subject: [chef] Re: Re: Re: Re: Re: Re: CHEF-3930: Run apt-get update
automagically if apt-get install fails

I also feel like this is crossing the line. Where now we are doing
something that we weren’t asked to do simply because we think we know what
you want. What happens if that is exactly what a user doesn’t want. Tho I
can’t see why an apt-get update would be a bad thing or break anything on
the surface. It just smells bad. If it is to be implemented it has to be an
optional disable.

On the Side note I like the idea of error callbacks for providers or
retry hooks. Something we could code at a high level for some providers to
do when failure happens to try to recover. This was brought up and then
dismissed, but I could see there being some interesting uses. This being
one of them.

On Wed, Feb 27, 2013 at 4:24 PM, Pete Cheslock petecheslock@gmail.com
wrote:
I would be +1 for this - in a previous life we had to often run apt-get
update two or three times in a run just to cover edge cases when a solution
like this could have helped us or made things as least less repetitive
during a chef run.

-Pete

On Wed, Feb 27, 2013 at 7:20 PM, John E. Vincent (lusis) <lusis.org+
chef-list@gmail.com> wrote:
On Wed, Feb 27, 2013 at 6:44 PM, Andrea Campi
andrea.campi@zephirworks.com wrote:

On Thu, Feb 28, 2013 at 12:19 AM, Bryan McLellan btm@loftninjas.org
wrote:

Is anyone opposed to the idea of it simply being a flag, that in the
case of a missing package, apt-get update or yum update is called and
a single retry is performed? That would likely solve 95% of the
problems I have when this happens.

On Wed, Feb 27, 2013 at 6:05 PM, Andrea Campi
andrea.campi@zephirworks.com wrote:

What about making it opt-in / opt-out with an attribute?
I don’t much care which way the default is, but it would sure be
nice to

have the option.

It would have to be a global attribute and the only established
pattern we have for that is Chef::Config options. These can kind of be
managed by templatizing the client.rb (e.g. chef-client cookbook) but
it’s less of a “flag” than would be convenient for this sort of thing.

The reason it can’t be a resource attribute is if you want it the
other way around, you’re populating “auto_update true/false” in every
package resource in every cookbook if you want it the other way
around.

D’oh, for some reason I was thinking “cookbook”.

But yeah, it feels awesome. Of course the apt package provider should
help me with my apt-cache! We also felt it violates a Chef Law, but
none of us could name it.

Yeah, it feels a bit “do what I mean”, but in this case I think that’s
quite

reasonable.

Playing devil’s advocate, how will that work with cookbooks that use
apt_repository, which runs apt-get update ?
Worst case there will be more than one apt-get update in a single run
(and

that probably only if the package is not there for real).
Can you think of other edge cases?


#12

On Fri, Mar 1, 2013 at 11:21 PM, Jesse Campbell hikeit@gmail.com wrote:

What about extending the current handler approach? Add the ability to catch
a failed resource (by name) and retry it, just like you can with a
begin/rescue.

I like the concept of having an exception handler be tied to a
resource or provider, or at least be able to determine what resource
or provider caused the exception, do some bits and retry.

Bryan


#13

Why don’t we make this a resource?

package “totally-sucks-devel”

on_failure “run apt-get update” do
match /^package/
notifies :run, "execute[apt-get-update]"
end

Where match is a regular expression that looks for names of resources.

Adam

On 3/4/13 9:06 AM, “Bryan McLellan” btm@loftninjas.org wrote:

On Fri, Mar 1, 2013 at 11:21 PM, Jesse Campbell hikeit@gmail.com wrote:

What about extending the current handler approach? Add the ability to
catch
a failed resource (by name) and retry it, just like you can with a
begin/rescue.

I like the concept of having an exception handler be tied to a
resource or provider, or at least be able to determine what resource
or provider caused the exception, do some bits and retry.

Bryan


#14

Oooh… that looks yummy. I like the idea of being able to stop a
service if a later resource (say something being deployed) fails.

Greg

On 03/04/2013 11:13 AM, Adam Jacob wrote:

Why don’t we make this a resource?

package “totally-sucks-devel”

on_failure “run apt-get update” do
match /^package/
notifies :run, "execute[apt-get-update]"
end

Where match is a regular expression that looks for names of resources.

Adam

On 3/4/13 9:06 AM, “Bryan McLellan” btm@loftninjas.org wrote:

On Fri, Mar 1, 2013 at 11:21 PM, Jesse Campbell hikeit@gmail.com wrote:

What about extending the current handler approach? Add the ability to
catch
a failed resource (by name) and retry it, just like you can with a
begin/rescue.
I like the concept of having an exception handler be tied to a
resource or provider, or at least be able to determine what resource
or provider caused the exception, do some bits and retry.

Bryan


#15

What would be a good way to prevent it from running in all cases?

only_if? not_if? Some provider specific parameter?

package “must-be-version-locked” do
version "foo"
end

package “totally-sucks-devel”

on_failure “run apt-get update” do
match /^package/
no_match /version/
notifies :run, "execute[apt-get-update]"
end

I like the concept, should be careful to set it up in a way that doesn’t
cause unexpected side effects, especially when including things from
dependencies. Would it be possible to scope it to the calling cookbook?

On Mon, Mar 4, 2013 at 1:29 PM, Greg Symons gsymons@drillinginfo.comwrote:

Oooh… that looks yummy. I like the idea of being able to stop a service
if a later resource (say something being deployed) fails.

Greg

On 03/04/2013 11:13 AM, Adam Jacob wrote:

Why don’t we make this a resource?

package “totally-sucks-devel”

on_failure “run apt-get update” do
match /^package/
notifies :run, "execute[apt-get-update]"
end

Where match is a regular expression that looks for names of resources.

Adam

On 3/4/13 9:06 AM, “Bryan McLellan” btm@loftninjas.org wrote:

On Fri, Mar 1, 2013 at 11:21 PM, Jesse Campbell hikeit@gmail.com

wrote:

What about extending the current handler approach? Add the ability to
catch
a failed resource (by name) and retry it, just like you can with a
begin/rescue.

I like the concept of having an exception handler be tied to a
resource or provider, or at least be able to determine what resource
or provider caused the exception, do some bits and retry.

Bryan


#16

I like this concept too, and add Bryan’s point about knowing what triggered
it, something like this (sorry, a good bit of pseudo code here)

package “totally-sucks-devel”

on_failure “run apt-get update” do
match /^package/
block do <------- optional, is passed a failed_resource
object
puts "#{failed_resource.name} triggered this error!"
if failed_resource.name != "totally-sucks-devel"
raise
end
end
end

On Mon, Mar 4, 2013 at 1:54 PM, Andrew Gross andrew@yipit.com wrote:

What would be a good way to prevent it from running in all cases?

only_if? not_if? Some provider specific parameter?

package “must-be-version-locked” do
version "foo"
end

package “totally-sucks-devel”

on_failure “run apt-get update” do
match /^package/
no_match /version/
notifies :run, "execute[apt-get-update]"
end

I like the concept, should be careful to set it up in a way that doesn’t
cause unexpected side effects, especially when including things from
dependencies. Would it be possible to scope it to the calling cookbook?

On Mon, Mar 4, 2013 at 1:29 PM, Greg Symons gsymons@drillinginfo.comwrote:

Oooh… that looks yummy. I like the idea of being able to stop a service
if a later resource (say something being deployed) fails.

Greg

On 03/04/2013 11:13 AM, Adam Jacob wrote:

Why don’t we make this a resource?

package “totally-sucks-devel”

on_failure “run apt-get update” do
match /^package/
notifies :run, "execute[apt-get-update]"
end

Where match is a regular expression that looks for names of resources.

Adam

On 3/4/13 9:06 AM, “Bryan McLellan” btm@loftninjas.org wrote:

On Fri, Mar 1, 2013 at 11:21 PM, Jesse Campbell hikeit@gmail.com

wrote:

What about extending the current handler approach? Add the ability to
catch
a failed resource (by name) and retry it, just like you can with a
begin/rescue.

I like the concept of having an exception handler be tied to a
resource or provider, or at least be able to determine what resource
or provider caused the exception, do some bits and retry.

Bryan


#17

On Mon, Mar 4, 2013 at 12:13 PM, Adam Jacob adam@opscode.com wrote:

Why don’t we make this a resource?

package “totally-sucks-devel”

on_failure “run apt-get update” do
match /^package/
notifies :run, "execute[apt-get-update]"
end

Where match is a regular expression that looks for names of resources.

Totally, this feels really useful, even if it could be a terrible idea still.

On Mon, Mar 4, 2013 at 2:33 PM, Kevin Keane Subscription
subscription@kkeane.com wrote:

Isn’t the philosophy of Chef that resources should describe the final, desired, state of the system? This resource just feels “procedural” rather than “declarative”.

Or am I way off on this?

I agree that the “correct” fix for apt-get issues is tracking down
what’s wrong and fixing your run. For instance, if your instances are
from EC2 and your apt-cache is always stale, you should ensure that
apt-get update happens at the beginning of your run, perhaps being
very early in a base role. If you are having issues after installing a
repository, you should be using the functionality in the apt cookbook.
If you always need the latest update before a package install you have
to bite the bullet and always run apt-get update. However, in this
example it is clear that we’d solve a lot of frustration for a bunch
of people if we didn’t force them to do it the ideal way, and instead
gave them options to get it fixed and get back to work.

On the other hand, it’s hard to think up other situations like this
that justify a feature that someone can get themselves so far down the
wrong path with. I like the idea of handling network or mirror issues
that the tools might handle poorly.

on_failure “rotate mirrors for dumb_tool” do
match /^execute[dumb_tool]$/
notifies :run, "execute[swap_dumb_tool_mirrors]"
retries 3
end

aws_timeout = 30
aws_retries = 2

on_failure “AWS API timeout” do
match /-aws]$/
block do
Chef::Log.warn(“Failure in #{failed_resource}, maybe an AWS issue,
pausing #{aws_timeout} seconds and retrying #{aws_retries} times”)
sleep aws_timeout
end
retries aws_retries
end

on_failure “github is down” do
match /^git[our-code-/
block do
Chef::Log.warn(“Failed to act on repository
#{failed_resource.repository} in resource #{failed_resource}, trying
internal mirror”)
failed_resource.repository.gsub(/^http://github.com/opscode/,
“git://backup_git_mirror”)
end
end

Bryan


#18

Hey All,

I wanted to recap this thread so we can make a decision on it before it
gets lost in the shuffle:

Initial Pull Request Goal: Prevent failed Chef runs due to apt-get failures
that can be fixed by running apt-get update

Evolved Goal: Provide some way to handle ‘expected’ failures. Either per
block (notification style), as a global setting, or a special block that
can watch certain blocks.

Gotchas: Don’t make Chef do implicit things, aim to be explicit

On Mon, Mar 4, 2013 at 8:12 PM, Bryan McLellan btm@loftninjas.org wrote:

On Mon, Mar 4, 2013 at 12:13 PM, Adam Jacob adam@opscode.com wrote:

Why don’t we make this a resource?

package “totally-sucks-devel”

on_failure “run apt-get update” do
match /^package/
notifies :run, "execute[apt-get-update]"
end

Where match is a regular expression that looks for names of resources.

Totally, this feels really useful, even if it could be a terrible idea
still.

On Mon, Mar 4, 2013 at 2:33 PM, Kevin Keane Subscription
subscription@kkeane.com wrote:

Isn’t the philosophy of Chef that resources should describe the final,
desired, state of the system? This resource just feels “procedural” rather
than “declarative”.

Or am I way off on this?

I agree that the “correct” fix for apt-get issues is tracking down
what’s wrong and fixing your run. For instance, if your instances are
from EC2 and your apt-cache is always stale, you should ensure that
apt-get update happens at the beginning of your run, perhaps being
very early in a base role. If you are having issues after installing a
repository, you should be using the functionality in the apt cookbook.
If you always need the latest update before a package install you have
to bite the bullet and always run apt-get update. However, in this
example it is clear that we’d solve a lot of frustration for a bunch
of people if we didn’t force them to do it the ideal way, and instead
gave them options to get it fixed and get back to work.

On the other hand, it’s hard to think up other situations like this
that justify a feature that someone can get themselves so far down the
wrong path with. I like the idea of handling network or mirror issues
that the tools might handle poorly.

on_failure “rotate mirrors for dumb_tool” do
match /^execute[dumb_tool]$/
notifies :run, "execute[swap_dumb_tool_mirrors]"
retries 3
end

aws_timeout = 30
aws_retries = 2

on_failure “AWS API timeout” do
match /-aws]$/
block do
Chef::Log.warn(“Failure in #{failed_resource}, maybe an AWS issue,
pausing #{aws_timeout} seconds and retrying #{aws_retries} times”)
sleep aws_timeout
end
retries aws_retries
end

on_failure “github is down” do
match /^git[our-code-/
block do
Chef::Log.warn(“Failed to act on repository
#{failed_resource.repository} in resource #{failed_resource}, trying
internal mirror”)
failed_resource.repository.gsub(/^http://github.com/opscode/,
“git://backup_git_mirror”)
end
end

Bryan


#19

What about extending the current handler approach? Add the ability to catch
a failed resource (by name) and retry it, just like you can with a
begin/rescue.

On Fri, Mar 1, 2013 at 1:58 PM, Dylan Northrup chef@doc-x.net wrote:

It seems like that would still require updating every resource with that
bit of code.

I prefer to have a central location for me to explicitly define this
behavior and allow it to “do what I mean”. Having the default be to have
Chef refuse to cross the line and implicitly do what I expect it to do is
fine, but I do believe there should be an option I can turn on that will
tell Chef “Yeah, I know you’re doing more than I’m explicitly telling you
to do each and every time I configure individual resources, but I’m okay
with that because this is the behavior I expect and I’d prefer to to the
equivalent of going to my coffee vendor and ordering ‘The usual’ instead of
having to say every time ‘Large coffee, 3 creams, 1 sugar’ because that’s
always what I’m going to want unless I explicitly tell you differently.”

On Thu, Feb 28, 2013 at 1:50 PM, Jeffrey Hulten jeffh@automatedlabs.comwrote:

Perhaps being able to specify, for any given resource

on_failure_notifies :action, “resource_type[name]”),
:immediately_and_retry

would allow both problems to be solved?

Jeffrey Hulten
Principal Consultant at Automated Labs
jeffh@automatedlabs.com 206-853-5216
Skype: jeffhulten

On Feb 28, 2013, at 7:10 AM, Andrew Gross andrew@yipit.com wrote:

The issue here seems to be a conflict between the general Chef
philosophy and what a generally sane default. I noticed similar behavior in

http://tickets.opscode.com/browse/CHEF-3734

Where the git resource was behaving in a similar way to the deploy
resource, even though that was not what was explicitly asked for. In this
case it looks like the change is leaning towards making git do explicitly
what was asked and no more.

It feels similar to the conflict when writing an API, you want to have
a layer where you have definitions that provide explicit behavior so you
can manipulate the system. At the same time though you want to provide a
good interface to the end user and give them a smooth interface that avoids
many of the pitfalls associated with inexperience. Normally this is solved
by just writing two libraries and having the user facing library wrap the
explicit one, however Chef doesn’t seem to quite fit in as either choice
(You could also argue that Chef is the wrapper library and the OS is the
lower level API).

On a side note:

I like the idea of having a generic system for handling recovery from
simple issues. I can see the logic about not having Chef do implicit
changes that may be unexpected. A downside to using an explicit retry hook
for something like packages is that you end up having to write it in to
every block, making your recipes seem very cluttered. It would be nice if
along with a retry system there was a simpler way to modify defaults for a
provider without needing to monkey patch the resource definition or
wrapping the call with your own LWRP.

Andrew

On Thu, Feb 28, 2013 at 12:43 AM, Adam Jacob adam@opscode.com wrote:
Yeah, I’m not a huge fan, for the same reason – we move from you being
explicit, to our making decisions about behavior.

Adam

From: Jesse Nelson spheromak@gmail.com
Reply-To: "chef@lists.opscode.com" chef@lists.opscode.com
Date: Wednesday, February 27, 2013 8:07 PM
To: chef chef@lists.opscode.com
Subject: [chef] Re: Re: Re: Re: Re: Re: CHEF-3930: Run apt-get update
automagically if apt-get install fails

I also feel like this is crossing the line. Where now we are doing
something that we weren’t asked to do simply because we think we know what
you want. What happens if that is exactly what a user doesn’t want. Tho I
can’t see why an apt-get update would be a bad thing or break anything on
the surface. It just smells bad. If it is to be implemented it has to be an
optional disable.

On the Side note I like the idea of error callbacks for providers or
retry hooks. Something we could code at a high level for some providers to
do when failure happens to try to recover. This was brought up and then
dismissed, but I could see there being some interesting uses. This being
one of them.

On Wed, Feb 27, 2013 at 4:24 PM, Pete Cheslock petecheslock@gmail.com
wrote:
I would be +1 for this - in a previous life we had to often run apt-get
update two or three times in a run just to cover edge cases when a solution
like this could have helped us or made things as least less repetitive
during a chef run.

-Pete

On Wed, Feb 27, 2013 at 7:20 PM, John E. Vincent (lusis) <lusis.org+
chef-list@gmail.com> wrote:
On Wed, Feb 27, 2013 at 6:44 PM, Andrea Campi
andrea.campi@zephirworks.com wrote:

On Thu, Feb 28, 2013 at 12:19 AM, Bryan McLellan btm@loftninjas.org
wrote:

Is anyone opposed to the idea of it simply being a flag, that in the
case of a missing package, apt-get update or yum update is called and
a single retry is performed? That would likely solve 95% of the
problems I have when this happens.

On Wed, Feb 27, 2013 at 6:05 PM, Andrea Campi
andrea.campi@zephirworks.com wrote:

What about making it opt-in / opt-out with an attribute?
I don’t much care which way the default is, but it would sure be
nice to

have the option.

It would have to be a global attribute and the only established
pattern we have for that is Chef::Config options. These can kind of
be

managed by templatizing the client.rb (e.g. chef-client cookbook) but
it’s less of a “flag” than would be convenient for this sort of
thing.

The reason it can’t be a resource attribute is if you want it the
other way around, you’re populating “auto_update true/false” in every
package resource in every cookbook if you want it the other way
around.

D’oh, for some reason I was thinking “cookbook”.

But yeah, it feels awesome. Of course the apt package provider should
help me with my apt-cache! We also felt it violates a Chef Law, but
none of us could name it.

Yeah, it feels a bit “do what I mean”, but in this case I think
that’s quite

reasonable.

Playing devil’s advocate, how will that work with cookbooks that use
apt_repository, which runs apt-get update ?
Worst case there will be more than one apt-get update in a single run
(and

that probably only if the package is not there for real).
Can you think of other edge cases?