RE: Re: Re: Re: Re: Re: CHEF-3930: Run apt-get update automagically if apt-get install fails

Chef Infra (archive)
1

I didn't follow this discussion from the start, just saw this. Forgive me if I'm way off base here, but at first glance, this seems very wrong to me, the "it just feels strange" type of wrong.

Isn't the philosophy of Chef that resources should describe the final, desired, state of the system? This resource just feels "procedural" rather than "declarative".

Or am I way off on this?

-----Original Message-----
From: Adam Jacob [mailto:adam@opscode.com]
Sent: Monday, March 4, 2013 9:13 AM
To: chef@lists.opscode.com
Subject: [chef] Re: Re: Re: Re: Re: CHEF-3930: Run apt-get update automagically if apt-get install fails

Why don't we make this a resource?

package "totally-sucks-devel"

on_failure "run apt-get update" do
match /^package/
notifies :run, "execute[apt-get-update]"
end

Where match is a regular expression that looks for names of resources.

Adam

On 3/4/13 9:06 AM, "Bryan McLellan" btm@loftninjas.org wrote:

On Fri, Mar 1, 2013 at 11:21 PM, Jesse Campbell hikeit@gmail.com wrote:

What about extending the current handler approach? Add the ability to
catch a failed resource (by name) and retry it, just like you can
with a begin/rescue.

I like the concept of having an exception handler be tied to a resource
or provider, or at least be able to determine what resource or provider
caused the exception, do some bits and retry.

Bryan

2

I agree, but this would be easily fixed by making on_failure a resource property that takes a block as an argument instead of being a resource itself.

  • cassiano

On Monday, March 4, 2013 at 16:33, Kevin Keane Subscription wrote:

I didn't follow this discussion from the start, just saw this. Forgive me if I'm way off base here, but at first glance, this seems very wrong to me, the "it just feels strange" type of wrong.

Isn't the philosophy of Chef that resources should describe the final, desired, state of the system? This resource just feels "procedural" rather than "declarative".

Or am I way off on this?

-----Original Message-----
From: Adam Jacob [mailto:adam@opscode.com]
Sent: Monday, March 4, 2013 9:13 AM
To: chef@lists.opscode.com (mailto:chef@lists.opscode.com)
Subject: [chef] Re: Re: Re: Re: Re: CHEF-3930: Run apt-get update automagically if apt-get install fails

Why don't we make this a resource?

package "totally-sucks-devel"

on_failure "run apt-get update" do
match /^package/
notifies :run, "execute[apt-get-update]"
end

Where match is a regular expression that looks for names of resources.

Adam

On 3/4/13 9:06 AM, "Bryan McLellan" <btm@loftninjas.org (mailto:btm@loftninjas.org)> wrote:

On Fri, Mar 1, 2013 at 11:21 PM, Jesse Campbell <hikeit@gmail.com (mailto:hikeit@gmail.com)> wrote:

What about extending the current handler approach? Add the ability to
catch a failed resource (by name) and retry it, just like you can
with a begin/rescue.

I like the concept of having an exception handler be tied to a resource
or provider, or at least be able to determine what resource or provider
caused the exception, do some bits and retry.

Bryan

3

cassiano, i think you nailed it.

make on_failure function just like an only_if/not_if/notify thing:

package "totally-sucks-devel" do
on_failure "apt-get update", :retry_once
end

make on_failure accept either a string or a block, and an optional set of
arguments telling whether to retry and how many times
or alternatively, make it a block like thing:
package "totally-sucks-devel" do
on_failure do
notifies :run, "execute[apt-get-update]"
retry true
end
end

On Mon, Mar 4, 2013 at 2:44 PM, Cassiano Leal cassianoleal@gmail.comwrote:

I agree, but this would be easily fixed by making on_failure a resource
property that takes a block as an argument instead of being a resource
itself.

  • cassiano

On Monday, March 4, 2013 at 16:33, Kevin Keane Subscription wrote:

I didn't follow this discussion from the start, just saw this. Forgive me
if I'm way off base here, but at first glance, this seems very wrong to me,
the "it just feels strange" type of wrong.

Isn't the philosophy of Chef that resources should describe the final,
desired, state of the system? This resource just feels "procedural" rather
than "declarative".

Or am I way off on this?

-----Original Message-----
From: Adam Jacob [mailto:adam@opscode.com adam@opscode.com]
Sent: Monday, March 4, 2013 9:13 AM
To: chef@lists.opscode.com
Subject: [chef] Re: Re: Re: Re: Re: CHEF-3930: Run apt-get update
automagically if apt-get install fails

Why don't we make this a resource?

package "totally-sucks-devel"

on_failure "run apt-get update" do
match /^package/
notifies :run, "execute[apt-get-update]"
end

Where match is a regular expression that looks for names of resources.

Adam

On 3/4/13 9:06 AM, "Bryan McLellan" btm@loftninjas.org wrote:

On Fri, Mar 1, 2013 at 11:21 PM, Jesse Campbell hikeit@gmail.com wrote:

What about extending the current handler approach? Add the ability to
catch a failed resource (by name) and retry it, just like you can
with a begin/rescue.

I like the concept of having an exception handler be tied to a resource
or provider, or at least be able to determine what resource or provider
caused the exception, do some bits and retry.

Bryan

4

That works pretty well, I think the previous idea sprouted from the
constraint that we didnt want to have to add this block to every single
package block. Unsure how to resolve that with this syntax unless we could
specify global defaults for some of these things.

On Mon, Mar 4, 2013 at 3:22 PM, Jesse Campbell hikeit@gmail.com wrote:

cassiano, i think you nailed it.

make on_failure function just like an only_if/not_if/notify thing:

package "totally-sucks-devel" do
on_failure "apt-get update", :retry_once
end

make on_failure accept either a string or a block, and an optional set of
arguments telling whether to retry and how many times
or alternatively, make it a block like thing:
package "totally-sucks-devel" do
on_failure do
notifies :run, "execute[apt-get-update]"
retry true
end
end

On Mon, Mar 4, 2013 at 2:44 PM, Cassiano Leal cassianoleal@gmail.comwrote:

I agree, but this would be easily fixed by making on_failure a resource
property that takes a block as an argument instead of being a resource
itself.

  • cassiano

On Monday, March 4, 2013 at 16:33, Kevin Keane Subscription wrote:

I didn't follow this discussion from the start, just saw this. Forgive me
if I'm way off base here, but at first glance, this seems very wrong to me,
the "it just feels strange" type of wrong.

Isn't the philosophy of Chef that resources should describe the final,
desired, state of the system? This resource just feels "procedural" rather
than "declarative".

Or am I way off on this?

-----Original Message-----
From: Adam Jacob [mailto:adam@opscode.com adam@opscode.com]
Sent: Monday, March 4, 2013 9:13 AM
To: chef@lists.opscode.com
Subject: [chef] Re: Re: Re: Re: Re: CHEF-3930: Run apt-get update
automagically if apt-get install fails

Why don't we make this a resource?

package "totally-sucks-devel"

on_failure "run apt-get update" do
match /^package/
notifies :run, "execute[apt-get-update]"
end

Where match is a regular expression that looks for names of resources.

Adam

On 3/4/13 9:06 AM, "Bryan McLellan" btm@loftninjas.org wrote:

On Fri, Mar 1, 2013 at 11:21 PM, Jesse Campbell hikeit@gmail.com wrote:

What about extending the current handler approach? Add the ability to
catch a failed resource (by name) and retry it, just like you can
with a begin/rescue.

I like the concept of having an exception handler be tied to a resource
or provider, or at least be able to determine what resource or provider
caused the exception, do some bits and retry.

Bryan

5

Why not add recipe[apt] to your base role? This keeps things cache up to date.

--
John Dewey
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)

On Monday, March 4, 2013 at 12:44 PM, Andrew Gross wrote:

That works pretty well, I think the previous idea sprouted from the constraint that we didnt want to have to add this block to every single package block. Unsure how to resolve that with this syntax unless we could specify global defaults for some of these things.

On Mon, Mar 4, 2013 at 3:22 PM, Jesse Campbell <hikeit@gmail.com (mailto:hikeit@gmail.com)> wrote:

cassiano, i think you nailed it.

make on_failure function just like an only_if/not_if/notify thing:

package "totally-sucks-devel" do
on_failure "apt-get update", :retry_once
end

make on_failure accept either a string or a block, and an optional set of arguments telling whether to retry and how many times
or alternatively, make it a block like thing:
package "totally-sucks-devel" do
on_failure do
notifies :run, "execute[apt-get-update]"
retry true
end
end

On Mon, Mar 4, 2013 at 2:44 PM, Cassiano Leal <cassianoleal@gmail.com (mailto:cassianoleal@gmail.com)> wrote:

I agree, but this would be easily fixed by making on_failure a resource property that takes a block as an argument instead of being a resource itself.

  • cassiano

On Monday, March 4, 2013 at 16:33, Kevin Keane Subscription wrote:

I didn't follow this discussion from the start, just saw this. Forgive me if I'm way off base here, but at first glance, this seems very wrong to me, the "it just feels strange" type of wrong.

Isn't the philosophy of Chef that resources should describe the final, desired, state of the system? This resource just feels "procedural" rather than "declarative".

Or am I way off on this?

-----Original Message-----
From: Adam Jacob [mailto:adam@opscode.com]
Sent: Monday, March 4, 2013 9:13 AM
To: chef@lists.opscode.com (mailto:chef@lists.opscode.com)
Subject: [chef] Re: Re: Re: Re: Re: CHEF-3930: Run apt-get update automagically if apt-get install fails

Why don't we make this a resource?

package "totally-sucks-devel"

on_failure "run apt-get update" do
match /^package/
notifies :run, "execute[apt-get-update]"
end

Where match is a regular expression that looks for names of resources.

Adam

On 3/4/13 9:06 AM, "Bryan McLellan" <btm@loftninjas.org (mailto:btm@loftninjas.org)> wrote:

On Fri, Mar 1, 2013 at 11:21 PM, Jesse Campbell <hikeit@gmail.com (mailto:hikeit@gmail.com)> wrote:

What about extending the current handler approach? Add the ability to
catch a failed resource (by name) and retry it, just like you can
with a begin/rescue.

I like the concept of having an exception handler be tied to a resource
or provider, or at least be able to determine what resource or provider
caused the exception, do some bits and retry.

Bryan

6

Hehe things have come full circle now. The recipe[apt] solution isn't bad,
but it doesn't cover all cases (for more info check out the JIRA ticket).
I created this pull request with a solution to try to silence the issue by
just automatically retrying, but I was not thinking about the broader
picture. All in all, I just want my Chef runs to stop failing when they
need to run apt-get update.

On Mon, Mar 4, 2013 at 3:58 PM, John Dewey john@dewey.ws wrote:

Why not add recipe[apt] to your base role? This keeps things cache up
to date.

--
John Dewey
Sent with Sparrow http://www.sparrowmailapp.com/?sig

On Monday, March 4, 2013 at 12:44 PM, Andrew Gross wrote:

That works pretty well, I think the previous idea sprouted from the
constraint that we didnt want to have to add this block to every single
package block. Unsure how to resolve that with this syntax unless we could
specify global defaults for some of these things.

On Mon, Mar 4, 2013 at 3:22 PM, Jesse Campbell hikeit@gmail.com wrote:

cassiano, i think you nailed it.

make on_failure function just like an only_if/not_if/notify thing:

package "totally-sucks-devel" do
on_failure "apt-get update", :retry_once
end

make on_failure accept either a string or a block, and an optional set of
arguments telling whether to retry and how many times
or alternatively, make it a block like thing:
package "totally-sucks-devel" do
on_failure do
notifies :run, "execute[apt-get-update]"
retry true
end
end

On Mon, Mar 4, 2013 at 2:44 PM, Cassiano Leal cassianoleal@gmail.comwrote:

I agree, but this would be easily fixed by making on_failure a resource
property that takes a block as an argument instead of being a resource
itself.

  • cassiano

On Monday, March 4, 2013 at 16:33, Kevin Keane Subscription wrote:

I didn't follow this discussion from the start, just saw this. Forgive me
if I'm way off base here, but at first glance, this seems very wrong to me,
the "it just feels strange" type of wrong.

Isn't the philosophy of Chef that resources should describe the final,
desired, state of the system? This resource just feels "procedural" rather
than "declarative".

Or am I way off on this?

-----Original Message-----
From: Adam Jacob [mailto:adam@opscode.com adam@opscode.com]
Sent: Monday, March 4, 2013 9:13 AM
To: chef@lists.opscode.com
Subject: [chef] Re: Re: Re: Re: Re: CHEF-3930: Run apt-get update
automagically if apt-get install fails

Why don't we make this a resource?

package "totally-sucks-devel"

on_failure "run apt-get update" do
match /^package/
notifies :run, "execute[apt-get-update]"
end

Where match is a regular expression that looks for names of resources.

Adam

On 3/4/13 9:06 AM, "Bryan McLellan" btm@loftninjas.org wrote:

On Fri, Mar 1, 2013 at 11:21 PM, Jesse Campbell hikeit@gmail.com wrote:

What about extending the current handler approach? Add the ability to
catch a failed resource (by name) and retry it, just like you can
with a begin/rescue.

I like the concept of having an exception handler be tied to a resource
or provider, or at least be able to determine what resource or provider
caused the exception, do some bits and retry.

Bryan

7

On Monday, March 4, 2013 at 1:02 PM, Andrew Gross wrote:

Hehe things have come full circle now. The recipe[apt] solution isn't bad, but it doesn't cover all cases (for more info check out the JIRA ticket). I created this pull request with a solution to try to silence the issue by just automatically retrying, but I was not thinking about the broader picture. All in all, I just want my Chef runs to stop failing when they need to run apt-get update.

How do you get to that situation? Are you updating cache on a fixed schedule and there's a window between packages being updated upstream and your cache update? Are you adding a new repo and not updating?

--
Daniel DeLeo

8

Issue can come up when using a PPA, a new version will be pushed but 24
hours will not have elapsed before we want to update. We have already
added the PPA in an earlier run, so it is not possible to notify an execute
block or similar workarounds.

On Mon, Mar 4, 2013 at 4:13 PM, Daniel DeLeo dan@kallistec.com wrote:

On Monday, March 4, 2013 at 1:02 PM, Andrew Gross wrote:

Hehe things have come full circle now. The recipe[apt] solution isn't
bad, but it doesn't cover all cases (for more info check out the JIRA
ticket). I created this pull request with a solution to try to silence the
issue by just automatically retrying, but I was not thinking about the
broader picture. All in all, I just want my Chef runs to stop failing when
they need to run apt-get update.

How do you get to that situation? Are you updating cache on a fixed
schedule and there's a window between packages being updated upstream and
your cache update? Are you adding a new repo and not updating?

--
Daniel DeLeo

9

Hi,

On Tue, Mar 5, 2013 at 7:44 AM, Andrew Gross andrew@yipit.com wrote:

That works pretty well, I think the previous idea sprouted from the
constraint that we didnt want to have to add this block to every single
package block. Unsure how to resolve that with this syntax unless we could
specify global defaults for some of these things.

I would not classify it as setting of defaults but would look at it
from a different angle. In some scenarios you want/need to handle
cross-cutting concerns consistently across the compiled resource base.
Typically rules may be;

  • Use a single maven repository, regardless of what the recipe says
    when downloading artifacts
  • Retry once on failure to download artifacts
  • Use a single apt repository, regardless of what the recipe says when
    downloading packages
  • Retry twice on failure to download packages

In the development world these are typically described as aspects and
many different systems have converged on using interceptor driven
approach to implement this. However, I prottyped it a hacky way after
I saw chef-rewind by adding something like the following to a recipe
at the end of a run.

@run_context.resource_collection.each do |r|
r.some_var = 'mydefault' if r.is_a?(MyResource)
end

--
Cheers,

Peter Donald

10

I would argue this allows you to better specify the final, desired state
of the system. As it stands now, if a chef-run fails, your system is
left in an undefined state... it may be working, it may be working, but
in a degraded state, or it may be completely failed. Nothing in your
run_list specifies which of those states you are in after a failure. An
on_failure resource allows you to fully specify the failure modes of
your configuration. I think this is still declarative in that you're
declaring changes to the configuration based on the status of other
declared resources.

Greg

On 03/04/2013 01:33 PM, Kevin Keane Subscription wrote:

I didn't follow this discussion from the start, just saw this. Forgive me if I'm way off base here, but at first glance, this seems very wrong to me, the "it just feels strange" type of wrong.

Isn't the philosophy of Chef that resources should describe the final, desired, state of the system? This resource just feels "procedural" rather than "declarative".

Or am I way off on this?

-----Original Message-----
From: Adam Jacob [mailto:adam@opscode.com]
Sent: Monday, March 4, 2013 9:13 AM
To: chef@lists.opscode.com
Subject: [chef] Re: Re: Re: Re: Re: CHEF-3930: Run apt-get update automagically if apt-get install fails

Why don't we make this a resource?

package "totally-sucks-devel"

on_failure "run apt-get update" do
match /^package/
notifies :run, "execute[apt-get-update]"
end

Where match is a regular expression that looks for names of resources.

Adam

On 3/4/13 9:06 AM, "Bryan McLellan" btm@loftninjas.org wrote:

On Fri, Mar 1, 2013 at 11:21 PM, Jesse Campbell hikeit@gmail.com wrote:

What about extending the current handler approach? Add the ability to
catch a failed resource (by name) and retry it, just like you can
with a begin/rescue.
I like the concept of having an exception handler be tied to a resource
or provider, or at least be able to determine what resource or provider
caused the exception, do some bits and retry.

Bryan