How to control the order that Chef (solo) executes resources?

Hi all,
I'm having this problem:

Quick summary: my main recipe's default.rb looks like this:


include_recipe "build-essential"
include_recipe "mytardis::deps"
include_recipe "mytardis::nginx"
include_recipe "mytardis::postgresql"

Those recipes get "loaded" in that order, but the postgresql is the first
one to be "processed":
[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Loading Recipe mytardis via
include_recipe
[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Found recipe default in cookbook
mytardis
[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Loading Recipe build-essential via
include_recipe
[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Found recipe default in cookbook
build-essential
...

[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Loading Recipe postgresql::client
via include_recipe
[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Found recipe client in cookbook
postgresql
[Sun, 08 Jul 2012 05:14:33 +0200] INFO: Processing
package[postgresql-client] action install (postgresql::client line 37)
[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: package[postgresql-client]
checking package status for postgresql-client

Why is this? How can I get build-essential to run first?

(I'm doing all this through Vagrant. The target is a local Ubuntu VM. I'm
new to Chef, Vagrant, and Ruby - which may explain a lot.)

Steve

This is your culprit:

[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Loading Recipe
postgresql::client via include_recipe <-

Something has postgres as an include, probably at the top.

Sascha Bates
| sascha.bates@gmail.com | 612 850 0444 | sascha.bates@skype | sascha_bates@yahoo |

On 7/8/12 7:24 AM, Steve Bennett wrote:

[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Loading Recipe
postgresql::client via include_recipe

Also, consider defining a role to contain a run list.
http://wiki.opscode.com/display/chef/Roles
http://vagrantup.com/v1/docs/provisioners/chef_solo.html#Roles

On Sun, Jul 8, 2012 at 9:24 AM, Sascha Bates sascha.bates@gmail.com wrote:

This is your culprit:

[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Loading Recipe postgresql::client
via include_recipe <-

Something has postgres as an include, probably at the top.

Sascha Bates
| sascha.bates@gmail.com | 612 850 0444 | sascha.bates@skype |
sascha_bates@yahoo |

On 7/8/12 7:24 AM, Steve Bennett wrote:

[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Loading Recipe postgresql::client
via include_recipe

I actually would not define a role to build up the state of what your application server looks like.

Not only will this not solve your problem, it also introduces coupling issues between your cookbooks and your chef-repo.

Because roles are stored in your chef-repo and not distributed with your cookbooks, you should keep the logic of what all makes up your application server in a recipe in your applications cookbook. Later, you should include this recipe in a role or directly on the run_list.

A common reason that resources are seemingly run out of order are when you explicitly request they are run at compile time. You should check to see if any of your recipes are attempting this.

Jamie Winsor
@resetexistence

On Jul 8, 2012, at 6:28 AM, Mike miketheman@gmail.com wrote:

Also, consider defining a role to contain a run list.
http://wiki.opscode.com/display/chef/Roles
http://vagrantup.com/v1/docs/provisioners/chef_solo.html#Roles

On Sun, Jul 8, 2012 at 9:24 AM, Sascha Bates sascha.bates@gmail.com wrote:

This is your culprit:

[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Loading Recipe postgresql::client
via include_recipe <-

Something has postgres as an include, probably at the top.

Sascha Bates
| sascha.bates@gmail.com | 612 850 0444 | sascha.bates@skype |
sascha_bates@yahoo |

On 7/8/12 7:24 AM, Steve Bennett wrote:

[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Loading Recipe postgresql::client
via include_recipe

Hi Sascha,
Thanks for the suggestion. I tried commenting out the include_recipe that
I quoted before, so:

include_recipe "build-essential"
include_recipe "mytardis::deps"
include_recipe "mytardis::nginx"
#include_recipe "mytardis::postgresql"

In this form, the build no longer does any postgresql step, and proceeds
much further. So, I don't think there are any other "include_recipe
postgresql" lines. Any other ideas?

Are there any workarounds that would force build-essential to run first?

Steve

On 8 July 2012 23:24, Sascha Bates sascha.bates@gmail.com wrote:

This is your culprit:

[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Loading Recipe postgresql::client
via include_recipe <-

Something has postgres as an include, probably at the top.

Sascha Bates
| sascha.bates@gmail.com | 612 850 0444 | sascha.bates@skype | sascha_bates@yahoo |

On 7/8/12 7:24 AM, Steve Bennett wrote:

[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Loading Recipe postgresql::client
via include_recipe

Is it possible that your recipe mytardis::postgresql forces
installation of the client at compile time? This is a trick that's
often used to make sure that some resources are available early in the
run, and which can make things appear to happen in an order that is
different from what's described in the cookbook, but in fact it is
what the cookbook author intended.

-Peter

On Sun, Jul 8, 2012 at 6:49 PM, Steve Bennett stevage@gmail.com wrote:

Hi Sascha,
Thanks for the suggestion. I tried commenting out the include_recipe that
I quoted before, so:

include_recipe "build-essential"
include_recipe "mytardis::deps"
include_recipe "mytardis::nginx"
#include_recipe "mytardis::postgresql"

In this form, the build no longer does any postgresql step, and proceeds
much further. So, I don't think there are any other "include_recipe
postgresql" lines. Any other ideas?

Are there any workarounds that would force build-essential to run first?

Steve

On 8 July 2012 23:24, Sascha Bates sascha.bates@gmail.com wrote:

This is your culprit:

[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Loading Recipe postgresql::client
via include_recipe <-

Something has postgres as an include, probably at the top.

Sascha Bates
| sascha.bates@gmail.com | 612 850 0444 | sascha.bates@skype |
sascha_bates@yahoo |

On 7/8/12 7:24 AM, Steve Bennett wrote:

[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Loading Recipe postgresql::client
via include_recipe

Yes, Peter is correct. I'm looking at the postgres-client recipe and
it's forcing install at compile time:

http://community.opscode.com/cookbooks/postgresql/source

|pg_packages.||each| |do| ||pg_pack||
|||package pg_pack ||do|
|||action ||:nothing|
|||end||.run_action(||:install||)|
|end|
|gem_package ||"pg"| |do|
|||action ||:nothing|
|end||.run_action(||:install||)|

There's probably a good reason for this. You could change it so it's
not running at compile time but that will probably break something else
that assumes it's already available. Is this breaking your
build-essentials install or are you just trying to figure out why it's
running out of the order you assumed?

Sascha Bates
| sascha.bates@gmail.com | 612 850 0444 | sascha.bates@skype | sascha_bates@yahoo |

On 7/8/12 6:10 PM, Peter Norton wrote:

Is it possible that your recipe mytardis::postgresql forces
installation of the client at compile time? This is a trick that's
often used to make sure that some resources are available early in the
run, and which can make things appear to happen in an order that is
different from what's described in the cookbook, but in fact it is
what the cookbook author intended.

-Peter

On Sun, Jul 8, 2012 at 6:49 PM, Steve Bennettstevage@gmail.com wrote:

Hi Sascha,
Thanks for the suggestion. I tried commenting out the include_recipe that
I quoted before, so:

include_recipe "build-essential"
include_recipe "mytardis::deps"
include_recipe "mytardis::nginx"
#include_recipe "mytardis::postgresql"

In this form, the build no longer does any postgresql step, and proceeds
much further. So, I don't think there are any other "include_recipe
postgresql" lines. Any other ideas?

Are there any workarounds that would force build-essential to run first?

Steve

On 8 July 2012 23:24, Sascha Batessascha.bates@gmail.com wrote:

This is your culprit:

[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Loading Recipe postgresql::client
via include_recipe<-

Something has postgres as an include, probably at the top.

Sascha Bates
| sascha.bates@gmail.com | 612 850 0444 | sascha.bates@skype |
sascha_bates@yahoo |

On 7/8/12 7:24 AM, Steve Bennett wrote:

[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Loading Recipe postgresql::client
via include_recipe

It's running at compile time to ensure the postgres gem will install. The package is a dependency of that.

Jamie Winsor
@resetexistence

On Jul 8, 2012, at 5:57 PM, Sascha Bates sascha.bates@gmail.com wrote:

Yes, Peter is correct. I'm looking at the postgres-client recipe and it's forcing install at compile time:

http://community.opscode.com/cookbooks/postgresql/source

pg_packages.each do |pg_pack|
package pg_pack do
action :nothing
end.run_action(:install)
end

gem_package "pg" do
action :nothing
end.run_action(:install)

There's probably a good reason for this. You could change it so it's not running at compile time but that will probably break something else that assumes it's already available. Is this breaking your build-essentials install or are you just trying to figure out why it's running out of the order you assumed?
Sascha Bates
| sascha.bates@gmail.com | 612 850 0444 | sascha.bates@skype | sascha_bates@yahoo |

On 7/8/12 6:10 PM, Peter Norton wrote:

Is it possible that your recipe mytardis::postgresql forces
installation of the client at compile time? This is a trick that's
often used to make sure that some resources are available early in the
run, and which can make things appear to happen in an order that is
different from what's described in the cookbook, but in fact it is
what the cookbook author intended.

-Peter

On Sun, Jul 8, 2012 at 6:49 PM, Steve Bennett stevage@gmail.com wrote:

Hi Sascha,
Thanks for the suggestion. I tried commenting out the include_recipe that
I quoted before, so:

include_recipe "build-essential"
include_recipe "mytardis::deps"
include_recipe "mytardis::nginx"
#include_recipe "mytardis::postgresql"

In this form, the build no longer does any postgresql step, and proceeds
much further. So, I don't think there are any other "include_recipe
postgresql" lines. Any other ideas?

Are there any workarounds that would force build-essential to run first?

Steve

On 8 July 2012 23:24, Sascha Bates sascha.bates@gmail.com wrote:

This is your culprit:

[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Loading Recipe postgresql::client
via include_recipe <-

Something has postgres as an include, probably at the top.

Sascha Bates
| sascha.bates@gmail.com | 612 850 0444 | sascha.bates@skype |
sascha_bates@yahoo |

On 7/8/12 7:24 AM, Steve Bennett wrote:

[Sun, 08 Jul 2012 05:14:33 +0200] DEBUG: Loading Recipe postgresql::client
via include_recipe

On 9 July 2012 09:10, Peter Norton pn+chef-list@knewton.com wrote:

Is it possible that your recipe mytardis::postgresql forces
installation of the client at compile time? This is a trick that's
often used to make sure that some resources are available early in the
run, and which can make things appear to happen in an order that is
different from what's described in the cookbook, but in fact it is
what the cookbook author intended.

Ok, thanks, that looks like the problem.


pg_packages.each do |pg_pack|
package pg_pack do
action :nothing
end.run_action(:install)
end

Seems I'm not the first one to hit this issue either:

postgresql::server (in cookbooks, rather than site-cookbooks) is straight
from Apache I think:

Doing the same thing on build-essential (that is, adding
end.run_action(:install)) works - or at least, fails later on in the build
:slight_smile:

Sascha wrote:

There's probably a good reason for this. You could change it so it's not
running at compile time but that will probably break something else that
assumes it's already available.
Is this breaking your build-essentials install or are you just trying to
figure out why it's running out of the order you assumed?

The problem is that the Postgres client seems to need to build something
using C tools, and they're not installed by default on the minimal Ubuntu
distribution I'm using (from Vagrant). So, installing the client before
anything else
doesn't work - I need to install build-essential even
earlier.

Anyway, I guess I have a functional workaround for now - but I'm sure I'll
be back soon with other issues :slight_smile:

Steve