run_context.include_recipe in a recipe within a role

Hi everyone

I am having trouble with LWRP, include_recipe and roles.

I have a LWRP that includes some cookbooks :
run_context.include_recipe "git"
run_context.include_recipe "internal-cookbook"
run_context.include_recipe “build-essential”

The git cookbook is the one available at opscode.
I use this recipe in a variety of other internal cookbooks.

Here is my problem : when I add the cookbooks that use the LWRP directly to
a node’s run_list, it works perfectly but when I add them via a role, git
is not installed. It works for the two other recipes included in the LWRP
but not for git.

In the logs I have :
DEBUG: Loading Recipe git via include_recipe
DEBUG: Found recipe default in cookbook git
and then it goes to the next cookbook without the expected
INFO: Processing package[git] action install (base::default line 9)

I am running chef 11.4.4 on ubuntu 12.04 LTS

Do you guys have any ideas what is happening and how to solve this ?


Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37
http://perfect-memory.com

https://twitter.com/semanticbushttp://www.facebook.com/pages/Perfect-Memory/155555567821817

I am still tracking down my problem and I have some news.
It appears that he problem is not specific to the git cookbook.
Indeed it appears that all recipes included in a LWRP
'run_context.include_recipe' is only really added to the run_list if the
LWRP is called once during the run_list.

If the LWRP is called more than once, it tells me that the resource has
already been added to the run_list even if it has not.

Is this a known bug ? I am doing something wrong ?
I am failing to find any clues using my google-fu so I am turning to your
expertise.

2013/5/29 Dorian Jaminais dorian.jaminais@perfect-memory.com

Hi everyone

I am having trouble with LWRP, include_recipe and roles.

I have a LWRP that includes some cookbooks :
run_context.include_recipe "git"
run_context.include_recipe "internal-cookbook"
run_context.include_recipe "build-essential"

The git cookbook is the one available at opscode.
I use this recipe in a variety of other internal cookbooks.

Here is my problem : when I add the cookbooks that use the LWRP directly
to a node's run_list, it works perfectly but when I add them via a role,
git is not installed. It works for the two other recipes included in the
LWRP but not for git.

In the logs I have :
DEBUG: Loading Recipe git via include_recipe
DEBUG: Found recipe default in cookbook git
and then it goes to the next cookbook without the expected
INFO: Processing package[git] action install (base::default line 9)

I am running chef 11.4.4 on ubuntu 12.04 LTS

Do you guys have any ideas what is happening and how to solve this ?

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37
http://perfect-memory.com

https://twitter.com/semanticbushttp://www.facebook.com/pages/Perfect-Memory/155555567821817

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37
http://perfect-memory.com

https://twitter.com/semanticbushttp://www.facebook.com/pages/Perfect-Memory/155555567821817

On Wednesday, May 29, 2013 at 9:06 AM, Dorian Jaminais wrote:

I am still tracking down my problem and I have some news.
It appears that he problem is not specific to the git cookbook.
Indeed it appears that all recipes included in a LWRP 'run_context.include_recipe' is only really added to the run_list if the LWRP is called once during the run_list.

If the LWRP is called more than once, it tells me that the resource has already been added to the run_list even if it has not.

Is this a known bug ? I am doing something wrong ?
I am failing to find any clues using my google-fu so I am turning to your expertise.

2013/5/29 Dorian Jaminais <dorian.jaminais@perfect-memory.com (mailto:dorian.jaminais@perfect-memory.com)>

Hi everyone

I am having trouble with LWRP, include_recipe and roles.

I have a LWRP that includes some cookbooks :
run_context.include_recipe "git"
run_context.include_recipe "internal-cookbook"
run_context.include_recipe "build-essential"

This is a bit odd. Why does the LWRP do this instead of a recipe? Showing more code, log output, and explaining why you're doing things this way would help others to understand your issue better.

--
Daniel DeLeo

Okay

I am using a LWRP instead of a recipe because I want to deploy about 15
applications that are very similar.
All those applications have a dedicated cookbook but as the installation
process is always the same work flow, I have factorized it into a LWRP.

The work flow is pretty simple : assert that installation dependencies are
met, clone the repo, bundle install the needed gems.

I have attached my provider to this mail (which by the way is included in a
cookbook named gios).

In each application's cookbook I use :
gios_installer NAME do
name NAME
repository GIT_URI
command "./bin/truc"
version "master"
action :install
end

As stated in my previous emails, using the LWRP that way will include the
recipes ruby, git, base::certificate, pm-openssh::rsa and
pm-openssh::known_hosts only if the LWRP is called exactly once in the
run_list.

Thanks for your help

2013/5/29 Daniel DeLeo dan@kallistec.com

On Wednesday, May 29, 2013 at 9:06 AM, Dorian Jaminais wrote:

I am still tracking down my problem and I have some news.
It appears that he problem is not specific to the git cookbook.
Indeed it appears that all recipes included in a LWRP
'run_context.include_recipe' is only really added to the run_list if the
LWRP is called once during the run_list.

If the LWRP is called more than once, it tells me that the resource has
already been added to the run_list even if it has not.

Is this a known bug ? I am doing something wrong ?
I am failing to find any clues using my google-fu so I am turning to your
expertise.

2013/5/29 Dorian Jaminais dorian.jaminais@perfect-memory.com

Hi everyone

I am having trouble with LWRP, include_recipe and roles.

I have a LWRP that includes some cookbooks :
run_context.include_recipe "git"
run_context.include_recipe "internal-cookbook"
run_context.include_recipe "build-essential"

This is a bit odd. Why does the LWRP do this instead of a recipe? Showing
more code, log output, and explaining why you're doing things this way
would help others to understand your issue better.

--
Daniel DeLeo

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37
http://perfect-memory.com

https://twitter.com/semanticbushttp://www.facebook.com/pages/Perfect-Memory/155555567821817

Is there a reason you wouldn't, however, just add the "include_recipe"
statements to the default recipe for the LWRP cookbook then? It's not
uncommon to do that even for library cookbooks iirc and it gets around (I
think) the issue you're describing.

On Wed, May 29, 2013 at 12:39 PM, Dorian Jaminais <
dorian.jaminais@perfect-memory.com> wrote:

Okay

I am using a LWRP instead of a recipe because I want to deploy about 15
applications that are very similar.
All those applications have a dedicated cookbook but as the installation
process is always the same work flow, I have factorized it into a LWRP.

The work flow is pretty simple : assert that installation dependencies are
met, clone the repo, bundle install the needed gems.

I have attached my provider to this mail (which by the way is included in
a cookbook named gios).

In each application's cookbook I use :
gios_installer NAME do
name NAME
repository GIT_URI
command "./bin/truc"
version "master"
action :install
end

As stated in my previous emails, using the LWRP that way will include the
recipes ruby, git, base::certificate, pm-openssh::rsa and
pm-openssh::known_hosts only if the LWRP is called exactly once in the
run_list.

Thanks for your help

2013/5/29 Daniel DeLeo dan@kallistec.com

On Wednesday, May 29, 2013 at 9:06 AM, Dorian Jaminais wrote:

I am still tracking down my problem and I have some news.
It appears that he problem is not specific to the git cookbook.
Indeed it appears that all recipes included in a LWRP
'run_context.include_recipe' is only really added to the run_list if the
LWRP is called once during the run_list.

If the LWRP is called more than once, it tells me that the resource has
already been added to the run_list even if it has not.

Is this a known bug ? I am doing something wrong ?
I am failing to find any clues using my google-fu so I am turning to your
expertise.

2013/5/29 Dorian Jaminais dorian.jaminais@perfect-memory.com

Hi everyone

I am having trouble with LWRP, include_recipe and roles.

I have a LWRP that includes some cookbooks :
run_context.include_recipe "git"
run_context.include_recipe "internal-cookbook"
run_context.include_recipe "build-essential"

This is a bit odd. Why does the LWRP do this instead of a recipe? Showing
more code, log output, and explaining why you're doing things this way
would help others to understand your issue better.

--
Daniel DeLeo

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37
http://perfect-memory.com

https://twitter.com/semanticbushttp://www.facebook.com/pages/Perfect-Memory/155555567821817

Yes this can be a workaround, still I don't think the behavior I am getting
is normal (and don't understand why) and I'd prefer to find a cleaner way
than having to add a recipe just to include other recipes whereas it should
be the task of my LWRP.

2013/5/29 John E. Vincent (lusis) lusis.org+chef-list@gmail.com

Is there a reason you wouldn't, however, just add the "include_recipe"
statements to the default recipe for the LWRP cookbook then? It's not
uncommon to do that even for library cookbooks iirc and it gets around (I
think) the issue you're describing.

On Wed, May 29, 2013 at 12:39 PM, Dorian Jaminais <
dorian.jaminais@perfect-memory.com> wrote:

Okay

I am using a LWRP instead of a recipe because I want to deploy about 15
applications that are very similar.
All those applications have a dedicated cookbook but as the installation
process is always the same work flow, I have factorized it into a LWRP.

The work flow is pretty simple : assert that installation dependencies
are met, clone the repo, bundle install the needed gems.

I have attached my provider to this mail (which by the way is included in
a cookbook named gios).

In each application's cookbook I use :
gios_installer NAME do
name NAME
repository GIT_URI
command "./bin/truc"
version "master"
action :install
end

As stated in my previous emails, using the LWRP that way will include the
recipes ruby, git, base::certificate, pm-openssh::rsa and
pm-openssh::known_hosts only if the LWRP is called exactly once in the
run_list.

Thanks for your help

2013/5/29 Daniel DeLeo dan@kallistec.com

On Wednesday, May 29, 2013 at 9:06 AM, Dorian Jaminais wrote:

I am still tracking down my problem and I have some news.
It appears that he problem is not specific to the git cookbook.
Indeed it appears that all recipes included in a LWRP
'run_context.include_recipe' is only really added to the run_list if the
LWRP is called once during the run_list.

If the LWRP is called more than once, it tells me that the resource has
already been added to the run_list even if it has not.

Is this a known bug ? I am doing something wrong ?
I am failing to find any clues using my google-fu so I am turning to
your expertise.

2013/5/29 Dorian Jaminais dorian.jaminais@perfect-memory.com

Hi everyone

I am having trouble with LWRP, include_recipe and roles.

I have a LWRP that includes some cookbooks :
run_context.include_recipe "git"
run_context.include_recipe "internal-cookbook"
run_context.include_recipe "build-essential"

This is a bit odd. Why does the LWRP do this instead of a recipe?
Showing more code, log output, and explaining why you're doing things this
way would help others to understand your issue better.

--
Daniel DeLeo

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37
http://perfect-memory.com

https://twitter.com/semanticbushttp://www.facebook.com/pages/Perfect-Memory/155555567821817

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37
http://perfect-memory.com

https://twitter.com/semanticbushttp://www.facebook.com/pages/Perfect-Memory/155555567821817

I think you misunderstand. You'll have to call include_recipe to get access
to the lwrp anyway. So adding the include_recipe statements to "default.rb"
for that lwrp cookbook doesn't require any additional work/change. They
just get pulled in at the time the LWRP is included.

Unless I'm missing something obvious here.

On Wed, May 29, 2013 at 12:46 PM, Dorian Jaminais <
dorian.jaminais@perfect-memory.com> wrote:

Yes this can be a workaround, still I don't think the behavior I am
getting is normal (and don't understand why) and I'd prefer to find a
cleaner way than having to add a recipe just to include other recipes
whereas it should be the task of my LWRP.

2013/5/29 John E. Vincent (lusis) lusis.org+chef-list@gmail.com

Is there a reason you wouldn't, however, just add the "include_recipe"

statements to the default recipe for the LWRP cookbook then? It's not
uncommon to do that even for library cookbooks iirc and it gets around (I
think) the issue you're describing.

On Wed, May 29, 2013 at 12:39 PM, Dorian Jaminais <
dorian.jaminais@perfect-memory.com> wrote:

Okay

I am using a LWRP instead of a recipe because I want to deploy about 15
applications that are very similar.
All those applications have a dedicated cookbook but as the installation
process is always the same work flow, I have factorized it into a LWRP.

The work flow is pretty simple : assert that installation dependencies
are met, clone the repo, bundle install the needed gems.

I have attached my provider to this mail (which by the way is included
in a cookbook named gios).

In each application's cookbook I use :
gios_installer NAME do
name NAME
repository GIT_URI
command "./bin/truc"
version "master"
action :install
end

As stated in my previous emails, using the LWRP that way will include
the recipes ruby, git, base::certificate, pm-openssh::rsa and
pm-openssh::known_hosts only if the LWRP is called exactly once in the
run_list.

Thanks for your help

2013/5/29 Daniel DeLeo dan@kallistec.com

On Wednesday, May 29, 2013 at 9:06 AM, Dorian Jaminais wrote:

I am still tracking down my problem and I have some news.
It appears that he problem is not specific to the git cookbook.
Indeed it appears that all recipes included in a LWRP
'run_context.include_recipe' is only really added to the run_list if the
LWRP is called once during the run_list.

If the LWRP is called more than once, it tells me that the resource has
already been added to the run_list even if it has not.

Is this a known bug ? I am doing something wrong ?
I am failing to find any clues using my google-fu so I am turning to
your expertise.

2013/5/29 Dorian Jaminais dorian.jaminais@perfect-memory.com

Hi everyone

I am having trouble with LWRP, include_recipe and roles.

I have a LWRP that includes some cookbooks :
run_context.include_recipe "git"
run_context.include_recipe "internal-cookbook"
run_context.include_recipe "build-essential"

This is a bit odd. Why does the LWRP do this instead of a recipe?
Showing more code, log output, and explaining why you're doing things this
way would help others to understand your issue better.

--
Daniel DeLeo

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37
http://perfect-memory.com

https://twitter.com/semanticbushttp://www.facebook.com/pages/Perfect-Memory/155555567821817

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37
http://perfect-memory.com

https://twitter.com/semanticbushttp://www.facebook.com/pages/Perfect-Memory/155555567821817

i think its fine to include recipes via provider (or rather execute a whole
recipe)

create a subcontext , and a mini chef run inside it, and converge the node.
this is how we used to reuse existing chef resource inside an lwrp earlier..
yfeldblum had a gist on it..

http://realityforge.org/code/2012/07/17/lwrp-notify-on-changed-resources.html

On Wed, May 29, 2013 at 9:46 AM, Dorian Jaminais <
dorian.jaminais@perfect-memory.com> wrote:

Yes this can be a workaround, still I don't think the behavior I am
getting is normal (and don't understand why) and I'd prefer to find a
cleaner way than having to add a recipe just to include other recipes
whereas it should be the task of my LWRP.

2013/5/29 John E. Vincent (lusis) lusis.org+chef-list@gmail.com

Is there a reason you wouldn't, however, just add the "include_recipe"

statements to the default recipe for the LWRP cookbook then? It's not
uncommon to do that even for library cookbooks iirc and it gets around (I
think) the issue you're describing.

On Wed, May 29, 2013 at 12:39 PM, Dorian Jaminais <
dorian.jaminais@perfect-memory.com> wrote:

Okay

I am using a LWRP instead of a recipe because I want to deploy about 15
applications that are very similar.
All those applications have a dedicated cookbook but as the installation
process is always the same work flow, I have factorized it into a LWRP.

The work flow is pretty simple : assert that installation dependencies
are met, clone the repo, bundle install the needed gems.

I have attached my provider to this mail (which by the way is included
in a cookbook named gios).

In each application's cookbook I use :
gios_installer NAME do
name NAME
repository GIT_URI
command "./bin/truc"
version "master"
action :install
end

As stated in my previous emails, using the LWRP that way will include
the recipes ruby, git, base::certificate, pm-openssh::rsa and
pm-openssh::known_hosts only if the LWRP is called exactly once in the
run_list.

Thanks for your help

2013/5/29 Daniel DeLeo dan@kallistec.com

On Wednesday, May 29, 2013 at 9:06 AM, Dorian Jaminais wrote:

I am still tracking down my problem and I have some news.
It appears that he problem is not specific to the git cookbook.
Indeed it appears that all recipes included in a LWRP
'run_context.include_recipe' is only really added to the run_list if the
LWRP is called once during the run_list.

If the LWRP is called more than once, it tells me that the resource has
already been added to the run_list even if it has not.

Is this a known bug ? I am doing something wrong ?
I am failing to find any clues using my google-fu so I am turning to
your expertise.

2013/5/29 Dorian Jaminais dorian.jaminais@perfect-memory.com

Hi everyone

I am having trouble with LWRP, include_recipe and roles.

I have a LWRP that includes some cookbooks :
run_context.include_recipe "git"
run_context.include_recipe "internal-cookbook"
run_context.include_recipe "build-essential"

This is a bit odd. Why does the LWRP do this instead of a recipe?
Showing more code, log output, and explaining why you're doing things this
way would help others to understand your issue better.

--
Daniel DeLeo

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37
http://perfect-memory.com

https://twitter.com/semanticbushttp://www.facebook.com/pages/Perfect-Memory/155555567821817

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37
http://perfect-memory.com

https://twitter.com/semanticbushttp://www.facebook.com/pages/Perfect-Memory/155555567821817

very neat.

Notifications from the lwrp are one thing, but IMO include_recipe shouldn't
need to be in the lwrp because as John mentioned the cookbook that provides
the lwrp still needs to be included to make use of the specific lwrp and as
such will run the default recipe. It should also result in a simpler lwrp.
This seems easier to reason about as well since the lwrp is not the
distribution mechanism for functionality, the cookbook is.

As Dan said though, we probably need to see more code to understand the
full issue

On Wed, May 29, 2013 at 10:07 AM, Ranjib Dey dey.ranjib@gmail.com wrote:

i think its fine to include recipes via provider (or rather execute a
whole recipe)

create a subcontext , and a mini chef run inside it, and converge the
node. this is how we used to reuse existing chef resource inside an lwrp
earlier..
yfeldblum had a gist on it..

LWRP notifying on changed resources

On Wed, May 29, 2013 at 9:46 AM, Dorian Jaminais <
dorian.jaminais@perfect-memory.com> wrote:

Yes this can be a workaround, still I don't think the behavior I am
getting is normal (and don't understand why) and I'd prefer to find a
cleaner way than having to add a recipe just to include other recipes
whereas it should be the task of my LWRP.

2013/5/29 John E. Vincent (lusis) lusis.org+chef-list@gmail.com

Is there a reason you wouldn't, however, just add the "include_recipe"

statements to the default recipe for the LWRP cookbook then? It's not
uncommon to do that even for library cookbooks iirc and it gets around (I
think) the issue you're describing.

On Wed, May 29, 2013 at 12:39 PM, Dorian Jaminais <
dorian.jaminais@perfect-memory.com> wrote:

Okay

I am using a LWRP instead of a recipe because I want to deploy about 15
applications that are very similar.
All those applications have a dedicated cookbook but as the
installation process is always the same work flow, I have factorized it
into a LWRP.

The work flow is pretty simple : assert that installation dependencies
are met, clone the repo, bundle install the needed gems.

I have attached my provider to this mail (which by the way is included
in a cookbook named gios).

In each application's cookbook I use :
gios_installer NAME do
name NAME
repository GIT_URI
command "./bin/truc"
version "master"
action :install
end

As stated in my previous emails, using the LWRP that way will include
the recipes ruby, git, base::certificate, pm-openssh::rsa and
pm-openssh::known_hosts only if the LWRP is called exactly once in the
run_list.

Thanks for your help

2013/5/29 Daniel DeLeo dan@kallistec.com

On Wednesday, May 29, 2013 at 9:06 AM, Dorian Jaminais wrote:

I am still tracking down my problem and I have some news.
It appears that he problem is not specific to the git cookbook.
Indeed it appears that all recipes included in a LWRP
'run_context.include_recipe' is only really added to the run_list if the
LWRP is called once during the run_list.

If the LWRP is called more than once, it tells me that the resource
has already been added to the run_list even if it has not.

Is this a known bug ? I am doing something wrong ?
I am failing to find any clues using my google-fu so I am turning to
your expertise.

2013/5/29 Dorian Jaminais dorian.jaminais@perfect-memory.com

Hi everyone

I am having trouble with LWRP, include_recipe and roles.

I have a LWRP that includes some cookbooks :
run_context.include_recipe "git"
run_context.include_recipe "internal-cookbook"
run_context.include_recipe "build-essential"

This is a bit odd. Why does the LWRP do this instead of a recipe?
Showing more code, log output, and explaining why you're doing things this
way would help others to understand your issue better.

--
Daniel DeLeo

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37
http://perfect-memory.com

https://twitter.com/semanticbushttp://www.facebook.com/pages/Perfect-Memory/155555567821817

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37
http://perfect-memory.com

https://twitter.com/semanticbushttp://www.facebook.com/pages/Perfect-Memory/155555567821817

try this

rc =run_context.dup
rc.resource_collection = Chef::ResourceCollection.new
rc.load_recipe("foo::bar")
Chef::Runner.new(rc).converge

.. for your case u'll be adding all the recipes,
make sure u capture the updated resources. and set the
new_resource.updated_by_last_action accordingly

On Wed, May 29, 2013 at 9:06 AM, Dorian Jaminais <
dorian.jaminais@perfect-memory.com> wrote:

I am still tracking down my problem and I have some news.
It appears that he problem is not specific to the git cookbook.
Indeed it appears that all recipes included in a LWRP
'run_context.include_recipe' is only really added to the run_list if the
LWRP is called once during the run_list.

If the LWRP is called more than once, it tells me that the resource has
already been added to the run_list even if it has not.

Is this a known bug ? I am doing something wrong ?
I am failing to find any clues using my google-fu so I am turning to your
expertise.

2013/5/29 Dorian Jaminais dorian.jaminais@perfect-memory.com

Hi everyone

I am having trouble with LWRP, include_recipe and roles.

I have a LWRP that includes some cookbooks :
run_context.include_recipe "git"
run_context.include_recipe "internal-cookbook"
run_context.include_recipe "build-essential"

The git cookbook is the one available at opscode.
I use this recipe in a variety of other internal cookbooks.

Here is my problem : when I add the cookbooks that use the LWRP directly
to a node's run_list, it works perfectly but when I add them via a role,
git is not installed. It works for the two other recipes included in the
LWRP but not for git.

In the logs I have :
DEBUG: Loading Recipe git via include_recipe
DEBUG: Found recipe default in cookbook git
and then it goes to the next cookbook without the expected
INFO: Processing package[git] action install (base::default line 9)

I am running chef 11.4.4 on ubuntu 12.04 LTS

Do you guys have any ideas what is happening and how to solve this ?

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37
http://perfect-memory.com

https://twitter.com/semanticbushttp://www.facebook.com/pages/Perfect-Memory/155555567821817

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37
http://perfect-memory.com

https://twitter.com/semanticbushttp://www.facebook.com/pages/Perfect-Memory/155555567821817

Try sticking a "use_inline_resources" at the top of your LWRP file.
It does that whole run_context pattern to allow for notifications to work.
I'm not sure if include_recipe will work.
-s

On Wed, May 29, 2013 at 1:39 PM, Ranjib Dey dey.ranjib@gmail.com wrote:

try this

rc =run_context.dup
rc.resource_collection = Chef::ResourceCollection.new
rc.load_recipe("foo::bar")
Chef::Runner.new(rc).converge

.. for your case u'll be adding all the recipes,
make sure u capture the updated resources. and set the
new_resource.updated_by_last_action accordingly

On Wed, May 29, 2013 at 9:06 AM, Dorian Jaminais <
dorian.jaminais@perfect-memory.com> wrote:

I am still tracking down my problem and I have some news.
It appears that he problem is not specific to the git cookbook.
Indeed it appears that all recipes included in a LWRP
'run_context.include_recipe' is only really added to the run_list if the
LWRP is called once during the run_list.

If the LWRP is called more than once, it tells me that the resource has
already been added to the run_list even if it has not.

Is this a known bug ? I am doing something wrong ?
I am failing to find any clues using my google-fu so I am turning to your
expertise.

2013/5/29 Dorian Jaminais dorian.jaminais@perfect-memory.com

Hi everyone

I am having trouble with LWRP, include_recipe and roles.

I have a LWRP that includes some cookbooks :
run_context.include_recipe "git"
run_context.include_recipe "internal-cookbook"
run_context.include_recipe "build-essential"

The git cookbook is the one available at opscode.
I use this recipe in a variety of other internal cookbooks.

Here is my problem : when I add the cookbooks that use the LWRP directly
to a node's run_list, it works perfectly but when I add them via a role,
git is not installed. It works for the two other recipes included in the
LWRP but not for git.

In the logs I have :
DEBUG: Loading Recipe git via include_recipe
DEBUG: Found recipe default in cookbook git
and then it goes to the next cookbook without the expected
INFO: Processing package[git] action install (base::default line 9)

I am running chef 11.4.4 on ubuntu 12.04 LTS

Do you guys have any ideas what is happening and how to solve this ?

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37
http://perfect-memory.com

https://twitter.com/semanticbushttp://www.facebook.com/pages/Perfect-Memory/155555567821817

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37
http://perfect-memory.com

https://twitter.com/semanticbushttp://www.facebook.com/pages/Perfect-Memory/155555567821817

i think both are fine, the non lwrp way. u'll be introducing the
conditionals inside the recipe.. which will

  1. reduce the declarativeness of the recipe
  2. u'll be limited by the compile time info , in certain cases
  3. testing will be harder

the lwrp opens up a much cleaner way to reuse this, brings the conditionals
inside an abstraction, and also allows you to set your resource's update
status depending upon the entire dependency (i.e. now you can say i was
updated if any of the dependant resource were update), which you might
exploit using notification mechanism is anywhere else... u'll lose most of
this if you push these logic in the recipe...
no?

On Wed, May 29, 2013 at 10:38 AM, Sean Escriva sean.escriva@gmail.comwrote:

very neat.

Notifications from the lwrp are one thing, but IMO include_recipe
shouldn't need to be in the lwrp because as John mentioned the cookbook
that provides the lwrp still needs to be included to make use of the
specific lwrp and as such will run the default recipe. It should also
result in a simpler lwrp. This seems easier to reason about as well since
the lwrp is not the distribution mechanism for functionality, the cookbook
is.

As Dan said though, we probably need to see more code to understand the
full issue

On Wed, May 29, 2013 at 10:07 AM, Ranjib Dey dey.ranjib@gmail.com wrote:

i think its fine to include recipes via provider (or rather execute a
whole recipe)

create a subcontext , and a mini chef run inside it, and converge the
node. this is how we used to reuse existing chef resource inside an lwrp
earlier..
yfeldblum had a gist on it..

LWRP notifying on changed resources

On Wed, May 29, 2013 at 9:46 AM, Dorian Jaminais <
dorian.jaminais@perfect-memory.com> wrote:

Yes this can be a workaround, still I don't think the behavior I am
getting is normal (and don't understand why) and I'd prefer to find a
cleaner way than having to add a recipe just to include other recipes
whereas it should be the task of my LWRP.

2013/5/29 John E. Vincent (lusis) lusis.org+chef-list@gmail.com

Is there a reason you wouldn't, however, just add the "include_recipe"

statements to the default recipe for the LWRP cookbook then? It's not
uncommon to do that even for library cookbooks iirc and it gets around (I
think) the issue you're describing.

On Wed, May 29, 2013 at 12:39 PM, Dorian Jaminais <
dorian.jaminais@perfect-memory.com> wrote:

Okay

I am using a LWRP instead of a recipe because I want to deploy about
15 applications that are very similar.
All those applications have a dedicated cookbook but as the
installation process is always the same work flow, I have factorized it
into a LWRP.

The work flow is pretty simple : assert that installation dependencies
are met, clone the repo, bundle install the needed gems.

I have attached my provider to this mail (which by the way is included
in a cookbook named gios).

In each application's cookbook I use :
gios_installer NAME do
name NAME
repository GIT_URI
command "./bin/truc"
version "master"
action :install
end

As stated in my previous emails, using the LWRP that way will include
the recipes ruby, git, base::certificate, pm-openssh::rsa and
pm-openssh::known_hosts only if the LWRP is called exactly once in the
run_list.

Thanks for your help

2013/5/29 Daniel DeLeo dan@kallistec.com

On Wednesday, May 29, 2013 at 9:06 AM, Dorian Jaminais wrote:

I am still tracking down my problem and I have some news.
It appears that he problem is not specific to the git cookbook.
Indeed it appears that all recipes included in a LWRP
'run_context.include_recipe' is only really added to the run_list if the
LWRP is called once during the run_list.

If the LWRP is called more than once, it tells me that the resource
has already been added to the run_list even if it has not.

Is this a known bug ? I am doing something wrong ?
I am failing to find any clues using my google-fu so I am turning to
your expertise.

2013/5/29 Dorian Jaminais dorian.jaminais@perfect-memory.com

Hi everyone

I am having trouble with LWRP, include_recipe and roles.

I have a LWRP that includes some cookbooks :
run_context.include_recipe "git"
run_context.include_recipe "internal-cookbook"
run_context.include_recipe "build-essential"

This is a bit odd. Why does the LWRP do this instead of a recipe?
Showing more code, log output, and explaining why you're doing things this
way would help others to understand your issue better.

--
Daniel DeLeo

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37
http://perfect-memory.com

https://twitter.com/semanticbushttp://www.facebook.com/pages/Perfect-Memory/155555567821817

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37
http://perfect-memory.com

https://twitter.com/semanticbushttp://www.facebook.com/pages/Perfect-Memory/155555567821817

Just to understand better, do I actually need to include a library cookbook's recipe in the run_list of a node in order to use its LWRPs?

I remember a thread not long ago in this mailing list about "how to test library cookbooks without recipes" or something like that, and someone specifically mentioned that requiring the lib cookbook in the metadata file would be enough.

Just checking. :slight_smile:

  • cassiano

On Wednesday, May 29, 2013 at 14:45, Ranjib Dey wrote:

i think both are fine, the non lwrp way. u'll be introducing the conditionals inside the recipe.. which will

  1. reduce the declarativeness of the recipe
  2. u'll be limited by the compile time info , in certain cases
  3. testing will be harder

the lwrp opens up a much cleaner way to reuse this, brings the conditionals inside an abstraction, and also allows you to set your resource's update status depending upon the entire dependency (i.e. now you can say i was updated if any of the dependant resource were update), which you might exploit using notification mechanism is anywhere else... u'll lose most of this if you push these logic in the recipe...
no?

On Wed, May 29, 2013 at 10:38 AM, Sean Escriva <sean.escriva@gmail.com (mailto:sean.escriva@gmail.com)> wrote:

very neat.

Notifications from the lwrp are one thing, but IMO include_recipe shouldn't need to be in the lwrp because as John mentioned the cookbook that provides the lwrp still needs to be included to make use of the specific lwrp and as such will run the default recipe. It should also result in a simpler lwrp. This seems easier to reason about as well since the lwrp is not the distribution mechanism for functionality, the cookbook is.

As Dan said though, we probably need to see more code to understand the full issue

On Wed, May 29, 2013 at 10:07 AM, Ranjib Dey <dey.ranjib@gmail.com (mailto:dey.ranjib@gmail.com)> wrote:

i think its fine to include recipes via provider (or rather execute a whole recipe)

create a subcontext , and a mini chef run inside it, and converge the node. this is how we used to reuse existing chef resource inside an lwrp earlier..
yfeldblum had a gist on it..

LWRP notifying on changed resources

On Wed, May 29, 2013 at 9:46 AM, Dorian Jaminais <dorian.jaminais@perfect-memory.com (mailto:dorian.jaminais@perfect-memory.com)> wrote:

Yes this can be a workaround, still I don't think the behavior I am getting is normal (and don't understand why) and I'd prefer to find a cleaner way than having to add a recipe just to include other recipes whereas it should be the task of my LWRP.

2013/5/29 John E. Vincent (lusis) <lusis.org+chef-list@gmail.com (mailto:lusis.org+chef-list@gmail.com)>

Is there a reason you wouldn't, however, just add the "include_recipe" statements to the default recipe for the LWRP cookbook then? It's not uncommon to do that even for library cookbooks iirc and it gets around (I think) the issue you're describing.

On Wed, May 29, 2013 at 12:39 PM, Dorian Jaminais <dorian.jaminais@perfect-memory.com (mailto:dorian.jaminais@perfect-memory.com)> wrote:

Okay

I am using a LWRP instead of a recipe because I want to deploy about 15 applications that are very similar.
All those applications have a dedicated cookbook but as the installation process is always the same work flow, I have factorized it into a LWRP.

The work flow is pretty simple : assert that installation dependencies are met, clone the repo, bundle install the needed gems.

I have attached my provider to this mail (which by the way is included in a cookbook named gios).

In each application's cookbook I use :
gios_installer NAME do
name NAME
repository GIT_URI
command "./bin/truc"
version "master"
action :install
end

As stated in my previous emails, using the LWRP that way will include the recipes ruby, git, base::certificate, pm-openssh::rsa and pm-openssh::known_hosts only if the LWRP is called exactly once in the run_list.

Thanks for your help

2013/5/29 Daniel DeLeo <dan@kallistec.com (mailto:dan@kallistec.com)>

On Wednesday, May 29, 2013 at 9:06 AM, Dorian Jaminais wrote:

I am still tracking down my problem and I have some news.
It appears that he problem is not specific to the git cookbook.
Indeed it appears that all recipes included in a LWRP 'run_context.include_recipe' is only really added to the run_list if the LWRP is called once during the run_list.

If the LWRP is called more than once, it tells me that the resource has already been added to the run_list even if it has not.

Is this a known bug ? I am doing something wrong ?
I am failing to find any clues using my google-fu so I am turning to your expertise.

2013/5/29 Dorian Jaminais <dorian.jaminais@perfect-memory.com (mailto:dorian.jaminais@perfect-memory.com)>

Hi everyone

I am having trouble with LWRP, include_recipe and roles.

I have a LWRP that includes some cookbooks :
run_context.include_recipe "git"
run_context.include_recipe "internal-cookbook"
run_context.include_recipe "build-essential"

This is a bit odd. Why does the LWRP do this instead of a recipe? Showing more code, log output, and explaining why you're doing things this way would help others to understand your issue better.

--
Daniel DeLeo

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37 (tel:%2B33%206%2095%2010%2095%2037)

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37

no.. you dont have to.. the libraries and lwrps are available just by
declaring the dependency, u dont have to include the parent cookbook's
default or any other recipe.

On Wed, May 29, 2013 at 11:00 AM, Cassiano Leal cassianoleal@gmail.comwrote:

Just to understand better, do I actually need to include a library
cookbook's recipe in the run_list of a node in order to use its LWRPs?

I remember a thread not long ago in this mailing list about "how to test
library cookbooks without recipes" or something like that, and someone
specifically mentioned that requiring the lib cookbook in the metadata file
would be enough.

Just checking. :slight_smile:

  • cassiano

On Wednesday, May 29, 2013 at 14:45, Ranjib Dey wrote:

i think both are fine, the non lwrp way. u'll be introducing the
conditionals inside the recipe.. which will

  1. reduce the declarativeness of the recipe
  2. u'll be limited by the compile time info , in certain cases
  3. testing will be harder

the lwrp opens up a much cleaner way to reuse this, brings the
conditionals inside an abstraction, and also allows you to set your
resource's update status depending upon the entire dependency (i.e. now you
can say i was updated if any of the dependant resource were update), which
you might exploit using notification mechanism is anywhere else... u'll
lose most of this if you push these logic in the recipe...
no?

On Wed, May 29, 2013 at 10:38 AM, Sean Escriva sean.escriva@gmail.comwrote:

very neat.

Notifications from the lwrp are one thing, but IMO include_recipe
shouldn't need to be in the lwrp because as John mentioned the cookbook
that provides the lwrp still needs to be included to make use of the
specific lwrp and as such will run the default recipe. It should also
result in a simpler lwrp. This seems easier to reason about as well since
the lwrp is not the distribution mechanism for functionality, the cookbook
is.

As Dan said though, we probably need to see more code to understand the
full issue

On Wed, May 29, 2013 at 10:07 AM, Ranjib Dey dey.ranjib@gmail.com wrote:

i think its fine to include recipes via provider (or rather execute a
whole recipe)

create a subcontext , and a mini chef run inside it, and converge the
node. this is how we used to reuse existing chef resource inside an lwrp
earlier..
yfeldblum had a gist on it..

LWRP notifying on changed resources

On Wed, May 29, 2013 at 9:46 AM, Dorian Jaminais <
dorian.jaminais@perfect-memory.com> wrote:

Yes this can be a workaround, still I don't think the behavior I am
getting is normal (and don't understand why) and I'd prefer to find a
cleaner way than having to add a recipe just to include other recipes
whereas it should be the task of my LWRP.

2013/5/29 John E. Vincent (lusis) lusis.org+chef-list@gmail.com

Is there a reason you wouldn't, however, just add the "include_recipe"
statements to the default recipe for the LWRP cookbook then? It's not
uncommon to do that even for library cookbooks iirc and it gets around (I
think) the issue you're describing.

On Wed, May 29, 2013 at 12:39 PM, Dorian Jaminais <
dorian.jaminais@perfect-memory.com> wrote:

Okay

I am using a LWRP instead of a recipe because I want to deploy about 15
applications that are very similar.
All those applications have a dedicated cookbook but as the installation
process is always the same work flow, I have factorized it into a LWRP.

The work flow is pretty simple : assert that installation dependencies are
met, clone the repo, bundle install the needed gems.

I have attached my provider to this mail (which by the way is included in
a cookbook named gios).

In each application's cookbook I use :
gios_installer NAME do
name NAME
repository GIT_URI
command "./bin/truc"
version "master"
action :install
end

As stated in my previous emails, using the LWRP that way will include the
recipes ruby, git, base::certificate, pm-openssh::rsa and
pm-openssh::known_hosts only if the LWRP is called exactly once in the
run_list.

Thanks for your help

2013/5/29 Daniel DeLeo dan@kallistec.com

On Wednesday, May 29, 2013 at 9:06 AM, Dorian Jaminais wrote:

I am still tracking down my problem and I have some news.
It appears that he problem is not specific to the git cookbook.
Indeed it appears that all recipes included in a LWRP
'run_context.include_recipe' is only really added to the run_list if the
LWRP is called once during the run_list.

If the LWRP is called more than once, it tells me that the resource has
already been added to the run_list even if it has not.

Is this a known bug ? I am doing something wrong ?
I am failing to find any clues using my google-fu so I am turning to your
expertise.

2013/5/29 Dorian Jaminais dorian.jaminais@perfect-memory.com

Hi everyone

I am having trouble with LWRP, include_recipe and roles.

I have a LWRP that includes some cookbooks :
run_context.include_recipe "git"
run_context.include_recipe "internal-cookbook"
run_context.include_recipe "build-essential"

This is a bit odd. Why does the LWRP do this instead of a recipe? Showing
more code, log output, and explaining why you're doing things this way
would help others to understand your issue better.

--
Daniel DeLeo

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37
http://perfect-memory.com

https://twitter.com/semanticbushttp://www.facebook.com/pages/Perfect-Memory/155555567821817

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37
http://perfect-memory.com

https://twitter.com/semanticbushttp://www.facebook.com/pages/Perfect-Memory/155555567821817

I'm still confused by this. The windows and runit resources, as an example,
are not available to a recipe unless include_recipe is used at the top.
They've always failed for me otherwise (regardless of including the
dependency in metadata.rb). I never bothered to revisit if this was a bug
and I've always done include_recipe "runit" (or any recipe for that matter)
where I needed the resources from that LWRP in the recipe itself.

I'm totally willing to admit I've been wrong all this time but there was a
reason I did it at one point and it fixed the problem at the time. I just
never went back and challenged that assumption.

On Wed, May 29, 2013 at 2:58 PM, Ranjib Dey dey.ranjib@gmail.com wrote:

no.. you dont have to.. the libraries and lwrps are available just by
declaring the dependency, u dont have to include the parent cookbook's
default or any other recipe.

On Wed, May 29, 2013 at 11:00 AM, Cassiano Leal cassianoleal@gmail.comwrote:

Just to understand better, do I actually need to include a library
cookbook's recipe in the run_list of a node in order to use its LWRPs?

I remember a thread not long ago in this mailing list about "how to test
library cookbooks without recipes" or something like that, and someone
specifically mentioned that requiring the lib cookbook in the metadata file
would be enough.

Just checking. :slight_smile:

  • cassiano

On Wednesday, May 29, 2013 at 14:45, Ranjib Dey wrote:

i think both are fine, the non lwrp way. u'll be introducing the
conditionals inside the recipe.. which will

  1. reduce the declarativeness of the recipe
  2. u'll be limited by the compile time info , in certain cases
  3. testing will be harder

the lwrp opens up a much cleaner way to reuse this, brings the
conditionals inside an abstraction, and also allows you to set your
resource's update status depending upon the entire dependency (i.e. now you
can say i was updated if any of the dependant resource were update), which
you might exploit using notification mechanism is anywhere else... u'll
lose most of this if you push these logic in the recipe...
no?

On Wed, May 29, 2013 at 10:38 AM, Sean Escriva sean.escriva@gmail.comwrote:

very neat.

Notifications from the lwrp are one thing, but IMO include_recipe
shouldn't need to be in the lwrp because as John mentioned the cookbook
that provides the lwrp still needs to be included to make use of the
specific lwrp and as such will run the default recipe. It should also
result in a simpler lwrp. This seems easier to reason about as well since
the lwrp is not the distribution mechanism for functionality, the cookbook
is.

As Dan said though, we probably need to see more code to understand the
full issue

On Wed, May 29, 2013 at 10:07 AM, Ranjib Dey dey.ranjib@gmail.comwrote:

i think its fine to include recipes via provider (or rather execute a
whole recipe)

create a subcontext , and a mini chef run inside it, and converge the
node. this is how we used to reuse existing chef resource inside an lwrp
earlier..
yfeldblum had a gist on it..

http://realityforge.org/code/2012/07/17/lwrp-notify-on-changed-resources.html

On Wed, May 29, 2013 at 9:46 AM, Dorian Jaminais <
dorian.jaminais@perfect-memory.com> wrote:

Yes this can be a workaround, still I don't think the behavior I am
getting is normal (and don't understand why) and I'd prefer to find a
cleaner way than having to add a recipe just to include other recipes
whereas it should be the task of my LWRP.

2013/5/29 John E. Vincent (lusis) lusis.org+chef-list@gmail.com

Is there a reason you wouldn't, however, just add the "include_recipe"
statements to the default recipe for the LWRP cookbook then? It's not
uncommon to do that even for library cookbooks iirc and it gets around (I
think) the issue you're describing.

On Wed, May 29, 2013 at 12:39 PM, Dorian Jaminais <
dorian.jaminais@perfect-memory.com> wrote:

Okay

I am using a LWRP instead of a recipe because I want to deploy about 15
applications that are very similar.
All those applications have a dedicated cookbook but as the installation
process is always the same work flow, I have factorized it into a LWRP.

The work flow is pretty simple : assert that installation dependencies
are met, clone the repo, bundle install the needed gems.

I have attached my provider to this mail (which by the way is included in
a cookbook named gios).

In each application's cookbook I use :
gios_installer NAME do
name NAME
repository GIT_URI
command "./bin/truc"
version "master"
action :install
end

As stated in my previous emails, using the LWRP that way will include the
recipes ruby, git, base::certificate, pm-openssh::rsa and
pm-openssh::known_hosts only if the LWRP is called exactly once in the
run_list.

Thanks for your help

2013/5/29 Daniel DeLeo dan@kallistec.com

On Wednesday, May 29, 2013 at 9:06 AM, Dorian Jaminais wrote:

I am still tracking down my problem and I have some news.
It appears that he problem is not specific to the git cookbook.
Indeed it appears that all recipes included in a LWRP
'run_context.include_recipe' is only really added to the run_list if the
LWRP is called once during the run_list.

If the LWRP is called more than once, it tells me that the resource has
already been added to the run_list even if it has not.

Is this a known bug ? I am doing something wrong ?
I am failing to find any clues using my google-fu so I am turning to your
expertise.

2013/5/29 Dorian Jaminais dorian.jaminais@perfect-memory.com

Hi everyone

I am having trouble with LWRP, include_recipe and roles.

I have a LWRP that includes some cookbooks :
run_context.include_recipe "git"
run_context.include_recipe "internal-cookbook"
run_context.include_recipe "build-essential"

This is a bit odd. Why does the LWRP do this instead of a recipe? Showing
more code, log output, and explaining why you're doing things this way
would help others to understand your issue better.

--
Daniel DeLeo

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37
http://perfect-memory.com

https://twitter.com/semanticbushttp://www.facebook.com/pages/Perfect-Memory/155555567821817

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37
http://perfect-memory.com

https://twitter.com/semanticbushttp://www.facebook.com/pages/Perfect-Memory/155555567821817

On Wednesday, May 29, 2013 at 12:07 PM, John E. Vincent (lusis) wrote:

I'm still confused by this. The windows and runit resources, as an example, are not available to a recipe unless include_recipe is used at the top. They've always failed for me otherwise (regardless of including the dependency in metadata.rb). I never bothered to revisit if this was a bug and I've always done include_recipe "runit" (or any recipe for that matter) where I needed the resources from that LWRP in the recipe itself.

I'm totally willing to admit I've been wrong all this time but there was a reason I did it at one point and it fixed the problem at the time. I just never went back and challenged that assumption.
LWRPs, resource definitions, etc. are loaded from all available cookbooks (modulo a caveat that chef-solo now requires metadata deps to consider a cookbook "available"). Relevant Chef 11 code: chef/lib/chef/run_context/cookbook_compiler.rb at main · chef/chef · GitHub

If you're talking about a LWRP or resource definition actually being able to succeed, then you will definitely need whatever system state it depends on to be available. Using runit as an example, you could probably write a "install-runit-my-own-way" cookbook to get runit on the system and then use the runit resource (resource definition until recently) without issue, but it definitely won't work if you haven't installed runit.

Bringing this back to the OP's issue, I think including a recipe (the normal way) to install prerequisites of a LWRP is the right way to go about it. If you are doing wizardry with chef internals, either 1) you're making something more complicated than it should be, 2) chef is making something more complicated than it should be, or 3) you're a wizard. So far, I'm inclined to think that including recipes from a provider falls into category #1, since you can just as easily and elegantly solve the problem with a role (or role cookbook if that's your thing). Am I missing something?

--
Daniel DeLeo

What about the case where you include_recipe from your provider in a rescue gem require. so get the gem installed and continue on.

I think this case is ok. If the wrapper wants to change the behavior they can, and as an author I can set an attribute to disable the include or return from the included recipe. Possibly Install some other gem that satisfies the dep.

The reason for wanting to do this is hide the implementation.

LWRPs should just work for the common case IMO.

An alternative that i have used is an explicit cookbook::library recipe that handles the setup for just the lib/LWRP. Usage. I like this as an operator because I can see this on the run_list and know the intent.

On May 29, 2013, at 3:43 PM, Daniel DeLeo dan@kallistec.com wrote:

On Wednesday, May 29, 2013 at 12:07 PM, John E. Vincent (lusis) wrote:

I'm still confused by this. The windows and runit resources, as an example, are not available to a recipe unless include_recipe is used at the top. They've always failed for me otherwise (regardless of including the dependency in metadata.rb). I never bothered to revisit if this was a bug and I've always done include_recipe "runit" (or any recipe for that matter) where I needed the resources from that LWRP in the recipe itself.

I'm totally willing to admit I've been wrong all this time but there was a reason I did it at one point and it fixed the problem at the time. I just never went back and challenged that assumption.
LWRPs, resource definitions, etc. are loaded from all available cookbooks (modulo a caveat that chef-solo now requires metadata deps to consider a cookbook "available"). Relevant Chef 11 code: chef/lib/chef/run_context/cookbook_compiler.rb at main · chef/chef · GitHub

If you're talking about a LWRP or resource definition actually being able to succeed, then you will definitely need whatever system state it depends on to be available. Using runit as an example, you could probably write a "install-runit-my-own-way" cookbook to get runit on the system and then use the runit resource (resource definition until recently) without issue, but it definitely won't work if you haven't installed runit.

Bringing this back to the OP's issue, I think including a recipe (the normal way) to install prerequisites of a LWRP is the right way to go about it. If you are doing wizardry with chef internals, either 1) you're making something more complicated than it should be, 2) chef is making something more complicated than it should be, or 3) you're a wizard. So far, I'm inclined to think that including recipes from a provider falls into category #1, since you can just as easily and elegantly solve the problem with a role (or role cookbook if that's your thing). Am I missing something?

--
Daniel DeLeo

lusis: I think include_recipe is no longer required to access libraries in chef 11, but I vaguely rembemer this being an issue in chef <= 10. I'm pretty sure that's why many library cookbooks have nop recipes/default.rb files.

Also, include_recipe used to be unavailable in the lwrp dsl, which was one of the many reasons I just used heavyweight RPs for a long time. I'm pretty sure this is also no longer true.

And now I want to grumble like an old man. Stupid newfangled DSLs fixing all the things that confused me when I started...

~j

On 2013-05-29, at 12:07, John E. Vincent (lusis) lusis.org+chef-list@gmail.com wrote:

I'm still confused by this. The windows and runit resources, as an example, are not available to a recipe unless include_recipe is used at the top. They've always failed for me otherwise (regardless of including the dependency in metadata.rb). I never bothered to revisit if this was a bug and I've always done include_recipe "runit" (or any recipe for that matter) where I needed the resources from that LWRP in the recipe itself.

I'm totally willing to admit I've been wrong all this time but there was a reason I did it at one point and it fixed the problem at the time. I just never went back and challenged that assumption.

On Wed, May 29, 2013 at 2:58 PM, Ranjib Dey dey.ranjib@gmail.com wrote:
no.. you dont have to.. the libraries and lwrps are available just by declaring the dependency, u dont have to include the parent cookbook's default or any other recipe.

On Wed, May 29, 2013 at 11:00 AM, Cassiano Leal cassianoleal@gmail.com wrote:
Just to understand better, do I actually need to include a library cookbook's recipe in the run_list of a node in order to use its LWRPs?

I remember a thread not long ago in this mailing list about "how to test library cookbooks without recipes" or something like that, and someone specifically mentioned that requiring the lib cookbook in the metadata file would be enough.

Just checking. :slight_smile:

  • cassiano

On Wednesday, May 29, 2013 at 14:45, Ranjib Dey wrote:

i think both are fine, the non lwrp way. u'll be introducing the conditionals inside the recipe.. which will

  1. reduce the declarativeness of the recipe
  2. u'll be limited by the compile time info , in certain cases
  3. testing will be harder

the lwrp opens up a much cleaner way to reuse this, brings the conditionals inside an abstraction, and also allows you to set your resource's update status depending upon the entire dependency (i.e. now you can say i was updated if any of the dependant resource were update), which you might exploit using notification mechanism is anywhere else... u'll lose most of this if you push these logic in the recipe...
no?

On Wed, May 29, 2013 at 10:38 AM, Sean Escriva sean.escriva@gmail.com wrote:

very neat.

Notifications from the lwrp are one thing, but IMO include_recipe shouldn't need to be in the lwrp because as John mentioned the cookbook that provides the lwrp still needs to be included to make use of the specific lwrp and as such will run the default recipe. It should also result in a simpler lwrp. This seems easier to reason about as well since the lwrp is not the distribution mechanism for functionality, the cookbook is.

As Dan said though, we probably need to see more code to understand the full issue

On Wed, May 29, 2013 at 10:07 AM, Ranjib Dey dey.ranjib@gmail.com wrote:

i think its fine to include recipes via provider (or rather execute a whole recipe)

create a subcontext , and a mini chef run inside it, and converge the node. this is how we used to reuse existing chef resource inside an lwrp earlier..
yfeldblum had a gist on it..

http://realityforge.org/code/2012/07/17/lwrp-notify-on-changed-resources.html

On Wed, May 29, 2013 at 9:46 AM, Dorian Jaminais dorian.jaminais@perfect-memory.com wrote:

Yes this can be a workaround, still I don't think the behavior I am getting is normal (and don't understand why) and I'd prefer to find a cleaner way than having to add a recipe just to include other recipes whereas it should be the task of my LWRP.

2013/5/29 John E. Vincent (lusis) lusis.org+chef-list@gmail.com

Is there a reason you wouldn't, however, just add the "include_recipe" statements to the default recipe for the LWRP cookbook then? It's not uncommon to do that even for library cookbooks iirc and it gets around (I think) the issue you're describing.

On Wed, May 29, 2013 at 12:39 PM, Dorian Jaminais dorian.jaminais@perfect-memory.com wrote:

Okay

I am using a LWRP instead of a recipe because I want to deploy about 15 applications that are very similar.
All those applications have a dedicated cookbook but as the installation process is always the same work flow, I have factorized it into a LWRP.

The work flow is pretty simple : assert that installation dependencies are met, clone the repo, bundle install the needed gems.

I have attached my provider to this mail (which by the way is included in a cookbook named gios).

In each application's cookbook I use :
gios_installer NAME do
name NAME
repository GIT_URI
command "./bin/truc"
version "master"
action :install
end

As stated in my previous emails, using the LWRP that way will include the recipes ruby, git, base::certificate, pm-openssh::rsa and pm-openssh::known_hosts only if the LWRP is called exactly once in the run_list.

Thanks for your help

2013/5/29 Daniel DeLeo dan@kallistec.com

On Wednesday, May 29, 2013 at 9:06 AM, Dorian Jaminais wrote:

I am still tracking down my problem and I have some news.
It appears that he problem is not specific to the git cookbook.
Indeed it appears that all recipes included in a LWRP 'run_context.include_recipe' is only really added to the run_list if the LWRP is called once during the run_list.

If the LWRP is called more than once, it tells me that the resource has already been added to the run_list even if it has not.

Is this a known bug ? I am doing something wrong ?
I am failing to find any clues using my google-fu so I am turning to your expertise.

2013/5/29 Dorian Jaminais dorian.jaminais@perfect-memory.com

Hi everyone

I am having trouble with LWRP, include_recipe and roles.

I have a LWRP that includes some cookbooks :
run_context.include_recipe "git"
run_context.include_recipe "internal-cookbook"
run_context.include_recipe "build-essential"

This is a bit odd. Why does the LWRP do this instead of a recipe? Showing more code, log output, and explaining why you're doing things this way would help others to understand your issue better.

--
Daniel DeLeo

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37

Hi,

On Thu, May 30, 2013 at 8:43 AM, Daniel DeLeo dan@kallistec.com wrote:

Bringing this back to the OP's issue, I think including a recipe (the normal
way) to install prerequisites of a LWRP is the right way to go about it. If
you are doing wizardry with chef internals, either 1) you're making
something more complicated than it should be, 2) chef is making something
more complicated than it should be, or 3) you're a wizard. So far, I'm
inclined to think that including recipes from a provider falls into category
#1, since you can just as easily and elegantly solve the problem with a role
(or role cookbook if that's your thing). Am I missing something?

I think it is more useful to see it as a failure of chef rather than a
failure of the user. i.e. Option 2.

In my mind LWRPs are a mechanism for creating composable
abstractions. Anything you can do in a recipe should be possible to do
in a LWRP. Where this is not true I would consider it a bug.

For this specific scenario I can easily see a use for it. In our
environment, many of our recipes are data driven. Data is collected
from search + rules + external systems (ldap/db etc) and then this is
interpreted and resources defined based on the data. So often we will
not know which LWRPs will be active (and thus which precursor recipes
should be included) until the data is interpreted.

That said I thought include_recipe worked from within LWRPs...

--
Cheers,

Peter Donald

Thank you all for your responses.

@peter : run_context.include_recipe works but only if the LWRP is called
once in during the chef_run.

I have included a recipe that installs all the dependencies for the LWRP to
work.
This works perfectly, even if still I don't understand why I had this
strange behavior with run_context.include_recipe

2013/5/30 Peter Donald peter@realityforge.org

Hi,

On Thu, May 30, 2013 at 8:43 AM, Daniel DeLeo dan@kallistec.com wrote:

Bringing this back to the OP's issue, I think including a recipe (the
normal
way) to install prerequisites of a LWRP is the right way to go about it.
If
you are doing wizardry with chef internals, either 1) you're making
something more complicated than it should be, 2) chef is making something
more complicated than it should be, or 3) you're a wizard. So far, I'm
inclined to think that including recipes from a provider falls into
category
#1, since you can just as easily and elegantly solve the problem with a
role
(or role cookbook if that's your thing). Am I missing something?

I think it is more useful to see it as a failure of chef rather than a
failure of the user. i.e. Option 2.

In my mind LWRPs are a mechanism for creating composable
abstractions. Anything you can do in a recipe should be possible to do
in a LWRP. Where this is not true I would consider it a bug.

For this specific scenario I can easily see a use for it. In our
environment, many of our recipes are data driven. Data is collected
from search + rules + external systems (ldap/db etc) and then this is
interpreted and resources defined based on the data. So often we will
not know which LWRPs will be active (and thus which precursor recipes
should be included) until the data is interpreted.

That said I thought include_recipe worked from within LWRPs...

--
Cheers,

Peter Donald

--
Dorian JAMINAIS
System Administrator
+33 6 95 10 95 37
http://perfect-memory.com

https://twitter.com/semanticbushttp://www.facebook.com/pages/Perfect-Memory/155555567821817