Object-Oriented Chef


#1

Hi,

It would be nice if there was better support for the native Ruby Chef classes/methods aside from the Chef::Recipe DSL with its fixed structure of cookbooks/recipes, etc. It seems to me that with access to the run_context class and the various resources, one could implement multiple run/convergence queues easily enough (by instantiating different run_contexts and operating on them), for example, and it would also open up the ability to use traditional classes/modules for adding structure and reusability, rather than the limited Chef::Recipe API. I understand that the current DSL is concise, convenient for beginners and mimics what Puppet uses, but it lacks structural power.

Thoughts?

Jos


#2

On Sep 16, 2015, at 3:44 PM, Jos Backus josb@mobileiron.com wrote:

Hi,

It would be nice if there was better support for the native Ruby Chef classes/methods aside from the Chef::Recipe DSL with its fixed structure of cookbooks/recipes, etc. It seems to me that with access to the run_context class and the various resources, one could implement multiple run/convergence queues easily enough (by instantiating different run_contexts and operating on them), for example, and it would also open up the ability to use traditional classes/modules for adding structure and reusability, rather than the limited Chef::Recipe API. I understand that the current DSL is concise, convenient for beginners and mimics what Puppet uses, but it lacks structural power.

This is exactly what my Poise (https://github.com/poise/poise) library and John’s Resource library do. This rabbit hole runs deep, happy to answer questions for you, though IRC (#chef on Freenode) might be a better medium.

–Noah


#3

Hi Noah,

On 9/16/15, 3:47 PM, “Noah Kantrowitz” noah@coderanger.net wrote:

On Sep 16, 2015, at 3:44 PM, Jos Backus josb@mobileiron.com wrote:

Hi,

It would be nice if there was better support for the native Ruby Chef classes/methods aside from the Chef::Recipe DSL with its fixed structure of cookbooks/recipes, etc. It seems to me that with access to the run_context class and the various resources, one could implement multiple run/convergence queues easily enough (by instantiating different run_contexts and operating on them), for example, and it would also open up the ability to use traditional classes/modules for adding structure and reusability, rather than the limited Chef::Recipe API. I understand that the current DSL is concise, convenient for beginners and mimics what Puppet uses, but it lacks structural power.

This is exactly what my Poise (https://github.com/poise/poise) library and John’s Resource library do. This rabbit hole runs deep, happy to answer questions for you, though IRC (#chef on Freenode) might be a better medium.

Thanks for the quick reply!

Looking at Poise, I still see recipes and cookbooks, which means ultimately we are still tied to the Chef::Recipe DSL. Iow, Poise seems to be an alternative to writing LWRPs. Maybe I misunderstand?

What I’m interested in is a pure Ruby solution without cookbooks/recipes. I want to write classes and modules with Chef::Resource::File.new (etc.) calls, not `file “foo” do … end’. I think of the Chef DSL as one somewhat convenient but not very powerful interface to the underlying Chef Resource classes it fronts; one could use those classes independently of the DSL. E.g.

run_context = Chef::RunContext.new(Chef::Node.new, {}, Chef::EventDispatch::Dispatcher.new)

Chef::Resource::Directory.new(“/my_dir", run_context) do |res|
res.mode 0755
res.owner ‘root’
res.group ‘root’
res.run_action(:create)
end

Chef::Resource::File.new(“/my_dir/foo", run_context) do |res|
res.mode 0644
res.owner ‘root’
res.group ‘root’
res.run_action(:create)
res.content = “some content"
end

result = run_context.converge

Of course the above could live in classes/modules as is normal with Ruby code.

Hope this explains what I’m interested in a little better.

Jos

–Noah


#4

On Sep 16, 2015, at 4:04 PM, Jos Backus josb@mobileiron.com wrote:

Hi Noah,

On 9/16/15, 3:47 PM, “Noah Kantrowitz” noah@coderanger.net wrote:

On Sep 16, 2015, at 3:44 PM, Jos Backus josb@mobileiron.com wrote:

Hi,

It would be nice if there was better support for the native Ruby Chef classes/methods aside from the Chef::Recipe DSL with its fixed structure of cookbooks/recipes, etc. It seems to me that with access to the run_context class and the various resources, one could implement multiple run/convergence queues easily enough (by instantiating different run_contexts and operating on them), for example, and it would also open up the ability to use traditional classes/modules for adding structure and reusability, rather than the limited Chef::Recipe API. I understand that the current DSL is concise, convenient for beginners and mimics what Puppet uses, but it lacks structural power.

This is exactly what my Poise (https://github.com/poise/poise) library and John’s Resource library do. This rabbit hole runs deep, happy to answer questions for you, though IRC (#chef on Freenode) might be a better medium.

Thanks for the quick reply!

Looking at Poise, I still see recipes and cookbooks, which means ultimately we are still tied to the Chef::Recipe DSL. Iow, Poise seems to be an alternative to writing LWRPs. Maybe I misunderstand?

What I’m interested in is a pure Ruby solution without cookbooks/recipes. I want to write classes and modules with Chef::Resource::File.new (etc.) calls, not `file “foo” do … end’. I think of the Chef DSL as one somewhat convenient but not very powerful interface to the underlying Chef Resource classes it fronts; one could use those classes independently of the DSL. E.g.

run_context = Chef::RunContext.new(Chef::Node.new, {}, Chef::EventDispatch::Dispatcher.new)

Chef::Resource::Directory.new(“/my_dir", run_context) do |res|
res.mode 0755
res.owner ‘root’
res.group ‘root’
res.run_action(:create)
end

Chef::Resource::File.new(“/my_dir/foo", run_context) do |res|
res.mode 0644
res.owner ‘root’
res.group ‘root’
res.run_action(:create)
res.content = “some content"
end

result = run_context.converge

Of course the above could live in classes/modules as is normal with Ruby code.

Hope this explains what I’m interested in a little better.

The DSL is used for convenience on declaring thought, though you could easily use code like you describe if you add run_context.resource_collection << in there. Otherwise Poise is pretty literally what you describe. See https://github.com/poise/poise-service/blob/master/lib/poise_service/resources/poise_service_user.rb for an example. That also uses Halite which is a thing I wrote to let me write all my code as standard gems and convert them to cookbooks on the fly. The creation and management of nested converge cycles is a bit more complex than you describe, and I’ve bundled it all inside helper APIs like notifying_block. That method creates a nested run based on the code inside it. Like I said, this is a very complex topic and I would happily not spam the whole list all day if you have specific questions. You can find me as “coderanger” on IRC or HangOps Slack.

–Noah


#5

right, and thats what makes chef very compelling. Those who wanted similar
use cases, have done it. Here are a few more examples[1], [2]. But the
recipe DSL layer not only makes thing concise, it also eases adopting,
implementing things with chef if you are new. For those who are coming form
cfengine or puppet, i think its easier than groking raw ruby. I also think
from a standard CM style execution (i.e building a set of resources and
then check & repair cycle) recipe makes a lot of sense, since that are the
authoritative sources for resource declaration. Also, this makes up what we
call cookbook, which is recipe+ artifacts required for recipe execution
that can be shared as independent. Chef community is built around that. The
cookbook route also allows segments (files, templates etc)… which are then
exploited by recipes/resources. In your example you have not used ohai
yet, for any non-trivial work, you’ll node ohai data as well… recipe/dsl
layer takes care of all those things.

i think, chef already have a decent design for raw consumption (i.e. use
chef from any ruby apps). We dont have the necessary docs or blogs around
it. Also a better understanding of Chef’s public API (so that version
compatibility can be maintained ). But let us know exactly what you are
looking for and we can tell you the core chef api.

i 'll also say that theres nothing inherent in the chef DSL layer that
restrict you from using raw ruby library modules etc… we can take an
example and explore that may be…

[1]https://github.com/ranjib/chef-stage
[2]https://github.com/ranjib/chef-lxc/blob/master/lib/chef/lxc_helper.rb

On Wed, Sep 16, 2015 at 3:44 PM, Jos Backus josb@mobileiron.com wrote:

Hi,

It would be nice if there was better support for the native Ruby Chef
classes/methods aside from the Chef::Recipe DSL with its fixed structure of
cookbooks/recipes, etc. It seems to me that with access to the run_context
class and the various resources, one could implement multiple
run/convergence queues easily enough (by instantiating different
run_contexts and operating on them), for example, and it would also open up
the ability to use traditional classes/modules for adding structure and
reusability, rather than the limited Chef::Recipe API. I understand that
the current DSL is concise, convenient for beginners and mimics what Puppet
uses, but it lacks structural power.

Thoughts?

Jos


#6

Hi Ranjib,

Noah helpfully pointed me to Halite which seems exactly what I am after.

You make good points about the DSL easing adoption and making the tool friendly for non-Rubyists. My objection to cookbooks is chiefly that we already have gems and classes/modules, and cookbooks and recipes are very limited compared to those other structures. In fact, when starting out with Chef a couple of months ago, I was very surprised to find that cookbooks could not nest arbitrarily.

I’d expect Ohai integration in a pure Ruby Chef environment not to be much of a problem - surely an API such as ohai.data[‘foo’] would be sufficient?

Your LXCHelper example is tantalizingly close and shows most of the plumbing I’d expect to be there, but instead of the instance_eval, there would just be a bunch of method calls to classes that push resources on some resource collection associated with a run_context, as Noah suggested.

I personally would appreciate more documentation of this use case of Chef, for people that are comfortable with Ruby and feel limited by the current cookbook/recipe paradigm (“What do you mean, recipes can’t take parameters?”).

Thanks for your feedback and suggestions!

Jos

From: Ranjib Dey
Reply-To: "chef@lists.opscode.commailto:chef@lists.opscode.com"
Date: Wednesday, September 16, 2015 at 4:30 PM
To: "chef@lists.opscode.commailto:chef@lists.opscode.com"
Subject: [chef] Re: Object-Oriented Chef

right, and thats what makes chef very compelling. Those who wanted similar use cases, have done it. Here are a few more examples[1], [2]. But the recipe DSL layer not only makes thing concise, it also eases adopting, implementing things with chef if you are new. For those who are coming form cfengine or puppet, i think its easier than groking raw ruby. I also think from a standard CM style execution (i.e building a set of resources and then check & repair cycle) recipe makes a lot of sense, since that are the authoritative sources for resource declaration. Also, this makes up what we call cookbook, which is recipe+ artifacts required for recipe execution that can be shared as independent. Chef community is built around that. The cookbook route also allows segments (files, templates etc)… which are then exploited by recipes/resources. In your example you have not used ohai yet, for any non-trivial work, you’ll node ohai data as well… recipe/dsl layer takes care of all those things.

i think, chef already have a decent design for raw consumption (i.e. use chef from any ruby apps). We dont have the necessary docs or blogs around it. Also a better understanding of Chef’s public API (so that version compatibility can be maintained ). But let us know exactly what you are looking for and we can tell you the core chef api.

i 'll also say that theres nothing inherent in the chef DSL layer that restrict you from using raw ruby library modules etc… we can take an example and explore that may be…

[1]https://github.com/ranjib/chef-stage
[2]https://github.com/ranjib/chef-lxc/blob/master/lib/chef/lxc_helper.rb

On Wed, Sep 16, 2015 at 3:44 PM, Jos Backus <josb@mobileiron.commailto:josb@mobileiron.com> wrote:
Hi,

It would be nice if there was better support for the native Ruby Chef classes/methods aside from the Chef::Recipe DSL with its fixed structure of cookbooks/recipes, etc. It seems to me that with access to the run_context class and the various resources, one could implement multiple run/convergence queues easily enough (by instantiating different run_contexts and operating on them), for example, and it would also open up the ability to use traditional classes/modules for adding structure and reusability, rather than the limited Chef::Recipe API. I understand that the current DSL is concise, convenient for beginners and mimics what Puppet uses, but it lacks structural power.

Thoughts?

Jos


#7

inline responses

On Wed, Sep 16, 2015 at 5:07 PM, Jos Backus josb@mobileiron.com wrote:

Hi Ranjib,

Noah helpfully pointed me to Halite which seems exactly what I am after.

awesome

You make good points about the DSL easing adoption and making the tool
friendly for non-Rubyists. My objection to cookbooks is chiefly that we
already have gems and classes/modules, and cookbooks and recipes are very
limited compared to those other structures.

yes. but as i said, cookbooks are different due to static contents &
segment (gems have only ruby files and executables)
In fact, when starting out with Chef a couple of months ago, I was very
surprised to find that cookbooks could not nest arbitrarily.

yeah, … there are several limitations… all they just dont have all the
joys of pure gems. use libraries to bridge.

I’d expect Ohai integration in a pure Ruby Chef environment not to be much
of a problem - surely an API such as ohai.data[‘foo’] would be sufficient?

its per easy, o = Ohai::System.new. o.run_plugins. chef injects this data
inside node object during startup, which appears as automatic attributes.

Your LXCHelper example is tantalizingly close and shows most of the
plumbing I’d expect to be there, but instead of the instance_eval, there
would just be a bunch of method calls to classes that push resources on
some resource collection associated with a run_context, as Noah suggested.

right. i prefer instance eval, for the same reason we instance_eval on
recipe.

I personally would appreciate more documentation of this use case of Chef,
for people that are comfortable with Ruby and feel limited by the current
cookbook/recipe paradigm (“What do you mean, recipes can’t take
parameters?”).

to me attributes are recipe parameters.
Im sorry you feel limited, lets see if we can make this betters.
I think of chef as a generic infrastructure library in ruby. knife being
the CLI front end, client being the implementation of CM pattern, chef
server being a metadata source. Now, unless you need CM, you dont need the
client bits, use the subsystem. In past we have two question on the same
line. First, what about a completely serializable DSL (i dont know if you
can call that dsl), like yaml or json. Second, what if we have an explicit
DSL, i.e. instead of instance_eval, we implement a class or something
similar. To me both of this two are two extremes. First one reduces
composability (u are bound by the platform), while second on allow some
composition but not with some quirks (certain things dont work :-)), which
albeit is annoying from pure ruby perspective. But remember theres nothing
in Chef that prevent you from doing what you want. i,e. you can you
cookbook syncer to grab things, and then build node, construct run list,
execute it.

i dont which one is best (serializable files, implicit DSL of current chef,
or explicit DSL, the one you are talking about). i’ll be very interested to
know others opinion. I see there are limited perf benefits, and a lot of
ease for pure ruby devs. But lot bigger learning curve for non-rubyist,
and also a lot more verbosity. Unless your resource list is lot smaller
than your all other logic (which is rarely the case, in my experience)
explicit dsl will be pretty verbose.

it will be nice if you specify an use case instead, and we compare how you
could have implemented it if explicit DSL was there, and then we share how
currently we can do this. we can compare its lot easier that way

Thanks for your feedback and suggestions!

Jos

From: Ranjib Dey
Reply-To: "chef@lists.opscode.com"
Date: Wednesday, September 16, 2015 at 4:30 PM
To: "chef@lists.opscode.com"
Subject: [chef] Re: Object-Oriented Chef

right, and thats what makes chef very compelling. Those who wanted similar
use cases, have done it. Here are a few more examples[1], [2]. But the
recipe DSL layer not only makes thing concise, it also eases adopting,
implementing things with chef if you are new. For those who are coming form
cfengine or puppet, i think its easier than groking raw ruby. I also think
from a standard CM style execution (i.e building a set of resources and
then check & repair cycle) recipe makes a lot of sense, since that are the
authoritative sources for resource declaration. Also, this makes up what we
call cookbook, which is recipe+ artifacts required for recipe execution
that can be shared as independent. Chef community is built around that. The
cookbook route also allows segments (files, templates etc)… which are then
exploited by recipes/resources. In your example you have not used ohai
yet, for any non-trivial work, you’ll node ohai data as well… recipe/dsl
layer takes care of all those things.

i think, chef already have a decent design for raw consumption (i.e. use
chef from any ruby apps). We dont have the necessary docs or blogs around
it. Also a better understanding of Chef’s public API (so that version
compatibility can be maintained ). But let us know exactly what you are
looking for and we can tell you the core chef api.

i 'll also say that theres nothing inherent in the chef DSL layer that
restrict you from using raw ruby library modules etc… we can take an
example and explore that may be…

[1]https://github.com/ranjib/chef-stage
[2]https://github.com/ranjib/chef-lxc/blob/master/lib/chef/lxc_helper.rb

On Wed, Sep 16, 2015 at 3:44 PM, Jos Backus josb@mobileiron.com wrote:

Hi,

It would be nice if there was better support for the native Ruby Chef
classes/methods aside from the Chef::Recipe DSL with its fixed structure of
cookbooks/recipes, etc. It seems to me that with access to the run_context
class and the various resources, one could implement multiple
run/convergence queues easily enough (by instantiating different
run_contexts and operating on them), for example, and it would also open up
the ability to use traditional classes/modules for adding structure and
reusability, rather than the limited Chef::Recipe API. I understand that
the current DSL is concise, convenient for beginners and mimics what Puppet
uses, but it lacks structural power.

Thoughts?

Jos


#8

In fact, when starting out with Chef a couple of months ago, I was very surprised to find that cookbooks could not nest arbitrarily.

Usually when I ear that, I’ve the reaction “You’re doing it wrong”.

If you need arbitrary nesting of cookbooks, than you’re not anymore
describing a state but the steps to get to this state.

If you use Chef to replace scripts and run it only on-demand I see why
you’re looking for this, but IMHO you’re using the wrong tool in this
case as Chef is aimed to be run periodically to enforce a state, so you
should avoid nested steps.

There’s some case where a “dual run” is needed as you can’t do the whole
state in one pass, in this case my usual way is to set a tag on node on
the first pass, skip what has to be skipped on the first pass when the
tag is absent, and use a execute resource to schedule the next run in 1
min or more, depending on the use case.

I do this to provision mysql clusters, I can’t tell who the replication
partner will be before it’s up, so I schedule a new chef run every 2
mins until the partner is found in search and the initial replication is
done.

This leave unused code in the long term run for those boxes, but it’s
better than looping within the run waiting the other box to get in
(which may never be true if your vagrant driver provision sequentially
as the vSphere plugin did before 1.0.2 for example)

This was my $.2 on the subject :slight_smile:


#9

inline

From: Ranjib Dey
Reply-To: "chef@lists.opscode.commailto:chef@lists.opscode.com"
Date: Wednesday, September 16, 2015 at 6:17 PM
To: "chef@lists.opscode.commailto:chef@lists.opscode.com"
Subject: [chef] Re: Re: Re: Object-Oriented Chef

inline responses

On Wed, Sep 16, 2015 at 5:07 PM, Jos Backus <josb@mobileiron.commailto:josb@mobileiron.com> wrote:
Hi Ranjib,

Noah helpfully pointed me to Halite which seems exactly what I am after.

awesome
You make good points about the DSL easing adoption and making the tool friendly for non-Rubyists. My objection to cookbooks is chiefly that we already have gems and classes/modules, and cookbooks and recipes are very limited compared to those other structures.
yes. but as i said, cookbooks are different due to static contents & segment (gems have only ruby files and executables)
In fact, when starting out with Chef a couple of months ago, I was very surprised to find that cookbooks could not nest arbitrarily.

yeah, … there are several limitations… all they just dont have all the joys of pure gems. use libraries to bridge.

[Jos] Sure, but cookbook-free Chef would be even better, no?

I’d expect Ohai integration in a pure Ruby Chef environment not to be much of a problem - surely an API such as ohai.data[‘foo’] would be sufficient?

its per easy, o = Ohai::System.new. o.run_plugins. chef injects this data inside node object during startup, which appears as automatic attributes.

[Jos] Yes, that’s how it works.

Your LXCHelper example is tantalizingly close and shows most of the plumbing I’d expect to be there, but instead of the instance_eval, there would just be a bunch of method calls to classes that push resources on some resource collection associated with a run_context, as Noah suggested.
right. i prefer instance eval, for the same reason we instance_eval on recipe.

[Jos] The downside of using instance_eval is that this

def meth
"bar"
end

ruby_block “foo” do
block do
puts “foo” + meth
end
end

doesn’t work. (Yes, I know how to work around this but that’s not the point.)

I personally would appreciate more documentation of this use case of Chef, for people that are comfortable with Ruby and feel limited by the current cookbook/recipe paradigm (“What do you mean, recipes can’t take parameters?”).

to me attributes are recipe parameters.

[Jos] It means that attributes become part of the method signature of each recipe, i.e. global state. Like this:

def foo
puts “Hi, #$attr”
end

$attr = “foo”

foo

$attr = “bar”

foo

It means that rather than being able to glance at the method signature to spot what a method can see, you have to look at all the attribute key paths as it’s one giant Hash (not even mentioning the complication of the various precedence levels, which personally I feel is a design defect, but I’m sure that’s not a popular opinion around here :slight_smile: ). So in my opinion attributes are a poor substitute for proper parameterization.

Im sorry you feel limited, lets see if we can make this betters.

[Jos] Great, thanks! I appreciate your efforts. I do think Chef has great potential if the focus remains on the resource collection and ways to generate it (cookbooks and the Recipe DSL being just one way).

I think of chef as a generic infrastructure library in ruby. knife being the CLI front end, client being the implementation of CM pattern, chef server being a metadata source. Now, unless you need CM, you dont need the client bits, use the subsystem. In past we have two question on the same line. First, what about a completely serializable DSL (i dont know if you can call that dsl), like yaml or json. Second, what if we have an explicit DSL, i.e. instead of instance_eval, we implement a class or something similar. To me both of this two are two extremes. First one reduces composability (u are bound by the platform), while second on allow some composition but not with some quirks (certain things dont work :-)), which albeit is annoying from pure ruby perspective. But remember theres nothing in Chef that prevent you from doing what you want. i,e. you can you cookbook syncer to grab things, and then build node, construct run list, execute it.

[Jos] Understood, with the caveat that I personally think that Chef would improve if it was less cookbook-centric. Maybe one of the reasons the current state of cookbooks is what it is because cookbooks with their global state and limited composability (include_recipe is about it, and it takes no parameters) make it easy to write poor code? Yes, you can write LWRPs (or even native Ruby) but there the issue is that the community has so many options that it’s not clear what the way forward is, and it leads to fragmentation and duplication of effort. I’d like to see the community converge (pun intended) on a single, well-supported easy to use LWRP solution at least. I don’t want to have to install the LWRP framework of the day just to be able to use some solution someone has built using it because it was cool.

(Sorry for the slightly off-topic tangent.)

i dont which one is best (serializable files, implicit DSL of current chef, or explicit DSL, the one you are talking about). i’ll be very interested to know others opinion. I see there are limited perf benefits, and a lot of ease for pure ruby devs. But lot bigger learning curve for non-rubyist, and also a lot more verbosity. Unless your resource list is lot smaller than your all other logic (which is rarely the case, in my experience) explicit dsl will be pretty verbose.

[Jos] Yes, it will be verbose, like Java perhaps :wink: But I think the tradeoff in terms of composition power is worth it. You could even still have a Cookbook and a Recipe class if you prefer that structure, but skip the DSL.

it will be nice if you specify an use case instead, and we compare how you could have implemented it if explicit DSL was there, and then we share how currently we can do this. we can compare its lot easier that way

[Jos] Well, the current example I’m struggling with is that I’m trying to install two Java versions at the same time with the Oracle JCE components. but the recipes in that cookbook use a bunch of global state so even though I monkey patched the DSL with include_recipe_again and I’m able to run the oracle_jce recipe twice, I have to figure out what node key paths it uses and set those before the include. This is a pain. Another issue I had is that in a recipe you can’t easily define methods and use them in resource blocks, like I showed above (they have to be defined on the class or instance of the resource instead). This is cumbersome as well. I believe to me as someone who knows Ruby this would be more straightforward if it was all just Ruby to begin with.

Thanks again for your insights and comments.

Jos

Thanks for your feedback and suggestions!

Jos

From: Ranjib Dey
Reply-To: "chef@lists.opscode.commailto:chef@lists.opscode.com"
Date: Wednesday, September 16, 2015 at 4:30 PM
To: "chef@lists.opscode.commailto:chef@lists.opscode.com"
Subject: [chef] Re: Object-Oriented Chef

right, and thats what makes chef very compelling. Those who wanted similar use cases, have done it. Here are a few more examples[1], [2]. But the recipe DSL layer not only makes thing concise, it also eases adopting, implementing things with chef if you are new. For those who are coming form cfengine or puppet, i think its easier than groking raw ruby. I also think from a standard CM style execution (i.e building a set of resources and then check & repair cycle) recipe makes a lot of sense, since that are the authoritative sources for resource declaration. Also, this makes up what we call cookbook, which is recipe+ artifacts required for recipe execution that can be shared as independent. Chef community is built around that. The cookbook route also allows segments (files, templates etc)… which are then exploited by recipes/resources. In your example you have not used ohai yet, for any non-trivial work, you’ll node ohai data as well… recipe/dsl layer takes care of all those things.

i think, chef already have a decent design for raw consumption (i.e. use chef from any ruby apps). We dont have the necessary docs or blogs around it. Also a better understanding of Chef’s public API (so that version compatibility can be maintained ). But let us know exactly what you are looking for and we can tell you the core chef api.

i 'll also say that theres nothing inherent in the chef DSL layer that restrict you from using raw ruby library modules etc… we can take an example and explore that may be…

[1]https://github.com/ranjib/chef-stage
[2]https://github.com/ranjib/chef-lxc/blob/master/lib/chef/lxc_helper.rb

On Wed, Sep 16, 2015 at 3:44 PM, Jos Backus <josb@mobileiron.commailto:josb@mobileiron.com> wrote:
Hi,

It would be nice if there was better support for the native Ruby Chef classes/methods aside from the Chef::Recipe DSL with its fixed structure of cookbooks/recipes, etc. It seems to me that with access to the run_context class and the various resources, one could implement multiple run/convergence queues easily enough (by instantiating different run_contexts and operating on them), for example, and it would also open up the ability to use traditional classes/modules for adding structure and reusability, rather than the limited Chef::Recipe API. I understand that the current DSL is concise, convenient for beginners and mimics what Puppet uses, but it lacks structural power.

Thoughts?

Jos


#10

In Ruby code it’s common to nest directories. E.g.

require 'chef/mixin/params_validate’

You can’t do the same with cookbooks, they have a fixed structure. My argument has nothing to do with resource convergence.

Thanks for your comment!

From: Tensibai
Reply-To: "chef@lists.opscode.commailto:chef@lists.opscode.com"
Date: Thursday, September 17, 2015 at 1:11 AM
To: "chef@lists.opscode.commailto:chef@lists.opscode.com"
Subject: [chef] Re: Re: Re: Object-Oriented Chef

In fact, when starting out with Chef a couple of months ago, I was very surprised to find that cookbooks could not nest arbitrarily.

Usually when I ear that, I’ve the reaction “You’re doing it wrong”.

If you need arbitrary nesting of cookbooks, than you’re not anymore describing a state but the steps to get to this state.

If you use Chef to replace scripts and run it only on-demand I see why you’re looking for this, but IMHO you’re using the wrong tool in this case as Chef is aimed to be run periodically to enforce a state, so you should avoid nested steps.

There’s some case where a “dual run” is needed as you can’t do the whole state in one pass, in this case my usual way is to set a tag on node on the first pass, skip what has to be skipped on the first pass when the tag is absent, and use a execute resource to schedule the next run in 1 min or more, depending on the use case.

I do this to provision mysql clusters, I can’t tell who the replication partner will be before it’s up, so I schedule a new chef run every 2 mins until the partner is found in search and the initial replication is done.

This leave unused code in the long term run for those boxes, but it’s better than looping within the run waiting the other box to get in (which may never be true if your vagrant driver provision sequentially as the vSphere plugin did before 1.0.2 for example)

This was my $.2 on the subject :slight_smile:


#11

Sorry, I misunderstood what you mean in this case (and still don’t see
exactly the use case for it, just organizing recipes ?)

Regarding your use case with two java version I assume the way to go
would be to move the recipe into a LWRP and call the LWRP with
corresponding attributes twice in a wrapper cookbook. (You may even do a
PR for this)

This sounds less likely to break in future than monkey patching the DSL
in my opinion.

In Ruby code it’s common to nest directories. E.g.

require ‘chef/mixin/params_validate’

You can’t do the same with cookbooks, they have a fixed structure. My argument has nothing to do with resource convergence.

Thanks for your comment!

From: Tensibai
Reply-To: "chef@lists.opscode.com"
Date: Thursday, September 17, 2015 at 1:11 AM
To: "chef@lists.opscode.com"
Subject: [chef] Re: Re: Re: Object-Oriented Chef

In fact, when starting out with Chef a couple of months ago, I was very surprised to find that cookbooks could not nest arbitrarily.

Usually when I ear that, I’ve the reaction “You’re doing it wrong”.

If you need arbitrary nesting of cookbooks, than you’re not anymore describing a state but the steps to get to this state.

If you use Chef to replace scripts and run it only on-demand I see why you’re looking for this, but IMHO you’re using the wrong tool in this case as Chef is aimed to be run periodically to enforce a state, so you should avoid nested steps.

There’s some case where a “dual run” is needed as you can’t do the whole state in one pass, in this case my usual way is to set a tag on node on the first pass, skip what has to be skipped on the first pass when the tag is absent, and use a execute resource to schedule the next run in 1 min or more, depending on the use case.

I do this to provision mysql clusters, I can’t tell who the replication partner will be before it’s up, so I schedule a new chef run every 2 mins until the partner is found in search and the initial replication is done.

This leave unused code in the long term run for those boxes, but it’s better than looping within the run waiting the other box to get in (which may never be true if your vagrant driver provision sequentially as the vSphere plugin did before 1.0.2 for example)

This was my $.2 on the subject :slight_smile:


#12

Correct, it’s an organization limitation. While I understand it’s on purpose, I still think its arbitrary - they are not really cookbooks, etc.

You are right about the java cookbook being poorly structured. And this is a recurring issue that I have with supermarket cookbooks. My guess is this is because the bar the DSL offers is so low. I agree that monkey-patching is generally undesirable.

Cheers,
Jos

-------- Original Message --------
From: Tensibai tensibai@iabis.net
Date: Fri, Sep 18, 2015, 1:07 AM
To: chef@lists.opscode.com
Subject: [chef] Re: Re: Re: Re: Re: Object-Oriented Chef

Sorry, I misunderstood what you mean in this case (and still don’t see exactly the use case for it, just organizing recipes ?)

Regarding your use case with two java version I assume the way to go would be to move the recipe into a LWRP and call the LWRP with corresponding attributes twice in a wrapper cookbook. (You may even do a PR for this)

This sounds less likely to break in future than monkey patching the DSL in my opinion.

In Ruby code it’s common to nest directories. E.g.

require ‘chef/mixin/params_validate’

You can’t do the same with cookbooks, they have a fixed structure. My argument has nothing to do with resource convergence.

Thanks for your comment!

From: Tensibai
Reply-To: "chef@lists.opscode.commailto:chef@lists.opscode.com"
Date: Thursday, September 17, 2015 at 1:11 AM
To: "chef@lists.opscode.commailto:chef@lists.opscode.com"
Subject: [chef] Re: Re: Re: Object-Oriented Chef

In fact, when starting out with Chef a couple of months ago, I was very surprised to find that cookbooks could not nest arbitrarily.

Usually when I ear that, I’ve the reaction “You’re doing it wrong”.

If you need arbitrary nesting of cookbooks, than you’re not anymore describing a state but the steps to get to this state.

If you use Chef to replace scripts and run it only on-demand I see why you’re looking for this, but IMHO you’re using the wrong tool in this case as Chef is aimed to be run periodically to enforce a state, so you should avoid nested steps.

There’s some case where a “dual run” is needed as you can’t do the whole state in one pass, in this case my usual way is to set a tag on node on the first pass, skip what has to be skipped on the first pass when the tag is absent, and use a execute resource to schedule the next run in 1 min or more, depending on the use case.

I do this to provision mysql clusters, I can’t tell who the replication partner will be before it’s up, so I schedule a new chef run every 2 mins until the partner is found in search and the initial replication is done.

This leave unused code in the long term run for those boxes, but it’s better than looping within the run waiting the other box to get in (which may never be true if your vagrant driver provision sequentially as the vSphere plugin did before 1.0.2 for example)

This was my $.2 on the subject :slight_smile: