CHEF-3762: LWRP DSL for subclassing needed?


#1

http://tickets.opscode.com/browse/CHEF-3762

There’s been some talk about adding a DSL for subclassing LWRPs. I’ve heard both sentiments that it would be useful and questions if it makes something justifiably easier.

Can those promoting it make a case? Can anyone else soap box about their own experience?


Bryan McLellan | opscode | technical program manager, open source
© 206.607.7108 | (t) @btmspox | (b) http://blog.loftninjas.org


#2

On Jan 28, 2013, at 2:45 PM, Bryan McLellan wrote:

http://tickets.opscode.com/browse/CHEF-3762

There’s been some talk about adding a DSL for subclassing LWRPs. I’ve heard both sentiments that it would be useful and questions if it makes something justifiably easier.

Can those promoting it make a case? Can anyone else soap box about their own experience?

I’ve been pushing hard for this for a long time now. As some examples of places this would be directly useful, I’ve spent part of today reviewing a new Mercural resource/provider which has to subclass Chef::Resource::Scm and as such can’t use the simpler LWRP syntax. Another example is any of the multitude of service resources which currently either duplicate code or use the “heavy” syntax. Beyond just subclassing built-in pieces, the application cookbook would gain a lot by allowing subclassing of individual sub-resources in places where things were mostly like a standard deployment but just need a little tweak, like (to use Django examples because that is what I’m familiar with) a requirements.txt in a non-standard place, or setting up the database connection a little differently. This would allow much more fine-grained overriding than is possible right now within the bounds of the LWRP system, and hopefully would reduce the number of times people copy-paste-fork parts of upstream cookbooks. In my bright and glorious LWRP-ified future this will only get more important over time, as I hope more cookbooks use LWRPs to provide internal modularity like this, just as you would see in any other OO API.

–Noah


#3

On Monday, January 28, 2013 at 3:06 PM, Noah Kantrowitz wrote:

On Jan 28, 2013, at 2:45 PM, Bryan McLellan wrote:

http://tickets.opscode.com/browse/CHEF-3762

There’s been some talk about adding a DSL for subclassing LWRPs. I’ve heard both sentiments that it would be useful and questions if it makes something justifiably easier.

Can those promoting it make a case? Can anyone else soap box about their own experience?

I’ve been pushing hard for this for a long time now. As some examples of places this would be directly useful, I’ve spent part of today reviewing a new Mercural resource/provider which has to subclass Chef::Resource::Scm and as such can’t use the simpler LWRP syntax. Another example is any of the multitude of service resources which currently either duplicate code or use the “heavy” syntax. Beyond just subclassing built-in pieces, the application cookbook would gain a lot by allowing subclassing of individual sub-resources in places where things were mostly like a standard deployment but just need a little tweak, like (to use Django examples because that is what I’m familiar with) a requirements.txt in a non-standard place, or setting up the database connection a little differently. This would allow much more fine-grained overriding than is possible right now within the bounds of the LWRP system, and hopefully would reduce the number of times people copy-paste-fork parts of upstrea
m cookbooks. In my bright and glorious LWRP-ified future this will only get more important over time, as I hope more cookbooks use LWRPs to provide internal modularity like this, just as you would see in any other OO API.

–Noah
Once you know about subclassing and so on, is it really that onerous to use the “heavy” syntax?

class FooProvider < Chef::Provider::Scm
end

…doesn’t seem like a ton of work.


Daniel DeLeo


#4

On Jan 28, 2013, at 3:11 PM, Daniel DeLeo wrote:

On Monday, January 28, 2013 at 3:06 PM, Noah Kantrowitz wrote:

On Jan 28, 2013, at 2:45 PM, Bryan McLellan wrote:

http://tickets.opscode.com/browse/CHEF-3762

There’s been some talk about adding a DSL for subclassing LWRPs. I’ve heard both sentiments that it would be useful and questions if it makes something justifiably easier.

Can those promoting it make a case? Can anyone else soap box about their own experience?

I’ve been pushing hard for this for a long time now. As some examples of places this would be directly useful, I’ve spent part of today reviewing a new Mercural resource/provider which has to subclass Chef::Resource::Scm and as such can’t use the simpler LWRP syntax. Another example is any of the multitude of service resources which currently either duplicate code or use the “heavy” syntax. Beyond just subclassing built-in pieces, the application cookbook would gain a lot by allowing subclassing of individual sub-resources in places where things were mostly like a standard deployment but just need a little tweak, like (to use Django examples because that is what I’m familiar with) a requirements.txt in a non-standard place, or setting up the database connection a little differently. This would allow much more fine-grained overriding than is possible right now within the bounds of the LWRP system, and hopefully would reduce the number of times people copy-paste-fork parts of upstream cookbooks. In my bright and glorious LWRP-ified future this will only get more important over time, as I hope more cookbooks use LWRPs to provide internal modularity like this, just as you would see in any other OO API.

–Noah
Once you know about subclassing and so on, is it really that onerous to use the “heavy” syntax?

class FooProvider < Chef::Provider::Scm
end

…doesn’t seem like a ton of work.

Helpers like #attribute and #action don’t work in non-LWRP files as far as I know. More to the point it confuses people seeing “heavy” resource and providers living under libraries/, so I think we should try to minimize the need for them. Additionally is there any negative to this change? It makes LWRPs more flexible without adding complexity unless you specifically request it as an author. Those that don’t want to subclass things can just ignore the new functionality entirely.

–Noah


#5

Yo,

On 29 January 2013 11:45, Bryan McLellan btm@opscode.com wrote:

http://tickets.opscode.com/browse/CHEF-3762

There’s been some talk about adding a DSL for subclassing LWRPs. I’ve heard both sentiments that it would be useful and questions if it makes something justifiably easier.

Can those promoting it make a case? Can anyone else soap box about their own experience?

There are definitely a few cases where I’ve used a heavy-weight
resource in a Library file instead of being able to subclass one of
the existing (core Chef) resources.

If it was possible to declare a lwrp provider/resource as a subclass
of an existing heavy-or-lwrp I think this would definitely not be
worth adding a DSL for…? Does the snippet that Dan posted below
actually work in an LWRP provider file?

–AJ


Bryan McLellan | opscode | technical program manager, open source
© 206.607.7108 | (t) @btmspox | (b) http://blog.loftninjas.org


#6

One option for the helpers is to move the helpers into a mixin which is
included by in the lightweight base classes, and which may be included
explicitly in any non-lightweight resource or provider class.

What confuses many people, from what I’ve seen in IRC and elsewhere, is
that they’re not sure how the lightweight resource and provider DSL’s are a
very thin wrapper around just writing yourself a resource or provider
class. It’s not terribly useful to re-implement Ruby inside of a Ruby-based
embedded DSL just for the kicks. Better that it be explicitly clear how the
lightweight DSLs are a thin wrapper around writing your own resource and
provider classes, and that people learn the structural relationship between
the two superficial alternatives.

  • Jay

On Mon, Jan 28, 2013 at 6:15 PM, Noah Kantrowitz noah@coderanger.netwrote:

On Jan 28, 2013, at 3:11 PM, Daniel DeLeo wrote:

On Monday, January 28, 2013 at 3:06 PM, Noah Kantrowitz wrote:

On Jan 28, 2013, at 2:45 PM, Bryan McLellan wrote:

http://tickets.opscode.com/browse/CHEF-3762

There’s been some talk about adding a DSL for subclassing LWRPs. I’ve
heard both sentiments that it would be useful and questions if it makes
something justifiably easier.

Can those promoting it make a case? Can anyone else soap box about
their own experience?

I’ve been pushing hard for this for a long time now. As some examples
of places this would be directly useful, I’ve spent part of today reviewing
a new Mercural resource/provider which has to subclass Chef::Resource::Scm
and as such can’t use the simpler LWRP syntax. Another example is any of
the multitude of service resources which currently either duplicate code or
use the “heavy” syntax. Beyond just subclassing built-in pieces, the
application cookbook would gain a lot by allowing subclassing of individual
sub-resources in places where things were mostly like a standard deployment
but just need a little tweak, like (to use Django examples because that is
what I’m familiar with) a requirements.txt in a non-standard place, or
setting up the database connection a little differently. This would allow
much more fine-grained overriding than is possible right now within the
bounds of the LWRP system, and hopefully would reduce the number of times
people copy-paste-fork parts of upstream cookbooks. In my bright and
glorious LWRP-ified future this will only get more important over time, as
I hope more cookbooks use LWRPs to provide internal modularity like this,
just as you would see in any other OO API.

–Noah
Once you know about subclassing and so on, is it really that onerous to
use the “heavy” syntax?

class FooProvider < Chef::Provider::Scm
end

…doesn’t seem like a ton of work.

Helpers like #attribute and #action don’t work in non-LWRP files as far as
I know. More to the point it confuses people seeing “heavy” resource and
providers living under libraries/, so I think we should try to minimize the
need for them. Additionally is there any negative to this change? It makes
LWRPs more flexible without adding complexity unless you specifically
request it as an author. Those that don’t want to subclass things can just
ignore the new functionality entirely.

–Noah


#7

On Jan 28, 2013, at 3:45 PM, Jay Feldblum wrote:

One option for the helpers is to move the helpers into a mixin which is included by in the lightweight base classes, and which may be included explicitly in any non-lightweight resource or provider class.

This is exactly the approach the application cookbook has used, but it gets very verbose and ungainly, and leads to related code being all over the place. It also doesn’t address built-in classes in Chef unless those are moved out into modules as well.

–Noah


#8

I mean the Chef project moving the helpers into a built-in mixin.

If Chef::Resource::LWRPBase had a simple implementation:

class Chef::Resource::LWRPBase < Chef::Resource
  # the stuff related to load-from-file

  include Chef::Resource::LightweightMixin
end

With all of the complexity being moved into the module
Chef::Resource::LightweightMixin, then anything else could include these
methods as well, including subclasses of existing built-in resources.

Likewise, providers.

Cheers,
Jay

On Mon, Jan 28, 2013 at 6:48 PM, Noah Kantrowitz noah@coderanger.netwrote:

On Jan 28, 2013, at 3:45 PM, Jay Feldblum wrote:

One option for the helpers is to move the helpers into a mixin which is
included by in the lightweight base classes, and which may be included
explicitly in any non-lightweight resource or provider class.

This is exactly the approach the application cookbook has used, but it
gets very verbose and ungainly, and leads to related code being all over
the place. It also doesn’t address built-in classes in Chef unless those
are moved out into modules as well.

–Noah