Use of require in chef


#1

Ohai Chefs,

I would like to rewrite the way chef loads itself, but before I begin, I
would like everyone who’s interested to have an opportunity to offer their
opinions on some of the sticky issues surrounding this.

First, some background: currently, chef uses require() throughout the source
tree, with each file require()-ing the files that it needs to work. As one
example, cookbook_loader.rb has the following at the top, just below the
license:

require ‘chef/config’

require ‘chef/cookbook’

require ‘chef/cookbook/metadata’

While this approach has the advantage of communicating file dependencies
with the dependent code, I think it needs to be rewritten because many of
the require statements overlap and ruby’s exact behavior for overlapping
require statements varies from version to version. Most notably, the code as
it is now causes a NameError for uninitialized constants in ruby 1.9 and,
less importantly, in some 1.8.x rubies before 1.8.5. In addition to
compatibility issues, the current file load process causes some files to be
loaded multiple times, leading to ruby warnings and kludgy unless defined?(Const) workarounds.

Before making any changes, I’d love to hear from everyone about the
following issues:

  • Relative load paths vs. Absolute
    There is an open ticket, CHEF-557 [1], which states that relative load paths
    are evil because other libraries with identical relative paths may take
    precedence over a file in chef and be loaded instead. For example,
    “not_chef/lib/chef/node.rb” could be loaded instead of chef’s node.rb. I
    don’t foresee this particular issue being common, but this approach does
    have the benefit of making multiple loads of the same file basically
    impossible and prevents all load path related problems (by ignoring the load
    path entirely). The code would look like:

CHEF_LIB_ROOT = File.expand_path(File.dirname(FILE))
require CHEF_LIB_ROOT + “/chef/node.rb”

Alternatively, some argue for the relative approach, with the same “you’re
doing it wrong” zeal. One example is posted on the Riding Rails blog [2].
The argument here is that the package manager, usually rubygems, will take
care of the load path, so stop worrying about it.

Personally, I lean a little towards the relative approach, but I’ve used
both in my personal code. The absolute paths are certainly more verbose but
less error prone. The inherent verbosity of this approach could be reduced
by defining a method to encapsulate the require and absolute path, such as
Chef.requires "chef/node" or, defined on Object, `chef_requires
"chef/node"

Some of you I’ve talked to already have strong opinions about this; I’ll
happily do either.

  • Namespace issues
    First of all, I’d like for Chef-the-namespace to be a module. This is much
    more flexible than as a class. For example, spec_helper.rb for the
    client/solo (the chef/chef directory) could have include Chef and
    eliminate the need for to prefix every constant with Chef:: (i.e., specs
    could use Node instead of Chef::Node).

I also prefer the pattern for classes such as Resource and Provider to have
a “Base” class that subclasses inherit from. For example, I’d prefer to have

module Chef

module Resource

class Base

  # move everything in Chef::Resource here

end

end

end

module Chef

module Resource

class Package < Chef::Resource::Base

  # ...

end

end

end

The advantage of this approach is that Chef::Resource is just a namespace
module, so chef/resource.rb can require chef/resource/base and then require
chef/resource/* and there won’t be any errors from undefined constants. This
keeps file loads grouped together by function, and they don’t need to be
"snuck in" at the bottom of the file.

  • Don’t require rubygems
    Finally, it should be possible to move any require “rubygems” into the
    executables and out of the lib/ dir. This would put chef in line with
    accepted practices and hopefully make packaging a bit easier.

Looking forward to hearing your opinions on this.

Daniel DeLeo

  1. http://tickets.opscode.com/browse/CHEF-557
  2. http://weblog.rubyonrails.org/2009/9/1/gem-packaging-best-practices

#2

On Fri, Nov 13, 2009 at 10:12 AM, Daniel DeLeo dan@kallistec.com wrote:

  • Don’t require rubygems
    Finally, it should be possible to move any require “rubygems” into the
    executables and out of the lib/ dir. This would put chef in line with
    accepted practices and hopefully make packaging a bit easier.

I’m still of the opinion that the require for rubygems shouldn’t be in
the chef/ohai source, and should either be in the distributions gem
wrapper binaries or in the distributions packaging if required. For
the chatter about why, see the comments in the slew of sticks. Ya’ll
are, of course, encouraged to convince me otherwise.

http://tickets.opscode.com/browse/OHAI-140
http://tickets.opscode.com/browse/OHAI-119
http://tickets.opscode.com/browse/CHEF-669
http://tickets.opscode.com/browse/CHEF-531


#3

If you have my vote, I vote with Bryan McLellan here.

I would rather avoid the rubygems dependency completely if possible, I know it raises some hackles and makes people worried. But I think most of us have seen how much fun rubygems can be, and some of us remember minigems fondly…

On Nov 13, 2009, at 10:26 AM, Bryan McLellan wrote:

On Fri, Nov 13, 2009 at 10:12 AM, Daniel DeLeo dan@kallistec.com wrote:

  • Don’t require rubygems
    Finally, it should be possible to move any require “rubygems” into the
    executables and out of the lib/ dir. This would put chef in line with
    accepted practices and hopefully make packaging a bit easier.

I’m still of the opinion that the require for rubygems shouldn’t be in
the chef/ohai source, and should either be in the distributions gem
wrapper binaries or in the distributions packaging if required. For
the chatter about why, see the comments in the slew of sticks. Ya’ll
are, of course, encouraged to convince me otherwise.

http://tickets.opscode.com/browse/OHAI-140
http://tickets.opscode.com/browse/OHAI-119
http://tickets.opscode.com/browse/CHEF-669
http://tickets.opscode.com/browse/CHEF-531

!DSPAM:4afda4f724111804284693!


#4

While we’re attempting to figure this stuff out once and for all, this is a
good issue to bring up.

As rubygems will modify the executables on install, the only downside to
banishing require "rubygems" everywhere is that you’d have to run, for
example, ruby -rubygems bin/chef-client to run the code out of a git
clone. This can easily be tucked away from view inside the rake tasks for
running a development instance or integration tests, so the impact would be
minimal.

So, I’ll +1 on this; I doubt anyone has a compelling reason that they need
to run a git clone of chef and can’trake install the gems, but if that’s
you, feel free to comment.

Dan Deleo

On Fri, Nov 13, 2009 at 11:42 AM, Scott M. Likens scott@likens.us wrote:

If you have my vote, I vote with Bryan McLellan here.

I would rather avoid the rubygems dependency completely if possible, I know
it raises some hackles and makes people worried. But I think most of us
have seen how much fun rubygems can be, and some of us remember minigems
fondly…

On Nov 13, 2009, at 10:26 AM, Bryan McLellan wrote:

On Fri, Nov 13, 2009 at 10:12 AM, Daniel DeLeo dan@kallistec.com
wrote:

  • Don’t require rubygems
    Finally, it should be possible to move any require “rubygems” into the
    executables and out of the lib/ dir. This would put chef in line with
    accepted practices and hopefully make packaging a bit easier.

I’m still of the opinion that the require for rubygems shouldn’t be in
the chef/ohai source, and should either be in the distributions gem
wrapper binaries or in the distributions packaging if required. For
the chatter about why, see the comments in the slew of sticks. Ya’ll
are, of course, encouraged to convince me otherwise.

http://tickets.opscode.com/browse/OHAI-140
http://tickets.opscode.com/browse/OHAI-119
http://tickets.opscode.com/browse/CHEF-669
http://tickets.opscode.com/browse/CHEF-531

!DSPAM:4afda4f724111804284693!


#5

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

There is also the RUBYOPT environment variable. Not saying it’s the
right approach, but I wanted to bring it up for completeness.

  • -C

Daniel DeLeo wrote:

While we’re attempting to figure this stuff out once and for all,
this is a good issue to bring up.

As rubygems will modify the executables on install, the only
downside to banishing require "rubygems" everywhere is that you’d
have to run, for example, ruby -rubygems bin/chef-client to run
the code out of a git clone. This can easily be tucked away from
view inside the rake tasks for running a development instance or
integration tests, so the impact would be minimal.

So, I’ll +1 on this; I doubt anyone has a compelling reason that
they need to run a git clone of chef and can’trake install the
gems, but if that’s you, feel free to comment.

Dan Deleo

On Fri, Nov 13, 2009 at 11:42 AM, Scott M. Likens scott@likens.us
wrote:

If you have my vote, I vote with Bryan McLellan here.

I would rather avoid the rubygems dependency completely if
possible, I know it raises some hackles and makes people worried.
But I think most of us have seen how much fun rubygems can be,
and some of us remember minigems fondly…

On Nov 13, 2009, at 10:26 AM, Bryan McLellan wrote:

On Fri, Nov 13, 2009 at 10:12 AM, Daniel DeLeo
dan@kallistec.com
wrote:

  • Don’t require rubygems Finally, it should be possible to
    move any require “rubygems” into the executables and out of
    the lib/ dir. This would put chef in line with accepted
    practices and hopefully make packaging a bit easier.
    I’m still of the opinion that the require for rubygems
    shouldn’t be in the chef/ohai source, and should either be in
    the distributions gem wrapper binaries or in the distributions
    packaging if required. For the chatter about why, see the
    comments in the slew of sticks. Ya’ll are, of course,
    encouraged to convince me otherwise.

http://tickets.opscode.com/browse/OHAI-140
http://tickets.opscode.com/browse/OHAI-119
http://tickets.opscode.com/browse/CHEF-669
http://tickets.opscode.com/browse/CHEF-531

!DSPAM:4afda4f724111804284693!

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.12 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iQEcBAEBAgAGBQJK/bc/AAoJEKv60eeL/nAXTUoH/19qxU675PFI9T880V2f5zdk
DJeg9ztrhTzxoTrEQJwLtPmokWz1vg/I3HxnRSrTM3hjLBlkHmSL05VMzJ1FCRqV
PZj19cs6qO4+oTtLYXQz1FytG5Iul/nxz6RgBiQxmmtxwamPCD9WpZKyl+nhhSSl
I/NxHZVA29DQ4iZXPV8AiqvLm+tWDRlszq9KzholWNETvo40e+w7vVp38xLHZhAn
L4GzpeLlxKgGbTDXm7h4YA00xBl3C9yQSodAAdS1hhSVP9fVxBwqcdVFGl6MqyWm
lZdYrC2cpVJ759RMMWywnv56Rfu3mDNaD7+/Wdmiq1M0uux7TrDPRxTNZ/xs7zs=
=AP1s
-----END PGP SIGNATURE-----


#6

On Fri, Nov 13, 2009 at 11:23 AM, Daniel DeLeo dan@kallistec.com wrote:

As rubygems will modify the executables on install, the only downside to
banishing require "rubygems" everywhere is that you’d have to run, for
example, ruby -rubygems bin/chef-client to run the code out of a git
clone. This can easily be tucked away from view inside the rake tasks for
running a development instance or integration tests, so the impact would be
minimal.

I just want to point out that this shouldn’t be a requirement if you
have all of the required libraries in your ruby $LOAD_PATH. If you
have mixlib-* and everything else required installed via your
distributions packaging into the $LOAD_PATH, life is fine. ‘ruby
-rubygems’ is a shortcut that gets you the rubygems custom_require
that replaces Kernel#require and activates a specific version of a gem
then requires it from the gem path. Granted, during development you
often have versions of libraries that are not yet packaged by your
distribution, and it is standard practice to install those as gem
packages.