Conditional dependencies

I’ve been searching the mailing lists and JIRA etc. for discussion related
to this issue. What I have found so far seems to cover a few different use
cases.

cookbook metadata should include maintainer_repository
http://tickets.opscode.com/browse/CHEF-286

The depends cookbook DSL should take an optional platform_family argument
http://tickets.opscode.com/browse/CHEF-3282

Implement cookbook metadata “suggests” and "recommends"
http://tickets.opscode.com/browse/CHEF-3270

Enforce / Implement platform “supports” in cookbook metadata
http://tickets.opscode.com/browse/CHEF-3871

The problem that we are having occurs when a mandatory cookbook’s
dependencies change based on attributes. As others have described, this can
cause issues of slowness or confusion when cookbooks unsupported on the
client system are required. As an example, I would like to develop a
cookbook that will require the MySQL or PostgreSQL cookbooks but can also
ignore this dependency with an unset db_type attribute.

I’m not convinced that the “suggests” and “recommends” qualifiers would
satisfy this use case. It seems more like the virtual package concept (
http://www.debian.org/doc/debian-policy/ch-relationships.html#s-virtual),
which the “provides” metadata support was presumably intended for.

I didn’t want to hack together something for our issue, but I’m happy to
work on this if a path forward can be agreed on. I’m on irc as well as
mattw.

thanks,

Matt Whiteley mwhiteley@engineyard.com

Ohai,

On Jun 6, 2013, at 9:46 AM, Matt Whiteley mwhiteley@engineyard.com wrote:

cookbook metadata should include maintainer_repository
http://tickets.opscode.com/browse/CHEF-286

The depends cookbook DSL should take an optional platform_family argument
http://tickets.opscode.com/browse/CHEF-3282

Implement cookbook metadata "suggests" and "recommends"
http://tickets.opscode.com/browse/CHEF-3270

Enforce / Implement platform "supports" in cookbook metadata
http://tickets.opscode.com/browse/CHEF-3871

Thanks for collecting these into a handy single list :slight_smile:

The problem that we are having occurs when a mandatory cookbook's dependencies change based on attributes. As others have described, this can cause issues of slowness or confusion when cookbooks unsupported on the client system are required. As an example, I would like to develop a cookbook that will require the MySQL or PostgreSQL cookbooks but can also ignore this dependency with an unset db_type attribute.

The thing about attributes is that it is very common to set behavior in a cookbook based on the value of an attribute, and to set the default value of said attributes based on the node's platform.

A particular cookbook might be specifically supported/tested on a certain platforms (like, "Debian" or "CentOS"). It might "just work" on other platforms without modification. It might require modifying attributes, or retrieving additional dependencies.

Generally many cookbooks "just work" because the [metadata] dependencies are set to cover the various use cases from the attributes. I think this is good, and it is the least surprising thing to most people.

However, it does mean you may end up with code you don't need on your nodes. For example, the "git" cookbook supports installing the 'git' package on a variety of platforms including Windows. This means, due to the lack of a "package" resource in Core Chef for Windows (yet[0]), we need the windows cookbook, which has windows_package. If we don't, then anyone who wants to just drop 'recipe[git]' on a Windows node will get an exception, find the need to get the windows cookbook, and wonder why Chef doesn't support Windows as a first class citizen[1].

Also, the git cookbook supports setting up a Git server. This brings in a dependency on runit, because we use a runit service to set up the daemon. This means that people who just want the git package so they can use the tool on their system may be confused why they have runit (they don't use runit, they use sysv/init, wtf!).

This goes on and on for various cookbooks, use cases, platforms. It's a fairly nuanced and difficult problem to solve. Someone will always be confused or irritated with this, because either:

  1. things don't work by default without more effect (and domain knowledge), or
  2. they have a bunch of code they didn't want/need being transferred about.

Comments in the tickets keeps the conversation in one place, but for folks who wish to contribute a fix for this, we need to ensure that we're consistent across the various codebases where this is affected (chef client, chef server, cookbook metadata).

Thank you for bringing this up!

Cheers,
Joshua

[0]: We're working on adding a package resource for Windows, so the windows cookbook is no longer required, but I don't have an ETA.
[1]: I don't feel this is true, but it feels that way when we have things like this :(.

I understand what you're saying about the Windows package issue and
personally it has never bothered me, I only included it since it seems
related to our issue and far more common in other people's comments.

I'm hoping to solve more of a dependency injection issue. A cookbook will
require one of a number of other cookbooks, but not all of them. Sure, we
could just disable the food critic rule checking depends statements, but it
seems like a problem better solved in a more comprehensive manner that
could help the other similar issues at the same time.

If a ticket is a better place for this, just let me know. I thought this
would get more discussion going so I could know what ticket to make
eventually.

On Thu, Jun 6, 2013 at 1:39 PM, Joshua Timberman joshua@opscode.com wrote:

Ohai,

On Jun 6, 2013, at 9:46 AM, Matt Whiteley mwhiteley@engineyard.com
wrote:

cookbook metadata should include maintainer_repository
http://tickets.opscode.com/browse/CHEF-286

The depends cookbook DSL should take an optional platform_family argument
http://tickets.opscode.com/browse/CHEF-3282

Implement cookbook metadata "suggests" and "recommends"
http://tickets.opscode.com/browse/CHEF-3270

Enforce / Implement platform "supports" in cookbook metadata
http://tickets.opscode.com/browse/CHEF-3871

Thanks for collecting these into a handy single list :slight_smile:

The problem that we are having occurs when a mandatory cookbook's
dependencies change based on attributes. As others have described, this can
cause issues of slowness or confusion when cookbooks unsupported on the
client system are required. As an example, I would like to develop a
cookbook that will require the MySQL or PostgreSQL cookbooks but can also
ignore this dependency with an unset db_type attribute.

The thing about attributes is that it is very common to set behavior in a
cookbook based on the value of an attribute, and to set the default value
of said attributes based on the node's platform.

A particular cookbook might be specifically supported/tested on a certain
platforms (like, "Debian" or "CentOS"). It might "just work" on other
platforms without modification. It might require modifying attributes, or
retrieving additional dependencies.

Generally many cookbooks "just work" because the [metadata] dependencies
are set to cover the various use cases from the attributes. I think this is
good, and it is the least surprising thing to most people.

However, it does mean you may end up with code you don't need on your
nodes. For example, the "git" cookbook supports installing the 'git'
package on a variety of platforms including Windows. This means, due to the
lack of a "package" resource in Core Chef for Windows (yet[0]), we need the
windows cookbook, which has windows_package. If we don't, then anyone who
wants to just drop 'recipe[git]' on a Windows node will get an exception,
find the need to get the windows cookbook, and wonder why Chef doesn't
support Windows as a first class citizen[1].

Also, the git cookbook supports setting up a Git server. This brings in a
dependency on runit, because we use a runit service to set up the daemon.
This means that people who just want the git package so they can use the
tool on their system may be confused why they have runit (they don't use
runit, they use sysv/init, wtf!).

This goes on and on for various cookbooks, use cases, platforms. It's a
fairly nuanced and difficult problem to solve. Someone will always be
confused or irritated with this, because either:

  1. things don't work by default without more effect (and domain
    knowledge), or
  2. they have a bunch of code they didn't want/need being transferred about.

Comments in the tickets keeps the conversation in one place, but for folks
who wish to contribute a fix for this, we need to ensure that we're
consistent across the various codebases where this is affected (chef
client, chef server, cookbook metadata).

Thank you for bringing this up!

Cheers,
Joshua

[0]: We're working on adding a package resource for Windows, so the
windows cookbook is no longer required, but I don't have an ETA.
[1]: I don't feel this is true, but it feels that way when we have things
like this :(.

--
Matt Whiteley mwhiteley@engineyard.com