Ownership Changes of Chef Cookbooks Revisited


#1

A little over six months have gone by since many of the cookbooks were
handed off to various community maintainers. Now seems like a good time
to continue this discussion with wisdom at hand; I am sure many others,
like myself may have questions and surely have ideas on how to improve.

Many of the cookbook maintainers have done an outstanding job of working
with the community, resolving issues, and accepting and mentoring
changes. Unfortunately, there are a few which have not met the quality
standards which preceded their maintainer-ship. Having said this, I
would like to discuss the Chef governance policy with regards to
maintainers whom have been unable to fulfill their duties, and the
process of ensuring these cookbooks continue to have high quality going
forward.

Some questions I have are:

  • What is the current governance of cookbooks which were previously
    maintained by Chef (Opscode)?

  • How can the community get involved in this governance process to
    ensure high quality standards persist for said cookbooks?

Please do not hesitate to share your questions or thoughts on the
subject of the cookbooks which were handed off. As mentioned, I am sure
there are many others with related questions and ideas on the topic.

-Ryan H.
This electronic message contains information which may be confidential or privileged. The information is intended for the use of the individual or entity named above. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of the contents of this information is prohibited. If you have received this electronic transmission in error, please notify us by e-mail at (postmaster@rapid7.com) immediately.


#2

Hi,

Please note I don’t work for Opscode (Getchef) so this is just my view.

On 5/23/14 1:58 PM, Ryan Hass wrote:

A little over six months have gone by since many of the cookbooks were
handed off to various community maintainers. Now seems like a good time
to continue this discussion with wisdom at hand; I am sure many others,
like myself may have questions and surely have ideas on how to improve.

Many of the cookbook maintainers have done an outstanding job of working
with the community, resolving issues, and accepting and mentoring
changes. Unfortunately, there are a few which have not met the quality
standards which preceded their maintainer-ship. Having said this, I
would like to discuss the Chef governance policy with regards to
maintainers whom have been unable to fulfill their duties, and the
process of ensuring these cookbooks continue to have high quality going
forward.

Some questions I have are:

  • What is the current governance of cookbooks which were previously
    maintained by Chef (Opscode)?

  • How can the community get involved in this governance process to
    ensure high quality standards persist for said cookbooks?
    Issues and Pull requests are you best friend here. Taking back a
    cookbook is likely not easy; it would be easiest to fork it and point
    everyone to the new fork? There likely should be a policy here or
    otherwise we’ll step on peoples feelings.

(squishy!)

Please do not hesitate to share your questions or thoughts on the
subject of the cookbooks which were handed off. As mentioned, I am sure
there are many others with related questions and ideas on the topic.

-Ryan H.
This electronic message contains information which may be confidential or privileged. The information is intended for the use of the individual or entity named above. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of the contents of this information is prohibited. If you have received this electronic transmission in error, please notify us by e-mail at (postmaster@rapid7.com) immediately.

!DSPAM:537fbdd4209001804284693!


#3

Just spit balling here…

Perhaps this sort of thing could be taken into account in features of the
"new community site". I’m thinking of the rating system - there was some
discussion at the community summit of having multi-dimensional ratings,
like maturity, quality, features, stability, etc. I think maintainer
commitment could be a category, perhaps.

Alternatively, one could imagine metrics being displayed on the community
site - something like average time to respond to an issue, issues open
longer than X days, etc.

Or: a button to flag the cookbook as needing a new maintainer. That would
need moderation, I imagine.

On May 24, 2014 3:44 PM, “Scott M. Likens” scott@likens.us wrote:

Hi,

Please note I don’t work for Opscode (Getchef) so this is just my view.

On 5/23/14 1:58 PM, Ryan Hass wrote:

A little over six months have gone by since many of the cookbooks were
handed off to various community maintainers. Now seems like a good time
to continue this discussion with wisdom at hand; I am sure many others,
like myself may have questions and surely have ideas on how to improve.

Many of the cookbook maintainers have done an outstanding job of working
with the community, resolving issues, and accepting and mentoring
changes. Unfortunately, there are a few which have not met the quality
standards which preceded their maintainer-ship. Having said this, I
would like to discuss the Chef governance policy with regards to
maintainers whom have been unable to fulfill their duties, and the
process of ensuring these cookbooks continue to have high quality going
forward.

Some questions I have are:

  • What is the current governance of cookbooks which were previously
    maintained by Chef (Opscode)?

  • How can the community get involved in this governance process to
    ensure high quality standards persist for said cookbooks?

Issues and Pull requests are you best friend here. Taking back a cookbook
is likely not easy; it would be easiest to fork it and point everyone to
the new fork? There likely should be a policy here or otherwise we’ll step
on peoples feelings.

(squishy!)

Please do not hesitate to share your questions or thoughts on the
subject of the cookbooks which were handed off. As mentioned, I am sure
there are many others with related questions and ideas on the topic.

-Ryan H.
This electronic message contains information which may be confidential or
privileged. The information is intended for the use of the individual or
entity named above. If you are not the intended recipient, be aware that
any disclosure, copying, distribution or use of the contents of this
information is prohibited. If you have received this electronic
transmission in error, please notify us by e-mail at (
postmaster@rapid7.com) immediately.

!DSPAM:537fbdd4209001804284693!


#4

My concerns come from some of the cookbooks which have a large number of
outstanding pull requests as well as open issues. Often they have not
actually seen pull rquests merged, commented on, or otherwise closed.

Forking is always an option, just not the best one in my opinion.
Moreover, with this approach the issues remain – without a common well
maintained upstream cookbook, amongst other issues, it becomes very
difficult to find “the right cookbook.”

On 05/24/2014 12:44 PM, Scott M. Likens wrote:

  • How can the community get involved in this governance process to
    ensure high quality standards persist for said cookbooks?
    Issues and Pull requests are you best friend here. Taking back a
    cookbook is likely not easy; it would be easiest to fork it and point
    everyone to the new fork? There likely should be a policy here or
    otherwise we’ll step on peoples feelings.

(squishy!)

This electronic message contains information which may be confidential or privileged. The information is intended for the use of the individual or entity named above. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of the contents of this information is prohibited. If you have received this electronic transmission in error, please notify us by e-mail at (postmaster@rapid7.com) immediately.


#5

Can anyone at Chef disclose estimated time-lines and planned features
for the “new community site?” I think it is a good long term solution.

Nevertheless, If we are a long way out from launch of this new site it
makes sense to discuss a policy and governance approach.

On 05/24/2014 12:58 PM, Clinton Wolfe wrote:

Just spit balling here…

Perhaps this sort of thing could be taken into account in features of the
"new community site". I’m thinking of the rating system - there was some
discussion at the community summit of having multi-dimensional ratings,
like maturity, quality, features, stability, etc. I think maintainer
commitment could be a category, perhaps.

Alternatively, one could imagine metrics being displayed on the community
site - something like average time to respond to an issue, issues open
longer than X days, etc.

Or: a button to flag the cookbook as needing a new maintainer. That would
need moderation, I imagine.

On May 24, 2014 3:44 PM, “Scott M. Likens” scott@likens.us wrote:

Hi,

Please note I don’t work for Opscode (Getchef) so this is just my view.

On 5/23/14 1:58 PM, Ryan Hass wrote:

A little over six months have gone by since many of the cookbooks were
handed off to various community maintainers. Now seems like a good time
to continue this discussion with wisdom at hand; I am sure many others,
like myself may have questions and surely have ideas on how to improve.

Many of the cookbook maintainers have done an outstanding job of working
with the community, resolving issues, and accepting and mentoring
changes. Unfortunately, there are a few which have not met the quality
standards which preceded their maintainer-ship. Having said this, I
would like to discuss the Chef governance policy with regards to
maintainers whom have been unable to fulfill their duties, and the
process of ensuring these cookbooks continue to have high quality going
forward.

Some questions I have are:

  • What is the current governance of cookbooks which were previously
    maintained by Chef (Opscode)?

  • How can the community get involved in this governance process to
    ensure high quality standards persist for said cookbooks?

Issues and Pull requests are you best friend here. Taking back a cookbook
is likely not easy; it would be easiest to fork it and point everyone to
the new fork? There likely should be a policy here or otherwise we’ll step
on peoples feelings.

(squishy!)

Please do not hesitate to share your questions or thoughts on the
subject of the cookbooks which were handed off. As mentioned, I am sure
there are many others with related questions and ideas on the topic.

-Ryan H.
This electronic message contains information which may be confidential or
privileged. The information is intended for the use of the individual or
entity named above. If you are not the intended recipient, be aware that
any disclosure, copying, distribution or use of the contents of this
information is prohibited. If you have received this electronic
transmission in error, please notify us by e-mail at (
postmaster@rapid7.com) immediately.

!DSPAM:537fbdd4209001804284693!

This electronic message contains information which may be confidential or privileged. The information is intended for the use of the individual or entity named above. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of the contents of this information is prohibited. If you have received this electronic transmission in error, please notify us by e-mail at (postmaster@rapid7.com) immediately.


#6

I can speak to that…

So the hope is that we will see a “soft opening” of the new community site, aka supermarket in the next few weeks. With that said, http://lists.opscode.com/sympa/arc/chef/2014-05/msg00279.html has a pretty good summary of the state of things.

As far as adding things like the metrics below or the button to flag a cookbook as needing a new maintainer you can submit a Pull Request to https://github.com/opscode/supermarket.

We are actively looking for contributions and would love to see the community hack on what they see as important.

With that said, please reach out via IRC, Gitter, this list, email or whatever if you have any specific questions.

Thanks!

— cwebber

On May 27, 2014, at 5:10 PM, Ryan Hass ryan_hass@rapid7.com wrote:

Can anyone at Chef disclose estimated time-lines and planned features
for the “new community site?” I think it is a good long term solution.

Nevertheless, If we are a long way out from launch of this new site it
makes sense to discuss a policy and governance approach.

On 05/24/2014 12:58 PM, Clinton Wolfe wrote:

Just spit balling here…

Perhaps this sort of thing could be taken into account in features of the
"new community site". I’m thinking of the rating system - there was some
discussion at the community summit of having multi-dimensional ratings,
like maturity, quality, features, stability, etc. I think maintainer
commitment could be a category, perhaps.

Alternatively, one could imagine metrics being displayed on the community
site - something like average time to respond to an issue, issues open
longer than X days, etc.

Or: a button to flag the cookbook as needing a new maintainer. That would
need moderation, I imagine.

On May 24, 2014 3:44 PM, “Scott M. Likens” scott@likens.us wrote:

Hi,

Please note I don’t work for Opscode (Getchef) so this is just my view.

On 5/23/14 1:58 PM, Ryan Hass wrote:

A little over six months have gone by since many of the cookbooks were
handed off to various community maintainers. Now seems like a good time
to continue this discussion with wisdom at hand; I am sure many others,
like myself may have questions and surely have ideas on how to improve.

Many of the cookbook maintainers have done an outstanding job of working
with the community, resolving issues, and accepting and mentoring
changes. Unfortunately, there are a few which have not met the quality
standards which preceded their maintainer-ship. Having said this, I
would like to discuss the Chef governance policy with regards to
maintainers whom have been unable to fulfill their duties, and the
process of ensuring these cookbooks continue to have high quality going
forward.

Some questions I have are:

  • What is the current governance of cookbooks which were previously
    maintained by Chef (Opscode)?

  • How can the community get involved in this governance process to
    ensure high quality standards persist for said cookbooks?

Issues and Pull requests are you best friend here. Taking back a cookbook
is likely not easy; it would be easiest to fork it and point everyone to
the new fork? There likely should be a policy here or otherwise we’ll step
on peoples feelings.

(squishy!)

Please do not hesitate to share your questions or thoughts on the
subject of the cookbooks which were handed off. As mentioned, I am sure
there are many others with related questions and ideas on the topic.

-Ryan H.
This electronic message contains information which may be confidential or
privileged. The information is intended for the use of the individual or
entity named above. If you are not the intended recipient, be aware that
any disclosure, copying, distribution or use of the contents of this
information is prohibited. If you have received this electronic
transmission in error, please notify us by e-mail at (
postmaster@rapid7.com) immediately.

!DSPAM:537fbdd4209001804284693!

This electronic message contains information which may be confidential or privileged. The information is intended for the use of the individual or entity named above. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of the contents of this information is prohibited. If you have received this electronic transmission in error, please notify us by e-mail at (postmaster@rapid7.com) immediately.


#7

Ryan,

Pull requests get ignored for many reasons (sometimes even personal)
quality; the maintainer does not see the point (for a few reasons). Not
every pull request should be merged.

What cookbooks are you referring to here?

It should be also noted that sometimes the pull requests collected in
opscode-cookbooks too; why was not always obvious to the end user. It
has happened in the past; and usually the best suggestion I can give is
to help.

On 5/27/14 5:03 PM, Ryan Hass wrote:

My concerns come from some of the cookbooks which have a large number of
outstanding pull requests as well as open issues. Often they have not
actually seen pull rquests merged, commented on, or otherwise closed.

Forking is always an option, just not the best one in my opinion.
Moreover, with this approach the issues remain – without a common well
maintained upstream cookbook, amongst other issues, it becomes very
difficult to find “the right cookbook.”

On 05/24/2014 12:44 PM, Scott M. Likens wrote:

  • How can the community get involved in this governance process to
    ensure high quality standards persist for said cookbooks?
    Issues and Pull requests are you best friend here. Taking back a
    cookbook is likely not easy; it would be easiest to fork it and point
    everyone to the new fork? There likely should be a policy here or
    otherwise we’ll step on peoples feelings.

(squishy!)
This electronic message contains information which may be confidential or privileged. The information is intended for the use of the individual or entity named above. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of the contents of this information is prohibited. If you have received this electronic transmission in error, please notify us by e-mail at (postmaster@rapid7.com) immediately.

!DSPAM:538527e9100361804284693!


#8

Hey Ryan,

I’m actually gonna shift this thread to chef@ so that it gets broader
visibility.

I’ve been meaning to address some of the points here since we had the
Cookbook Governance BoF at ChefConf (and I have been remiss in not
posting a summary from that). I have a short deck that summarized how
I set up the conversation at the BoF:

Chef Cookbook Governance BoF at ChefConf from Julian Dunn

What we heard from folks there is that the community wants Chef
(Software, Inc.) to be more prescriptive about managing the namespaces
on the community site. In other words, folks who were at the BoF
wouldn’t mind if we set some standards for evicting unresponsive
maintainers from those namespaces, and/or transferring ownership to
others who want to take over cookbooks – provided, of course, that we
have made a good faith effort to contact maintainers & encourage them
to do their duty.

I’d like to hear from others who weren’t at the BoF if they also feel
this way. If you do, then I think we (Chef Software, Inc.) should work
with the community to develop some of these rules, and then enforce
them, to the extent that ‘enforce’ means ‘be the broker of last resort
in case of dispute or cookbook abandonment’.

I’ve also asked Chris Webber, our community software developer, to
weigh in on this thread. He has some thoughts that may be pertinent.

  • Julian

On Fri, May 23, 2014 at 4:58 PM, Ryan Hass ryan_hass@rapid7.com wrote:

A little over six months have gone by since many of the cookbooks were
handed off to various community maintainers. Now seems like a good time
to continue this discussion with wisdom at hand; I am sure many others,
like myself may have questions and surely have ideas on how to improve.

Many of the cookbook maintainers have done an outstanding job of working
with the community, resolving issues, and accepting and mentoring
changes. Unfortunately, there are a few which have not met the quality
standards which preceded their maintainer-ship. Having said this, I
would like to discuss the Chef governance policy with regards to
maintainers whom have been unable to fulfill their duties, and the
process of ensuring these cookbooks continue to have high quality going
forward.

Some questions I have are:

  • What is the current governance of cookbooks which were previously
    maintained by Chef (Opscode)?

  • How can the community get involved in this governance process to
    ensure high quality standards persist for said cookbooks?

Please do not hesitate to share your questions or thoughts on the
subject of the cookbooks which were handed off. As mentioned, I am sure
there are many others with related questions and ideas on the topic.

-Ryan H.
This electronic message contains information which may be confidential or privileged. The information is intended for the use of the individual or entity named above. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of the contents of this information is prohibited. If you have received this electronic transmission in error, please notify us by e-mail at (postmaster@rapid7.com) immediately.


[ Julian C. Dunn jdunn@aquezada.com * Sorry, I’m ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]


#9

So first off, thank you for the email Ryan. These issues are super important and I want to make sure we address them in a thoughtful and open manner.

So a couple of things stand out to me:

  1. At this point, software quality isn’t really a solved problem. We have great tools like ChefSpec, Test Kitchen, Rubocop and Foodcritic to give us information about the quality of our code but they are not 100%. With that said, we should probably work towards a community standard of what constitutes a quality cookbook, if only to give those of us writing cookbooks a yardstick to measure ourselves by.

  2. I worry about the idea of “enforcing” high quality. Things like setting standards for unresponsiveness and cookbook abandonment make a ton of sense to me, but the problem with enforcing around quality is that it becomes a very dangerous path and I think I would rather see us get some of the other aspects down before we even begin that process.

  3. There is another idea that we really haven’t discussed around ownership. I don’t have enough background to know the actual agreements around the cookbooks that the community has taken on, but it doesn’t seem like the cool thing to just take back a cookbook. Additionally, I worry about the precedent that doing so would set as the associated implications of doing such a thing. I think about it this way… Just because a sysadmin has the ability to read all the email on a system doesn’t mean it is right of them to do so. There are usually organizational policies and guidelines that govern when it is acceptable to do so. If we start down a path of governance and defining rules around these things, this would be a place I would like to see us address.

I really would like to hear what the community thinks about all of the things presented. More than anything, I would like to know if working towards a governance policy or set of guidelines is something the community wants undertake. I know we (Chef Software, Inc.) would love to be a part of facilitating that discussion and are key stakeholders, but I don’t think any of us want to go down the Governance path if there isn’t support from the greater community to do so.

Thanks!

— cwebber

On May 23, 2014, at 4:56 PM, Julian C. Dunn jdunn@aquezada.com wrote:

Hey Ryan,

I’m actually gonna shift this thread to chef@ so that it gets broader
visibility.

I’ve been meaning to address some of the points here since we had the
Cookbook Governance BoF at ChefConf (and I have been remiss in not
posting a summary from that). I have a short deck that summarized how
I set up the conversation at the BoF:
http://www.slideshare.net/JulianDunn/chef-cookbook-governance-bof-at-chefconf

What we heard from folks there is that the community wants Chef
(Software, Inc.) to be more prescriptive about managing the namespaces
on the community site. In other words, folks who were at the BoF
wouldn’t mind if we set some standards for evicting unresponsive
maintainers from those namespaces, and/or transferring ownership to
others who want to take over cookbooks – provided, of course, that we
have made a good faith effort to contact maintainers & encourage them
to do their duty.

I’d like to hear from others who weren’t at the BoF if they also feel
this way. If you do, then I think we (Chef Software, Inc.) should work
with the community to develop some of these rules, and then enforce
them, to the extent that ‘enforce’ means ‘be the broker of last resort
in case of dispute or cookbook abandonment’.

I’ve also asked Chris Webber, our community software developer, to
weigh in on this thread. He has some thoughts that may be pertinent.

  • Julian

On Fri, May 23, 2014 at 4:58 PM, Ryan Hass ryan_hass@rapid7.com wrote:

A little over six months have gone by since many of the cookbooks were
handed off to various community maintainers. Now seems like a good time
to continue this discussion with wisdom at hand; I am sure many others,
like myself may have questions and surely have ideas on how to improve.

Many of the cookbook maintainers have done an outstanding job of working
with the community, resolving issues, and accepting and mentoring
changes. Unfortunately, there are a few which have not met the quality
standards which preceded their maintainer-ship. Having said this, I
would like to discuss the Chef governance policy with regards to
maintainers whom have been unable to fulfill their duties, and the
process of ensuring these cookbooks continue to have high quality going
forward.

Some questions I have are:

  • What is the current governance of cookbooks which were previously
    maintained by Chef (Opscode)?

  • How can the community get involved in this governance process to
    ensure high quality standards persist for said cookbooks?

Please do not hesitate to share your questions or thoughts on the
subject of the cookbooks which were handed off. As mentioned, I am sure
there are many others with related questions and ideas on the topic.

-Ryan H.
This electronic message contains information which may be confidential or privileged. The information is intended for the use of the individual or entity named above. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of the contents of this information is prohibited. If you have received this electronic transmission in error, please notify us by e-mail at (postmaster@rapid7.com) immediately.


[ Julian C. Dunn jdunn@aquezada.com * Sorry, I’m ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]


#10

Hi Ryan!

We’ve been thinking about this quite a bit lately. With Supermarket on its
way, this is the perfect time to rekindle the conversation.

To answer you first question, we retain a commit bit on cookbooks which
were previously maintained by Chef, but trust the maintainers to do what
they think is best. Currently, issues with maintainer-ship are handled on
an individual case by case basis.

Like Julian mentioned, we’re looking at other projects (Arch Linux, Perl)
for examples of governance policies. The key is to be super clear about
exactly what issues such a policy is intended to solve, and not to
discourage or alienate people when enforcing such policies. Issues like how
to handle abandoned projects clearly need to be addressed.

One of the exciting things about Supermarket shipping, is we’ll finally
have a hackable community site. Instrumenting that to provide visibility
into usage patterns is going to allow us to make better decisions. It also
gives us the freedom to start tinkering around with things like name
spacing.

-s

On Fri, May 23, 2014 at 9:08 PM, Christopher Webber cwebber@getchef.comwrote:

So first off, thank you for the email Ryan. These issues are super
important and I want to make sure we address them in a thoughtful and open
manner.

So a couple of things stand out to me:

  1. At this point, software quality isn’t really a solved problem. We have
    great tools like ChefSpec, Test Kitchen, Rubocop and Foodcritic to give us
    information about the quality of our code but they are not 100%. With that
    said, we should probably work towards a community standard of what
    constitutes a quality cookbook, if only to give those of us writing
    cookbooks a yardstick to measure ourselves by.

  2. I worry about the idea of “enforcing” high quality. Things like setting
    standards for unresponsiveness and cookbook abandonment make a ton of sense
    to me, but the problem with enforcing around quality is that it becomes a
    very dangerous path and I think I would rather see us get some of the other
    aspects down before we even begin that process.

  3. There is another idea that we really haven’t discussed around
    ownership. I don’t have enough background to know the actual agreements
    around the cookbooks that the community has taken on, but it doesn’t seem
    like the cool thing to just take back a cookbook. Additionally, I worry
    about the precedent that doing so would set as the associated implications
    of doing such a thing. I think about it this way… Just because a sysadmin
    has the ability to read all the email on a system doesn’t mean it is right
    of them to do so. There are usually organizational policies and guidelines
    that govern when it is acceptable to do so. If we start down a path of
    governance and defining rules around these things, this would be a place I
    would like to see us address.

I really would like to hear what the community thinks about all of the
things presented. More than anything, I would like to know if working
towards a governance policy or set of guidelines is something the community
wants undertake. I know we (Chef Software, Inc.) would love to be a part of
facilitating that discussion and are key stakeholders, but I don’t think
any of us want to go down the Governance path if there isn’t support from
the greater community to do so.

Thanks!

— cwebber

On May 23, 2014, at 4:56 PM, Julian C. Dunn jdunn@aquezada.com wrote:

Hey Ryan,

I’m actually gonna shift this thread to chef@ so that it gets broader
visibility.

I’ve been meaning to address some of the points here since we had the
Cookbook Governance BoF at ChefConf (and I have been remiss in not
posting a summary from that). I have a short deck that summarized how
I set up the conversation at the BoF:

http://www.slideshare.net/JulianDunn/chef-cookbook-governance-bof-at-chefconf

What we heard from folks there is that the community wants Chef
(Software, Inc.) to be more prescriptive about managing the namespaces
on the community site. In other words, folks who were at the BoF
wouldn’t mind if we set some standards for evicting unresponsive
maintainers from those namespaces, and/or transferring ownership to
others who want to take over cookbooks – provided, of course, that we
have made a good faith effort to contact maintainers & encourage them
to do their duty.

I’d like to hear from others who weren’t at the BoF if they also feel
this way. If you do, then I think we (Chef Software, Inc.) should work
with the community to develop some of these rules, and then enforce
them, to the extent that ‘enforce’ means ‘be the broker of last resort
in case of dispute or cookbook abandonment’.

I’ve also asked Chris Webber, our community software developer, to
weigh in on this thread. He has some thoughts that may be pertinent.

  • Julian

On Fri, May 23, 2014 at 4:58 PM, Ryan Hass ryan_hass@rapid7.com wrote:

A little over six months have gone by since many of the cookbooks were
handed off to various community maintainers. Now seems like a good time
to continue this discussion with wisdom at hand; I am sure many others,
like myself may have questions and surely have ideas on how to improve.

Many of the cookbook maintainers have done an outstanding job of working
with the community, resolving issues, and accepting and mentoring
changes. Unfortunately, there are a few which have not met the quality
standards which preceded their maintainer-ship. Having said this, I
would like to discuss the Chef governance policy with regards to
maintainers whom have been unable to fulfill their duties, and the
process of ensuring these cookbooks continue to have high quality going
forward.

Some questions I have are:

  • What is the current governance of cookbooks which were previously
    maintained by Chef (Opscode)?

  • How can the community get involved in this governance process to
    ensure high quality standards persist for said cookbooks?

Please do not hesitate to share your questions or thoughts on the
subject of the cookbooks which were handed off. As mentioned, I am sure
there are many others with related questions and ideas on the topic.

-Ryan H.
This electronic message contains information which may be confidential
or privileged. The information is intended for the use of the individual or
entity named above. If you are not the intended recipient, be aware that
any disclosure, copying, distribution or use of the contents of this
information is prohibited. If you have received this electronic
transmission in error, please notify us by e-mail at (
postmaster@rapid7.com) immediately.


[ Julian C. Dunn jdunn@aquezada.com * Sorry, I’m ]
[ WWW: http://www.aquezada.com/staff/julian * only Web 1.0 ]
[ gopher://sdf.org/1/users/keymaker/ * compliant! ]
[ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9 ]


#11

I would really like to see Chef not be the SPOF for cookbook sharing and
governance. I would love to see someone aggressively fork the community
site and implement their own code quality standards, supported features
and governance. Because as long as we think we’re trying to produce the
one apache cookbook that works on every RHEL, Debian, Ubuntu, Gentoo,
Arch, Solaris and AIX distro for the past 10 years and supports binary
and source installation, etc we’re going to produce a mess which
satisfies almost nobody. The models of rubygems and CPAN and other
library sites for languages are a really bad metaphor for this domain
because we don’t just have the problem of running on a few different
versions of a language, but running on a huge matrix of different O/Ses
and distros and all the ways that sysadmins like to maintain their
servers. There will never be the OneTrueWayToInstallApache.

It would also be good to see better competition in the ‘marketplace’ for
cookbook governance. I’d love to see us be able to be lazy and just
steal good ideas that other people had implemented and had turned out to
be successful (and also learn from the failures that happen). Ideally
Chef, the company, just gets out of the business of managing cookbooks
entirely and we can focus on building the software behind it all. I
don’t think that happens unless you have a robust, self-supporting
community, though, and right now it feels too much like the Chef
community is just looking entirely at the company to provide all the
leadership here. I don’t think that’s a good thing.

And when people start talking about "enforcement of community standards"
when there’s just one game in town, my inner anarchist gets very upset.
I’d prefer to see different communities, and if one of them has a
complete internal political meltdown over moderation and governance then
you can fork it all and patch the politics.

On 5/23/14, 6:14 PM, Sean OMeara wrote:

Hi Ryan!

We’ve been thinking about this quite a bit lately. With Supermarket on
its way, this is the perfect time to rekindle the conversation.

To answer you first question, we retain a commit bit on cookbooks
which were previously maintained by Chef, but trust the maintainers to
do what they think is best. Currently, issues with maintainer-ship are
handled on an individual case by case basis.

Like Julian mentioned, we’re looking at other projects (Arch Linux,
Perl) for examples of governance policies. The key is to be super
clear about exactly what issues such a policy is intended to solve,
and not to discourage or alienate people when enforcing such policies.
Issues like how to handle abandoned projects clearly need to be addressed.

One of the exciting things about Supermarket shipping, is we’ll
finally have a hackable community site. Instrumenting that to provide
visibility into usage patterns is going to allow us to make better
decisions. It also gives us the freedom to start tinkering around with
things like name spacing.

-s

On Fri, May 23, 2014 at 9:08 PM, Christopher Webber
<cwebber@getchef.com mailto:cwebber@getchef.com> wrote:

So first off, thank you for the email Ryan. These issues are super
important and I want to make sure we address them in a thoughtful
and open manner.

So a couple of things stand out to me:

1) At this point, software quality isn’t really a solved problem.
We have great tools like ChefSpec, Test Kitchen, Rubocop and
Foodcritic to give us information about the quality of our code
but they are not 100%. With that said, we should probably work
towards a community standard of what constitutes a quality
cookbook, if only to give those of us writing cookbooks a
yardstick to measure ourselves by.

2) I worry about the idea of “enforcing” high quality. Things like
setting standards for unresponsiveness and cookbook abandonment
make a ton of sense to me, but the problem with enforcing around
quality is that it becomes a very dangerous path and I think I
would rather see us get some of the other aspects down before we
even begin that process.

3) There is another idea that we really haven’t discussed around
ownership. I don’t have enough background to know the actual
agreements around the cookbooks that the community has taken on,
but it doesn’t seem like the cool thing to just take back a
cookbook. Additionally, I worry about the precedent that doing so
would set as the associated implications of doing such a thing. I
think about it this way… Just because a sysadmin has the ability
to read all the email on a system doesn’t mean it is right of them
to do so. There are usually organizational policies and guidelines
that govern when it is acceptable to do so. If we start down a
path of governance and defining rules around these things, this
would be a place I would like to see us address.

I really would like to hear what the community thinks about all of
the things presented. More than anything, I would like to know if
working towards a governance policy or set of guidelines is
something the community wants undertake. I know we (Chef Software,
Inc.) would love to be a part of facilitating that discussion and
are key stakeholders, but I don’t think any of us want to go down
the Governance path if there isn’t support from the greater
community to do so.

Thanks!

— cwebber

On May 23, 2014, at 4:56 PM, Julian C. Dunn <jdunn@aquezada.com
<mailto:jdunn@aquezada.com>> wrote:

> Hey Ryan,
>
> I'm actually gonna shift this thread to chef@ so that it gets
broader
> visibility.
>
> I've been meaning to address some of the points here since we
had the
> Cookbook Governance BoF at ChefConf (and I have been remiss in not
> posting a summary from that). I have a short deck that
summarized how
> I set up the conversation at the BoF:
>
http://www.slideshare.net/JulianDunn/chef-cookbook-governance-bof-at-chefconf
>
> What we heard from folks there is that the community wants Chef
> (Software, Inc.) to be more prescriptive about managing the
namespaces
> on the community site. In other words, folks who were at the BoF
> wouldn't mind if we set some standards for evicting unresponsive
> maintainers from those namespaces, and/or transferring ownership to
> others who want to take over cookbooks -- provided, of course,
that we
> have made a good faith effort to contact maintainers & encourage
them
> to do their duty.
>
> I'd like to hear from others who weren't at the BoF if they also
feel
> this way. If you do, then I think we (Chef Software, Inc.)
should work
> with the community to develop some of these rules, and then enforce
> them, to the extent that 'enforce' means 'be the broker of last
resort
> in case of dispute or cookbook abandonment'.
>
> I've also asked Chris Webber, our community software developer, to
> weigh in on this thread. He has some thoughts that may be pertinent.
>
> - Julian
>
> On Fri, May 23, 2014 at 4:58 PM, Ryan Hass <ryan_hass@rapid7.com
<mailto:ryan_hass@rapid7.com>> wrote:
>> A little over six months have gone by since many of the
cookbooks were
>> handed off to various community maintainers. Now seems like a
good time
>> to continue this discussion with wisdom at hand; I am sure many
others,
>> like myself may have questions and surely have ideas on how to
improve.
>>
>> Many of the cookbook maintainers have done an outstanding job
of working
>> with the community, resolving issues, and accepting and mentoring
>> changes. Unfortunately, there are a few which have not met the
quality
>> standards which preceded their maintainer-ship. Having said this, I
>> would like to discuss the Chef governance policy with regards to
>> maintainers whom have been unable to fulfill their duties, and the
>> process of ensuring these cookbooks continue to have high
quality going
>> forward.
>>
>> Some questions I have are:
>>
>> * What is the current governance of cookbooks which were previously
>> maintained by Chef (Opscode)?
>>
>> * How can the community get involved in this governance process to
>> ensure high quality standards persist for said cookbooks?
>>
>> Please do not hesitate to share your questions or thoughts on the
>> subject of the cookbooks which were handed off. As mentioned, I
am sure
>> there are many others with related questions and ideas on the
topic.
>>
>> -Ryan H.
>> This electronic message contains information which may be
confidential or privileged. The information is intended for the
use of the individual or entity named above. If you are not the
intended recipient, be aware that any disclosure, copying,
distribution or use of the contents of this information is
prohibited. If you have received this electronic transmission in
error, please notify us by e-mail at (postmaster@rapid7.com
<mailto:postmaster@rapid7.com>) immediately.
>>
>
>
>
> --
> [ Julian C. Dunn <jdunn@aquezada.com
<mailto:jdunn@aquezada.com>>          * Sorry, I'm    ]
> [ WWW: http://www.aquezada.com/staff/julian    * only Web 1.0  ]
> [ gopher://sdf.org/1/users/keymaker/
<http://sdf.org/1/users/keymaker/>   * compliant!    ]
> [ PGP: 91B3 7A9D 683C 7C16 715F 442C 6065 D533 FDC2 05B9       ]

#12

On Sun, May 25, 2014 at 12:41 PM, Lamont Granquist lamont@opscode.com
wrote:

I would really like to see Chef not be the SPOF for cookbook sharing and
governance. I would love to see someone aggressively fork the community
site and implement their own code quality standards, supported features and
governance. Because as long as we think we’re trying to produce the one
apache cookbook that works on every RHEL, Debian, Ubuntu, Gentoo, Arch,
Solaris and AIX distro for the past 10 years and supports binary and source
installation, etc we’re going to produce a mess which satisfies almost
nobody. The models of rubygems and CPAN and other library sites for
languages are a really bad metaphor for this domain because we don’t just
have the problem of running on a few different versions of a language, but
running on a huge matrix of different O/Ses and distros and all the ways
that sysadmins like to maintain their servers. There will never be the
OneTrueWayToInstallApache.

We have already started doing this for Amazon Linux.

Our focus is on producing a supported set cookbooks that codifies
and complies very closely to AWS Security and Architecture Best
Practices. Where we can’t fully automate, we provide extensive
documentation on how the cookbook should be used.

As an example of how we do this, please see -

https://github.com/lambda-linux-cookbooks/ll-postgresql-devops-helper
https://github.com/lambda-linux-cookbooks/ll-yum#ll_yum_createrepo
https://github.com/lambda-linux-cookbooks/ll-postgresql

There are a number of quality standards that we implement in our
cookbooks. I’ll highlight a few here,

  1. We manually audit every cookbook to ensure that there is no funny
    business going on and aggressively prune out code paths that are not
    relevant to our context.

  2. We have a much shallower dependency chain. As a general rule
    we do not exceed two levels of dependency as complexity increases
    exponentially for every additional level of cookbook dependency.
    This is bad for security.

    In addition, all ll-COOKBOOKS depend only on other ll-COOKBOOKS.
    This means entire dependency chain is audited.

  3. We have a very strong notion of public APIs in our cookbooks. If
    we break public APIs (documented default attributes, recipes) for
    our users, we will fix it.

    Most of our cookbooks are designed to be used via wrapper-cookbook
    and base-cookbook patterns.

  4. We separate out build-time vs. run-time concerns in cookbooks. Our
    preferred approach is to build the packages separately and inject
    them where necessary into the workflow. This approach removes the
    need for build-essential in many cookbooks and also significantly
    reduces the time needed to complete Chef Run.

There are more things that we do, but you get the idea. :slight_smile:

If you are on Amazon Linux/EC2, feel free to check out and use the
cookbooks that we’ve already published at,

https://github.com/lambda-linux-cookbooks

I don’t mean to hijack this thread. Lamont is making a very good
point and I wanted to give an example of what is possible.

If this is an approach that is interesting to you, I’ll be at DevOps
Days Silicon Valley and we can continue the conversation there. I can
also show you some other things this approach allows us to do.

Thank you!

Best,
Rajiv


#13

On Tue May 27 10:05:48 2014, Rajiv Ranganath wrote:

I don’t mean to hijack this thread. Lamont is making a very good
point and I wanted to give an example of what is possible.

If this is an approach that is interesting to you, I’ll be at DevOps
Days Silicon Valley and we can continue the conversation there. I can
also show you some other things this approach allows us to do.

Thank you!

Best,
Rajiv

I don’t think this is a hijack at all. What I’d really like to see is
either the public supermarket instance supporting namespaces a bit more
explicitly, or else the ability for you to run your own supermarket
instance and the tools like berkshelf and knife cookbook site supporting
pulling/pushing to your instance via some simple config. And I’d prefer
to discuss whatever problems you are seeing with supporting your own
community than trying to fix the governance problems in the existing
Chef community site (although to be fair we should probably come up with
better governance policy there, but I think if we can start seeing
community forks of the community site that the requirements for the
existing community site will start to change – and I kind of hope the
community will run faster at this problem than we can respond to it and
fix the problems while we’re still trying to react, and that we should
focus on trying to enable what you’re doing).

And Amazon Linux is a great example since its RHEL-like and we already
don’t do RHEL entirely well (see SELinux + apache cookbook) and then
Amazon Linux is that odd mashup of different RHELs with fedora bits in
it which makes it have other strange edge conditions, and nobody uses it
much around the office here. It’ll most likely be much better served by
your fork.