Contributing to Cookbooks (was chef & hoptoad)


#1

Ohai!

Begin forwarded message:

On a related topic… I still don’t quite grok how one can contribute back to the cookbooks…
there are multiple repos of cookbooks… and multiple repos of single cookbooks… so its
kinda hard to contribute.

Cookbooks are treated like any of Opscode’s open source projects. There is a minor difference in terms of release.

Development happens in the github repository, http://github.com/opscode/cookbooks/. Tickets are tracked in the COOK project in tickets.opscode.com. In order to contribute, since they’re Apache licensed, you do need to have signed a CLA. The workflow for contributing is the same as the other projects (particularly Chef, but also Ohai or the Mixlibs). For the documentation on how to contribute, see the Chef wiki:

http://wiki.opscode.com/display/opscode/Contributing

Setting up a repository:

http://wiki.opscode.com/display/opscode/Working+with+Git

If you have any questions at all on the process, please send them to the list, ask on IRC or feel free to ask me directly.

Now, the minor difference, on the release of cookbooks. While Chef, Ohai and the other code is released as RubyGems, we release the Chef cookbooks to the Cookbook site, http://cookbooks.opscode.com. Now that knife has the ability to work with the site directly, we recommend a workflow that utilizes the Opscode chef-repo as a Git repository, adding cookbooks as “vendor branches”. For more information on this workflow, see:

http://help.opscode.com/faqs/start/working-with-cookbooks

(it uses an example cookbook, getting-started)

So far I’ve found the existing cookbooks to be a very useful crib for writing my own… often
I copy one, hack away the stuff I’m not interested in (apache, etc) and then add the few
small twiddles I need (e.g. nginx).

Nothing wrong with this at all, they were intended to be provided as a ‘baseline’ that you can then customize for your environment. Further, using the workflow described on the help site, you not only have your local modifications, but you can “track” the upstream changes and merge them in as well.

Also, I tend to work on a recipe until it works for me… and then stop working on it… and
I don’t bother making it generic enough for anybody to use… me being lazy but I think
the contributing back hurdle is too high.

If the contribution hurdle is too high thats our bad and we should make it clearer! Hopefully the information above will help you, and we look forward to your contributions!

Thanks for using Chef and the feedback on how we can make it better.


Opscode, Inc
Joshua Timberman, Senior Solutions Engineer
C: 720.334.RUBY E: joshua@opscode.com


#2

On Thu, Jul 15, 2010 at 2:43 PM, Joshua Timberman joshua@opscode.com wrote:

Thanks for using Chef and the feedback on how we can make it better.

Don’t make chef-server (i.e. knife) the de-facto (and only
easy/sustainable) way to participate in and contribute to the opscode
cookbook community, if chef-solo works fine and chef-server is
overkill for my particular usage scenario (never got a response to my
previous detailed post on this topic). Sorry, but you said you liked
feedback :slight_smile:

Thanks,
– Chad


#3

Hi Chad!

On Jul 16, 2010, at 12:24 AM, Chad Woolley wrote:

Don’t make chef-server (i.e. knife) the de-facto (and only
easy/sustainable) way to participate in and contribute to the opscode
cookbook community, if chef-solo works fine and chef-server is
overkill for my particular usage scenario (never got a response to my
previous detailed post on this topic). Sorry, but you said you liked
feedback :slight_smile:

And I do like feedback, I must have missed the email you posted before, and that’s no good of me!

The short version of the tale is that we (Opscode) work with a Chef Server, so when writing recipes, they are focused on working in that context. Please do open tickets (with patches if possible) to have our recipes work with chef-solo as well. It’s not that we see Solo as a second-class citizen, we’re just not using it for our own stuff.


Opscode, Inc
Joshua Timberman, Senior Solutions Engineer
C: 720.334.RUBY E: joshua@opscode.com


#4

I guess I have two separate roadblocks to contributing:

First I mainly spend my time with chef developing cookbooks, so how would I ‘contribute’ a cookbook while not incurring a lot of overhead ? For example, I create a cookbook that I haven’t seen in the wild, I’d like to publish it for others to use and enhance, but since it’s in my private repo, how would I do that?

One idea would be that I have another cookbooks repo where I publish the cookbook and be able to import it using the existing vendor stuff knife provides but use it against other repo’s than just the cookbooks site.

Secondly, I’ve run into times where I have found bugs in chef that I’ve traced into the code and figured out ‘why’ its broke. I’ve gone ahead and reported the bugs, but I’d love to be able to just fix it myself. However I end up just working around the problem in my cookbooks because I’m not sure how I would fix chef and then deploy it so my nodes can take advantage of my fix since in some clients I have pre-built AMI’s with chef pre-installed, but I’m transitioning to the bootstrap way. In either case, I’ve now forked chef and taken some burden of responsibility of packaging my own chef. So far I haven’t done that because of the extra burden. I’d love to hear any tips on making that easier.

Thanks,

Alex

On Jul 15, 2010, at 2:43 PM, Joshua Timberman wrote:

Ohai!

Begin forwarded message:

On a related topic… I still don’t quite grok how one can contribute back to the cookbooks…
there are multiple repos of cookbooks… and multiple repos of single cookbooks… so its
kinda hard to contribute.

Cookbooks are treated like any of Opscode’s open source projects. There is a minor difference in terms of release.

Development happens in the github repository, http://github.com/opscode/cookbooks/. Tickets are tracked in the COOK project in tickets.opscode.com. In order to contribute, since they’re Apache licensed, you do need to have signed a CLA. The workflow for contributing is the same as the other projects (particularly Chef, but also Ohai or the Mixlibs). For the documentation on how to contribute, see the Chef wiki:

http://wiki.opscode.com/display/opscode/Contributing

Setting up a repository:

http://wiki.opscode.com/display/opscode/Working+with+Git

If you have any questions at all on the process, please send them to the list, ask on IRC or feel free to ask me directly.

Now, the minor difference, on the release of cookbooks. While Chef, Ohai and the other code is released as RubyGems, we release the Chef cookbooks to the Cookbook site, http://cookbooks.opscode.com. Now that knife has the ability to work with the site directly, we recommend a workflow that utilizes the Opscode chef-repo as a Git repository, adding cookbooks as “vendor branches”. For more information on this workflow, see:

http://help.opscode.com/faqs/start/working-with-cookbooks

(it uses an example cookbook, getting-started)

So far I’ve found the existing cookbooks to be a very useful crib for writing my own… often
I copy one, hack away the stuff I’m not interested in (apache, etc) and then add the few
small twiddles I need (e.g. nginx).

Nothing wrong with this at all, they were intended to be provided as a ‘baseline’ that you can then customize for your environment. Further, using the workflow described on the help site, you not only have your local modifications, but you can “track” the upstream changes and merge them in as well.

Also, I tend to work on a recipe until it works for me… and then stop working on it… and
I don’t bother making it generic enough for anybody to use… me being lazy but I think
the contributing back hurdle is too high.

If the contribution hurdle is too high thats our bad and we should make it clearer! Hopefully the information above will help you, and we look forward to your contributions!

Thanks for using Chef and the feedback on how we can make it better.


Opscode, Inc
Joshua Timberman, Senior Solutions Engineer
C: 720.334.RUBY E: joshua@opscode.com


#5

On Jul 15, 2010, at 3:31 PM, Alex Soto wrote:

First I mainly spend my time with chef developing cookbooks, so how would I ‘contribute’ a cookbook while not incurring a lot of overhead ? For example, I create a cookbook that I haven’t seen in the wild, I’d like to publish it for others to use and enhance, but since it’s in my private repo, how would I do that?

Thanks to Josh’s informative emails, I’ve been pondering on this in the background most of the afternoon.

Looking at my site-cookbooks I notice that there’s a pattern developing. I tend to create subclass cookbooks.
So, for example I’ll create site-cookbooks/jm_memcached, which includes cookbooks/memcached, and then
adds my own twiddles. I think this addresses the public vs private problem.

Adding my initials to the name is kinda icky, but if this is a common pattern then Chef/Knife could codify
it with some namespace support, or cookbook inheritance, or something.

John


John Merrells
http://johnmerrells.com
+1.415.244.5808


#6

I think the knife cookbook vendor command takes care of the need to have your own jm_cookbook. Just vendor the cookbook, make your tweaks. If you wish to update from upstream, you pull use the vendor command again and fix any merge conflicts.

Take with a grain of salt. I’ve vendored but haven’t needed to pull in upstream changes yet, but I’m assuming if my changes are minor, merges should be fairly painless.

Alex

On Jul 15, 2010, at 5:07 PM, John Merrells wrote:

On Jul 15, 2010, at 3:31 PM, Alex Soto wrote:

First I mainly spend my time with chef developing cookbooks, so how would I ‘contribute’ a cookbook while not incurring a lot of overhead ? For example, I create a cookbook that I haven’t seen in the wild, I’d like to publish it for others to use and enhance, but since it’s in my private repo, how would I do that?

Thanks to Josh’s informative emails, I’ve been pondering on this in the background most of the afternoon.

Looking at my site-cookbooks I notice that there’s a pattern developing. I tend to create subclass cookbooks.
So, for example I’ll create site-cookbooks/jm_memcached, which includes cookbooks/memcached, and then
adds my own twiddles. I think this addresses the public vs private problem.

Adding my initials to the name is kinda icky, but if this is a common pattern then Chef/Knife could codify
it with some namespace support, or cookbook inheritance, or something.

John


John Merrells
http://johnmerrells.com
+1.415.244.5808


#7

I didn’t know about the vendor thing. But, I think I wrote my cookbooks before knife could do that. I should go rtfm the manual again…

John

On Jul 15, 2010, at 5:29 PM, Alex Soto wrote:

I think the knife cookbook vendor command takes care of the need to have your own jm_cookbook. Just vendor the cookbook, make your tweaks. If you wish to update from upstream, you pull use the vendor command again and fix any merge conflicts.

Take with a grain of salt. I’ve vendored but haven’t needed to pull in upstream changes yet, but I’m assuming if my changes are minor, merges should be fairly painless.

Alex

On Jul 15, 2010, at 5:07 PM, John Merrells wrote:

On Jul 15, 2010, at 3:31 PM, Alex Soto wrote:

First I mainly spend my time with chef developing cookbooks, so how would I ‘contribute’ a cookbook while not incurring a lot of overhead ? For example, I create a cookbook that I haven’t seen in the wild, I’d like to publish it for others to use and enhance, but since it’s in my private repo, how would I do that?

Thanks to Josh’s informative emails, I’ve been pondering on this in the background most of the afternoon.

Looking at my site-cookbooks I notice that there’s a pattern developing. I tend to create subclass cookbooks.
So, for example I’ll create site-cookbooks/jm_memcached, which includes cookbooks/memcached, and then
adds my own twiddles. I think this addresses the public vs private problem.

Adding my initials to the name is kinda icky, but if this is a common pattern then Chef/Knife could codify
it with some namespace support, or cookbook inheritance, or something.

John


John Merrells
http://johnmerrells.com
+1.415.244.5808


John Merrells
http://johnmerrells.com
+1.415.244.5808


#8

Ohai!

On Jul 15, 2010, at 4:31 PM, Alex Soto wrote:

First I mainly spend my time with chef developing cookbooks, so how would I ‘contribute’ a cookbook while not incurring a lot of overhead ? For example, I create a cookbook that I haven’t seen in the wild, I’d like to publish it for others to use and enhance, but since it’s in my private repo, how would I do that?

If you would like to simply publish a new cookbook that doesn’t “exist” in the wild yet, the best place to put that is the Opscode cookbooks site.

Then other people can discover and use your cookbook by downloading it with the ‘vendor branch’ workflow.

One idea would be that I have another cookbooks repo where I publish the cookbook and be able to import it using the existing vendor stuff knife provides but use it against other repo’s than just the cookbooks site.

Currently knife only supports the Opscode cookbooks site, since it provides an API that we can use directly. I do not knife if the dev team has plans to implement support for other sites or methods by which to ‘vendor’ cookbooks. The advantage of the Opscode site vs say GitHub is that it integrates well with the Platform, and its a better known quantity for the API usage. But I’ll let Adam or Dan speak up about potential roadmap topics in that regard.

Secondly, I’ve run into times where I have found bugs in chef that I’ve traced into the code and figured out ‘why’ its broke. I’ve gone ahead and reported the bugs, but I’d love to be able to just fix it myself. However I end up just working around the problem in my cookbooks because I’m not sure how I would fix chef and then deploy it so my nodes can take advantage of my fix since in some clients I have pre-built AMI’s with chef pre-installed, but I’m transitioning to the bootstrap way. In either case, I’ve now forked chef and taken some burden of responsibility of packaging my own chef. So far I haven’t done that because of the extra burden. I’d love to hear any tips on making that easier.

If you have found bugs in Chef itself and know how to fix them, we’d love a topic branch in your fork of the Chef repo that we can merge into master. That does of course require a contributor license agreement, which I see you are on the list of ‘approved contributors’ already, so thank you! Follow the working with git flow on the wiki page I linked and you should have a path to victory. If you need assistance with development, particularly with updating or adding feature/unit testing for your code, please drop by #chef-hacking or post to the chef development list and someone will be able to help you. We can’t help if we don’t know what your needs are :-).

Thank you!


Opscode, Inc
Joshua Timberman, Senior Solutions Engineer
C: 720.334.RUBY E: joshua@opscode.com


#9

On Jul 15, 2010, at 7:05 PM, Joshua Timberman wrote:

On Jul 15, 2010, at 4:31 PM, Alex Soto wrote:

First I mainly spend my time with chef developing cookbooks, so how would I ‘contribute’ a cookbook while not incurring a lot of overhead ? For example, I create a cookbook that I haven’t seen in the wild, I’d like to publish it for others to use and enhance, but since it’s in my private repo, how would I do that?

If you would like to simply publish a new cookbook that doesn’t “exist” in the wild yet, the best place to put that is the Opscode cookbooks site.

Then other people can discover and use your cookbook by downloading it with the ‘vendor branch’ workflow.

Great news. I was under the impression that the cookbooks site was just a distribution method for the cookbooks in the opscode cookbooks repo. I wasn’t aware I could just push things up there. What happens to a cookbook I push up there? Does it get reviewed, or does it go right up on the site?

One idea would be that I have another cookbooks repo where I publish the cookbook and be able to import it using the existing vendor stuff knife provides but use it against other repo’s than just the cookbooks site.

Currently knife only supports the Opscode cookbooks site, since it provides an API that we can use directly. I do not knife if the dev team has plans to implement support for other sites or methods by which to ‘vendor’ cookbooks. The advantage of the Opscode site vs say GitHub is that it integrates well with the Platform, and its a better known quantity for the API usage. But I’ll let Adam or Dan speak up about potential roadmap topics in that regard.
I looked at the code, and I see what you mean, but it seems like it’s just pulling down a tarball. By pointing to another repo and using git archive, you can mimic that behavior. Unfortunately, it doesn’t seem that github allows using that command to pull a single cookbook as a tarball so it may not be useful for github, but private github repo’s work that way.


#10

On Fri, Jul 16, 2010 at 5:47 PM, Alex Soto apsoto@gmail.com wrote:

Currently knife only supports the Opscode cookbooks site, since it provides an API that we can use directly. I do not knife if the dev team has plans to implement support for other sites or methods by which to ‘vendor’ cookbooks. The advantage of the Opscode site vs say GitHub is that it integrates well with the Platform, and its a better known quantity for the API usage. But I’ll let Adam or Dan speak up about potential roadmap topics in that regard.
I looked at the code, and I see what you mean, but it seems like it’s just pulling down a tarball. By pointing to another repo and using git archive, you can mimic that behavior. Unfortunately, it doesn’t seem that github allows using that command to pull a single cookbook as a tarball so it may not be useful for github, but private github repo’s work that way.

Yes, that’s exactly what I would like to see- a parallel mode for
knife which works with a plain git repo, or even just github. This
would allow people who don’t use chef-server to still fork and
participate in the opscode cookbook community.

When I first started with chef, I tried briefly to be a good citizen
and do this manually fork opscodes’ cookbooks using git topic branches
and do pull requests, tickets, etc. However, it’s just too much work
(i.e. much easier to just write all my cookbooks from scratch using
opscode’s cookbooks as an example, and not fork opscode’s cookbooks or
contribute anything back).

Since my from-scratch cookbooks were so much simpler, focused, and
non-buggy; I’m not sure I would fork opscode’s again even if it were
easy to contribute back, but that’s a discussion for another day :wink:
chef-solo and ohai are the bomb in any case, thanks a lot for them.

– Chad


#11

On Jul 16, 2010, at 18:47, Alex Soto apsoto@gmail.com wrote:

Great news. I was under the impression that the cookbooks site was just a distribution method for the cookbooks in the opscode cookbooks repo. I wasn’t aware I could just push things up there. What happens to a cookbook I push up there? Does it get reviewed, or does it go right up on the site?

You definitely can post your cookbooks to the site. It has a flat namespace, so first come, first served on names. They are not reviewed before being posted, other than a simple check that happens on the metadata for things like the version and directory structure. Other users can download and review on their own and then rate with a star system and/or add comments.

In an upcoming version of chef, we will be adding the capability to knife that will upload cookbooks to the site, directly from your chef-repo.


#12

On Fri, Jul 16, 2010 at 9:28 PM, Chad Woolley thewoolleyman@gmail.com wrote:

Yes, that’s exactly what I would like to see- a parallel mode for
knife which works with a plain git repo, or even just github. This
would allow people who don’t use chef-server to still fork and
participate in the opscode cookbook community.

Patches welcome!

There is nothing stopping you from contributing to the cookbook
community, though - all you need to do is package up your cookbook and
publish it. It’s not specific to the Chef server. At the same time,
not every cookbook is going to work with chef-solo - things like
Search and Data Bags are always going to be a bummer there.

When I first started with chef, I tried briefly to be a good citizen
and do this manually fork opscodes’ cookbooks using git topic branches
and do pull requests, tickets, etc. However, it’s just too much work
(i.e. much easier to just write all my cookbooks from scratch using
opscode’s cookbooks as an example, and not fork opscode’s cookbooks or
contribute anything back).

There is nothing wrong with writing your own cookbooks - you’re not
being a bad citizen. What you might want to do after the fact is
publish yours, because others might prefer the way you do it to the
way we do. TIMTOWTDI is the rule here.

Since my from-scratch cookbooks were so much simpler, focused, and
non-buggy; I’m not sure I would fork opscode’s again even if it were
easy to contribute back, but that’s a discussion for another day :wink:
chef-solo and ohai are the bomb in any case, thanks a lot for them.

Right, that’s the point - if you like what you’re doing, and it’s
awesome, don’t stop doing it. But consider sharing it, so that other
people who also like what you are doing can do it your way instead.

Adam


Opscode, Inc.
Adam Jacob, CTO
T: (206) 508-7449 E: adam@opscode.com