Documenting Cookbooks

Chef Infra (archive)
1

Hi,

One of the things I have struggled with is documenting our chef cookbooks.
Last week I had a few hours to spare so I put together a way of knife
plugin that helps generate the README for a cookbook. The basic concept is
to markup metadata.rb and the source files and generate documentation based
on the annotations, similar to javadoc.

I have written up the first step at [1] but am looking for feedback on the
general approach. Step 1 was to generate the README. For step 2 I am
thinking of allowing you to point the tool at a chef server (or a local
repository) and generate documentation for the whole set of cookbooks and
thus generate the documentation for the suite (either as PDF or html).

So is this of interest to anyone else? Anyone have opinions on the best way
to do this?

[1] http://realityforge.org/code/2013/04/01/documenting-cookbooks.html

Cheers,

Peter Donald

2

So is this of interest to anyone else?

Yes!

A thousand times yes.

I like that it doesn't omit the undocumented stuff in
the resulting README.md.

3

Peter,

This may be of interest, too:

http://rubygems.org/gems/knife-cookbook-readme

It was recently mentioned in the Food Fight Show newsletter.

-Nathen

From: Peter Donald <peter@realityforge.orgmailto:peter@realityforge.org>
Reply-To: "chef@lists.opscode.commailto:chef@lists.opscode.com" <chef@lists.opscode.commailto:chef@lists.opscode.com>
Date: Tuesday, April 2, 2013 7:14 PM
To: Chef Mailing List <chef@lists.opscode.commailto:chef@lists.opscode.com>
Subject: [chef] Documenting Cookbooks

Hi,

One of the things I have struggled with is documenting our chef cookbooks. Last week I had a few hours to spare so I put together a way of knife plugin that helps generate the README for a cookbook. The basic concept is to markup metadata.rb and the source files and generate documentation based on the annotations, similar to javadoc.

I have written up the first step at [1] but am looking for feedback on the general approach. Step 1 was to generate the README. For step 2 I am thinking of allowing you to point the tool at a chef server (or a local repository) and generate documentation for the whole set of cookbooks and thus generate the documentation for the suite (either as PDF or html).

So is this of interest to anyone else? Anyone have opinions on the best way to do this?

[1] http://realityforge.org/code/2013/04/01/documenting-cookbooks.html

Cheers,

Peter Donald

4

This may be of interest, too:

knife-cookbook-readme | RubyGems.org | your community gem host

It's based on that.

http://realityforge.org/code/2013/04/01/documenting-cookbooks.html

"Mathias Lafeldt wrote a knife plugin [knife-cookbook-readme]
that generates an initial README.md from the metadata.rb file
in a a cookbook.
...
So I decided to try and extend Mathias's work..."

Jeff (not the "I" above)

5

This is great

On Tue, Apr 2, 2013 at 6:00 PM, Jeff Blaine jblaine@kickflop.net wrote:

This may be of interest, too:

http://rubygems.org/gems/**knife-cookbook-readmehttp://rubygems.org/gems/knife-cookbook-readme

It's based on that.

http://realityforge.org/code/**2013/04/01/documenting-**cookbooks.htmlhttp://realityforge.org/code/2013/04/01/documenting-cookbooks.html

"Mathias Lafeldt wrote a knife plugin [knife-cookbook-readme]
that generates an initial README.md from the metadata.rb file
in a a cookbook.
...
So I decided to try and extend Mathias's work..."

Jeff (not the "I" above)

6

Have we all forgotten about yard-chef?

On Wednesday, April 3, 2013, Jesse Nelson wrote:

This is great

On Tue, Apr 2, 2013 at 6:00 PM, Jeff Blaine <jblaine@kickflop.net<javascript:_e({}, 'cvml', 'jblaine@kickflop.net');>

wrote:

This may be of interest, too:

http://rubygems.org/gems/**knife-cookbook-readmehttp://rubygems.org/gems/knife-cookbook-readme

It's based on that.

http://realityforge.org/code/**2013/04/01/documenting-**cookbooks.htmlhttp://realityforge.org/code/2013/04/01/documenting-cookbooks.html

"Mathias Lafeldt wrote a knife plugin [knife-cookbook-readme]
that generates an initial README.md from the metadata.rb file
in a a cookbook.
...
So I decided to try and extend Mathias's work..."

Jeff (not the "I" above)

7

Hi

I do not havea lot ofcookbooks, but a few big ones. Therefore I use a self written small ruby programm. My opinion was to generate a single markdown file for every text file in a cookbook within the markdown blocks like:

#begin md

#First heading

Some text.

* list1

…

#end md

because I think in a realy huge cookbook it is not a good idea to write all explanations to one big readme file. Anotherreason for thisis that sometimese.g.several equivalentresources, are stackedandyou do nothave to describeeach individually. By that it is possible to describethebehavior, less than the sysntax of a recipe, lwrp etc.

Moreover, it isalsopossiblewith this methodto addnormalcommentsfor otherdevelopers.

Nowto extractthe blocks, I usemy program. It searches with regularexpressions theblocks, deletes theadditionalcontentand paste them into a simpletext filewith the extension “.md”.

#First heading

Some text.

  • list1

…

At this point I can convert them with pandoc to html or somthing else.

I knowthat thisbrings manyproblemswith it, but myprogram does the job and I hopethat someone else finds a bettersolution.

So I wouldbe glad to geta gooddocumentationprogramwhich willmeettherequirements.
> So is this of interest to anyone else?

Yes!

A thousand times yes.

I like that it doesn’t omit the undocumented stuff in

the resulting README.md.

8

my $.02 –

I’ve always been a big fan of documenting the code “with the code.” I find
it odd to document the attributes in the metdata.rb rather than the
attributes file(s). Same with recipes, etc.

9

Hi Peter,

On Wed, Apr 3, 2013 at 1:14 AM, Peter Donald peter@realityforge.org wrote:

So is this of interest to anyone else?

Thanks for this. This definitely speaks to a pain point we have as well.

Anyone have opinions on the best way to do this?

I agree with Brian that code should be documented with the code. In
the case of attributes, though, the metadata spec we get from Opscode
doesn't seem to allow for this. So, it would be a big win to be able
to document attributes in the attributes file(s) using the '#<>'
convention, then have the attribute specifications inserted into the
metadata file during the documentation phase (knife cookbook doc).

What do you think? Would this be possible with the plugin?

  • Dimitri
10

Hi Mike,

On Wed, Apr 3, 2013 at 5:02 AM, Mike miketheman@gmail.com wrote:

Have we all forgotten about yard-chef?
GitHub - rightscale-cookbooks/yard-chef

I'd say that yard-chef speaks to a different use-case, doesn't it?
What the knife-cookbook-doc plugin does is "just" generate the README
file. Both tools could be used on the same cookbook in a
complementary fashion: knife-cookbook-doc to generate the README and
yard-chef to generate browsable docs with your choice of template.

  • Dimitri
11

On Wed, Apr 3, 2013 at 8:30 AM, Dimitri Aivaliotis aglarond@gmail.comwrote:

Both tools could be used on the same cookbook in a
complementary fashion: knife-cookbook-doc to generate the README and
yard-chef to generate browsable docs with your choice of template.

I've never used Yard before. Is the syntax for Yard the same as this
knife-cookbook-doc plugin? It would suck to have to write documentation
twice. :slight_smile:

--
John Alberts

12

It would suck to have to write documentation twice. :slight_smile:

Absolutely true.

So I think the root question is: Where should a cookbook's
documentation live? In its entirety in the README? 1 or in a doc/
directory, like many other software projects, or hell, in the
metadata.rb [2]?

I think a README is a great place to say "hey, here's the thing,
here's some basics, check out the documentation for further details".
It's the Step 1 of using a cookbook, read the README.

Should the README be a comprehensive manual of everything in this
cookbook? I think not. Document the code in the code.

And until Chef Server has anything to do with attributes in
metadata.rb, they are just "spam" and non-canonical, duplicated
attributes that may fall from accuracy/sync as time goes by.

I'm going to look at promoting/working with yard-chef more to achieve
the approach of documentation in the source, which renders html on the
side.

Curious to hear if there are other approaches that are being used.

-Mike

On Wed, Apr 3, 2013 at 9:51 AM, John Alberts john.m.alberts@gmail.com wrote:

On Wed, Apr 3, 2013 at 8:30 AM, Dimitri Aivaliotis aglarond@gmail.com
wrote:

Both tools could be used on the same cookbook in a
complementary fashion: knife-cookbook-doc to generate the README and
yard-chef to generate browsable docs with your choice of template.

I've never used Yard before. Is the syntax for Yard the same as this
knife-cookbook-doc plugin? It would suck to have to write documentation
twice. :slight_smile:

--
John Alberts

13

I'm going to look at promoting/working with yard-chef more to achieve
the approach of documentation in the source, which renders html on the
side.

"YARD does not impose a specific markup. ... YARD also supports
textile and markdown via the command-line switch or .yardopts file"

Sounds good to me.

Above all, I think it's essential to the Chef community/ecosystem
that there's 1 acknowledged and recommended way to do cookbook
documentation.

14

On Thu, Apr 4, 2013 at 7:42 AM, Mike miketheman@gmail.com wrote:

And until Chef Server has anything to do with attributes in
metadata.rb, they are just "spam" and non-canonical, duplicated
attributes that may fall from accuracy/sync as time goes by.

+1

This is why I'm -1 on any documentation "standard" that says put the
documentation in some place other than in the attributes file. I've seen
cookbooks here and there that have "documentation" in metadata.rb that has
drifted from the attributes file(s).

15

Does anyone have any examples or tutorials for using yard-chef? I glanced
at some of the rightscale cookbooks, since they created yard-chef, and I
don't see them using it in any of their cookbooks that I looked at.

On Fri, Apr 5, 2013 at 5:46 AM, Brian Akins brian@akins.org wrote:

On Thu, Apr 4, 2013 at 7:42 AM, Mike miketheman@gmail.com wrote:

And until Chef Server has anything to do with attributes in
metadata.rb, they are just "spam" and non-canonical, duplicated
attributes that may fall from accuracy/sync as time goes by.

+1

This is why I'm -1 on any documentation "standard" that says put the
documentation in some place other than in the attributes file. I've seen
cookbooks here and there that have "documentation" in metadata.rb that has
drifted from the attributes file(s).

--
John Alberts

16

On Wed, Apr 3, 2013 at 2:02 PM, Mike miketheman@gmail.com wrote:

Have we all forgotten about yard-chef?
GitHub - rightscale-cookbooks/yard-chef

Interesting ... I never knew about it. It does seem a little abandoned or
at least the instructions don't seem to work. It has a more interesting
approach to collecting the metadata which is better in some ways. However
it is still heavily focused on describing the code rather than providing
user-level documentation.

On Wed, Apr 3, 2013 at 5:28 PM, Oleg Volotov oleg.volotov@gmx.de wrote:

Now to extract the blocks, I use my program. It searches with regular
expressions the blocks, deletes the additional content and paste them
into a simple text file with the extension ".md".

The approach I was using is not that much more sophisticated than that :slight_smile:

On Wed, Apr 3, 2013 at 9:57 PM, Brian Akins brian@akins.org wrote:

I've always been a big fan of documenting the code "with the code." I
find it odd to document the attributes in the metadata.rb rather than the
attributes file(s). Same with recipes, etc.

I agree. An early version of the plugin actually did this by executing the
attribute files. I ran into problems where some of the cookbooks actually
had logic in them - particularly when interacting with ohai data and this
was difficult to mock out. In theory you could go back to a regex/extract
approach but that also has it's own problems.

On Thu, Apr 4, 2013 at 12:13 AM, Dimitri Aivaliotis aglarond@gmail.com
wrote:

What do you think? Would this be possible with the plugin?

My plan was to have an foodcritic plugin that verified the metadata.rb from
the attributes file but it would certainly be possible to update the
metadata.rb from the source. You could also add in the authoritative list
of recipes based on scanning the source ...

On Fri, Apr 5, 2013 at 12:27 AM, Jeff Blaine jblaine@kickflop.net wrote:

Above all, I think it's essential to the Chef community/ecosystem
that there's 1 acknowledged and recommended way to do cookbook
documentation.

+1

--
Cheers,

Peter Donald

17

On Sat, Apr 6, 2013 at 4:23 AM, Peter Donald peter@realityforge.org wrote:

On Wed, Apr 3, 2013 at 9:57 PM, Brian Akins brian@akins.org wrote:

I've always been a big fan of documenting the code "with the code." I
find it odd to document the attributes in the metadata.rb rather than the
attributes file(s). Same with recipes, etc.

I agree. An early version of the plugin actually did this by executing the
attribute files. I ran into problems where some of the cookbooks actually
had logic in them - particularly when interacting with ohai data and this
was difficult to mock out. In theory you could go back to a regex/extract
approach but that also has it's own problems.

Maybe you can mock ohai using GitHub - chefspec/fauxhai: Easily mock full ohai data?

Cheers, Torben

--

Cheers,

Peter Donald

18

I've used it successfully by following the README. However, if you don't
populate your cookbooks' metadata.rb, you won't see anything exciting.

I really like the idea of yard-chef, but in order for it to become truly,
indispensably awesome we'd need for it to document attributes from a
comment immediately preceding the first mention of the attribute and to
parse a recipe's commented preamble as a docstring. Those both seem to be
pretty common patterns of usage both inside and outside our organization.

(and I'm another believer in making the code conform to what people are
doing instead of the other way around, whenever possible ... :smiley: )

On Fri, Apr 5, 2013 at 7:20 AM, John Alberts john.m.alberts@gmail.comwrote:

Does anyone have any examples or tutorials for using yard-chef? I glanced
at some of the rightscale cookbooks, since they created yard-chef, and I
don't see them using it in any of their cookbooks that I looked at.

On Fri, Apr 5, 2013 at 5:46 AM, Brian Akins brian@akins.org wrote:

On Thu, Apr 4, 2013 at 7:42 AM, Mike miketheman@gmail.com wrote:

And until Chef Server has anything to do with attributes in
metadata.rb, they are just "spam" and non-canonical, duplicated
attributes that may fall from accuracy/sync as time goes by.

+1

This is why I'm -1 on any documentation "standard" that says put the
documentation in some place other than in the attributes file. I've seen
cookbooks here and there that have "documentation" in metadata.rb that has
drifted from the attributes file(s).

--
John Alberts