Chef cookbook documentation generator from attributes files


#1

Hello,

I started working on a project to generate README (markdown)
documentation from attributes files:

This project started while working on the openstack cookbooks[1]. The
attributes files there have a lot of comments that would be useful for
users: which attributes are available to be configured and what they do,
how etc. This documentation also has the advantage of being more valid
and up-to-date since it is in the code.

README files don’t get nearly as many changes as the comments in the
attributes files. However, the README is generally the frontpage of a
cookbook which users generally go to read. So it would make sense to
have the documentation mirrored from the attributes files.

chef-attrdoc looks at the attributes/default.rb file, groups blocks of
attribute assignments with the first comment above them and then inserts
all of this into the cookbook README’s Attributes section.

I am aware of other tools which generate documentation from cookbook
filse, but they either do not use the attributes file as a source or
ignore comments.

Here are some examples of generated attributes sections from openstack
cookbooks:


-> https://gist.github.com/mapleoin/6886586


-> https://gist.github.com/mapleoin/6886493

This is a very early release with some known bugs. I’d really like to
know what the wider chef community thinks :slight_smile:

Thanks,
Ionuț

[1] https://wiki.openstack.org/wiki/Chef/GettingStarted#Code


#2

Nice work Ionuț. I like where this is going, especially for the stack forge openstack cookbooks.

John

On Thursday, October 10, 2013 at 8:09 AM, Ionuț Arțăriși wrote:

Hello,

I started working on a project to generate README (markdown)
documentation from attributes files:
https://github.com/mapleoin/chef-attrdoc

This project started while working on the openstack cookbooks[1]. The
attributes files there have a lot of comments that would be useful for
users: which attributes are available to be configured and what they do,
how etc. This documentation also has the advantage of being more valid
and up-to-date since it is in the code.

README files don’t get nearly as many changes as the comments in the
attributes files. However, the README is generally the frontpage of a
cookbook which users generally go to read. So it would make sense to
have the documentation mirrored from the attributes files.

chef-attrdoc looks at the attributes/default.rb file, groups blocks of
attribute assignments with the first comment above them and then inserts
all of this into the cookbook README’s Attributes section.

I am aware of other tools which generate documentation from cookbook
filse, but they either do not use the attributes file as a source or
ignore comments.

Here are some examples of generated attributes sections from openstack
cookbooks:

https://github.com/stackforge/cookbook-openstack-compute/blob/aa42f5c09a445cde7267e4b4d00a6ce893aa481e/attributes/default.rb
-> https://gist.github.com/mapleoin/6886586

https://github.com/stackforge/cookbook-openstack-identity/blob/2e6b8b9c6788ae28fbc362c77c53a51c040b49a6/attributes/default.rb
-> https://gist.github.com/mapleoin/6886493

This is a very early release with some known bugs. I’d really like to
know what the wider chef community thinks :slight_smile:

Thanks,
Ionuț

[1] https://wiki.openstack.org/wiki/Chef/GettingStarted#Code


#3

I added this kind of functionality to yard-chef. I expect to have final
legal approval to submit the pull request to Rightscale any day now. (I
can talk about this since they’ve approved the contribution, they’re just
looking over the CLA now…)

It’s pretty cool - if you point it at a cookbooks directory, it will
actually build up a searchable index of node attributes and add that to the
yardoc site.

Actually, I’ve just realized … I can’t submit a pull request yet (that
Rightscale can legally accept) because I can’t sign the CLA until I hear
back that it’s OK for me to do so, but it’s probably safe for me to put up
on GitHub now…

So, um … I’ll see if I can post that in the next couple hours.

On Thu, Oct 10, 2013 at 9:23 AM, John Dewey john@dewey.ws wrote:

Nice work Ionuț. I like where this is going, especially for the stack
forge openstack cookbooks.

John

On Thursday, October 10, 2013 at 8:09 AM, Ionuț Arțăriși wrote:

Hello,

I started working on a project to generate README (markdown)
documentation from attributes files:
https://github.com/mapleoin/chef-attrdoc

This project started while working on the openstack cookbooks[1]. The
attributes files there have a lot of comments that would be useful for
users: which attributes are available to be configured and what they do,
how etc. This documentation also has the advantage of being more valid
and up-to-date since it is in the code.

README files don’t get nearly as many changes as the comments in the
attributes files. However, the README is generally the frontpage of a
cookbook which users generally go to read. So it would make sense to
have the documentation mirrored from the attributes files.

chef-attrdoc looks at the attributes/default.rb file, groups blocks of
attribute assignments with the first comment above them and then inserts
all of this into the cookbook README’s Attributes section.

I am aware of other tools which generate documentation from cookbook
filse, but they either do not use the attributes file as a source or
ignore comments.

Here are some examples of generated attributes sections from openstack
cookbooks:

https://github.com/stackforge/cookbook-openstack-compute/blob/aa42f5c09a445cde7267e4b4d00a6ce893aa481e/attributes/default.rb
-> https://gist.github.com/mapleoin/6886586

https://github.com/stackforge/cookbook-openstack-identity/blob/2e6b8b9c6788ae28fbc362c77c53a51c040b49a6/attributes/default.rb
-> https://gist.github.com/mapleoin/6886493

This is a very early release with some known bugs. I’d really like to
know what the wider chef community thinks :slight_smile:

Thanks,
Ionuț

[1] https://wiki.openstack.org/wiki/Chef/GettingStarted#Code


#4

Okay, managed to find the right person to do the
forking/permission-setting, so here’s my yard-chef changes with attribute
parsing / documentation generation:

If you’ve already used yard-chef, it should work in exactly the same way,
but now it differentiates between resource attributes and node attributes.

Do let me know if you run into any bugs. I’ve been generating
documentation sites for a body of cookbooks internally for a couple months
with this, so I know it can work.

Again, it is my intention to submit this branch back to Rightscale but I’m
waiting on legal approval before I can sign the CLA. I hope to have it
submitted (and maybe even a new official version released?) before the
community summit next month.

On Thu, Oct 10, 2013 at 1:49 PM, steve . leftathome@gmail.com wrote:

I added this kind of functionality to yard-chef. I expect to have final
legal approval to submit the pull request to Rightscale any day now. (I
can talk about this since they’ve approved the contribution, they’re just
looking over the CLA now…)

It’s pretty cool - if you point it at a cookbooks directory, it will
actually build up a searchable index of node attributes and add that to the
yardoc site.

Actually, I’ve just realized … I can’t submit a pull request yet (that
Rightscale can legally accept) because I can’t sign the CLA until I hear
back that it’s OK for me to do so, but it’s probably safe for me to put up
on GitHub now…

So, um … I’ll see if I can post that in the next couple hours.

On Thu, Oct 10, 2013 at 9:23 AM, John Dewey john@dewey.ws wrote:

Nice work Ionuț. I like where this is going, especially for the stack
forge openstack cookbooks.

John

On Thursday, October 10, 2013 at 8:09 AM, Ionuț Arțăriși wrote:

Hello,

I started working on a project to generate README (markdown)
documentation from attributes files:
https://github.com/mapleoin/chef-attrdoc

This project started while working on the openstack cookbooks[1]. The
attributes files there have a lot of comments that would be useful for
users: which attributes are available to be configured and what they do,
how etc. This documentation also has the advantage of being more valid
and up-to-date since it is in the code.

README files don’t get nearly as many changes as the comments in the
attributes files. However, the README is generally the frontpage of a
cookbook which users generally go to read. So it would make sense to
have the documentation mirrored from the attributes files.

chef-attrdoc looks at the attributes/default.rb file, groups blocks of
attribute assignments with the first comment above them and then inserts
all of this into the cookbook README’s Attributes section.

I am aware of other tools which generate documentation from cookbook
filse, but they either do not use the attributes file as a source or
ignore comments.

Here are some examples of generated attributes sections from openstack
cookbooks:

https://github.com/stackforge/cookbook-openstack-compute/blob/aa42f5c09a445cde7267e4b4d00a6ce893aa481e/attributes/default.rb
-> https://gist.github.com/mapleoin/6886586

https://github.com/stackforge/cookbook-openstack-identity/blob/2e6b8b9c6788ae28fbc362c77c53a51c040b49a6/attributes/default.rb
-> https://gist.github.com/mapleoin/6886493

This is a very early release with some known bugs. I’d really like to
know what the wider chef community thinks :slight_smile:

Thanks,
Ionuț

[1] https://wiki.openstack.org/wiki/Chef/GettingStarted#Code


#5

Hi,

On Fri, Oct 11, 2013 at 2:09 AM, Ionuț Arțăriși iartarisi@suse.cz wrote:

I am aware of other tools which generate documentation from cookbook
filse, but they either do not use the attributes file as a source or
ignore comments.

The “knife cookbook doc” plugin [1] will scan the attributes directory
to generate the README in much the same way. It also scans the recipes
directory to build documentation for recipes from the code, LWRPs as
well. It also allows you to mix in markdown from another location (ie.
doc/overview.md).

You can see an example of the generated readme at [2] from annotated
source files.

[1] https://github.com/realityforge/knife-cookbook-doc
[2] https://github.com/realityforge/chef-glassfish


Cheers,

Peter Donald


#6

On 10/13/2013 05:49 AM, Peter Donald wrote:

Hi,

On Fri, Oct 11, 2013 at 2:09 AM, Ionuț Arțăriși iartarisi@suse.cz wrote:

I am aware of other tools which generate documentation from cookbook
filse, but they either do not use the attributes file as a source or
ignore comments.
The “knife cookbook doc” plugin [1] will scan the attributes directory
to generate the README in much the same way. It also scans the recipes
directory to build documentation for recipes from the code, LWRPs as
well. It also allows you to mix in markdown from another location (ie.
doc/overview.md).

You can see an example of the generated readme at [2] from annotated
source files.

[1] https://github.com/realityforge/knife-cookbook-doc
[2] https://github.com/realityforge/chef-glassfish

That’s cool and I’ve looked at this before, though I was confused by the
additional syntax before seeing the example.

Compared to chef-attrdoc, knife-cookbook-doc:

  • rewrites the whole README file and requires additional template
    files, whereas chef-attrdoc just looks for the Attributes section and
    rewrites that, leaving everything else intact
  • does not support groups of attribute declarations documented at the
    same time; only one-to-one comment to attribute
  • doesn’t handle hashes, the generated output is: Defaults to { ... }
  • requires additional markdown, comment lines have to start with #<>
  • requires knife and all its dependencies which also adds to the
    startup time (the entire doc run is 4 seconds here, vs 0.1 for chef-attrdoc)

It’s not clear which are advantages and which are disadvantages (and I’m
biased) so I’ll just leave this list as is. It would be cool to get some
opinions on these differences :slight_smile:

-Ionuț