System configuration tool comparison

Chef Infra (archive)
1

Hi,

Since there are a lot of system configuration tools (both commercial
and open-source), it is difficult to know what tool if right for you.
Therefore, we developed a framework that you can use to compare system
configuration tools. This framework, together with a set of 11 tools
that are evaluated using this framework will be published in a paper
and presented at the LISA’10 conference (http://www.usenix.org/events/
lisa10).

Since Chef is a well known tool within the system administration
community, we included it in our comparison. We also set up a website
that accompanies the paper. On this website, you will find the full
evaluation of Chef at http://sysconfigtools.cer-wiki.info/tool/chef
My question to you: if you want to help us making this website and our
paper better, please review Chef (or other tools you are familiar
with) and provide us with comments, either by email or using the
comment-facility on the website.

The website URL is temporary, we are going to migrate this website a
’official’ university URL. You will see that for each tool, there
exist a large number of properties that are evaluated. Clicking on
these properties gives you an overlay with more information about that
property.

Note that the deadline for the paper submission is the 24th of august.
Comments that might trigger chances in the paper should be received
before that date. We are going to maintain this website, so comments
are also welcome after the 24th of august.

Thank you very much in advance

Kind Regards

–
Bart Vanbrabant
Doctoral Researcher
DistriNet, dept. Computer Science, K.U.Leuven Belgium

Thomas Delaet
Post-doctoral Researcher
DistriNet, dept. Computer Science, K.U.Leuven Belgium

Disclaimer: http://www.kuleuven.be/cwis/email_disclaimer.htm

2

Hi,

2010/8/13 Bart Vanbrabant bart.vanbrabant@cs.kuleuven.be

Since Chef is a well known tool within the system administration
community, we included it in our comparison. We also set up a website
that accompanies the paper. On this website, you will find the full
evaluation of Chef at http://sysconfigtools.cer-wiki.info/tool/chef
My question to you: if you want to help us making this website and our
paper better, please review Chef (or other tools you are familiar
with) and provide us with comments, either by email or using the
comment-facility on the website.

Sounds very interesting ! Do you have a precise grid to evaluate subjective
criterias ? There are a couple of things I would answer by a "it depends on
how you implement it" or just "it depends". How will you address this kind
of question ?

The main one is a common troll about Chef : for the moment you evaluate the
"Ease of use" to "hard" because it's a ruby DSL. But in the real world, the
lambda sysadmin don't even know he is writing ruby. In most cases, the
syntax is much much more simple than Cfengine for instance.

It has been an argument among others to choose puppet over chef in my
organization. But no one seem to understand that tools with a so-called
"specialized configuration syntax" always end up in re-inventing the wheel
(see on puppet MLs the endless discussions about how variables are scoped,
about if/then/else syntax, etc). Anyway it's just an example, it's not
totally true with Puppet 2.6, which now has a real builtin ruby DSL to write
recipes.

I don't want to start a new troll at all, I'd rather like to understand how
people address those questions and compare things. In my opinion, the only
good answer to such questions is either "it depends on your needs and
situation", or much more factual metrics, but there would be a lot to take
into account, with little chance we have clearer ideas at the end.

Does anybody here has a solid "sales argument" for that ?

Cheers

Jean-Baptiste

3

Hi Bart!

Thank you for including Chef in your comparison!

As Chef is an actively developed project, the version you reviewed, 0.8.8, is at this point 5 months old. We have had a 'major' release (0.9.0) in that time and several point releases (current is 0.9.8). New features for consideration in your evaluation:

  • Windows is now a supported platform (as of 0.8.14), and gets better with each release.
  • The Opscode Platform (our commercial product offering) supports strong role-based access control (as of 0.9.0). This is API based rather than system-user based.

Also a couple clarifications on specific points on the list:

  • User interface, the web interface is mainly used for management of components. Reporting is a feature we may develop more in the future.
  • Deployment architecture, when used client/server that is true (centralized, pull), though Chef can also be used in a standalone mode called Chef Solo.
  • Monitoring the infrastructure can be done a variety of ways (status from knife or the webui, exception/reporting handlers, querying the server data/search indexes). Also, Opscode provides a few cookbooks for configuring monitoring tools (notably nagios and munin).
  • Versioning support is in early phases for cookbooks uploaded to a Chef Server, and will be fully developed in an upcoming release.
  • Available documentation should also include that Opscode provides open training materials that are under a creative commons license.

Feel free to ask if you have any further questions.

On Aug 13, 2010, at 2:56 AM, Bart Vanbrabant wrote:

Hi,

Since there are a lot of system configuration tools (both commercial
and open-source), it is difficult to know what tool if right for you.
Therefore, we developed a framework that you can use to compare system
configuration tools. This framework, together with a set of 11 tools
that are evaluated using this framework will be published in a paper
and presented at the LISA'10 conference (http://www.usenix.org/events/
lisa10).

Since Chef is a well known tool within the system administration
community, we included it in our comparison. We also set up a website
that accompanies the paper. On this website, you will find the full
evaluation of Chef at http://sysconfigtools.cer-wiki.info/tool/chef
My question to you: if you want to help us making this website and our
paper better, please review Chef (or other tools you are familiar
with) and provide us with comments, either by email or using the
comment-facility on the website.

The website URL is temporary, we are going to migrate this website a
'official' university URL. You will see that for each tool, there
exist a large number of properties that are evaluated. Clicking on
these properties gives you an overlay with more information about that
property.

Note that the deadline for the paper submission is the 24th of august.
Comments that might trigger chances in the paper should be received
before that date. We are going to maintain this website, so comments
are also welcome after the 24th of august.

Thank you very much in advance

Kind Regards

--
Bart Vanbrabant
Doctoral Researcher
DistriNet, dept. Computer Science, K.U.Leuven Belgium

Thomas Delaet
Post-doctoral Researcher
DistriNet, dept. Computer Science, K.U.Leuven Belgium

Disclaimer: KU Leuven disclaimer voor e-mails - KU Leuven

--
Opscode, Inc
Joshua Timberman, Technical Evangelist
C: 720.334.RUBY E: joshua@opscode.com

4

Hi,

I have a question about the access control. To what can access control
be applied? Is this fine grained per parameter or very coarse per
device?

I'll update the chef page based on your feedback.

gr,

Bart

On Sat, 2010-08-14 at 02:35 -0600, Joshua Timberman wrote:

Hi Bart!

Thank you for including Chef in your comparison!

As Chef is an actively developed project, the version you reviewed, 0.8.8, is at this point 5 months old. We have had a 'major' release (0.9.0) in that time and several point releases (current is 0.9.8). New features for consideration in your evaluation:

  • Windows is now a supported platform (as of 0.8.14), and gets better with each release.
  • The Opscode Platform (our commercial product offering) supports strong role-based access control (as of 0.9.0). This is API based rather than system-user based.

Also a couple clarifications on specific points on the list:

  • User interface, the web interface is mainly used for management of components. Reporting is a feature we may develop more in the future.
  • Deployment architecture, when used client/server that is true (centralized, pull), though Chef can also be used in a standalone mode called Chef Solo.
  • Monitoring the infrastructure can be done a variety of ways (status from knife or the webui, exception/reporting handlers, querying the server data/search indexes). Also, Opscode provides a few cookbooks for configuring monitoring tools (notably nagios and munin).
  • Versioning support is in early phases for cookbooks uploaded to a Chef Server, and will be fully developed in an upcoming release.
  • Available documentation should also include that Opscode provides open training materials that are under a creative commons license.

Feel free to ask if you have any further questions.

On Aug 13, 2010, at 2:56 AM, Bart Vanbrabant wrote:

Hi,

Since there are a lot of system configuration tools (both commercial
and open-source), it is difficult to know what tool if right for you.
Therefore, we developed a framework that you can use to compare system
configuration tools. This framework, together with a set of 11 tools
that are evaluated using this framework will be published in a paper
and presented at the LISA'10 conference (Upcoming USENIX Conferences | USENIX
lisa10).

Since Chef is a well known tool within the system administration
community, we included it in our comparison. We also set up a website
that accompanies the paper. On this website, you will find the full
evaluation of Chef at http://sysconfigtools.cer-wiki.info/tool/chef
My question to you: if you want to help us making this website and our
paper better, please review Chef (or other tools you are familiar
with) and provide us with comments, either by email or using the
comment-facility on the website.

The website URL is temporary, we are going to migrate this website a
'official' university URL. You will see that for each tool, there
exist a large number of properties that are evaluated. Clicking on
these properties gives you an overlay with more information about that
property.

Note that the deadline for the paper submission is the 24th of august.
Comments that might trigger chances in the paper should be received
before that date. We are going to maintain this website, so comments
are also welcome after the 24th of august.

Thank you very much in advance

Kind Regards

--
Bart Vanbrabant
Doctoral Researcher
DistriNet, dept. Computer Science, K.U.Leuven Belgium

Thomas Delaet
Post-doctoral Researcher
DistriNet, dept. Computer Science, K.U.Leuven Belgium

Disclaimer: KU Leuven disclaimer voor e-mails - KU Leuven

--
Bart Vanbrabant bart.vanbrabant@cs.kuleuven.be

Disclaimer: KU Leuven disclaimer voor e-mails - KU Leuven

5

Hello,

Just to be clear, the ACLs are currently only available in the commercial Opscode Platform offering, not in the open source Chef Server. The client libraries are open source and unaffected, other than potentially being restricted by ACL.

The access control can be applied to any object managed by Chef.

  • Nodes
  • Clients
  • Roles
  • Cookbooks[0]
  • Data Bags

The permissions can be managed by groups or per-user.

Further documentation for managing permissions in the Opscode Platform Management Console can be found on our help site:

Thanks!

[0]: Permissions can be assigned to specific cookbooks; i.e. user jtimberman can access the 'apache2' cookbook but not the 'mysql' cookbook.

On Aug 16, 2010, at 2:30 AM, Bart Vanbrabant wrote:

I have a question about the access control. To what can access control
be applied? Is this fine grained per parameter or very coarse per
device?

I'll update the chef page based on your feedback.

--
Opscode, Inc
Joshua Timberman, Technical Evangelist
C: 720.334.RUBY E: joshua@opscode.com

6

And to be very clear, access to each of those kinds of objects (and others
not listed but probably not important for your purposes) is separated into
CREATE, READ, UPDATE, DELETE, and GRANT (required to view or change the
object's ACLs) Access Control Entries (ACEs) so that a person may have the
ability to read one cookbook but not modify it, while having complete access
to another and no access to yet a third. Each ACE can have groups or actors,
and groups can contain both actors and other groups.

The one sentence description: It's fine-grain RBAC via discretionary ACLs
with (possibly nested) groups.

Hope that helps,
Chris

On Mon, Aug 16, 2010 at 9:35 AM, Joshua Timberman joshua@opscode.comwrote:

Hello,

Just to be clear, the ACLs are currently only available in the commercial
Opscode Platform offering, not in the open source Chef Server. The client
libraries are open source and unaffected, other than potentially being
restricted by ACL.

The access control can be applied to any object managed by Chef.

  • Nodes
  • Clients
  • Roles
  • Cookbooks[0]
  • Data Bags

The permissions can be managed by groups or per-user.

Further documentation for managing permissions in the Opscode Platform
Management Console can be found on our help site:

Chef Support for Automation & DevOps | Chef

Thanks!

[0]: Permissions can be assigned to specific cookbooks; i.e. user
jtimberman can access the 'apache2' cookbook but not the 'mysql' cookbook.

On Aug 16, 2010, at 2:30 AM, Bart Vanbrabant wrote:

I have a question about the access control. To what can access control
be applied? Is this fine grained per parameter or very coarse per
device?

I'll update the chef page based on your feedback.

--
Opscode, Inc
Joshua Timberman, Technical Evangelist
C: 720.334.RUBY E: joshua@opscode.com