Attribute Validation with chef-attribute-validator


#1

Hello all,
At the Chef Community Summit 3 today, I presented a mechanism for defining,
evaluating, and handling validation rules for node attributes. It’s packaged
as a rubygem (http://github.com/clintoncwolfe/chef-attribute-validator) which
implements the rules engine, and a thin cookbook
(http://github.com/clintoncwolfe/attribute-validator) which installs the gem
and then runs the validation at either compile-time or convergence time,
depending what the user places in their runlist. The gem may also be used
directly, either from recipes or from CI tooling, etc.

Rules are defined as attributes themselves, so it’s easy for an
implementation cookbook to define validation rules right alongside the defaults
for the attributes. Rules use wildcarded paths (using shell glob wildcards) to
select attributes, and have a number of checks to select from (type, regex,
etc…).

The code is fairly young but well-tested; there are some missing features,
but it’s useful now. I’d welcome any feedback/suggestions, here or as github
issues. And collaborators welcome :).

There was also some discussion about incorporating the functionality into
chef core. I’d generally welcome that, though I think we should allow the
gem/cookbook approach to mature a bit to see how the functionality evolves.

One particular suggestion was to examine the (rule-defining) cookbook’s
metadata.rb, and look at the ‘attribute’ DSL. In discussion, the consensus
seemed to be that the attribute DSL was intended to define validation rules,
but nothing was enforcing them. To me, reading the metadata DSL docs
(http://docs.opscode.com/config_rb_metadata.html), the actual intent wasn’t
clear; it seemed like the attribute properties might actually be intended to
generate a UI to set attributes.

So: should the chef-attribute-validator gem try to read rules from
metadata.rb (in addition to, or instead of, reading the rules from attributes)?

If the metadata.rb is the authoritative source of validation rules, should it
be extended to somehow allow wildcarding?

– Clinton Wolfe


#2

On Tuesday, November 12, 2013 at 9:27 PM, Clinton Wolfe wrote:

Hello all,
At the Chef Community Summit 3 today, I presented a mechanism for defining,
evaluating, and handling validation rules for node attributes.

(snipped)

One particular suggestion was to examine the (rule-defining) cookbook’s
metadata.rb, and look at the ‘attribute’ DSL. In discussion, the consensus
seemed to be that the attribute DSL was intended to define validation rules,
but nothing was enforcing them. To me, reading the metadata DSL docs
(http://docs.opscode.com/config_rb_metadata.html), the actual intent wasn’t
clear; it seemed like the attribute properties might actually be intended to
generate a UI to set attributes.

So: should the chef-attribute-validator gem try to read rules from
metadata.rb (in addition to, or instead of, reading the rules from attributes)?

If the metadata.rb is the authoritative source of validation rules, should it
be extended to somehow allow wildcarding?

I think this could be pretty difficult. When cookbook metadata was originally created, only the “depends” field was actually implemented in Chef. Other fields have been used by various applications, in WebUIs and the like. But! Since chef doesn’t use these fields or validate them (and in some cases doesn’t really define what they should contain), in the wild you’ll find all manner of things in there. In a major version bump (Chef 12.0), it’s acceptable to change that, if it’s the best solution. Even so, if there’s a different option that doesn’t break compatibility, I think that would be preferred since we like to get everyone on the new hotness as soon as possible.

Finally, my own inclination is that it’s better to define these things in code. As a major sticking point, there’s no “proc” type in JSON (how could there be?), but that would give you the most sane path to implementing validations that the framework doesn’t support yet. For example, if the framework doesn’t support integer ranges, I can easily write a bit of code to check if a number is in a range (so it’s a valid port number, say). With static data, such a thing wouldn’t be possible.

– Clinton Wolfe


Daniel DeLeo


#3

Finally, my own inclination is that it’s better to define these things in code. As a major sticking point, there’s no “proc” type in JSON (how could there be?), but that would give you the most sane
path to implementing validations that the framework doesn’t support yet. For example, if the framework doesn’t support integer ranges, I can easily write a bit of code to check if a number is in a
range (so it’s a valid port number, say). With static data, such a thing wouldn’t be possible.

This is my preference as well. I’d prefer to define rules either very
close to where attribute defaults are set (in attribute files
typically) or where they are used. We discussed the Proc-type
validation in the summit meeting and I think it is vital to make this
feature extensible.

On Fri, Nov 15, 2013 at 9:29 AM, Daniel DeLeo dan@kallistec.com wrote:

On Tuesday, November 12, 2013 at 9:27 PM, Clinton Wolfe wrote:

Hello all,
At the Chef Community Summit 3 today, I presented a mechanism for defining,
evaluating, and handling validation rules for node attributes.

(snipped)

One particular suggestion was to examine the (rule-defining) cookbook’s
metadata.rb, and look at the ‘attribute’ DSL. In discussion, the consensus
seemed to be that the attribute DSL was intended to define validation rules,
but nothing was enforcing them. To me, reading the metadata DSL docs
(http://docs.opscode.com/config_rb_metadata.html), the actual intent wasn’t
clear; it seemed like the attribute properties might actually be intended to
generate a UI to set attributes.

So: should the chef-attribute-validator gem try to read rules from
metadata.rb (in addition to, or instead of, reading the rules from
attributes)?

If the metadata.rb is the authoritative source of validation rules, should
it
be extended to somehow allow wildcarding?

I think this could be pretty difficult. When cookbook metadata was
originally created, only the “depends” field was actually implemented in
Chef. Other fields have been used by various applications, in WebUIs and the
like. But! Since chef doesn’t use these fields or validate them (and in some
cases doesn’t really define what they should contain), in the wild you’ll
find all manner of things in there. In a major version bump (Chef 12.0),
it’s acceptable to change that, if it’s the best solution. Even so, if
there’s a different option that doesn’t break compatibility, I think that
would be preferred since we like to get everyone on the new hotness as soon
as possible.

Finally, my own inclination is that it’s better to define these things in
code. As a major sticking point, there’s no “proc” type in JSON (how could
there be?), but that would give you the most sane path to implementing
validations that the framework doesn’t support yet. For example, if the
framework doesn’t support integer ranges, I can easily write a bit of code
to check if a number is in a range (so it’s a valid port number, say). With
static data, such a thing wouldn’t be possible.

– Clinton Wolfe


Daniel DeLeo


#4

On Fri, Nov 15, 2013 at 11:29 AM, Daniel DeLeo dan@kallistec.com wrote:

Finally, my own inclination is that it’s better to define these things in
code. As a major sticking point, there’s no “proc” type in JSON (how could
there be?), but that would give you the most sane path to implementing
validations that the framework doesn’t support yet. For example, if the
framework doesn’t support integer ranges, I can easily write a bit of code
to check if a number is in a range (so it’s a valid port number, say). With
static data, such a thing wouldn’t be possible.

Not suggesting we use it, but JSON Schema does support things like ranges
and other attributes, though it still doesn’t let you use code:
http://json-schema.org/latest/json-schema-validation.html#anchor12

Nathan L Smith

smith@opscode.com


#5

FYI, I added proc support in v0.3.0 of the chef-attribute-validator gem, a
mere 2 days ago :slight_smile:

Sounds like reading metadata.rb for attribute validation rules is fraught
with peril. I’ll leave it be, then.

On Fri, Nov 15, 2013 at 12:46 PM, Nathan Smith smith@opscode.com wrote:

On Fri, Nov 15, 2013 at 11:29 AM, Daniel DeLeo dan@kallistec.com wrote:

Finally, my own inclination is that it’s better to define these things in
code. As a major sticking point, there’s no “proc” type in JSON (how could
there be?), but that would give you the most sane path to implementing
validations that the framework doesn’t support yet. For example, if the
framework doesn’t support integer ranges, I can easily write a bit of code
to check if a number is in a range (so it’s a valid port number, say). With
static data, such a thing wouldn’t be possible.

Not suggesting we use it, but JSON Schema does support things like ranges
and other attributes, though it still doesn’t let you use code:
http://json-schema.org/latest/json-schema-validation.html#anchor12

Nathan L Smith

smith@opscode.com