CHEF-3580: Do you keep node artifacts in source control?

What goes in them? Anything more than a run list? What attribute
precedence are you using?

If the node exists, would you prefer ‘knife node from file’ to:

  1. fail
  2. overwrite the node (all attributes)
  3. overwrite the node, then expand the run list
  4. merge with the existing node

(Current result is #2)

Why?

What if you weren’t as smart and awesome as you are today?


Bryan McLellan | opscode | technical program manager, open source
© 206.607.7108 | (t) @btmspox | (b) http://blog.loftninjas.org

http://tickets.opscode.com/browse/CHEF-3580

I have never even thought of storing nodes in source control.
I don't know what the value would be, especially since as your code
evolves, you may create new attributes and then your node artifacts
would be out of sync with the ones your code creates.

I'm guessing by the lack of response to this message, not too many
others are doing this either? It seems like there might be a use case
for it, but I haven't thought of one yet.

-Mike Fiedler

On Fri, Nov 2, 2012 at 1:11 PM, Bryan McLellan btm@opscode.com wrote:

What goes in them? Anything more than a run list? What attribute
precedence are you using?

If the node exists, would you prefer 'knife node from file' to:

  1. fail
  2. overwrite the node (all attributes)
  3. overwrite the node, then expand the run list
  4. merge with the existing node

(Current result is #2)

Why?

What if you weren't as smart and awesome as you are today?


Bryan McLellan | opscode | technical program manager, open source
(c) 206.607.7108 | (t) @btmspox | (b) http://blog.loftninjas.org

http://tickets.opscode.com/browse/CHEF-3580

On Fri, Nov 2, 2012 at 6:11 PM, Bryan McLellan btm@opscode.com wrote:

What goes in them? Anything more than a run list? What attribute
precedence are you using?

We do keep nodes in git.
We use GitHub - andreacampi/knife-export-plugins: A set of export plugins for Chef's `knife`. to dump nodes;
that takes care of keeping things predictable enough that a diff is
meaningful.

My only gripe with this are the cookbooks that needlessly set normal
attributes… If we could reduce those to the bare minimum, I'd be a happy
camper.

If the node exists, would you prefer 'knife node from file' to:

  1. fail
  2. overwrite the node (all attributes)
  3. overwrite the node, then expand the run list
  4. merge with the existing node

(Current result is #2)

Why?

What if you weren't as smart and awesome as you are today?

I'm not sure I understand #3… fully expanding the run list, i.e. saving to
the node the list of recipes that happen to appear on the run list, if we
were to expand roles right now?
That doesn't sound terribly useful, it will be out of date as soon as any
role is edited.
But I'm probably wrong and you meant something else.

I don't have a strong preference one way or another, but:

#2 is convenient if you are very rigorous in your workflow. If you are not,
you may well end up losing attributes.
Also, it doesn't really play well with cookbooks that set normal attribute
unless you make sure to commit those (like I do).

I feel like #4 is probably more newbie-friendly.

On Saturday, November 3, 2012 at 4:45 PM, Andrea Campi wrote:

On Fri, Nov 2, 2012 at 6:11 PM, Bryan McLellan <btm@opscode.com (mailto:btm@opscode.com)> wrote:

What goes in them? Anything more than a run list? What attribute
precedence are you using?

We do keep nodes in git.
We use GitHub - andreacampi/knife-export-plugins: A set of export plugins for Chef's `knife`. to dump nodes; that takes care of keeping things predictable enough that a diff is meaningful.

My only gripe with this are the cookbooks that needlessly set normal attributes… If we could reduce those to the bare minimum, I'd be a happy camper.

If the node exists, would you prefer 'knife node from file' to:

  1. fail
  2. overwrite the node (all attributes)
  3. overwrite the node, then expand the run list
  4. merge with the existing node

(Current result is #2)

Why?

What if you weren't as smart and awesome as you are today?

I'm not sure I understand #3… fully expanding the run list, i.e. saving to the node the list of recipes that happen to appear on the run list, if we were to expand roles right now?
That doesn't sound terribly useful, it will be out of date as soon as any role is edited.
But I'm probably wrong and you meant something else.

There's some context missing here. In Chef 11, Node objects internally store default and override attributes from roles and environments separately from attributes from cookbooks. These attributes are applied when the run_list is expanded (Node#expand!). Therefore, it's possible for knife, when creating or editing a node, to expand the run_list, setting the automatic node[:roles] and node[:recipes] attributes along with the attributes sourced from roles and environments. In the case of editing a node, attributes from cookbooks would be lost until you run chef-client on that node.

This could be combined with option #4, so that normal attributes and automatic attributes on the node are retained, but default and override are replaced.

Side note, a longer post on the attribute changes I described will be coming next week sometime.

I don't have a strong preference one way or another, but:

#2 is convenient if you are very rigorous in your workflow. If you are not, you may well end up losing attributes.
Also, it doesn't really play well with cookbooks that set normal attribute unless you make sure to commit those (like I do).

I feel like #4 is probably more newbie-friendly.

--
Daniel DeLeo

On Sat, Nov 3, 2012 at 7:23 PM, Mike miketheman@gmail.com wrote:

I'm guessing by the lack of response to this message, not too many
others are doing this either? It seems like there might be a use case
for it, but I haven't thought of one yet.

I think there's a natural progression that most self-taught systems
administrators go through in regard to our server's identity, myself
included. We usually start off with a couple servers and we give them
somewhat random names that we like. Then as we get more we come up
with naming schemes like roman gods or transformers. We're able to
remember these names well and we can remember that mercury is our smtp
and anti-spam server. Eventually we have so many servers we just want
them all named by function, and then later adding location to the
hostname or a subdomain. In the post-automation days, there's now a
camp of people who prefer to name servers a unique random identifier,
and convey their purpose and location in metadata (and perhaps an
automated DNS name for convenience).

There's more than a couple folks out there with a limited number of
servers who still consider a server a "unique snowflake" and manage
its JSON representation as the source of truth about it. Back to the
other side of the coin, when I worked solely in operations at Opscode,
we kept a "run book" that listed each type of server and the knife
command to re-create it (given our git repository), which doubled as a
nice bit of a DR plan. Servers were meant to be thrown away and
resources were treated as a commodity.

On Sat, Nov 3, 2012 at 7:45 PM, Andrea Campi
andrea.campi@zephirworks.com wrote:

I'm not sure I understand #3… fully expanding the run list, i.e. saving to
the node the list of recipes that happen to appear on the run list, if we
were to expand roles right now?
That doesn't sound terribly useful, it will be out of date as soon as any
role is edited.
But I'm probably wrong and you meant something else.

Some folks who re-create nodes from json regularly as a source of
truth, rather than rebuilding the server, find that their search-based
recipes then begin to fail because the 'roles' and 'recipes' attribute
is not populated and many of their roles are children of the roles
listed on the server. Nothing prevents knife from doing this when it
creates the node object. This is CHEF-3528, btw.

On Sat, Nov 3, 2012 at 8:31 PM, Daniel DeLeo dan@kallistec.com wrote:

There's some context missing here. In Chef 11, Node objects internally store
default and override attributes from roles and environments separately from
attributes from cookbooks. These attributes are applied when the run_list is
expanded (Node#expand!). Therefore, it's possible for knife, when creating
or editing a node, to expand the run_list, setting the automatic
node[:roles] and node[:recipes] attributes along with the attributes sourced
from roles and environments. In the case of editing a node, attributes from
cookbooks would be lost until you run chef-client on that node.

This could be combined with option #4, so that normal attributes and
automatic attributes on the node are retained, but default and override are
replaced.

Right, which is relevant to #3. I was generally hoping to get more
detailed workflow use cases from those who store node objects in their
repositories though.

Bryan