Run lists and dependencies (Was: chef-client , apply role)

Chef Infra (archive)
1

(changed the subject to fork the conversation)

In the described circumstance, I still want things that must exist on the
box for the application to run to be on the dependency list, not the run
list.

Why not just use an LWRP in each cookbook to add file system mounts as
opposed to building attributes and praying someone else mounts what you
need? Alternately you can build a list of attributes across all your
cookbooks during compile time and use those attributes during convergence
when you perform a mount. Separating requirements like you describe adds
another possibility of failure if the run list changes between versions,
that being forgetting to set the run list during an upgrade. When managing
multiple data centers with many engineers touching deployments, forgetting
that is a very real possibility, even with run books. Further, run lists
are not versioned so if you ever make a mistake, it's up to you to remember
what the run list needed to look like, more than just remembering what
component lives on that box.

In my opinion, multiple entries on a run list means multiple applications
are running on that node. A run list entry should mean that the entirety of
what is required for that app to start and function in a default state will
be deployed. Any change to that default state should be set in attributes
in the environment or on the node.

On Fri, Jan 3, 2014 at 6:01 AM, Dylan Northrup chef@doc-x.net wrote:

Not to digress too much, but I use explicit dependencies from metadata.rb
in individual cookbooks (and, less often, roles and environments... at
least until they are able to be versioned) when they're required by the
app/service. But for other, more general "things that should be on a host"
cookbooks, those may gather information from other cookbooks in the run
list to determine how to proceed. There's an implicit dependency in those
more general cookbooks that guides my suggestion to run a full chef-client
run including every cookbook in the run_list.

As an example, let's say I've got multiple apps with multiple development
groups working on those individual apps. Apps are set up using the
individual cookbooks, but I want to get user accounts, sudoers, etc set up
using a single 'users' cookbook. The app cookbooks can set node attributes
that will be read by the users cookbook to determine what user accounts to
put on the node, who to put in sudoers, etc. If I have a single cookbook
for "mount external file systems", it's conceivable individual cookbooks
would be registering remote file systems to mount and a separate cookbook
in the run list (either explicitly or included) would do the actual
mounting of all remote file systems.

If you flip this around and say each app cookbook should manage file
system mounts (something reasonable, I'll admit, given the typically small
number of file system mounts) or adding users (something that quickly gets
out of hand given any reasonably sized shop) that quickly becomes a
maintenance/ops nightmare and possible security issue since there's no way
to insure users are being added correctly, file system mount options are
standardized, etc. Being able to specify requests from multiple different
locations, but have those requests acted upon in a single location allows
for flexibility while maintaining ease of updates and increasing ease of
debugging.

Digression regarding dependencies aside, I think the idea of only running
a single cookbook on a node is something that was not intended. If you
only want a single cookbook to be run on a node, then that's the only thing
that should be in the run_list, because next time chef-client runs, it'll
run everything in the run_list anyway. If you want to make sure your
results after adding a cookbook to the run_list, you'll need to have the
chef-client run execute every cookbook if you want to make sure your test
results will be consistent with what will eventually happen with production.

But that's just my opinion. I could be wrong.

On Fri, Jan 3, 2014 at 2:14 AM, Josiah Kiehl bluepojo@gmail.com wrote:

If you have dependencies on other entries on your run list, you're using
run lists in a way they are not intended to be used.

Dependencies should be pulled in via "depends" not via run list entries.

On Thu, Jan 2, 2014 at 6:18 AM, Dylan Northrup chef@doc-x.net wrote:

If you don't run the entire run_list, you may be missing attributes that
should be needed during the chef-client run. As an example, if your
run_list were 'wrapper' cookbooks (that contain attributes, and then call
other cookbooks), if you had any dependency on those attributes (such as
"what data center is this node in?", "is this node dev, ref or prod?",
"what service is this node providing?", etc) you wouldn't have access to
them.

I'd suggest that there's likely something interesting you're trying to
do, but the way you're trying to go about it will cause more headaches down
the road than you think and there is likely a better way to go about what
you're wanting to do. If you want to simply update node attributes, knife
node edit is one way (though not the easiest to automate). A quick google
search shows 'knife-set-attribute' at
GitHub - amian84/knife-set-attribute: Knife set_attribute is a plugin to knife to change attributes values in your nodes may be another way
(though I've not used it, so YMMV).

But these are still methods of changing the model on the chef server and
you'll still need to do a full chef-client run on the client to render the
change(s) on the node. And, in the end, doing the full chef-client run is
the best practice from a reliability standpoint. When making a change you
want to make sure it's fully and completely rendered onto a node the way it
will be every time chef-client will run in the future. If it doesn't,
there's a pretty large opportunity for you to be surprised after the next
automated chef-client run when something goes boom because of that
attribute change 'that simply updated one thing that was ephemeral and
should have little effect'.

But that's just my opinion. I could be wrong.

On Thu, Jan 2, 2014 at 5:28 AM, Sam Darwin samuel.d.darwin@gmail.comwrote:

is there a way to run:

chef-client -o "role[abc]"

that is, to apply a role without running the entire run_list ?

this should not be ephemeral and have little effect... it should cause
all the
results that having the role in the real run_list would have, update the
chef-server with attributes in a persistent way and everything else.
I
would also add the role to the real run_list . But I don't want to
run all
recipes on the client at that moment.

2

Comments inline.

On Fri, Jan 3, 2014 at 3:18 PM, Josiah Kiehl bluepojo@gmail.com wrote:

(changed the subject to fork the conversation)

In the described circumstance, I still want things that must exist on the
box for the application to run to be on the dependency list, not the run
list.

I don't think we disagree on this. If something is required for an app
to run, it should be a dependency in the run list (via metadata.rb in a
cookbook or via role/environment). I'm simply saying functionality that is
separate from the app, but which might require input from the app should be
in a separate cookbook and, possibly, a separate run_list entry.

Why not just use an LWRP in each cookbook to add file system mounts as
opposed to building attributes and praying someone else mounts what you
need? Alternately you can build a list of attributes across all your
cookbooks during compile time and use those attributes during convergence
when you perform a mount.

The second statement here is what I was trying to convey. If that did not
come across, I apologize for the misunderstanding. In my preferred
scenario, individual cookbooks in the run_list add to attributes about
things they'd like to happen (users to get added to the box, mounts to get
mounted, monitors to be created, etc) and the cookbook that is responsible
for those (either directly via run_list or indirectly via wrapper
cookbook/role/environment) handles the heavy lifting.

Separating requirements like you describe adds another possibility of
failure if the run list changes between versions, that being forgetting to
set the run list during an upgrade. When managing multiple data centers
with many engineers touching deployments, forgetting that is a very real
possibility, even with run books. Further, run lists are not versioned so
if you ever make a mistake, it's up to you to remember what the run list
needed to look like, more than just remembering what component lives on
that box.

I've never had occasion to add or remove entries from the run_list for a
deployed node while versioning unless the explicit purpose of the
versioning is the addition of a new cookbook. I've changed pinned versions
of cookbooks, but not removed them. I'd think that if you're modifying
run_lists by adding or removing entries between versions of an app, I'd
take a look at your process. That might be an anti-pattern that should be
avoided given the ephemeral nature of the run_list and the inability to
version it.

In my opinion, multiple entries on a run list means multiple applications

are running on that node. A run list entry should mean that the entirety of
what is required for that app to start and function in a default state will
be deployed. Any change to that default state should be set in attributes
in the environment or on the node.

I believe each entry on the run list should encapsulate a specific meaning.
For each application on a host, there should be an entry on the run list.
If you're using the same monitoring cookbook across your infrastructure
(and, if you can pass attributes into that cookbook that get parsed to
create monitors on a per-application basis, I would think you would use the
same cookbook for reasons stated in my previous reply), that should be an
entry in the run_list. If you're using a cookbook to create users on your
nodes, that should be an entry in your run_list. If you find there's a
proliferation of "Standard cookbooks we want everywhere", then collect up
those and put them into a standard cookbook that goes into your run_list to
clean things up. But the standard cookbook should be able to take
attributes set by other cookbooks and, if present, do cool things with them.

Hopefully this clears things up a bit.

On Fri, Jan 3, 2014 at 6:01 AM, Dylan Northrup chef@doc-x.net wrote:

Not to digress too much, but I use explicit dependencies from metadata.rb
in individual cookbooks (and, less often, roles and environments... at
least until they are able to be versioned) when they're required by the
app/service. But for other, more general "things that should be on a host"
cookbooks, those may gather information from other cookbooks in the run
list to determine how to proceed. There's an implicit dependency in those
more general cookbooks that guides my suggestion to run a full chef-client
run including every cookbook in the run_list.

As an example, let's say I've got multiple apps with multiple development
groups working on those individual apps. Apps are set up using the
individual cookbooks, but I want to get user accounts, sudoers, etc set up
using a single 'users' cookbook. The app cookbooks can set node attributes
that will be read by the users cookbook to determine what user accounts to
put on the node, who to put in sudoers, etc. If I have a single cookbook
for "mount external file systems", it's conceivable individual cookbooks
would be registering remote file systems to mount and a separate cookbook
in the run list (either explicitly or included) would do the actual
mounting of all remote file systems.

If you flip this around and say each app cookbook should manage file
system mounts (something reasonable, I'll admit, given the typically small
number of file system mounts) or adding users (something that quickly gets
out of hand given any reasonably sized shop) that quickly becomes a
maintenance/ops nightmare and possible security issue since there's no way
to insure users are being added correctly, file system mount options are
standardized, etc. Being able to specify requests from multiple different
locations, but have those requests acted upon in a single location allows
for flexibility while maintaining ease of updates and increasing ease of
debugging.

Digression regarding dependencies aside, I think the idea of only running
a single cookbook on a node is something that was not intended. If you
only want a single cookbook to be run on a node, then that's the only thing
that should be in the run_list, because next time chef-client runs, it'll
run everything in the run_list anyway. If you want to make sure your
results after adding a cookbook to the run_list, you'll need to have the
chef-client run execute every cookbook if you want to make sure your test
results will be consistent with what will eventually happen with production.

But that's just my opinion. I could be wrong.

On Fri, Jan 3, 2014 at 2:14 AM, Josiah Kiehl bluepojo@gmail.com wrote:

If you have dependencies on other entries on your run list, you're using
run lists in a way they are not intended to be used.

Dependencies should be pulled in via "depends" not via run list entries.

On Thu, Jan 2, 2014 at 6:18 AM, Dylan Northrup chef@doc-x.net wrote:

If you don't run the entire run_list, you may be missing attributes
that should be needed during the chef-client run. As an example, if your
run_list were 'wrapper' cookbooks (that contain attributes, and then call
other cookbooks), if you had any dependency on those attributes (such as
"what data center is this node in?", "is this node dev, ref or prod?",
"what service is this node providing?", etc) you wouldn't have access to
them.

I'd suggest that there's likely something interesting you're trying to
do, but the way you're trying to go about it will cause more headaches down
the road than you think and there is likely a better way to go about what
you're wanting to do. If you want to simply update node attributes, knife
node edit is one way (though not the easiest to automate). A quick google
search shows 'knife-set-attribute' at
GitHub - amian84/knife-set-attribute: Knife set_attribute is a plugin to knife to change attributes values in your nodes may be another way
(though I've not used it, so YMMV).

But these are still methods of changing the model on the chef server
and you'll still need to do a full chef-client run on the client to render
the change(s) on the node. And, in the end, doing the full chef-client run
is the best practice from a reliability standpoint. When making a change
you want to make sure it's fully and completely rendered onto a node the
way it will be every time chef-client will run in the future. If it
doesn't, there's a pretty large opportunity for you to be surprised after
the next automated chef-client run when something goes boom because of that
attribute change 'that simply updated one thing that was ephemeral and
should have little effect'.

But that's just my opinion. I could be wrong.

On Thu, Jan 2, 2014 at 5:28 AM, Sam Darwin samuel.d.darwin@gmail.comwrote:

is there a way to run:

chef-client -o "role[abc]"

that is, to apply a role without running the entire run_list ?

this should not be ephemeral and have little effect... it should cause
all the
results that having the role in the real run_list would have, update
the
chef-server with attributes in a persistent way and everything else.
I
would also add the role to the real run_list . But I don't want to
run all
recipes on the client at that moment.