My approach is to use knife node upload from file <node>.json whenever I
need to change something on the node. Ideally, I would want to make sure
that whatever I upload from-file must be already commited to version
control for traceability reasons (I’ve seen some approaches to ensure this,
but don’t quite remember them…).
But where do you keep these files? I was considering to put them under
chef-repo/nodes as the chef-repo git repo seems to be the right place for
me. But since there is no such directory in the default chef-repo structure
I was wondering if you guys know a better place for it?
On Mon, Apr 23, 2012 at 07:36, Torben Knerr ukio@gmx.de wrote:
Ohai Chefs,
But where do you keep these files? I was considering to put them under
chef-repo/nodes as the chef-repo git repo seems to be the right place for
me. But since there is no such directory in the default chef-repo structure
I was wondering if you guys know a better place for it?
On Mon, Apr 23, 2012 at 07:36, Torben Knerr ukio@gmx.de wrote:
Ohai Chefs,
But where do you keep these files? I was considering to put them under
chef-repo/nodes as the chef-repo git repo seems to be the right place for
me. But since there is no such directory in the default chef-repo structure
I was wondering if you guys know a better place for it?
@AJ: how do you keep track of modifications to nodes, e.g. who added a new
role or cookbook? Does Chef Server keep track of that?
I was thinking that infrastructure-as-code wise one would keep the nodes in
source control as well, as I would do for databags (and I see good reasons
for doing that). Is this just a matter of taste or did I miss something?
Cheers,
Torben
Am 23.04.2012 10:59 schrieb "AJ Christensen" aj@junglist.gen.nz:
I don't keep these. We only store them for backups when deleting
nodes, in the event that some data required for recovery is persisted
to node data.
I wouldn't recommend the overhead of ensuring nodes are synchronized
from server to disk and back, but perhaps a tool could be built for it
(jenkins?)
Embrace chaos, destroy nodes regularly -- build for failure.
On Mon, Apr 23, 2012 at 07:36, Torben Knerr ukio@gmx.de wrote:
Ohai Chefs,
But where do you keep these files? I was considering to put them under
chef-repo/nodes as the chef-repo git repo seems to be the right place
for
me. But since there is no such directory in the default chef-repo
structure
I was wondering if you guys know a better place for it?
@AJ: how do you keep track of modifications to nodes, e.g. who added a new
role or cookbook? Does Chef Server keep track of that?
I was thinking that infrastructure-as-code wise one would keep the nodes in
source control as well, as I would do for databags (and I see good reasons
for doing that). Is this just a matter of taste or did I miss something?
On Mon, Apr 23, 2012 at 07:36, Torben Knerr ukio@gmx.de wrote:
Ohai Chefs,
But where do you keep these files? I was considering to put them under
chef-repo/nodes as the chef-repo git repo seems to be the right place
for
me. But since there is no such directory in the default chef-repo
structure
I was wondering if you guys know a better place for it?
In the sphere of applications, we often have a codebase and a database.
Only the code is stored in source control. None of the data is stored in
source control, but we safeguard the data by taking frequent backups.
One approach to using Chef, an approach paralleling applications, is as
follows. Cookbooks are always code. Nodes and data-bags are always data.
Some roles and environments qualify as code and some qualify as data; this
varies from role to role and from environment to environment. Keep
cookbooks and the code-ish roles and environments in source control; write
code-ish roles and environments in Ruby. Keep node and data-bag data, along
with the data-ish roles and environments, out of source control and left to
the database; write data-ish roles and environments in JSON. To safeguard
the data, take frequent backups of the database.
What are people's thoughts on this way of using Chef?
On Mon, Apr 23, 2012 at 12:31 PM, Juanje Ojeda Croissier < juanje.ojeda@gmail.com> wrote:
@AJ: how do you keep track of modifications to nodes, e.g. who added a
new
role or cookbook? Does Chef Server keep track of that?
I was thinking that infrastructure-as-code wise one would keep the nodes
in
source control as well, as I would do for databags (and I see good
reasons
for doing that). Is this just a matter of taste or did I miss something?
On Mon, Apr 23, 2012 at 07:36, Torben Knerr ukio@gmx.de wrote:
Ohai Chefs,
But where do you keep these files? I was considering to put them
under
chef-repo/nodes as the chef-repo git repo seems to be the right place
for
me. But since there is no such directory in the default chef-repo
structure
I was wondering if you guys know a better place for it?
@AJ: how do you keep track of modifications to nodes, e.g. who added a
new
role or cookbook? Does Chef Server keep track of that?
I was thinking that infrastructure-as-code wise one would keep the nodes
in
source control as well, as I would do for databags (and I see good
reasons
for doing that). Is this just a matter of taste or did I miss something?
On Mon, Apr 23, 2012 at 07:36, Torben Knerr ukio@gmx.de wrote:
Ohai Chefs,
But where do you keep these files? I was considering to put them
under
chef-repo/nodes as the chef-repo git repo seems to be the right place
for
me. But since there is no such directory in the default chef-repo
structure
I was wondering if you guys know a better place for it?
In the sphere of applications, we often have a codebase and a database.
Only the code is stored in source control. None of the data is stored in
source control, but we safeguard the data by taking frequent backups.
One approach to using Chef, an approach paralleling applications, is as
follows. Cookbooks are always code. Nodes and data-bags are always data.
Some roles and environments qualify as code and some qualify as data; this
varies from role to role and from environment to environment. Keep
cookbooks and the code-ish roles and environments in source control; write
code-ish roles and environments in Ruby. Keep node and data-bag data, along
with the data-ish roles and environments, out of source control and left to
the database; write data-ish roles and environments in JSON. To safeguard
the data, take frequent backups of the database.
Can you give an example of code-ish vs. data-ish
roles/environments/databags?
For me all of them qualify as "code" because they are represented as text
and have the benefit that they can be easily version controlled, diffed and
merged.
What are people's thoughts on this way of using Chef?
On Mon, Apr 23, 2012 at 12:31 PM, Juanje Ojeda Croissier < juanje.ojeda@gmail.com> wrote:
@AJ: how do you keep track of modifications to nodes, e.g. who added a
new
role or cookbook? Does Chef Server keep track of that?
I was thinking that infrastructure-as-code wise one would keep the
nodes in
source control as well, as I would do for databags (and I see good
reasons
for doing that). Is this just a matter of taste or did I miss
something?
On Mon, Apr 23, 2012 at 07:36, Torben Knerr ukio@gmx.de wrote:
Ohai Chefs,
But where do you keep these files? I was considering to put them
under
chef-repo/nodes as the chef-repo git repo seems to be the right
place
for
me. But since there is no such directory in the default chef-repo
structure
I was wondering if you guys know a better place for it?
E.g., roles named "mysql-pair-1", "mysql-pair-2", "mysql-pair-3",
"mysql-pair-4", etc., that are all essentially identical to each other but
have the effect of allowing you to set up multiple mysql pairs. You don't
need every single pair in source control, if they are all essentially the
same. You might have a "mysql-pair" role which is included in each of the
above, and that might be in source control.
On Mon, Apr 23, 2012 at 1:12 PM, Torben Knerr ukio@gmx.de wrote:
In the sphere of applications, we often have a codebase and a database.
Only the code is stored in source control. None of the data is stored in
source control, but we safeguard the data by taking frequent backups.
One approach to using Chef, an approach paralleling applications, is as
follows. Cookbooks are always code. Nodes and data-bags are always data.
Some roles and environments qualify as code and some qualify as data; this
varies from role to role and from environment to environment. Keep
cookbooks and the code-ish roles and environments in source control; write
code-ish roles and environments in Ruby. Keep node and data-bag data, along
with the data-ish roles and environments, out of source control and left to
the database; write data-ish roles and environments in JSON. To safeguard
the data, take frequent backups of the database.
Can you give an example of code-ish vs. data-ish
roles/environments/databags?
For me all of them qualify as "code" because they are represented as text
and have the benefit that they can be easily version controlled, diffed and
merged.
What are people's thoughts on this way of using Chef?
On Mon, Apr 23, 2012 at 12:31 PM, Juanje Ojeda Croissier < juanje.ojeda@gmail.com> wrote:
@AJ: how do you keep track of modifications to nodes, e.g. who added
a new
role or cookbook? Does Chef Server keep track of that?
I was thinking that infrastructure-as-code wise one would keep the
nodes in
source control as well, as I would do for databags (and I see good
reasons
for doing that). Is this just a matter of taste or did I miss
something?
On Mon, Apr 23, 2012 at 07:36, Torben Knerr ukio@gmx.de wrote:
Ohai Chefs,
But where do you keep these files? I was considering to put them
under
chef-repo/nodes as the chef-repo git repo seems to be the right
place
for
me. But since there is no such directory in the default chef-repo
structure
I was wondering if you guys know a better place for it?
@AJ: how do you keep track of modifications to nodes, e.g. who added a new
role or cookbook? Does Chef Server keep track of that?
I was thinking that infrastructure-as-code wise one would keep the nodes in
source control as well, as I would do for databags (and I see good reasons
for doing that). Is this just a matter of taste or did I miss something?
The best-practice-goal is for nodes to be more of a commodity and not
have anything unique about them beyond their run_list. Data should be
stored elsewhere, which saves you the trouble of 1) backing up nodes
and 2) sifting through nodes for their unique data.
The Chef Server doesn't keep an audit trail, but Opscode expects to
ship some features in Chef 11 that would be useful here.
@AJ: how do you keep track of modifications to nodes, e.g. who added a
new
role or cookbook? Does Chef Server keep track of that?
I was thinking that infrastructure-as-code wise one would keep the
nodes in
source control as well, as I would do for databags (and I see good
reasons
for doing that). Is this just a matter of taste or did I miss something?
The best-practice-goal is for nodes to be more of a commodity and not
have anything unique about them beyond their run_list. Data should be
stored elsewhere, which saves you the trouble of 1) backing up nodes
and 2) sifting through nodes for their unique data.
Never thought about backups this way, but it's totally neat that you need
to backup only the data stores, not any app servers or the like anymore...
But what about attribute overrides on node level? Is that something that we
should rather avoid and better do any cookbook configuration on a role
level instead?
The Chef Server doesn't keep an audit trail, but Opscode expects to
ship some features in Chef 11 that would be useful here.
On Tue, Apr 24, 2012 at 3:41 PM, Torben Knerr ukio@gmx.de wrote:
But what about attribute overrides on node level? Is that something that we
should rather avoid and better do any cookbook configuration on a role level
instead?
We have been able to avoid node specific attributes in our
infrastructure. Attributes are all in roles or environments. Sometimes
we store the configuration for a particular application in a databag.
Then our recipes may read the databag (often based on the current
environment) and publish relevant attributes from the databag into the
node - but usually we only do this when we need to use these
attributes in searches from other nodes. So far ... this approach
seems to be working ok.
On Tue, Apr 24, 2012 at 3:41 PM, Torben Knerr ukio@gmx.de wrote:
But what about attribute overrides on node level? Is that something that we
should rather avoid and better do any cookbook configuration on a role level
instead?
We have been able to avoid node specific attributes in our
infrastructure. Attributes are all in roles or environments. Sometimes
we store the configuration for a particular application in a databag.
Then our recipes may read the databag (often based on the current
environment) and publish relevant attributes from the databag into the
node - but usually we only do this when we need to use these
attributes in searches from other nodes. So far ... this approach
seems to be working ok.
On Tue, Apr 24, 2012 at 3:41 PM, Torben Knerr ukio@gmx.de wrote:
But what about attribute overrides on node level? Is that something that we
should rather avoid and better do any cookbook configuration on a role level
instead?
We have been able to avoid node specific attributes in our
infrastructure. Attributes are all in roles or environments. Sometimes
we store the configuration for a particular application in a databag.
Then our recipes may read the databag (often based on the current
environment) and publish relevant attributes from the databag into the
node - but usually we only do this when we need to use these
attributes in searches from other nodes. So far ... this approach
seems to be working ok.
On Tue, Apr 24, 2012 at 1:41 AM, Torben Knerr ukio@gmx.de wrote:
But what about attribute overrides on node level? Is that something that we
should rather avoid and better do any cookbook configuration on a role level
instead?
Generally avoid them. Ask yourself, is this something specific about
this computer, or about what it is doing? Usually the override is
configuring what the server is doing, like setting a configuration
value for your web server. Shouldn't all your web servers be
configured the same way?
Sometimes your computer will be unique, like it has more or less ram
than another and you have to tune -Xmx for a Java VM, or whatever. Let
the automation do the work and upfactor this to the cookbook to choose
this value based on the environment, like the total memory installed
(via ohai) other roles assigned to the node.
The Chef Server doesn't keep an audit trail, but Opscode expects to
ship some features in Chef 11 that would be useful here.
Interested to hear more if you can tell already.
We've got code written that audits actions on the API side, but it is
in the newer server code that is erlang based which we haven't
released yet. We're getting started on the planning portion of that
release over the next couple of months, so it's too early to make any
claims about what specific features will look like.
On Tue, Apr 24, 2012 at 3:41 PM, Torben Knerr ukio@gmx.de wrote:
But what about attribute overrides on node level? Is that something
that we
should rather avoid and better do any cookbook configuration on a role
level
instead?
We have been able to avoid node specific attributes in our
infrastructure. Attributes are all in roles or environments. Sometimes
we store the configuration for a particular application in a databag.
Then our recipes may read the databag (often based on the current
environment) and publish relevant attributes from the databag into the
node - but usually we only do this when we need to use these
attributes in searches from other nodes. So far ... this approach
seems to be working ok.
On Tue, Apr 24, 2012 at 6:48 PM, Bryan McLellan btm@loftninjas.org wrote:
On Tue, Apr 24, 2012 at 1:41 AM, Torben Knerr ukio@gmx.de wrote:
But what about attribute overrides on node level? Is that something that
we
should rather avoid and better do any cookbook configuration on a role
level
instead?
Generally avoid them. Ask yourself, is this something specific about
this computer, or about what it is doing? Usually the override is
configuring what the server is doing, like setting a configuration
value for your web server. Shouldn't all your web servers be
configured the same way?
Sometimes your computer will be unique, like it has more or less ram
than another and you have to tune -Xmx for a Java VM, or whatever. Let
the automation do the work and upfactor this to the cookbook to choose
this value based on the environment, like the total memory installed
(via ohai) other roles assigned to the node.
The Chef Server doesn't keep an audit trail, but Opscode expects to
ship some features in Chef 11 that would be useful here.
Interested to hear more if you can tell already.
We've got code written that audits actions on the API side, but it is
in the newer server code that is erlang based which we haven't
released yet. We're getting started on the planning portion of that
release over the next couple of months, so it's too early to make any
claims about what specific features will look like.
Bryan
Sounds very reasonable and makes totally sense to me. Lesson learned.
