Using chef for our own applications/services' configuration?


#1

(apologies if this has been previously asked & answered; the mailing list
archives are sort of hard to use.)

We’re investigating using chef, among other purposes to better manage
cross-system “application” configuration. Specifically, we have services
from multiple projects running on multiple servers, and at the moment each
project has their own configuration file, despite the fact that there is
configuration that would be best shared among them.

I’m quite sure this could be helped by chef but I’m having a little trouble
completely wrapping my head around how we would do this with chef. Seems
like all of the chef docs & tutorials are focused on provisioning and
configuring services like apache, mysql, etc. (And, as I’ve read elsewhere,
a lot of the chef doc seems somewhat abstract and doesn’t always relate
these abstract concepts to concrete usage.)

Does anybody have any samples/suggestions/etc?

Thanks!

dwh


#2

Denis,

Take a look at the “application” cookbooks folder on github* for an
idea of managing in-house application configurations. We’re using chef
to manage things like log4j.xml files, spring properties files and
other things. The benefit you get from using the same tool to manage
your application configuration that you use for system configuration
is that your application configurations are now infrastructure aware.

Take database servers:
If you have a node in Chef tagged as your master db server, you can
now refer to that information in your application configuration. If
the role of db master moves to another box, your application
configuration will be updated on next run.

On Fri, Feb 25, 2011 at 8:51 AM, Denis Haskin denis@haskinferguson.net wrote:

(apologies if this has been previously asked & answered; the mailing list
archives are sort of hard to use.)
We’re investigating using chef, among other purposes to better manage
cross-system “application” configuration. Specifically, we have services
from multiple projects running on multiple servers, and at the moment each
project has their own configuration file, despite the fact that there is
configuration that would be best shared among them.
I’m quite sure this could be helped by chef but I’m having a little trouble
completely wrapping my head around how we would do this with chef. Seems
like all of the chef docs & tutorials are focused on provisioning and
configuring services like apache, mysql, etc. (And, as I’ve read elsewhere,
a lot of the chef doc seems somewhat abstract and doesn’t always relate
these abstract concepts to concrete usage.)
Does anybody have any samples/suggestions/etc?
Thanks!
dwh


#3

Thanks; I’ll look at those in a little more detail. I thought that was the
right thing to look at but wasn’t sure.

dwh

On Fri, Feb 25, 2011 at 10:15 AM, John E. Vincent (lusis) <lusis.org+
chef-list@gmail.com> wrote:

Denis,

Take a look at the “application” cookbooks folder on github* for an
idea of managing in-house application configurations. We’re using chef
to manage things like log4j.xml files, spring properties files and
other things. The benefit you get from using the same tool to manage
your application configuration that you use for system configuration
is that your application configurations are now infrastructure aware.

Take database servers:
If you have a node in Chef tagged as your master db server, you can
now refer to that information in your application configuration. If
the role of db master moves to another box, your application
configuration will be updated on next run.

On Fri, Feb 25, 2011 at 8:51 AM, Denis Haskin denis@haskinferguson.net
wrote:

(apologies if this has been previously asked & answered; the mailing list
archives are sort of hard to use.)
We’re investigating using chef, among other purposes to better manage
cross-system “application” configuration. Specifically, we have services
from multiple projects running on multiple servers, and at the moment
each
project has their own configuration file, despite the fact that there is
configuration that would be best shared among them.
I’m quite sure this could be helped by chef but I’m having a little
trouble
completely wrapping my head around how we would do this with chef. Seems
like all of the chef docs & tutorials are focused on provisioning and
configuring services like apache, mysql, etc. (And, as I’ve read
elsewhere,
a lot of the chef doc seems somewhat abstract and doesn’t always relate
these abstract concepts to concrete usage.)
Does anybody have any samples/suggestions/etc?
Thanks!
dwh


#4

Dennis,

Welcome to my world. We started that project in earnest last fall. We have a
number of environments and instances of application suites in those
environments and needed desperately to simplify our application
configuration. We, too, came to the conclusion that Chef looked like a good
tool because of it’s hierarchy of configuration and flexibility.

We spent months (literally) wrapping our heads around the problem. That was
the most painful part of the process. What we came out with was a model of
our environments and our data that we could plug into Chef. Do not skimp on
the modelling part. I’ve written some about the process here
http://blog.geeksgonemad.com/2011/01/chef-is-my-documentation.html. And
inside that post is a link to a presentation I’ve made public that discusses
some of our modeling in detail.

I still don’t have anything posted giving examples of our choices of when we
decided to use roles vs. databags, vs other Cheffy things. I’ll post that to
my blog soon-ish as we are in the thick of rolling out our solution right
now. Once I do some demo presentations internally I’ll have some material to
post. We’re using it to manage java properties and logging files, tomcat
server.xml, apache vhosts, any configuration file that any piece of our
application needs. We are even using Chef to create our Control Tier project
xml files.

I will warn you that one constraint of Chef may or may not be a significant
impact to how you manage and deploy configurations. Because Chef is purely a
convergence tool, you cannot manage configuration files individually. There
is only one giant runlist that will have to make every configuration file
every run. (see my thread earlier on this mailing list about ad-hock
runlists http://lists.opscode.com/sympa/arc/chef/2011-01/msg00130.html). I
am conspiring with DTO Solutions (Control Tier and Rundeck) to work out ways
of getting Chef to behave more like direct control than convergence. Our
compromise is that we have built a process where Chef will create all
configuration files but not in a live location and we will have some other
piece of orchestration software deploy the necessary configuration files
when the time is right.

John’s post came in as I’m writing this. I agree that some of the greatest
benefit we are seeing is that our configuration is now data-driven and as
much as possible derivable. Having system and application configuration in
one place is a really great thing.

Dan

On Fri, Feb 25, 2011 at 8:51 AM, Denis Haskin denis@haskinferguson.netwrote:

(apologies if this has been previously asked & answered; the mailing list
archives are sort of hard to use.)

We’re investigating using chef, among other purposes to better manage
cross-system “application” configuration. Specifically, we have services
from multiple projects running on multiple servers, and at the moment each
project has their own configuration file, despite the fact that there is
configuration that would be best shared among them.

I’m quite sure this could be helped by chef but I’m having a little trouble
completely wrapping my head around how we would do this with chef. Seems
like all of the chef docs & tutorials are focused on provisioning and
configuring services like apache, mysql, etc. (And, as I’ve read elsewhere,
a lot of the chef doc seems somewhat abstract and doesn’t always relate
these abstract concepts to concrete usage.)

Does anybody have any samples/suggestions/etc?

Thanks!

dwh


#5

Dan – superb, and thanks for the caveats and heads-ups. We’ll look at your
doc & posts and hopefully can stand on your shoulders to not spend months on
it…

I don’t think the convergence issue (as you call it) should be a problem
for us; we’re already moving toward consolidated config files anyway (if I
understood you correctly).

I’m sure we’ll be back with more questions. Thanks!

dwh

On Fri, Feb 25, 2011 at 10:26 AM, Dan Nemec dan@nemecfamily.com wrote:

Dennis,

Welcome to my world. We started that project in earnest last fall. We have
a number of environments and instances of application suites in those
environments and needed desperately to simplify our application
configuration. We, too, came to the conclusion that Chef looked like a good
tool because of it’s hierarchy of configuration and flexibility.

We spent months (literally) wrapping our heads around the problem. That was
the most painful part of the process. What we came out with was a model of
our environments and our data that we could plug into Chef. Do not skimp on
the modelling part. I’ve written some about the process here
http://blog.geeksgonemad.com/2011/01/chef-is-my-documentation.html. And
inside that post is a link to a presentation I’ve made public that discusses
some of our modeling in detail.

I still don’t have anything posted giving examples of our choices of when
we decided to use roles vs. databags, vs other Cheffy things. I’ll post that
to my blog soon-ish as we are in the thick of rolling out our solution right
now. Once I do some demo presentations internally I’ll have some material to
post. We’re using it to manage java properties and logging files, tomcat
server.xml, apache vhosts, any configuration file that any piece of our
application needs. We are even using Chef to create our Control Tier project
xml files.

I will warn you that one constraint of Chef may or may not be a significant
impact to how you manage and deploy configurations. Because Chef is purely a
convergence tool, you cannot manage configuration files individually. There
is only one giant runlist that will have to make every configuration file
every run. (see my thread earlier on this mailing list about ad-hock
runlists http://lists.opscode.com/sympa/arc/chef/2011-01/msg00130.html). I
am conspiring with DTO Solutions (Control Tier and Rundeck) to work out ways
of getting Chef to behave more like direct control than convergence. Our
compromise is that we have built a process where Chef will create all
configuration files but not in a live location and we will have some other
piece of orchestration software deploy the necessary configuration files
when the time is right.

John’s post came in as I’m writing this. I agree that some of the greatest
benefit we are seeing is that our configuration is now data-driven and as
much as possible derivable. Having system and application configuration in
one place is a really great thing.

Dan

On Fri, Feb 25, 2011 at 8:51 AM, Denis Haskin denis@haskinferguson.netwrote:

(apologies if this has been previously asked & answered; the mailing list
archives are sort of hard to use.)

We’re investigating using chef, among other purposes to better manage
cross-system “application” configuration. Specifically, we have services
from multiple projects running on multiple servers, and at the moment each
project has their own configuration file, despite the fact that there is
configuration that would be best shared among them.

I’m quite sure this could be helped by chef but I’m having a little
trouble completely wrapping my head around how we would do this with chef.
Seems like all of the chef docs & tutorials are focused on provisioning and
configuring services like apache, mysql, etc. (And, as I’ve read elsewhere,
a lot of the chef doc seems somewhat abstract and doesn’t always relate
these abstract concepts to concrete usage.)

Does anybody have any samples/suggestions/etc?

Thanks!

dwh


#6

I’m not sure why regenerating all the config files every time matters? If the underlying parameters are the same the files will remain unchanged, so there will be no net effect. If the underlying parameters have changed, then the file should be regenerated anyway. Thats the whole idea.

Brian

Sent from my iPhone

On Feb 25, 2011, at 7:41 AM, Denis Haskin denis@haskinferguson.net wrote:

Dan – superb, and thanks for the caveats and heads-ups. We’ll look at your doc & posts and hopefully can stand on your shoulders to not spend months on it…

I don’t think the convergence issue (as you call it) should be a problem for us; we’re already moving toward consolidated config files anyway (if I understood you correctly).

I’m sure we’ll be back with more questions. Thanks!

dwh

On Fri, Feb 25, 2011 at 10:26 AM, Dan Nemec dan@nemecfamily.com wrote:
Dennis,

Welcome to my world. We started that project in earnest last fall. We have a number of environments and instances of application suites in those environments and needed desperately to simplify our application configuration. We, too, came to the conclusion that Chef looked like a good tool because of it’s hierarchy of configuration and flexibility.

We spent months (literally) wrapping our heads around the problem. That was the most painful part of the process. What we came out with was a model of our environments and our data that we could plug into Chef. Do not skimp on the modelling part. I’ve written some about the process here http://blog.geeksgonemad.com/2011/01/chef-is-my-documentation.html. And inside that post is a link to a presentation I’ve made public that discusses some of our modeling in detail.

I still don’t have anything posted giving examples of our choices of when we decided to use roles vs. databags, vs other Cheffy things. I’ll post that to my blog soon-ish as we are in the thick of rolling out our solution right now. Once I do some demo presentations internally I’ll have some material to post. We’re using it to manage java properties and logging files, tomcat server.xml, apache vhosts, any configuration file that any piece of our application needs. We are even using Chef to create our Control Tier project xml files.

I will warn you that one constraint of Chef may or may not be a significant impact to how you manage and deploy configurations. Because Chef is purely a convergence tool, you cannot manage configuration files individually. There is only one giant runlist that will have to make every configuration file every run. (see my thread earlier on this mailing list about ad-hock runlists http://lists.opscode.com/sympa/arc/chef/2011-01/msg00130.html). I am conspiring with DTO Solutions (Control Tier and Rundeck) to work out ways of getting Chef to behave more like direct control than convergence. Our compromise is that we have built a process where Chef will create all configuration files but not in a live location and we will have some other piece of orchestration software deploy the necessary configuration files when the time is right.

John’s post came in as I’m writing this. I agree that some of the greatest benefit we are seeing is that our configuration is now data-driven and as much as possible derivable. Having system and application configuration in one place is a really great thing.

Dan

On Fri, Feb 25, 2011 at 8:51 AM, Denis Haskin denis@haskinferguson.net wrote:
(apologies if this has been previously asked & answered; the mailing list archives are sort of hard to use.)

We’re investigating using chef, among other purposes to better manage cross-system “application” configuration. Specifically, we have services from multiple projects running on multiple servers, and at the moment each project has their own configuration file, despite the fact that there is configuration that would be best shared among them.

I’m quite sure this could be helped by chef but I’m having a little trouble completely wrapping my head around how we would do this with chef. Seems like all of the chef docs & tutorials are focused on provisioning and configuring services like apache, mysql, etc. (And, as I’ve read elsewhere, a lot of the chef doc seems somewhat abstract and doesn’t always relate these abstract concepts to concrete usage.)

Does anybody have any samples/suggestions/etc?

Thanks!

dwh


#7

I agree. On paper convergence should not create any problems and is
(sometimes) the ideal solution. If you can do it, then great. You are in a
better place than me.

The hypothetical scenario showing the risk of convergence of
application configuration looks like this. I do not work at a continuous
deployment shop. All changes must be scheduled and approved through a change
management process.

On Monday Admin A is told to make a configuration change to Application X on
Thursday. He has an approved change ticket for the change. On Tuesday Admin
B realizes he needs to make a configuration change to Application Y on
Friday. He has an approved ticket for the change. Now, if Admin B checks in
his change before Friday, then his change will be deployed when Admin A
deploys his change on Thursday. So, the timing of any change to
configuration becomes super-critical so that only the expected change goes
out at the expected time.

If you have the ability to deploy only the configuration for Application X,
then it doesn’t matter when a change is made to Application Y. Risk is lower
when you can deploy only the thing you know you are deploying.

Convergence is good for system change, but for my requirements, it
complicates application change.

As I said, we worked around this by having Chef create configuration into an
offline location, and deployment software moves the appropriate
configuration into the live location with an appropriate orchestrated
workflow.

Dan

On Fri, Feb 25, 2011 at 11:02 AM, Brian McKelvey brian@worlize.com wrote:

I’m not sure why regenerating all the config files every time matters? If
the underlying parameters are the same the files will remain unchanged, so
there will be no net effect. If the underlying parameters have changed,
then the file should be regenerated anyway. Thats the whole idea.

Brian

Sent from my iPhone

On Feb 25, 2011, at 7:41 AM, Denis Haskin denis@haskinferguson.net
wrote:

Dan – superb, and thanks for the caveats and heads-ups. We’ll look at
your doc & posts and hopefully can stand on your shoulders to not spend
months on it…

I don’t think the convergence issue (as you call it) should be a problem
for us; we’re already moving toward consolidated config files anyway (if I
understood you correctly).

I’m sure we’ll be back with more questions. Thanks!

dwh

On Fri, Feb 25, 2011 at 10:26 AM, Dan Nemec < dan@nemecfamily.com
dan@nemecfamily.com> wrote:

Dennis,

Welcome to my world. We started that project in earnest last fall. We have
a number of environments and instances of application suites in those
environments and needed desperately to simplify our application
configuration. We, too, came to the conclusion that Chef looked like a good
tool because of it’s hierarchy of configuration and flexibility.

We spent months (literally) wrapping our heads around the problem. That
was the most painful part of the process. What we came out with was a model
of our environments and our data that we could plug into Chef. Do not skimp
on the modelling part. I’ve written some about the process here http://blog.geeksgonemad.com/2011/01/chef-is-my-documentation.html
http://blog.geeksgonemad.com/2011/01/chef-is-my-documentation.html. And
inside that post is a link to a presentation I’ve made public that discusses
some of our modeling in detail.

I still don’t have anything posted giving examples of our choices of when
we decided to use roles vs. databags, vs other Cheffy things. I’ll post that
to my blog soon-ish as we are in the thick of rolling out our solution right
now. Once I do some demo presentations internally I’ll have some material to
post. We’re using it to manage java properties and logging files, tomcat
server.xml, apache vhosts, any configuration file that any piece of our
application needs. We are even using Chef to create our Control Tier project
xml files.

I will warn you that one constraint of Chef may or may not be a
significant impact to how you manage and deploy configurations. Because Chef
is purely a convergence tool, you cannot manage configuration files
individually. There is only one giant runlist that will have to make every
configuration file every run. (see my thread earlier on this mailing list
about ad-hock runlists http://lists.opscode.com/sympa/arc/chef/2011-01/msg00130.html
http://lists.opscode.com/sympa/arc/chef/2011-01/msg00130.html). I am
conspiring with DTO Solutions (Control Tier and Rundeck) to work out ways of
getting Chef to behave more like direct control than convergence. Our
compromise is that we have built a process where Chef will create all
configuration files but not in a live location and we will have some other
piece of orchestration software deploy the necessary configuration files
when the time is right.

John’s post came in as I’m writing this. I agree that some of the greatest
benefit we are seeing is that our configuration is now data-driven and as
much as possible derivable. Having system and application configuration in
one place is a really great thing.

Dan

On Fri, Feb 25, 2011 at 8:51 AM, Denis Haskin <denis@haskinferguson.net
denis@haskinferguson.net> wrote:

(apologies if this has been previously asked & answered; the mailing list
archives are sort of hard to use.)

We’re investigating using chef, among other purposes to better manage
cross-system “application” configuration. Specifically, we have services
from multiple projects running on multiple servers, and at the moment each
project has their own configuration file, despite the fact that there is
configuration that would be best shared among them.

I’m quite sure this could be helped by chef but I’m having a little
trouble completely wrapping my head around how we would do this with chef.
Seems like all of the chef docs & tutorials are focused on provisioning and
configuring services like apache, mysql, etc. (And, as I’ve read elsewhere,
a lot of the chef doc seems somewhat abstract and doesn’t always relate
these abstract concepts to concrete usage.)

Does anybody have any samples/suggestions/etc?

Thanks!

dwh


#8

Got it. Sounds unpleasant. :frowning:

Sent from my iPhone

On Feb 25, 2011, at 8:44 AM, Dan Nemec dan@nemecfamily.com wrote:

I agree. On paper convergence should not create any problems and is (sometimes) the ideal solution. If you can do it, then great. You are in a better place than me.

The hypothetical scenario showing the risk of convergence of application configuration looks like this. I do not work at a continuous deployment shop. All changes must be scheduled and approved through a change management process.

On Monday Admin A is told to make a configuration change to Application X on Thursday. He has an approved change ticket for the change. On Tuesday Admin B realizes he needs to make a configuration change to Application Y on Friday. He has an approved ticket for the change. Now, if Admin B checks in his change before Friday, then his change will be deployed when Admin A deploys his change on Thursday. So, the timing of any change to configuration becomes super-critical so that only the expected change goes out at the expected time.

If you have the ability to deploy only the configuration for Application X, then it doesn’t matter when a change is made to Application Y. Risk is lower when you can deploy only the thing you know you are deploying.

Convergence is good for system change, but for my requirements, it complicates application change.

As I said, we worked around this by having Chef create configuration into an offline location, and deployment software moves the appropriate configuration into the live location with an appropriate orchestrated workflow.

Dan

On Fri, Feb 25, 2011 at 11:02 AM, Brian McKelvey brian@worlize.com wrote:
I’m not sure why regenerating all the config files every time matters? If the underlying parameters are the same the files will remain unchanged, so there will be no net effect. If the underlying parameters have changed, then the file should be regenerated anyway. Thats the whole idea.

Brian

Sent from my iPhone

On Feb 25, 2011, at 7:41 AM, Denis Haskin denis@haskinferguson.net wrote:

Dan – superb, and thanks for the caveats and heads-ups. We’ll look at your doc & posts and hopefully can stand on your shoulders to not spend months on it…

I don’t think the convergence issue (as you call it) should be a problem for us; we’re already moving toward consolidated config files anyway (if I understood you correctly).

I’m sure we’ll be back with more questions. Thanks!

dwh

On Fri, Feb 25, 2011 at 10:26 AM, Dan Nemec dan@nemecfamily.com wrote:
Dennis,

Welcome to my world. We started that project in earnest last fall. We have a number of environments and instances of application suites in those environments and needed desperately to simplify our application configuration. We, too, came to the conclusion that Chef looked like a good tool because of it’s hierarchy of configuration and flexibility.

We spent months (literally) wrapping our heads around the problem. That was the most painful part of the process. What we came out with was a model of our environments and our data that we could plug into Chef. Do not skimp on the modelling part. I’ve written some about the process here http://blog.geeksgonemad.com/2011/01/chef-is-my-documentation.html. And inside that post is a link to a presentation I’ve made public that discusses some of our modeling in detail.

I still don’t have anything posted giving examples of our choices of when we decided to use roles vs. databags, vs other Cheffy things. I’ll post that to my blog soon-ish as we are in the thick of rolling out our solution right now. Once I do some demo presentations internally I’ll have some material to post. We’re using it to manage java properties and logging files, tomcat server.xml, apache vhosts, any configuration file that any piece of our application needs. We are even using Chef to create our Control Tier project xml files.

I will warn you that one constraint of Chef may or may not be a significant impact to how you manage and deploy configurations. Because Chef is purely a convergence tool, you cannot manage configuration files individually. There is only one giant runlist that will have to make every configuration file every run. (see my thread earlier on this mailing list about ad-hock runlists http://lists.opscode.com/sympa/arc/chef/2011-01/msg00130.html). I am conspiring with DTO Solutions (Control Tier and Rundeck) to work out ways of getting Chef to behave more like direct control than convergence. Our compromise is that we have built a process where Chef will create all configuration files but not in a live location and we will have some other piece of orchestration software deploy the necessary configuration files when the time is right.

John’s post came in as I’m writing this. I agree that some of the greatest benefit we are seeing is that our configuration is now data-driven and as much as possible derivable. Having system and application configuration in one place is a really great thing.

Dan

On Fri, Feb 25, 2011 at 8:51 AM, Denis Haskin denis@haskinferguson.net wrote:
(apologies if this has been previously asked & answered; the mailing list archives are sort of hard to use.)

We’re investigating using chef, among other purposes to better manage cross-system “application” configuration. Specifically, we have services from multiple projects running on multiple servers, and at the moment each project has their own configuration file, despite the fact that there is configuration that would be best shared among them.

I’m quite sure this could be helped by chef but I’m having a little trouble completely wrapping my head around how we would do this with chef. Seems like all of the chef docs & tutorials are focused on provisioning and configuring services like apache, mysql, etc. (And, as I’ve read elsewhere, a lot of the chef doc seems somewhat abstract and doesn’t always relate these abstract concepts to concrete usage.)

Does anybody have any samples/suggestions/etc?

Thanks!

dwh


#9

Without going off-topic too far…

Brian, the process Dan is mentioning is pretty much de rigueur in most
corporate environments. There are plenty of cases where external rules
(i.e. PCIDSS/HIPPA/ACRONYM) create additional process that prevents
what you were saying. Many times those externals are legacy but there
are some legitimate cases where industry or government regulation
prevents you from going all in. Sometimes it’s inherited regulation.
I.e. you may not be required for your industry but because you service
another industry that DOES have the regulation, it get’s passed down
to you.

Other times, that level of process is there because of past screw ups.
People don’t trust the tool because someone who was doing manual work
in the past screwed something up. So now there are layers of
bureaucratic bs to go through to get changes out.

On Fri, Feb 25, 2011 at 12:07 PM, Brian McKelvey brian@worlize.com wrote:

Got it. Sounds unpleasant. :frowning:

Sent from my iPhone
On Feb 25, 2011, at 8:44 AM, Dan Nemec dan@nemecfamily.com wrote:

I agree. On paper convergence should not create any problems and is
(sometimes) the ideal solution. If you can do it, then great. You are in a
better place than me.
The hypothetical scenario showing the risk of convergence of
application configuration looks like this. I do not work at a continuous
deployment shop. All changes must be scheduled and approved through a change
management process.
On Monday Admin A is told to make a configuration change to Application X on
Thursday. He has an approved change ticket for the change. On Tuesday Admin
B realizes he needs to make a configuration change to Application Y on
Friday. He has an approved ticket for the change. Now, if Admin B checks in
his change before Friday, then his change will be deployed when Admin A
deploys his change on Thursday. So, the timing of any change to
configuration becomes super-critical so that only the expected change goes
out at the expected time.
If you have the ability to deploy only the configuration for Application X,
then it doesn’t matter when a change is made to Application Y. Risk is lower
when you can deploy only the thing you know you are deploying.

Convergence is good for system change, but for my requirements, it
complicates application change.
As I said, we worked around this by having Chef create configuration into an
offline location, and deployment software moves the appropriate
configuration into the live location with an appropriate orchestrated
workflow.
Dan

On Fri, Feb 25, 2011 at 11:02 AM, Brian McKelvey brian@worlize.com wrote:

I’m not sure why regenerating all the config files every time matters? If
the underlying parameters are the same the files will remain unchanged, so
there will be no net effect. If the underlying parameters have changed,
then the file should be regenerated anyway. Thats the whole idea.
Brian

Sent from my iPhone
On Feb 25, 2011, at 7:41 AM, Denis Haskin denis@haskinferguson.net
wrote:

Dan – superb, and thanks for the caveats and heads-ups. We’ll look at
your doc & posts and hopefully can stand on your shoulders to not spend
months on it…
I don’t think the convergence issue (as you call it) should be a problem
for us; we’re already moving toward consolidated config files anyway (if I
understood you correctly).
I’m sure we’ll be back with more questions. Thanks!
dwh

On Fri, Feb 25, 2011 at 10:26 AM, Dan Nemec dan@nemecfamily.com wrote:

Dennis,
Welcome to my world. We started that project in earnest last fall. We
have a number of environments and instances of application suites in those
environments and needed desperately to simplify our application
configuration. We, too, came to the conclusion that Chef looked like a good
tool because of it’s hierarchy of configuration and flexibility.
We spent months (literally) wrapping our heads around the problem. That
was the most painful part of the process. What we came out with was a model
of our environments and our data that we could plug into Chef. Do not skimp
on the modelling part. I’ve written some about the process
here http://blog.geeksgonemad.com/2011/01/chef-is-my-documentation.html. And
inside that post is a link to a presentation I’ve made public that discusses
some of our modeling in detail.
I still don’t have anything posted giving examples of our choices of when
we decided to use roles vs. databags, vs other Cheffy things. I’ll post that
to my blog soon-ish as we are in the thick of rolling out our solution right
now. Once I do some demo presentations internally I’ll have some material to
post. We’re using it to manage java properties and logging files, tomcat
server.xml, apache vhosts, any configuration file that any piece of our
application needs. We are even using Chef to create our Control Tier project
xml files.
I will warn you that one constraint of Chef may or may not be a
significant impact to how you manage and deploy configurations. Because Chef
is purely a convergence tool, you cannot manage configuration files
individually. There is only one giant runlist that will have to make every
configuration file every run. (see my thread earlier on this mailing list
about ad-hock
runlists http://lists.opscode.com/sympa/arc/chef/2011-01/msg00130.html). I
am conspiring with DTO Solutions (Control Tier and Rundeck) to work out ways
of getting Chef to behave more like direct control than convergence. Our
compromise is that we have built a process where Chef will create all
configuration files but not in a live location and we will have some other
piece of orchestration software deploy the necessary configuration files
when the time is right.
John’s post came in as I’m writing this. I agree that some of the
greatest benefit we are seeing is that our configuration is now data-driven
and as much as possible derivable. Having system and application
configuration in one place is a really great thing.
Dan

On Fri, Feb 25, 2011 at 8:51 AM, Denis Haskin denis@haskinferguson.net
wrote:

(apologies if this has been previously asked & answered; the mailing
list archives are sort of hard to use.)
We’re investigating using chef, among other purposes to better manage
cross-system “application” configuration. Specifically, we have services
from multiple projects running on multiple servers, and at the moment each
project has their own configuration file, despite the fact that there is
configuration that would be best shared among them.
I’m quite sure this could be helped by chef but I’m having a little
trouble completely wrapping my head around how we would do this with chef.
Seems like all of the chef docs & tutorials are focused on provisioning and
configuring services like apache, mysql, etc. (And, as I’ve read elsewhere,
a lot of the chef doc seems somewhat abstract and doesn’t always relate
these abstract concepts to concrete usage.)
Does anybody have any samples/suggestions/etc?
Thanks!
dwh