Got it. Sounds unpleasant. 
I agree. On paper convergence should not create any problems and is (sometimes) the ideal solution. If you can do it, then great. You are in a better place than me.
The hypothetical scenario showing the risk of convergence of application configuration looks like this. I do not work at a continuous deployment shop. All changes must be scheduled and approved through a change management process.
On Monday Admin A is told to make a configuration change to Application X on Thursday. He has an approved change ticket for the change. On Tuesday Admin B realizes he needs to make a configuration change to Application Y on Friday. He has an approved ticket for the change. Now, if Admin B checks in his change before Friday, then his change will be deployed when Admin A deploys his change on Thursday. So, the timing of any change to configuration becomes super-critical so that only the expected change goes out at the expected time.
If you have the ability to deploy only the configuration for Application X, then it doesn't matter when a change is made to Application Y. Risk is lower when you can deploy only the thing you know you are deploying.
Convergence is good for system change, but for my requirements, it complicates application change.
As I said, we worked around this by having Chef create configuration into an offline location, and deployment software moves the appropriate configuration into the live location with an appropriate orchestrated workflow.
Dan
On Fri, Feb 25, 2011 at 11:02 AM, Brian McKelvey brian@worlize.com wrote:
I'm not sure why regenerating all the config files every time matters? If the underlying parameters are the same the files will remain unchanged, so there will be no net effect. If the underlying parameters have changed, then the file should be regenerated anyway. Thats the whole idea.
Brian
Sent from my iPhone
On Feb 25, 2011, at 7:41 AM, Denis Haskin denis@haskinferguson.net wrote:
Dan -- superb, and thanks for the caveats and heads-ups. We'll look at your doc & posts and hopefully can stand on your shoulders to not spend months on it...
I don't think the convergence issue (as you call it) should be a problem for us; we're already moving toward consolidated config files anyway (if I understood you correctly).
I'm sure we'll be back with more questions. Thanks!
dwh
On Fri, Feb 25, 2011 at 10:26 AM, Dan Nemec dan@nemecfamily.com wrote:
Dennis,
Welcome to my world. We started that project in earnest last fall. We have a number of environments and instances of application suites in those environments and needed desperately to simplify our application configuration. We, too, came to the conclusion that Chef looked like a good tool because of it's hierarchy of configuration and flexibility.
We spent months (literally) wrapping our heads around the problem. That was the most painful part of the process. What we came out with was a model of our environments and our data that we could plug into Chef. Do not skimp on the modelling part. I've written some about the process here Migrating To Devops: Chef Is My Documentation. And inside that post is a link to a presentation I've made public that discusses some of our modeling in detail.
I still don't have anything posted giving examples of our choices of when we decided to use roles vs. databags, vs other Cheffy things. I'll post that to my blog soon-ish as we are in the thick of rolling out our solution right now. Once I do some demo presentations internally I'll have some material to post. We're using it to manage java properties and logging files, tomcat server.xml, apache vhosts, any configuration file that any piece of our application needs. We are even using Chef to create our Control Tier project xml files.
I will warn you that one constraint of Chef may or may not be a significant impact to how you manage and deploy configurations. Because Chef is purely a convergence tool, you cannot manage configuration files individually. There is only one giant runlist that will have to make every configuration file every run. (see my thread earlier on this mailing list about ad-hock runlists chef - [chef] One-Shot runlists with inheritance). I am conspiring with DTO Solutions (Control Tier and Rundeck) to work out ways of getting Chef to behave more like direct control than convergence. Our compromise is that we have built a process where Chef will create all configuration files but not in a live location and we will have some other piece of orchestration software deploy the necessary configuration files when the time is right.
John's post came in as I'm writing this. I agree that some of the greatest benefit we are seeing is that our configuration is now data-driven and as much as possible derivable. Having system and application configuration in one place is a really great thing.
Dan
On Fri, Feb 25, 2011 at 8:51 AM, Denis Haskin denis@haskinferguson.net wrote:
(apologies if this has been previously asked & answered; the mailing list archives are sort of hard to use.)
We're investigating using chef, among other purposes to better manage cross-system "application" configuration. Specifically, we have services from multiple projects running on multiple servers, and at the moment each project has their own configuration file, despite the fact that there is configuration that would be best shared among them.
I'm quite sure this could be helped by chef but I'm having a little trouble completely wrapping my head around how we would do this with chef. Seems like all of the chef docs & tutorials are focused on provisioning and configuring services like apache, mysql, etc. (And, as I've read elsewhere, a lot of the chef doc seems somewhat abstract and doesn't always relate these abstract concepts to concrete usage.)
Does anybody have any samples/suggestions/etc?
Thanks!
dwh