# Managing system files with Chef

**URL:** <https://discourse.chef.io/t/managing-system-files-with-chef/1870>\
**Category:** Chef Infra (archive)\
**Created:** [May 5, 2011, 4:03pm UTC](https://discourse.chef.io/t/managing-system-files-with-chef/1870 "2011-05-05T16:03:48Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Sascha\_Bates](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/sascha_bates/32/504_2.png) [@Sascha\_Bates](https://discourse.chef.io/u/Sascha_Bates)\
**Post date:** [May 5, 2011, 4:03pm UTC](https://discourse.chef.io/t/managing-system-files-with-chef/1870/1 "2011-05-05T16:03:48Z")

</div>

Chef wants to manage config files as templates. I get that. I’m wondering  
how people are handling system configuration files that are large and  
sometimes unwieldy? For example, in Red Hat, the /etc/sendmail.cf is 700  
lines and we update just one. Should I really keep a template or cookbook  
file for something like that? We are running primarily RH4 and RH5 and have  
identified the following choices for system configuration files:

Appropriate them as templates or manage them with complex string  
substitution in ruby/bash blocks and manual checks to enforce idempotence.

The first has pitfalls. We shouldn’t be managing system files at that  
level. Patching should be an automated process, and trying to keep an eye  
on system files during the patch process is not something I’m interested in.  
Also it means that only one recipe can ever manage that file. In most  
cases that’s fine, but in others, it won’t work. Example: I have two  
different cookbooks that add keys to the root authorized\_keys file in some  
cases. Or what if two different things want to insert things in the  
modprobe.conf? I have a system config recipe that inserts lines into the  
modprobe.conf to disable IPv6 and a KVM recipe that inserts the kvm module  
lines.

On the other hand, using regex string substitutions in files is complex, can  
be messy looking and I’m getting the impression this method is discouraged.  
I also regard the use of the bash block as the last resort in a recipe. But  
it has the easiest way to do string substitution in files (using sed). I  
have been using this to manage settings in sshd\_config, [sendmail.cf](http://sendmail.cf),  
[submit.cf](http://submit.cf), and a few other places.

So those are my thoughts. What are others doing to manage ssh configs,  
sendmail, snmp, module configs, etc? I’m really curious.

Sascha

---

<div class="post-metadata">

**Author:** ![kallistec](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/kallistec/32/23_2.png) [@kallistec](https://discourse.chef.io/u/kallistec)\
**Post date:** [May 5, 2011, 4:18pm UTC](https://discourse.chef.io/t/managing-system-files-with-chef/1870/2 "2011-05-05T16:18:37Z")

</div>

On Thursday, May 5, 2011 at 9:03 AM, Sascha Bates wrote:  
Chef wants to manage config files as templates. I get that. I'm wondering how people are handling system configuration files that are large and sometimes unwieldy? For example, in Red Hat, the /etc/sendmail.cf is 700 lines and we update just one. Should I really keep a template or cookbook file for something like that? We are running primarily RH4 and RH5 and have identified the following choices for system configuration files:

> 

The reason for this is that it's declarative. When you do it this way, you get the same config file when you build a new machine as you have on one that's managed with chef for a long time. Consider the following scenario:

1. You patch line X of file FOO using regular expressions
2. You patch line X of file FOO using regular expressions (again)
3. When you build a new server, the regex in step 2 doesn't match and your file doesn't get patched at all.

It's not hard to imagine many variations of this problem.

The ideal situation is that your applications support config.d/ directories, but this isn't always the case. Anyway, I'd recommend carefully considering this problem before you decide to go off the happy path.

--  
Dan DeLeo

---

<div class="post-metadata">

**Author:** ![Sascha\_Bates](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/sascha_bates/32/504_2.png) [@Sascha\_Bates](https://discourse.chef.io/u/Sascha_Bates)\
**Post date:** [May 5, 2011, 4:22pm UTC](https://discourse.chef.io/t/managing-system-files-with-chef/1870/3 "2011-05-05T16:22:37Z")

</div>

Yes, but I won't necessarily know that one of my system files was patched  
and then overwritten by a template will I? So that goes both ways.

On Thu, May 5, 2011 at 11:18 AM, Daniel DeLeo [dan@kallistec.com](mailto:dan@kallistec.com) wrote:

> On Thursday, May 5, 2011 at 9:03 AM, Sascha Bates wrote:
> 
> Chef wants to manage config files as templates. I get that. I'm wondering  
> how people are handling system configuration files that are large and  
> sometimes unwieldy? For example, in Red Hat, the /etc/sendmail.cf is 700  
> lines and we update just one. Should I really keep a template or cookbook  
> file for something like that? We are running primarily RH4 and RH5 and have  
> identified the following choices for system configuration files:
> 
> The reason for this is that it's declarative. When you do it this way, you  
> get the same config file when you build a new machine as you have on one  
> that's managed with chef for a long time. Consider the following scenario:
> 
> 1. You patch line X of file FOO using regular expressions
> 2. You patch line X of file FOO using regular expressions (again)
> 3. When you build a new server, the regex in step 2 doesn't match and your  
> file doesn't get patched at all.
> 
> It's not hard to imagine many variations of this problem.
> 
> The ideal situation is that your applications support config.d/  
> directories, but this isn't always the case. Anyway, I'd recommend carefully  
> considering this problem before you decide to go off the happy path.
> 
> --  
> Dan DeLeo

---

<div class="post-metadata">

**Author:** ![Bryan\_McLellan](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/bryan_mclellan/32/81_2.png) [@Bryan\_McLellan](https://discourse.chef.io/u/Bryan_McLellan)\
**Post date:** [May 5, 2011, 4:24pm UTC](https://discourse.chef.io/t/managing-system-files-with-chef/1870/4 "2011-05-05T16:24:53Z")

</div>

On Thu, May 5, 2011 at 9:03 AM, Sascha Bates [sascha.bates@gmail.com](mailto:sascha.bates@gmail.com) wrote:

> Chef wants to manage config files as templates. I get that. I'm wondering  
> how people are handling system configuration files that are large and  
> sometimes unwieldy? For example, in Red Hat, the /etc/sendmail.cf is 700  
> lines and we update just one. Should I really keep a template or cookbook  
> file for something like that? We are running primarily RH4 and RH5 and have  
> identified the following choices for system configuration files:

If there is a variable that I'm setting, I'll usually use a template.  
If I'm patching a file, for instance I recently had to make a one line  
change to a Wordpress file, I'll use cookbook\_file.

> Appropriate them as templates or manage them with complex string  
> substitution in ruby/bash blocks and manual checks to enforce idempotence.

I think the fact that you used the word "complex" there makes it the  
losing choice. At least with templates you can follow the code and it  
should be more straightforward.

> The first has pitfalls. We shouldn't be managing system files at that  
> level. Patching should be an automated process, and trying to keep an eye  
> on system files during the patch process is not something I'm interested in.

Can you give a use case? I'm not sure exactly what you mean by  
managing system files.

In the case of my wordpress example, I consider it a temporary hack  
and that the fix should be upstreamed. I tied the cookbook file to the  
wordpress release using only\_if. My rule of thumb is that managing any  
file with Chef that comes out of a deb package that isn't recognized  
as a config file by the package should be temporary.

sendmail.cf is a configuration file though, I don't see why you  
wouldn't want to manage it with Chef as a template.

> Also it means that only one recipe can ever manage that file. In most  
> cases that's fine, but in others, it won't work. Example: I have two  
> different cookbooks that add keys to the root authorized\_keys file in some  
> cases. Or what if two different things want to insert things in the  
> modprobe.conf? I have a system config recipe that inserts lines into the  
> modprobe.conf to disable IPv6 and a KVM recipe that inserts the kvm module  
> lines.

In both cases, you want to upfactor the code to another cookbook and  
have either a definition or an LWRP that you call from the other  
cookbooks to make the change. How you do this depends on a couple  
things, like if you have '.d' functionality for what you're changing.  
In the case of modprobe on Ubuntu, I would have a definition that  
creates a modprobe.d/resource.conf file for each call, similar to the  
iptables\_rule definition [1]. In the case of an authorized\_keys file,  
you could have a definition that walks the file when called to ensure  
that the line is there and appends if it is not.

On Thu, May 5, 2011 at 9:22 AM, Sascha Bates [sascha.bates@gmail.com](mailto:sascha.bates@gmail.com) wrote:

> Yes, but I won't necessarily know that one of my system files was patched  
> and then overwritten by a template will I? So that goes both ways.

If Chef manages the file, then your process needs to be that all  
changes go through Chef. It becomes the authoritative source for the  
file. You should always avoid one-offs by changing files manually  
anyway.

Bryan

[1] [https://github.com/opscode/cookbooks/blob/master/iptables/definitions/iptables\_rule.rb](https://github.com/opscode/cookbooks/blob/master/iptables/definitions/iptables_rule.rb)

---

<div class="post-metadata">

**Author:** ![jasonjwwilliams](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/jasonjwwilliams/32/101_2.png) [@jasonjwwilliams](https://discourse.chef.io/u/jasonjwwilliams)\
**Post date:** [May 5, 2011, 5:34pm UTC](https://discourse.chef.io/t/managing-system-files-with-chef/1870/5 "2011-05-05T17:34:44Z")

</div>

Hi Sasha

> The first has pitfalls. We shouldn't be managing system files at that  
> level. Patching should be an automated process, and trying to keep an eye  
> on system files during the patch process is not something I'm interested in.  
> Also it means that only one recipe can ever manage that file. In most  
> cases that's fine, but in others, it won't work. Example: I have two  
> different cookbooks that add keys to the root authorized\_keys file in some  
> cases. Or what if two different things want to insert things in the  
> modprobe.conf? I have a system config recipe that inserts lines into the  
> modprobe.conf to disable IPv6 and a KVM recipe that inserts the kvm module  
> lines.

We're in the same boat on things like Nagios NRPE configs where  
depending on the role of the server different checks need to be  
present or not. We're planning to handle it with templates and  
conditional Ruby blocks. Our plan is to use the roles the server  
belongs to activate particular blocks for checks specific to those  
roles. I would imagine a similar solution might work in your scenario.  
That would be much cleaner than patching I would imagine.

-J

---

<div class="post-metadata">

**Author:** ![Rob\_Guttman](https://avatars.discourse-cdn.com/v4/letter/r/a9adbd/32.png) [@Rob\_Guttman](https://discourse.chef.io/u/Rob_Guttman)\
**Post date:** [May 5, 2011, 5:54pm UTC](https://discourse.chef.io/t/managing-system-files-with-chef/1870/6 "2011-05-05T17:54:12Z")

</div>

> The ideal situation is that your applications support config.d/  
> directories ..

Bingo. Nagios uses such directories aplenty - they work very well for us.

- Rob

On Thu, May 5, 2011 at 1:34 PM, Jason J. W. Williams \<  
[jasonjwwilliams@gmail.com](mailto:jasonjwwilliams@gmail.com)\> wrote:

> Hi Sasha
> 
> > The first has pitfalls. We shouldn't be managing system files at that  
> > level. Patching should be an automated process, and trying to keep an  
> > eye  
> > on system files during the patch process is not something I'm interested  
> > in.  
> > Also it means that only one recipe can ever manage that file. In most  
> > cases that's fine, but in others, it won't work. Example: I have two  
> > different cookbooks that add keys to the root authorized\_keys file in  
> > some  
> > cases. Or what if two different things want to insert things in the  
> > modprobe.conf? I have a system config recipe that inserts lines into the  
> > modprobe.conf to disable IPv6 and a KVM recipe that inserts the kvm  
> > module  
> > lines.
> 
> We're in the same boat on things like Nagios NRPE configs where  
> depending on the role of the server different checks need to be  
> present or not. We're planning to handle it with templates and  
> conditional Ruby blocks. Our plan is to use the roles the server  
> belongs to activate particular blocks for checks specific to those  
> roles. I would imagine a similar solution might work in your scenario.  
> That would be much cleaner than patching I would imagine.
> 
> -J

---

<div class="post-metadata">

**Author:** ![Jeffrey\_Hulten1](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/jeffrey_hulten1/32/585_2.png) [@Jeffrey\_Hulten1](https://discourse.chef.io/u/Jeffrey_Hulten1)\
**Post date:** [May 5, 2011, 6:45pm UTC](https://discourse.chef.io/t/managing-system-files-with-chef/1870/7 "2011-05-05T18:45:44Z")

</div>

Now if only there was an /etc/hosts.d

On Thu, May 5, 2011 at 12:54 PM, Rob Guttman [robguttman@gmail.com](mailto:robguttman@gmail.com) wrote:

> > The ideal situation is that your applications support config.d/  
> > directories ..
> 
> Bingo. Nagios uses such directories aplenty - they work very well for us.
> 
> - Rob
> 
> On Thu, May 5, 2011 at 1:34 PM, Jason J. W. Williams  
> [jasonjwwilliams@gmail.com](mailto:jasonjwwilliams@gmail.com) wrote:
> 
> > Hi Sasha
> > 
> > > The first has pitfalls. We shouldn't be managing system files at that  
> > > level. Patching should be an automated process, and trying to keep an  
> > > eye  
> > > on system files during the patch process is not something I'm interested  
> > > in.  
> > > Also it means that only one recipe can ever manage that file. In most  
> > > cases that's fine, but in others, it won't work. Example: I have two  
> > > different cookbooks that add keys to the root authorized\_keys file in  
> > > some  
> > > cases. Or what if two different things want to insert things in the  
> > > modprobe.conf? I have a system config recipe that inserts lines into  
> > > the  
> > > modprobe.conf to disable IPv6 and a KVM recipe that inserts the kvm  
> > > module  
> > > lines.
> > 
> > We're in the same boat on things like Nagios NRPE configs where  
> > depending on the role of the server different checks need to be  
> > present or not. We're planning to handle it with templates and  
> > conditional Ruby blocks. Our plan is to use the roles the server  
> > belongs to activate particular blocks for checks specific to those  
> > roles. I would imagine a similar solution might work in your scenario.  
> > That would be much cleaner than patching I would imagine.
> > 
> > -J

--  
Jeffrey Hulten  
Principal Consultant at Automated Labs, LLC  
[jeffh@automatedlabs.com](mailto:jeffh@automatedlabs.com) 206-853-5216