Chef + managed-directory pain


#1

Hi,

has anybody a solution how to realize managed directories with chef ?

remote_directory with “purge true” isn’t a solution in the following cases:

Ubuntu:

Case 1:

  1. clean out /etc/apt/sources.list.d/

  2. create apt-repo-files with “apt_repository” in /etc/apt/sources.list.d/

  3. remote_directory-resource deletes the files from 2.

Case 2:

  1. clean out /etc/apache2/conf.d/

  2. generate apache config-files (with template resource) in
    /etc/apache2/conf.d/

  3. remote_directory-resource deletes the files from 2.

I’ve looked at https://github.com/zts/chef-cookbook-managed_directory ,but
ths solution has the same problems.

Thats the last big hurdle on our migration to chef, I hope somebody have
an idea.


#2

Hello,

I has an idea:

On 23 August 2012 03:06, bjunity@gmail.com bjunity@gmail.com wrote:

Hi,

has anybody a solution how to realize managed directories with chef ?

remote_directory with “purge true” isn’t a solution in the following cases:

Ubuntu:

Case 1:

  1. clean out /etc/apt/sources.list.d/

  2. create apt-repo-files with “apt_repository” in /etc/apt/sources.list.d/

  3. remote_directory-resource deletes the files from 2.

Don’t mix resources (and providers) to manage the same content.

Dynamically consume data pertaining to your infrastructure and create
the Resources required via the Recipe DSL:

node[‘apt_repositories’][‘enabled_repositories’].each do |repo|
apt_repository repo[‘name’] do

end
end

A similar (contrived) example could be made for the other scenario you
posted below.

Generally, I’d (personally) say jamming files in a cookbook and
remote_directory transferring them around is an anti-pattern that
should be avoided (YMMV)

–AJ

Case 2:

  1. clean out /etc/apache2/conf.d/

  2. generate apache config-files (with template resource) in
    /etc/apache2/conf.d/

  3. remote_directory-resource deletes the files from 2.

I’ve looked at https://github.com/zts/chef-cookbook-managed_directory ,but
ths solution has the same problems.

Thats the last big hurdle on our migration to chef, I hope somebody have an
idea.


#3

Hi,

maybe my explanation wasn’t clear enough.

The goal is to have only files managed by chef in a directory ( imagine
/etc/apache2/conf.d/ or /etc/apt/sources.list.d/).

On most systems you don’t know what’s in these directories (for example
bringing existing/legacy systems under chef control)

Example:

#delete unmanged files in directory “/etc/apache2/conf.d” (“empty” is an
empty-directory (only with a .placeholder file because chef fails if the
source-directory is empty). With this approach you can clean out a
directory:

remote_directory “/etc/apache2/conf.d” do
source "empty"
mode "0755"
files_backup 0
recursive true
purge true
end

#generate (config)-file
file “/etc/apache2/conf.d/charset.conf” do
content "some content\n"
end

The problem with this approach is that now on every chef run the file gets
recreated and for a short period of time the file is unavailable (and for
example if you use notifications on the file resource these are triggered
now on every chef run even when the file content hasn’t changed)

In puppet this works and I’ve used this a lot to realize managed
directory’s.

https://github.com/zts/chef-cookbook-managed_directory tries to solve the
problem, but unfortunately has the same issues.

I think managed directory’s are best-practice because you only have
full-control what’s in the directory with managed directories.

Thanks for help,

Bjoern

2012/8/22 AJ Christensen aj@junglist.gen.nz

Hello,

I has an idea:

On 23 August 2012 03:06, bjunity@gmail.com bjunity@gmail.com wrote:

Hi,

has anybody a solution how to realize managed directories with chef ?

remote_directory with “purge true” isn’t a solution in the following
cases:

Ubuntu:

Case 1:

  1. clean out /etc/apt/sources.list.d/

  2. create apt-repo-files with “apt_repository” in
    /etc/apt/sources.list.d/

  3. remote_directory-resource deletes the files from 2.

Don’t mix resources (and providers) to manage the same content.

Dynamically consume data pertaining to your infrastructure and create
the Resources required via the Recipe DSL:

node[‘apt_repositories’][‘enabled_repositories’].each do |repo|
apt_repository repo[‘name’] do

end
end

A similar (contrived) example could be made for the other scenario you
posted below.

Generally, I’d (personally) say jamming files in a cookbook and
remote_directory transferring them around is an anti-pattern that
should be avoided (YMMV)

–AJ

Case 2:

  1. clean out /etc/apache2/conf.d/

  2. generate apache config-files (with template resource) in
    /etc/apache2/conf.d/

  3. remote_directory-resource deletes the files from 2.

I’ve looked at https://github.com/zts/chef-cookbook-managed_directory,but
ths solution has the same problems.

Thats the last big hurdle on our migration to chef, I hope somebody
have an
idea.


#4

I think I’ve got an idea for you. If the requirement is “start fresh”,

  1. Destroy ( execute rm -rf) and create (directory) a fresh /etc/apache2/conf.d/ with a sigil file .chef-managed only_if that file does not exist.

  2. Drop chef file and template resources there with a clear conscience.

This will not detect subsequent “off the ranch” additions, but I’d argue if that is a risk you’ve got other problems to solve.

But… if you really need to re-baseline every time, there is no mechanism in chef to do that per-se with templates.

You can rebaseline using the above with remote_dir and purge if the content is only files, then the correct way to do it is actually drop all the files into your cookbook under files/default/apache2-conf.d rather than individual file resources, but I’m not a big fan of that.


Aaron Peterson

On Wednesday, August 22, 2012 at 18:44, bjunity@gmail.com wrote:

Hi,

maybe my explanation wasn’t clear enough.

The goal is to have only files managed by chef in a directory ( imagine /etc/apache2/conf.d/ or /etc/apt/sources.list.d/).

On most systems you don’t know what’s in these directories (for example bringing existing/legacy systems under chef control)

Example:

#delete unmanged files in directory “/etc/apache2/conf.d” (“empty” is an empty-directory (only with a .placeholder file because chef fails if the
source-directory is empty). With this approach you can clean out a directory:

remote_directory “/etc/apache2/conf.d” do
source "empty"
mode "0755"
files_backup 0
recursive true
purge true
end

#generate (config)-file
file “/etc/apache2/conf.d/charset.conf” do
content "some content\n"
end

The problem with this approach is that now on every chef run the file gets recreated and for a short period of time the file is unavailable (and for example if you use notifications on the file resource these are triggered now on every chef run even when the file content hasn’t changed)

In puppet this works and I’ve used this a lot to realize managed directory’s.

https://github.com/zts/chef-cookbook-managed_directory tries to solve the problem, but unfortunately has the same issues.

I think managed directory’s are best-practice because you only have full-control what’s in the directory with managed directories.

Thanks for help,

Bjoern

2012/8/22 AJ Christensen <aj@junglist.gen.nz (mailto:aj@junglist.gen.nz)>

Hello,

I has an idea:

On 23 August 2012 03:06, bjunity@gmail.com (mailto:bjunity@gmail.com) <bjunity@gmail.com (mailto:bjunity@gmail.com)> wrote:

Hi,

has anybody a solution how to realize managed directories with chef ?

remote_directory with “purge true” isn’t a solution in the following cases:

Ubuntu:

Case 1:

  1. clean out /etc/apt/sources.list.d/

  2. create apt-repo-files with “apt_repository” in /etc/apt/sources.list.d/

  3. remote_directory-resource deletes the files from 2.

Don’t mix resources (and providers) to manage the same content.

Dynamically consume data pertaining to your infrastructure and create
the Resources required via the Recipe DSL:

node[‘apt_repositories’][‘enabled_repositories’].each do |repo|
apt_repository repo[‘name’] do

end
end

A similar (contrived) example could be made for the other scenario you
posted below.

Generally, I’d (personally) say jamming files in a cookbook and
remote_directory transferring them around is an anti-pattern that
should be avoided (YMMV)

–AJ

Case 2:

  1. clean out /etc/apache2/conf.d/

  2. generate apache config-files (with template resource) in
    /etc/apache2/conf.d/

  3. remote_directory-resource deletes the files from 2.

I’ve looked at https://github.com/zts/chef-cookbook-managed_directory ,but
ths solution has the same problems.

Thats the last big hurdle on our migration to chef, I hope somebody have an
idea.


#5

Hi Aaron,

thanks for reply.

I’'m actually using your workaround for first-time installs to start fresh,
but this doesn’t solve the problem with subsequent "off the ranch"
additions as you stated.

This will not detect subsequent “off the ranch” additions, but I’d argue
if that is a risk you’ve got other
problems to solve.

I agree, but my understanding of an configuration management system is to
bring back the node to full compliance at ANY moment (even if there were “off
the ranch” additions).

You can rebaseline using the above with remote_dir and purge if the
content is only files, then the correct
way to do it is actually drop all the files into your cookbook under
files/default/apache2-conf.d rather than
individual file resources, but I’m not a big fan of that.

If there are dynamically created files in this directory you can’t drop
these files as files in the cookbook on the chef-server (besides that, notifies
on individual file resources aren’t possible with that.)


Bjoern

2012/8/23 Aaron Peterson metaxis@gmail.com

I think I’ve got an idea for you. If the requirement is “start fresh”,

  1. Destroy ( execute rm -rf) and create (directory) a fresh /etc/apache2/conf.d/
    with a sigil file .chef-managed only_if that file does not exist.

  2. Drop chef file and template resources there with a clear conscience.

This will not detect subsequent “off the ranch” additions, but I’d argue
if that is a risk you’ve got other problems to solve.

But… if you really need to re-baseline every time, there is no mechanism
in chef to do that per-se with templates.

You can rebaseline using the above with remote_dir and purge if the
content is only files, then the correct way to do it is actually drop all
the files into your cookbook under files/default/apache2-conf.d rather than
individual file resources, but I’m not a big fan of that.


Aaron Peterson

On Wednesday, August 22, 2012 at 18:44, bjunity@gmail.com wrote:

Hi,

maybe my explanation wasn’t clear enough.

The goal is to have only files managed by chef in a directory ( imagine
/etc/apache2/conf.d/ or /etc/apt/sources.list.d/).

On most systems you don’t know what’s in these directories (for example
bringing existing/legacy systems under chef control)

Example:

#delete unmanged files in directory “/etc/apache2/conf.d” (“empty” is an
empty-directory (only with a .placeholder file because chef fails if the
source-directory is empty). With this approach you can clean out a
directory:

remote_directory “/etc/apache2/conf.d” do
source "empty"
mode "0755"
files_backup 0
recursive true
purge true
end

#generate (config)-file
file “/etc/apache2/conf.d/charset.conf” do
content "some content\n"
end

The problem with this approach is that now on every chef run the file gets
recreated and for a short period of time the file is unavailable (and for
example if you use notifications on the file resource these are triggered
now on every chef run even when the file content hasn’t changed)

In puppet this works and I’ve used this a lot to realize managed
directory’s.

https://github.com/zts/chef-cookbook-managed_directory tries to solve the
problem, but unfortunately has the same issues.

I think managed directory’s are best-practice because you only have
full-control what’s in the directory with managed directories.

Thanks for help,

Bjoern

2012/8/22 AJ Christensen aj@junglist.gen.nz

Hello,

I has an idea:

On 23 August 2012 03:06, bjunity@gmail.com bjunity@gmail.com wrote:

Hi,

has anybody a solution how to realize managed directories with chef ?

remote_directory with “purge true” isn’t a solution in the following
cases:

Ubuntu:

Case 1:

  1. clean out /etc/apt/sources.list.d/

  2. create apt-repo-files with “apt_repository” in
    /etc/apt/sources.list.d/

  3. remote_directory-resource deletes the files from 2.

Don’t mix resources (and providers) to manage the same content.

Dynamically consume data pertaining to your infrastructure and create
the Resources required via the Recipe DSL:

node[‘apt_repositories’][‘enabled_repositories’].each do |repo|
apt_repository repo[‘name’] do

end
end

A similar (contrived) example could be made for the other scenario you
posted below.

Generally, I’d (personally) say jamming files in a cookbook and
remote_directory transferring them around is an anti-pattern that
should be avoided (YMMV)

–AJ

Case 2:

  1. clean out /etc/apache2/conf.d/

  2. generate apache config-files (with template resource) in
    /etc/apache2/conf.d/

  3. remote_directory-resource deletes the files from 2.

I’ve looked at https://github.com/zts/chef-cookbook-managed_directory,but
ths solution has the same problems.

Thats the last big hurdle on our migration to chef, I hope somebody
have an
idea.