Announcing zap-0.0.1

Ohai!

At the summit, I gave a presentation on the zap cookbook. Navigating my employer’s legal department proved daunting so I did a clean room reimplementation hosted at https://github.com/nvwls/zap.

v0.0.1 only supports the zap_directory LWRP. Support for cron and services are planned next. Enjoy!

From the README:
Library cookbook for building authoritative resource sets.

One of the common pitfalls in chef land is the pattern of one deleting a resource definition from a recipe and the user wondering why the resource still exists on the system.

For example, on Monday a cronjob is added:

cron ‘collect stats’ do
action :create
minute 0
command '/usr/local/bin/collect-stats | mailto ops@nvwls.com
end
After a few days, the issue is figured out and that cron resource is removed from the recipe. After uploading the new cookbook, they wonder why they are still receiving email.

The issue that chef is great for describing actions. I mean, action is part of the DSL.

At the 2013 Opscode Communit Summit, Matt Ray and I had discussion regarding this issue. The name authoritative cookbook was coined. If chef is deploying files to a .d directory, if there are files in that directory not converged by a resource, those files should be removed.

This pattern has been added to https://github.com/Youscribe/sysctl-cookbook

hi Joe
thanks for sharing the cookbook. we have the exactly same issue (nginx
conf.d , iptables rules and several others), and I am in process of
opensourcing our conf.d style (authoritative) implementation.
a major challenge we faced with modeling the delete un-managed files was
how to detect them. In your implementation you are doing by iterating the
resource collection. I am not sure if you can get a fully expanded resource
collection from any resource. Because chef allows dynamic resource
addition, at best you can get only the resource list that was statically
added (across all chef recipes) and dynamically added till the point of
detection (in your case till the zap_directory resource declaration),
notification brings another twist in this whole dynamic expansion. In
simpler terms what it means is, if i use an lwrp ( that expands into a set
of files inside the zap directory) after the zap_directory resource
declaration, will they be deleted in next run? because zap_directory
provider cant see them in resource collection.

In our implementation we use a report handler to do the deletion. the logic
is ditto (via iterating resource collection). I find it bit controversial
(i dont want any report handler to change my system's state or do
destructive operations), but this implementation does a bug free & clean
job. and provides some basic stuff on top of which we can improve.

I'm very happy to see your implementation :slight_smile: . thanks again for sharing
this.

regards
ranjib

On Sun, Nov 24, 2013 at 8:31 AM, Joe Nuspl nuspl@nvwls.com wrote:

Ohai!

At the summit, I gave a presentation on the zap cookbook. Navigating my
employer's legal department proved daunting so I did a clean room
reimplementation hosted at GitHub - nvwls/zap: Library cookbook for garbage collecting chef controlled resource sets..

v0.0.1 only supports the zap_directory LWRP. Support for cron and
services are planned next. Enjoy!

From the README:

Library cookbook for building authoritative resource sets.

One of the common pitfalls in chef land is the pattern of one deleting a
resource definition from a recipe and the user wondering why the resource
still exists on the system.

For example, on Monday a cronjob is added:

cron 'collect stats' do
action :create
minute 0
command '/usr/local/bin/collect-stats | mailto ops@nvwls.com'end

After a few days, the issue is figured out and that cron resource is
removed from the recipe. After uploading the new cookbook, they wonder why
they are still receiving email.

The issue that chef is great for describing actions. I mean, action is
part of the DSL.

At the 2013 Opscode Communit Summit, Matt Ray and I had discussion
regarding this issue. The name authoritative cookbook was coined. If
chef is deploying files to a .d directory, if there are files in that
directory not converged by a resource, those files should be removed.

This pattern has been added to
GitHub - Youscribe/sysctl-cookbook: DEPRECATED

Original I had implemented this as report handler but doing these actions in a report handler smelled funny (or tasted bad).

I've improved the LWRP by splitting it into two phases (implemented as two actions). :delay is the default action. When it runs, it clones the resource, sets the action to :run, and appends it to the end of the resource collection.

I'm working on making the LWRP more generic and being able to override the collect the existing, collect the resource, and eliminate cruft stages with your own routines. All in good time.

For the time being, v0.0.4 has been released.

Joe

On Nov 24, 2013, at 9:31 PM, Ranjib Dey dey.ranjib@gmail.com wrote:

hi Joe
thanks for sharing the cookbook. we have the exactly same issue (nginx conf.d , iptables rules and several others), and I am in process of opensourcing our conf.d style (authoritative) implementation.
a major challenge we faced with modeling the delete un-managed files was how to detect them. In your implementation you are doing by iterating the resource collection. I am not sure if you can get a fully expanded resource collection from any resource. Because chef allows dynamic resource addition, at best you can get only the resource list that was statically added (across all chef recipes) and dynamically added till the point of detection (in your case till the zap_directory resource declaration), notification brings another twist in this whole dynamic expansion. In simpler terms what it means is, if i use an lwrp ( that expands into a set of files inside the zap directory) after the zap_directory resource declaration, will they be deleted in next run? because zap_directory provider cant see them in resource collection.

In our implementation we use a report handler to do the deletion. the logic is ditto (via iterating resource collection). I find it bit controversial (i dont want any report handler to change my system's state or do destructive operations), but this implementation does a bug free & clean job. and provides some basic stuff on top of which we can improve.

I'm very happy to see your implementation :slight_smile: . thanks again for sharing this.

regards
ranjib

On Sun, Nov 24, 2013 at 8:31 AM, Joe Nuspl nuspl@nvwls.com wrote:
Ohai!

At the summit, I gave a presentation on the zap cookbook. Navigating my employer's legal department proved daunting so I did a clean room reimplementation hosted at GitHub - nvwls/zap: Library cookbook for garbage collecting chef controlled resource sets..

v0.0.1 only supports the zap_directory LWRP. Support for cron and services are planned next. Enjoy!

From the README:
Library cookbook for building authoritative resource sets.

One of the common pitfalls in chef land is the pattern of one deleting a resource definition from a recipe and the user wondering why the resource still exists on the system.

For example, on Monday a cronjob is added:

cron 'collect stats' do
action :create
minute 0
command '/usr/local/bin/collect-stats | mailto ops@nvwls.com'
end
After a few days, the issue is figured out and that cron resource is removed from the recipe. After uploading the new cookbook, they wonder why they are still receiving email.

The issue that chef is great for describing actions. I mean, action is part of the DSL.

At the 2013 Opscode Communit Summit, Matt Ray and I had discussion regarding this issue. The name authoritative cookbook was coined. If chef is deploying files to a .d directory, if there are files in that directory not converged by a resource, those files should be removed.

This pattern has been added to GitHub - Youscribe/sysctl-cookbook: DEPRECATED