How would I do automatic compressing and archiving of old files via chef


#1

I used to have some CFEngine rules that would look for old files and compress them in place. A subsequent rule would look for these compressed files and, if they were older still, would move them to an archive directory. A final rule would erase the compressed, archived files if they became very old.

With CFEngine, I could have rules that only did this once a day, using something like a .Hour01 class on a statement inside of its files section.

Is there a straightforward way of doing this ala Chef? I could create a set of code blocks or put cron jobs on the systems, but I am wondering if there is a solution similar to CFEngine’s approach.


Stephen Corbesero, Performance Engineer
Synchronoss - Mobile Innovation for a Connected World
stephen.corbesero@synchronoss.com | www.synchronoss.com


#2

I would encourage you to look into using log rotate rules. Placing a log rotate configuration with fairly straight forward rules can accomplish all of these tasks.

Graham

  • Graham

On Thursday, January 16, 2014 at 9:15 AM, Stephen Corbesero wrote:

I used to have some CFEngine rules that would look for old files and compress them in place. A subsequent rule would look for these compressed files and, if they were older still, would move them to an archive directory. A final rule would erase the compressed, archived files if they became very old.

With CFEngine, I could have rules that only did this once a day, using something like a .Hour01 class on a statement inside of its files section.

Is there a straightforward way of doing this ala Chef? I could create a set of code blocks or put cron jobs on the systems, but I am wondering if there is a solution similar to CFEngine’s approach.


Stephen Corbesero, Performance Engineer
Synchronoss - Mobile Innovation for a Connected World
stephen.corbesero@synchronoss.com | www.synchronoss.com (http://www.synchronoss.com)


#3

[Shameless Plug]

I wrote the janitor cookbook for the deletion of files that are based on
various criteria like age, size etc. similar to how cfengine did. The only
part I am missing is the archival of the same files.

Contributions are welcome!

On Thu, Jan 16, 2014 at 10:15 AM, Stephen Corbesero <
Stephen.Corbesero@synchronoss.com> wrote:

I used to have some CFEngine rules that would look for old files and
compress them in place. A subsequent rule would look for these compressed
files and, if they were older still, would move them to an archive
directory. A final rule would erase the compressed, archived files if they
became very old.

With CFEngine, I could have rules that only did this once a day, using
something like a .Hour01 class on a statement inside of its files section.

Is there a straightforward way of doing this ala Chef? I could create a
set of code blocks or put cron jobs on the systems, but I am wondering if
there is a solution similar to CFEngine’s approach.

Stephen Corbesero, Performance Engineer

Synchronoss - Mobile Innovation for a Connected World

stephen.corbesero@synchronoss.com | www.synchronoss.com


Thanks,

Mark


#4

Another thing to consider is that after rotating logs (however you do it,)
it is often necessary to SIGHUP or restart the process which is attached to
those logs. In general, be aware of the implications of removing files from
under processes.

On Thu, Jan 16, 2014 at 9:28 AM, Graham Christensen graham@grahamc.comwrote:

I would encourage you to look into using log rotate rules. Placing a log
rotate configuration with fairly straight forward rules can accomplish all
of these tasks.

Graham

  • Graham

On Thursday, January 16, 2014 at 9:15 AM, Stephen Corbesero wrote:

I used to have some CFEngine rules that would look for old files and
compress them in place. A subsequent rule would look for these compressed
files and, if they were older still, would move them to an archive
directory. A final rule would erase the compressed, archived files if they
became very old.

With CFEngine, I could have rules that only did this once a day, using
something like a .Hour01 class on a statement inside of its files section.

Is there a straightforward way of doing this ala Chef? I could create a
set of code blocks or put cron jobs on the systems, but I am wondering if
there is a solution similar to CFEngine’s approach.

Stephen Corbesero, Performance Engineer

Synchronoss - Mobile Innovation for a Connected World

stephen.corbesero@synchronoss.com | www.synchronoss.com