Edit a file using chef


#1

Hi,

I saw a file resource which can be used to create and delete a file,but i want
to edit a file with values provided by the user.
Does chef provides such facility??


#2

minimin, u can use the template resource for this purpose

questions like this are probably better suited to the main chef mailing list

On Thu, Jun 14, 2012 at 1:11 PM, minmini17@gmail.com wrote:

Hi,

I saw a file resource which can be used to create and delete a file,but i
want
to edit a file with values provided by the user.
Does chef provides such facility??


#3

https://github.com/opscode/chef/blob/master/chef/lib/chef/util/file_edit.rb

On 14 June 2012 23:25, Bryan Berry bryan.berry@gmail.com wrote:

minimin, u can use the template resource for this purpose

questions like this are probably better suited to the main chef mailing list

On Thu, Jun 14, 2012 at 1:11 PM, minmini17@gmail.com wrote:

Hi,

I saw a file resource which can be used to create and delete a file,but i
want
to edit a file with values provided by the user.
Does chef provides such facility??


#4

I wonder if it’s time to finally turn that into a resource, even
though I think there is near universal agreement it’s not a great
idea?

Adam

On Thu, Jun 14, 2012 at 4:54 AM, AJ Christensen aj@junglist.gen.nz wrote:

https://github.com/opscode/chef/blob/master/chef/lib/chef/util/file_edit.rb

On 14 June 2012 23:25, Bryan Berry bryan.berry@gmail.com wrote:

minimin, u can use the template resource for this purpose

questions like this are probably better suited to the main chef mailing list

On Thu, Jun 14, 2012 at 1:11 PM, minmini17@gmail.com wrote:

Hi,

I saw a file resource which can be used to create and delete a file,but i
want
to edit a file with values provided by the user.
Does chef provides such facility??


Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com


#5

I think its a great idea.

Its only bad for you if you overuse it.

Used sparingly it can get some jobs done where the constraints make
updating the whole file annoying.

On 6/14/12 2:02 PM, Adam Jacob wrote:

I wonder if it’s time to finally turn that into a resource, even
though I think there is near universal agreement it’s not a great
idea?

Adam

On Thu, Jun 14, 2012 at 4:54 AM, AJ Christensenaj@junglist.gen.nz wrote:

https://github.com/opscode/chef/blob/master/chef/lib/chef/util/file_edit.rb

On 14 June 2012 23:25, Bryan Berrybryan.berry@gmail.com wrote:

minimin, u can use the template resource for this purpose

questions like this are probably better suited to the main chef mailing list

On Thu, Jun 14, 2012 at 1:11 PM,minmini17@gmail.com wrote:

Hi,

I saw a file resource which can be used to create and delete a file,but i
want
to edit a file with values provided by the user.
Does chef provides such facility??


#6

+1

On Thu, Jun 14, 2012 at 11:02 PM, Adam Jacob adam@opscode.com wrote:

I wonder if it’s time to finally turn that into a resource, even
though I think there is near universal agreement it’s not a great
idea?

Adam

On Thu, Jun 14, 2012 at 4:54 AM, AJ Christensen aj@junglist.gen.nz
wrote:

https://github.com/opscode/chef/blob/master/chef/lib/chef/util/file_edit.rb

On 14 June 2012 23:25, Bryan Berry bryan.berry@gmail.com wrote:

minimin, u can use the template resource for this purpose

questions like this are probably better suited to the main chef mailing
list

On Thu, Jun 14, 2012 at 1:11 PM, minmini17@gmail.com wrote:

Hi,

I saw a file resource which can be used to create and delete a
file,but i

want
to edit a file with values provided by the user.
Does chef provides such facility??


Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com


#7

Lamont "I promise I’ll only use it correctly, tactfully, and rarely"
Granquist, your argument does not persuade. :). But even if you held
true to that…

-1, I could never recommend this, and it fails most methodological
tests that Chef works under.

It should be trivial to take the whole file and templatize it. If you
cannot just easily take the whole file under management, that means
there is interestingly different content “out there” that represents
unmanaged yet important operational state. Now you are making
successive error-prone edits to a collection of interestingly
different files, and there is no guarantee that you will not have
drift unless you have already analyzed the differences across the
collection, at which point you might as well manage the whole thing.

Even if it is consistent, or the file differences are reproduced
normally by an underlying OS or software process, I had better only
add but never delete file edits to recipes, because if ever a machine
misses a past edit because it is off or whatever, or we launch a new
machine, we won’t be creating the file correctly.

Generally, if you cannot always guarantee the recreation of the file
in its current state, you have entombed state and risk and have failed
to create an infrastructure you can reason about or rebuild.

Making it a full resource is the Chef imprimatur to use it, and this
isn’t just rope, it’s already tied into a noose and permanently
attached to a scaffold. IMO.

-a

On Jun 14, 2012, at 14:02, Adam Jacob adam@opscode.com wrote:

I wonder if it’s time to finally turn that into a resource, even
though I think there is near universal agreement it’s not a great
idea?

Adam

On Thu, Jun 14, 2012 at 4:54 AM, AJ Christensen aj@junglist.gen.nz wrote:

https://github.com/opscode/chef/blob/master/chef/lib/chef/util/file_edit.rb

On 14 June 2012 23:25, Bryan Berry bryan.berry@gmail.com wrote:

minimin, u can use the template resource for this purpose

questions like this are probably better suited to the main chef mailing list

On Thu, Jun 14, 2012 at 1:11 PM, minmini17@gmail.com wrote:

Hi,

I saw a file resource which can be used to create and delete a file,but i
want
to edit a file with values provided by the user.
Does chef provides such facility??


Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com


#8

This exchange (Aaron and Lamont) is almost exactly the conversation
Adam and I had when I first had Nuo write file_edit and that’s why we
decided to bury it. It’s useful, but cuts against the grain.
Pragmatically though, there are files over which you may not have full
control. We constantly tell people that you can ease into as much
Chef as you can take, without having to rigidly accept our model.
Doesn’t saying "but you can only manage whole files, rendered by Chef"
blunt that argument?

-C

On Fri, Jun 15, 2012 at 7:21 AM, Aaron Peterson aaron@opscode.com wrote:

Lamont "I promise I’ll only use it correctly, tactfully, and rarely"
Granquist, your argument does not persuade. :). But even if you held
true to that…

-1, I could never recommend this, and it fails most methodological
tests that Chef works under.

It should be trivial to take the whole file and templatize it. If you
cannot just easily take the whole file under management, that means
there is interestingly different content “out there” that represents
unmanaged yet important operational state. Now you are making
successive error-prone edits to a collection of interestingly
different files, and there is no guarantee that you will not have
drift unless you have already analyzed the differences across the
collection, at which point you might as well manage the whole thing.

Even if it is consistent, or the file differences are reproduced
normally by an underlying OS or software process, I had better only
add but never delete file edits to recipes, because if ever a machine
misses a past edit because it is off or whatever, or we launch a new
machine, we won’t be creating the file correctly.

Generally, if you cannot always guarantee the recreation of the file
in its current state, you have entombed state and risk and have failed
to create an infrastructure you can reason about or rebuild.

Making it a full resource is the Chef imprimatur to use it, and this
isn’t just rope, it’s already tied into a noose and permanently
attached to a scaffold. IMO.

-a

On Jun 14, 2012, at 14:02, Adam Jacob adam@opscode.com wrote:

I wonder if it’s time to finally turn that into a resource, even
though I think there is near universal agreement it’s not a great
idea?

Adam

On Thu, Jun 14, 2012 at 4:54 AM, AJ Christensen aj@junglist.gen.nz wrote:

https://github.com/opscode/chef/blob/master/chef/lib/chef/util/file_edit.rb

On 14 June 2012 23:25, Bryan Berry bryan.berry@gmail.com wrote:

minimin, u can use the template resource for this purpose

questions like this are probably better suited to the main chef mailing list

On Thu, Jun 14, 2012 at 1:11 PM, minmini17@gmail.com wrote:

Hi,

I saw a file resource which can be used to create and delete a file,but i
want
to edit a file with values provided by the user.
Does chef provides such facility??


Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com


Christopher Brown, Chief Technical Officer, Opscode, Inc.
T: (425) 502-5522, E: cb@opscode.com
IRC, Github: skeptomai
Twitter: @skeptomai


#9

Problem is that while this is always the most correct way to do things,
that it may not be the correct way for the business that is using chef
to get things done right now. Its quite possible that you’ve got a huge
mess on your hands and the easiest way to alleviate 90% of the pain is
to simply enforce one line of state in a config file that has a thousand
unique snowflakes that have been deployed by hand over the last decade.
The ‘right’ thing to do is clean up the whole mess, but that often takes
10x the effort, since you often need to audit the whole thing and figure
out what deltas in all of the config options are actual necessary
business requirements and what is cargo cult and deltas that are
unnecessary. Leaving that unmanaged state out there is often messy, but
I’ve made that decision a lot in order to move forward in increments.
We also leave most of the state of the system unmanaged because most of
the state doesn’t matter.

There’s also other use cases, such as using bundled chef-solo to manage
state for an application that needs to ship and be integrated with
uncontrolled external config management systems. Such code may need to
enforce state in a file, but the rest of the state of that file is
complete out of your control and fundamentally cannot be brought under
your own management.

Within the same company you can wind up seeing this problem where
departments may have written tooling that edits files already and config
management systems may need to integrate with those and chef may need to
make partial edits to the portion of the file that it ‘controls’. The
right solution there is always to fix that separation of concerns and
make chef authoritative and to call a tool in order so that chef could
render the whole template, but stupid thing like that can make for
political snarls in large organizations with legacy infrastructure and
legacy teams, and I don’t think chef should be in the business of
inserting itself into politics.

And this goes to the whole meme of being “opinionated”, and I’m already
highly skeptical and tired of that meme and think its a bad one for this
kind of product. Its also why I don’t use CFEngine anymore because of
some of the opinions of that crowd that made it almost deliberately
difficult to integrate that product with a central database. I really
don’t like using tools that have opinions that force me into making
certain cost/benefit decisions in my job, I need to be able to weigh
those decisions myself.

So, yes, I’ve run into this decision over and over through the years,
and have quite often determined that it is far from trivial to take
the whole file and templatize it (its often far from trivial to
understand the actual requirements from the cargo culting), and have
chosen to either do the expedient thing for the business and clean up a
fragment of it first, or simple to ignore the rest of the config. In
some cases I’ve actually written code that does “if the file exists,
edit it to enforce this one line, if the file does not exist, then use
my nice standardized template” to gain eventual convergence and reduce
config deltas over time while at the same time punting the existing
mess, and allowing me to test my standardized template every time a new
server goes into prod.

And the whole use of “our model” and such is language that is kind of
annoying. Chef didn’t invent that, its just common practice that its
the better way of doing things in the unix sysadmin world, but its not
the /only/ configuration management design problem out there – although
for a small startup with 40 nodes in the cloud that has been using chef
since day #1 it may be the only one that you need. When talking about
walking into thousands of unmanaged servers grown organically over a
decade and on different major flavors of operating systems… You will
need every trick in the book.

And being able to use every trick in the book and be able to easily get
the job done – no matter how broken the existing systems and no matter
how ingrained the departmental politics – is what is going to make chef
awesome to work with for practicing/suffering devops/admins out there…

On 6/15/12 7:29 AM, Christopher Brown wrote:

This exchange (Aaron and Lamont) is almost exactly the conversation
Adam and I had when I first had Nuo write file_edit and that’s why we
decided to bury it. It’s useful, but cuts against the grain.
Pragmatically though, there are files over which you may not have full
control. We constantly tell people that you can ease into as much
Chef as you can take, without having to rigidly accept our model.
Doesn’t saying "but you can only manage whole files, rendered by Chef"
blunt that argument?

-C

On Fri, Jun 15, 2012 at 7:21 AM, Aaron Petersonaaron@opscode.com wrote:

Lamont "I promise I’ll only use it correctly, tactfully, and rarely"
Granquist, your argument does not persuade. :). But even if you held
true to that…

-1, I could never recommend this, and it fails most methodological
tests that Chef works under.

It should be trivial to take the whole file and templatize it. If you
cannot just easily take the whole file under management, that means
there is interestingly different content “out there” that represents
unmanaged yet important operational state. Now you are making
successive error-prone edits to a collection of interestingly
different files, and there is no guarantee that you will not have
drift unless you have already analyzed the differences across the
collection, at which point you might as well manage the whole thing.

Even if it is consistent, or the file differences are reproduced
normally by an underlying OS or software process, I had better only
add but never delete file edits to recipes, because if ever a machine
misses a past edit because it is off or whatever, or we launch a new
machine, we won’t be creating the file correctly.

Generally, if you cannot always guarantee the recreation of the file
in its current state, you have entombed state and risk and have failed
to create an infrastructure you can reason about or rebuild.

Making it a full resource is the Chef imprimatur to use it, and this
isn’t just rope, it’s already tied into a noose and permanently
attached to a scaffold. IMO.

-a

On Jun 14, 2012, at 14:02, Adam Jacobadam@opscode.com wrote:

I wonder if it’s time to finally turn that into a resource, even
though I think there is near universal agreement it’s not a great
idea?

Adam

On Thu, Jun 14, 2012 at 4:54 AM, AJ Christensenaj@junglist.gen.nz wrote:

https://github.com/opscode/chef/blob/master/chef/lib/chef/util/file_edit.rb

On 14 June 2012 23:25, Bryan Berrybryan.berry@gmail.com wrote:

minimin, u can use the template resource for this purpose

questions like this are probably better suited to the main chef mailing list

On Thu, Jun 14, 2012 at 1:11 PM,minmini17@gmail.com wrote:

Hi,

I saw a file resource which can be used to create and delete a file,but i
want
to edit a file with values provided by the user.
Does chef provides such facility??


Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com


#10

I completely agreed with Aaron until I read Lamont’s reply and now I
agree with both of them. I guess the difference between having the
resource and not having it is that the first trusts people to use the
tool correctly while the second forces them to do what you want and
leaves them without the flexibility in a situation where it could be a
life saver.

In my opinion, it’s very similar to the script/execute resources which
are abused mightily every day. Shelling out should be a last resort
when there’s no other way to do things, but if you examine codebases of
the inexperienced, you will find egregious examples of shell scripts
migrated nearly verbatim into Chef script blocks. I speak from
experience. In my time, I have both abused the script resource and
written ruby blocks to append to files. I also learned the error of my
ways and refactored everything later becaues it was the right thing to do.

Not creating the resource because it’s low priority is one thing, but
opposing its creation because you think it’s crime against nature…? I
concur that it’s the wrong way to operate, but I seem to recall hearing
that Chef grew out of some people’s frustration with Puppet being
opinionated and not allowing them to do certain things. I don’t know if
that’s an urban myth or not, but we might want to consider that as well
before refusing to create a resource based on the One True Way.

Sascha

On 6/15/12 11:13 AM, Lamont Granquist wrote:

Problem is that while this is always the most correct way to do
things, that it may not be the correct way for the business that is
using chef to get things done right now. Its quite possible that
you’ve got a huge mess on your hands and the easiest way to alleviate
90% of the pain is to simply enforce one line of state in a config
file that has a thousand unique snowflakes that have been deployed by
hand over the last decade. The ‘right’ thing to do is clean up the
whole mess, but that often takes 10x the effort, since you often need
to audit the whole thing and figure out what deltas in all of the
config options are actual necessary business requirements and what is
cargo cult and deltas that are unnecessary. Leaving that unmanaged
state out there is often messy, but I’ve made that decision a lot in
order to move forward in increments. We also leave most of the state
of the system unmanaged because most of the state doesn’t matter.

There’s also other use cases, such as using bundled chef-solo to
manage state for an application that needs to ship and be integrated
with uncontrolled external config management systems. Such code may
need to enforce state in a file, but the rest of the state of that
file is complete out of your control and fundamentally cannot be
brought under your own management.

Within the same company you can wind up seeing this problem where
departments may have written tooling that edits files already and
config management systems may need to integrate with those and chef
may need to make partial edits to the portion of the file that it
’controls’. The right solution there is always to fix that separation
of concerns and make chef authoritative and to call a tool in order so
that chef could render the whole template, but stupid thing like that
can make for political snarls in large organizations with legacy
infrastructure and legacy teams, and I don’t think chef should be in
the business of inserting itself into politics.

And this goes to the whole meme of being “opinionated”, and I’m
already highly skeptical and tired of that meme and think its a bad
one for this kind of product. Its also why I don’t use CFEngine
anymore because of some of the opinions of that crowd that made it
almost deliberately difficult to integrate that product with a central
database. I really don’t like using tools that have opinions that
force me into making certain cost/benefit decisions in my job, I need
to be able to weigh those decisions myself.

So, yes, I’ve run into this decision over and over through the years,
and have quite often determined that it is far from trivial to take
the whole file and templatize it (its often far from trivial to
understand the actual requirements from the cargo culting), and have
chosen to either do the expedient thing for the business and clean up
a fragment of it first, or simple to ignore the rest of the config.
In some cases I’ve actually written code that does “if the file
exists, edit it to enforce this one line, if the file does not exist,
then use my nice standardized template” to gain eventual convergence
and reduce config deltas over time while at the same time punting the
existing mess, and allowing me to test my standardized template every
time a new server goes into prod.

And the whole use of “our model” and such is language that is kind of
annoying. Chef didn’t invent that, its just common practice that its
the better way of doing things in the unix sysadmin world, but its not
the /only/ configuration management design problem out there –
although for a small startup with 40 nodes in the cloud that has been
using chef since day #1 it may be the only one that you need. When
talking about walking into thousands of unmanaged servers grown
organically over a decade and on different major flavors of operating
systems… You will need every trick in the book.

And being able to use every trick in the book and be able to easily
get the job done – no matter how broken the existing systems and no
matter how ingrained the departmental politics – is what is going to
make chef awesome to work with for practicing/suffering devops/admins
out there…

On 6/15/12 7:29 AM, Christopher Brown wrote:

This exchange (Aaron and Lamont) is almost exactly the conversation
Adam and I had when I first had Nuo write file_edit and that’s why we
decided to bury it. It’s useful, but cuts against the grain.
Pragmatically though, there are files over which you may not have full
control. We constantly tell people that you can ease into as much
Chef as you can take, without having to rigidly accept our model.
Doesn’t saying "but you can only manage whole files, rendered by Chef"
blunt that argument?

-C

On Fri, Jun 15, 2012 at 7:21 AM, Aaron Petersonaaron@opscode.com
wrote:

Lamont "I promise I’ll only use it correctly, tactfully, and rarely"
Granquist, your argument does not persuade. :). But even if you held
true to that…

-1, I could never recommend this, and it fails most methodological
tests that Chef works under.

It should be trivial to take the whole file and templatize it. If you
cannot just easily take the whole file under management, that means
there is interestingly different content “out there” that represents
unmanaged yet important operational state. Now you are making
successive error-prone edits to a collection of interestingly
different files, and there is no guarantee that you will not have
drift unless you have already analyzed the differences across the
collection, at which point you might as well manage the whole thing.

Even if it is consistent, or the file differences are reproduced
normally by an underlying OS or software process, I had better only
add but never delete file edits to recipes, because if ever a machine
misses a past edit because it is off or whatever, or we launch a new
machine, we won’t be creating the file correctly.

Generally, if you cannot always guarantee the recreation of the file
in its current state, you have entombed state and risk and have failed
to create an infrastructure you can reason about or rebuild.

Making it a full resource is the Chef imprimatur to use it, and this
isn’t just rope, it’s already tied into a noose and permanently
attached to a scaffold. IMO.

-a

On Jun 14, 2012, at 14:02, Adam Jacobadam@opscode.com wrote:

I wonder if it’s time to finally turn that into a resource, even
though I think there is near universal agreement it’s not a great
idea?

Adam

On Thu, Jun 14, 2012 at 4:54 AM, AJ
Christensenaj@junglist.gen.nz wrote:

https://github.com/opscode/chef/blob/master/chef/lib/chef/util/file_edit.rb

On 14 June 2012 23:25, Bryan Berrybryan.berry@gmail.com wrote:

minimin, u can use the template resource for this purpose

questions like this are probably better suited to the main chef
mailing list

On Thu, Jun 14, 2012 at 1:11 PM,minmini17@gmail.com wrote:

Hi,

I saw a file resource which can be used to create and delete a
file,but i
want
to edit a file with values provided by the user.
Does chef provides such facility??


Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com


#11

+1

If you have both the script and execute resources (with root by default!),
you are allowing users to shoot themselves in the foot much more than if
you allow regular expression search and replace with a resource.

Tell the story in the documentation. Evangelize the Chef-preferred way but
then concede there are times when there is no ideal situation. Show users
how much flexibility you are giving them. Also just like with any
potentially dangerous tool, explain the potential consequences and give
them fair warning. I believe users will feel empowered. More and more,
developers can say to sysadmins “look, we can start implementing this
incrementally.” There are a lot of folks who will appreciate this.

I see it being an extension of the file resource.

On Fri, Jun 15, 2012 at 12:13 PM, Lamont Granquist lamont@opscode.comwrote:

Problem is that while this is always the most correct way to do things,
that it may not be the correct way for the business that is using chef to
get things done right now. Its quite possible that you’ve got a huge mess
on your hands and the easiest way to alleviate 90% of the pain is to simply
enforce one line of state in a config file that has a thousand unique
snowflakes that have been deployed by hand over the last decade. The
’right’ thing to do is clean up the whole mess, but that often takes 10x
the effort, since you often need to audit the whole thing and figure out
what deltas in all of the config options are actual necessary business
requirements and what is cargo cult and deltas that are unnecessary.
Leaving that unmanaged state out there is often messy, but I’ve made that
decision a lot in order to move forward in increments. We also leave most
of the state of the system unmanaged because most of the state doesn’t
matter.

There’s also other use cases, such as using bundled chef-solo to manage
state for an application that needs to ship and be integrated with
uncontrolled external config management systems. Such code may need to
enforce state in a file, but the rest of the state of that file is complete
out of your control and fundamentally cannot be brought under your own
management.

Within the same company you can wind up seeing this problem where
departments may have written tooling that edits files already and config
management systems may need to integrate with those and chef may need to
make partial edits to the portion of the file that it ‘controls’. The
right solution there is always to fix that separation of concerns and make
chef authoritative and to call a tool in order so that chef could render
the whole template, but stupid thing like that can make for political
snarls in large organizations with legacy infrastructure and legacy teams,
and I don’t think chef should be in the business of inserting itself into
politics.

And this goes to the whole meme of being “opinionated”, and I’m already
highly skeptical and tired of that meme and think its a bad one for this
kind of product. Its also why I don’t use CFEngine anymore because of some
of the opinions of that crowd that made it almost deliberately difficult to
integrate that product with a central database. I really don’t like using
tools that have opinions that force me into making certain cost/benefit
decisions in my job, I need to be able to weigh those decisions myself.

So, yes, I’ve run into this decision over and over through the years, and
have quite often determined that it is far from trivial to take the whole
file and templatize it (its often far from trivial to understand the actual
requirements from the cargo culting), and have chosen to either do the
expedient thing for the business and clean up a fragment of it first, or
simple to ignore the rest of the config. In some cases I’ve actually
written code that does "if the file exists, edit it to enforce this one
line, if the file does not exist, then use my nice standardized template"
to gain eventual convergence and reduce config deltas over time while at
the same time punting the existing mess, and allowing me to test my
standardized template every time a new server goes into prod.

And the whole use of “our model” and such is language that is kind of
annoying. Chef didn’t invent that, its just common practice that its the
better way of doing things in the unix sysadmin world, but its not the
/only/ configuration management design problem out there – although for a
small startup with 40 nodes in the cloud that has been using chef since day
#1 it may be the only one that you need. When talking about walking into
thousands of unmanaged servers grown organically over a decade and on
different major flavors of operating systems… You will need every trick
in the book.

And being able to use every trick in the book and be able to easily get
the job done – no matter how broken the existing systems and no matter how
ingrained the departmental politics – is what is going to make chef
awesome to work with for practicing/suffering devops/admins out there…

On 6/15/12 7:29 AM, Christopher Brown wrote:

This exchange (Aaron and Lamont) is almost exactly the conversation
Adam and I had when I first had Nuo write file_edit and that’s why we
decided to bury it. It’s useful, but cuts against the grain.
Pragmatically though, there are files over which you may not have full
control. We constantly tell people that you can ease into as much
Chef as you can take, without having to rigidly accept our model.
Doesn’t saying "but you can only manage whole files, rendered by Chef"
blunt that argument?

-C

On Fri, Jun 15, 2012 at 7:21 AM, Aaron Petersonaaron@opscode.com
wrote:

Lamont "I promise I’ll only use it correctly, tactfully, and rarely"
Granquist, your argument does not persuade. :). But even if you held
true to that…

-1, I could never recommend this, and it fails most methodological
tests that Chef works under.

It should be trivial to take the whole file and templatize it. If you
cannot just easily take the whole file under management, that means
there is interestingly different content “out there” that represents
unmanaged yet important operational state. Now you are making
successive error-prone edits to a collection of interestingly
different files, and there is no guarantee that you will not have
drift unless you have already analyzed the differences across the
collection, at which point you might as well manage the whole thing.

Even if it is consistent, or the file differences are reproduced
normally by an underlying OS or software process, I had better only
add but never delete file edits to recipes, because if ever a machine
misses a past edit because it is off or whatever, or we launch a new
machine, we won’t be creating the file correctly.

Generally, if you cannot always guarantee the recreation of the file
in its current state, you have entombed state and risk and have failed
to create an infrastructure you can reason about or rebuild.

Making it a full resource is the Chef imprimatur to use it, and this
isn’t just rope, it’s already tied into a noose and permanently
attached to a scaffold. IMO.

-a

On Jun 14, 2012, at 14:02, Adam Jacobadam@opscode.com wrote:

I wonder if it’s time to finally turn that into a resource, even

though I think there is near universal agreement it’s not a great
idea?

Adam

On Thu, Jun 14, 2012 at 4:54 AM, AJ Christensen<aj@junglist.gen.nz**>
wrote:

https://github.com/opscode/chef/blob/master/chef/lib/
chef/util/file_edit.rbhttps://github.com/opscode/chef/blob/master/chef/lib/chef/util/file_edit.rb

On 14 June 2012 23:25, Bryan Berrybryan.berry@gmail.com wrote:

minimin, u can use the template resource for this purpose

questions like this are probably better suited to the main chef
mailing list

On Thu, Jun 14, 2012 at 1:11 PM,minmini17@gmail.com wrote:

Hi,

I saw a file resource which can be used to create and delete a
file,but i
want
to edit a file with values provided by the user.
Does chef provides such facility??


Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com


Patrick Collins


#12

Is the File resource currently considered idempotent? If so, does adding
this capability break that contract?

A resource that is no longer idempotent seems like a bad idea for backwards
compatibility. Maybe that’s not a concern.

(chef newbie)
Sam

On Fri, Jun 15, 2012 at 2:33 PM, Pat Collins pat@burned.com wrote:

+1

If you have both the script and execute resources (with root by default!),
you are allowing users to shoot themselves in the foot much more than if
you allow regular expression search and replace with a resource.

Tell the story in the documentation. Evangelize the Chef-preferred way but
then concede there are times when there is no ideal situation. Show users
how much flexibility you are giving them. Also just like with any
potentially dangerous tool, explain the potential consequences and give
them fair warning. I believe users will feel empowered. More and more,
developers can say to sysadmins “look, we can start implementing this
incrementally.” There are a lot of folks who will appreciate this.

I see it being an extension of the file resource.

On Fri, Jun 15, 2012 at 12:13 PM, Lamont Granquist lamont@opscode.comwrote:

Problem is that while this is always the most correct way to do things,
that it may not be the correct way for the business that is using chef to
get things done right now. Its quite possible that you’ve got a huge mess
on your hands and the easiest way to alleviate 90% of the pain is to simply
enforce one line of state in a config file that has a thousand unique
snowflakes that have been deployed by hand over the last decade. The
’right’ thing to do is clean up the whole mess, but that often takes 10x
the effort, since you often need to audit the whole thing and figure out
what deltas in all of the config options are actual necessary business
requirements and what is cargo cult and deltas that are unnecessary.
Leaving that unmanaged state out there is often messy, but I’ve made that
decision a lot in order to move forward in increments. We also leave most
of the state of the system unmanaged because most of the state doesn’t
matter.

There’s also other use cases, such as using bundled chef-solo to manage
state for an application that needs to ship and be integrated with
uncontrolled external config management systems. Such code may need to
enforce state in a file, but the rest of the state of that file is complete
out of your control and fundamentally cannot be brought under your own
management.

Within the same company you can wind up seeing this problem where
departments may have written tooling that edits files already and config
management systems may need to integrate with those and chef may need to
make partial edits to the portion of the file that it ‘controls’. The
right solution there is always to fix that separation of concerns and make
chef authoritative and to call a tool in order so that chef could render
the whole template, but stupid thing like that can make for political
snarls in large organizations with legacy infrastructure and legacy teams,
and I don’t think chef should be in the business of inserting itself into
politics.

And this goes to the whole meme of being “opinionated”, and I’m already
highly skeptical and tired of that meme and think its a bad one for this
kind of product. Its also why I don’t use CFEngine anymore because of some
of the opinions of that crowd that made it almost deliberately difficult to
integrate that product with a central database. I really don’t like using
tools that have opinions that force me into making certain cost/benefit
decisions in my job, I need to be able to weigh those decisions myself.

So, yes, I’ve run into this decision over and over through the years, and
have quite often determined that it is far from trivial to take the whole
file and templatize it (its often far from trivial to understand the actual
requirements from the cargo culting), and have chosen to either do the
expedient thing for the business and clean up a fragment of it first, or
simple to ignore the rest of the config. In some cases I’ve actually
written code that does "if the file exists, edit it to enforce this one
line, if the file does not exist, then use my nice standardized template"
to gain eventual convergence and reduce config deltas over time while at
the same time punting the existing mess, and allowing me to test my
standardized template every time a new server goes into prod.

And the whole use of “our model” and such is language that is kind of
annoying. Chef didn’t invent that, its just common practice that its the
better way of doing things in the unix sysadmin world, but its not the
/only/ configuration management design problem out there – although for a
small startup with 40 nodes in the cloud that has been using chef since day
#1 it may be the only one that you need. When talking about walking into
thousands of unmanaged servers grown organically over a decade and on
different major flavors of operating systems… You will need every trick
in the book.

And being able to use every trick in the book and be able to easily get
the job done – no matter how broken the existing systems and no matter how
ingrained the departmental politics – is what is going to make chef
awesome to work with for practicing/suffering devops/admins out there…

On 6/15/12 7:29 AM, Christopher Brown wrote:

This exchange (Aaron and Lamont) is almost exactly the conversation
Adam and I had when I first had Nuo write file_edit and that’s why we
decided to bury it. It’s useful, but cuts against the grain.
Pragmatically though, there are files over which you may not have full
control. We constantly tell people that you can ease into as much
Chef as you can take, without having to rigidly accept our model.
Doesn’t saying "but you can only manage whole files, rendered by Chef"
blunt that argument?

-C

On Fri, Jun 15, 2012 at 7:21 AM, Aaron Petersonaaron@opscode.com
wrote:

Lamont "I promise I’ll only use it correctly, tactfully, and rarely"
Granquist, your argument does not persuade. :). But even if you held
true to that…

-1, I could never recommend this, and it fails most methodological
tests that Chef works under.

It should be trivial to take the whole file and templatize it. If you
cannot just easily take the whole file under management, that means
there is interestingly different content “out there” that represents
unmanaged yet important operational state. Now you are making
successive error-prone edits to a collection of interestingly
different files, and there is no guarantee that you will not have
drift unless you have already analyzed the differences across the
collection, at which point you might as well manage the whole thing.

Even if it is consistent, or the file differences are reproduced
normally by an underlying OS or software process, I had better only
add but never delete file edits to recipes, because if ever a machine
misses a past edit because it is off or whatever, or we launch a new
machine, we won’t be creating the file correctly.

Generally, if you cannot always guarantee the recreation of the file
in its current state, you have entombed state and risk and have failed
to create an infrastructure you can reason about or rebuild.

Making it a full resource is the Chef imprimatur to use it, and this
isn’t just rope, it’s already tied into a noose and permanently
attached to a scaffold. IMO.

-a

On Jun 14, 2012, at 14:02, Adam Jacobadam@opscode.com wrote:

I wonder if it’s time to finally turn that into a resource, even

though I think there is near universal agreement it’s not a great
idea?

Adam

On Thu, Jun 14, 2012 at 4:54 AM, AJ Christensen<aj@junglist.gen.nz**>
wrote:

https://github.com/opscode/chef/blob/master/chef/lib/
chef/util/file_edit.rbhttps://github.com/opscode/chef/blob/master/chef/lib/chef/util/file_edit.rb

On 14 June 2012 23:25, Bryan Berrybryan.berry@gmail.com wrote:

minimin, u can use the template resource for this purpose

questions like this are probably better suited to the main chef
mailing list

On Thu, Jun 14, 2012 at 1:11 PM,minmini17@gmail.com wrote:

Hi,

I saw a file resource which can be used to create and delete a
file,but i
want
to edit a file with values provided by the user.
Does chef provides such facility??


Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com


Patrick Collins


#13

I would see this as a new provider that extends the File resource, and not a modification to File itself. I think that’s also what Pat was suggesting.

Sent from my mobile device

-----Original Message-----
From: Sam Dengler samdengler@gmail.com
Date: Fri, 15 Jun 2012 14:45:06
To: Pat Collinspat@burned.com
Cc: Chef Devchef-dev@lists.opscode.com
Subject: [chef-dev] Re: Re: Re: Re: Re: Re: Re: Re: edit a file using chef

Is the File resource currently considered idempotent? If so, does adding this capability break that contract?

A resource that is no longer idempotent seems like a bad idea for backwards compatibility. Maybe that’s not a concern.

(chef newbie)
Sam

On Fri, Jun 15, 2012 at 2:33 PM, Pat Collins <pat@burned.com mailto:pat@burned.com > wrote:
+1

If you have both the script and execute resources (with root by default!), you are allowing users to shoot themselves in the foot much more than if you allow regular expression search and replace with a resource.

Tell the story in the documentation. Evangelize the Chef-preferred way but then concede there are times when there is no ideal situation. Show users how much flexibility you are giving them. Also just like with any potentially dangerous tool, explain the potential consequences and give them fair warning. I believe users will feel empowered. More and more, developers can say to sysadmins “look, we can start implementing this incrementally.” There are a lot of folks who will appreciate this.

I see it being an extension of the file resource.

On Fri, Jun 15, 2012 at 12:13 PM, Lamont Granquist <lamont@opscode.com mailto:lamont@opscode.com > wrote:

Problem is that while this is always the most correct way to do things, that it may not be the correct way for the business that is using chef to get things done right now. Its quite possible that you’ve got a huge mess on your hands and the easiest way to alleviate 90% of the pain is to simply enforce one line of state in a config file that has a thousand unique snowflakes that have been deployed by hand over the last decade. The ‘right’ thing to do is clean up the whole mess, but that often takes 10x the effort, since you often need to audit the whole thing and figure out what deltas in all of the config options are actual necessary business requirements and what is cargo cult and deltas that are unnecessary. Leaving that unmanaged state out there is often messy, but I’ve made that decision a lot in order to move forward in increments. We also leave most of the state of the system unmanaged because most of the state doesn’t matter.

There’s also other use cases, such as using bundled chef-solo to manage state for an application that needs to ship and be integrated with uncontrolled external config management systems. Such code may need to enforce state in a file, but the rest of the state of that file is complete out of your control and fundamentally cannot be brought under your own management.

Within the same company you can wind up seeing this problem where departments may have written tooling that edits files already and config management systems may need to integrate with those and chef may need to make partial edits to the portion of the file that it ‘controls’. The right solution there is always to fix that separation of concerns and make chef authoritative and to call a tool in order so that chef could render the whole template, but stupid thing like that can make for political snarls in large organizations with legacy infrastructure and legacy teams, and I don’t think chef should be in the business of inserting itself into politics.

And this goes to the whole meme of being “opinionated”, and I’m already highly skeptical and tired of that meme and think its a bad one for this kind of product. Its also why I don’t use CFEngine anymore because of some of the opinions of that crowd that made it almost deliberately difficult to integrate that product with a central database. I really don’t like using tools that have opinions that force me into making certain cost/benefit decisions in my job, I need to be able to weigh those decisions myself.

So, yes, I’ve run into this decision over and over through the years, and have quite often determined that it is far from trivial to take the whole file and templatize it (its often far from trivial to understand the actual requirements from the cargo culting), and have chosen to either do the expedient thing for the business and clean up a fragment of it first, or simple to ignore the rest of the config. In some cases I’ve actually written code that does “if the file exists, edit it to enforce this one line, if the file does not exist, then use my nice standardized template” to gain eventual convergence and reduce config deltas over time while at the same time punting the existing mess, and allowing me to test my standardized template every time a new server goes into prod.

And the whole use of “our model” and such is language that is kind of annoying. Chef didn’t invent that, its just common practice that its the better way of doing things in the unix sysadmin world, but its not the /only/ configuration management design problem out there – although for a small startup with 40 nodes in the cloud that has been using chef since day #1 it may be the only one that you need. When talking about walking into thousands of unmanaged servers grown organically over a decade and on different major flavors of operating systems… You will need every trick in the book.

And being able to use every trick in the book and be able to easily get the job done – no matter how broken the existing systems and no matter how ingrained the departmental politics – is what is going to make chef awesome to work with for practicing/suffering devops/admins out there…

On 6/15/12 7:29 AM, Christopher Brown wrote:
This exchange (Aaron and Lamont) is almost exactly the conversation
Adam and I had when I first had Nuo write file_edit and that’s why we
decided to bury it. It’s useful, but cuts against the grain.
Pragmatically though, there are files over which you may not have full
control. We constantly tell people that you can ease into as much
Chef as you can take, without having to rigidly accept our model.
Doesn’t saying "but you can only manage whole files, rendered by Chef"
blunt that argument?

-C

On Fri, Jun 15, 2012 at 7:21 AM, Aaron Peterson<aaron@opscode.com mailto:aaron@opscode.com > wrote:
Lamont "I promise I’ll only use it correctly, tactfully, and rarely"
Granquist, your argument does not persuade. :). But even if you held
true to that…

-1, I could never recommend this, and it fails most methodological
tests that Chef works under.

It should be trivial to take the whole file and templatize it. If you
cannot just easily take the whole file under management, that means
there is interestingly different content “out there” that represents
unmanaged yet important operational state. Now you are making
successive error-prone edits to a collection of interestingly
different files, and there is no guarantee that you will not have
drift unless you have already analyzed the differences across the
collection, at which point you might as well manage the whole thing.

Even if it is consistent, or the file differences are reproduced
normally by an underlying OS or software process, I had better only
add but never delete file edits to recipes, because if ever a machine
misses a past edit because it is off or whatever, or we launch a new
machine, we won’t be creating the file correctly.

Generally, if you cannot always guarantee the recreation of the file
in its current state, you have entombed state and risk and have failed
to create an infrastructure you can reason about or rebuild.

Making it a full resource is the Chef imprimatur to use it, and this
isn’t just rope, it’s already tied into a noose and permanently
attached to a scaffold. IMO.

-a

On Jun 14, 2012, at 14:02, Adam Jacob<adam@opscode.com mailto:adam@opscode.com > wrote:

I wonder if it’s time to finally turn that into a resource, even
though I think there is near universal agreement it’s not a great
idea?

Adam

On Thu, Jun 14, 2012 at 4:54 AM, AJ Christensen<aj@junglist.gen.nz mailto:aj@junglist.gen.nz > wrote:
https://github.com/opscode/chef/blob/master/chef/lib/chef/util/file_edit.rb

On 14 June 2012 23:25, Bryan Berry<bryan.berry@gmail.com mailto:bryan.berry@gmail.com > wrote:
minimin, u can use the template resource for this purpose

questions like this are probably better suited to the main chef mailing list

On Thu, Jun 14, 2012 at 1:11 PM,<minmini17@gmail.com mailto:minmini17@gmail.com > wrote:

Hi,

I saw a file resource which can be used to create and delete a file,but i
want
to edit a file with values provided by the user.
Does chef provides such facility??


Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 tel:(206)%20619-7151 E: adam@opscode.com mailto:adam@opscode.com


Patrick Collins


#14

When I started with Chef I found a case where I needed something for
replace lines or checked in a file and I didn’t find. Everyone told me
to use templates and so, but sometimes the app doesn’t support and
it’s worse the workaround for using templates.

I did create my LWRP (on top of ‘file_edit’) and I tried to do it in a
more idempotent and safe way (probably not enogh, though), but I knew
it was just for a few special cases, no a silver bullet and it has to
be used carefully.

IMHO, if you have this in mind, it is much better to have a resource
which try to care about the idempotence, that use scripts and execute
resources to deal with those cases.

In case anyone lik to use or improve the LWRP, it’s here:
http://community.opscode.com/cookbooks/conf

On Fri, Jun 15, 2012 at 7:33 PM, Pat Collins pat@burned.com wrote:

+1

If you have both the script and execute resources (with root by default!),
you are allowing users to shoot themselves in the foot much more than if you
allow regular expression search and replace with a resource.

Tell the story in the documentation. Evangelize the Chef-preferred way but
then concede there are times when there is no ideal situation. Show users
how much flexibility you are giving them. Also just like with any
potentially dangerous tool, explain the potential consequences and give them
fair warning. I believe users will feel empowered. More and more, developers
can say to sysadmins "look, we can start implementing this incrementally."
There are a lot of folks who will appreciate this.

I see it being an extension of the file resource.

On Fri, Jun 15, 2012 at 12:13 PM, Lamont Granquist lamont@opscode.com
wrote:

Problem is that while this is always the most correct way to do things,
that it may not be the correct way for the business that is using chef to
get things done right now. Its quite possible that you’ve got a huge mess
on your hands and the easiest way to alleviate 90% of the pain is to simply
enforce one line of state in a config file that has a thousand unique
snowflakes that have been deployed by hand over the last decade. The
’right’ thing to do is clean up the whole mess, but that often takes 10x the
effort, since you often need to audit the whole thing and figure out what
deltas in all of the config options are actual necessary business
requirements and what is cargo cult and deltas that are unnecessary.
Leaving that unmanaged state out there is often messy, but I’ve made that
decision a lot in order to move forward in increments. We also leave most
of the state of the system unmanaged because most of the state doesn’t
matter.

There’s also other use cases, such as using bundled chef-solo to manage
state for an application that needs to ship and be integrated with
uncontrolled external config management systems. Such code may need to
enforce state in a file, but the rest of the state of that file is complete
out of your control and fundamentally cannot be brought under your own
management.

Within the same company you can wind up seeing this problem where
departments may have written tooling that edits files already and config
management systems may need to integrate with those and chef may need to
make partial edits to the portion of the file that it ‘controls’. The right
solution there is always to fix that separation of concerns and make chef
authoritative and to call a tool in order so that chef could render the
whole template, but stupid thing like that can make for political snarls in
large organizations with legacy infrastructure and legacy teams, and I don’t
think chef should be in the business of inserting itself into politics.

And this goes to the whole meme of being “opinionated”, and I’m already
highly skeptical and tired of that meme and think its a bad one for this
kind of product. Its also why I don’t use CFEngine anymore because of some
of the opinions of that crowd that made it almost deliberately difficult to
integrate that product with a central database. I really don’t like using
tools that have opinions that force me into making certain cost/benefit
decisions in my job, I need to be able to weigh those decisions myself.

So, yes, I’ve run into this decision over and over through the years, and
have quite often determined that it is far from trivial to take the whole
file and templatize it (its often far from trivial to understand the actual
requirements from the cargo culting), and have chosen to either do the
expedient thing for the business and clean up a fragment of it first, or
simple to ignore the rest of the config. In some cases I’ve actually
written code that does “if the file exists, edit it to enforce this one
line, if the file does not exist, then use my nice standardized template” to
gain eventual convergence and reduce config deltas over time while at the
same time punting the existing mess, and allowing me to test my standardized
template every time a new server goes into prod.

And the whole use of “our model” and such is language that is kind of
annoying. Chef didn’t invent that, its just common practice that its the
better way of doing things in the unix sysadmin world, but its not the
/only/ configuration management design problem out there – although for a
small startup with 40 nodes in the cloud that has been using chef since day
#1 it may be the only one that you need. When talking about walking into
thousands of unmanaged servers grown organically over a decade and on
different major flavors of operating systems… You will need every trick
in the book.

And being able to use every trick in the book and be able to easily get
the job done – no matter how broken the existing systems and no matter how
ingrained the departmental politics – is what is going to make chef awesome
to work with for practicing/suffering devops/admins out there…

On 6/15/12 7:29 AM, Christopher Brown wrote:

This exchange (Aaron and Lamont) is almost exactly the conversation
Adam and I had when I first had Nuo write file_edit and that’s why we
decided to bury it. It’s useful, but cuts against the grain.
Pragmatically though, there are files over which you may not have full
control. We constantly tell people that you can ease into as much
Chef as you can take, without having to rigidly accept our model.
Doesn’t saying "but you can only manage whole files, rendered by Chef"
blunt that argument?

-C

On Fri, Jun 15, 2012 at 7:21 AM, Aaron Petersonaaron@opscode.com
wrote:

Lamont "I promise I’ll only use it correctly, tactfully, and rarely"
Granquist, your argument does not persuade. :). But even if you held
true to that…

-1, I could never recommend this, and it fails most methodological
tests that Chef works under.

It should be trivial to take the whole file and templatize it. If you
cannot just easily take the whole file under management, that means
there is interestingly different content “out there” that represents
unmanaged yet important operational state. Now you are making
successive error-prone edits to a collection of interestingly
different files, and there is no guarantee that you will not have
drift unless you have already analyzed the differences across the
collection, at which point you might as well manage the whole thing.

Even if it is consistent, or the file differences are reproduced
normally by an underlying OS or software process, I had better only
add but never delete file edits to recipes, because if ever a machine
misses a past edit because it is off or whatever, or we launch a new
machine, we won’t be creating the file correctly.

Generally, if you cannot always guarantee the recreation of the file
in its current state, you have entombed state and risk and have failed
to create an infrastructure you can reason about or rebuild.

Making it a full resource is the Chef imprimatur to use it, and this
isn’t just rope, it’s already tied into a noose and permanently
attached to a scaffold. IMO.

-a

On Jun 14, 2012, at 14:02, Adam Jacobadam@opscode.com wrote:

I wonder if it’s time to finally turn that into a resource, even
though I think there is near universal agreement it’s not a great
idea?

Adam

On Thu, Jun 14, 2012 at 4:54 AM, AJ Christensenaj@junglist.gen.nz
wrote:

https://github.com/opscode/chef/blob/master/chef/lib/chef/util/file_edit.rb

On 14 June 2012 23:25, Bryan Berrybryan.berry@gmail.com wrote:

minimin, u can use the template resource for this purpose

questions like this are probably better suited to the main chef
mailing list

On Thu, Jun 14, 2012 at 1:11 PM,minmini17@gmail.com wrote:

Hi,

I saw a file resource which can be used to create and delete a
file,but i
want
to edit a file with values provided by the user.
Does chef provides such facility??


Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com


Patrick Collins


Juanje


#15

Yeah - I think I was wrong, and you were right: we should’ve made it a resource.

I was wrong about dry run mode and OpenID, too. :slight_smile:

Adam

On Fri, Jun 15, 2012 at 7:29 AM, Christopher Brown cb@opscode.com wrote:

This exchange (Aaron and Lamont) is almost exactly the conversation
Adam and I had when I first had Nuo write file_edit and that’s why we
decided to bury it. It’s useful, but cuts against the grain.
Pragmatically though, there are files over which you may not have full
control. We constantly tell people that you can ease into as much
Chef as you can take, without having to rigidly accept our model.
Doesn’t saying "but you can only manage whole files, rendered by Chef"
blunt that argument?

-C

On Fri, Jun 15, 2012 at 7:21 AM, Aaron Peterson aaron@opscode.com wrote:

Lamont "I promise I’ll only use it correctly, tactfully, and rarely"
Granquist, your argument does not persuade. :). But even if you held
true to that…

-1, I could never recommend this, and it fails most methodological
tests that Chef works under.

It should be trivial to take the whole file and templatize it. If you
cannot just easily take the whole file under management, that means
there is interestingly different content “out there” that represents
unmanaged yet important operational state. Now you are making
successive error-prone edits to a collection of interestingly
different files, and there is no guarantee that you will not have
drift unless you have already analyzed the differences across the
collection, at which point you might as well manage the whole thing.

Even if it is consistent, or the file differences are reproduced
normally by an underlying OS or software process, I had better only
add but never delete file edits to recipes, because if ever a machine
misses a past edit because it is off or whatever, or we launch a new
machine, we won’t be creating the file correctly.

Generally, if you cannot always guarantee the recreation of the file
in its current state, you have entombed state and risk and have failed
to create an infrastructure you can reason about or rebuild.

Making it a full resource is the Chef imprimatur to use it, and this
isn’t just rope, it’s already tied into a noose and permanently
attached to a scaffold. IMO.

-a

On Jun 14, 2012, at 14:02, Adam Jacob adam@opscode.com wrote:

I wonder if it’s time to finally turn that into a resource, even
though I think there is near universal agreement it’s not a great
idea?

Adam

On Thu, Jun 14, 2012 at 4:54 AM, AJ Christensen aj@junglist.gen.nz wrote:

https://github.com/opscode/chef/blob/master/chef/lib/chef/util/file_edit.rb

On 14 June 2012 23:25, Bryan Berry bryan.berry@gmail.com wrote:

minimin, u can use the template resource for this purpose

questions like this are probably better suited to the main chef mailing list

On Thu, Jun 14, 2012 at 1:11 PM, minmini17@gmail.com wrote:

Hi,

I saw a file resource which can be used to create and delete a file,but i
want
to edit a file with values provided by the user.
Does chef provides such facility??


Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com


Christopher Brown, Chief Technical Officer, Opscode, Inc.
T: (425) 502-5522, E: cb@opscode.com
IRC, Github: skeptomai
Twitter: @skeptomai


Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com


#16

For backward compatibility I suppose it would be better as a separate
resource.

Doing some more thinking about this, it seems like a really common use case
is performing search/replace on entire lines in configuration files. I’ve
used FileEdit to manipulate /etc/hosts and sshd.conf many times. The more
complicated or lengthy the configuration file seems to be, the more I just
want to concentrate on ensuring parts of the file are as I expect. For
example, if I want to turn off password authentication in sshd, I don’t
want to have to take control of the whole file – I just want to make sure
the directive is set to “No.”

I’m going to throw out some sample code here. Bare with me.

file_contents “/etc/hosts” do
replace_line(/^127.0.0.1/) do |match|
# returning a string uses FileEdit to replace line
"#{match} test.mydomain.com" unless match.include? “test.mydomain.com
# returning nil doesn’t replace anything
end
end

file_contents “/etc/ssh/sshd_config” do
replace_line(/^#?PasswordAuthentication/) { “PasswordAuthentication No” }
end

Of course it’s up to who implements it to keep it idempotent.

I’m not fully convinced it’s a good idea to implement add, append, replace,
and remove. It seems “replace_line” is a good compromise and fulfills the
use cases I can think of.

On Fri, Jun 15, 2012 at 2:57 PM, Juanje Ojeda Croissier <
juanje.ojeda@gmail.com> wrote:

When I started with Chef I found a case where I needed something for
replace lines or checked in a file and I didn’t find. Everyone told me
to use templates and so, but sometimes the app doesn’t support and
it’s worse the workaround for using templates.

I did create my LWRP (on top of ‘file_edit’) and I tried to do it in a
more idempotent and safe way (probably not enogh, though), but I knew
it was just for a few special cases, no a silver bullet and it has to
be used carefully.

IMHO, if you have this in mind, it is much better to have a resource
which try to care about the idempotence, that use scripts and execute
resources to deal with those cases.

In case anyone lik to use or improve the LWRP, it’s here:
http://community.opscode.com/cookbooks/conf

On Fri, Jun 15, 2012 at 7:33 PM, Pat Collins pat@burned.com wrote:

+1

If you have both the script and execute resources (with root by
default!),
you are allowing users to shoot themselves in the foot much more than if
you
allow regular expression search and replace with a resource.

Tell the story in the documentation. Evangelize the Chef-preferred way
but
then concede there are times when there is no ideal situation. Show users
how much flexibility you are giving them. Also just like with any
potentially dangerous tool, explain the potential consequences and give
them
fair warning. I believe users will feel empowered. More and more,
developers
can say to sysadmins "look, we can start implementing this
incrementally."
There are a lot of folks who will appreciate this.

I see it being an extension of the file resource.

On Fri, Jun 15, 2012 at 12:13 PM, Lamont Granquist lamont@opscode.com
wrote:

Problem is that while this is always the most correct way to do things,
that it may not be the correct way for the business that is using chef
to

get things done right now. Its quite possible that you’ve got a huge
mess

on your hands and the easiest way to alleviate 90% of the pain is to
simply

enforce one line of state in a config file that has a thousand unique
snowflakes that have been deployed by hand over the last decade. The
’right’ thing to do is clean up the whole mess, but that often takes
10x the

effort, since you often need to audit the whole thing and figure out
what

deltas in all of the config options are actual necessary business
requirements and what is cargo cult and deltas that are unnecessary.
Leaving that unmanaged state out there is often messy, but I’ve made
that

decision a lot in order to move forward in increments. We also leave
most

of the state of the system unmanaged because most of the state doesn’t
matter.

There’s also other use cases, such as using bundled chef-solo to manage
state for an application that needs to ship and be integrated with
uncontrolled external config management systems. Such code may need to
enforce state in a file, but the rest of the state of that file is
complete

out of your control and fundamentally cannot be brought under your own
management.

Within the same company you can wind up seeing this problem where
departments may have written tooling that edits files already and config
management systems may need to integrate with those and chef may need to
make partial edits to the portion of the file that it ‘controls’. The
right

solution there is always to fix that separation of concerns and make
chef

authoritative and to call a tool in order so that chef could render the
whole template, but stupid thing like that can make for political
snarls in

large organizations with legacy infrastructure and legacy teams, and I
don’t

think chef should be in the business of inserting itself into politics.

And this goes to the whole meme of being “opinionated”, and I’m already
highly skeptical and tired of that meme and think its a bad one for this
kind of product. Its also why I don’t use CFEngine anymore because of
some

of the opinions of that crowd that made it almost deliberately
difficult to

integrate that product with a central database. I really don’t like
using

tools that have opinions that force me into making certain cost/benefit
decisions in my job, I need to be able to weigh those decisions myself.

So, yes, I’ve run into this decision over and over through the years,
and

have quite often determined that it is far from trivial to take the
whole

file and templatize it (its often far from trivial to understand the
actual

requirements from the cargo culting), and have chosen to either do the
expedient thing for the business and clean up a fragment of it first, or
simple to ignore the rest of the config. In some cases I’ve actually
written code that does “if the file exists, edit it to enforce this one
line, if the file does not exist, then use my nice standardized
template” to

gain eventual convergence and reduce config deltas over time while at
the

same time punting the existing mess, and allowing me to test my
standardized

template every time a new server goes into prod.

And the whole use of “our model” and such is language that is kind of
annoying. Chef didn’t invent that, its just common practice that its
the

better way of doing things in the unix sysadmin world, but its not the
/only/ configuration management design problem out there – although
for a

small startup with 40 nodes in the cloud that has been using chef since
day

#1 it may be the only one that you need. When talking about walking
into

thousands of unmanaged servers grown organically over a decade and on
different major flavors of operating systems… You will need every
trick

in the book.

And being able to use every trick in the book and be able to easily get
the job done – no matter how broken the existing systems and no matter
how

ingrained the departmental politics – is what is going to make chef
awesome

to work with for practicing/suffering devops/admins out there…

On 6/15/12 7:29 AM, Christopher Brown wrote:

This exchange (Aaron and Lamont) is almost exactly the conversation
Adam and I had when I first had Nuo write file_edit and that’s why we
decided to bury it. It’s useful, but cuts against the grain.
Pragmatically though, there are files over which you may not have full
control. We constantly tell people that you can ease into as much
Chef as you can take, without having to rigidly accept our model.
Doesn’t saying "but you can only manage whole files, rendered by Chef"
blunt that argument?

-C

On Fri, Jun 15, 2012 at 7:21 AM, Aaron Petersonaaron@opscode.com
wrote:

Lamont "I promise I’ll only use it correctly, tactfully, and rarely"
Granquist, your argument does not persuade. :). But even if you held
true to that…

-1, I could never recommend this, and it fails most methodological
tests that Chef works under.

It should be trivial to take the whole file and templatize it. If you
cannot just easily take the whole file under management, that means
there is interestingly different content “out there” that represents
unmanaged yet important operational state. Now you are making
successive error-prone edits to a collection of interestingly
different files, and there is no guarantee that you will not have
drift unless you have already analyzed the differences across the
collection, at which point you might as well manage the whole thing.

Even if it is consistent, or the file differences are reproduced
normally by an underlying OS or software process, I had better only
add but never delete file edits to recipes, because if ever a machine
misses a past edit because it is off or whatever, or we launch a new
machine, we won’t be creating the file correctly.

Generally, if you cannot always guarantee the recreation of the file
in its current state, you have entombed state and risk and have failed
to create an infrastructure you can reason about or rebuild.

Making it a full resource is the Chef imprimatur to use it, and this
isn’t just rope, it’s already tied into a noose and permanently
attached to a scaffold. IMO.

-a

On Jun 14, 2012, at 14:02, Adam Jacobadam@opscode.com wrote:

I wonder if it’s time to finally turn that into a resource, even
though I think there is near universal agreement it’s not a great
idea?

Adam

On Thu, Jun 14, 2012 at 4:54 AM, AJ Christensenaj@junglist.gen.nz
wrote:

https://github.com/opscode/chef/blob/master/chef/lib/chef/util/file_edit.rb

On 14 June 2012 23:25, Bryan Berrybryan.berry@gmail.com wrote:

minimin, u can use the template resource for this purpose

questions like this are probably better suited to the main chef
mailing list

On Thu, Jun 14, 2012 at 1:11 PM,minmini17@gmail.com wrote:

Hi,

I saw a file resource which can be used to create and delete a
file,but i
want
to edit a file with values provided by the user.
Does chef provides such facility??


Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com


Patrick Collins


Juanje

http://about.me/juanje


Patrick Collins


#17

Actually, I found some cases were ‘append’ or ‘add’ (after some line)
is needed/useful.
For example, what if you need your line with “test.mydomain.com” to be
at the /etc/hosts but you don’t have that string there to be replaced?
To replace the line is ok if the line (or a patern) already exists,
but the first time, maybe it doesn’t. It should be nice to have a way
to add at the end of the file or at some specific point (some apps
need specific order in they conffiles).

On Fri, Jun 15, 2012 at 9:09 PM, Pat Collins pat@burned.com wrote:

For backward compatibility I suppose it would be better as a separate
resource.

Doing some more thinking about this, it seems like a really common use case
is performing search/replace on entire lines in configuration files. I’ve
used FileEdit to manipulate /etc/hosts and sshd.conf many times. The more
complicated or lengthy the configuration file seems to be, the more I just
want to concentrate on ensuring parts of the file are as I expect. For
example, if I want to turn off password authentication in sshd, I don’t want
to have to take control of the whole file – I just want to make sure the
directive is set to “No.”

I’m going to throw out some sample code here. Bare with me.

file_contents “/etc/hosts” do
replace_line(/^127.0.0.1/) do |match|
# returning a string uses FileEdit to replace line
"#{match} test.mydomain.com" unless match.include? “test.mydomain.com
# returning nil doesn’t replace anything
end
end

file_contents “/etc/ssh/sshd_config” do
replace_line(/^#?PasswordAuthentication/) { “PasswordAuthentication No” }
end

Of course it’s up to who implements it to keep it idempotent.

I’m not fully convinced it’s a good idea to implement add, append, replace,
and remove. It seems “replace_line” is a good compromise and fulfills the
use cases I can think of.

On Fri, Jun 15, 2012 at 2:57 PM, Juanje Ojeda Croissier
juanje.ojeda@gmail.com wrote:

When I started with Chef I found a case where I needed something for
replace lines or checked in a file and I didn’t find. Everyone told me
to use templates and so, but sometimes the app doesn’t support and
it’s worse the workaround for using templates.

I did create my LWRP (on top of ‘file_edit’) and I tried to do it in a
more idempotent and safe way (probably not enogh, though), but I knew
it was just for a few special cases, no a silver bullet and it has to
be used carefully.

IMHO, if you have this in mind, it is much better to have a resource
which try to care about the idempotence, that use scripts and execute
resources to deal with those cases.

In case anyone lik to use or improve the LWRP, it’s here:
http://community.opscode.com/cookbooks/conf

On Fri, Jun 15, 2012 at 7:33 PM, Pat Collins pat@burned.com wrote:

+1

If you have both the script and execute resources (with root by
default!),
you are allowing users to shoot themselves in the foot much more than if
you
allow regular expression search and replace with a resource.

Tell the story in the documentation. Evangelize the Chef-preferred way
but
then concede there are times when there is no ideal situation. Show
users
how much flexibility you are giving them. Also just like with any
potentially dangerous tool, explain the potential consequences and give
them
fair warning. I believe users will feel empowered. More and more,
developers
can say to sysadmins "look, we can start implementing this
incrementally."
There are a lot of folks who will appreciate this.

I see it being an extension of the file resource.

On Fri, Jun 15, 2012 at 12:13 PM, Lamont Granquist lamont@opscode.com
wrote:

Problem is that while this is always the most correct way to do things,
that it may not be the correct way for the business that is using chef
to
get things done right now. Its quite possible that you’ve got a huge
mess
on your hands and the easiest way to alleviate 90% of the pain is to
simply
enforce one line of state in a config file that has a thousand unique
snowflakes that have been deployed by hand over the last decade. The
’right’ thing to do is clean up the whole mess, but that often takes
10x the
effort, since you often need to audit the whole thing and figure out
what
deltas in all of the config options are actual necessary business
requirements and what is cargo cult and deltas that are unnecessary.
Leaving that unmanaged state out there is often messy, but I’ve made
that
decision a lot in order to move forward in increments. We also leave
most
of the state of the system unmanaged because most of the state doesn’t
matter.

There’s also other use cases, such as using bundled chef-solo to manage
state for an application that needs to ship and be integrated with
uncontrolled external config management systems. Such code may need to
enforce state in a file, but the rest of the state of that file is
complete
out of your control and fundamentally cannot be brought under your own
management.

Within the same company you can wind up seeing this problem where
departments may have written tooling that edits files already and
config
management systems may need to integrate with those and chef may need
to
make partial edits to the portion of the file that it ‘controls’. The
right
solution there is always to fix that separation of concerns and make
chef
authoritative and to call a tool in order so that chef could render the
whole template, but stupid thing like that can make for political
snarls in
large organizations with legacy infrastructure and legacy teams, and I
don’t
think chef should be in the business of inserting itself into politics.

And this goes to the whole meme of being “opinionated”, and I’m already
highly skeptical and tired of that meme and think its a bad one for
this
kind of product. Its also why I don’t use CFEngine anymore because of
some
of the opinions of that crowd that made it almost deliberately
difficult to
integrate that product with a central database. I really don’t like
using
tools that have opinions that force me into making certain cost/benefit
decisions in my job, I need to be able to weigh those decisions myself.

So, yes, I’ve run into this decision over and over through the years,
and
have quite often determined that it is far from trivial to take the
whole
file and templatize it (its often far from trivial to understand the
actual
requirements from the cargo culting), and have chosen to either do the
expedient thing for the business and clean up a fragment of it first,
or
simple to ignore the rest of the config. In some cases I’ve actually
written code that does “if the file exists, edit it to enforce this one
line, if the file does not exist, then use my nice standardized
template” to
gain eventual convergence and reduce config deltas over time while at
the
same time punting the existing mess, and allowing me to test my
standardized
template every time a new server goes into prod.

And the whole use of “our model” and such is language that is kind of
annoying. Chef didn’t invent that, its just common practice that its
the
better way of doing things in the unix sysadmin world, but its not the
/only/ configuration management design problem out there – although
for a
small startup with 40 nodes in the cloud that has been using chef since
day
#1 it may be the only one that you need. When talking about walking
into
thousands of unmanaged servers grown organically over a decade and on
different major flavors of operating systems… You will need every
trick
in the book.

And being able to use every trick in the book and be able to easily get
the job done – no matter how broken the existing systems and no matter
how
ingrained the departmental politics – is what is going to make chef
awesome
to work with for practicing/suffering devops/admins out there…

On 6/15/12 7:29 AM, Christopher Brown wrote:

This exchange (Aaron and Lamont) is almost exactly the conversation
Adam and I had when I first had Nuo write file_edit and that’s why we
decided to bury it. It’s useful, but cuts against the grain.
Pragmatically though, there are files over which you may not have full
control. We constantly tell people that you can ease into as much
Chef as you can take, without having to rigidly accept our model.
Doesn’t saying "but you can only manage whole files, rendered by Chef"
blunt that argument?

-C

On Fri, Jun 15, 2012 at 7:21 AM, Aaron Petersonaaron@opscode.com
wrote:

Lamont "I promise I’ll only use it correctly, tactfully, and rarely"
Granquist, your argument does not persuade. :). But even if you held
true to that…

-1, I could never recommend this, and it fails most methodological
tests that Chef works under.

It should be trivial to take the whole file and templatize it. If you
cannot just easily take the whole file under management, that means
there is interestingly different content “out there” that represents
unmanaged yet important operational state. Now you are making
successive error-prone edits to a collection of interestingly
different files, and there is no guarantee that you will not have
drift unless you have already analyzed the differences across the
collection, at which point you might as well manage the whole thing.

Even if it is consistent, or the file differences are reproduced
normally by an underlying OS or software process, I had better only
add but never delete file edits to recipes, because if ever a machine
misses a past edit because it is off or whatever, or we launch a new
machine, we won’t be creating the file correctly.

Generally, if you cannot always guarantee the recreation of the file
in its current state, you have entombed state and risk and have
failed
to create an infrastructure you can reason about or rebuild.

Making it a full resource is the Chef imprimatur to use it, and this
isn’t just rope, it’s already tied into a noose and permanently
attached to a scaffold. IMO.

-a

On Jun 14, 2012, at 14:02, Adam Jacobadam@opscode.com wrote:

I wonder if it’s time to finally turn that into a resource, even
though I think there is near universal agreement it’s not a great
idea?

Adam

On Thu, Jun 14, 2012 at 4:54 AM, AJ Christensenaj@junglist.gen.nz
wrote:

https://github.com/opscode/chef/blob/master/chef/lib/chef/util/file_edit.rb

On 14 June 2012 23:25, Bryan Berrybryan.berry@gmail.com wrote:

minimin, u can use the template resource for this purpose

questions like this are probably better suited to the main chef
mailing list

On Thu, Jun 14, 2012 at 1:11 PM,minmini17@gmail.com wrote:

Hi,

I saw a file resource which can be used to create and delete a
file,but i
want
to edit a file with values provided by the user.
Does chef provides such facility??


Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com


Patrick Collins


Juanje

http://about.me/juanje


Patrick Collins


Juanje


#18

I wouldn’t use file_edit even if it’d be available and I think the way to template the file you need to change is sufficient. It’s just way more sane and pays in the long run to manage files as templates. We’ve had numerous problems with cfengine because of this in the past and stopped using it.

Regards,
Mike

On Jun 15, 2012, at 10:41 PM, Juanje Ojeda Croissier wrote:

Actually, I found some cases were ‘append’ or ‘add’ (after some line)
is needed/useful.
For example, what if you need your line with “test.mydomain.com” to be
at the /etc/hosts but you don’t have that string there to be replaced?
To replace the line is ok if the line (or a patern) already exists,
but the first time, maybe it doesn’t. It should be nice to have a way
to add at the end of the file or at some specific point (some apps
need specific order in they conffiles).

On Fri, Jun 15, 2012 at 9:09 PM, Pat Collins pat@burned.com wrote:

For backward compatibility I suppose it would be better as a separate
resource.

Doing some more thinking about this, it seems like a really common use case
is performing search/replace on entire lines in configuration files. I’ve
used FileEdit to manipulate /etc/hosts and sshd.conf many times. The more
complicated or lengthy the configuration file seems to be, the more I just
want to concentrate on ensuring parts of the file are as I expect. For
example, if I want to turn off password authentication in sshd, I don’t want
to have to take control of the whole file – I just want to make sure the
directive is set to “No.”

I’m going to throw out some sample code here. Bare with me.

file_contents “/etc/hosts” do
replace_line(/^127.0.0.1/) do |match|
# returning a string uses FileEdit to replace line
"#{match} test.mydomain.com" unless match.include? “test.mydomain.com
# returning nil doesn’t replace anything
end
end

file_contents “/etc/ssh/sshd_config” do
replace_line(/^#?PasswordAuthentication/) { “PasswordAuthentication No” }
end

Of course it’s up to who implements it to keep it idempotent.

I’m not fully convinced it’s a good idea to implement add, append, replace,
and remove. It seems “replace_line” is a good compromise and fulfills the
use cases I can think of.

On Fri, Jun 15, 2012 at 2:57 PM, Juanje Ojeda Croissier
juanje.ojeda@gmail.com wrote:

When I started with Chef I found a case where I needed something for
replace lines or checked in a file and I didn’t find. Everyone told me
to use templates and so, but sometimes the app doesn’t support and
it’s worse the workaround for using templates.

I did create my LWRP (on top of ‘file_edit’) and I tried to do it in a
more idempotent and safe way (probably not enogh, though), but I knew
it was just for a few special cases, no a silver bullet and it has to
be used carefully.

IMHO, if you have this in mind, it is much better to have a resource
which try to care about the idempotence, that use scripts and execute
resources to deal with those cases.

In case anyone lik to use or improve the LWRP, it’s here:
http://community.opscode.com/cookbooks/conf

On Fri, Jun 15, 2012 at 7:33 PM, Pat Collins pat@burned.com wrote:

+1

If you have both the script and execute resources (with root by
default!),
you are allowing users to shoot themselves in the foot much more than if
you
allow regular expression search and replace with a resource.

Tell the story in the documentation. Evangelize the Chef-preferred way
but
then concede there are times when there is no ideal situation. Show
users
how much flexibility you are giving them. Also just like with any
potentially dangerous tool, explain the potential consequences and give
them
fair warning. I believe users will feel empowered. More and more,
developers
can say to sysadmins "look, we can start implementing this
incrementally."
There are a lot of folks who will appreciate this.

I see it being an extension of the file resource.

On Fri, Jun 15, 2012 at 12:13 PM, Lamont Granquist lamont@opscode.com
wrote:

Problem is that while this is always the most correct way to do things,
that it may not be the correct way for the business that is using chef
to
get things done right now. Its quite possible that you’ve got a huge
mess
on your hands and the easiest way to alleviate 90% of the pain is to
simply
enforce one line of state in a config file that has a thousand unique
snowflakes that have been deployed by hand over the last decade. The
’right’ thing to do is clean up the whole mess, but that often takes
10x the
effort, since you often need to audit the whole thing and figure out
what
deltas in all of the config options are actual necessary business
requirements and what is cargo cult and deltas that are unnecessary.
Leaving that unmanaged state out there is often messy, but I’ve made
that
decision a lot in order to move forward in increments. We also leave
most
of the state of the system unmanaged because most of the state doesn’t
matter.

There’s also other use cases, such as using bundled chef-solo to manage
state for an application that needs to ship and be integrated with
uncontrolled external config management systems. Such code may need to
enforce state in a file, but the rest of the state of that file is
complete
out of your control and fundamentally cannot be brought under your own
management.

Within the same company you can wind up seeing this problem where
departments may have written tooling that edits files already and
config
management systems may need to integrate with those and chef may need
to
make partial edits to the portion of the file that it ‘controls’. The
right
solution there is always to fix that separation of concerns and make
chef
authoritative and to call a tool in order so that chef could render the
whole template, but stupid thing like that can make for political
snarls in
large organizations with legacy infrastructure and legacy teams, and I
don’t
think chef should be in the business of inserting itself into politics.

And this goes to the whole meme of being “opinionated”, and I’m already
highly skeptical and tired of that meme and think its a bad one for
this
kind of product. Its also why I don’t use CFEngine anymore because of
some
of the opinions of that crowd that made it almost deliberately
difficult to
integrate that product with a central database. I really don’t like
using
tools that have opinions that force me into making certain cost/benefit
decisions in my job, I need to be able to weigh those decisions myself.

So, yes, I’ve run into this decision over and over through the years,
and
have quite often determined that it is far from trivial to take the
whole
file and templatize it (its often far from trivial to understand the
actual
requirements from the cargo culting), and have chosen to either do the
expedient thing for the business and clean up a fragment of it first,
or
simple to ignore the rest of the config. In some cases I’ve actually
written code that does “if the file exists, edit it to enforce this one
line, if the file does not exist, then use my nice standardized
template” to
gain eventual convergence and reduce config deltas over time while at
the
same time punting the existing mess, and allowing me to test my
standardized
template every time a new server goes into prod.

And the whole use of “our model” and such is language that is kind of
annoying. Chef didn’t invent that, its just common practice that its
the
better way of doing things in the unix sysadmin world, but its not the
/only/ configuration management design problem out there – although
for a
small startup with 40 nodes in the cloud that has been using chef since
day
#1 it may be the only one that you need. When talking about walking
into
thousands of unmanaged servers grown organically over a decade and on
different major flavors of operating systems… You will need every
trick
in the book.

And being able to use every trick in the book and be able to easily get
the job done – no matter how broken the existing systems and no matter
how
ingrained the departmental politics – is what is going to make chef
awesome
to work with for practicing/suffering devops/admins out there…

On 6/15/12 7:29 AM, Christopher Brown wrote:

This exchange (Aaron and Lamont) is almost exactly the conversation
Adam and I had when I first had Nuo write file_edit and that’s why we
decided to bury it. It’s useful, but cuts against the grain.
Pragmatically though, there are files over which you may not have full
control. We constantly tell people that you can ease into as much
Chef as you can take, without having to rigidly accept our model.
Doesn’t saying "but you can only manage whole files, rendered by Chef"
blunt that argument?

-C

On Fri, Jun 15, 2012 at 7:21 AM, Aaron Petersonaaron@opscode.com
wrote:

Lamont "I promise I’ll only use it correctly, tactfully, and rarely"
Granquist, your argument does not persuade. :). But even if you held
true to that…

-1, I could never recommend this, and it fails most methodological
tests that Chef works under.

It should be trivial to take the whole file and templatize it. If you
cannot just easily take the whole file under management, that means
there is interestingly different content “out there” that represents
unmanaged yet important operational state. Now you are making
successive error-prone edits to a collection of interestingly
different files, and there is no guarantee that you will not have
drift unless you have already analyzed the differences across the
collection, at which point you might as well manage the whole thing.

Even if it is consistent, or the file differences are reproduced
normally by an underlying OS or software process, I had better only
add but never delete file edits to recipes, because if ever a machine
misses a past edit because it is off or whatever, or we launch a new
machine, we won’t be creating the file correctly.

Generally, if you cannot always guarantee the recreation of the file
in its current state, you have entombed state and risk and have
failed
to create an infrastructure you can reason about or rebuild.

Making it a full resource is the Chef imprimatur to use it, and this
isn’t just rope, it’s already tied into a noose and permanently
attached to a scaffold. IMO.

-a

On Jun 14, 2012, at 14:02, Adam Jacobadam@opscode.com wrote:

I wonder if it’s time to finally turn that into a resource, even
though I think there is near universal agreement it’s not a great
idea?

Adam

On Thu, Jun 14, 2012 at 4:54 AM, AJ Christensenaj@junglist.gen.nz
wrote:

https://github.com/opscode/chef/blob/master/chef/lib/chef/util/file_edit.rb

On 14 June 2012 23:25, Bryan Berrybryan.berry@gmail.com wrote:

minimin, u can use the template resource for this purpose

questions like this are probably better suited to the main chef
mailing list

On Thu, Jun 14, 2012 at 1:11 PM,minmini17@gmail.com wrote:

Hi,

I saw a file resource which can be used to create and delete a
file,but i
want
to edit a file with values provided by the user.
Does chef provides such facility??


Opscode, Inc.
Adam Jacob, Chief Customer Officer
T: (206) 619-7151 E: adam@opscode.com


Patrick Collins


Juanje

http://about.me/juanje


Patrick Collins


Juanje

http://about.me/juanje


Mike Adolphs
Stitenstrasse 24
23554 Luebeck

  • Germany -

Mail. mike@fooforge.com
Web. http://fooforge.com/
Github. https://github.com/fooforge/
XING. http://www.xing.com/profile/Mike_Adolphs


#19

]] Mike Adolphs

I wouldn’t use file_edit even if it’d be available and I think the way
to template the file you need to change is sufficient. It’s just way
more sane and pays in the long run to manage files as templates. We’ve
had numerous problems with cfengine because of this in the past and
stopped using it.

file editing is conceptually no different from having

include “/etc/foo/conf.d/*.conf”

in your package. I don’t see people complaining about that, and I’m
pretty sure somebody, somewhere has been bitten by an include that
grabbed a file they didn’t know was there and that caused problems.

Heck, if we drag it to its extreme conclusion, we shouldn’t have file
management in chef at all, since what you should be doing is managing
file systems as complete blocks. :stuck_out_tongue:

File editing is a tool. It’s a tool that can be used well and it’s a
tool that can be used badly. I think it’s important that Chef is
pragmatic and allows you to implement hacks when you need to, and
sometimes the hack you need is to do file editing. We should warn about
the dangers of blindly editing files in the documentation, but just
having the tool there is useful.


Tollef Fog Heen
UNIX is user friendly, it’s just picky about who its friends are


#20

I dig the pragmatic case, where it is essentially impossible to move
forward without piecemeal manipulation of a given file. It seems important
to note at that point you’re in it for bare manageability, and no longer in
the “reproduce the the whole infrastructure from scratch” use case. I think
that puts us out on a limb when talking about what Chef does or should do.

Having an in-built file edit facility is great, as is arbitrary ruby or
script evaluation or gem inclusion in Chef. They all let you get the job,
dirty or not, done in a rich unified context. I’m never going to argue
against that, and edit file and family is available whether or not we make
it core Chef or provide a resource interface. It goes a bit far to
instantly equate discussions of potential primitives instantly with the
"opinionated meme". Every existing primitive has plently of opinions baked
in to it.

The problem I will continue to have is that, though file edit is like
script/execute in that you have to bring your own idempotence contract,
with file edit - especially in the very situations where you’ve been forced
into this corner - it is orders of magnitude harder to write idempotency
correctly. If you fail to consider edge cases, overlaps, past typos, etc.,
I’ve seen the big old’ shark bite in the ass that is entirely not visible
in any of the current code, and the hunt begins on what horror you birthed
perhaps long ago so you can actually move forward again - but this time in
the middle of an outage.

I’d go so far as to argue that to be generally successful when approaching
non-trivial file edit tasks you have to build and maintain a mental map of
what is out there and what has gone before, a form of tribal knowledge that
might actually be the contrapositive of idempotent: if you are not in the
desired state, then you’d better know if you ever tried to apply the
function.

What would interest me is knowing how promoting file edit to a resource
could perhaps help people who find themselves wanting/ needing/ thinking
they need to use file edits.

-A

On Fri, Jun 15, 2012 at 1:06 PM, Adam Jacob adam@opscode.com wrote:

Yeah - I think I was wrong, and you were right: we should’ve made it a
resource.

I was wrong about dry run mode and OpenID, too. :slight_smile:

Adam

On Fri, Jun 15, 2012 at 7:29 AM, Christopher Brown cb@opscode.com wrote:

This exchange (Aaron and Lamont) is almost exactly the conversation
Adam and I had when I first had Nuo write file_edit and that’s why we
decided to bury it. It’s useful, but cuts against the grain.
Pragmatically though, there are files over which you may not have full
control. We constantly tell people that you can ease into as much
Chef as you can take, without having to rigidly accept our model.
Doesn’t saying "but you can only manage whole files, rendered by Chef"
blunt that argument?

-C

On Fri, Jun 15, 2012 at 7:21 AM, Aaron Peterson aaron@opscode.com
wrote:

Lamont "I promise I’ll only use it correctly, tactfully, and rarely"
Granquist, your argument does not persuade. :). But even if you held
true to that…

-1, I could never recommend this, and it fails most methodological
tests that Chef works under.

It should be trivial to take the whole file and templatize it. If you
cannot just easily take the whole file under management, that means
there is interestingly different content “out there” that represents
unmanaged yet important operational state. Now you are making
successive error-prone edits to a collection of interestingly
different files, and there is no guarantee that you will not have
drift unless you have already analyzed the differences across the
collection, at which point you might as well manage the whole thing.

Even if it is consistent, or the file differences are reproduced
normally by an underlying OS or software process, I had better only
add but never delete file edits to recipes, because if ever a machine
misses a past edit because it is off or whatever, or we launch a new
machine, we won’t be creating the file correctly.

Generally, if you cannot always guarantee the recreation of the file
in its current state, you have entombed state and risk and have failed
to create an infrastructure you can reason about or rebuild.

Making it a full resource is the Chef imprimatur to use it, and this
isn’t just rope, it’s already tied into a noose and permanently
attached to a scaffold. IMO.

-a

On Jun 14, 2012, at 14:02, Adam Jacob adam@opscode.com wrote:

I wonder if it’s time to finally turn that into a resource, even
though I think there is near universal agreement it’s not a great
idea?

Adam