Place file only once per version

Damien_Roche · August 15, 2013, 10:20am

Hello,

Would would be the best way to only place a file once per cookbook version. It’s modified and removed outside chef.

Regards
D.

Ranjib · August 15, 2013, 10:40am

inside a recipe

if node.attribute?('bar')
Chef::Log.info('already run')
else
ruby_block 'set_attribute' do
block do
nod.set['foo']='bar'
end
end
file '/foo/bar' do
content "foobar"
end
end

On Thu, Aug 15, 2013 at 3:20 AM, dcroche@gmail.com wrote:

Hello,

Would would be the best way to only place a file once per cookbook
version. It's modified and removed outside chef.

Regards
D.

Damien_Roche · August 15, 2013, 4:24pm

Is it possible to get a checksum of a cookbook file and store this in the attribute?

require 'Digest'
cb_filename = ?

if node.attribute['staged_checksum'] == Digest::SHA2.file(File.new(cb_filename))
Chef::Log.info("Already been staged")
else
Chef::Log.info("Update staging")
node.attribute['staged_checksum'] = Digest::SHA2.file(File.new(cb_filename))
end

On 15 Aug 2013, at 11:40, Ranjib Dey dey.ranjib@gmail.com wrote:

inside a recipe

if node.attribute?('bar')
Chef::Log.info('already run')
else
ruby_block 'set_attribute' do
block do
nod.set['foo']='bar'
end
end
file '/foo/bar' do
content "foobar"
end
end

On Thu, Aug 15, 2013 at 3:20 AM, dcroche@gmail.com wrote:
Hello,

Would would be the best way to only place a file once per cookbook version. It's modified and removed outside chef.

Regards
D.

Ranjib · August 15, 2013, 4:43pm

well, you can do that using ruby stdlib, as well as low level chef api
(that chef uses to make file/template/cookbookfile/remote_file [1]
idempotent). But why you want to do it? If you are doing it
inside a recipe, probably its a bad idea, consider pushing them inside
providers or libraries

On Thu, Aug 15, 2013 at 9:24 AM, dcroche@gmail.com wrote:

Is it possible to get a checksum of a cookbook file and store this in the
attribute?

require 'Digest'

cb_filename = ?
if node.attribute['staged_checksum'] == Digest::SHA2.file(File.new(cb_filename))
Chef::Log.info("Already been staged")else
Chef::Log.info("Update staging")

node.attribute['staged_checksum'] = Digest::SHA2.file(File.new(cb_filename))

end

On 15 Aug 2013, at 11:40, Ranjib Dey dey.ranjib@gmail.com wrote:

inside a recipe

if node.attribute?('bar')
Chef::Log.info('already run')
else
ruby_block 'set_attribute' do
block do
nod.set['foo']='bar'
end
end
file '/foo/bar' do
content "foobar"
end
end

On Thu, Aug 15, 2013 at 3:20 AM, dcroche@gmail.com wrote:

Hello,

Would would be the best way to only place a file once per cookbook
version. It's modified and removed outside chef.

Regards
D.

Ranjib · August 15, 2013, 4:44pm

oops , forgot share the reference
[1] chef/lib/chef/mixin/checksum.rb at main · chef/chef · GitHub

On Thu, Aug 15, 2013 at 9:43 AM, Ranjib Dey dey.ranjib@gmail.com wrote:

well, you can do that using ruby stdlib, as well as low level chef api
(that chef uses to make file/template/cookbookfile/remote_file [1]
idempotent). But why you want to do it? If you are doing it
inside a recipe, probably its a bad idea, consider pushing them inside
providers or libraries

On Thu, Aug 15, 2013 at 9:24 AM, dcroche@gmail.com wrote:

Is it possible to get a checksum of a cookbook file and store this in the
attribute?

require 'Digest'

cb_filename = ?
if node.attribute['staged_checksum'] == Digest::SHA2.file(File.new(cb_filename))
Chef::Log.info("Already been staged")else
Chef::Log.info("Update staging")

node.attribute['staged_checksum'] = Digest::SHA2.file(File.new(cb_filename))

end

On 15 Aug 2013, at 11:40, Ranjib Dey dey.ranjib@gmail.com wrote:

inside a recipe

if node.attribute?('bar')
Chef::Log.info('already run')
else
ruby_block 'set_attribute' do
block do
nod.set['foo']='bar'
end
end
file '/foo/bar' do
content "foobar"
end
end

On Thu, Aug 15, 2013 at 3:20 AM, dcroche@gmail.com wrote:

Hello,

Would would be the best way to only place a file once per cookbook
version. It's modified and removed outside chef.

Regards
D.

Andrew_Gross · August 15, 2013, 5:23pm

Whats wrong with

cookbook_file "foo" do
action :create
not_if { ::File.exists?("foo") }
end

On Thu, Aug 15, 2013 at 12:44 PM, Ranjib Dey dey.ranjib@gmail.com wrote:

oops , forgot share the reference
[1] chef/lib/chef/mixin/checksum.rb at main · chef/chef · GitHub

On Thu, Aug 15, 2013 at 9:43 AM, Ranjib Dey dey.ranjib@gmail.com wrote:

well, you can do that using ruby stdlib, as well as low level chef api
(that chef uses to make file/template/cookbookfile/remote_file [1]
idempotent). But why you want to do it? If you are doing it
inside a recipe, probably its a bad idea, consider pushing them inside
providers or libraries

On Thu, Aug 15, 2013 at 9:24 AM, dcroche@gmail.com wrote:

Is it possible to get a checksum of a cookbook file and store this in
the attribute?

require 'Digest'

cb_filename = ?
if node.attribute['staged_checksum'] == Digest::SHA2.file(File.new(cb_filename))
Chef::Log.info("Already been staged")else
Chef::Log.info("Update staging")

node.attribute['staged_checksum'] = Digest::SHA2.file(File.new(cb_filename))

end

On 15 Aug 2013, at 11:40, Ranjib Dey dey.ranjib@gmail.com wrote:

inside a recipe

if node.attribute?('bar')
Chef::Log.info('already run')
else
ruby_block 'set_attribute' do
block do
nod.set['foo']='bar'
end
end
file '/foo/bar' do
content "foobar"
end
end

On Thu, Aug 15, 2013 at 3:20 AM, dcroche@gmail.com wrote:

Hello,

Would would be the best way to only place a file once per cookbook
version. It's modified and removed outside chef.

Regards
D.

Mike · August 15, 2013, 5:39pm

Even better:

cookbook_file "foo" do
action :create_if_missing
end

Boom.
-M

On Thu, Aug 15, 2013 at 1:23 PM, Andrew Gross andrew@yipit.com wrote:

Whats wrong with

cookbook_file "foo" do
action :create
not_if { ::File.exists?("foo") }
end

On Thu, Aug 15, 2013 at 12:44 PM, Ranjib Dey dey.ranjib@gmail.com wrote:

oops , forgot share the reference
[1]
chef/lib/chef/mixin/checksum.rb at main · chef/chef · GitHub

On Thu, Aug 15, 2013 at 9:43 AM, Ranjib Dey dey.ranjib@gmail.com wrote:

well, you can do that using ruby stdlib, as well as low level chef api
(that chef uses to make file/template/cookbookfile/remote_file [1]
idempotent). But why you want to do it? If you are doing it
inside a recipe, probably its a bad idea, consider pushing them inside
providers or libraries

On Thu, Aug 15, 2013 at 9:24 AM, dcroche@gmail.com wrote:

Is it possible to get a checksum of a cookbook file and store this in
the attribute?

require 'Digest'

cb_filename = ?
if node.attribute['staged_checksum'] == Digest::SHA2.file(File.new(cb_filename))
Chef::Log.info("Already been staged")else
Chef::Log.info("Update staging")

node.attribute['staged_checksum'] = Digest::SHA2.file(File.new(cb_filename))

end

On 15 Aug 2013, at 11:40, Ranjib Dey dey.ranjib@gmail.com wrote:

inside a recipe

if node.attribute?('bar')
Chef::Log.info('already run')
else
ruby_block 'set_attribute' do
block do
nod.set['foo']='bar'
end
end
file '/foo/bar' do
content "foobar"
end
end

On Thu, Aug 15, 2013 at 3:20 AM, dcroche@gmail.com wrote:

Hello,

Would would be the best way to only place a file once per cookbook
version. It's modified and removed outside chef.

Regards
D.

Damien_Roche · August 15, 2013, 5:42pm

The file only needs to be placed once per unique version of the file and is
moved modified outside chef control.
On 15 Aug 2013 18:39, "Mike" miketheman@gmail.com wrote:

Even better:

cookbook_file "foo" do
action :create_if_missing
end

Boom.
-M

On Thu, Aug 15, 2013 at 1:23 PM, Andrew Gross andrew@yipit.com wrote:

Whats wrong with

cookbook_file "foo" do
action :create
not_if { ::File.exists?("foo") }
end

On Thu, Aug 15, 2013 at 12:44 PM, Ranjib Dey dey.ranjib@gmail.comwrote:

oops , forgot share the reference
[1]
chef/lib/chef/mixin/checksum.rb at main · chef/chef · GitHub

On Thu, Aug 15, 2013 at 9:43 AM, Ranjib Dey dey.ranjib@gmail.comwrote:

well, you can do that using ruby stdlib, as well as low level chef api
(that chef uses to make file/template/cookbookfile/remote_file [1]
idempotent). But why you want to do it? If you are doing it
inside a recipe, probably its a bad idea, consider pushing them inside
providers or libraries

On Thu, Aug 15, 2013 at 9:24 AM, dcroche@gmail.com wrote:

Is it possible to get a checksum of a cookbook file and store this in
the attribute?

require 'Digest'

cb_filename = ?
if node.attribute['staged_checksum'] == Digest::SHA2.file(File.new(cb_filename))
Chef::Log.info("Already been staged")else
Chef::Log.info("Update staging")

node.attribute['staged_checksum'] = Digest::SHA2.file(File.new(cb_filename))

end

On 15 Aug 2013, at 11:40, Ranjib Dey dey.ranjib@gmail.com wrote:

inside a recipe

if node.attribute?('bar')
Chef::Log.info('already run')
else
ruby_block 'set_attribute' do
block do
nod.set['foo']='bar'
end
end
file '/foo/bar' do
content "foobar"
end
end

On Thu, Aug 15, 2013 at 3:20 AM, dcroche@gmail.com wrote:

Hello,

Would would be the best way to only place a file once per cookbook
version. It's modified and removed outside chef.

Regards
D.

Mike · August 15, 2013, 5:46pm

I guess I'm not understanding what a "unique version" of a file is.

If I modify file 'foo', and put it in my cookbook, then the checksum will
differ from the in-place file on the target node, and be placed on disk
with action :create.

If I then modify the file on disk, then I've changed the checksum locally,
and Chef will clobber my change.

You may want to adopt a flow where you on-server modifications (which is an
interesting use-case, please elaborate) are done to a copy of your
Chef-goverend file.

-M

On Thu, Aug 15, 2013 at 1:42 PM, Damien Roche dcroche@gmail.com wrote:

The file only needs to be placed once per unique version of the file and
is moved modified outside chef control.
On 15 Aug 2013 18:39, "Mike" miketheman@gmail.com wrote:

Even better:

cookbook_file "foo" do
action :create_if_missing
end

Boom.
-M

On Thu, Aug 15, 2013 at 1:23 PM, Andrew Gross andrew@yipit.com wrote:

Whats wrong with

cookbook_file "foo" do
action :create
not_if { ::File.exists?("foo") }
end

On Thu, Aug 15, 2013 at 12:44 PM, Ranjib Dey dey.ranjib@gmail.comwrote:

oops , forgot share the reference
[1]
chef/lib/chef/mixin/checksum.rb at main · chef/chef · GitHub

On Thu, Aug 15, 2013 at 9:43 AM, Ranjib Dey dey.ranjib@gmail.comwrote:

well, you can do that using ruby stdlib, as well as low level chef api
(that chef uses to make file/template/cookbookfile/remote_file [1]
idempotent). But why you want to do it? If you are doing it
inside a recipe, probably its a bad idea, consider pushing them inside
providers or libraries

On Thu, Aug 15, 2013 at 9:24 AM, dcroche@gmail.com wrote:

Is it possible to get a checksum of a cookbook file and store this in
the attribute?

require 'Digest'

cb_filename = ?
if node.attribute['staged_checksum'] == Digest::SHA2.file(File.new(cb_filename))
Chef::Log.info("Already been staged")else
Chef::Log.info("Update staging")

node.attribute['staged_checksum'] = Digest::SHA2.file(File.new(cb_filename))

end

On 15 Aug 2013, at 11:40, Ranjib Dey dey.ranjib@gmail.com wrote:

inside a recipe

if node.attribute?('bar')
Chef::Log.info('already run')
else
ruby_block 'set_attribute' do
block do
nod.set['foo']='bar'
end
end
file '/foo/bar' do
content "foobar"
end
end

On Thu, Aug 15, 2013 at 3:20 AM, dcroche@gmail.com wrote:

Hello,

Would would be the best way to only place a file once per cookbook
version. It's modified and removed outside chef.

Regards
D.

Davis_Bruce · August 15, 2013, 8:13pm

We resolved a similar problem with our Splunk implementation.

The problem: File churn caused by Chef and Splunk

Chef creates authentication.conf from a template which includes and unencrypted bindDNpassword.
Splunk takes the contents for authentication.conf and encrypts the bindDNpassword.
Chef runs, and creates authentication.conf with an unencrypted bindDNpassword.
Repeat from step 2.

This happened every time chef-client ran.

Our solution:
Create a template (authentication.conf.erb) on the chef server that creates a local template (authentication.conf.erb on the node).
The local template on the node creates authentication.conf.
Authentication.conf only changes if the local template on the node or the bindDNpassword changes.

Create a 0 byte local template file (just a place holder):

file “/etc/system/local/authentication.conf.erb” do
action :create_if_missing
end

Use a template resource to create the authentication.conf from the LOCAL template authentication.conf.erb (created in step 3):

NOTE: action: nothing
NOTE: local true

template “/etc/system/local/authentication.conf” do
source “/etc/system/local/authentication.conf.erb” # On the local node
local true
notifies :restart, resources(:service => “splunk”) # Splunk encrypts the bindDNpassword
action :nothing
end

Create the LOCAL authentication.conf.erb template and use notifies to create authentication.conf defined (in step 2 above) with the unencrypted bindDNpassword:

NOTE: notifies: create, resource(:template ……), :immediately

template “/etc/system/local/authentication.conf.erb” do # On the local node
source “server/authentication.conf.erb” # On the Chef server
notifies :create, resources(:template => “/etc/system/local/authentication.conf”), :immediately
end

Cookbook TEMPLATE config (CRITICAL) on the chef server:

NOTE: the double percent sign: bindDNpassword = <%%= @ldap_bind_dn_passwd %>
This is critical for creating a local template with a variable.

authentication.conf.erb
bindDNpassword = <%%= @ldap_bind_dn_passwd %>

The result:

A local template on the node with the bindDNpassword variable:

/etc/system/local/authentication.conf.erb
bindDNpassword = <%= @ldap_bind_dn_passwd %>

The actual authentication.conf file changes ONLY if the LOCAL template changes authentication.conf.erb with an unencrypted bindDNpassword:

/etc/system/local/authentication.conf
bindDNpassword = UNENCRYPTEDPASSWORD # Put in place by Chef

Splunk reads authentication.conf and encrypts the bindDNpassword:
/etc/system/local/authentication.conf
bindDNpassword = ENCRYPTEDPASSWORD # Encrypted by Splunk

I know this is clear as mud, but I hope it helps.

Bruce Davis
bdavis2-consultant@scholastic.com

From: Mike <miketheman@gmail.com mailto:miketheman@gmail.com>
Reply-To: "chef@lists.opscode.com mailto:chef@lists.opscode.com" <chef@lists.opscode.com mailto:chef@lists.opscode.com>
Date: Thursday, August 15, 2013 1:46 PM
To: "chef@lists.opscode.com mailto:chef@lists.opscode.com" <chef@lists.opscode.com mailto:chef@lists.opscode.com>
Subject: [chef] Re: Re: Re: Re: Re: Re: Place file only once per version

I guess I’m not understanding what a “unique version” of a file is.

If I modify file ‘foo’, and put it in my cookbook, then the checksum will differ from the in-place file on the target node, and be placed on disk with action :create.

If I then modify the file on disk, then I’ve changed the checksum locally, and Chef will clobber my change.

You may want to adopt a flow where you on-server modifications (which is an interesting use-case, please elaborate) are done to a copy of your Chef-goverend file.

-M

On Thu, Aug 15, 2013 at 1:42 PM, Damien Roche <dcroche@gmail.com mailto:dcroche@gmail.com> wrote:

The file only needs to be placed once per unique version of the file and is moved modified outside chef control.

On 15 Aug 2013 18:39, “Mike” <miketheman@gmail.com mailto:miketheman@gmail.com> wrote:
Even better:

cookbook_file “foo” do
action :create_if_missing
end

Boom.
-M

On Thu, Aug 15, 2013 at 1:23 PM, Andrew Gross <andrew@yipit.com mailto:andrew@yipit.com> wrote:
Whats wrong with

cookbook_file “foo” do
action :create
not_if { ::File.exists?(“foo”) }
end

On Thu, Aug 15, 2013 at 12:44 PM, Ranjib Dey <dey.ranjib@gmail.com mailto:dey.ranjib@gmail.com> wrote:
oops , forgot share the reference
[1] https://github.com/opscode/chef/blob/master/lib/chef/mixin/checksum.rb

On Thu, Aug 15, 2013 at 9:43 AM, Ranjib Dey <dey.ranjib@gmail.com mailto:dey.ranjib@gmail.com> wrote:
well, you can do that using ruby stdlib, as well as low level chef api (that chef uses to make file/template/cookbookfile/remote_file [1] idempotent). But why you want to do it? If you are doing it
inside a recipe, probably its a bad idea, consider pushing them inside providers or libraries

On Thu, Aug 15, 2013 at 9:24 AM, <dcroche@gmail.com mailto:dcroche@gmail.com> wrote:
Is it possible to get a checksum of a cookbook file and store this in the attribute?

require ‘Digest’

cb_filename = ?

if node.attribute[‘staged_checksum’] == Digest::SHA2.file(File.new(cb_filename))
Chef::Log.info(“Already been staged”)else
Chef::Log.info(“Update staging”)

    node.attribute['staged_checksum'] = Digest::SHA2.file(File.new(cb_filename))

end

On 15 Aug 2013, at 11:40, Ranjib Dey <dey.ranjib@gmail.com mailto:dey.ranjib@gmail.com> wrote:

inside a recipe

if node.attribute?(‘bar’)
Chef::Log.info(‘already run’)
else
ruby_block ‘set_attribute’ do
block do
nod.set[‘foo’]='bar’
end
end
file ‘/foo/bar’ do
content "foobar"
end
end

On Thu, Aug 15, 2013 at 3:20 AM, <dcroche@gmail.com mailto:dcroche@gmail.com> wrote:
Hello,

Would would be the best way to only place a file once per cookbook version. It’s modified and removed outside chef.

Regards
D.

Damien_Roche · August 15, 2013, 8:19pm

Hi Mike,

I only need the file to be placed once per version a process outside chef moves this file to a working location and removes it when done. But if the file was to be created again it would trigger the same process again.
This is why I need only place it once per unique checksum. It's not a big deal to change the outside process just time consuming. I think its an option to only create the file once per checksum on the chef side too.

Hope that makes sense.

On 15 Aug 2013, at 18:46, Mike miketheman@gmail.com wrote:

I guess I'm not understanding what a "unique version" of a file is.

If I modify file 'foo', and put it in my cookbook, then the checksum will differ from the in-place file on the target node, and be placed on disk with action :create.

If I then modify the file on disk, then I've changed the checksum locally, and Chef will clobber my change.

You may want to adopt a flow where you on-server modifications (which is an interesting use-case, please elaborate) are done to a copy of your Chef-goverend file.

-M

On Thu, Aug 15, 2013 at 1:42 PM, Damien Roche dcroche@gmail.com wrote:
The file only needs to be placed once per unique version of the file and is moved modified outside chef control.

On 15 Aug 2013 18:39, "Mike" miketheman@gmail.com wrote:
Even better:

cookbook_file "foo" do
action :create_if_missing
end

Boom.
-M

On Thu, Aug 15, 2013 at 1:23 PM, Andrew Gross andrew@yipit.com wrote:
Whats wrong with

cookbook_file "foo" do
action :create
not_if { ::File.exists?("foo") }
end

On Thu, Aug 15, 2013 at 12:44 PM, Ranjib Dey dey.ranjib@gmail.com wrote:
oops , forgot share the reference
[1] chef/lib/chef/mixin/checksum.rb at main · chef/chef · GitHub

On Thu, Aug 15, 2013 at 9:43 AM, Ranjib Dey dey.ranjib@gmail.com wrote:
well, you can do that using ruby stdlib, as well as low level chef api (that chef uses to make file/template/cookbookfile/remote_file [1] idempotent). But why you want to do it? If you are doing it
inside a recipe, probably its a bad idea, consider pushing them inside providers or libraries

On Thu, Aug 15, 2013 at 9:24 AM, dcroche@gmail.com wrote:
Is it possible to get a checksum of a cookbook file and store this in the attribute?

require 'Digest'
cb_filename = ?

if node.attribute['staged_checksum'] == Digest::SHA2.file(File.new(cb_filename))
Chef::Log.info("Already been staged")
else
Chef::Log.info("Update staging")
node.attribute['staged_checksum'] = Digest::SHA2.file(File.new(cb_filename))

end

On 15 Aug 2013, at 11:40, Ranjib Dey dey.ranjib@gmail.com wrote:

inside a recipe

if node.attribute?('bar')
Chef::Log.info('already run')
else
ruby_block 'set_attribute' do
block do
nod.set['foo']='bar'
end
end
file '/foo/bar' do
content "foobar"
end
end

On Thu, Aug 15, 2013 at 3:20 AM, dcroche@gmail.com wrote:
Hello,

Would would be the best way to only place a file once per cookbook version. It's modified and removed outside chef.

Regards
D.

Russ_Lavoie · August 16, 2013, 11:50am

An option is to have chef lay down a versioned file and symlink the versioned file to a name the application is expecting.

default['file_version'] = 1.2
template "file-#{node[file_version]}.txt" do
source "file"
...
...
end

link "/path/to/source/file-#{node[file_version]}.txt" do
to "/path/to/expected/by/app/file
end

Then you can unlink the previous version in a similar way.

Hope that helps a bit.

From: "dcroche@gmail.com" dcroche@gmail.com
To: chef@lists.opscode.com
Sent: Thursday, August 15, 2013 3:19 PM
Subject: [chef] Re: Place file only once per version