Cookbok file-- copy only if new

Chef Infra (archive)
1

Hi,

This is what i am trying to do, i build my war file and copy it in the chef
cookbook files/default folder and then run chef-client on my deployment
node assuming it will copy the new file.

My recipe says cookbook file “file_name” … , I ma not sure what should
i write it in the action statement
As the *war file name will always be the same *so create or create if
missing does not make sense,

Please help.

2

The :create_if_missing action will only put the file in place if it doesn't
already exist.

The .war file can be in the cookbook/files/default/file_name.war. Then, use
the action:

cookbook_file "/destination/path/to/file_name.war" do
action :create_if_missing"
end

See cookbook_file Resource

--
Alan Ivey

**

Senior Systems Administrator

EchoDitto****

email: alan@echoditto.com****

phone: 202-285-9455 ext 115

**

aim: echodittoalan

On Thu, Sep 12, 2013 at 2:29 PM, Manoj Thakkar manoj.thakkar@gmail.comwrote:

Hi,

This is what i am trying to do, i build my war file and copy it in the
chef cookbook files/default folder and then run chef-client on my
deployment node assuming it will copy the new file.

My recipe says cookbook file "file_name" ...... , I ma not sure what
should i write it in the action statement
As the *war file name will always be the same *so create or create if
missing does not make sense,

Please help.

3

On Sep 12, 2013, at 3:00 PM, Alan Ivey alan@echoditto.com wrote:

The :create_if_missing action will only put the file in place if it doesn't already exist.

The .war file can be in the cookbook/files/default/file_name.war. Then, use the action:

cookbook_file "/destination/path/to/file_name.war" do
action :create_if_missing"
end

Yeah, but if there's an old war file that's already there, will it replace that with the new one?

I think the OP wants an "action :create" here, and leave it up to Chef to check the hash signature of the war file to see if it actually needs to be updated or not.

--
Brad Knowles brad@shub-internet.org
LinkedIn Profile: http://tinyurl.com/y8kpxu

4

Hi Manoj -

If I am understanding what you are trying to do, I don't believe that using
:create_if_missing will work for you (which is consistent with what you
suspect).

My understanding is that you want to do the following:

  • Create a .war file that has the same name for every version of the file.
  • Only copy the file when there is a new version
    ++ But that new version will have the same name

Is that correct?

If so, I believe that using remote_file is your best option. It will
allow you to specify a checksum and will only copy the file if the checksum
matches what you specify.

Alternatively, you could use cookbook_file, but use a conditional that
requires they checksum you calculate is different from what is deployed.

On Thu, Sep 12, 2013 at 1:00 PM, Alan Ivey alan@echoditto.com wrote:

The :create_if_missing action will only put the file in place if it
doesn't already exist.

The .war file can be in the cookbook/files/default/file_name.war. Then,
use the action:

cookbook_file "/destination/path/to/file_name.war" do
action :create_if_missing"
end

See http://docs.opscode.com/resource_cookbook_file.html#actions

--
Alan Ivey

**

Senior Systems Administrator

EchoDitto****

email: alan@echoditto.com****

phone: 202-285-9455 ext 115

**

aim: echodittoalan

On Thu, Sep 12, 2013 at 2:29 PM, Manoj Thakkar manoj.thakkar@gmail.comwrote:

Hi,

This is what i am trying to do, i build my war file and copy it in the
chef cookbook files/default folder and then run chef-client on my
deployment node assuming it will copy the new file.

My recipe says cookbook file "file_name" ...... , I ma not sure what
should i write it in the action statement
As the *war file name will always be the same *so create or create if
missing does not make sense,

Please help.

5

Also, remote_file has been upgraded to allow conditional GETs. So if the
files are being hosted on something that supports if-modified-since, then
you can just use a standard action :create and it will only update the file
if the remote file has been updated.

They you wouldn't necessarily have to modify the checksum in the cookbook
each time you update the file.

use_last_modified Indicates that If-Modified-Since headers are enabled. Set
to false to disable If-Modified-Since headers. To use this setting,
use_conditional_get must also be set to true. Default value: true.

On Thu, Sep 12, 2013 at 2:23 PM, Jordan Dea-Mattson <
jdm-lists@dea-mattson.com> wrote:

Hi Manoj -

If I am understanding what you are trying to do, I don't believe that
using :create_if_missing will work for you (which is consistent with what
you suspect).

My understanding is that you want to do the following:

  • Create a .war file that has the same name for every version of the file.
  • Only copy the file when there is a new version
    ++ But that new version will have the same name

Is that correct?

If so, I believe that using remote_file is your best option. It will
allow you to specify a checksum and will only copy the file if the checksum
matches what you specify.

Alternatively, you could use cookbook_file, but use a conditional that
requires they checksum you calculate is different from what is deployed.

On Thu, Sep 12, 2013 at 1:00 PM, Alan Ivey alan@echoditto.com wrote:

The :create_if_missing action will only put the file in place if it
doesn't already exist.

The .war file can be in the cookbook/files/default/file_name.war. Then,
use the action:

cookbook_file "/destination/path/to/file_name.war" do
action :create_if_missing"
end

See http://docs.opscode.com/resource_cookbook_file.html#actions

--
Alan Ivey

**

Senior Systems Administrator

EchoDitto****

email: alan@echoditto.com****

phone: 202-285-9455 ext 115

**

aim: echodittoalan

On Thu, Sep 12, 2013 at 2:29 PM, Manoj Thakkar manoj.thakkar@gmail.comwrote:

Hi,

This is what i am trying to do, i build my war file and copy it in the
chef cookbook files/default folder and then run chef-client on my
deployment node assuming it will copy the new file.

My recipe says cookbook file "file_name" ...... , I ma not sure what
should i write it in the action statement
As the *war file name will always be the same *so create or create if
missing does not make sense,

Please help.

6

Excellent point Benjamin! And one that I missed making and only thought
about after my initial post (and hadn't gotten around to posting a
follow-up).

I think we need to consider a refactor that combines cookbook_file,
'file' and remote_file into a single resource. We can maintain the three
aliases for background compatibility.

Jordan

On Thu, Sep 12, 2013 at 1:32 PM, Benjamin Bytheway bbytheway@gmail.comwrote:

Also, remote_file has been upgraded to allow conditional GETs. So if the
files are being hosted on something that supports if-modified-since, then
you can just use a standard action :create and it will only update the file
if the remote file has been updated.

They you wouldn't necessarily have to modify the checksum in the cookbook
each time you update the file.

use_last_modified Indicates that If-Modified-Since headers are enabled.
Set to false to disable If-Modified-Since headers. To use this setting,
use_conditional_get must also be set to true. Default value: true.

remote_file Resource

On Thu, Sep 12, 2013 at 2:23 PM, Jordan Dea-Mattson <
jdm-lists@dea-mattson.com> wrote:

Hi Manoj -

If I am understanding what you are trying to do, I don't believe that
using :create_if_missing will work for you (which is consistent with what
you suspect).

My understanding is that you want to do the following:

  • Create a .war file that has the same name for every version of the file.
  • Only copy the file when there is a new version
    ++ But that new version will have the same name

Is that correct?

If so, I believe that using remote_file is your best option. It will
allow you to specify a checksum and will only copy the file if the checksum
matches what you specify.

Alternatively, you could use cookbook_file, but use a conditional that
requires they checksum you calculate is different from what is deployed.

On Thu, Sep 12, 2013 at 1:00 PM, Alan Ivey alan@echoditto.com wrote:

The :create_if_missing action will only put the file in place if it
doesn't already exist.

The .war file can be in the cookbook/files/default/file_name.war. Then,
use the action:

cookbook_file "/destination/path/to/file_name.war" do
action :create_if_missing"
end

See cookbook_file Resource

--
Alan Ivey

**

Senior Systems Administrator

EchoDitto****

email: alan@echoditto.com****

phone: 202-285-9455 ext 115

**

aim: echodittoalan

On Thu, Sep 12, 2013 at 2:29 PM, Manoj Thakkar manoj.thakkar@gmail.comwrote:

Hi,

This is what i am trying to do, i build my war file and copy it in the
chef cookbook files/default folder and then run chef-client on my
deployment node assuming it will copy the new file.

My recipe says cookbook file "file_name" ...... , I ma not sure what
should i write it in the action statement
As the *war file name will always be the same *so create or create if
missing does not make sense,

Please help.

7

Thanks everyone ,

I am still confused on how to achieve it what I have us a very normal everyday scenario in almost every company I think

  1. Do a build using Jenkins and copy it to chef server
  2. Run chef client on a set of nodes , all these nodes have multiple recipes in their runlist
    3.only run list that should be run is whose file is changed and the new file should be copied on that agent

Please help me do a write up for this

What i understand from remote_file is that it will not be able to automatically copy file from a location like chef server . I would have to copy the file on some NSF mount and make it available on all nodes which is too tedious and unnecessary

I have used puppet in past and puppet supports it as a secular functionality , I am sure I am missing some piece here

Please help

Sent from my iPhone

On Sep 12, 2013, at 1:47 PM, Jordan Dea-Mattson jdm-lists@dea-mattson.com wrote:

Excellent point Benjamin! And one that I missed making and only thought about after my initial post (and hadn't gotten around to posting a follow-up).

I think we need to consider a refactor that combines cookbook_file, 'file' and remote_file into a single resource. We can maintain the three aliases for background compatibility.

Jordan

On Thu, Sep 12, 2013 at 1:32 PM, Benjamin Bytheway bbytheway@gmail.com wrote:

Also, remote_file has been upgraded to allow conditional GETs. So if the files are being hosted on something that supports if-modified-since, then you can just use a standard action :create and it will only update the file if the remote file has been updated.

They you wouldn't necessarily have to modify the checksum in the cookbook each time you update the file.

use_last_modified Indicates that If-Modified-Since headers are enabled. Set to false to disable If-Modified-Since headers. To use this setting, use_conditional_get must also be set to true. Default value: true.

remote_file Resource

On Thu, Sep 12, 2013 at 2:23 PM, Jordan Dea-Mattson jdm-lists@dea-mattson.com wrote:

Hi Manoj -

If I am understanding what you are trying to do, I don't believe that using :create_if_missing will work for you (which is consistent with what you suspect).

My understanding is that you want to do the following:

  • Create a .war file that has the same name for every version of the file.
  • Only copy the file when there is a new version
    ++ But that new version will have the same name

Is that correct?

If so, I believe that using remote_file is your best option. It will allow you to specify a checksum and will only copy the file if the checksum matches what you specify.

Alternatively, you could use cookbook_file, but use a conditional that requires they checksum you calculate is different from what is deployed.

On Thu, Sep 12, 2013 at 1:00 PM, Alan Ivey alan@echoditto.com wrote:

The :create_if_missing action will only put the file in place if it doesn't already exist.

The .war file can be in the cookbook/files/default/file_name.war. Then, use the action:

cookbook_file "/destination/path/to/file_name.war" do
action :create_if_missing"
end

See cookbook_file Resource

--
Alan Ivey
Senior Systems Administrator
EchoDitto
email: alan@echoditto.com
phone: 202-285-9455 ext 115
aim: echodittoalan

On Thu, Sep 12, 2013 at 2:29 PM, Manoj Thakkar manoj.thakkar@gmail.com wrote:

Hi,

This is what i am trying to do, i build my war file and copy it in the chef cookbook files/default folder and then run chef-client on my deployment node assuming it will copy the new file.

My recipe says cookbook file "file_name" ...... , I ma not sure what should i write it in the action statement
As the war file name will always be the same so create or create if missing does not make sense,

Please help.

8

Ah, apologies for misunderstanding that part of the question.

On Thu, Sep 12, 2013 at 4:14 PM, Brad Knowles brad@shub-internet.orgwrote:

On Sep 12, 2013, at 3:00 PM, Alan Ivey alan@echoditto.com wrote:

The :create_if_missing action will only put the file in place if it
doesn't already exist.

The .war file can be in the cookbook/files/default/file_name.war. Then,
use the action:

cookbook_file "/destination/path/to/file_name.war" do
action :create_if_missing"
end

Yeah, but if there's an old war file that's already there, will it replace
that with the new one?

I think the OP wants an "action :create" here, and leave it up to Chef to
check the hash signature of the war file to see if it actually needs to be
updated or not.

--
Brad Knowles brad@shub-internet.org
LinkedIn Profile: http://tinyurl.com/y8kpxu

9

On Thursday, September 12, 2013 at 9:41 PM, Manoj wrote:

Thanks everyone ,

I am still confused on how to achieve it what I have us a very normal everyday scenario in almost every company I think

  1. Do a build using Jenkins and copy it to chef server
  2. Run chef client on a set of nodes , all these nodes have multiple recipes in their runlist
    3.only run list that should be run is whose file is changed and the new file should be copied on that agent

Please help me do a write up for this

What i understand from remote_file is that it will not be able to automatically copy file from a location like chef server . I would have to copy the file on some NSF mount and make it available on all nodes which is too tedious and unnecessary
Where do you put your artifacts after jenkins builds them? Use jenkins built-in artifact storage?

If that's the case, chef-client can fetch the file directly from jenkins as long as you have the URL.

If it's a requirement that chef-server store the artifact, you're gonna have to upload the artifact as part of a cookbook and use cookbook_file. This can be a real pain if you manage the cookbook with version control (which you should do).

I have used puppet in past and puppet supports it as a secular functionality , I am sure I am missing some piece here

Please help
Chef has an override run list feature where you can run just a specific recipe (or several).

--
Daniel DeLeo

10

I can have the chef client download the artifacts sure but as i said the artifacts name are the same I would not know what's downloaded from repository
Putting it in the chef server from Jenkins is my way of tracking what I am deploying to chef client , I think it's pretty standard from traceability point of virw to put data in chef server for jenkins

Are you saying there is no way a cookbook file can do a checksum or see if Ike is modified . I was reading cookbook_fie documentation and it says it does copy file only Fi modified in theory

Sent from my iPhone

On Sep 13, 2013, at 9:09 AM, Daniel DeLeo dan@kallistec.com wrote:

On Thursday, September 12, 2013 at 9:41 PM, Manoj wrote:

Thanks everyone ,

I am still confused on how to achieve it what I have us a very normal everyday scenario in almost every company I think

  1. Do a build using Jenkins and copy it to chef server
  2. Run chef client on a set of nodes , all these nodes have multiple recipes in their runlist
    3.only run list that should be run is whose file is changed and the new file should be copied on that agent

Please help me do a write up for this

What i understand from remote_file is that it will not be able to automatically copy file from a location like chef server . I would have to copy the file on some NSF mount and make it available on all nodes which is too tedious and unnecessary
Where do you put your artifacts after jenkins builds them? Use jenkins built-in artifact storage?

If that's the case, chef-client can fetch the file directly from jenkins as long as you have the URL.

If it's a requirement that chef-server store the artifact, you're gonna have to upload the artifact as part of a cookbook and use cookbook_file. This can be a real pain if you manage the cookbook with version control (which you should do).

I have used puppet in past and puppet supports it as a secular functionality , I am sure I am missing some piece here

Please help
Chef has an override run list feature where you can run just a specific recipe (or several).

--
Daniel DeLeo

11

On Friday, September 13, 2013 at 9:24 AM, Manoj wrote:

I can have the chef client download the artifacts sure but as i said the artifacts name are the same I would not know what's downloaded from repository
Putting it in the chef server from Jenkins is my way of tracking what I am deploying to chef client , I think it's pretty standard from traceability point of virw to put data in chef server for jenkins

Are you saying there is no way a cookbook file can do a checksum or see if Ike is modified . I was reading cookbook_fie documentation and it says it does copy file only Fi modified in theory
All of the file resources are idempotent and will only update a file if it's modified. In the remote_file case, if you are on 11.6+ and your server supports conditional GET, then the file will not be re-downloaded; if you're on an earlier version of chef, the file has to be redownloaded for chef to check if it's changed. You can use the checksum attribute on the resource to avoid this.

--
Daniel DeLeo

12

Thanks for the input

This is what I am seeing on my server 11.4 I have a cookbook_file resource where I am copying a file a-1.1.0-snapshot.war , when I copy a new war file to the cookbook files/default location and run chef client the chef client says uptodate and don't download the file

Please help me with the proper fix for this

Sent from my iPhone

On Sep 13, 2013, at 9:31 AM, Daniel DeLeo dan@kallistec.com wrote:

On Friday, September 13, 2013 at 9:24 AM, Manoj wrote:

I can have the chef client download the artifacts sure but as i said the artifacts name are the same I would not know what's downloaded from repository
Putting it in the chef server from Jenkins is my way of tracking what I am deploying to chef client , I think it's pretty standard from traceability point of virw to put data in chef server for jenkins

Are you saying there is no way a cookbook file can do a checksum or see if Ike is modified . I was reading cookbook_fie documentation and it says it does copy file only Fi modified in theory
All of the file resources are idempotent and will only update a file if it's modified. In the remote_file case, if you are on 11.6+ and your server supports conditional GET, then the file will not be re-downloaded; if you're on an earlier version of chef, the file has to be redownloaded for chef to check if it's changed. You can use the checksum attribute on the resource to avoid this.

--
Daniel DeLeo

13

Daniel,Alan,

Help me understand this if i copy a new file to the cookbook , do i need to
a cookbook upload also ? or the chef-client will see the new file without
the cookbook upload ?

sorry for too many questions and discussions i am new to all this.

Thanks
Manoj

On Fri, Sep 13, 2013 at 9:50 AM, Manoj manoj.thakkar@gmail.com wrote:

Thanks for the input

This is what I am seeing on my server 11.4 I have a cookbook_file resource
where I am copying a file a-1.1.0-snapshot.war , when I copy a new war file
to the cookbook files/default location and run chef client the chef client
says uptodate and don't download the file

Please help me with the proper fix for this

Sent from my iPhone

On Sep 13, 2013, at 9:31 AM, Daniel DeLeo dan@kallistec.com wrote:

On Friday, September 13, 2013 at 9:24 AM, Manoj wrote:

I can have the chef client download the artifacts sure but as i said the
artifacts name are the same I would not know what's downloaded from
repository
Putting it in the chef server from Jenkins is my way of tracking what I am
deploying to chef client , I think it's pretty standard from traceability
point of virw to put data in chef server for jenkins

Are you saying there is no way a cookbook file can do a checksum or see if
Ike is modified . I was reading cookbook_fie documentation and it says it
does copy file only Fi modified in theory

All of the file resources are idempotent and will only update a file if
it's modified. In the remote_file case, if you are on 11.6+ and your server
supports conditional GET, then the file will not be re-downloaded; if
you're on an earlier version of chef, the file has to be redownloaded for
chef to check if it's changed. You can use the checksum attribute on the
resource to avoid this.

--
Daniel DeLeo

14

On Sep 13, 2013, at 2:50 PM, Manoj Thakkar manoj.thakkar@gmail.com wrote:

Help me understand this if i copy a new file to the cookbook , do i need to a cookbook upload also ? or the chef-client will see the new file without the cookbook upload ?

Any changes made to the cookbook will not be seen anywhere until a new copy is uploaded to the Chef server.

--
Brad Knowles brad@shub-internet.org
LinkedIn Profile: http://tinyurl.com/y8kpxu

15

(It's okay, Manoi, there are other, equally new kitchen boys watching
these posts attentively.)

On 9/13/2013 1:50 PM, Manoj Thakkar wrote:

Daniel,Alan,

Help me understand this if i copy a new file to the cookbook , do i
need to a cookbook upload also ? or the chef-client will see the new
file without the cookbook upload ?

sorry for too many questions and discussions i am new to all this.

Thanks
Manoj

On Fri, Sep 13, 2013 at 9:50 AM, Manoj <manoj.thakkar@gmail.com
mailto:manoj.thakkar@gmail.com> wrote:

Thanks for the input

This is what I am seeing on my server 11.4 I have a cookbook_file
resource where I am copying a file a-1.1.0-snapshot.war , when I
copy a new war file to the cookbook files/default location and run
chef client the chef client says uptodate and don't download the file

Please help me with the proper fix for this

Sent from my iPhone

On Sep 13, 2013, at 9:31 AM, Daniel DeLeo <dan@kallistec.com
<mailto:dan@kallistec.com>> wrote:

On Friday, September 13, 2013 at 9:24 AM, Manoj wrote:

I can have the chef client download the artifacts sure but as i
said the artifacts name are the same I would not know what's
downloaded from repository
Putting it in the chef server from Jenkins is my way of tracking
what I am deploying to chef client , I think it's pretty
standard from traceability point of virw to put data in chef
server for jenkins

Are you saying there is no way a cookbook file can do a checksum
or see if Ike is modified . I was reading cookbook_fie
documentation and it says it does copy file only Fi modified in
theory

All of the file resources are idempotent and will only update a
file if it's modified. In the remote_file case, if you are on
11.6+ and your server supports conditional GET, then the file
will not be re-downloaded; if you're on an earlier version of
chef, the file has to be redownloaded for chef to check if it's
changed. You can use the checksum attribute on the resource to
avoid this.


-- 
Daniel DeLeo