Best way to install from tarballs?


#1

How is the best way to handle installations from tarballs? Currently,
we have several tarballs that we use to deploy software. These tarballs
are just extracted to the correct directory and the software is
installed.

I want the software installation to be idempotent and not have the
recipe do any more work than necessary. I also want the recipe to
replace a file that may have been overwritten by someone or some other
process. I do not want to have to extract the tarball on every run .
I considered extracting the tarball into the file portion of the
cookbook, but it is in many cases several thousand files that is in
these tarballs.

Anyone have any ideas?

Randy


#2

I created the java_ark lwrp exactly for this use case. While it is in the
java cookbook, it is not actually specific to java

On Fri, Jan 20, 2012 at 3:28 PM, Van Fossan,Randy vanfossr@oclc.org wrote:

How is the best way to handle installations from tarballs? Currently, we
have several tarballs that we use to deploy software. These tarballs are
just extracted to the correct directory and the software is installed.****


I want the software installation to be idempotent and not have the recipe
do any more work than necessary. I also want the recipe to replace a file
that may have been overwritten by someone or some other process. I do
not want to have to extract the tarball on every run . I considered
extracting the tarball into the file portion of the cookbook, but it is in
many cases several thousand files that is in these tarballs.****


Anyone have any ideas?****


Randy****


#3

serve the tar ball via file resource or remote file resource,( specify
checksum to make it idempotent), notify an un-tar execute resource from
the earlier file/remote file resource. you should not check for individual
file content/permission check if the number of files are in thousands. You
might want to use tar against the uncompressed directory and then verify
the checksum .

On Fri, Jan 20, 2012 at 8:01 PM, Bryan Berry bryan.berry@gmail.com wrote:

I created the java_ark lwrp exactly for this use case. While it is in the
java cookbook, it is not actually specific to java

https://github.com/bryanwb/cookbooks/blob/master/java/README.md

On Fri, Jan 20, 2012 at 3:28 PM, Van Fossan,Randy vanfossr@oclc.orgwrote:

How is the best way to handle installations from tarballs? Currently, we
have several tarballs that we use to deploy software. These tarballs are
just extracted to the correct directory and the software is installed.***
*


I want the software installation to be idempotent and not have the recipe
do any more work than necessary. I also want the recipe to replace a file
that may have been overwritten by someone or some other process. I do
not want to have to extract the tarball on every run . I considered
extracting the tarball into the file portion of the cookbook, but it is in
many cases several thousand files that is in these tarballs.****


Anyone have any ideas?****


Randy****


#4

my java_ark is basically a DSL for doing that

here is an example. The one difference is that it currently doesn’t
support sourcing tarballs from cookbook_files but I could add that. Right
now it expects a url pointing to the tarball

here is an example

java_ark “jdk” do
url 'http://download.oracle.com/otn-pub/java/jdk/6u29-b11/jdk-6u29-linux-x64.bin
checksum 'a8603fa62045ce2164b26f7c04859cd548ffe0e33bfc979d9fa73df42e3b3365’
app_home '/usr/local/java/default’
bin_cmds [“java”, “javac”]
action :install
end

On Fri, Jan 20, 2012 at 3:41 PM, Ranjib Dey ranjibd@thoughtworks.comwrote:

serve the tar ball via file resource or remote file resource,( specify
checksum to make it idempotent), notify an un-tar execute resource from
the earlier file/remote file resource. you should not check for individual
file content/permission check if the number of files are in thousands. You
might want to use tar against the uncompressed directory and then verify
the checksum .

On Fri, Jan 20, 2012 at 8:01 PM, Bryan Berry bryan.berry@gmail.comwrote:

I created the java_ark lwrp exactly for this use case. While it is in the
java cookbook, it is not actually specific to java

https://github.com/bryanwb/cookbooks/blob/master/java/README.md

On Fri, Jan 20, 2012 at 3:28 PM, Van Fossan,Randy vanfossr@oclc.orgwrote:

How is the best way to handle installations from tarballs? Currently,
we have several tarballs that we use to deploy software. These tarballs
are just extracted to the correct directory and the software is installed.



I want the software installation to be idempotent and not have the
recipe do any more work than necessary. I also want the recipe to replace
a file that may have been overwritten by someone or some other process.
I do not want to have to extract the tarball on every run . I considered
extracting the tarball into the file portion of the cookbook, but it is in
many cases several thousand files that is in these tarballs.****


Anyone have any ideas?****


Randy****


#5

oh… sorry :-). Didn’t went through your providers… will check them now.

On Fri, Jan 20, 2012 at 8:15 PM, Bryan Berry bryan.berry@gmail.com wrote:

my java_ark is basically a DSL for doing that

here is an example. The one difference is that it currently doesn’t
support sourcing tarballs from cookbook_files but I could add that. Right
now it expects a url pointing to the tarball

here is an example

java_ark “jdk” do
url 'http://download.oracle.com/otn-pub/java/jdk/6u29-b11/jdk-6u29-linux-x64.bin
checksum 'a8603fa62045ce2164b26f7c04859cd548ffe0e33bfc979d9fa73df42e3b3365’
app_home '/usr/local/java/default’
bin_cmds [“java”, “javac”]
action :install
end

On Fri, Jan 20, 2012 at 3:41 PM, Ranjib Dey ranjibd@thoughtworks.comwrote:

serve the tar ball via file resource or remote file resource,( specify
checksum to make it idempotent), notify an un-tar execute resource from
the earlier file/remote file resource. you should not check for individual
file content/permission check if the number of files are in thousands. You
might want to use tar against the uncompressed directory and then verify
the checksum .

On Fri, Jan 20, 2012 at 8:01 PM, Bryan Berry bryan.berry@gmail.comwrote:

I created the java_ark lwrp exactly for this use case. While it is in
the java cookbook, it is not actually specific to java

https://github.com/bryanwb/cookbooks/blob/master/java/README.md

On Fri, Jan 20, 2012 at 3:28 PM, Van Fossan,Randy vanfossr@oclc.orgwrote:

How is the best way to handle installations from tarballs? Currently,
we have several tarballs that we use to deploy software. These tarballs
are just extracted to the correct directory and the software is installed.



I want the software installation to be idempotent and not have the
recipe do any more work than necessary. I also want the recipe to replace
a file that may have been overwritten by someone or some other process.
I do not want to have to extract the tarball on every run . I considered
extracting the tarball into the file portion of the cookbook, but it is in
many cases several thousand files that is in these tarballs.****


Anyone have any ideas?****


Randy****


#6

On Fri, Jan 20, 2012 at 2:28 PM, Van Fossan,Randy vanfossr@oclc.org wrote:

How is the best way to handle installations from tarballs? Currently, we
have several tarballs that we use to deploy software. These tarballs are
just extracted to the correct directory and the software is installed.

We started out with a “tar_package” LWRP, similar to the approach
mentioned by the other replies on this thread, but now use FPM1 to
create RPMs of the file we previously shipped in a tarball. The
tarballs we’re concerned with are updated very infrequently, so the
extra step of building the RPM isn’t a problem for us.

Zac


#7

Indeed , even we do use the same strategy to deploy some of our apps. We
have the fpm mediated rpm creation step as part of our CI. Also, there are
mcollective based triggers that executed “createrepo” on the rpm repo
server and "chef-client --once " to the app servers. But as things evolved,
in some cases we are overburdened with git-fpm–publish-createrepo-chef
route and in some other cases we have many sources of truth (rpm version/
git commit hash). As of now , i dont see a ‘one solution fits all
situation’ approach for distributing build artifacts. But never the less we
should work/innovate towards that

On Fri, Jan 20, 2012 at 9:24 PM, Zac Stevens zts@cryptocracy.com wrote:

On Fri, Jan 20, 2012 at 2:28 PM, Van Fossan,Randy vanfossr@oclc.org
wrote:

How is the best way to handle installations from tarballs? Currently, we
have several tarballs that we use to deploy software. These tarballs are
just extracted to the correct directory and the software is installed.

We started out with a “tar_package” LWRP, similar to the approach
mentioned by the other replies on this thread, but now use FPM1 to
create RPMs of the file we previously shipped in a tarball. The
tarballs we’re concerned with are updated very infrequently, so the
extra step of building the RPM isn’t a problem for us.

Zac