Bulk expansion of templates

Chef Infra (archive)
1

Hi all -

Before I go off the rails and start trying to iterate over “n” number of
directories and their contents, is there a great way to say, “hey, in this
templates directory, move everything one directory higher (or FQP)”?

I don’t see any sort of wild carding kinda stuff around the templates or
being able to specify a templates directory to be processed.

Thanks for any tips.

2

I'm actually confused by the question. Can you explain it again, perhaps a
bit differently, perhaps with a more explicit example of your intent?

Adam

On 11/29/12 7:15 AM, "Maven User" maven.2.user@gmail.com wrote:

Hi all -

Before I go off the rails and start trying to iterate over "n" number of
directories and their contents, is there a great way to say, "hey, in
this templates directory, move everything one directory higher (or FQP)"?

I don't see any sort of wild carding kinda stuff around the templates or
being able to specify a templates directory to be processed.

Thanks for any tips.

3

Sure -

So because of the way we structure our deployable units (zip files) we wind
up supplying templates inside there (maven injects the erb syntax into
these files).

The structure looks like this:

foo
|_bin
|_conf
|_templates
|_bin
|_conf

Essentially, I want to take any of the directories inside templates, and
process the files inside to identical directories one higher.

I don't want to have a hard coded list of templates anywhere.

If this were ant or maven or something, I'd just use something like
templates/**.

Does this help explain a bit more? I'm not sure how awesome my ASCII art
will look like once posted....

On Thu, Nov 29, 2012 at 10:37 AM, Adam Jacob adam@opscode.com wrote:

I'm actually confused by the question. Can you explain it again, perhaps a
bit differently, perhaps with a more explicit example of your intent?

Adam

On 11/29/12 7:15 AM, "Maven User" maven.2.user@gmail.com wrote:

Hi all -

Before I go off the rails and start trying to iterate over "n" number of
directories and their contents, is there a great way to say, "hey, in
this templates directory, move everything one directory higher (or FQP)"?

I don't see any sort of wild carding kinda stuff around the templates or
being able to specify a templates directory to be processed.

Thanks for any tips.

4

Something like this?

remote_file "/srv/app/deploy.zip" do
end

On 11/29/12 7:45 AM, "Maven User" maven.2.user@gmail.com wrote:

Sure -

So because of the way we structure our deployable units (zip files) we
wind up supplying templates inside there (maven injects the erb syntax
into these files).

The structure looks like this:

foo
|_bin
|_conf
|_templates
|_bin
|_conf

Essentially, I want to take any of the directories inside templates, and
process the files inside to identical directories one higher.

I don't want to have a hard coded list of templates anywhere.

If this were ant or maven or something, I'd just use something like
templates/**.

Does this help explain a bit more? I'm not sure how awesome my ASCII art
will look like once posted....

On Thu, Nov 29, 2012 at 10:37 AM, Adam Jacob
adam@opscode.com wrote:

I'm actually confused by the question. Can you explain it again, perhaps a
bit differently, perhaps with a more explicit example of your intent?

Adam

On 11/29/12 7:15 AM, "Maven User" maven.2.user@gmail.com wrote:

Hi all -

Before I go off the rails and start trying to iterate over "n" number of
directories and their contents, is there a great way to say, "hey, in
this templates directory, move everything one directory higher (or FQP)"?

I don't see any sort of wild carding kinda stuff around the templates or
being able to specify a templates directory to be processed.

Thanks for any tips.

5

Ack!

Something like this:

remote_file "/srv/app/deploy.zip" do
source "http://example.com/deploy.zip"
end

execute "unzip /srv/app/deploy.zip"

ruby_block "process templates" do
source <<-EOH
Directory["/srv/app/templates/**"].each do |file|
...
end
EOH
End

Should do the trick (where Š is fill in the blanks).

Adam

On 11/29/12 7:45 AM, "Maven User" maven.2.user@gmail.com wrote:

Sure -

So because of the way we structure our deployable units (zip files) we
wind up supplying templates inside there (maven injects the erb syntax
into these files).

The structure looks like this:

foo
|_bin
|_conf
|_templates
|_bin
|_conf

Essentially, I want to take any of the directories inside templates, and
process the files inside to identical directories one higher.

I don't want to have a hard coded list of templates anywhere.

If this were ant or maven or something, I'd just use something like
templates/**.

Does this help explain a bit more? I'm not sure how awesome my ASCII art
will look like once posted....

On Thu, Nov 29, 2012 at 10:37 AM, Adam Jacob
adam@opscode.com wrote:

I'm actually confused by the question. Can you explain it again, perhaps a
bit differently, perhaps with a more explicit example of your intent?

Adam

On 11/29/12 7:15 AM, "Maven User" maven.2.user@gmail.com wrote:

Hi all -

Before I go off the rails and start trying to iterate over "n" number of
directories and their contents, is there a great way to say, "hey, in
this templates directory, move everything one directory higher (or FQP)"?

I don't see any sort of wild carding kinda stuff around the templates or
being able to specify a templates directory to be processed.

Thanks for any tips.

6

I'm not sure what that does exactly - but I thought remote_file just copied
some remote file down.

I don't understand how that would expand the tokens inside a template.

The structure looks like this:

foo
|_bin
|_conf
|_templates
|_bin
|_conf

Essentially, I want to take any of the directories inside templates, and
process the files inside to identical directories one higher.

In "foo/templates/conf" we have property files for example. Inside the
property files, we have erb type syntax.

In maven and ant, you can say "copy the contents of templates one directory
higher and expand all tokens you find inside these files".

There are two things I'm trying to solve for - I want to configure my
product as late as possible AND I don't want to maintain a list of property
files and their locations.

The ULTIMATE goal of mine is to create a lwrp so I don't have all these
very similar cookbooks with only slightly different configuration files
listed.

7

Hmmm - ok, it's a bit clearer now, but this is an area I'm not familiar
with.

Can't I do:

Directory["/path/to/templates/**"].each do |file|
template /tmp/foo/file do
local true
source file
end
end

??

I'm not sure how to reference the file variable in the actual template
resource now tho...

On Thu, Nov 29, 2012 at 11:38 AM, Adam Jacob adam@opscode.com wrote:

Ack!

Something like this:

remote_file "/srv/app/deploy.zip" do
source "http://example.com/deploy.zip"
end

execute "unzip /srv/app/deploy.zip"

ruby_block "process templates" do
source <<-EOH
Directory["/srv/app/templates/**"].each do |file|
...
end
EOH
End

Should do the trick (where Š is fill in the blanks).

Adam

On 11/29/12 7:45 AM, "Maven User" maven.2.user@gmail.com wrote:

Sure -

So because of the way we structure our deployable units (zip files) we
wind up supplying templates inside there (maven injects the erb syntax
into these files).

The structure looks like this:

foo
|_bin
|_conf
|_templates
|_bin
|_conf

Essentially, I want to take any of the directories inside templates, and
process the files inside to identical directories one higher.

I don't want to have a hard coded list of templates anywhere.

If this were ant or maven or something, I'd just use something like
templates/**.

Does this help explain a bit more? I'm not sure how awesome my ASCII art
will look like once posted....

On Thu, Nov 29, 2012 at 10:37 AM, Adam Jacob
adam@opscode.com wrote:

I'm actually confused by the question. Can you explain it again, perhaps a
bit differently, perhaps with a more explicit example of your intent?

Adam

On 11/29/12 7:15 AM, "Maven User" maven.2.user@gmail.com wrote:

Hi all -

Before I go off the rails and start trying to iterate over "n" number of
directories and their contents, is there a great way to say, "hey, in
this templates directory, move everything one directory higher (or FQP)"?

I don't see any sort of wild carding kinda stuff around the templates or
being able to specify a templates directory to be processed.

Thanks for any tips.

8

The only issue is that the "each" loop as you have it will be run at
compile-time. So if you don't create the directory full of templates until
convergence (eg: with Chef), that will be an empty array. This is why Adam
was using the "ruby_block" resource.

Of couse the ruby_block resource might have its own problems. I don't know
whether something like this will actually work (I think it won't)

ruby_block "process templates" do
Directory["/srv/app/templates/**"].each do |filename|
template filename do
#stuff
end
end
end

-Matt Moretti

On Thu, Nov 29, 2012 at 2:35 PM, Maven User maven.2.user@gmail.com wrote:

Hmmm - ok, it's a bit clearer now, but this is an area I'm not familiar
with.

Can't I do:

Directory["/path/to/templates/**"].each do |file|
template /tmp/foo/file do
local true
source file
end
end

??

I'm not sure how to reference the file variable in the actual template
resource now tho...

On Thu, Nov 29, 2012 at 11:38 AM, Adam Jacob adam@opscode.com wrote:

Ack!

Something like this:

remote_file "/srv/app/deploy.zip" do
source "http://example.com/deploy.zip"
end

execute "unzip /srv/app/deploy.zip"

ruby_block "process templates" do
source <<-EOH
Directory["/srv/app/templates/**"].each do |file|
...
end
EOH
End

Should do the trick (where Š is fill in the blanks).

Adam

On 11/29/12 7:45 AM, "Maven User" maven.2.user@gmail.com wrote:

Sure -

So because of the way we structure our deployable units (zip files) we
wind up supplying templates inside there (maven injects the erb syntax
into these files).

The structure looks like this:

foo
|_bin
|_conf
|_templates
|_bin
|_conf

Essentially, I want to take any of the directories inside templates, and
process the files inside to identical directories one higher.

I don't want to have a hard coded list of templates anywhere.

If this were ant or maven or something, I'd just use something like
templates/**.

Does this help explain a bit more? I'm not sure how awesome my ASCII art
will look like once posted....

On Thu, Nov 29, 2012 at 10:37 AM, Adam Jacob
adam@opscode.com wrote:

I'm actually confused by the question. Can you explain it again, perhaps
a
bit differently, perhaps with a more explicit example of your intent?

Adam

On 11/29/12 7:15 AM, "Maven User" maven.2.user@gmail.com wrote:

Hi all -

Before I go off the rails and start trying to iterate over "n" number of
directories and their contents, is there a great way to say, "hey, in
this templates directory, move everything one directory higher (or
FQP)"?

I don't see any sort of wild carding kinda stuff around the templates or
being able to specify a templates directory to be processed.

Thanks for any tips.

9

You could use the ruby_block to generate (and store) an array of template
paths to interact with, during convergence phase after the extraction.

Cheers,

--AJ

On 30 November 2012 09:38, Matthew Moretti werebus@gmail.com wrote:

The only issue is that the "each" loop as you have it will be run at
compile-time. So if you don't create the directory full of templates until
convergence (eg: with Chef), that will be an empty array. This is why Adam
was using the "ruby_block" resource.

Of couse the ruby_block resource might have its own problems. I don't
know whether something like this will actually work (I think it won't)

ruby_block "process templates" do
Directory["/srv/app/templates/**"].each do |filename|
template filename do
#stuff
end
end
end

-Matt Moretti

On Thu, Nov 29, 2012 at 2:35 PM, Maven User maven.2.user@gmail.comwrote:

Hmmm - ok, it's a bit clearer now, but this is an area I'm not familiar
with.

Can't I do:

Directory["/path/to/templates/**"].each do |file|
template /tmp/foo/file do
local true
source file
end
end

??

I'm not sure how to reference the file variable in the actual template
resource now tho...

On Thu, Nov 29, 2012 at 11:38 AM, Adam Jacob adam@opscode.com wrote:

Ack!

Something like this:

remote_file "/srv/app/deploy.zip" do
source "http://example.com/deploy.zip"
end

execute "unzip /srv/app/deploy.zip"

ruby_block "process templates" do
source <<-EOH
Directory["/srv/app/templates/**"].each do |file|
...
end
EOH
End

Should do the trick (where Š is fill in the blanks).

Adam

On 11/29/12 7:45 AM, "Maven User" maven.2.user@gmail.com wrote:

Sure -

So because of the way we structure our deployable units (zip files) we
wind up supplying templates inside there (maven injects the erb syntax
into these files).

The structure looks like this:

foo
|_bin
|_conf
|_templates
|_bin
|_conf

Essentially, I want to take any of the directories inside templates, and
process the files inside to identical directories one higher.

I don't want to have a hard coded list of templates anywhere.

If this were ant or maven or something, I'd just use something like
templates/**.

Does this help explain a bit more? I'm not sure how awesome my ASCII
art
will look like once posted....

On Thu, Nov 29, 2012 at 10:37 AM, Adam Jacob
adam@opscode.com wrote:

I'm actually confused by the question. Can you explain it again,
perhaps a
bit differently, perhaps with a more explicit example of your intent?

Adam

On 11/29/12 7:15 AM, "Maven User" maven.2.user@gmail.com wrote:

Hi all -

Before I go off the rails and start trying to iterate over "n" number
of
directories and their contents, is there a great way to say, "hey, in
this templates directory, move everything one directory higher (or
FQP)"?

I don't see any sort of wild carding kinda stuff around the templates
or
being able to specify a templates directory to be processed.

Thanks for any tips.

10

Hey guys -

Thanks for the feed back!

I have this much working:

Dir["/some/path/to/templates/**/*"].each do |file|
if (File.file?(file))
filename = File.basename(file)
template "/tmp/foo/#{filename}" do
local true
source "#{file}"
end
end
end

The only outstanding issue with this is the path is NOT
maintained/preserved :-/

so if I have:

templates/subdir/textfile.txt

and the target location is:

/tmp/foo

I wind up with:

/tmp/foo/textfile.txt

when I really want

/tmp/foo/subdir/textfile.txt

I think I need to add in a directory resource....

On Thu, Nov 29, 2012 at 3:39 PM, AJ Christensen aj@junglist.gen.nz wrote:

You could use the ruby_block to generate (and store) an array of template
paths to interact with, during convergence phase after the extraction.

Cheers,

--AJ

On 30 November 2012 09:38, Matthew Moretti werebus@gmail.com wrote:

The only issue is that the "each" loop as you have it will be run at
compile-time. So if you don't create the directory full of templates until
convergence (eg: with Chef), that will be an empty array. This is why Adam
was using the "ruby_block" resource.

Of couse the ruby_block resource might have its own problems. I don't
know whether something like this will actually work (I think it won't)

ruby_block "process templates" do
Directory["/srv/app/templates/**"].each do |filename|
template filename do
#stuff
end
end
end

-Matt Moretti

On Thu, Nov 29, 2012 at 2:35 PM, Maven User maven.2.user@gmail.comwrote:

Hmmm - ok, it's a bit clearer now, but this is an area I'm not familiar
with.

Can't I do:

Directory["/path/to/templates/**"].each do |file|
template /tmp/foo/file do
local true
source file
end
end

??

I'm not sure how to reference the file variable in the actual template
resource now tho...

On Thu, Nov 29, 2012 at 11:38 AM, Adam Jacob adam@opscode.com wrote:

Ack!

Something like this:

remote_file "/srv/app/deploy.zip" do
source "http://example.com/deploy.zip"
end

execute "unzip /srv/app/deploy.zip"

ruby_block "process templates" do
source <<-EOH
Directory["/srv/app/templates/**"].each do |file|
...
end
EOH
End

Should do the trick (where Š is fill in the blanks).

Adam

On 11/29/12 7:45 AM, "Maven User" maven.2.user@gmail.com wrote:

Sure -

So because of the way we structure our deployable units (zip files) we
wind up supplying templates inside there (maven injects the erb syntax
into these files).

The structure looks like this:

foo
|_bin
|_conf
|_templates
|_bin
|_conf

Essentially, I want to take any of the directories inside templates,
and
process the files inside to identical directories one higher.

I don't want to have a hard coded list of templates anywhere.

If this were ant or maven or something, I'd just use something like
templates/**.

Does this help explain a bit more? I'm not sure how awesome my ASCII
art
will look like once posted....

On Thu, Nov 29, 2012 at 10:37 AM, Adam Jacob
adam@opscode.com wrote:

I'm actually confused by the question. Can you explain it again,
perhaps a
bit differently, perhaps with a more explicit example of your intent?

Adam

On 11/29/12 7:15 AM, "Maven User" maven.2.user@gmail.com wrote:

Hi all -

Before I go off the rails and start trying to iterate over "n" number
of
directories and their contents, is there a great way to say, "hey, in
this templates directory, move everything one directory higher (or
FQP)"?

I don't see any sort of wild carding kinda stuff around the templates
or
being able to specify a templates directory to be processed.

Thanks for any tips.

11

I'm not sure anyone but me even cares - but if anyone is looking to move an
entire directory structure of templates without specifying each template
individually - maybe this will help them too!

Dir["/your/path/to/your/templates/**/*"].each do |file|
if (File.file?(file))
filename = File.basename(file)
newfilename = File.absolute_path(file).split("templates/")
if (newfilename[1].rindex('/') != nil && newfilename[1].rindex('/') >
1)
dirstocreate = newfilename[1][0, newfilename[1].rindex('/')]
directory "/path/you/want/them/created/in/#{dirstocreate}" do
recursive true
action :create
end
end
template "//path/you/want/them/created/in/#{newfilename[1]}" do
local true
source "#{file}"
end
end
end

seemed to work for me - hope this helps someone else!

On Thu, Nov 29, 2012 at 3:46 PM, Maven User maven.2.user@gmail.com wrote:

Hey guys -

Thanks for the feed back!

I have this much working:

Dir["/some/path/to/templates/**/*"].each do |file|
if (File.file?(file))
filename = File.basename(file)
template "/tmp/foo/#{filename}" do
local true
source "#{file}"
end
end
end

The only outstanding issue with this is the path is NOT
maintained/preserved :-/

so if I have:

templates/subdir/textfile.txt

and the target location is:

/tmp/foo

I wind up with:

/tmp/foo/textfile.txt

when I really want

/tmp/foo/subdir/textfile.txt

I think I need to add in a directory resource....

On Thu, Nov 29, 2012 at 3:39 PM, AJ Christensen aj@junglist.gen.nzwrote:

You could use the ruby_block to generate (and store) an array of template
paths to interact with, during convergence phase after the extraction.

Cheers,

--AJ

On 30 November 2012 09:38, Matthew Moretti werebus@gmail.com wrote:

The only issue is that the "each" loop as you have it will be run at
compile-time. So if you don't create the directory full of templates until
convergence (eg: with Chef), that will be an empty array. This is why Adam
was using the "ruby_block" resource.

Of couse the ruby_block resource might have its own problems. I don't
know whether something like this will actually work (I think it won't)

ruby_block "process templates" do
Directory["/srv/app/templates/**"].each do |filename|
template filename do
#stuff
end
end
end

-Matt Moretti

On Thu, Nov 29, 2012 at 2:35 PM, Maven User maven.2.user@gmail.comwrote:

Hmmm - ok, it's a bit clearer now, but this is an area I'm not familiar
with.

Can't I do:

Directory["/path/to/templates/**"].each do |file|
template /tmp/foo/file do
local true
source file
end
end

??

I'm not sure how to reference the file variable in the actual template
resource now tho...

On Thu, Nov 29, 2012 at 11:38 AM, Adam Jacob adam@opscode.com wrote:

Ack!

Something like this:

remote_file "/srv/app/deploy.zip" do
source "http://example.com/deploy.zip"
end

execute "unzip /srv/app/deploy.zip"

ruby_block "process templates" do
source <<-EOH
Directory["/srv/app/templates/**"].each do |file|
...
end
EOH
End

Should do the trick (where Š is fill in the blanks).

Adam

On 11/29/12 7:45 AM, "Maven User" maven.2.user@gmail.com wrote:

Sure -

So because of the way we structure our deployable units (zip files) we
wind up supplying templates inside there (maven injects the erb syntax
into these files).

The structure looks like this:

foo
|_bin
|_conf
|_templates
|_bin
|_conf

Essentially, I want to take any of the directories inside templates,
and
process the files inside to identical directories one higher.

I don't want to have a hard coded list of templates anywhere.

If this were ant or maven or something, I'd just use something like
templates/**.

Does this help explain a bit more? I'm not sure how awesome my ASCII
art
will look like once posted....

On Thu, Nov 29, 2012 at 10:37 AM, Adam Jacob
adam@opscode.com wrote:

I'm actually confused by the question. Can you explain it again,
perhaps a
bit differently, perhaps with a more explicit example of your intent?

Adam

On 11/29/12 7:15 AM, "Maven User" maven.2.user@gmail.com wrote:

Hi all -

Before I go off the rails and start trying to iterate over "n"
number of
directories and their contents, is there a great way to say, "hey, in
this templates directory, move everything one directory higher (or
FQP)"?

I don't see any sort of wild carding kinda stuff around the
templates or
being able to specify a templates directory to be processed.

Thanks for any tips.