Need to copy a file after installation to another place


#1

I’m creating a node in a MongoDB replica set that furnishes a) an
arbiter, b) a configsvr and c) a sharding router. Unfortunately, as the
binary that implements the arbiter and the configsvr are one and the
same (mongod) and it cannot be multiply loaded, the customary action is
to make a copy of it. This sort of thing is only done on testing
instances (not in production).

So, one recipe installs MongoDB via the Debian package. mongod is on the
path //usr/bin/mongod/ and I want to make a copy of it on the path
//data/mongodb/bin/mongd/. My /configsvr.rb/ recipe ends with this:

# Copy /usr/bin/mongod to /data/mongodb/bin/mongod because a
separate copy
# must exist in order for the configuration server's mongod instance not
# to conflict with the arbiter's instance. This copy will be
referenced by
# /etc/init/mongodb-configsvr.conf.
file "/data/mongodb/bin/mongod" do
   action :create
   owner "mongodb"
   group "mongodb"
   mode 00755
   content IO.read( "/usr/bin/mongod" )
end

Sadly, this only creates a zero-length file.

I’m guessing this will be a Chef anti-pattern and I do want to hear the
cat-calls, but I’d also like a solution.

Profuse thanks for reading this.

Best,

Russ


#2

You could just create a symlink in the location you want.

Nic

On Wed, Sep 4, 2013 at 1:06 PM, Russell Bateman russ@windofkeltia.comwrote:

I’m creating a node in a MongoDB replica set that furnishes a) an
arbiter, b) a configsvr and c) a sharding router. Unfortunately, as the
binary that implements the arbiter and the configsvr are one and the same
(mongod) and it cannot be multiply loaded, the customary action is to make
a copy of it. This sort of thing is only done on testing instances (not in
production).

So, one recipe installs MongoDB via the Debian package. mongod is on the
path /usr/bin/mongod and I want to make a copy of it on the path *
/data/mongodb/bin/mongd*. My configsvr.rb recipe ends with this:

Copy /usr/bin/mongod to /data/mongodb/bin/mongod because a separate

copy

must exist in order for the configuration server’s mongod instance not

to conflict with the arbiter’s instance. This copy will be referenced by

/etc/init/mongodb-configsvr.conf.

file “/data/mongodb/bin/mongod” do
action :create
owner "mongodb"
group "mongodb"
mode 00755
content IO.read( “/usr/bin/mongod” )
end ****

Sadly, this only creates a zero-length file.

I’m guessing this will be a Chef anti-pattern and I do want to hear the
cat-calls, but I’d also like a solution.

Profuse thanks for reading this.

Best,

Russ


#3

On Wednesday, September 4, 2013 at 11:06 AM, Russell Bateman wrote:

I’m creating a node in a MongoDB replica set that furnishes a) an arbiter, b) a configsvr and c) a sharding router. Unfortunately, as the binary that implements the arbiter and the configsvr are one and the same (mongod) and it cannot be multiply loaded, the customary action is to make a copy of it. This sort of thing is only done on testing instances (not in production).

So, one recipe installs MongoDB via the Debian package. mongod is on the path /usr/bin/mongod and I want to make a copy of it on the path /data/mongodb/bin/mongd. My configsvr.rb recipe ends with this:

Copy /usr/bin/mongod to /data/mongodb/bin/mongod because a separate copy

must exist in order for the configuration server’s mongod instance not

to conflict with the arbiter’s instance. This copy will be referenced by

/etc/init/mongodb-configsvr.conf.

file “/data/mongodb/bin/mongod” do
action :create
owner "mongodb"
group "mongodb"
mode 00755
content IO.read( “/usr/bin/mongod” )
end Sadly, this only creates a zero-length file.

I’m guessing this will be a Chef anti-pattern and I do want to hear the cat-calls, but I’d also like a solution.

Profuse thanks for reading this.

Best,

Russ
In Chef 11.6 you can use a file:/// URL for a remote file to copy it from a local source.


Daniel DeLeo


#4

Thanks, many have tried that, but the OS will not be fooled. It must be
a copy.

On 9/4/2013 12:07 PM, Nic Grayson wrote:

You could just create a symlink in the location you want.

Nic

On Wed, Sep 4, 2013 at 1:06 PM, Russell Bateman <russ@windofkeltia.com
mailto:russ@windofkeltia.com> wrote:

I'm creating a node in a MongoDB replica set that furnishes a) an
arbiter, b) a configsvr and c) a sharding router. Unfortunately,
as the binary that implements the arbiter and the configsvr are
one and the same (mongod) and it cannot be multiply loaded, the
customary action is to make a copy of it. This sort of thing is
only done on testing instances (not in production).

So, one recipe installs MongoDB via the Debian package. mongod is
on the path //usr/bin/mongod/ and I want to make a copy of it on
the path //data/mongodb/bin/mongd/. My /configsvr.rb/ recipe ends
with this:

    # Copy /usr/bin/mongod to /data/mongodb/bin/mongod because a
    separate copy
    # must exist in order for the configuration server's mongod
    instance not
    # to conflict with the arbiter's instance. This copy will be
    referenced by
    # /etc/init/mongodb-configsvr.conf.
    file "/data/mongodb/bin/mongod" do
      action :create
      owner "mongodb"
      group "mongodb"
      mode 00755
      content IO.read( "/usr/bin/mongod" )
    end

Sadly, this only creates a zero-length file.

I'm guessing this will be a Chef anti-pattern and I do want to
hear the cat-calls, but I'd also like a solution.

Profuse thanks for reading this.

Best,

Russ

#5

Thanks for responding. I’m unclear on this. I’m copying a file from one
place on the client node to another place on the client node. Which of
these two places is “remote”?

On 9/4/2013 12:08 PM, Daniel DeLeo wrote:

On Wednesday, September 4, 2013 at 11:06 AM, Russell Bateman wrote:

I’m creating a node in a MongoDB replica set that furnishes a) an
arbiter, b) a configsvr and c) a sharding router. Unfortunately, as
the binary that implements the arbiter and the configsvr are one and
the same (mongod) and it cannot be multiply loaded, the customary
action is to make a copy of it. This sort of thing is only done on
testing instances (not in production).

So, one recipe installs MongoDB via the Debian package. mongod is on
the path //usr/bin/mongod/ and I want to make a copy of it on the
path //data/mongodb/bin/mongd/. My /configsvr.rb/ recipe ends with this:

# Copy /usr/bin/mongod to /data/mongodb/bin/mongod because a
separate copy
# must exist in order for the configuration server's mongod
instance not
# to conflict with the arbiter's instance. This copy will be
referenced by
# /etc/init/mongodb-configsvr.conf.
file "/data/mongodb/bin/mongod" do
  action :create
  owner "mongodb"
  group "mongodb"
  mode 00755
  content IO.read( "/usr/bin/mongod" )
end

Sadly, this only creates a zero-length file.

I’m guessing this will be a Chef anti-pattern and I do want to hear
the cat-calls, but I’d also like a solution.

Profuse thanks for reading this.

Best,

Russ
In Chef 11.6 you can use a file:/// URL for a remote file to copy it
from a local source.


Daniel DeLeo


#6

On Wednesday, September 4, 2013 at 11:11 AM, Russell Bateman wrote:

Thanks for responding. I’m unclear on this. I’m copying a file from one place on the client node to another place on the client node. Which of these two places is “remote”?

Not sure exactly what you’re commenting on, so…

Yeah, it’s a little odd that it’s called “remote_file” and now it copies files from a filesystem on a box to a filesystem on the same box. We recently rewrote the internals so the thing that used to be exclusively focused on fetching an HTTP resource and copying it to disk now internally has a few backends, which allows it to support ftp and file URLs. In the long term we could consider renaming it or adding an alias if the current situation impedes grok-ability

As for actually using it, you want to do

remote_file("/path/to/destination") do
source “file:///path/to/original”

other stuff

end


Daniel DeLeo

On 9/4/2013 12:08 PM, Daniel DeLeo wrote:

On Wednesday, September 4, 2013 at 11:06 AM, Russell Bateman wrote:

I’m creating a node in a MongoDB replica set that furnishes a) an arbiter, b) a configsvr and c) a sharding router. Unfortunately, as the binary that implements the arbiter and the configsvr are one and the same (mongod) and it cannot be multiply loaded, the customary action is to make a copy of it. This sort of thing is only done on testing instances (not in production).

So, one recipe installs MongoDB via the Debian package. mongod is on the path /usr/bin/mongod and I want to make a copy of it on the path /data/mongodb/bin/mongd. My configsvr.rb recipe ends with this:

Copy /usr/bin/mongod to /data/mongodb/bin/mongod because a separate copy

must exist in order for the configuration server’s mongod instance not

to conflict with the arbiter’s instance. This copy will be referenced by

/etc/init/mongodb-configsvr.conf.

file “/data/mongodb/bin/mongod” do
action :create
owner "mongodb"
group "mongodb"
mode 00755
content IO.read( “/usr/bin/mongod” )
end Sadly, this only creates a zero-length file.

I’m guessing this will be a Chef anti-pattern and I do want to hear the cat-calls, but I’d also like a solution.

Profuse thanks for reading this.

Best,

Russ
In Chef 11.6 you can use a file:/// URL for a remote file to copy it from a local source.


Daniel DeLeo


#7

Adopting that, …

remote_file( "/data/mongodb/bin/mongod" )
   source "file:///usr/bin/mongod"
   owner "mongodb"
   group "mongodb"
   mode 00755
end

I get:

Errno::ECONNREFUSED
----------------------------------
Connection refused - Connection refused connecting to :
for*/usr/bin/mongod/file:///usr/bin/mongod*, giving up

I’m playing around trying to trick it not prefix it with redundant path,
but to no avail so far.

Thanks.

On 9/4/2013 12:35 PM, Daniel DeLeo wrote:

On Wednesday, September 4, 2013 at 11:11 AM, Russell Bateman wrote:

Thanks for responding. I’m unclear on this. I’m copying a file from
one place on the client node to another place on the client node.
Which of these two places is “remote”?

Not sure exactly what you’re commenting on, so…

Yeah, it’s a little odd that it’s called “remote_file” and now it
copies files from a filesystem on a box to a filesystem on the same
box. We recently rewrote the internals so the thing that used to be
exclusively focused on fetching an HTTP resource and copying it to
disk now internally has a few backends, which allows it to support ftp
and file URLs. In the long term we could consider renaming it or
adding an alias if the current situation impedes grok-ability

As for actually using it, you want to do

remote_file("/path/to/destination") do
source “file:///path/to/original”

other stuff

end


Daniel DeLeo

On 9/4/2013 12:08 PM, Daniel DeLeo wrote:

On Wednesday, September 4, 2013 at 11:06 AM, Russell Bateman wrote:

I’m creating a node in a MongoDB replica set that furnishes a) an
arbiter, b) a configsvr and c) a sharding router. Unfortunately, as
the binary that implements the arbiter and the configsvr are one
and the same (mongod) and it cannot be multiply loaded, the
customary action is to make a copy of it. This sort of thing is
only done on testing instances (not in production).

So, one recipe installs MongoDB via the Debian package. mongod is
on the path //usr/bin/mongod/ and I want to make a copy of it on
the path //data/mongodb/bin/mongd/. My /configsvr.rb/ recipe ends
with this:

# Copy /usr/bin/mongod to /data/mongodb/bin/mongod because a
separate copy
# must exist in order for the configuration server's mongod
instance not
# to conflict with the arbiter's instance. This copy will be
referenced by
# /etc/init/mongodb-configsvr.conf.
file "/data/mongodb/bin/mongod" do
  action :create
  owner "mongodb"
  group "mongodb"
  mode 00755
  content IO.read( "/usr/bin/mongod" )
end

Sadly, this only creates a zero-length file.

I’m guessing this will be a Chef anti-pattern and I do want to hear
the cat-calls, but I’d also like a solution.

Profuse thanks for reading this.

Best,

Russ
In Chef 11.6 you can use a file:/// URL for a remote file to copy
it from a local source.


Daniel DeLeo


#8

On Wednesday, September 4, 2013 at 12:57 PM, Russell Bateman wrote:

Adopting that, …

remote_file( “/data/mongodb/bin/mongod” )
source “file:///usr/bin/mongod” (file:///usr/bin/mongod)
owner "mongodb"
group "mongodb"
mode 00755
end

I get:

Errno::ECONNREFUSED

Connection refused - Connection refused connecting to : for /usr/bin/mongod/file:///usr/bin/mongod, giving up
I’m playing around trying to trick it not prefix it with redundant path, but to no avail so far.

Thanks.
Are you using Chef 11.6.0?


Daniel DeLeo


#9

No, on the client I’m using 11.4.4-2. My chef server is running 10.18.1.

Incidentally, I also tried a ruby_block using FileUtils.cp() which
didn’t work either.

file "/data/mongodb/bin/mongodb" do
     action :create
     owner "mongodb"
     group "mongodb"
     mode 00644
end

require 'fileutils'

ruby_block "copy-mongod-binary"
     block do
         FileUtils.copy( "/usr/bin/mongod",
"/data/mongodb/bin/mongod", { :preserve => false } )
     end
end

On 9/4/2013 3:04 PM, Daniel DeLeo wrote:

On Wednesday, September 4, 2013 at 12:57 PM, Russell Bateman wrote:

Adopting that, …

remote_file( "/data/mongodb/bin/mongod" )
  source "file:///usr/bin/mongod" <file:///usr/bin/mongod>
  owner "mongodb"
  group "mongodb"
  mode 00755
end

I get:

Errno::ECONNREFUSED
----------------------------------
Connection refused - Connection refused connecting to :
for*/usr/bin/mongod/file:///usr/bin/mongod*, giving up

I’m playing around trying to trick it not prefix it with redundant
path, but to no avail so far.

Thanks.
Are you using Chef 11.6.0?


Daniel DeLeo


#10

While I’m not convinced that this is impossible in “normal” Chef recipe
code, I stumbled upon the idea of just using a script to do it. (I am,
after all, new to and still very perplexed by Chef.) Here’s what I did
to get around my inability to make the “remote file to remote file” copy
work:

# Create a script on the fly to copy the mongod daemon for our
configuration
# server to run off of. (The latter cannot share /usr/bin/mongod
because the
# arbiter is going to be using that one.)
bash "copy-mongod" do
   user "root"
   cwd "/data/mongodb"
   code <<-EOS
   cp /usr/bin/mongod ./bin
   chown mongodb:mongodb ./bin/mongod
   chmod a+x ./bin/mongod
   EOS
end

Thanks for bearing with me on this!

Russ

On 9/4/2013 3:22 PM, Russell Bateman wrote:

No, on the client I’m using 11.4.4-2. My chef server is running 10.18.1.

Incidentally, I also tried a ruby_block using FileUtils.cp() which
didn’t work either.

file "/data/mongodb/bin/mongodb" do
    action :create
    owner "mongodb"
    group "mongodb"
    mode 00644
end

require 'fileutils'

ruby_block "copy-mongod-binary"
    block do
        FileUtils.copy( "/usr/bin/mongod",
"/data/mongodb/bin/mongod", { :preserve => false } )
    end
end

On 9/4/2013 3:04 PM, Daniel DeLeo wrote:

On Wednesday, September 4, 2013 at 12:57 PM, Russell Bateman wrote:

Adopting that, …

remote_file( "/data/mongodb/bin/mongod" )
  source "file:///usr/bin/mongod" <file:///usr/bin/mongod>
  owner "mongodb"
  group "mongodb"
  mode 00755
end

I get:

Errno::ECONNREFUSED
----------------------------------
Connection refused - Connection refused connecting to :
for*/usr/bin/mongod/file:///usr/bin/mongod*, giving up

I’m playing around trying to trick it not prefix it with redundant
path, but to no avail so far.

Thanks.
Are you using Chef 11.6.0?


Daniel DeLeo


#11

On Thursday, September 5, 2013 at 8:43 AM, Russell Bateman wrote:

While I’m not convinced that this is impossible in “normal” Chef recipe code, I stumbled upon the idea of just using a script to do it. (I am, after all, new to and still very perplexed by Chef.) Here’s what I did to get around my inability to make the “remote file to remote file” copy work:

Create a script on the fly to copy the mongod daemon for our configuration

server to run off of. (The latter cannot share /usr/bin/mongod because the

arbiter is going to be using that one.)

bash “copy-mongod” do
user "root"
cwd "/data/mongodb"
code <<-EOS
cp /usr/bin/mongod ./bin
chown mongodb:mongodb ./bin/mongod
chmod a+x ./bin/mongod
EOS
end
Thanks for bearing with me on this!

Russ
I think what’s confusing you here is the compile phase vs. converge phase. Chef evaluates all of your recipes in one go, which results in an ordered list of resources to converge (the resource collection). Once the resource collection is compiled, Chef goes through the resource collection in order and configures each resource (or not, if it’s already configured).

This is (probably) why your implementation like

file “/path/to/copy” do
content IO.read "path/to/original"
end

didn’t work.

On 9/4/2013 3:22 PM, Russell Bateman wrote:

No, on the client I’m using 11.4.4-2. My chef server is running 10.18.1.
Copying local files via a “file://” URL requires Chef 11.6.


Daniel DeLeo