# More problems with remote\_file

**URL:** <https://discourse.chef.io/t/more-problems-with-remote-file/1195>\
**Category:** Chef Infra (archive)\
**Created:** [June 8, 2009, 5:16pm UTC](https://discourse.chef.io/t/more-problems-with-remote-file/1195 "2009-06-08T17:16:35Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Jeppe\_Nejsum\_Madsen](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/jeppe_nejsum_madsen/32/258_2.png) [@Jeppe\_Nejsum\_Madsen](https://discourse.chef.io/u/Jeppe_Nejsum_Madsen)\
**Post date:** [June 8, 2009, 5:16pm UTC](https://discourse.chef.io/t/more-problems-with-remote-file/1195/1 "2009-06-08T17:16:35Z")

</div>

Hi,

I seem to occasionally get some errors with remote\_file when running  
chef-solo. It is not consistently reproducible, but seems to occur  
when I rerun a chef-solo command and the target file already exists.

This is the stack trace:

Mon, 08 Jun 2009 18:16:20 +0200] DEBUG: /helloworld-1.war 100% done  
(7565054 of 7565054)  
[Mon, 08 Jun 2009 18:16:20 +0200] ERROR: remote\_file[test3] (/var/lib/  
gems/1.8/gems/chef-0.6.2/lib/chef/recipe.rb line 187) had an error:  
closed stream  
/var/lib/gems/1.8/gems/chef-0.6.2/lib/chef/provider/remote\_file.rb:  
76:in `close'/var/lib/gems/1.8/gems/chef-0.6.2/lib/chef/provider/ remote_file.rb:76:in`do\_remote\_file’/var/lib/gems/1.8/gems/chef-0.6.2/  
lib/chef/provider/remote\_file.rb:34:in `action_create'/var/lib/gems/ 1.8/gems/chef-0.6.2/lib/chef/runner.rb:87:in`send’/var/lib/gems/1.8/  
gems/chef-0.6.2/lib/chef/runner.rb:87:in `converge'/var/lib/gems/1.8/ gems/chef-0.6.2/lib/chef/runner.rb:85:in`each’/var/lib/gems/1.8/gems/  
chef-0.6.2/lib/chef/runner.rb:85:in `converge'/var/lib/gems/1.8/gems/ chef-0.6.2/lib/chef/resource_collection.rb:58:in`each’/var/lib/gems/  
1.8/gems/chef-0.6.2/lib/chef/resource\_collection.rb:57:in `each'/var/ lib/gems/1.8/gems/chef-0.6.2/lib/chef/runner.rb:63:in`converge’/var/  
lib/gems/1.8/gems/chef-0.6.2/lib/chef/client.rb:348:in `converge'/var/ lib/gems/1.8/gems/chef-0.6.2/lib/chef/client.rb:96:in`run\_solo’/var/  
lib/gems/1.8/gems/chef-0.6.2/bin/chef-solo:118/sbin/chef-solo:19:in  
`load'/sbin/chef-solo:19 /var/lib/gems/1.8/gems/chef-0.6.2/lib/chef/provider/remote_file.rb: 76:in`close’: closed stream (IOError)  
from /var/lib/gems/1.8/gems/chef-0.6.2/lib/chef/provider/  
remote\_file.rb:76:in `do_remote_file' from /var/lib/gems/1.8/gems/chef-0.6.2/lib/chef/provider/ remote_file.rb:34:in`action\_create’  
from /var/lib/gems/1.8/gems/chef-0.6.2/lib/chef/runner.rb:87:in `send' from /var/lib/gems/1.8/gems/chef-0.6.2/lib/chef/runner.rb:87:in`converge’  
from /var/lib/gems/1.8/gems/chef-0.6.2/lib/chef/runner.rb:85:in `each' from /var/lib/gems/1.8/gems/chef-0.6.2/lib/chef/runner.rb:85:in`converge’  
from /var/lib/gems/1.8/gems/chef-0.6.2/lib/chef/  
resource\_collection.rb:58:in `each' from /var/lib/gems/1.8/gems/chef-0.6.2/lib/chef/ resource_collection.rb:57:in`each’  
from /var/lib/gems/1.8/gems/chef-0.6.2/lib/chef/runner.rb:63:in  
`converge' from /var/lib/gems/1.8/gems/chef-0.6.2/lib/chef/client.rb:348:in`converge’  
from /var/lib/gems/1.8/gems/chef-0.6.2/lib/chef/client.rb:96:in  
`run_solo' from /var/lib/gems/1.8/gems/chef-0.6.2/bin/chef-solo:118 from /sbin/chef-solo:19:in`load’  
from /sbin/chef-solo:19

I’m not a ruby guy, but took a quick look at the code (chef 0.6.2) and  
at a first glance it seems that the filehandle may be closed twice  
(rest.rb:189) and in remote\_file:76. But if this is true, why doesn’t  
it always fail?

/Jeppe

---

<div class="post-metadata">

**Author:** ![Jeppe\_Nejsum\_Madsen](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/jeppe_nejsum_madsen/32/258_2.png) [@Jeppe\_Nejsum\_Madsen](https://discourse.chef.io/u/Jeppe_Nejsum_Madsen)\
**Post date:** [June 9, 2009, 9:58am UTC](https://discourse.chef.io/t/more-problems-with-remote-file/1195/2 "2009-06-09T09:58:39Z")

</div>

On Mon, Jun 8, 2009 at 7:16 PM, Jeppe Nejsum Madsen[jeppe@ingolfs.dk](mailto:jeppe@ingolfs.dk) wrote:

> Hi,
> 
> I seem to occasionally get some errors with remote\_file when running  
> chef-solo. It is not consistently reproducible, but seems to occur when I  
> rerun a chef-solo command and the target file already exists.

Hmm seems to be a platform thing. On an EC2 instance running Ubuntu  
8.04, I cannot reproduce this. On an EC2 instance running 9.04 I can  
consistently reproduce this....

/Jeppe

---

<div class="post-metadata">

**Author:** ![Jeppe\_Nejsum\_Madsen](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/jeppe_nejsum_madsen/32/258_2.png) [@Jeppe\_Nejsum\_Madsen](https://discourse.chef.io/u/Jeppe_Nejsum_Madsen)\
**Post date:** [June 9, 2009, 1:11pm UTC](https://discourse.chef.io/t/more-problems-with-remote-file/1195/3 "2009-06-09T13:11:44Z")

</div>

On Tue, Jun 9, 2009 at 11:58 AM, Jeppe Nejsum Madsen[jeppe@ingolfs.dk](mailto:jeppe@ingolfs.dk) wrote:

> On Mon, Jun 8, 2009 at 7:16 PM, Jeppe Nejsum Madsen[jeppe@ingolfs.dk](mailto:jeppe@ingolfs.dk) wrote:
> 
> > Hi,
> > 
> > I seem to occasionally get some errors with remote\_file when running  
> > chef-solo. It is not consistently reproducible, but seems to occur when I  
> > rerun a chef-solo command and the target file already exists.
> 
> Hmm seems to be a platform thing. On an EC2 instance running Ubuntu  
> 8.04, I cannot reproduce this. On an EC2 instance running 9.04 I can  
> consistently reproduce this....
> 
> /Jeppe

And just to keep the dialogue with myself: I'm pretty sure it's  
caused by Ruby 1.8.7 which is default on Ubuntu 9.04. Downgrading to  
1.8.6 solves the problem.

The strange part is that it only happens on some files. I've created a  
very small test scenario which consistently fails on 9.04 after the  
first successful run. After downgrading to Ruby 1.8.6, I can run this  
scenario several times without failure:

1. Create solo.rb  
cookbook\_path "/etc/chef/recipes/cookbooks"  
log\_level :debug  
file\_store\_path "/var/chef/files/"  
file\_cache\_path "/etc/chef/recipes"

2. gem install --no-ri --no-rdoc --bindir /sbin ohai chef --source  
[http://gems.opscode.com](http://gems.opscode.com) --source [http://gems.rubyforge.org](http://gems.rubyforge.org)

3. chef-solo -c solo.rb -j [http://jnmtest.s3.amazonaws.com/dna.json](http://jnmtest.s3.amazonaws.com/dna.json) -l  
debug -r [http://jnmtest.s3.amazonaws.com/cookbooks.tgz](http://jnmtest.s3.amazonaws.com/cookbooks.tgz)

4. chef-solo -c solo.rb -j [http://jnmtest.s3.amazonaws.com/dna.json](http://jnmtest.s3.amazonaws.com/dna.json) -l  
debug -r [http://jnmtest.s3.amazonaws.com/cookbooks.tgz](http://jnmtest.s3.amazonaws.com/cookbooks.tgz)

The last step always fails for the last file when running on 9.04 with  
Ruby 1.8.7 but works with Ruby 1.8.6

/Jeppe

---

<div class="post-metadata">

**Author:** ![Adam\_Jacob](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/adam_jacob/32/292_2.png) [@Adam\_Jacob](https://discourse.chef.io/u/Adam_Jacob)\
**Post date:** [June 10, 2009, 2:09am UTC](https://discourse.chef.io/t/more-problems-with-remote-file/1195/4 "2009-06-10T02:09:47Z")

</div>

Jeppe, can you try this with 0.7.0?

Sent from my iPhone

On Jun 9, 2009, at 6:11 AM, Jeppe Nejsum Madsen [jeppe@ingolfs.dk](mailto:jeppe@ingolfs.dk)  
wrote:

> On Tue, Jun 9, 2009 at 11:58 AM, Jeppe Nejsum  
> Madsen[jeppe@ingolfs.dk](mailto:jeppe@ingolfs.dk) wrote:
> 
> > On Mon, Jun 8, 2009 at 7:16 PM, Jeppe Nejsum  
> > Madsen[jeppe@ingolfs.dk](mailto:jeppe@ingolfs.dk) wrote:
> > 
> > > Hi,
> > > 
> > > I seem to occasionally get some errors with remote\_file when running  
> > > chef-solo. It is not consistently reproducible, but seems to occur  
> > > when I  
> > > rerun a chef-solo command and the target file already exists.
> > 
> > Hmm seems to be a platform thing. On an EC2 instance running Ubuntu  
> > 8.04, I cannot reproduce this. On an EC2 instance running 9.04 I can  
> > consistently reproduce this....
> > 
> > /Jeppe
> 
> And just to keep the dialogue with myself: I'm pretty sure it's  
> caused by Ruby 1.8.7 which is default on Ubuntu 9.04. Downgrading to  
> 1.8.6 solves the problem.
> 
> The strange part is that it only happens on some files. I've created a  
> very small test scenario which consistently fails on 9.04 after the  
> first successful run. After downgrading to Ruby 1.8.6, I can run this  
> scenario several times without failure:
> 
> 1. Create solo.rb  
> cookbook\_path "/etc/chef/recipes/cookbooks"  
> log\_level :debug  
> file\_store\_path "/var/chef/files/"  
> file\_cache\_path "/etc/chef/recipes"
> 
> 2. gem install --no-ri --no-rdoc --bindir /sbin ohai chef --source  
> [http://gems.opscode.com](http://gems.opscode.com) --source [http://gems.rubyforge.org](http://gems.rubyforge.org)
> 
> 3. chef-solo -c solo.rb -j [http://jnmtest.s3.amazonaws.com/dna.json](http://jnmtest.s3.amazonaws.com/dna.json) -l  
> debug -r [http://jnmtest.s3.amazonaws.com/cookbooks.tgz](http://jnmtest.s3.amazonaws.com/cookbooks.tgz)
> 
> 4. chef-solo -c solo.rb -j [http://jnmtest.s3.amazonaws.com/dna.json](http://jnmtest.s3.amazonaws.com/dna.json) -l  
> debug -r [http://jnmtest.s3.amazonaws.com/cookbooks.tgz](http://jnmtest.s3.amazonaws.com/cookbooks.tgz)
> 
> The last step always fails for the last file when running on 9.04 with  
> Ruby 1.8.7 but works with Ruby 1.8.6
> 
> /Jeppe

---

<div class="post-metadata">

**Author:** ![Jeppe\_Nejsum\_Madsen](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/jeppe_nejsum_madsen/32/258_2.png) [@Jeppe\_Nejsum\_Madsen](https://discourse.chef.io/u/Jeppe_Nejsum_Madsen)\
**Post date:** [June 10, 2009, 6:55am UTC](https://discourse.chef.io/t/more-problems-with-remote-file/1195/5 "2009-06-10T06:55:12Z")

</div>

Seems to work. At least unable to reproduce with the steps below. Nice!

Was this a deliberate fix or just fixed by coincidence?

/Jeppe

On 10/06/2009, at 04.09, Adam Jacob wrote:

> Jeppe, can you try this with 0.7.0?
> 
> Sent from my iPhone
> 
> On Jun 9, 2009, at 6:11 AM, Jeppe Nejsum Madsen [jeppe@ingolfs.dk](mailto:jeppe@ingolfs.dk)  
> wrote:
> 
> > On Tue, Jun 9, 2009 at 11:58 AM, Jeppe Nejsum  
> > Madsen[jeppe@ingolfs.dk](mailto:jeppe@ingolfs.dk) wrote:
> > 
> > > On Mon, Jun 8, 2009 at 7:16 PM, Jeppe Nejsum  
> > > Madsen[jeppe@ingolfs.dk](mailto:jeppe@ingolfs.dk) wrote:
> > > 
> > > > Hi,
> > > > 
> > > > I seem to occasionally get some errors with remote\_file when  
> > > > running  
> > > > chef-solo. It is not consistently reproducible, but seems to  
> > > > occur when I  
> > > > rerun a chef-solo command and the target file already exists.
> > > 
> > > Hmm seems to be a platform thing. On an EC2 instance running Ubuntu  
> > > 8.04, I cannot reproduce this. On an EC2 instance running 9.04 I can  
> > > consistently reproduce this....
> > > 
> > > /Jeppe
> > 
> > And just to keep the dialogue with myself: I'm pretty sure it's  
> > caused by Ruby 1.8.7 which is default on Ubuntu 9.04. Downgrading to  
> > 1.8.6 solves the problem.
> > 
> > The strange part is that it only happens on some files. I've  
> > created a  
> > very small test scenario which consistently fails on 9.04 after the  
> > first successful run. After downgrading to Ruby 1.8.6, I can run this  
> > scenario several times without failure:
> > 
> > 1. Create solo.rb  
> > cookbook\_path "/etc/chef/recipes/cookbooks"  
> > log\_level :debug  
> > file\_store\_path "/var/chef/files/"  
> > file\_cache\_path "/etc/chef/recipes"
> > 
> > 2. gem install --no-ri --no-rdoc --bindir /sbin ohai chef --source  
> > [http://gems.opscode.com](http://gems.opscode.com) --source [http://gems.rubyforge.org](http://gems.rubyforge.org)
> > 
> > 3. chef-solo -c solo.rb -j [http://jnmtest.s3.amazonaws.com/dna.json](http://jnmtest.s3.amazonaws.com/dna.json)  
> > -l  
> > debug -r [http://jnmtest.s3.amazonaws.com/cookbooks.tgz](http://jnmtest.s3.amazonaws.com/cookbooks.tgz)
> > 
> > 4. chef-solo -c solo.rb -j [http://jnmtest.s3.amazonaws.com/dna.json](http://jnmtest.s3.amazonaws.com/dna.json)  
> > -l  
> > debug -r [http://jnmtest.s3.amazonaws.com/cookbooks.tgz](http://jnmtest.s3.amazonaws.com/cookbooks.tgz)
> > 
> > The last step always fails for the last file when running on 9.04  
> > with  
> > Ruby 1.8.7 but works with Ruby 1.8.6
> > 
> > /Jeppe

---

<div class="post-metadata">

**Author:** ![Adam\_Jacob](https://sea2.discourse-cdn.com/flex016/user_avatar/discourse.chef.io/adam_jacob/32/292_2.png) [@Adam\_Jacob](https://discourse.chef.io/u/Adam_Jacob)\
**Post date:** [June 10, 2009, 2:41pm UTC](https://discourse.chef.io/t/more-problems-with-remote-file/1195/6 "2009-06-10T14:41:37Z")

</div>

It was deliberate - I added a guard to ensure that, no matter how we  
get the filehandle, it's only closed if it's open.

Adam

On Tue, Jun 9, 2009 at 11:55 PM, Jeppe Nejsum Madsen[jeppe@ingolfs.dk](mailto:jeppe@ingolfs.dk) wrote:

> Seems to work. At least unable to reproduce with the steps below. Nice!
> 
> Was this a deliberate fix or just fixed by coincidence?
> 
> /Jeppe
> 
> On 10/06/2009, at 04.09, Adam Jacob wrote:
> 
> > Jeppe, can you try this with 0.7.0?
> > 
> > Sent from my iPhone
> > 
> > On Jun 9, 2009, at 6:11 AM, Jeppe Nejsum Madsen [jeppe@ingolfs.dk](mailto:jeppe@ingolfs.dk) wrote:
> > 
> > > On Tue, Jun 9, 2009 at 11:58 AM, Jeppe Nejsum Madsen[jeppe@ingolfs.dk](mailto:jeppe@ingolfs.dk)  
> > > wrote:
> > > 
> > > > On Mon, Jun 8, 2009 at 7:16 PM, Jeppe Nejsum Madsen[jeppe@ingolfs.dk](mailto:jeppe@ingolfs.dk)  
> > > > wrote:
> > > > 
> > > > > Hi,
> > > > > 
> > > > > I seem to occasionally get some errors with remote\_file when running  
> > > > > chef-solo. It is not consistently reproducible, but seems to occur when  
> > > > > I  
> > > > > rerun a chef-solo command and the target file already exists.
> > > > 
> > > > Hmm seems to be a platform thing. On an EC2 instance running Ubuntu  
> > > > 8.04, I cannot reproduce this. On an EC2 instance running 9.04 I can  
> > > > consistently reproduce this....
> > > > 
> > > > /Jeppe
> > > 
> > > And just to keep the dialogue with myself: I'm pretty sure it's  
> > > caused by Ruby 1.8.7 which is default on Ubuntu 9.04. Downgrading to  
> > > 1.8.6 solves the problem.
> > > 
> > > The strange part is that it only happens on some files. I've created a  
> > > very small test scenario which consistently fails on 9.04 after the  
> > > first successful run. After downgrading to Ruby 1.8.6, I can run this  
> > > scenario several times without failure:
> > > 
> > > 1. Create solo.rb  
> > > cookbook\_path "/etc/chef/recipes/cookbooks"  
> > > log\_level :debug  
> > > file\_store\_path "/var/chef/files/"  
> > > file\_cache\_path "/etc/chef/recipes"
> > > 
> > > 2. gem install --no-ri --no-rdoc --bindir /sbin ohai chef --source  
> > > [http://gems.opscode.com](http://gems.opscode.com) --source [http://gems.rubyforge.org](http://gems.rubyforge.org)
> > > 
> > > 3. chef-solo -c solo.rb -j [http://jnmtest.s3.amazonaws.com/dna.json](http://jnmtest.s3.amazonaws.com/dna.json) -l  
> > > debug -r [http://jnmtest.s3.amazonaws.com/cookbooks.tgz](http://jnmtest.s3.amazonaws.com/cookbooks.tgz)
> > > 
> > > 4. chef-solo -c solo.rb -j [http://jnmtest.s3.amazonaws.com/dna.json](http://jnmtest.s3.amazonaws.com/dna.json) -l  
> > > debug -r [http://jnmtest.s3.amazonaws.com/cookbooks.tgz](http://jnmtest.s3.amazonaws.com/cookbooks.tgz)
> > > 
> > > The last step always fails for the last file when running on 9.04 with  
> > > Ruby 1.8.7 but works with Ruby 1.8.6
> > > 
> > > /Jeppe

--  
Opscode, Inc.  
Adam Jacob, CTO  
T: (206) 508-4759 E: [adam@opscode.com](mailto:adam@opscode.com)
