Swap consuming bug in cookbook_file, chef-solo v10.26, SmartOS

Chef Infra (archive)
1

Hi,

I have a simple resource as follows:

cookbook_file “/opt/local/share/da.json” do
source "da.json"
owner "root"
group "root"
mode "0644"
action :create_if_missing
notifies :reload, resources(:service => “apache”)
end

The source file is 8MB.
When chef needs to create this file, it suddenly consumes 750+MB of swap.

Is this a known bug? Any recommendations on a workaround other than not
using cookbook_file?

Thanks,
Yonah

2

Though this behavior is not expected , I'll suggest not to use chef as a
file server to distribute large binaries or config files . use NFS or a
vanilla lighthttpd server in to host the file and remotely_file resource
inside the chef recipes to install it instead.
For getting the root cause you can use ruby-prof with kcachegrind to check
which part of the close is actually consuming large memory

On Jun 30, 2013 6:21 AM, "Yonah Russ" opscode@yonahruss.com wrote:

Hi,

I have a simple resource as follows:

cookbook_file "/opt/local/share/da.json" do
source "da.json"
owner "root"
group "root"
mode "0644"
action :create_if_missing
notifies :reload, resources(:service => "apache")
end

The source file is 8MB.
When chef needs to create this file, it suddenly consumes 750+MB of swap.

Is this a known bug? Any recommendations on a workaround other than not
using cookbook_file?

Thanks,
Yonah

3

Hi,

Regarding the file server bit, we're using chef-solo+git specifically to
avoid setting up infrastructure (like chef or nfs or lighttpd servers) just
to manage our infrastructure.
I could easily workaround the bug by running cp source destination in a
bash/execute resource but it seems like the cookbook_file resource was
meant to be used for this :?
Is there a howto on using ruby-prof to find the problem here or chef
problems in general?

Thanks,
Yonah

On Sun, Jun 30, 2013 at 7:46 PM, Ranjib Dey dey.ranjib@gmail.com wrote:

Though this behavior is not expected , I'll suggest not to use chef as a
file server to distribute large binaries or config files . use NFS or a
vanilla lighthttpd server in to host the file and remotely_file resource
inside the chef recipes to install it instead.
For getting the root cause you can use ruby-prof with kcachegrind to check
which part of the close is actually consuming large memory

On Jun 30, 2013 6:21 AM, "Yonah Russ" opscode@yonahruss.com wrote:

Hi,

I have a simple resource as follows:

cookbook_file "/opt/local/share/da.json" do
source "da.json"
owner "root"
group "root"
mode "0644"
action :create_if_missing
notifies :reload, resources(:service => "apache")
end

The source file is 8MB.
When chef needs to create this file, it suddenly consumes 750+MB of swap.

Is this a known bug? Any recommendations on a workaround other than not
using cookbook_file?

Thanks,
Yonah

4

check if this helps

regards
ranjib

On Sun, Jun 30, 2013 at 11:52 AM, Yonah Russ opscode@yonahruss.com wrote:

Hi,

Regarding the file server bit, we're using chef-solo+git specifically to
avoid setting up infrastructure (like chef or nfs or lighttpd servers) just
to manage our infrastructure.
I could easily workaround the bug by running cp source destination in a
bash/execute resource but it seems like the cookbook_file resource was
meant to be used for this :?
Is there a howto on using ruby-prof to find the problem here or chef
problems in general?

Thanks,
Yonah

On Sun, Jun 30, 2013 at 7:46 PM, Ranjib Dey dey.ranjib@gmail.com wrote:

Though this behavior is not expected , I'll suggest not to use chef as a
file server to distribute large binaries or config files . use NFS or a
vanilla lighthttpd server in to host the file and remotely_file resource
inside the chef recipes to install it instead.
For getting the root cause you can use ruby-prof with kcachegrind to
check which part of the close is actually consuming large memory

On Jun 30, 2013 6:21 AM, "Yonah Russ" opscode@yonahruss.com wrote:

Hi,

I have a simple resource as follows:

cookbook_file "/opt/local/share/da.json" do
source "da.json"
owner "root"
group "root"
mode "0644"
action :create_if_missing
notifies :reload, resources(:service => "apache")
end

The source file is 8MB.
When chef needs to create this file, it suddenly consumes 750+MB of
swap.

Is this a known bug? Any recommendations on a workaround other than not
using cookbook_file?

Thanks,
Yonah