Resource consumption on chef-solo


#1

Hey Chef Lists,

I am running chef-solo on the standard vagrant lucid32 box. My host OSes are
either Mac Lion or Ubuntu (10.04 32 bit). I’m running into a serious resource
consumption error and I’m wondering if you’ve seen this before.

Several of the larger packages in my setup (build-essentials, python-dev, a few
others) seem to freeze my computer. I got bored after watching both CPUs pegged
and RAM full for 9 minutes. I’ve tested the actual apt instruction on the VM
(i.e. apt-get install -q -y python-dev) and it runs very quickly. I’ve also run
the exact same cookbooks on a VM on the Ubuntu host and found no problems.
Therefore, it appears to be a problem that only manifests on some machines
running chef-solo in a virtual machine.

Any advice?
Thanks,
Tommy Guy
Wave Accounting


#2

This is probably what you are experiencing:

http://help.opscode.com/discussions/problems/1174-memory-usage-goes-crazy-during-package-checking-package-status-apt
http://tickets.opscode.com/browse/CHEF-2947

The patch on the ticket fixed the issue for me :slight_smile:

We experienced this on physical machines (lucid 64bit), it is a bit of a race condition by the looks.

Cheers,
Dave.

-----Original Message-----
From: tommy@waveaccounting.com [mailto:tommy@waveaccounting.com]
Sent: Thursday, 8 March 2012 9:41 a.m.
To: chef@lists.opscode.com
Subject: [chef] Resource consumption on chef-solo

Hey Chef Lists,

I am running chef-solo on the standard vagrant lucid32 box. My host OSes are either Mac Lion or Ubuntu (10.04 32 bit). I’m running into a serious resource consumption error and I’m wondering if you’ve seen this before.

Several of the larger packages in my setup (build-essentials, python-dev, a few
others) seem to freeze my computer. I got bored after watching both CPUs pegged and RAM full for 9 minutes. I’ve tested the actual apt instruction on the VM (i.e. apt-get install -q -y python-dev) and it runs very quickly. I’ve also run the exact same cookbooks on a VM on the Ubuntu host and found no problems.
Therefore, it appears to be a problem that only manifests on some machines running chef-solo in a virtual machine.

Any advice?
Thanks,
Tommy Guy
Wave Accounting


#3

Hey Dave,

Thanks for the help. I installed chef from github including all of the
following packages:
chef mixlib-authentication mixlib-cli mixlib-config mixlib-log
mixlib-shellout ohai

The patch you send appears to be in place. I built the shellout gem using
gem build and installed it using gem install. The problem is still there.

While I don’t appear to have a memory problem, both CPUs in my Mac are
pegged at 100% of system time. Are there any similar places that could
force the same problem? Are there any logs that would help?

Thanks,
Tommy

On Wed, Mar 7, 2012 at 5:48 PM, David Leaver dave@smartrak.co.nz wrote:

This is probably what you are experiencing:

http://help.opscode.com/discussions/problems/1174-memory-usage-goes-crazy-during-package-checking-package-status-apt
http://tickets.opscode.com/browse/CHEF-2947

The patch on the ticket fixed the issue for me :slight_smile:

We experienced this on physical machines (lucid 64bit), it is a bit of a
race condition by the looks.

Cheers,
Dave.

-----Original Message-----
From: tommy@waveaccounting.com [mailto:tommy@waveaccounting.com]
Sent: Thursday, 8 March 2012 9:41 a.m.
To: chef@lists.opscode.com
Subject: [chef] Resource consumption on chef-solo

Hey Chef Lists,

I am running chef-solo on the standard vagrant lucid32 box. My host OSes
are either Mac Lion or Ubuntu (10.04 32 bit). I’m running into a serious
resource consumption error and I’m wondering if you’ve seen this before.

Several of the larger packages in my setup (build-essentials, python-dev,
a few
others) seem to freeze my computer. I got bored after watching both CPUs
pegged and RAM full for 9 minutes. I’ve tested the actual apt instruction
on the VM (i.e. apt-get install -q -y python-dev) and it runs very quickly.
I’ve also run the exact same cookbooks on a VM on the Ubuntu host and found
no problems.
Therefore, it appears to be a problem that only manifests on some machines
running chef-solo in a virtual machine.

Any advice?
Thanks,
Tommy Guy
Wave Accounting


#4

Update on the issue where chef-solo hung on some recipes. It looks like it
might limited to the lucid release on Ubunutu. I just ran on natty with no
problems, even using chef-solo 0.10.0.

Thanks again for the help.

Tommy

On Thu, Mar 8, 2012 at 12:24 PM, Tommy Guy tommy@waveaccounting.com wrote:

Hey Dave,

Thanks for the help. I installed chef from github including all of the
following packages:
chef mixlib-authentication mixlib-cli mixlib-config mixlib-log
mixlib-shellout ohai

The patch you send appears to be in place. I built the shellout gem using
gem build and installed it using gem install. The problem is still there.

While I don’t appear to have a memory problem, both CPUs in my Mac are
pegged at 100% of system time. Are there any similar places that could
force the same problem? Are there any logs that would help?

Thanks,
Tommy

On Wed, Mar 7, 2012 at 5:48 PM, David Leaver dave@smartrak.co.nz wrote:

This is probably what you are experiencing:

http://help.opscode.com/discussions/problems/1174-memory-usage-goes-crazy-during-package-checking-package-status-apt
http://tickets.opscode.com/browse/CHEF-2947

The patch on the ticket fixed the issue for me :slight_smile:

We experienced this on physical machines (lucid 64bit), it is a bit of a
race condition by the looks.

Cheers,
Dave.

-----Original Message-----
From: tommy@waveaccounting.com [mailto:tommy@waveaccounting.com]
Sent: Thursday, 8 March 2012 9:41 a.m.
To: chef@lists.opscode.com
Subject: [chef] Resource consumption on chef-solo

Hey Chef Lists,

I am running chef-solo on the standard vagrant lucid32 box. My host OSes
are either Mac Lion or Ubuntu (10.04 32 bit). I’m running into a serious
resource consumption error and I’m wondering if you’ve seen this before.

Several of the larger packages in my setup (build-essentials, python-dev,
a few
others) seem to freeze my computer. I got bored after watching both CPUs
pegged and RAM full for 9 minutes. I’ve tested the actual apt instruction
on the VM (i.e. apt-get install -q -y python-dev) and it runs very quickly.
I’ve also run the exact same cookbooks on a VM on the Ubuntu host and found
no problems.
Therefore, it appears to be a problem that only manifests on some
machines running chef-solo in a virtual machine.

Any advice?
Thanks,
Tommy Guy
Wave Accounting