Re: Re: Re: Chef 11 in production

Chef Infra (archive)
1

Thank you for the answers. I've started to use chef-server(11) in a
production, but still using chef-client 10.24.0 at the workstation and
nodes, because of memory leak problems with 11.4.0.

http://tickets.opscode.com/browse/CHEF-3432

On Tue, Apr 16, 2013 at 11:45 PM, steve . leftathome@gmail.com wrote:

We haven't formally discussed this at work, but I think our upgrade plan
will look something like this:

Once we get our mitts on the equivalent Private Chef version and install
it in testing, we'll get people to test their cookbooks off the test
endpoint. (We're looking forward to it, though -- knife-lastrun bombed out
our older Chef 10.x server through some bizarre bug that had nodes running
many node.save()s per Chef run and freaked out Solr. We're hoping this
goes away once the back-end switches to Postgres -- knife-lastrun's nice to
have, IMO.) With the new version on our test endpoint, we can run our
repos through our test framework until they all turn green again. Then
we'll know we're ready to roll it out to our thousands of production nodes..

On Tue, Apr 16, 2013 at 1:11 PM, Bryan McLellan btm@loftninjas.orgwrote:

On Tue, Apr 16, 2013 at 8:37 AM, nowarry@gmail.com wrote:

I am new in a Chef's community. Does anyone use already Chef 11.x in
production? Is it stable for production?

Yes.

Cycle Computing was an early adopter of Chef 11:
http://www.opscode.com/case-studies/opscode-case-study-cycle-computing.pdf

As well as Facebook:

http://www.opscode.com/press-releases/facebook-likes-opscode-and-private-chef/

And those were a couple of months ago. As John noted, it is in
production all over the place now.

Bryan

2

On Thursday, April 18, 2013 at 11:14 PM, Anton K wrote:

Thank you for the answers. I've started to use chef-server(11) in a production, but still using chef-client 10.24.0 at the workstation and nodes, because of memory leak problems with 11.4.0.

http://tickets.opscode.com/browse/CHEF-3432

Based on my investigation, the same bug is present in both 10.x and 11.x release tracks, but is completely worked around by running the chef-client with --fork. If you're not seeing any problem with 10.24, then you shouldn't have a problem with 11.4 either.

--
Daniel DeLeo

3

Based on my investigation, the same bug is present in both 10.x and 11.x
release tracks, but is completely worked around by running the chef-client
with --fork. If you're not seeing any problem with 10.24, then you
shouldn't have a problem with 11.4 either.

I found that chef-client 11.x in forked state generate many processes with
each call. Like this:

root 27101 0.0 1.7 107876 29384 ? Sl 10:45 0:00
/opt/chef/embedded/bin/ruby /usr/bin/chef-client -d -f -P
/var/run/chef/client.pid -c /etc/chef/client.rb -i 180 -s 20 -L
/var/log/chef/client.log

root 27858 0.0 1.7 107876 29380 ? Sl 10:45 0:00
/opt/chef/embedded/bin/ruby /usr/bin/chef-client -d -f -P
/var/run/chef/client.pid -c /etc/chef/client.rb -i 180 -s 20 -L
/var/log/chef/client.log

Chef-client 10.x does not have this problem with fork.

In the end, I've stopped at 11.2.0. This version free of memory leak(as my
investigation) without fork mode.

On Sat, Apr 20, 2013 at 1:20 AM, Daniel DeLeo dan@kallistec.com wrote:

On Thursday, April 18, 2013 at 11:14 PM, Anton K wrote:

Thank you for the answers. I've started to use chef-server(11) in a
production, but still using chef-client 10.24.0 at the workstation and
nodes, because of memory leak problems with 11.4.0.

http://tickets.opscode.com/browse/CHEF-3432

Based on my investigation, the same bug is present in both 10.x and 11.x
release tracks, but is completely worked around by running the chef-client
with --fork. If you're not seeing any problem with 10.24, then you
shouldn't have a problem with 11.4 either.

--
Daniel DeLeo

4

On Sun, Apr 21, 2013 at 9:09 PM, Anton K nowarry@gmail.com wrote:

Based on my investigation, the same bug is present in both 10.x and 11.x
release tracks, but is completely worked around by running the chef-client
with --fork. If you're not seeing any problem with 10.24, then you shouldn't
have a problem with 11.4 either.

I found that chef-client 11.x in forked state generate many processes with
each call. Like this:

We have hit the same problem. Except we have also found that --fork
was not sufficient - we ended up moving to executing it as a cron
script with a monitor that kills and retries when it gets stuck. (even
ignoring the memory problem ohai seems to get a little stuck checking
the uptodate-ness of apt packages).

--
Cheers,

Peter Donald

5

On 4/21/13 4:09 AM, Anton K wrote:

Based on my investigation, the same bug is present in both 10.x and
11.x release tracks, but is completely worked around by running the
chef-client with --fork. If you're not seeing any problem with
10.24, then you shouldn't have a problem with 11.4 either.

I found that chef-client 11.x in forked state generate many processes
with each call. Like this:
For those experiencing this, the core issue is described in the
following bug

http://tickets.opscode.com/browse/CHEF-3367

This issue is fixed on the Chef 10 branch but the fix hasn't yet been
released for Chef 11. It should be fixed in the next release.

Sincerely,

Steven