Hi all,
since some days my opensource-chef-server is acting up. Most hosts work
just fine, for some hosts erchef takes ~1000(!) seconds to process the
dependencies. And eats all the cpu. Thankfully this doesn’t block other
clients from getting their data. But some hosts haven’t been able to get
the data for a successfull chef run for some days now. It seems to be
some of my cookbooks, alone or in combination. But I am currently
lacking data to debug this.
Is there a way to make erchef output the dep-cycles its calculating per
request? Or is there a way to query dependencies on chef-server by hand?
I tried setting “erchef[‘log_level’] = :debug” in
/etc/chef-server/chef-server.rb followed by “chef-server-ctl
reconfigure” but that resulted only in one info-level logline per
request.
Thanks in advance,
Arnold
This is a known issue in the Open Source Server’s dependency resolver. This will be fixed in 11.1, which will be released soon. We’re expecting an RC to go out in a day or so and full release next week.
--
Daniel DeLeo
On Thursday, May 8, 2014 at 1:40 AM, Arnold Krille wrote:
Hi all,
since some days my opensource-chef-server is acting up. Most hosts work
just fine, for some hosts erchef takes ~1000(!) seconds to process the
dependencies. And eats all the cpu. Thankfully this doesn't block other
clients from getting their data. But some hosts haven't been able to get
the data for a successfull chef run for some days now. It seems to be
some of my cookbooks, alone or in combination. But I am currently
lacking data to debug this.
Is there a way to make erchef output the dep-cycles its calculating per
request? Or is there a way to query dependencies on chef-server by hand?
I tried setting "erchef['log_level'] = :debug" in
/etc/chef-server/chef-server.rb followed by "chef-server-ctl
reconfigure" but that resulted only in one info-level logline per
request.
Thanks in advance,
Arnold
You can use GitHub - coderanger/knife-solve: A knife plugin to see the Chef Server version solver solution for a given set of cookbooks. to debug these kinds of issues. Just recursively narrow the set of cookbooks you pass it until you find the minimal set that fails to solve (in a reasonable amount of time anyway). Often by then the error in the constraints is pretty clear.
--Noah
On May 8, 2014, at 1:40 AM, Arnold Krille arnold@arnoldarts.de wrote:
Hi all,
since some days my opensource-chef-server is acting up. Most hosts work just fine, for some hosts erchef takes ~1000(!) seconds to process the dependencies. And eats all the cpu. Thankfully this doesn't block other clients from getting their data. But some hosts haven't been able to get the data for a successfull chef run for some days now. It seems to be some of my cookbooks, alone or in combination. But I am currently lacking data to debug this.
Is there a way to make erchef output the dep-cycles its calculating per request? Or is there a way to query dependencies on chef-server by hand?
I tried setting "erchef['log_level'] = :debug" in /etc/chef-server/chef-server.rb followed by "chef-server-ctl reconfigure" but that resulted only in one info-level logline per request.
Thanks in advance,
Arnold
Hi,
thanks for that tip. Will look at it and hopefully remember this the
next time this issue arises.
For now I 'fixed'/circumvented the problem by updating the requirements
of our own cookbooks and uploading them, then deleting lots of older
version from the server. That made chef-server not hang on certain
nodes/recipes. Then I only had to adopt the version-constraints of the
environments to the latest versions...
Have fun,
Am Thu, 8 May 2014 14:22:09 -0700 schrieb Noah Kantrowitz
noah@coderanger.net:
You can use GitHub - coderanger/knife-solve: A knife plugin to see the Chef Server version solver solution for a given set of cookbooks. to debug these
kinds of issues. Just recursively narrow the set of cookbooks you
pass it until you find the minimal set that fails to solve (in a
reasonable amount of time anyway). Often by then the error in the
constraints is pretty clear.
--Noah
On May 8, 2014, at 1:40 AM, Arnold Krille arnold@arnoldarts.de
wrote:
Hi all,
since some days my opensource-chef-server is acting up. Most hosts
work just fine, for some hosts erchef takes ~1000(!) seconds to
process the dependencies. And eats all the cpu. Thankfully this
doesn't block other clients from getting their data. But some hosts
haven't been able to get the data for a successfull chef run for
some days now. It seems to be some of my cookbooks, alone or in
combination. But I am currently lacking data to debug this.
Is there a way to make erchef output the dep-cycles its calculating
per request? Or is there a way to query dependencies on chef-server
by hand? I tried setting "erchef['log_level'] = :debug"
in /etc/chef-server/chef-server.rb followed by "chef-server-ctl
reconfigure" but that resulted only in one info-level logline per
request.
Thanks in advance,
Arnold
Hi,
yeah, I have seen the ticket. Add me to the list of users waiting for
the release;-)
See my other answer to Noah how I solved my problem.
Have fun,
Am Thu, 8 May 2014 14:19:44 -0700
schrieb Daniel DeLeo dan@kallistec.com:
This is a known issue in the Open Source Server’s dependency
resolver. This will be fixed in 11.1, which will be released soon.
We’re expecting an RC to go out in a day or so and full release next
week.
--
Daniel DeLeo
On Thursday, May 8, 2014 at 1:40 AM, Arnold Krille wrote:
Hi all,
since some days my opensource-chef-server is acting up. Most hosts
work just fine, for some hosts erchef takes ~1000(!) seconds to
process the dependencies. And eats all the cpu. Thankfully this
doesn't block other clients from getting their data. But some hosts
haven't been able to get the data for a successfull chef run for
some days now. It seems to be some of my cookbooks, alone or in
combination. But I am currently lacking data to debug this.
Is there a way to make erchef output the dep-cycles its calculating
per request? Or is there a way to query dependencies on chef-server
by hand? I tried setting "erchef['log_level'] = :debug" in
/etc/chef-server/chef-server.rb followed by "chef-server-ctl
reconfigure" but that resulted only in one info-level logline per
request.
Thanks in advance,
Arnold
Catching up on mailing list content, just wanted to comment that you can
assist the depsolver by making sure that
none of your cookbooks 'depend' on themselves (and you don't want to
have larger loops setup anywhere either, but
those might be harder to track down if you've got a big mess of cookbooks).
Either knife upload and/or the chef server should arguably error or
automatically trim such self-dependencies out of
the cookbook metadata before trying to feed it to the resolver, but
AFAIK it still does not. There should probably be
a foodcritic rule added to check the name of the cookbook against any of
the depends lines and complain if there's
a match.
On 5/9/14, 1:36 AM, Arnold Krille wrote:
Hi,
thanks for that tip. Will look at it and hopefully remember this the
next time this issue arises.
For now I 'fixed'/circumvented the problem by updating the requirements
of our own cookbooks and uploading them, then deleting lots of older
version from the server. That made chef-server not hang on certain
nodes/recipes. Then I only had to adopt the version-constraints of the
environments to the latest versions...
Have fun,
Am Thu, 8 May 2014 14:22:09 -0700 schrieb Noah Kantrowitz
noah@coderanger.net:
You can use GitHub - coderanger/knife-solve: A knife plugin to see the Chef Server version solver solution for a given set of cookbooks. to debug these
kinds of issues. Just recursively narrow the set of cookbooks you
pass it until you find the minimal set that fails to solve (in a
reasonable amount of time anyway). Often by then the error in the
constraints is pretty clear.
--Noah
On May 8, 2014, at 1:40 AM, Arnold Krille arnold@arnoldarts.de
wrote:
Hi all,
since some days my opensource-chef-server is acting up. Most hosts
work just fine, for some hosts erchef takes ~1000(!) seconds to
process the dependencies. And eats all the cpu. Thankfully this
doesn't block other clients from getting their data. But some hosts
haven't been able to get the data for a successfull chef run for
some days now. It seems to be some of my cookbooks, alone or in
combination. But I am currently lacking data to debug this.
Is there a way to make erchef output the dep-cycles its calculating
per request? Or is there a way to query dependencies on chef-server
by hand? I tried setting "erchef['log_level'] = :debug"
in /etc/chef-server/chef-server.rb followed by "chef-server-ctl
reconfigure" but that resulted only in one info-level logline per
request.
Thanks in advance,
Arnold