Hi,
I’m having a bit of a hard time here.
Any knife command I issue takes approximately 15s to run. I’ve ruled out network issues by running knife by itself (http://pastie.org/8240023).
If I ‘top’ during the knife execution I can see a ruby process taking up a whole CPU core.
How can I profile that to find out what’s taking so long to process?
Cassiano Leal
http://cassianoleal.com
http://twitter.com/cassianoleal
I have run into this before. Was due to many gems installed.
Does this help?
chef - [chef] Re: Re: debugging slow knife?
On Thursday, August 15, 2013 at 1:01 PM, Cassiano Leal wrote:
Hi,
I’m having a bit of a hard time here.
Any knife command I issue takes approximately 15s to run. I’ve ruled out network issues by running knife by itself (http://pastie.org/8240023).
If I ‘top’ during the knife execution I can see a ruby process taking up a whole CPU core.
How can I profile that to find out what’s taking so long to process?
Cheers,
--Cassiano Leal
http://cassianoleal.com
http://twitter.com/cassianoleal
It totally did!
After cleaning up, time went down to 0.54s. After installing chefspec, this time doubled. Still acceptable, but I wouldn’t expect that this gem would increase the time. knife-ec2 increases about .02s to the run time.
Thanks for the help, this makes things much more tolerable! 
On August 15, 2013 at 17:10:57, John Dewey (john@dewey.ws) wrote:
I have run into this before. Was due to many gems installed.
Does this help?
http://lists.opscode.com/sympa/arc/chef/2011-05/msg00041.html
On Thursday, August 15, 2013 at 1:01 PM, Cassiano Leal wrote:
Hi,
I’m having a bit of a hard time here.
Any knife command I issue takes approximately 15s to run. I’ve ruled out network issues by running knife by itself (http://pastie.org/8240023).
If I ‘top’ during the knife execution I can see a ruby process taking up a whole CPU core.
How can I profile that to find out what’s taking so long to process?
Cassiano Leal
http://cassianoleal.com
http://twitter.com/cassianoleal
–
Cassiano Leal
http://cassianoleal.com
http://twitter.com/cassianoleal
Try using an up-to-date rvm that has bundle exec integration (rvm >=
1.1.0) or use rbenv with the rbenv-bundler installed and then setup a
Gemfile in your chef-repo and bundle install it. Run your knife
commands under that bundle and there's no longer any need to bundle
install --binstubs or bundle exec every command. With ruby-1.9.3-p429 or
ruby-2.0 run with that bundle it should be fairly fast.
Note, if you go down this path that you'll wind up wanting to keep
recent versions of chef, berkshelf and other command line tools in your
bundle which gets interesting when they have common deps like the json
gem. Right now there may be a bug in the bundler depsolving code where
it somehow solves to berkshelf pinning json to >= 1.8.0 where all I can
see is the ridley restriction of json >= 1.7.7 which is still compatible
with chef's json <= 1.7.7 restriction. If you let the berkshelf version
float you wind up with an old version of it, if you try to pin berkshelf
to ~> 2.0.8 and chef to 11.6.0 then it blows up. But if you also pin
the json gem to 1.7.7 explicitly then its perfectly happy. I've been
using rvm with ruby-2.0.0-p247 and this Gemfile:
% cat Gemfile
source 'https://rubygems.org'
ruby '2.0.0'
gem 'rake'
gem 'thor'
gem 'chef', '11.6.0'
gem 'json', '1.7.7'
gem 'chefspec'
gem 'foodcritic'
gem 'knife-spork'
gem 'berkshelf', '~> 2.0.8'
gem 'knife-ec2'
gem 'knife-hp'
Going down this road speeds up knife and is convenient. You can check
in your Gemfile.lock and get your whole team using the same versions of
your commandline toolchain, which is cool. But, you will probably get
intimately familiar with transitive deps between all these tools and the
json gem (which probably makes for a really good argument to check in
specific versions of Gemfile.lock that are found to work and bump
versions for the team only when the universe is aligned...)
Even if you don't go the whole way with setting up a Gemfile to limit
the number of gems that knife needs to search for plugins,
ruby-1.9.3-p429 or ruby-2.0 is pretty mandatory to speed up knife.
Unless you're using a very recent distro you will not have a recent
enough ruby in /usr/bin/ruby and must use either rvm or rbenv.
(Also note that ruby 2.0 requires chef-11.6 and may still have the odd bug)
On 8/15/13 1:10 PM, John Dewey wrote:
I have run into this before. Was due to many gems installed.
Does this help?
chef - [chef] Re: Re: debugging slow knife?On Thursday, August 15, 2013 at 1:01 PM, Cassiano Leal wrote:
Hi,
I’m having a bit of a hard time here.
Any knife command I issue takes approximately 15s to run. I’ve ruled
out network issues by running knife by itself
(http://pastie.org/8240023).If I ‘top’ during the knife execution I can see a ruby process taking
up a whole CPU core.How can I profile that to find out what’s taking so long to process?
Cheers,
Cassiano Leal
http://cassianoleal.com
http://twitter.com/cassianoleal
On Thu, Aug 15, 2013 at 11:35 PM, Cassiano Leal cassianoleal@gmail.com wrote:
After cleaning up, time went down to 0.54s. After installing chefspec, this
time doubled. Still acceptable, but I wouldn’t expect that this gem would
increase the time. knife-ec2 increases about .02s to the run time.
I'm quite sure the slowness is because of knife subcommand loading. It
crawl through files in all gems while trying to find knife commands.
Using Bundler to isolate loaded gems should indeed help.
--
Cheers,
Thanks for the tips. I don’t use a chef-repo per se, and I issue knife command from anywhere so the Gemfile approach is not an option.
I also don’t use RVM or rbenv (I ditched both in favour of chruby, which is much leaner), and I had been using 1.9.3-p429 all along. Tried with 2.0 and had similar results (~1s with a clean gem environment).
I’m happy for now. Knife’s slowness (when it happens again) will serve as a reminder to clean up gems that are only occupying disk space and taking away my time and patience.
On August 15, 2013 at 17:44:44, Lamont Granquist (lamont@opscode.com) wrote:
Try using an up-to-date rvm that has bundle exec integration (rvm >= 1.1.0) or use rbenv with the rbenv-bundler installed and then setup a Gemfile in your chef-repo and bundle install it. Run your knife commands under that bundle and there’s no longer any need to bundle install --binstubs or bundle exec every command. With ruby-1.9.3-p429 or ruby-2.0 run with that bundle it should be fairly fast.
Note, if you go down this path that you’ll wind up wanting to keep recent versions of chef, berkshelf and other command line tools in your bundle which gets interesting when they have common deps like the json gem. Right now there may be a bug in the bundler depsolving code where it somehow solves to berkshelf pinning json to >= 1.8.0 where all I can see is the ridley restriction of json >= 1.7.7 which is still compatible with chef’s json <= 1.7.7 restriction. If you let the berkshelf version float you wind up with an old version of it, if you try to pin berkshelf to ~> 2.0.8 and chef to 11.6.0 then it blows up. But if you also pin the json gem to 1.7.7 explicitly then its perfectly happy. I’ve been using rvm with ruby-2.0.0-p247 and this Gemfile:
% cat Gemfile
source ‘https://rubygems.org’
ruby ‘2.0.0’
gem 'rake’
gem 'thor’
gem ‘chef’, '11.6.0’
gem ‘json’, '1.7.7’
gem 'chefspec’
gem 'foodcritic’
gem 'knife-spork’
gem ‘berkshelf’, '~> 2.0.8’
gem 'knife-ec2’
gem ‘knife-hp’
Going down this road speeds up knife and is convenient. You can check in your Gemfile.lock and get your whole team using the same versions of your commandline toolchain, which is cool. But, you will probably get intimately familiar with transitive deps between all these tools and the json gem (which probably makes for a really good argument to check in specific versions of Gemfile.lock that are found to work and bump versions for the team only when the universe is aligned…)
Even if you don’t go the whole way with setting up a Gemfile to limit the number of gems that knife needs to search for plugins, ruby-1.9.3-p429 or ruby-2.0 is pretty mandatory to speed up knife. Unless you’re using a very recent distro you will not have a recent enough ruby in /usr/bin/ruby and must use either rvm or rbenv.
(Also note that ruby 2.0 requires chef-11.6 and may still have the odd bug)
On 8/15/13 1:10 PM, John Dewey wrote:
I have run into this before. Was due to many gems installed.
Does this help?
http://lists.opscode.com/sympa/arc/chef/2011-05/msg00041.html
On Thursday, August 15, 2013 at 1:01 PM, Cassiano Leal wrote:
Hi,
I’m having a bit of a hard time here.
Any knife command I issue takes approximately 15s to run. I’ve ruled out network issues by running knife by itself (http://pastie.org/8240023).
If I ‘top’ during the knife execution I can see a ruby process taking up a whole CPU core.
How can I profile that to find out what’s taking so long to process?
Cassiano Leal
http://cassianoleal.com
http://twitter.com/cassianoleal
–
Cassiano Leal
http://cassianoleal.com
http://twitter.com/cassianoleal
On August 16, 2013 at 09:35:24, Teemu Matilainen (teemu.matilainen@iki.fi) wrote:
On Thu, Aug 15, 2013 at 11:35 PM, Cassiano Leal cassianoleal@gmail.com wrote:
After cleaning up, time went down to 0.54s. After installing chefspec, this
time doubled. Still acceptable, but I wouldn’t expect that this gem would
increase the time. knife-ec2 increases about .02s to the run time.
I'm quite sure the slowness is because of knife subcommand loading. It
crawl through files in all gems while trying to find knife commands.
Doesn’t sound like the optimal way of dealing with gems. Maybe an actual plugins system would be better? Or some other way of letting knife know about installed commands?
Using Bundler to isolate loaded gems should indeed help.
--
Cheers,
--
Cassiano Leal
http://cassianoleal.com
http://twitter.com/cassianoleal
On Friday, August 16, 2013 at 5:48 AM, Cassiano Leal wrote:
On August 16, 2013 at 09:35:24, Teemu Matilainen (teemu.matilainen@iki.fi (mailto:teemu.matilainen@iki.fi)) wrote:
On Thu, Aug 15, 2013 at 11:35 PM, Cassiano Leal <cassianoleal@gmail.com (mailto:cassianoleal@gmail.com)> wrote:
After cleaning up, time went down to 0.54s. After installing chefspec, this
time doubled. Still acceptable, but I wouldn’t expect that this gem would
increase the time. knife-ec2 increases about .02s to the run time.I'm quite sure the slowness is because of knife subcommand loading. It
crawl through files in all gems while trying to find knife commands.
Doesn’t sound like the optimal way of dealing with gems. Maybe an actual plugins system would be better? Or some other way of letting knife know about installed commands?
This is something we're considering changing, especially since it's confusing for omnibus users to have to use /opt/chef/embedded/bin/gem. Substantive changes would have to wait for a major release, though.
--
Daniel DeLeo
On August 19, 2013 at 11:58:59, Daniel DeLeo (dan@kallistec.com) wrote:
On Friday, August 16, 2013 at 5:48 AM, Cassiano Leal wrote:
On August 16, 2013 at 09:35:24, Teemu Matilainen (teemu.matilainen@iki.fi) wrote:
On Thu, Aug 15, 2013 at 11:35 PM, Cassiano Leal cassianoleal@gmail.com wrote:
After cleaning up, time went down to 0.54s. After installing chefspec, this
time doubled. Still acceptable, but I wouldn’t expect that this gem would
increase the time. knife-ec2 increases about .02s to the run time.
I'm quite sure the slowness is because of knife subcommand loading. It
crawl through files in all gems while trying to find knife commands.
Doesn’t sound like the optimal way of dealing with gems. Maybe an actual plugins system would be better? Or some other way of letting knife know about installed commands?
This is something we're considering changing, especially since it's confusing for omnibus users to have to use /opt/chef/embedded/bin/gem. Substantive changes would have to wait for a major release, though.
That’s understandable, and I wouldn’t expect otherwise.
I’m good for the moment as knife is taking ~2s for anything (perfectly acceptable and much better than the previous ~16s!).
Cassiano Leal
http://cassianoleal.com
http://twitter.com/cassianoleal
On 8/19/13 11:28 AM, Cassiano Leal wrote:
I don’t use omnibus on my laptop, as I prefer to manage all my rubies
with a single version manager, so installing Chef as a gem is saner. I
also don’t like to use either gemsets or bundles to manage
command-line tools, since I usually want to run them from anywhere in
the shell without having to change context first.
you can install a knife-specific gemset under rvm and then alias knife
to rvm exec it under that gemset. not sure if you can do that with
rbenv or other tools.