Issue with easy_install_package provider

Hi,

First my scenario:

  • chef server 0.9.4
  • chef-client installed on remote node via gem install using rubygems-1.3.7.
  • remote node running Ubuntu Lucid 64-bit.

I tried to use the easy_install_package provider to install some
Python packages. It works most of the time, but it fails for packages
whose main module name (in terms of doing ‘import pkgname’) is
different from the name of the package.

Case in point is dnspython. If you run ‘easy_install dnspython’, you
need to then do ‘import dns’ in your code, and not ‘import dnspython’.
However, the provider tries to verify the existence of the package by
trying ‘import dnspython’ and it gets an exception which stops the
entire chef-client run.

Here is the entire traceback from chef.log:

ERROR: Running exception handlers
ERROR: Exception handlers complete
ERROR: Chef::Exceptions::ShellCommandFailed:Expected process to exit
0, but it exited with 1
---- Begin output of python -c “import dnspython; print dnspython.path” ----
STDOUT:
STDERR: Traceback (most recent call last):
File “”, line 1, in
ImportError: No module named dnspython
---- End output of python -c “import dnspython; print dnspython.path” ----
Ran python -c “import dnspython; print dnspython.path” returned 1
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/…/lib/chef/shell_out.rb:232:in
invalid!' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/shell_out.rb:218:inerror!’
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/…/lib/chef/mixin/shell_out.rb:33:in
shell_out!' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/provider/package/easy_install.rb:69:inload_current_resource’
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/…/lib/chef/resource.rb:394:in
run_action' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/runner.rb:53:inrun_action’
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/…/lib/chef/runner.rb:89:in
converge' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/runner.rb:89:ineach’
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/…/lib/chef/runner.rb:89:in
converge' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/resource_collection.rb:94 /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:incall’
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/…/lib/chef/resource_collection/stepable_iterator.rb:116:in
call_iterator_block' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/resource_collection/stepable_iterator.rb:85:instep’
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/…/lib/chef/resource_collection/stepable_iterator.rb:104:in
iterate' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/resource_collection/stepable_iterator.rb:55:ineach_with_index’
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/…/lib/chef/resource_collection.rb:92:in
execute_each_resource' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/runner.rb:84:inconverge’
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/…/lib/chef/client.rb:268:in
converge' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/client.rb:171:inrun’
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/…/lib/chef/application/client.rb:222:in
run_application' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/application/client.rb:212:inloop’
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/…/lib/chef/application/client.rb:212:in
run_application' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/application.rb:62:inrun’
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/chef-client:26
/usr/bin/chef-client:19:in `load’
/usr/bin/chef-client:19

Grig,

I happened to write that provider. I found that it is difficult to detect what python eggs are installed without attempting an import or scanning through sys.path. Currently the provider should do it in that order but it looks like it's exiting rather than getting rescued and trying to find it in the sys.path. I'll work up a patch and get back to you.

-Joe

On Nov 22, 2010, at 5:05 PM, Grig Gheorghiu wrote:

Hi,

First my scenario:

  • chef server 0.9.4
  • chef-client installed on remote node via gem install using rubygems-1.3.7.
  • remote node running Ubuntu Lucid 64-bit.

I tried to use the easy_install_package provider to install some
Python packages. It works most of the time, but it fails for packages
whose main module name (in terms of doing 'import pkgname') is
different from the name of the package.

Case in point is dnspython. If you run 'easy_install dnspython', you
need to then do 'import dns' in your code, and not 'import dnspython'.
However, the provider tries to verify the existence of the package by
trying 'import dnspython' and it gets an exception which stops the
entire chef-client run.

Here is the entire traceback from chef.log:

ERROR: Running exception handlers
ERROR: Exception handlers complete
ERROR: Chef::Exceptions::ShellCommandFailed:Expected process to exit
0, but it exited with 1
---- Begin output of python -c "import dnspython; print dnspython.path" ----
STDOUT:
STDERR: Traceback (most recent call last):
File "", line 1, in
ImportError: No module named dnspython
---- End output of python -c "import dnspython; print dnspython.path" ----
Ran python -c "import dnspython; print dnspython.path" returned 1
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/shell_out.rb:232:in
invalid!' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/shell_out.rb:218:in error!'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/mixin/shell_out.rb:33:in
shell_out!' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/provider/package/easy_install.rb:69:in load_current_resource'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/resource.rb:394:in
run_action' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/runner.rb:53:in run_action'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/runner.rb:89:in
converge' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/runner.rb:89:in each'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/runner.rb:89:in
converge' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/resource_collection.rb:94 /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in call'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in
call_iterator_block' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/resource_collection/stepable_iterator.rb:85:in step'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/resource_collection/stepable_iterator.rb:104:in
iterate' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/resource_collection/stepable_iterator.rb:55:in each_with_index'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/resource_collection.rb:92:in
execute_each_resource' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/runner.rb:84:in converge'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/client.rb:268:in
converge' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/client.rb:171:in run'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/application/client.rb:222:in
run_application' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/application/client.rb:212:in loop'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/application/client.rb:212:in
run_application' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/application.rb:62:in run'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/chef-client:26
/usr/bin/chef-client:19:in `load'
/usr/bin/chef-client:19

Name: Joseph A. Williams
Email: joe@joetify.com
Blog: http://www.joeandmotorboat.com/
Twitter: http://twitter.com/williamsjoe

On Mon, Nov 22, 2010 at 8:45 PM, Joe Williams joe@joetify.com wrote:

Grig,

I happened to write that provider. I found that it is difficult to detect what python eggs are installed without attempting an import or scanning through sys.path. Currently the provider should do it in that order but it looks like it's exiting rather than getting rescued and trying to find it in the sys.path. I'll work up a patch and get back to you.

-Joe

Hi Joe

Thanks for the quick reply. You're right, the original package name
(dnspython) should have been found by inspecting sys.path. Not sure
why it wasn't found. Here's what I get when I run this line (from
https://github.com/opscode/chef/blob/master/chef/lib/chef/provider/package/easy_install.rb)
on my target node:

python -c "import sys; print sys.path"

['', '/usr/local/lib/python2.6/dist-packages/Jinja2-2.5.5-py2.6.egg',
'/usr/local/lib/python2.6/dist-packages/isodate-0.4.3-py2.6.egg',
'/usr/local/lib/python2.6/dist-packages/eventlet-0.9.13-py2.6.egg',
'/usr/local/lib/python2.6/dist-packages/greenlet-0.3.1-py2.6-linux-x86_64.egg',
'/usr/local/lib/python2.6/dist-packages/DBUtils-1.0-py2.6.egg',
'/usr/local/lib/python2.6/dist-packages/boto-2.0b3-py2.6.egg',
'/usr/local/lib/python2.6/dist-packages/amqplib-0.6.1-py2.6.egg',
'/usr/local/lib/python2.6/dist-packages/dnspython-1.9.1-py2.6.egg',
'/usr/local/lib/python2.6/dist-packages/pydkim-0.3-py2.6.egg',
'/usr/local/lib/python2.6/dist-packages/Django-1.2.3-py2.6.egg',
'/usr/lib/python2.6', '/usr/lib/python2.6/plat-linux2',
'/usr/lib/python2.6/lib-tk', '/usr/lib/python2.6/lib-old',
'/usr/lib/python2.6/lib-dynload', '/usr/lib/python2.6/dist-packages',
'/usr/lib/python2.6/dist-packages/PIL',
'/usr/lib/pymodules/python2.6',
'/usr/lib/pymodules/python2.6/gtk-2.0',
'/usr/local/lib/python2.6/dist-packages']

Grig

On Nov 22, 2010, at 9:01 PM, Grig Gheorghiu wrote:

On Mon, Nov 22, 2010 at 8:45 PM, Joe Williams joe@joetify.com wrote:

Grig,

I happened to write that provider. I found that it is difficult to detect what python eggs are installed without attempting an import or scanning through sys.path. Currently the provider should do it in that order but it looks like it's exiting rather than getting rescued and trying to find it in the sys.path. I'll work up a patch and get back to you.

-Joe

Hi Joe

Thanks for the quick reply. You're right, the original package name
(dnspython) should have been found by inspecting sys.path. Not sure
why it wasn't found. Here's what I get when I run this line (from
https://github.com/opscode/chef/blob/master/chef/lib/chef/provider/package/easy_install.rb)
on my target node:

Try this patch out (works for me on a handful of packages):

http://friendpaste.com/4IpYnXoV83AAKd1KxT5TO3

If it works for you I'll create a bug ticket and get it submitted for the next release.

-Joe

Name: Joseph A. Williams
Email: joe@joetify.com
Blog: http://www.joeandmotorboat.com/
Twitter: http://twitter.com/williamsjoe

On Mon, Nov 22, 2010 at 9:28 PM, Joe Williams joe@joetify.com wrote:

Try this patch out (works for me on a handful of packages):
Friendpaste - Paste #4IpYnXoV83AAKd1KxT5TO3
If it works for you I'll create a bug ticket and get it submitted for the
next release.

Hmm..I applied the patch and it seems to have worked fine when the
dnspython package wasn't installed -- it went ahead and installed it.
However, the next time chef-client ran, the installation check failed
with:

ERROR: easy_install_package[dnspython]
(/srv/chef/cache/cookbooks/evite/recipes/default.rb:67:in from_file') had an error: Expected process to exit 0, but it exited with 1 ---- Begin output of python -c "import dnspython; print dnspython.__path__" ---- STDOUT: STDERR: Traceback (most recent call last): File "<string>", line 1, in <module> ImportError: No module named dnspython ---- End output of python -c "import dnspython; print dnspython.__path__" ---- Ran python -c "import dnspython; print dnspython.__path__" returned 1 /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/shell_out.rb:232:in invalid!'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/shell_out.rb:218:in
error!' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/mixin/shell_out.rb:33:in shell_out!'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/provider/package/easy_install.rb:69:in
load_current_resource' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/resource.rb:394:in run_action'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/runner.rb:53:in
run_action' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/runner.rb:89:in converge'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/runner.rb:89:in each' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/runner.rb:89:in converge'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/resource_collection.rb:94
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in
call' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in call_iterator_block'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/resource_collection/stepable_iterator.rb:85:in
step' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/resource_collection/stepable_iterator.rb:104:in iterate'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/resource_collection/stepable_iterator.rb:55:in
each_with_index' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/resource_collection.rb:92:in execute_each_resource'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/runner.rb:84:in
converge' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/client.rb:268:in converge'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/client.rb:171:in run' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/application/client.rb:222:in run_application'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/application/client.rb:212:in
loop' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/application/client.rb:212:in run_application'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/../lib/chef/application.rb:62:in
run' /usr/lib/ruby/gems/1.8/gems/chef-0.9.12/bin/chef-client:26 /usr/bin/chef-client:19:in load'
/usr/bin/chef-client:19
ERROR: Running exception handlers
ERROR: Exception handlers complete

Can you please paste your full working version of easy_install.rb?
Maybe I messed up when I applied the patch.

Grig

Since you're in the mode, Joe, I had trouble using the easy_install provider
for web.py http://webpy.org for similar (but perhaps not identical)
reasons.

Also, as a future enhancement, it would be handy if the easy_install
provider could handle installing local files as well. For now I've needed
to resort to custom execute resources.

Thanks for otherwise handy the provider, Joe!

  • Rob

On Tue, Nov 23, 2010 at 12:28 AM, Joe Williams joe@joetify.com wrote:

On Nov 22, 2010, at 9:01 PM, Grig Gheorghiu wrote:

On Mon, Nov 22, 2010 at 8:45 PM, Joe Williams joe@joetify.com wrote:

Grig,

I happened to write that provider. I found that it is difficult to detect
what python eggs are installed without attempting an import or scanning
through sys.path. Currently the provider should do it in that order but it
looks like it's exiting rather than getting rescued and trying to find it in
the sys.path. I'll work up a patch and get back to you.

-Joe

Hi Joe

Thanks for the quick reply. You're right, the original package name
(dnspython) should have been found by inspecting sys.path. Not sure
why it wasn't found. Here's what I get when I run this line (from

https://github.com/opscode/chef/blob/master/chef/lib/chef/provider/package/easy_install.rb
)
on my target node:

Try this patch out (works for me on a handful of packages):

Friendpaste - Paste #4IpYnXoV83AAKd1KxT5TO3

If it works for you I'll create a bug ticket and get it submitted for the
next release.

-Joe

Name: Joseph A. Williams
Email: joe@joetify.com
Blog: http://www.joeandmotorboat.com/
Twitter: http://twitter.com/williamsjoe

On Nov 22, 2010, at 9:51 PM, Grig Gheorghiu wrote:

On Mon, Nov 22, 2010 at 9:28 PM, Joe Williams joe@joetify.com wrote:

Try this patch out (works for me on a handful of packages):
Friendpaste - Paste #4IpYnXoV83AAKd1KxT5TO3
If it works for you I'll create a bug ticket and get it submitted for the
next release.

Can you please paste your full working version of easy_install.rb?
Maybe I messed up when I applied the patch.

I removed the path business so it seems the patch didn't get applied quite right. Here's the full file:

http://friendpaste.com/6olLDA3MZApdpZEJFGqNKa

-Joe

Name: Joseph A. Williams
Email: joe@joetify.com
Blog: http://www.joeandmotorboat.com/
Twitter: http://twitter.com/williamsjoe

On Mon, Nov 22, 2010 at 9:56 PM, Joe Williams joe@joetify.com wrote:

I removed the path business so it seems the patch didn't get applied
quite right. Here's the full file:
Friendpaste - Paste #6olLDA3MZApdpZEJFGqNKa

The problem, as it turns out, was that I had applied the patch to
easy_install.rb on the chef server, not the chef client....For some
reason I thought the client would get that updated file from the
server. In any case, once I patched the client it worked fine for the
2 packages I had issues with initially: dnspython and pydkim,

Thanks so much for your help, Joe! I hope your patch will make it in
the next release.

Grig

On Nov 22, 2010, at 10:17 PM, Grig Gheorghiu wrote:

On Mon, Nov 22, 2010 at 9:56 PM, Joe Williams joe@joetify.com wrote:

I removed the path business so it seems the patch didn't get applied
quite right. Here's the full file:
Friendpaste - Paste #6olLDA3MZApdpZEJFGqNKa

The problem, as it turns out, was that I had applied the patch to
easy_install.rb on the chef server, not the chef client....For some
reason I thought the client would get that updated file from the
server. In any case, once I patched the client it worked fine for the
2 packages I had issues with initially: dnspython and pydkim,

Thanks so much for your help, Joe! I hope your patch will make it in
the next release.

Grig, awesome, glad to help.

Rob, mind seeing if it works for you as well?

-Joe

Name: Joseph A. Williams
Email: joe@joetify.com
Blog: http://www.joeandmotorboat.com/
Twitter: http://twitter.com/williamsjoe

Joe, just checked. Unfortunately, web.py still doesn't work - although
admitedly it's an unusually named package:

[Tue, 23 Nov 2010 16:08:37 -0500] ERROR: easy_install_package[web.py]
(/srv/chef/cache/cookbooks/xxxxx/recipes/default.rb:18:in from_file') had an error: Expected process to exit 0, but it exited with 1 ---- Begin output of python -c "import web.py; print web.py.__path__" ---- STDOUT: STDERR: Traceback (most recent call last): File "<string>", line 1, in <module> ImportError: No module named py ---- End output of python -c "import web.py; print web.py.__path__" ---- Ran python -c "import web.py; print web.py.__path__" returned 1 /usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/shell_out.rb:232:in invalid!'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/shell_out.rb:218:in
error!' /usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/mixin/shell_out.rb:33:in shell_out!'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/provider/package/easy_install.rb:69:in
load_current_resource' /usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/resource.rb:394:in run_action'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/runner.rb:53:in
run_action' /usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/runner.rb:89:in converge'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/runner.rb:89:in
each' /usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/runner.rb:89:in converge'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/resource_collection.rb:94
/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in
call' /usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in call_iterator_block'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/resource_collection/stepable_iterator.rb:85:in
step' /usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/resource_collection/stepable_iterator.rb:104:in iterate'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/resource_collection/stepable_iterator.rb:55:in
each_with_index' /usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/resource_collection.rb:92:in execute_each_resource'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/runner.rb:84:in
converge' /usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/client.rb:268:in converge'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/client.rb:171:in
run' /usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/application/client.rb:222:in run_application'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/application/client.rb:212:in
loop' /usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/application/client.rb:212:in run_application'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/application.rb:62:in
run' /usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/chef-client:26 /usr/bin/chef-client:19:in load'
/usr/bin/chef-client:19
[Tue, 23 Nov 2010 16:08:37 -0500] ERROR: Running exception handlers
[Tue, 23 Nov 2010 16:08:37 -0500] ERROR: Exception handlers complete
/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/shell_out.rb:232:in
`invalid!': Expected process to exit 0, but it exited with 1
(Chef::Exceptions::ShellCommandFailed)
---- Begin output of python -c "import web.py; print web.py.path" ----
STDOUT:
STDERR: Traceback (most recent call last):
File "", line 1, in
ImportError: No module named py
---- End output of python -c "import web.py; print web.py.path" ----

Note that I don't actually use the EasyInstall provider because it doesn't
support local files. For our setup, we maintain all of our packages in an
S3 bucket, download and install them. So I've resorted to using an Execute
resource to install python packages and touch a file when each is successful
for idempotence. I would prefer to use the EasyInstall provider if it
supported local files as its idempotence check is more robust (except for
web.py, of course :slight_smile: ).

  • Rob

On Tue, Nov 23, 2010 at 1:23 AM, Joe Williams joe@joetify.com wrote:

On Nov 22, 2010, at 10:17 PM, Grig Gheorghiu wrote:

On Mon, Nov 22, 2010 at 9:56 PM, Joe Williams joe@joetify.com wrote:

I removed the path business so it seems the patch didn't get applied
quite right. Here's the full file:
Friendpaste - Paste #6olLDA3MZApdpZEJFGqNKa

The problem, as it turns out, was that I had applied the patch to
easy_install.rb on the chef server, not the chef client....For some
reason I thought the client would get that updated file from the
server. In any case, once I patched the client it worked fine for the
2 packages I had issues with initially: dnspython and pydkim,

Thanks so much for your help, Joe! I hope your patch will make it in
the next release.

Grig, awesome, glad to help.

Rob, mind seeing if it works for you as well?

-Joe

Name: Joseph A. Williams
Email: joe@joetify.com
Blog: http://www.joeandmotorboat.com/
Twitter: http://twitter.com/williamsjoe

I applied the patch correctly this time and it worked for web.py. Nice
work, Joe!

  • Rob

On Tue, Nov 23, 2010 at 4:16 PM, Rob Guttman robguttman@gmail.com wrote:

Joe, just checked. Unfortunately, web.py still doesn't work - although
admitedly it's an unusually named package:

[Tue, 23 Nov 2010 16:08:37 -0500] ERROR: easy_install_package[web.py]
(/srv/chef/cache/cookbooks/xxxxx/recipes/default.rb:18:in `from_file') had
an error:

Expected process to exit 0, but it exited with 1
---- Begin output of python -c "import web.py; print web.py.path" ----

STDOUT:
STDERR: Traceback (most recent call last):
File "", line 1, in
ImportError: No module named py
---- End output of python -c "import web.py; print web.py.path" ----
Ran python -c "import web.py; print web.py.path" returned 1
/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/shell_out.rb:232:in
invalid!' /usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/shell_out.rb:218:in error!'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/mixin/shell_out.rb:33:in
shell_out!' /usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/provider/package/easy_install.rb:69:in load_current_resource'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/resource.rb:394:in
run_action' /usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/runner.rb:53:in run_action'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/runner.rb:89:in
converge' /usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/runner.rb:89:in each'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/runner.rb:89:in
`converge'

/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/resource_collection.rb:94
/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in
call' /usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/resource_collection/stepable_iterator.rb:116:in call_iterator_block'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/resource_collection/stepable_iterator.rb:85:in
step' /usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/resource_collection/stepable_iterator.rb:104:in iterate'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/resource_collection/stepable_iterator.rb:55:in
each_with_index' /usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/resource_collection.rb:92:in execute_each_resource'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/runner.rb:84:in
converge' /usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/client.rb:268:in converge'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/client.rb:171:in
run' /usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/application/client.rb:222:in run_application'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/application/client.rb:212:in
loop' /usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/application/client.rb:212:in run_application'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/application.rb:62:in
`run'
/usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/chef-client:26

/usr/bin/chef-client:19:in load' /usr/bin/chef-client:19 [Tue, 23 Nov 2010 16:08:37 -0500] ERROR: Running exception handlers [Tue, 23 Nov 2010 16:08:37 -0500] ERROR: Exception handlers complete /usr/lib/ruby/gems/1.8/gems/chef-0.9.10/bin/../lib/chef/shell_out.rb:232:in invalid!': Expected process to exit 0, but it exited with 1
(Chef::Exceptions::ShellCommandFailed)
---- Begin output of python -c "import web.py; print web.py.path" ----

STDOUT:
STDERR: Traceback (most recent call last):
File "", line 1, in
ImportError: No module named py
---- End output of python -c "import web.py; print web.py.path" ----

Note that I don't actually use the EasyInstall provider because it doesn't
support local files. For our setup, we maintain all of our packages in an
S3 bucket, download and install them. So I've resorted to using an Execute
resource to install python packages and touch a file when each is successful
for idempotence. I would prefer to use the EasyInstall provider if it
supported local files as its idempotence check is more robust (except for
web.py, of course :slight_smile: ).

  • Rob

On Tue, Nov 23, 2010 at 1:23 AM, Joe Williams joe@joetify.com wrote:

On Nov 22, 2010, at 10:17 PM, Grig Gheorghiu wrote:

On Mon, Nov 22, 2010 at 9:56 PM, Joe Williams joe@joetify.com wrote:

I removed the path business so it seems the patch didn't get
applied
quite right. Here's the full file:
Friendpaste - Paste #6olLDA3MZApdpZEJFGqNKa

The problem, as it turns out, was that I had applied the patch to
easy_install.rb on the chef server, not the chef client....For some
reason I thought the client would get that updated file from the
server. In any case, once I patched the client it worked fine for the
2 packages I had issues with initially: dnspython and pydkim,

Thanks so much for your help, Joe! I hope your patch will make it in
the next release.

Grig, awesome, glad to help.

Rob, mind seeing if it works for you as well?

-Joe

Name: Joseph A. Williams
Email: joe@joetify.com
Blog: http://www.joeandmotorboat.com/
Twitter: http://twitter.com/williamsjoe

On Nov 23, 2010, at 1:44 PM, Rob Guttman wrote:

I applied the patch correctly this time and it worked for web.py. Nice work, Joe!

Awesome, glad it worked.

-Joe

Name: Joseph A. Williams
Email: joe@joetify.com
Blog: http://www.joeandmotorboat.com/
Twitter: http://twitter.com/williamsjoe

I came across the same problems, I created a library to handle
easy_install (also because I had to use python26 instead of python):

related to:
http://tickets.opscode.com/browse/CHEF-1633
http://tickets.opscode.com/browse/CHEF-1634

what I did:

The idea is to pass extra parameter to the install method:
version = options.delete(:version)
module_name = options.delete(:module) || name
file_name = options.delete(:file) || name

:name is the name of the package, the arg of easy_install
-> module_name is used to get the version of an installed module
-> file_name is used to check if the package is installed

Note:
It seems http://tickets.opscode.com/browse/CHEF-1634 (module_name) has
been put in 0.9.12

I'll make a request for the file_name trick

--Gilles

On Tue, Nov 23, 2010 at 1:47 PM, Joe Williams joe@joetify.com wrote:

On Nov 23, 2010, at 1:44 PM, Rob Guttman wrote:

I applied the patch correctly this time and it worked for web.py. Nice work, Joe!

Awesome, glad it worked.

-Joe

Name: Joseph A. Williams
Email: joe@joetify.com
Blog: http://www.joeandmotorboat.com/
Twitter: http://twitter.com/williamsjoe

On Nov 23, 2010, at 2:20 PM, Gilles Devaux wrote:

Note:
It seems http://tickets.opscode.com/browse/CHEF-1634 (module_name) has
been put in 0.9.12

Cool, although it doesn't look like it's been merged yet (https://github.com/opscode/chef/commits/master/chef/lib/chef/provider/package/easy_install.rb).

I imagine both of ours will get put into 0.9.14 or 0.10.

-Joe

On Tue, Nov 23, 2010 at 1:47 PM, Joe Williams joe@joetify.com wrote:

On Nov 23, 2010, at 1:44 PM, Rob Guttman wrote:

I applied the patch correctly this time and it worked for web.py. Nice work, Joe!

Awesome, glad it worked.

-Joe

Name: Joseph A. Williams
Email: joe@joetify.com
Blog: http://www.joeandmotorboat.com/
Twitter: http://twitter.com/williamsjoe

Name: Joseph A. Williams
Email: joe@joetify.com
Blog: http://www.joeandmotorboat.com/
Twitter: http://twitter.com/williamsjoe

I've update the jira.

On Tue, Nov 23, 2010 at 3:01 PM, Joe Williams joe@joetify.com wrote:

On Nov 23, 2010, at 2:20 PM, Gilles Devaux wrote:

Note:
It seems http://tickets.opscode.com/browse/CHEF-1634 (module_name) has
been put in 0.9.12

Cool, although it doesn't look like it's been merged yet
(https://github.com/opscode/chef/commits/master/chef/lib/chef/provider/package/easy_install.rb).
I imagine both of ours will get put into 0.9.14 or 0.10.
-Joe

On Tue, Nov 23, 2010 at 1:47 PM, Joe Williams joe@joetify.com wrote:

On Nov 23, 2010, at 1:44 PM, Rob Guttman wrote:

I applied the patch correctly this time and it worked for web.py. Nice
work, Joe!

Awesome, glad it worked.

-Joe

Name: Joseph A. Williams

Email: joe@joetify.com

Blog: http://www.joeandmotorboat.com/

Twitter: http://twitter.com/williamsjoe

Name: Joseph A. Williams
Email: joe@joetify.com
Blog: http://www.joeandmotorboat.com/
Twitter: http://twitter.com/williamsjoe

I hope these patches will also fix the problem when two versions of python
are installed.
http://tickets.opscode.com/browse/CHEF-1889

On Wed, Nov 24, 2010 at 2:50 AM, Gilles Devaux gilles.devaux@gmail.comwrote:

I've update the jira.

On Tue, Nov 23, 2010 at 3:01 PM, Joe Williams joe@joetify.com wrote:

On Nov 23, 2010, at 2:20 PM, Gilles Devaux wrote:

Note:
It seems http://tickets.opscode.com/browse/CHEF-1634 (module_name) has
been put in 0.9.12

Cool, although it doesn't look like it's been merged yet
(
https://github.com/opscode/chef/commits/master/chef/lib/chef/provider/package/easy_install.rb
).
I imagine both of ours will get put into 0.9.14 or 0.10.
-Joe

On Tue, Nov 23, 2010 at 1:47 PM, Joe Williams joe@joetify.com wrote:

On Nov 23, 2010, at 1:44 PM, Rob Guttman wrote:

I applied the patch correctly this time and it worked for web.py. Nice
work, Joe!

Awesome, glad it worked.

-Joe

Name: Joseph A. Williams

Email: joe@joetify.com

Blog: http://www.joeandmotorboat.com/

Twitter: http://twitter.com/williamsjoe

Name: Joseph A. Williams
Email: joe@joetify.com
Blog: http://www.joeandmotorboat.com/
Twitter: http://twitter.com/williamsjoe

--
Mike Scherbakov

On Nov 24, 2010, at 6:53 AM, Mike Scherbakov wrote:

I hope these patches will also fix the problem when two versions of python are installed.
http://tickets.opscode.com/browse/CHEF-1889

Mike,

I am pretty sure Gilles' patch (CHEF-1634) takes care of that.

-Joe

Name: Joseph A. Williams
Email: joe@joetify.com
Blog: http://www.joeandmotorboat.com/
Twitter: http://twitter.com/williamsjoe

It does

On Wed, Nov 24, 2010 at 8:20 AM, Joe Williams joe@joetify.com wrote:

On Nov 24, 2010, at 6:53 AM, Mike Scherbakov wrote:

I hope these patches will also fix the problem when two versions of python
are installed.
http://tickets.opscode.com/browse/CHEF-1889

Mike,
I am pretty sure Gilles' patch (CHEF-1634) takes care of that.
-Joe

Name: Joseph A. Williams
Email: joe@joetify.com
Blog: http://www.joeandmotorboat.com/
Twitter: http://twitter.com/williamsjoe