Chef you dont support python.....why?


#1

I am trying to bootstrap an server. I used to use “recipe[poise-python]” to pip install packages. Now I get the below error. How do fix?

* python_package[setuptools] action upgrade
    
    ================================================================================
    Error executing action `upgrade` on resource 'python_package[setuptools]'
    ================================================================================
    
    Mixlib::ShellOut::ShellCommandFailed
    ------------------------------------
    Expected process to exit with [0], but received '1'
    ---- Begin output of ["/usr/bin/python", "-", "setuptools"] ----
    STDOUT: 
    STDERR: The python_package resource requires pip >= 6.1.0, currently 10.0.0
    ---- End output of ["/usr/bin/python", "-", "setuptools"] ----
    Ran ["/usr/bin/python", "-", "setuptools"] returned 1
    
    Cookbook Trace:
    ---------------
    /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:207:in `tap'
    /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:207:in `language_command_shell_out!'
    /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:222:in `block in language_command_mixin'
    /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:317:in `pip_command'
    /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:341:in `pip_outdated'
    /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:209:in `check_package_versions'
    /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:189:in `load_current_resource'
    
    Resource Declaration:
    ---------------------
    # In /var/chef/cache/cookbooks/supervisor/recipes/default.rb
    
     32: python_package "setuptools" do
     33:   action :upgrade
     34: end
     35: 
    
    Compiled Resource:
    ------------------
    # Declared in /var/chef/cache/cookbooks/supervisor/recipes/default.rb:32:in `from_file'
    
    python_package("setuptools") do
      package_name "setuptools"
      action [:upgrade]
      default_guard_interpreter :default
      declared_type :python_package
      cookbook_name "supervisor"
      recipe_name "default"
      parent_python nil
      timeout 900
    end
    
    System Info:
    ------------
    chef_version=14.0.202
    platform=ubuntu
    platform_version=16.04
    ruby=ruby 2.5.1p57 (2018-03-29 revision 63029) [x86_64-linux]
    program_name=/usr/bin/chef-solo
    executable=/opt/chef/bin/chef-solo
    

Running handlers:
[2018-04-17T01:46:45+00:00] ERROR: Running exception handlers
Running handlers complete
[2018-04-17T01:46:45+00:00] ERROR: Exception handlers complete
Chef Client failed. 2 resources updated in 23 seconds
[2018-04-17T01:46:45+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
[2018-04-17T01:46:45+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
[2018-04-17T01:46:45+00:00] FATAL: Mixlib::ShellOut::ShellCommandFailed: python_package[setuptools] (supervisor::default line 32) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of ["/usr/bin/python", "-", "setuptools"] ----
STDOUT: 
STDERR: The python_package resource requires pip >= 6.1.0, currently 10.0.0
---- End output of ["/usr/bin/python", "-", "setuptools"] ----
Ran ["/usr/bin/python", "-", "setuptools"] returned 1

#2

As I’ve now said in line 5 different places including numerous open issues on the poise-python repo, I’m working on it. The pip 10 release happened to fall at a time when I don’t have a ton of spare cycles so it’s taken a few days to get back on top of the all the build failures from both this and Chef 14. It’s coming though.


#3

Well my question is now more for chef…then why do you not support python package installs? I had a production server die and what should have been a 10 min boot took several hours to refactor the code from poise-python. You used to have easy_install and for years that resource had dubious results with random failures. Why cant you support a reliable pip_install?


#4

I don’t work for Chef Software, they are not involved with any of the poise-* suite of cookbooks. The problem is very specifically that Pip released a new major version three days ago, it is otherwise quite stable. Such are the joys of open source? If you think three days is too long a turnaround for a major release on a one-person volunteer project then I am happy to provide you with a full refund for all my cookbooks.


#5

For me yes 3 days is long… the only cookbooks I use not built by me are build-essential and poise-python. Started out with more but I found one by one over time they tend to fail due to these type of issues and the complexity of supporting multiple platforms. the extent I used poise-python was pip_install but easy enough to use bash which I will have to do just a lot of refactoring so now I am down to just using build-essential. This is something that should be supported by chef.

…of course not your issue it is what it is… but I cant be down


#6

The fix was posted in all the various threads and whatnot. For anyone else that finds this before the release goes up (we’ll see how much longer I can stay awake):

You can use either the pip_version property on the python_runtime resource or node attributes (override[‘poise-python’][‘options’][‘pip_version’]) to set the version for managed runtimes back to 9.0.3 which works.


#7

Is the python “setuptools” the one from the Ubuntu distribution, or a more recent one being installed dynamically from Chef? I’ve had considerable issues with Python modules and dependencies breaking each other when allowing Chef to do “pip install” rather than using “yum_package” on the RHEL and CentOS hosts I work on, and I tent to publish RPM’s for those packages to ensure pre-availability of defined, compatible versions of all modules. While “setuptools” is in the

It looks like that may not be the problem here, it looks like the comparision of version “10.,0.0” of “pip” itself is being incorrectly reported as less than 6.1.0. But is an older version of “pip” available, on that Ubuntu release, from the Ubuntu maintainers? One that can be gracefully rolled back to, from the vendor, before the other chef cookbooks run? Maintaining a bill-of-materials of system components for production systems is a common task, and this can include python components from the OS vendor.


#8

The incorrect error message from the version check is funny (I checked for /^1/ rather than /^1\./) but isn’t the actual problem, there are a bunch of compat changes needed for Pip 10 itself, as well as some changes related to Chef 14.


#9

Poise-python 1.7.0 is up with full support for both Pip 10 and Chef 14.


#10

Hi I am still facing the python problems on my environment, please see the below error

python_package[setuptools] action install
192.168.1.122
192.168.1.122 ================================================================================
192.168.1.122 Error executing action install on resource 'python_package[setuptools]'
192.168.1.122 ================================================================================
192.168.1.122
192.168.1.122 Mixlib::ShellOut::ShellCommandFailed
192.168.1.122 ------------------------------------
192.168.1.122 Expected process to exit with [0], but received '1'
192.168.1.122 ---- Begin output of ["/usr/bin/python", "-", "setuptools"] ----
192.168.1.122 STDOUT:
192.168.1.122 STDERR: Traceback (most recent call last):
192.168.1.122 File "", line 42, in
192.168.1.122 AttributeError: type object 'InstallRequirement' has no attribute 'from_line'
192.168.1.122 ---- End output of ["/usr/bin/python", "-", "setuptools"] ----
192.168.1.122 Ran ["/usr/bin/python", "-", "setuptools"] returned 1
192.168.1.122
192.168.1.122 Cookbook Trace:
192.168.1.122 ---------------
192.168.1.122 /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:207:in tap' 192.168.1.122 /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:207:inlanguage_command_shell_out!'
192.168.1.122 /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:222:in block in language_command_mixin' 192.168.1.122 /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:329:inpip_command'
192.168.1.122 /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:353:in pip_outdated' 192.168.1.122 /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:221:incheck_package_versions'
192.168.1.122 /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:200:in load_current_resource' 192.168.1.122 /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:innotifying_block'
192.168.1.122 /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:50:in action_install' 192.168.1.122 192.168.1.122 Resource Declaration: 192.168.1.122 --------------------- 192.168.1.122 # In /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb 192.168.1.122 192.168.1.122 138: python_package 'setuptools' do 192.168.1.122 139: parent_python new_resource 192.168.1.122 140: version setuptools_version if setuptools_version.is_a?(String) 192.168.1.122 141: end 192.168.1.122 142: end 192.168.1.122 192.168.1.122 Compiled Resource: 192.168.1.122 ------------------ 192.168.1.122 # Declared in /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:138:ininstall_setuptools'
192.168.1.122
192.168.1.122 python_package("setuptools") do
192.168.1.122 package_name "setuptools"
192.168.1.122 action [:install]
192.168.1.122 default_guard_interpreter :default
192.168.1.122 declared_type :python_package
192.168.1.122 cookbook_name "cloudcli"
192.168.1.122 parent_python python_runtime[2]
192.168.1.122 timeout 900
192.168.1.122 end
192.168.1.122
192.168.1.122 System Info:
192.168.1.122 ------------
192.168.1.122 chef_version=13.11.3
192.168.1.122 platform=centos
192.168.1.122 platform_version=7.5.1804
192.168.1.122 ruby=ruby 2.4.4p296 (2018-03-28 revision 63013) [x86_64-linux]
192.168.1.122 program_name=chef-client worker: ppid=12615;start=08:46:12;
192.168.1.122 executable=/opt/chef/bin/chef-client
192.168.1.122
192.168.1.122
192.168.1.122 ================================================================================
192.168.1.122 Error executing action install on resource 'python_runtime[2]'
192.168.1.122 ================================================================================
192.168.1.122
192.168.1.122 Mixlib::ShellOut::ShellCommandFailed
192.168.1.122 ------------------------------------
192.168.1.122 python_package[setuptools] (/var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb line 138) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
192.168.1.122 ---- Begin output of ["/usr/bin/python", "-", "setuptools"] ----
192.168.1.122 STDOUT:
192.168.1.122 STDERR: Traceback (most recent call last):
192.168.1.122 File "", line 42, in
192.168.1.122 AttributeError: type object 'InstallRequirement' has no attribute 'from_line'
192.168.1.122 ---- End output of ["/usr/bin/python", "-", "setuptools"] ----
192.168.1.122 Ran ["/usr/bin/python", "-", "setuptools"] returned 1
192.168.1.122
192.168.1.122 Cookbook Trace:
192.168.1.122 ---------------
192.168.1.122 /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:207:in tap' 192.168.1.122 /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:207:inlanguage_command_shell_out!'
192.168.1.122 /var/chef/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:222:in block in language_command_mixin' 192.168.1.122 /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:329:inpip_command'
192.168.1.122 /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:353:in pip_outdated' 192.168.1.122 /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:221:incheck_package_versions'
192.168.1.122 /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:200:in load_current_resource' 192.168.1.122 /var/chef/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:innotifying_block'
192.168.1.122 /var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:50:in action_install' 192.168.1.122 192.168.1.122 Resource Declaration: 192.168.1.122 --------------------- 192.168.1.122 # In /var/chef/cache/cookbooks/cloudcli/recipes/_aws_linux.rb 192.168.1.122 192.168.1.122 20: python_runtime node['cloudcli']['aws']['python']['version'] do 192.168.1.122 21: provider node['cloudcli']['aws']['python']['provider'] 192.168.1.122 22: end 192.168.1.122 23: 192.168.1.122 192.168.1.122 Compiled Resource: 192.168.1.122 ------------------ 192.168.1.122 # Declared in /var/chef/cache/cookbooks/cloudcli/recipes/_aws_linux.rb:20:infrom_file'
192.168.1.122
192.168.1.122 python_runtime("2") do
192.168.1.122 provider PoisePython::PythonProviders::System
192.168.1.122 action [:install]
192.168.1.122 default_guard_interpreter :default
192.168.1.122 subresources [python_package[awscli], python_runtime_pip[2], python_package[setuptools], python_package[wheel], python_package[virtualenv]]
192.168.1.122 declared_type :python_runtime
192.168.1.122 cookbook_name "cloudcli"
192.168.1.122 recipe_name "_aws_linux"
192.168.1.122 get_pip_url "https://bootstrap.pypa.io/get-pip.py"
192.168.1.122 pip_version true
192.168.1.122 setuptools_version true
192.168.1.122 version "2"
192.168.1.122 virtualenv_version true
192.168.1.122 wheel_version true
192.168.1.122 end
192.168.1.122
192.168.1.122 System Info:
192.168.1.122 ------------
192.168.1.122 chef_version=13.11.3
192.168.1.122 platform=centos
192.168.1.122 platform_version=7.5.1804
192.168.1.122 ruby=ruby 2.4.4p296 (2018-03-28 revision 63013) [x86_64-linux]
192.168.1.122 program_name=chef-client worker: ppid=12615;start=08:46:12;
192.168.1.122 executable=/opt/chef/bin/chef-client
192.168.1.122
192.168.1.122
192.168.1.122 Running handlers:
192.168.1.122 [2018-10-08T08:46:24+00:00] ERROR: Running exception handlers
192.168.1.122 Running handlers complete
192.168.1.122 [2018-10-08T08:46:24+00:00] ERROR: Exception handlers complete
192.168.1.122 Chef Client failed. 0 resources updated in 12 seconds
192.168.1.122 [2018-10-08T08:46:24+00:00] FATAL: Stacktrace dumped to /var/chef/cache/chef-stacktrace.out
192.168.1.122 [2018-10-08T08:46:24+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
192.168.1.122 [2018-10-08T08:46:24+00:00] ERROR: python_runtime[2] (cloudcli::_aws_linux line 20) had an error: Mixlib::ShellOut::ShellCommandFailed: python_package[setuptools] (/var/chef/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb line 138) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
192.168.1.122 ---- Begin output of ["/usr/bin/python", "-", "setuptools"] ----
192.168.1.122 STDOUT:
192.168.1.122 STDERR: Traceback (most recent call last):
192.168.1.122 File "", line 42, in
192.168.1.122 AttributeError: type object 'InstallRequirement' has no attribute 'from_line'
192.168.1.122 ---- End output of ["/usr/bin/python", "-", "setuptools"] ----
192.168.1.122 Ran ["/usr/bin/python", "-", "setuptools"] returned 1
192.168.1.122 [2018-10-08T08:46:24+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)


#11

Can somebody help me on the above request? @coderanger i am quite new to chef... pls help...
I am using cloudcli 1.2.0 and poise-python 1.7.0 cookbooks together on chef 13.11.3 on centos7


#12

Hi noble1,

You should start a new topic when dealing with a different error, but for the sake of expediency: the bootstrap.pypa.io/get-pip.py script was broken recently.

See https://github.com/poise/poise-python/issues/133 for a workaround.


#13

Yeah I got it... Thank you for the update @ccrebolder