Ohai Ruby plugin: system ruby vs. running ruby?


#1

Ohai Chefs, we’d like to get your feedback on an issue that’s been reported a few times for ohai[1].

The problem is that ohai shells out to the ruby executable in order to populate the languages/ruby attributes.

This is good because:

  • It detects the “system” ruby even if you have several rubies installed, or are running Chef with a different ruby (i.e., running chef under REE or whatever). This is consistent with the other language plugins which detect python (but not python26) java, and so on. It’s not consistent with the C plugin which will detect all of GCC, MS, IBM, Sun, and HP-UX compilers.

This is bad because:

  • It detects a different ruby than the one you’re using for Chef.
  • It does not detect a ruby installed as ruby18 or ruby19 (or rbx or jruby for that matter)
  • It’s slower than using rbconfig within the same ruby process.

There’s a few things we could do about this:

  1. Always use the Ruby that you’re using for ohai/chef-client. If we do this, we could avoid shelling out entirely and the ruby plugin would run faster.
  2. Try to run ruby18 or ruby19 (and/or rbx, jruby, etc.) if there is no ruby in the PATH.
  3. Do the same as #2, but try to find every Ruby we can, e.g., detect all of ruby, ruby18, ruby19, rbx, jruby.

To be honest, I’m not sure what people are using the languages/ruby attributes for, so I’m not able to make a judgement about which of these options is best (or maybe there’s another option?). If this is something that matters to you, please chime in and explain your use case and which course of action would be most helpful to you.

Thanks!


Dan DeLeo

  1. http://tickets.opscode.com/browse/OHAI-172 is the canonical ticket, there are some duplicates as well.

#2

I suppose that, firstly, we need ohai RVM plugin.

And next, we need to report data default ruby.

I.e. system ruby without RVM, or current (rvm use --default) ruby in case of
RVM.

2011/8/31 Daniel DeLeo dan@kallistec.com

Ohai Chefs, we’d like to get your feedback on an issue that’s been
reported a few times for ohai[1].

The problem is that ohai shells out to the ruby executable in order to
populate the languages/ruby attributes.

This is good because:

  • It detects the “system” ruby even if you have several rubies installed,
    or are running Chef with a different ruby (i.e., running chef under REE or
    whatever). This is consistent with the other language plugins which detect
    python (but not python26) java, and so on. It’s not consistent with the
    C plugin which will detect all of GCC, MS, IBM, Sun, and HP-UX compilers.

This is bad because:

  • It detects a different ruby than the one you’re using for Chef.
  • It does not detect a ruby installed as ruby18 or ruby19 (or rbx or
    jruby for that matter)
  • It’s slower than using rbconfig within the same ruby process.

There’s a few things we could do about this:

  1. Always use the Ruby that you’re using for ohai/chef-client. If we do
    this, we could avoid shelling out entirely and the ruby plugin would run
    faster.
  2. Try to run ruby18 or ruby19 (and/or rbx, jruby, etc.) if there
    is no ruby in the PATH.
  3. Do the same as #2, but try to find every Ruby we can, e.g., detect all
    of ruby, ruby18, ruby19, rbx, jruby.

To be honest, I’m not sure what people are using the languages/ruby
attributes for, so I’m not able to make a judgement about which of these
options is best (or maybe there’s another option?). If this is something
that matters to you, please chime in and explain your use case and which
course of action would be most helpful to you.

Thanks!


Dan DeLeo

  1. http://tickets.opscode.com/browse/OHAI-172 is the canonical ticket,
    there are some duplicates as well.

#3

On Aug 31, 2011, at 2:32 PM, Daniel DeLeo wrote:

There’s a few things we could do about this:

  1. Always use the Ruby that you’re using for ohai/chef-client. If we do this, we could avoid shelling out entirely and the ruby plugin would run faster.

For those of us who have used the Omnibus installer, there is an embedded version of Ruby that is included. The path to that embedded version should be included in any Chef or Chef-related code that runs on the system, and that process should be instantiated at install/compile time – no need to search around for which version of ruby to run, you brought your own version of ruby with you, and you know that’s the one you always want to use.

Moreover, the user path should not have to be modified to include the directory for /opt/opscode/embedded/bin/ruby, nor should there be a symlink to this executable created in /usr/bin, /usr/local/bin, or any other standard (or semi-standard) location.

This entire problem can be fixed in stone at the time of install/compile, and if you need to upgrade (or downgrade) the embedded version of ruby, you should be able to do that pretty much completely independently of everything else on the system. That’s why it’s called “embedded”.

We had some discussions on this topic in #chef. I would encourage you to go check the logs.


Brad Knowles bknowles@ihiji.com


#4

On Wednesday, August 31, 2011 at 12:50 PM, Brad Knowles wrote:

On Aug 31, 2011, at 2:32 PM, Daniel DeLeo wrote:

There’s a few things we could do about this:

  1. Always use the Ruby that you’re using for ohai/chef-client. If we do this, we could avoid shelling out entirely and the ruby plugin would run faster.

For those of us who have used the Omnibus installer, there is an embedded version of Ruby that is included. The path to that embedded version should be included in any Chef or Chef-related code that runs on the system, and that process should be instantiated at install/compile time – no need to search around for which version of ruby to run, you brought your own version of ruby with you, and you know that’s the one you always want to use.

Moreover, the user path should not have to be modified to include the directory for /opt/opscode/embedded/bin/ruby, nor should there be a symlink to this executable created in /usr/bin, /usr/local/bin, or any other standard (or semi-standard) location.

This entire problem can be fixed in stone at the time of install/compile, and if you need to upgrade (or downgrade) the embedded version of ruby, you should be able to do that pretty much completely independently of everything else on the system. That’s why it’s called “embedded”.

We had some discussions on this topic in #chef. I would encourage you to go check the logs.

So, if your ohai attributes for languages/ruby are empty, what breaks?


Dan DeLeo


Brad Knowles <bknowles@ihiji.com (mailto:bknowles@ihiji.com)>


#5

On Aug 31, 2011, at 2:52 PM, Daniel DeLeo wrote:

So, if your ohai attributes for languages/ruby are empty, what breaks?

In my case, everything related to Chef. The system-provided version of ruby & rubygems is old enough that it will not work with anything related to Chef, and yet there are other parts of the system that are dependent on that older version being installed and being installed in the system-standard place. And they probably also do stupid things like assuming that the path is always set up correctly so if they just shell out to “ruby” it will pick up the right system-provided version.

So, I can’t upgrade the system-provided version of ruby & ruby-gems, but I have to have the newer version of ruby & ruby-gems in order to get Chef & ohai to work.

Both code bases want to assume that there is one and only one installation of ruby on the system, and it is necessarily the version that they need.

That’s a majorly, heinously, bad assumption.

You can’t go back and fix all the legacy code that wants to make that one-and-only-one assumption, but you can at least ensure that all the new code will always be able to find the right version that it actually needs, without breaking or otherwise causing any disturbance for any of the older code.

So, if you’re going to program in Ruby, and you’re going to build a system that requires newer version of the code than what we can get with a system-standard install, then the requirement falls onto you to make sure that you make things work properly. Don’t break my existing production systems with your new code, and don’t make me jump through hoops to get two incompatible versions installed.


Brad Knowles bknowles@ihiji.com


#6

On Wednesday, August 31, 2011 at 12:59 PM, Brad Knowles wrote:

On Aug 31, 2011, at 2:52 PM, Daniel DeLeo wrote:

So, if your ohai attributes for languages/ruby are empty, what breaks?

In my case, everything related to Chef. The system-provided version of ruby & rubygems is old enough that it will not work with anything related to Chef, and yet there are other parts of the system that are dependent on that older version being installed and being installed in the system-standard place. And they probably also do stupid things like assuming that the path is always set up correctly so if they just shell out to “ruby” it will pick up the right system-provided version.

So, I can’t upgrade the system-provided version of ruby & ruby-gems, but I have to have the newer version of ruby & ruby-gems in order to get Chef & ohai to work.

Both code bases want to assume that there is one and only one installation of ruby on the system, and it is necessarily the version that they need.

That’s a majorly, heinously, bad assumption.

You can’t go back and fix all the legacy code that wants to make that one-and-only-one assumption, but you can at least ensure that all the new code will always be able to find the right version that it actually needs, without breaking or otherwise causing any disturbance for any of the older code.

So, if you’re going to program in Ruby, and you’re going to build a system that requires newer version of the code than what we can get with a system-standard install, then the requirement falls onto you to make sure that you make things work properly. Don’t break my existing production systems with your new code, and don’t make me jump through hoops to get two incompatible versions installed.


Brad Knowles <bknowles@ihiji.com (mailto:bknowles@ihiji.com)>
The change that I’m talking about is only related to how ohai collects data about what ruby you have installed on the system. In Chef you can access this data as node[:languages][:ruby]. Are you currently depending on this data to return information about the system ruby?


Dan DeLeo


#7

On Aug 31, 2011, at 3:03 PM, Daniel DeLeo wrote:

The change that I’m talking about is only related to how ohai collects data about what ruby you have installed on the system. In Chef you can access this data as node[:languages][:ruby]. Are you currently depending on this data to return information about the system ruby?

Right now, we’re just trying to figure out how to use Chef to help us rebuild our infrastructure so that it is more manageable and scalable. The Omnibus installer goes a way towards that, but there is still the issue of having to set the user’s path environment variable to include the /opt/opscode/embedded/bin directory before anything else because replacing the standard system-provided version of /usr/bin/ruby with symlinks to the new binary would cause all the old system-provided programs to break.

This is CentOS 5.6. We’re not back on RHEL4 or CentOS 5.1 or something – this is relatively recent, to the degree that anything from RHEL/CentOS is recent.

We’re not currently actually using Chef or ohai for anything, but if the code depends on a specific version of ruby & ruby-gems being installed and that version is different from what the standard system-provided version is, then you need to find a way to handle that exception within your own code. This problem should be handled at compile/install time, so that I don’t even have to modify the user path environment variable to include the new embedded subdirectory location.

You need to decide how you’re going to handle potentially having several different versions of the same program installed (think multiple versions of python as well as ruby, etc…). Our code doesn’t care, at least not at the moment. But you can be assured that whatever code we do have that might be sensitive to what version of what program is installed, we will handle the pathing/location issues to that within our own code.


Brad Knowles bknowles@ihiji.com


#8

I see the need to differentiate the chef ruby (using rbconf) to figure
out what chef is running under. assuming you may need this for
developing some in-recipe things. I.e. i want to detect ruby 1.8 or
1.9 in my recipe. I know we can do that other ways in the recipe but
node['ruby"][“chef”][“version”] makes sense to me.

I think pulling all other rubies is also good. For the times when i
want to unsure some other script code or framework will work.
node[“ruby”][“system”][“jruby”]

I also see that you would want to detect the available rubies under
rvm. node[“ruby”][“rvm”] for similar reasons.

On Wed, Aug 31, 2011 at 1:03 PM, Daniel DeLeo dan@kallistec.com wrote:

On Wednesday, August 31, 2011 at 12:59 PM, Brad Knowles wrote:

On Aug 31, 2011, at 2:52 PM, Daniel DeLeo wrote:

So, if your ohai attributes for languages/ruby are empty, what breaks?

In my case, everything related to Chef. The system-provided version of ruby & rubygems is old enough that it will not work with anything related to Chef, and yet there are other parts of the system that are dependent on that older version being installed and being installed in the system-standard place. And they probably also do stupid things like assuming that the path is always set up correctly so if they just shell out to “ruby” it will pick up the right system-provided version.

So, I can’t upgrade the system-provided version of ruby & ruby-gems, but I have to have the newer version of ruby & ruby-gems in order to get Chef & ohai to work.

Both code bases want to assume that there is one and only one installation of ruby on the system, and it is necessarily the version that they need.

That’s a majorly, heinously, bad assumption.

You can’t go back and fix all the legacy code that wants to make that one-and-only-one assumption, but you can at least ensure that all the new code will always be able to find the right version that it actually needs, without breaking or otherwise causing any disturbance for any of the older code.

So, if you’re going to program in Ruby, and you’re going to build a system that requires newer version of the code than what we can get with a system-standard install, then the requirement falls onto you to make sure that you make things work properly. Don’t break my existing production systems with your new code, and don’t make me jump through hoops to get two incompatible versions installed.


Brad Knowles <bknowles@ihiji.com (mailto:bknowles@ihiji.com)>
The change that I’m talking about is only related to how ohai collects data about what ruby you have installed on the system. In Chef you can access this data as node[:languages][:ruby]. Are you currently depending on this data to return information about the system ruby?


Dan DeLeo


#9

Brad, why do you need to include /opt/opscode/embedded/bin in the
path? All the binaries that omnibus installs are using the right
interpreter on the shebang line, so this should not be needed at all.

Adam

On Wed, Aug 31, 2011 at 1:25 PM, Brad Knowles bknowles@ihiji.com wrote:

On Aug 31, 2011, at 3:03 PM, Daniel DeLeo wrote:

The change that I’m talking about is only related to how ohai collects data about what ruby you have installed on the system. In Chef you can access this data as node[:languages][:ruby]. Are you currently depending on this data to return information about the system ruby?

Right now, we’re just trying to figure out how to use Chef to help us rebuild our infrastructure so that it is more manageable and scalable. The Omnibus installer goes a way towards that, but there is still the issue of having to set the user’s path environment variable to include the /opt/opscode/embedded/bin directory before anything else because replacing the standard system-provided version of /usr/bin/ruby with symlinks to the new binary would cause all the old system-provided programs to break.

This is CentOS 5.6. We’re not back on RHEL4 or CentOS 5.1 or something – this is relatively recent, to the degree that anything from RHEL/CentOS is recent.

We’re not currently actually using Chef or ohai for anything, but if the code depends on a specific version of ruby & ruby-gems being installed and that version is different from what the standard system-provided version is, then you need to find a way to handle that exception within your own code. This problem should be handled at compile/install time, so that I don’t even have to modify the user path environment variable to include the new embedded subdirectory location.

You need to decide how you’re going to handle potentially having several different versions of the same program installed (think multiple versions of python as well as ruby, etc…). Our code doesn’t care, at least not at the moment. But you can be assured that whatever code we do have that might be sensitive to what version of what program is installed, we will handle the pathing/location issues to that within our own code.


Brad Knowles bknowles@ihiji.com


Opscode, Inc.
Adam Jacob, Chief Product Officer
T: (206) 619-7151 E: adam@opscode.com


#10

On Aug 31, 2011, at 6:29 PM, Adam Jacob wrote:

Brad, why do you need to include /opt/opscode/embedded/bin in the
path? All the binaries that omnibus installs are using the right
interpreter on the shebang line, so this should not be needed at all.

That should be the case, and that may now be the case, but as of the time the ticket for CHEF-2578 was filed, that was not the case.

If you can convince me that the issue in this ticket is resolved and that you’re not creating any symlinks from outside of /opt/opscode into subdirectories of that directory structure, and you’re not dependent on people making sure that they put /opt/opscode/embedded/bin into their $PATH before anything else (like /usr/bin), then I’ll be glad to shut up.


Brad Knowles bknowles@ihiji.com


#11

On Wednesday, August 31, 2011 at 4:55 PM, Brad Knowles wrote:

On Aug 31, 2011, at 6:29 PM, Adam Jacob wrote:

Brad, why do you need to include /opt/opscode/embedded/bin in the
path? All the binaries that omnibus installs are using the right
interpreter on the shebang line, so this should not be needed at all.

That should be the case, and that may now be the case, but as of the time the ticket for CHEF-2578 was filed, that was not the case.

If you can convince me that the issue in this ticket is resolved and that you’re not creating any symlinks from outside of /opt/opscode into subdirectories of that directory structure, and you’re not dependent on people making sure that they put /opt/opscode/embedded/bin into their $PATH before anything else (like /usr/bin), then I’ll be glad to shut up.


Brad Knowles <bknowles@ihiji.com (mailto:bknowles@ihiji.com)>
This ohai issue is unrelated to omnibus or any other installation method. What I’m considering here is how ohai detects information about what ruby interpreter you have installed–ohai is just the messenger. I’m not personally working on the omnibus project (though I’m excited about it), so I’m not up to speed on the considerations for this pathing issue. But it’s only tangentially related to how ohai would detect what versions of ruby you have installed.


Dan DeLeo


#12

On Wednesday, August 31, 2011 at 1:32 PM, Jesse Nelson wrote:

I see the need to differentiate the chef ruby (using rbconf) to figure
out what chef is running under. assuming you may need this for
developing some in-recipe things. I.e. i want to detect ruby 1.8 or
1.9 in my recipe. I know we can do that other ways in the recipe but
node['ruby"][“chef”][“version”] makes sense to me.

I think pulling all other rubies is also good. For the times when i
want to unsure some other script code or framework will work.
node[“ruby”][“system”][“jruby”]

I also see that you would want to detect the available rubies under
rvm. node[“ruby”][“rvm”] for similar reasons.

For the sake of argument, you could use RUBY_VERSION to get the version of ruby that your code is running under. But this will be less obvious to Chef users who are used to getting that data via the node object (via ohai).

I think rvm support in ohai would be great. Does it belong in the core ruby plugin, or in a separate plugin?

I also see merit in reporting all the rubies that ohai can find. Is there a clean way to do this without breaking existing recipes that use ohai’s current ruby plugin? What should ohai search for?


Dan DeLeo

On Wed, Aug 31, 2011 at 1:03 PM, Daniel DeLeo <dan@kallistec.com (mailto:dan@kallistec.com)> wrote:

On Wednesday, August 31, 2011 at 12:59 PM, Brad Knowles wrote:

On Aug 31, 2011, at 2:52 PM, Daniel DeLeo wrote:

So, if your ohai attributes for languages/ruby are empty, what breaks?

In my case, everything related to Chef. The system-provided version of ruby & rubygems is old enough that it will not work with anything related to Chef, and yet there are other parts of the system that are dependent on that older version being installed and being installed in the system-standard place. And they probably also do stupid things like assuming that the path is always set up correctly so if they just shell out to “ruby” it will pick up the right system-provided version.

So, I can’t upgrade the system-provided version of ruby & ruby-gems, but I have to have the newer version of ruby & ruby-gems in order to get Chef & ohai to work.

Both code bases want to assume that there is one and only one installation of ruby on the system, and it is necessarily the version that they need.

That’s a majorly, heinously, bad assumption.

You can’t go back and fix all the legacy code that wants to make that one-and-only-one assumption, but you can at least ensure that all the new code will always be able to find the right version that it actually needs, without breaking or otherwise causing any disturbance for any of the older code.

So, if you’re going to program in Ruby, and you’re going to build a system that requires newer version of the code than what we can get with a system-standard install, then the requirement falls onto you to make sure that you make things work properly. Don’t break my existing production systems with your new code, and don’t make me jump through hoops to get two incompatible versions installed.


Brad Knowles <bknowles@ihiji.com (mailto:bknowles@ihiji.com)>
The change that I’m talking about is only related to how ohai collects data about what ruby you have installed on the system. In Chef you can access this data as node[:languages][:ruby]. Are you currently depending on this data to return information about the system ruby?


Dan DeLeo


#13

On Aug 31, 2011, at 8:19 PM, Daniel DeLeo wrote:

This ohai issue is unrelated to omnibus or any other installation method. What I’m considering here is how ohai detects information about what ruby interpreter you have installed–ohai is just the messenger. I’m not personally working on the omnibus project (though I’m excited about it), so I’m not up to speed on the considerations for this pathing issue. But it’s only tangentially related to how ohai would detect what versions of ruby you have installed.

Sorry, I misunderstood. I was under the impression that you were working on how ohai itself was working and how it was trying to figure out what the “right” version of ruby was for it to be using – i.e., the system provided ruby vs. the embedded version, etc…

I can see that some people would expect a single canonical result, whereas others would prefer a multivariate result that includes all the versions of ruby that can be found on the system. However, I don’t know how deeply you should go inspecting the system to try to find all the various hidden copies of ruby – e.g., should the omnibus embedded version be returned in the list of versions of ruby that are found?


Brad Knowles bknowles@ihiji.com