Gem install on Ubuntu 12

Hi all,
This is a bug report/heads up, not a support request - my issue is solved.

I ran into an issue a couple of days ago with the default (omnibus?)
chef-client install on Ubuntu 12.04. Basically, it somehow ends up with two
versions of Gems (or just Ruby? - I'm not a Ruby person) installed, in
different locations: Gems 1.8 in /var/lib/gems, and 1.9.1 somewhere else
(/opt/...something...chef-embedded I think). The upshot of this was that a
cookbook that worked on Ubuntu 10.04 didn't work - a postgres gem ("pg")
installed, but couldn't be found by a later step that required it. It was
present, but not available to the ruby environment in use at the time.

People on IRC helped with a workaround: installing chef-client with the
specific distribution, "-d ubuntu12.04".

I'm just mentioning this so the relevant people are aware and can hopefully
fix it. The recipe I ran into problems with is here:

Steve

I think the thing is, if you want to run Chef with Ubuntu 12, you want to
switch the default version of ruby away from ruby1.8. So you'd want to do
"update-alternatives --config ruby" to point the default to ruby1.9 (and
associated binaries such as "gem", "ri", "rdoc", etc.).

~~ StormeRider ~~

"With hearts immortal, we stand before our lives
A soul against oblivion, forever asking why
This upside down symphony in a paradox called life
Our hearts immortal
What you give to love, will never die"

On Sun, Jul 22, 2012 at 9:29 PM, Steve Bennett stevage@gmail.com wrote:

Hi all,
This is a bug report/heads up, not a support request - my issue is
solved.

I ran into an issue a couple of days ago with the default (omnibus?)
chef-client install on Ubuntu 12.04. Basically, it somehow ends up with two
versions of Gems (or just Ruby? - I'm not a Ruby person) installed, in
different locations: Gems 1.8 in /var/lib/gems, and 1.9.1 somewhere else
(/opt/...something...chef-embedded I think). The upshot of this was that a
cookbook that worked on Ubuntu 10.04 didn't work - a postgres gem ("pg")
installed, but couldn't be found by a later step that required it. It was
present, but not available to the ruby environment in use at the time.

People on IRC helped with a workaround: installing chef-client with the
specific distribution, "-d ubuntu12.04".

I'm just mentioning this so the relevant people are aware and can
hopefully fix it. The recipe I ran into problems with is here:
GitHub - stevage/mytardis-chef: Chef repository for installing MyTardis

Steve

Doing that would change the system ruby to the one installed by omnibus which defeats many of the purposes of installing another non-system ruby for use with chef.

-ash

On 23 Jul 2012, at 05:48, Morgan Blackthorne wrote:

I think the thing is, if you want to run Chef with Ubuntu 12, you want to switch the default version of ruby away from ruby1.8. So you'd want to do "update-alternatives --config ruby" to point the default to ruby1.9 (and associated binaries such as "gem", "ri", "rdoc", etc.).

~~ StormeRider ~~

"With hearts immortal, we stand before our lives
A soul against oblivion, forever asking why
This upside down symphony in a paradox called life
Our hearts immortal
What you give to love, will never die"

On Sun, Jul 22, 2012 at 9:29 PM, Steve Bennett stevage@gmail.com wrote:
Hi all,
This is a bug report/heads up, not a support request - my issue is solved.

I ran into an issue a couple of days ago with the default (omnibus?) chef-client install on Ubuntu 12.04. Basically, it somehow ends up with two versions of Gems (or just Ruby? - I'm not a Ruby person) installed, in different locations: Gems 1.8 in /var/lib/gems, and 1.9.1 somewhere else (/opt/...something...chef-embedded I think). The upshot of this was that a cookbook that worked on Ubuntu 10.04 didn't work - a postgres gem ("pg") installed, but couldn't be found by a later step that required it. It was present, but not available to the ruby environment in use at the time.

People on IRC helped with a workaround: installing chef-client with the specific distribution, "-d ubuntu12.04".

I'm just mentioning this so the relevant people are aware and can hopefully fix it. The recipe I ran into problems with is here: GitHub - stevage/mytardis-chef: Chef repository for installing MyTardis

Steve

I fear you've seen a case where an out of date cookbook in an old
version of a snapshot repository hasn't been updated to use 'chef_gem'
resource to install gem to the Opscode Chef embedded Ruby
installation.

In the culprit recipe, if you change gem_package "pg" etc to chef_gem
"pg", that should remedy it!

Most of the Opscode cookbooks on the community site have been updated
to support the new Omnibus based installations.

At Heavy Water, we developed the 'chef_gem' cookbook to allow "Forward
Compatibility" for ~> 0.10.0 to use the 'chef_gem' resource.

I'd additionally suggest the 'omnibus_updater' and 'gem_installer'
cookbooks too.

Hope this helps,

--AJ

http://community.opscode.com/cookbooks/chef_gem
http://community.opscode.com/cookbooks/gem_installer
http://community.opscode.com/cookbooks/omnibus_updater

On 23 July 2012 19:38, Ash Berlin ash_opscode@firemirror.com wrote:

Doing that would change the system ruby to the one installed by omnibus
which defeats many of the purposes of installing another non-system ruby for
use with chef.

-ash

On 23 Jul 2012, at 05:48, Morgan Blackthorne wrote:

I think the thing is, if you want to run Chef with Ubuntu 12, you want to
switch the default version of ruby away from ruby1.8. So you'd want to do
"update-alternatives --config ruby" to point the default to ruby1.9 (and
associated binaries such as "gem", "ri", "rdoc", etc.).

~~ StormeRider ~~

"With hearts immortal, we stand before our lives
A soul against oblivion, forever asking why
This upside down symphony in a paradox called life
Our hearts immortal
What you give to love, will never die"

On Sun, Jul 22, 2012 at 9:29 PM, Steve Bennett stevage@gmail.com wrote:

Hi all,
This is a bug report/heads up, not a support request - my issue is
solved.

I ran into an issue a couple of days ago with the default (omnibus?)
chef-client install on Ubuntu 12.04. Basically, it somehow ends up with two
versions of Gems (or just Ruby? - I'm not a Ruby person) installed, in
different locations: Gems 1.8 in /var/lib/gems, and 1.9.1 somewhere else
(/opt/...something...chef-embedded I think). The upshot of this was that a
cookbook that worked on Ubuntu 10.04 didn't work - a postgres gem ("pg")
installed, but couldn't be found by a later step that required it. It was
present, but not available to the ruby environment in use at the time.

People on IRC helped with a workaround: installing chef-client with the
specific distribution, "-d ubuntu12.04".

I'm just mentioning this so the relevant people are aware and can
hopefully fix it. The recipe I ran into problems with is here:
GitHub - stevage/mytardis-chef: Chef repository for installing MyTardis

Steve

On 23 July 2012 17:52, AJ Christensen aj@junglist.gen.nz wrote:

I fear you've seen a case where an out of date cookbook in an old
version of a snapshot repository hasn't been updated to use 'chef_gem'
resource to install gem to the Opscode Chef embedded Ruby
installation.

In the culprit recipe, if you change gem_package "pg" etc to chef_gem
"pg", that should remedy it!

Should gem_package be deprecated, then?

Most of the Opscode cookbooks on the community site have been updated
to support the new Omnibus based installations.

At Heavy Water, we developed the 'chef_gem' cookbook to allow "Forward
Compatibility" for ~> 0.10.0 to use the 'chef_gem' resource.

Whee, confusing. From http://community.opscode.com/cookbooks/chef_gem:

--
This cookbook is a transition cookbook aimed at helping move from the
pre-chef_gem era to the post chef_gem era. The chef_gem resource was
introducde in Chef 0.10.9, providing an easy mechanism for installing and
using gems required by Chef internally. When used within an omnibus
installation, chef_gems are installed within the embedded Ruby.

From the perspective of someone who just wants to get a cookbook to work,
this is a lot of confusing detail about the internals of Chef, gems, etc.
What is the "post-Chef_gem era"? Is chef_gem being phased out?

Steve

I do not believe that 'chef_gem' is being phased out. Opscode could
confirm -- Perhaps the documentation author (one of my team mates)
meant that the 'chef_gem' Heavy Water cookbook would no longer be
required post 10.12.0 or so.

In some states, you may have deployed Chef via gem to an embedded
system interpreter. In this case, the chef_gem cookbook,
omnibus_updater and gem_installer cookbooks will allow you to
transition to newer cookbooks that use the 'chef_gem' method of
installing a gem to the embedded Opscode Chef interpreter instead of
the system interpreter (which generally requires another Ruby
installation).

In some states, you may have deployed a system and an updated Chef
'full stack' omnibus installation may have been released, again, our
cookbooks will help.

In alternate states, you may have deployed multiple rubies to a single
system and require the capability to install gems to any-which of
them; again, our cookbooks will allow you to achieve this.

I can't possibly know which of these states your infrastructure is in;
I leave the adventure up to you :slight_smile:

I neglected to mention one, possibly helpful in the event you require
a system Ruby installation [0]

--AJ

[0] http://community.opscode.com/cookbooks/ruby_installer

On 23 July 2012 20:05, Steve Bennett stevage@gmail.com wrote:

On 23 July 2012 17:52, AJ Christensen aj@junglist.gen.nz wrote:

I fear you've seen a case where an out of date cookbook in an old
version of a snapshot repository hasn't been updated to use 'chef_gem'
resource to install gem to the Opscode Chef embedded Ruby
installation.

In the culprit recipe, if you change gem_package "pg" etc to chef_gem
"pg", that should remedy it!

Should gem_package be deprecated, then?

Most of the Opscode cookbooks on the community site have been updated
to support the new Omnibus based installations.

At Heavy Water, we developed the 'chef_gem' cookbook to allow "Forward
Compatibility" for ~> 0.10.0 to use the 'chef_gem' resource.

Whee, confusing. From http://community.opscode.com/cookbooks/chef_gem:

--
This cookbook is a transition cookbook aimed at helping move from the
pre-chef_gem era to the post chef_gem era. The chef_gem resource was
introducde in Chef 0.10.9, providing an easy mechanism for installing and
using gems required by Chef internally. When used within an omnibus
installation, chef_gems are installed within the embedded Ruby.

From the perspective of someone who just wants to get a cookbook to work,
this is a lot of confusing detail about the internals of Chef, gems, etc.
What is the "post-Chef_gem era"? Is chef_gem being phased out?

Steve

On Mon, Jul 23, 2012 at 12:29 AM, Steve Bennett stevage@gmail.com wrote:

I ran into an issue a couple of days ago with the default (omnibus?)
chef-client install on Ubuntu 12.04. Basically, it somehow ends up with two
versions of Gems (or just Ruby? - I'm not a Ruby person) installed, in
different locations: Gems 1.8 in /var/lib/gems, and 1.9.1 somewhere else
(/opt/...something...chef-embedded I think). The upshot of this was that a
cookbook that worked on Ubuntu 10.04 didn't work - a postgres gem ("pg")
installed, but couldn't be found by a later step that required it. It was
present, but not available to the ruby environment in use at the time.

This is by design. Omnibus Chef ships with its own version of Ruby. It
is stable for running Chef, but should never be used for any other
purpose. You can choose to not have a system ruby installed at all if
you wish. You can install your Ruby using rvm, rbenv, or any other
magic you want, and it won't bother Chef's own version of Ruby.

People on IRC helped with a workaround: installing chef-client with the
specific distribution, "-d ubuntu12.04".

This particular template installs Chef under the system's Rubygems,
which is installed as a package from upstream. This is perfectly fine
as long as that's the version of Ruby you want to use for everything
else you do with ruby, provided of course that you do something else
with Ruby besides running Chef.

On Mon, Jul 23, 2012 at 12:48 AM, Morgan Blackthorne
stormerider@gmail.com wrote:

I think the thing is, if you want to run Chef with Ubuntu 12, you want to
switch the default version of ruby away from ruby1.8. So you'd want to do
"update-alternatives --config ruby" to point the default to ruby1.9 (and
associated binaries such as "gem", "ri", "rdoc", etc.).

Chef runs fine with Ruby 1.8. Future versions of Debian and Ubuntu
will have not only a newer Ruby, but a nicer redesign of the packaging
scheme for managing Ruby libraries and different versions of the Ruby
API.

On Mon, Jul 23, 2012 at 4:05 AM, Steve Bennett stevage@gmail.com wrote:

Should gem_package be deprecated, then?

In the course of developing the Omnibus installer we realized that the
gem_package provider was being used for two different use cases; one
for installing a gem to your system to use, such as rails to run a web
service, and the other to install a gem to use in Chef recipes, such
as the mysql gem to connect to a database and modify it. Thus, we
created chef_gem in CHEF-2879 [1].

gem_package should be used in cookbooks when you want to install a gem
to your system for use outside of Chef. Since we don't know what Ruby
is your system Ruby, we use the Ruby that Chef is running under unless
you specify otherwise with the gem_binary attribute. However, under
Omnibus, we know that you're running Chef under the copy of Ruby that
ships with Omnibus and that it is not your system ruby, so we walk
your path in search of a ruby binary.

chef_gem should be used in cookbooks when you want to use a gem in a
chef recipe. It will install a gem immediately, which means in the
compilation phase of cookbook processing so the gem will be available
when you run other Ruby code (outside of a ruby block resource) in
your cookbook. You cannot change which gem binary this provider uses
because it always uses the Ruby that Chef is running under. This
provider does all the magic that you used to have to add to
gem_package to do this, such as running Gem.clear_paths. This is
simpler and reduces code duplication.

That the postgresql cookbook is broken under omnibus is a known bug
filed under COOK-1406 [2]. This is a somewhat unique issue such that
the postgres gem compiles extensions which use openssl and Omnibus has
its own copy of openssl. There are other dependencies required to
compile the postgresql extension which are installed to the system
using packages. However those packages are compiled against a
difference version of openssl than is included with Omnibus. It is not
trivial to prevent the Rubygems included with omnibus to not include
that version of openssl when compiling. I've done some testing with
this and talked to some of the folks smarter than me at Opscode and we
don't see a better path forward than compiling those bits from source
inside of Omnibus. The postgresql cookbook will be updated to do so.

--
Bryan McLellan | opscode | technical program manager, open source
(c) 206.607.7108 | (t) @btmspox | (b) http://blog.loftninjas.org

[1] http://tickets.opscode.com/browse/CHEF-2879
[2] http://tickets.opscode.com/browse/COOK-1406