Best I can tell, Joshua Timberman has finished the refactoring of mysql,
postgresql, and database cookbooks, and chef 10.14 has been released,
both of which were precursors to untangling the chicken-and-egg mess of
needing a libpq that’s compatible with Omnibus ruby.
Soooo… what’s left? I’m currently installing chef via ubuntu-gems, but
that creates its own problems when I replace system ruby with rvm…
Jay
On Sep 24, 2012, at 11:30 AM, Jay Levitt jay.levitt@gmail.com wrote:
Best I can tell, Joshua Timberman has finished the refactoring of mysql, postgresql, and database cookbooks, and chef 10.14 has been released, both of which were precursors to untangling the chicken-and-egg mess of needing a libpq that's compatible with Omnibus ruby.
Soooo.. what's left? I'm currently installing chef via ubuntu-gems, but that creates its own problems when I replace system ruby with rvm..
IIRC, the point of Omnibus was to try to help make sure that ... things ... like rvm and rbenv would not be necessary with Chef. For a developer, tools like rvm and rbenv can be handy, but from a systems perspective they are a nightmare. If you have no choice, my understanding is that rbenv is not as bad as rvm, because at least rbenv doesn't try to do things via the environment settings which are supposed to change as you cd into and out of various directories or when otherwise encountering .rvmrc files.
Maybe the better solution is to try to fix the pg gem so that it finds libpq in the correct location?
I know there are postgresql cookbooks for Chef. I'm really surprised that the pg gem can't find libpq when run under Omnibus chef.
--
Brad Knowles brad@shub-internet.org
LinkedIn Profile: http://tinyurl.com/y8kpxu
Seems there's multiple different problems in that ticket that got
conflated together.
The postgresql::ruby recipe should work on systems that have older
openssl's in order to install build-essentials at compile-time, to
install the system pg libraries and headers and to compile and install
the pg gem as a 'chef_gem'. That solves the first issue in that ticket.
The problem comes with newer systems where the openssl in the system is
incompatible with the one that omnibus ships. The problem there is that
the ruby binary in omnibus is linked against the openssl library in
omibus, but the pg libraries on the system are linked against the system
pg libraries and if they use different ABIs to talk to openssl when you
try to 'chef_gem' install the pg gem you'll wind up with the omnibus
version of openssl linked in, and the pg libraries will barf with
unresolved symbols because they want the later version of the openssl libs.
Upgrading the openssl libraries in omnibus would just shove this problem
around to different distros (now centos5/6, and ubuntu-10.04 would be
busted). Shipping pg binaries in omnibus is another solution but its a
bit annoying because we probably want to match the pg version against
the version shipping with the distro, and that hasn't been what omnibus
has been about, its been to ship a consistent version of the libraries
in it everywhere.
The better solution is probably to compile the pg libraries from source,
making sure to get -I and -L flags right to pick up the omnibus openssl
and install those in the omnibus directory. Do that at compile time
after installing build-essentials, but before calling chef_gem.
On 9/24/12 11:30 AM, Jay Levitt wrote:
Best I can tell, Joshua Timberman has finished the refactoring of
mysql, postgresql, and database cookbooks, and chef 10.14 has been
released, both of which were precursors to untangling the
chicken-and-egg mess of needing a libpq that's compatible with Omnibus
ruby.Soooo.. what's left? I'm currently installing chef via ubuntu-gems,
but that creates its own problems when I replace system ruby with rvm..Jay
Lamont Granquist wrote:
On 9/24/12 11:30 AM, Jay Levitt wrote:
Best I can tell, Joshua Timberman has finished the refactoring of mysql,
postgresql, and database cookbooks, and chef 10.14 has been released,
both of which were precursors to untangling the chicken-and-egg mess
of needing a libpq that's compatible with Omnibus ruby.Soooo.. what's left? I'm currently installing chef via ubuntu-gems, but
that creates its own problems when I replace system ruby with rvm..
The problem comes with newer systems where the openssl in the system is
incompatible with the one that omnibus ships. The problem there is that the
ruby binary in omnibus is linked against the openssl library in omibus, but
the pg libraries on the system are linked against the system pg libraries
and if they use different ABIs to talk to openssl when you try to 'chef_gem'
install the pg gem you'll wind up with the omnibus version of openssl linked
in, and the pg libraries will barf with unresolved symbols because they want
the later version of the openssl libs.Upgrading the openssl libraries in omnibus would just shove this problem
around to different distros (now centos5/6, and ubuntu-10.04 would be
busted). Shipping pg binaries in omnibus is another solution but its a bit
annoying because we probably want to match the pg version against the
version shipping with the distro, and that hasn't been what omnibus has been
about, its been to ship a consistent version of the libraries in it everywhere.
I'm not so sure about that. On the one hand, psql (and therefore libpq) is
considered backwards-compatible enough that committers just proposed
removing the "WARNING: Some features might not work" when you connect to
older servers.
On the other hand, this doesn't help if you have a newer version in the
distro, specifically a newer, incompatible server version that this libpq
needs to talk to. This can be solved by revving pg versions often enough in
omnibus.
On the third hand, nothing helps if your server is on a different machine,
and THAT machine has a newer version of the server than this machine's
libpq, and chef doesn't even try to protect you here. So why do we care
more if the server's on the same machine?
I'm new here, but this feels OK to me; we're (presumably) not doing complex
database access in chef recipes that requires bleeding-edge libpq (if you're
even on a bleeding-edge distro); we're creating a database and some roles
and other scaffoldy stuff.
What do you think?
The better solution is probably to compile the pg libraries from source,
making sure to get -I and -L flags right to pick up the omnibus openssl and
install those in the omnibus directory. Do that at compile time after
installing build-essentials, but before calling chef_gem.
Which cookbook should this go in? I could take a crack at it, though
anything that slows the first converge makes me sad, and Postgres compiles
take two minutes on a fast machine (though that's the whole server;
client-only may be faster).
On 9/25/12 6:40 AM, Jay Levitt wrote:
What do you think?
So far that just isn't what omnibus has been about. I understand the
argument, but then we need to be revving libpg and chasing after the
latest stable versions and that's a shift of what we've been doing with
omnibus which has been more about producing a stable platform for chef
itself to run.
The better solution is probably to compile the pg libraries from source,
making sure to get -I and -L flags right to pick up the omnibus
openssl and
install those in the omnibus directory. Do that at compile time after
installing build-essentials, but before calling chef_gem.Which cookbook should this go in? I could take a crack at it, though
anything that slows the first converge makes me sad, and Postgres
compiles take two minutes on a fast machine (though that's the whole
server; client-only may be faster).
It should be a postgresql cookbook recipe to install the client from source.
To speed this up, it could be compiled once, installed into a different
directory outside of /opt/opscode (but build with the -I and -L flags
pointed at the openssl in omnibus) then with chef_gem with appropriate
options to point at it, and you can fpm that libpg up into an rpm/deb
and install it from an internal mirror or from S3 or whatever. You
could probably even try to just fpm it up from inside of /opt/opscode,
but I don't know how messy that would be. Same goes for the pg gem
itself as well (and if you can just point fpm at the omnibus ruby, that
might be trivial to package the pg gem)
On Tue, Sep 25, 2012 at 7:50 PM, Lamont Granquist lamont@opscode.comwrote:
On 9/25/12 6:40 AM, Jay Levitt wrote:
The better solution is probably to compile the pg libraries from source,
making sure to get -I and -L flags right to pick up the omnibus openssl
and
install those in the omnibus directory. Do that at compile time after
installing build-essentials, but before calling chef_gem.Which cookbook should this go in? I could take a crack at it, though
anything that slows the first converge makes me sad, and Postgres compiles
take two minutes on a fast machine (though that's the whole server;
client-only may be faster).It should be a postgresql cookbook recipe to install the client from
source.To speed this up, it could be compiled once, installed into a different
directory outside of /opt/opscode (but build with the -I and -L flags
pointed at the openssl in omnibus) then with chef_gem with appropriate
options to point at it, and you can fpm that libpg up into an rpm/deb and
install it from an internal mirror or from S3 or whatever. You could
probably even try to just fpm it up from inside of /opt/opscode, but I
don't know how messy that would be. Same goes for the pg gem itself as well
(and if you can just point fpm at the omnibus ruby, that might be trivial
to package the pg gem)
Hi Jay,
did you actually come to fixing the postgres cookbook for omnibus installs?
Need it too, but don't want to look for further workaround if it has been
done already.
Cheers,
Torben