Chef, CentOS/RedHat, and rubygem-rest-client updates

Chef Infra (archive)
1

What's the status of chef on CentOS/RedHat? We've currently installed
chef as described at

http://wiki.opscode.com/display/chef/Installation+on+RHEL+and+CentOS+5+with+RPMs

but I get the feeling that chef isn't very well supported on
CentOS/RedHat. For example, the latest RPM packaged version is 0.9.8,
which is quite old, and isn't packaged by Opscode anyway. (Unlike the
Debian/Ubuntu packages.) One of its dependencies is
rubygem-rest-client (from EPEL), and this was updated to version 1.6.1
in the last day or so, which breaks chef-client 0.9.8:

chef-client

/usr/lib/ruby/site_ruby/1.8/rubygems.rb:827:in
`report_activate_error': RubyGem version error: rest-client(1.6.1 not

= 1.0.4, <= 1.5.1) (Gem::LoadError)
from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:261:in activate' from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:296:in activate'
from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:295:in each' from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:295:in activate'
from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:68:in `gem'
from /usr/bin/chef-client:18

Will chef + CentOS be a viable combination in the future, or should I
look to moving to a different distribution? (Or figure out how to
install/package chef myself, which might be about as much work...)

Cheers,
Michael

--
http://beebo.org

2

I run Chef 0.9.14 on CentOS quite happily. A few caveats:

  • Upstream Ruby 1.8.7 (not the Enterprise Ruby equivalent) is needed for
    the server -- I use it everywhere, with a StackScript (my VPS provider is
    Linode; these are their paradigm for post-installation scriptage) installing
    the interpreter in /opt/ruby when my nodes are provisioned. (I also have
    some equivalent veewee scripts for building appropriate Vagrant baseboxes,
    and can publish these if desired).
  • Use gems, not RPMs, for package management.
  • Expect to need to modify many of the Opscode recipes.

On Fri, Mar 25, 2011 at 7:07 AM, Michael Stillwell mjs@beebo.org wrote:

What's the status of chef on CentOS/RedHat? We've currently installed
chef as described at

http://wiki.opscode.com/display/chef/Installation+on+RHEL+and+CentOS+5+with+RPMs

but I get the feeling that chef isn't very well supported on
CentOS/RedHat. For example, the latest RPM packaged version is 0.9.8,
which is quite old, and isn't packaged by Opscode anyway. (Unlike the
Debian/Ubuntu packages.) One of its dependencies is
rubygem-rest-client (from EPEL), and this was updated to version 1.6.1
in the last day or so, which breaks chef-client 0.9.8:

chef-client

/usr/lib/ruby/site_ruby/1.8/rubygems.rb:827:in
`report_activate_error': RubyGem version error: rest-client(1.6.1 not

= 1.0.4, <= 1.5.1) (Gem::LoadError)
from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:261:in activate' from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:296:in activate'
from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:295:in each' from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:295:in activate'
from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:68:in `gem'
from /usr/bin/chef-client:18

Will chef + CentOS be a viable combination in the future, or should I
look to moving to a different distribution? (Or figure out how to
install/package chef myself, which might be about as much work...)

Cheers,
Michael

--
http://beebo.org

3

Michael,

There are tons of us who are using chef-client on CentOS5. The trick,
which may not be documented cleanly in the wiki, is to use a custom
bootstrap file. Don't rely on any packaged RPMs of chef-client. The
server component is a bit more work.

One thing you'll find, however, is that most/many of the cookbooks are
designed around ubuntu. You'll spend a good bit of time working to
massage those into a RHEL/CentOS world view.

If you need any advice/tips, I'm usually on IRC during the day and can
throw a few gists/github repos your way to help out.

Just look for 'lusis' in Chef Infra (archive).

On Fri, Mar 25, 2011 at 8:07 AM, Michael Stillwell mjs@beebo.org wrote:

What's the status of chef on CentOS/RedHat? We've currently installed
chef as described at

http://wiki.opscode.com/display/chef/Installation+on+RHEL+and+CentOS+5+with+RPMs

but I get the feeling that chef isn't very well supported on
CentOS/RedHat. For example, the latest RPM packaged version is 0.9.8,
which is quite old, and isn't packaged by Opscode anyway. (Unlike the
Debian/Ubuntu packages.) One of its dependencies is
rubygem-rest-client (from EPEL), and this was updated to version 1.6.1
in the last day or so, which breaks chef-client 0.9.8:

chef-client

/usr/lib/ruby/site_ruby/1.8/rubygems.rb:827:in
`report_activate_error': RubyGem version error: rest-client(1.6.1 not

= 1.0.4, <= 1.5.1) (Gem::LoadError)
from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:261:in activate' from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:296:in activate'
from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:295:in each' from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:295:in activate'
from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:68:in `gem'
from /usr/bin/chef-client:18

Will chef + CentOS be a viable combination in the future, or should I
look to moving to a different distribution? (Or figure out how to
install/package chef myself, which might be about as much work...)

Cheers,
Michael

--
http://beebo.org

4

And beware CentOS's env_reset in the default /etc/sudoers file. More here:
http://wiki.opscode.com/display/chef/User+Environment+PATH+Sanity

Basically if you run "sudo chef-client" data will not be collected by Ohai correctly. This can easily be remedied by running "chef-client" as root...or having a custom "/etc/sudoers" file.

Seth

--
Opscode, Inc.
Seth Chisamore, Technical Evangelist
T: (404) 348-0505 E: schisamo@opscode.com
Twitter, IRC, Github: schisamo

On Friday, March 25, 2011 at 10:37 AM, John E. Vincent (lusis) wrote:

Michael,

There are tons of us who are using chef-client on CentOS5. The trick,
which may not be documented cleanly in the wiki, is to use a custom
bootstrap file. Don't rely on any packaged RPMs of chef-client. The
server component is a bit more work.

One thing you'll find, however, is that most/many of the cookbooks are
designed around ubuntu. You'll spend a good bit of time working to
massage those into a RHEL/CentOS world view.

If you need any advice/tips, I'm usually on IRC during the day and can
throw a few gists/github repos your way to help out.

Just look for 'lusis' in Chef Infra (archive).

On Fri, Mar 25, 2011 at 8:07 AM, Michael Stillwell mjs@beebo.org wrote:

What's the status of chef on CentOS/RedHat? We've currently installed
chef as described at

http://wiki.opscode.com/display/chef/Installation+on+RHEL+and+CentOS+5+with+RPMs

but I get the feeling that chef isn't very well supported on
CentOS/RedHat. For example, the latest RPM packaged version is 0.9.8,
which is quite old, and isn't packaged by Opscode anyway. (Unlike the
Debian/Ubuntu packages.) One of its dependencies is
rubygem-rest-client (from EPEL), and this was updated to version 1.6.1
in the last day or so, which breaks chef-client 0.9.8:

chef-client

/usr/lib/ruby/site_ruby/1.8/rubygems.rb:827:in
`report_activate_error': RubyGem version error: rest-client(1.6.1 not

= 1.0.4, <= 1.5.1) (Gem::LoadError)
from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:261:in activate' from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:296:in activate'
from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:295:in each' from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:295:in activate'
from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:68:in `gem'
from /usr/bin/chef-client:18

Will chef + CentOS be a viable combination in the future, or should I
look to moving to a different distribution? (Or figure out how to
install/package chef myself, which might be about as much work...)

Cheers,
Michael

--
http://beebo.org

5

Ohai!

On Mar 25, 2011, at 8:37 AM, John E. Vincent (lusis) wrote:

One thing you'll find, however, is that most/many of the cookbooks are
designed around ubuntu. You'll spend a good bit of time working to
massage those into a RHEL/CentOS world view.

And we're working to improve those in the near future :-).

--
Opscode, Inc
Joshua Timberman, Technical Evangelist
IRC, Skype, Twitter, Github: jtimberman

6

+1 for better RHEL/CentOS support and supported RPMs especially. I
really like Chef but it's a hard sell at a company with hundreds or
thousands of RHEL servers at the moment. A couple of you mentioned
only using gems, but going that route I seem to keep hitting Ruby path
issues. Plus I don't really like the idea of having to install
compilers and devel packages on all my prod machines so they can gem
install various libraries. Wondering if I need to break down and just
roll my own RPMs until Opscode has a supported yum repo.

KC Braunschweig

On Fri, Mar 25, 2011 at 10:14 AM, Joshua Timberman joshua@opscode.com wrote:

Ohai!
And we're working to improve those in the near future :-).

7

I maintain RPMs targeted towards Fedora. They should work on RHEL6 /
SL6 and Centos6 whenever they get around to putting it out

http://yum.afistfulofservers.net/affs-chef/fedora/13/x86_64/

-s

On Fri, Mar 25, 2011 at 1:46 PM, KC Braunschweig
kcbraunschweig@gmail.com wrote:

+1 for better RHEL/CentOS support and supported RPMs especially. I
really like Chef but it's a hard sell at a company with hundreds or
thousands of RHEL servers at the moment. A couple of you mentioned
only using gems, but going that route I seem to keep hitting Ruby path
issues. Plus I don't really like the idea of having to install
compilers and devel packages on all my prod machines so they can gem
install various libraries. Wondering if I need to break down and just
roll my own RPMs until Opscode has a supported yum repo.

KC Braunschweig

On Fri, Mar 25, 2011 at 10:14 AM, Joshua Timberman joshua@opscode.com wrote:

Ohai!
And we're working to improve those in the near future :-).

8

We use Chef with CentOS 5 at Plaxo. CentOS 4 also, although the init
script is a bit broken but it was a minor fix.

I agree that the packages are heavily Ubuntu-oriented. But Plaxo had its
own way of doing things, so I had to roll out my own cookbooks anyhow.
(That's why it's probably not worth sharing with the community, because
they're not really generically applicable).

I'm using 0.9.14 and it's been working out fine. I started with EPEL+ELFF
RPMs, and then I started building my own rubygems-chef RPMs as needed.
You'd also want to upgrade CouchDB, because the EPEL's CouchDB is 0.11.2,
and it's somewhat unstable. I used EPEL's SRPM as the basis for 1.0.2, and
it's been running fine since then.

I rely on my own shell script to bootstrap. We have physical servers and
VMWare Vms, and there's no nice way to do this via knife at this point.
You can install Chef and assign a run_list to the host with kickstart's
post script, but I find that you'd have to be pretty disciplined about
pre-assigning proper hostname and IP in DHCP.

-Paul

On 3/25/11 5:07 AM, "Michael Stillwell" mjs@beebo.org wrote:

What's the status of chef on CentOS/RedHat? We've currently installed
chef as described at

http://wiki.opscode.com/display/chef/Installation+on+RHEL+and+CentOS+5+wit
h+RPMs

but I get the feeling that chef isn't very well supported on
CentOS/RedHat. For example, the latest RPM packaged version is 0.9.8,
which is quite old, and isn't packaged by Opscode anyway. (Unlike the
Debian/Ubuntu packages.) One of its dependencies is
rubygem-rest-client (from EPEL), and this was updated to version 1.6.1
in the last day or so, which breaks chef-client 0.9.8:

chef-client

/usr/lib/ruby/site_ruby/1.8/rubygems.rb:827:in
`report_activate_error': RubyGem version error: rest-client(1.6.1 not

= 1.0.4, <= 1.5.1) (Gem::LoadError)
from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:261:in activate' from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:296:in activate'
from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:295:in each' from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:295:in activate'
from /usr/lib/ruby/site_ruby/1.8/rubygems.rb:68:in `gem'
from /usr/bin/chef-client:18

Will chef + CentOS be a viable combination in the future, or should I
look to moving to a different distribution? (Or figure out how to
install/package chef myself, which might be about as much work...)

Cheers,
Michael

--
http://beebo.org

9

KC,

As far as rolling your own RPMS and needing compilers, I would HIGHLY
suggest looking at FPM to roll your RPMS. *I've actually built a few
scripts to create ruby 1.9.2 RPMS for RHEL5 boxen for each arch and it
would be just as easy (because FPM supports gems to rpm) to use the
same "build" system to package the chef-client and dependent gems.

Another option that I've been trying to push my various opscode
friends into is trying to ensure that at a minimum, the client works
under jruby. Then it's an even easier packaging scenario.

On Fri, Mar 25, 2011 at 1:46 PM, KC Braunschweig
kcbraunschweig@gmail.com wrote:

+1 for better RHEL/CentOS support and supported RPMs especially. I
really like Chef but it's a hard sell at a company with hundreds or
thousands of RHEL servers at the moment. A couple of you mentioned
only using gems, but going that route I seem to keep hitting Ruby path
issues. Plus I don't really like the idea of having to install
compilers and devel packages on all my prod machines so they can gem
install various libraries. Wondering if I need to break down and just
roll my own RPMs until Opscode has a supported yum repo.

KC Braunschweig

On Fri, Mar 25, 2011 at 10:14 AM, Joshua Timberman joshua@opscode.com wrote:

Ohai!
And we're working to improve those in the near future :-).

10

Now that you mention hundreds or thousands of servers...
What's the biggest that you find Chef can handle? I probably have about
100-ish hosts in Chef, and probably more in the near future. I have 6 thin
processes running chef-server and sometimes I find that it's pretty slow.
Running on 4 cores, 16GB RAM (excessive I know), and I find that it's
pretty CPU-bound, often busy at chef-server.

-Paul

On 3/25/11 11:00 AM, "Sean OMeara" someara@gmail.com wrote:

I maintain RPMs targeted towards Fedora. They should work on RHEL6 /
SL6 and Centos6 whenever they get around to putting it out

http://yum.afistfulofservers.net/affs-chef/fedora/13/x86_64/

-s

On Fri, Mar 25, 2011 at 1:46 PM, KC Braunschweig
kcbraunschweig@gmail.com wrote:

+1 for better RHEL/CentOS support and supported RPMs especially. I
really like Chef but it's a hard sell at a company with hundreds or
thousands of RHEL servers at the moment. A couple of you mentioned
only using gems, but going that route I seem to keep hitting Ruby path
issues. Plus I don't really like the idea of having to install
compilers and devel packages on all my prod machines so they can gem
install various libraries. Wondering if I need to break down and just
roll my own RPMs until Opscode has a supported yum repo.

KC Braunschweig

On Fri, Mar 25, 2011 at 10:14 AM, Joshua Timberman joshua@opscode.com
wrote:

Ohai!
And we're working to improve those in the near future :-).

11

On Fri, Mar 25, 2011 at 2:11 PM, Paul Choi paulchoi@plaxo.com wrote:

Now that you mention hundreds or thousands of servers...
What's the biggest that you find Chef can handle? I probably have about
100-ish hosts in Chef, and probably more in the near future. I have 6 thin
processes running chef-server and sometimes I find that it's pretty slow.
Running on 4 cores, 16GB RAM (excessive I know), and I find that it's
pretty CPU-bound, often busy at chef-server.

-Paul

Have you moved Couch/Rabbit/solr to it's own distinct resources yet?
That would be my first step if not.

12

On Fri, Mar 25, 2011 at 11:11 AM, Paul Choi paulchoi@plaxo.com wrote:

Now that you mention hundreds or thousands of servers...
What's the biggest that you find Chef can handle?

We don't have much of a rollout yet. Still working with the issues
mentioned. Definitely post your experiences as you scale your Chef
server though as that's next on the list to tackle. With so many folks
using the platform, those of us running chef server ourselves have to
stick together. :slight_smile:

KC

13

Is there any documentation on specifying a separate host for
Couchdb/rabbit or solr? Server.rb assumes everything is localhost at this
point. FYI, I'm still running 0.9.8 on server, and 0.9.14 on clients.

-Paul

On 3/25/11 11:16 AM, "John E. Vincent (lusis)"
lusis.org+chef-list@gmail.com wrote:

On Fri, Mar 25, 2011 at 2:11 PM, Paul Choi paulchoi@plaxo.com wrote:

Now that you mention hundreds or thousands of servers...
What's the biggest that you find Chef can handle? I probably have about
100-ish hosts in Chef, and probably more in the near future. I have 6
thin
processes running chef-server and sometimes I find that it's pretty
slow.
Running on 4 cores, 16GB RAM (excessive I know), and I find that it's
pretty CPU-bound, often busy at chef-server.

-Paul

Have you moved Couch/Rabbit/solr to it's own distinct resources yet?
That would be my first step if not.

14

Well, I found that you can set "couchdb_url" in /etc/chef/server.rb... By
grepping through chef gem's lib dir...

Although, if there's a doc page that I can just look at for available
params, without grepping through the source, that'd be great. Or if
there's some ruby way of generating documentation (like rdoc?)? But I'm
not all that familiar with Ruby ecosystem.

And after upgrading couchdb from 0.11 to 1.0.2, couchdb seems slower...
I'm going to have to dig into that deeper.

-Paul

On 3/25/11 11:24 AM, "Paul Choi" paulchoi@plaxo.com wrote:

Is there any documentation on specifying a separate host for
Couchdb/rabbit or solr? Server.rb assumes everything is localhost at this
point. FYI, I'm still running 0.9.8 on server, and 0.9.14 on clients.

-Paul

On 3/25/11 11:16 AM, "John E. Vincent (lusis)"
lusis.org+chef-list@gmail.com wrote:

On Fri, Mar 25, 2011 at 2:11 PM, Paul Choi paulchoi@plaxo.com wrote:

Now that you mention hundreds or thousands of servers...
What's the biggest that you find Chef can handle? I probably have about
100-ish hosts in Chef, and probably more in the near future. I have 6
thin
processes running chef-server and sometimes I find that it's pretty
slow.
Running on 4 cores, 16GB RAM (excessive I know), and I find that it's
pretty CPU-bound, often busy at chef-server.

-Paul

Have you moved Couch/Rabbit/solr to it's own distinct resources yet?
That would be my first step if not.

15

Off hand, I'm not sure. Worse case you could iptable the ports locally
to the remote servers.

On Fri, Mar 25, 2011 at 2:24 PM, Paul Choi paulchoi@plaxo.com wrote:

Is there any documentation on specifying a separate host for
Couchdb/rabbit or solr? Server.rb assumes everything is localhost at this
point. FYI, I'm still running 0.9.8 on server, and 0.9.14 on clients.

-Paul

On 3/25/11 11:16 AM, "John E. Vincent (lusis)"
lusis.org+chef-list@gmail.com wrote:

On Fri, Mar 25, 2011 at 2:11 PM, Paul Choi paulchoi@plaxo.com wrote:

Now that you mention hundreds or thousands of servers...
What's the biggest that you find Chef can handle? I probably have about
100-ish hosts in Chef, and probably more in the near future. I have 6
thin
processes running chef-server and sometimes I find that it's pretty
slow.
Running on 4 cores, 16GB RAM (excessive I know), and I find that it's
pretty CPU-bound, often busy at chef-server.

-Paul

Have you moved Couch/Rabbit/solr to it's own distinct resources yet?
That would be my first step if not.

16

On Fri, Mar 25, 2011 at 1:07 PM, Michael Stillwell mjs@beebo.org wrote:

What's the status of chef on CentOS/RedHat? We've currently installed
chef as described at

http://wiki.opscode.com/display/chef/Installation+on+RHEL+and+CentOS+5+with+RPMs

but I get the feeling that chef isn't very well supported on
CentOS/RedHat. For example, the latest RPM packaged version is 0.9.8,
which is quite old, and isn't packaged by Opscode anyway. (Unlike the

Hi Michael,

As others have said, chef is working very nicely on CentOS 5, both as a
server and as a client.

I have built my own RPMs of chef 0.9.14, using a bunch of files from
ELFF (thanks guys!). They are available for CentOS 5 i386 and x86_64
here: http://pakk.96b.it/repositories/el5/
My packages are not really meant to be integrated with RedHat/CentOS'
default ruby though, as I am also packaging ruby 1.9 (with a recent GC
patch).

Installation is not well (err... at all) documented, but:

rpm -ivh http://pakk.96b.it/repositories/el5//pakk-release.rpm

yum install chef-client (or chef-server)

should work, where arch is either i386 or x86_64.
Please note that you need to configure EPEL's repositories before.

I am the only user of those packages (I have not yet publicly announced
the repository), and they work for me, but if anyone has any problem
with them, I'd be glad to help. Patches are even more welcomed, as
the SPEC files are published here: https://github.com/eolamey/pakk

17

On Mar 25, 2011, at 2:11 PM, Paul Choi wrote:

Running on 4 cores, 16GB RAM (excessive I know), and I find that it's
pretty CPU-bound, often busy at chef-server.

We use nginx in front of unicorn on our chef-server (16 ruby processes). We still run couchdb and solr on the same server. We replicated couch to a "cold" backup server. We run about the same number of clients now twice an hour and see ~ 10-15% CPU usage on an old 4-core server we currently use for chef-server. We will soon have a few thousand on it, so I'll let you know. I think letting nginx keep up with the client connections and keepalives helps a good bit. The ruby process only have to process well formed, very fast requests as nginx buffers both the request and the response, plus nginx takes up only a few MB of RAM.

--Brian