Anyone interested in a "hardware" cookbook?

AtomicPenguin (Eric Wolfe) has put a good bit of work into the yumrepo
recipes like vmware-tools and dell which detect and installs virtualization
and management tools specific to those hardware or virtualization platforms.

I would want the similar functionality for a VM running in VMware Fusion,
virtualbox, Xen, etc where adding the “recipe[hardware]” adds the
paravirtualization tools specific to that platform.

For machine running on actual bare metal, I want “recipe[hardware]” to
detect and install the hardware management tools, be they Dell, HP, IBM,
etc.

recipes

default – detect current hardware or virt platform and install related
tools
vmware-tools - install the paravirtualization tools for vmware
virtualbox - see vmware-tools
hp - install hp-health tools for bare metal machine
dell - install dell openmanage
ibm - …

anyone else interested in such a cookbook? I would only do the subset that
relates to the platforms i use.

By the way, credit for this idea belongs to jpalmer who is creating
something like this for puppet

I'll be interested, i can do the dell/openmange stuff.

On Thu, Dec 1, 2011 at 1:34 PM, Bryan Berry bryan.berry@gmail.com wrote:

AtomicPenguin (Eric Wolfe) has put a good bit of work into the yumrepo
recipes like vmware-tools and dell which detect and installs virtualization
and management tools specific to those hardware or virtualization platforms.

I would want the similar functionality for a VM running in VMware Fusion,
virtualbox, Xen, etc where adding the "recipe[hardware]" adds the
paravirtualization tools specific to that platform.

For machine running on actual bare metal, I want "recipe[hardware]" to
detect and install the hardware management tools, be they Dell, HP, IBM,
etc.

recipes

default -- detect current hardware or virt platform and install related
tools
vmware-tools - install the paravirtualization tools for vmware
virtualbox - see vmware-tools
hp - install hp-health tools for bare metal machine
dell - install dell openmanage
ibm - ....

anyone else interested in such a cookbook? I would only do the subset that
relates to the platforms i use.

By the way, credit for this idea belongs to jpalmer who is creating
something like this for puppet

For the bare metal part, I have a nice simple cookbook we're using for
installing Dell OMSA on a fleet of Dell servers. I'd be happy to
contribute it ...

On 11-12-01 03:04 AM, Bryan Berry wrote:

AtomicPenguin (Eric Wolfe) has put a good bit of work into the yumrepo
recipes like vmware-tools and dell which detect and installs
virtualization and management tools specific to those hardware or
virtualization platforms.

I would want the similar functionality for a VM running in VMware
Fusion, virtualbox, Xen, etc where adding the "recipe[hardware]" adds
the paravirtualization tools specific to that platform.

For machine running on actual bare metal, I want "recipe[hardware]" to
detect and install the hardware management tools, be they Dell, HP, IBM,
etc.

recipes

default -- detect current hardware or virt platform and install related
tools
vmware-tools - install the paravirtualization tools for vmware
virtualbox - see vmware-tools
hp - install hp-health tools for bare metal machine
dell - install dell openmanage
ibm - ....

anyone else interested in such a cookbook? I would only do the subset
that relates to the platforms i use.

By the way, credit for this idea belongs to jpalmer who is creating
something like this for puppet

--
Graham Stewart graham.stewart@utoronto.ca 416-550-2806
Network and Storage Services Manager, Information Technology Services
University of Toronto Libraries
130 St. George Street
Toronto, Ontario, Canada M5S 1A5

Graham, that would be great

On Thu, Dec 1, 2011 at 3:06 PM, Graham Stewart
graham.stewart@utoronto.cawrote:

For the bare metal part, I have a nice simple cookbook we're using for
installing Dell OMSA on a fleet of Dell servers. I'd be happy to contribute
it ...

On 11-12-01 03:04 AM, Bryan Berry wrote:

AtomicPenguin (Eric Wolfe) has put a good bit of work into the yumrepo
recipes like vmware-tools and dell which detect and installs
virtualization and management tools specific to those hardware or
virtualization platforms.

I would want the similar functionality for a VM running in VMware
Fusion, virtualbox, Xen, etc where adding the "recipe[hardware]" adds
the paravirtualization tools specific to that platform.

For machine running on actual bare metal, I want "recipe[hardware]" to
detect and install the hardware management tools, be they Dell, HP, IBM,
etc.

recipes

default -- detect current hardware or virt platform and install related
tools
vmware-tools - install the paravirtualization tools for vmware
virtualbox - see vmware-tools
hp - install hp-health tools for bare metal machine
dell - install dell openmanage
ibm - ....

anyone else interested in such a cookbook? I would only do the subset
that relates to the platforms i use.

By the way, credit for this idea belongs to jpalmer who is creating
something like this for puppet

--
Graham Stewart graham.stewart@utoronto.ca 416-550-2806
Network and Storage Services Manager, Information Technology Services
University of Toronto Libraries
130 St. George Street
Toronto, Ontario, Canada M5S 1A5

So right now, the existing yumrepo cookbook is a collection of simple
recipes utilizing the yum_repo and yum_key LWRP pieces. I modeled the
default recipe to bootstrap our RHEL/CentOS servers and provide a
baseline of third-party packages early in the run_list. So the whole
cookbook and recipes included are opinionated examples from the fine
folks of the community who have worked on them.

I guess what I am getting at, is maybe the cookbook serves better as
documentation, than a one size fits all re-usable cookbook. Talking
with a few of the end users at the summit, most RHEL/CentOS users seem
to depend on EPEL for something. The latest yum cookbook also has an
EPEL recipe which installs by RPM rather than templating out the .repo
file. Which I think demonstrates most Chef on RHEL users at least need
EPEL for something. The zenoss cookbook also duplicates the
yumrepo::zenoss recipe I have, and it totally makes sense to ship that
code with a cookbook which manages zenoss the service.

The more that I think about shipping the repo management code in a
cookbook that addresses a specific problem space, the more I think that
is probably "the right Chef way". I am open to deprecating the existing
yumrepo, and refactoring that out into individual cookbooks. Simply
keep the deprecated code out there as a functional and documented
example. I'm open to thoughts from real-world and potential users of
the existing cookbook. Yet, one of the problems discussed in length at
the summit is Debian/Ubuntu and RHEL/CentOS world view separation in
Chef cooking. One could make the "case node[:platform]" problem even
worse than what has already been seen in cookbooks.

As far as a "hardware" recipe that covers multi-vendor rack-and-stack
nodes and also "virtual" hardware, I don't think that is the right
approach, at least right now. Those are two different problem spaces.
On the "real hardware" side, our Dell OMSA stuff would make a good fit
in the snmp cookbook. Is that also what the hp-health helper exists to
do, serve up a vendor specific OID tree exposing hardware metrics? On
the other hand, the virtualization tools are hypervisor helper
libraries. Because those tools in the virtualization space have nothing
to do with hardware monitoring or snmp, right?

I am really leaning towards it is better to address a specific problem
space really well, than a multi-faceted cookbook addressing many problem
spaces in a mediocre manner. Once those individual problems are
addressed well, I think it makes sense to bring them all together in a
more generalized cookbook like the "application" or "database" cookbooks.

Existing yumrepo default does this:

  1. Configures yum.conf by inclusion of yum::yum
  2. Configures EPEL repo, but installs nothing specific from EPEL.
    Duplicate exists in yum::epel
  3. Configures Dell repos. Installs OpenManage, starts OpenManage, and
    optionally downloads hardware specific firmware. Does it belong in a
    hardware, dell, or snmp cookbook?
  4. Configures VMWare repo. Installs vmware-tools helper libraries for
    VMs on an ESX hypervisor. Tries to safely remove any manually installed
    vmware-tools, in favor of something shipped and managed by yum. So does
    that belong in a vmware-tools, or virtualization cookbook? Because this
    has really nothing to do with monitoring the VM.

In summary, vmware-tools and Dell OMSA recipes exist. There might be a
misuse of cookbook naming/organization which hide that fact. Like I
said, its an opinionated cookbook. I would love to codify and automate
other people's hardware and virtualization vendor problems. It is hard
for any one person, or only a few, to deliver them all. Anyone who
wants to see more of an "application" or "database" meta-cookbook in
these areas could take on an under-represented problem in that space.
If they don't know how/where to start, I for one, am willing to help.
But one cannot effectively codify a problem they have not seen or
understand. I'm just trying to better understand the problem, and bring
some of the summit ideas and concerns into this discussion.

Eric G. Wolfe
Senior Linux Administrator,
IT Infrastructure Systems

Marshall University Computing Services
Drinko Library 428-K
One John Marshall Dr.
Huntington, WV 25755
Phone: 304.696.3428
Email: eric.wolfe@marshall.edu

A programming language is low level when its programs require attention
to the irrelevant.

On 12/01/2011 09:27 AM, Bryan Berry wrote:

Graham, that would be great

On Thu, Dec 1, 2011 at 3:06 PM, Graham Stewart
<graham.stewart@utoronto.ca mailto:graham.stewart@utoronto.ca> wrote:

For the bare metal part, I have a nice simple cookbook we're using
for installing Dell OMSA on a fleet of Dell servers. I'd be happy
to contribute it ...


On 11-12-01 03:04 AM, Bryan Berry wrote:

    AtomicPenguin (Eric Wolfe) has put a good bit of work into the
    yumrepo
    recipes like vmware-tools and dell which detect and installs
    virtualization and management tools specific to those hardware or
    virtualization platforms.

    I would want the similar functionality for a VM running in VMware
    Fusion, virtualbox, Xen, etc where adding the
    "recipe[hardware]" adds
    the paravirtualization tools specific to that platform.

    For machine running on actual bare metal, I want
    "recipe[hardware]" to
    detect and install the hardware management tools, be they
    Dell, HP, IBM,
    etc.

    recipes

    default -- detect current hardware or virt platform and
    install related
    tools
    vmware-tools - install the paravirtualization tools for vmware
    virtualbox - see vmware-tools
    hp  - install hp-health tools for bare metal machine
    dell - install dell openmanage
    ibm - ....

    anyone else interested in such a cookbook? I would only do the
    subset
    that relates to the platforms i use.


    By the way, credit for this idea belongs to jpalmer who is
    creating
    something like this for puppet


-- 
Graham Stewart graham.stewart@utoronto.ca
<mailto:graham.stewart@utoronto.ca> 416-550-2806 <tel:416-550-2806>
Network and Storage Services Manager, Information Technology Services
University of Toronto Libraries
130 St. George Street
Toronto, Ontario, Canada M5S 1A5

Great comments as usual Atomic Penguin, I mean Eric :wink:

On Fri, Dec 2, 2011 at 4:43 AM, Eric G. Wolfe eric.wolfe@marshall.eduwrote:

**
documented example. I'm open to thoughts from real-world and potential
users of the existing cookbook. Yet, one of the problems discussed in
length at the summit is Debian/Ubuntu and RHEL/CentOS world view separation
in Chef cooking. One could make the "case node[:platform]" problem even
worse than what has already been seen in cookbooks.

Could you elaborate on that problem? Is the problem that some cookbooks
only address EL or deb or is it that accommodating EL and deb in the same
recipe is turning recipe code into spaghetti?

As far as a "hardware" recipe that covers multi-vendor rack-and-stack
nodes and also "virtual" hardware, I don't think that is the right
approach, at least right now. Those are two different problem spaces. On
the "real hardware" side, our Dell OMSA stuff would make a good fit in the
snmp cookbook. Is that also what the hp-health helper exists to do, serve
up a vendor specific OID tree exposing hardware metrics? On the other
hand, the virtualization tools are hypervisor helper libraries. Because
those tools in the virtualization space have nothing to do with hardware
monitoring or snmp, right?

True, good point

I am really leaning towards it is better to address a specific problem
space really well, than a multi-faceted cookbook addressing many problem
spaces in a mediocre manner. Once those individual problems are addressed
well, I think it makes sense to bring them all together in a more
generalized cookbook like the "application" or "database" cookbooks.

Existing yumrepo default does this:

  1. Configures yum.conf by inclusion of yum::yum
  2. Configures EPEL repo, but installs nothing specific from EPEL.
    Duplicate exists in yum::epel
  3. Configures Dell repos. Installs OpenManage, starts OpenManage, and
    optionally downloads hardware specific firmware. Does it belong in a
    hardware, dell, or snmp cookbook?
  4. Configures VMWare repo. Installs vmware-tools helper libraries for VMs
    on an ESX hypervisor. Tries to safely remove any manually installed
    vmware-tools, in favor of something shipped and managed by yum. So does
    that belong in a vmware-tools, or virtualization cookbook? Because this
    has really nothing to do with monitoring the VM.

In summary, vmware-tools and Dell OMSA recipes exist. There might be a

misuse of cookbook naming/organization which hide that fact. Like I said,
its an opinionated cookbook. I would love to codify and automate other
people's hardware and virtualization vendor problems. It is hard for any
one person, or only a few, to deliver them all. Anyone who wants to see
more of an "application" or "database" meta-cookbook in these areas could
take on an under-represented problem in that space. If they don't know
how/where to start, I for one, am willing to help. But one cannot
effectively codify a problem they have not seen or understand. I'm just
trying to better understand the problem, and bring some of the summit ideas
and concerns into this discussion.

The problem is essentially laziness on my part. I want a recipe(s) from one
cookbook that installs vmware-tools on my vm whether it is EL or deb. I
want the same functionality for virtualbox. I also need different recipes
for vmware-tools because my vmware-administrator insists on using the
tarball that comes w/ the ESX host.

A hardware cookbook seems a good enough place for myself and others to dump
such recipes. I will probably put the the hw monitoring stuff in there as
well. When and if it grows beyond a few recipes, I will move it into a
separate coookbook.

You point out correctly that hardware monitoring is far different from
paravirtualization tools. I would prefer to start w/ a broad cookbook and
refactor it later than start w/ a small optimized one.

On 12/01/2011 09:27 AM, Bryan Berry wrote:

Graham, that would be great

On Thu, Dec 1, 2011 at 3:06 PM, Graham Stewart <
graham.stewart@utoronto.ca> wrote:

For the bare metal part, I have a nice simple cookbook we're using for
installing Dell OMSA on a fleet of Dell servers. I'd be happy to contribute
it ...

On 11-12-01 03:04 AM, Bryan Berry wrote:

AtomicPenguin (Eric Wolfe) has put a good bit of work into the yumrepo
recipes like vmware-tools and dell which detect and installs
virtualization and management tools specific to those hardware or
virtualization platforms.

I would want the similar functionality for a VM running in VMware
Fusion, virtualbox, Xen, etc where adding the "recipe[hardware]" adds
the paravirtualization tools specific to that platform.

For machine running on actual bare metal, I want "recipe[hardware]" to
detect and install the hardware management tools, be they Dell, HP, IBM,
etc.

recipes

default -- detect current hardware or virt platform and install related
tools
vmware-tools - install the paravirtualization tools for vmware
virtualbox - see vmware-tools
hp - install hp-health tools for bare metal machine
dell - install dell openmanage
ibm - ....

anyone else interested in such a cookbook? I would only do the subset
that relates to the platforms i use.

By the way, credit for this idea belongs to jpalmer who is creating
something like this for puppet

--
Graham Stewart graham.stewart@utoronto.ca 416-550-2806
Network and Storage Services Manager, Information Technology Services
University of Toronto Libraries
130 St. George Street
Toronto, Ontario, Canada M5S 1A5

Comments in-line.

On 12/02/2011 03:14 AM, Bryan Berry wrote:

Great comments as usual Atomic Penguin, I mean Eric :wink:

On Fri, Dec 2, 2011 at 4:43 AM, Eric G. Wolfe <eric.wolfe@marshall.edu
mailto:eric.wolfe@marshall.edu> wrote:

documented example.  I'm open to thoughts from real-world and
potential users of the existing cookbook.  Yet, one of the
problems discussed in length at the summit is Debian/Ubuntu and
RHEL/CentOS world view separation in Chef cooking.  One could make
the "case node[:platform]" problem even worse than what has
already been seen in cookbooks.

Could you elaborate on that problem? Is the problem that some
cookbooks only address EL or deb or is it that accommodating EL and
deb in the same recipe is turning recipe code into spaghetti?
Yeah, so there is no standardization of service names or package names
across distributions, or even different versions of the same version for
that matter. So if one wants to write a generalized cookbook to work
multiple distributions, one would need to write a lot of "case" selector
code for all the edge cases. Which can muddle up a recipe with this
spaghetti code. Probably the best way to write simplified generalized
recipes, right now, is to parameterize those selectable pieces out of
the recipe and hide them in the attributes file. That way you still
accomplish the selection of edge cases, and your recipe code remains
more readable.

As far as a "hardware" recipe that covers multi-vendor
rack-and-stack nodes and also "virtual" hardware, I don't think
that is the right approach, at least right now.  Those are two
different problem spaces.  On the "real hardware" side, our Dell
OMSA stuff would make a good fit in the snmp cookbook.  Is that
also what the hp-health helper exists to do, serve up a vendor
specific OID tree exposing hardware metrics?  On the other hand,
the virtualization tools are hypervisor helper libraries.  Because
those tools in the virtualization space have nothing to do with
hardware monitoring or snmp, right?

True, good point

I am really leaning towards it is better to address a specific
problem space really well, than a multi-faceted cookbook
addressing many problem spaces in a mediocre manner.  Once those
individual problems are addressed well, I think it makes sense to
bring them all together in a more generalized cookbook like the
"application" or "database" cookbooks.

Existing yumrepo default does this:
1. Configures yum.conf by inclusion of yum::yum
2. Configures EPEL repo, but installs nothing specific from EPEL. 
Duplicate exists in yum::epel
3. Configures Dell repos.  Installs OpenManage, starts OpenManage,
and optionally downloads hardware specific firmware.  Does it
belong in a hardware, dell, or snmp cookbook?
4. Configures VMWare repo.  Installs vmware-tools helper libraries
for VMs on an ESX hypervisor.  Tries to safely remove any manually
installed vmware-tools, in favor of something shipped and managed
by yum.  So does that belong in a vmware-tools, or virtualization
cookbook?  Because this has really nothing to do with monitoring
the VM.

In summary, vmware-tools and Dell OMSA recipes exist.  There might
be a misuse of cookbook naming/organization which hide that fact. 
Like I said, its an opinionated cookbook.  I would love to codify
and automate other people's hardware and virtualization vendor
problems.  It is hard for any one person, or only a few, to
deliver them all.  Anyone who wants to see more of an
"application" or "database" meta-cookbook in these areas could
take on an under-represented problem in that space.  If they don't
know how/where to start, I for one, am willing to help.  But one
cannot effectively codify a problem they have not seen or
understand.  I'm just trying to better understand the problem, and
bring some of the summit ideas and concerns into this discussion.

The problem is essentially laziness on my part. I want a recipe(s)
from one cookbook that installs vmware-tools on my vm whether it is EL
or deb. I want the same functionality for virtualbox. I also need
different recipes for vmware-tools because my vmware-administrator
insists on using the tarball that comes w/ the ESX host.
I think that is probably doable either way you go, tarball install or
package manager install. The manual install process is slightly
different now than previous versions, so you may have to include two
different installation constructs to cover both versions. When I wrote
my own version, our goal was to get vmware-tools installed/upgraded
consistently across hosts. We chose to standardize on the yum managed
version. Therefore I nuked the older manual installed versions, and
installed the version shipped via yum.

I am taking a guess that your vmware admin doesn't like the "unmanaged"
semantic that ESX reports when using the packages from VMWare's own
repository. All that really means is vmware-tools is not managed by
ESX, the goal here is to manage that component with Chef. I would like
to see the Windows piece be automated for this space as well, since
VMWare has the Windows packages out there in their public repository.

A hardware cookbook seems a good enough place for myself and others to
dump such recipes. I will probably put the the hw monitoring stuff in
there as well. When and if it grows beyond a few recipes, I will move
it into a separate coookbook.

You point out correctly that hardware monitoring is far different from
paravirtualization tools. I would prefer to start w/ a broad cookbook
and refactor it later than start w/ a small optimized one.

There is this Perl analogy about the camel that it is a horse designed
by committee. It would probably be easier for a single person to design
a duck, otter, and beaver. Once we have those pieces, it will be easier
for the committee to build a better platypus.

On Fri, 02 Dec 2011 11:45:50 -0500, “Eric G. Wolfe” wrote:
I am
taking a guess that your vmware admin doesn’t like the "unmanaged"
semantic that ESX reports when using the packages from VMWare’s own
repository. All that really means is vmware-tools is not managed by ESX,
the goal here is to manage that component with Chef. I would like to see
the Windows piece be automated for this space as well, since VMWare has
the Windows packages out there in their public repository.

Well,
having tools unmanaged block the update from Virtual hardware 4 to 7, in
this case you have to uninstall the repo tools and isntall manually
before upgrading.

I also saw problems with tools being updated before
the esx has been patched, causing latency into guest and even freeze.

So I also prefer to stick on the local package usually instead of
using repo.

On 12/02/2011 12:14 PM, Tensibai wrote:

On Fri, 02 Dec 2011 11:45:50 -0500, "Eric G. Wolfe"
eric.wolfe@marshall.edu wrote:

I am taking a guess that your vmware admin doesn't like the
"unmanaged" semantic that ESX reports when using the packages from
VMWare's own repository. All that really means is vmware-tools is
not managed by ESX, the goal here is to manage that component with
Chef. I would like to see the Windows piece be automated for this
space as well, since VMWare has the Windows packages out there in
their public repository.

Well, having tools unmanaged block the update from Virtual hardware 4
to 7, in this case you have to uninstall the repo tools and isntall
manually before upgrading.

I also saw problems with tools being updated before the esx has been
patched, causing latency into guest and even freeze.

So I also prefer to stick on the local package usually instead of
using repo.

Interesting problem. We haven't made the jump yet to ESX 5, which will
have the same problem of upgrading from Virtual hardware version 7 to
8. From the node's perspective there isn't really any way to detect the
VMWare host version within the VM guest and make an intelligent decision
in the recipe, is there?

I'd like to see the recipes others have come up with for vmware-tools.
Here is what I have:
https://github.com/atomic-penguin/cookbooks/blob/yumrepo/yumrepo/attributes/vmware-tools.rb,
https://github.com/atomic-penguin/cookbooks/blob/yumrepo/yumrepo/recipes/vmware-tools.rb

Le 02.12.2011 23:02, Eric G. Wolfe a écrit :

On 12/02/2011
12:14 PM, Tensibai wrote:

On Fri, 02 Dec 2011 11:45:50 -0500,
"Eric G. Wolfe"wrote:

I am taking a guess that your vmware
admin doesn't like the "unmanaged" semantic that ESX reports when using
the packages from VMWare's own repository. All that really means is
vmware-tools is not managed by ESX, the goal here is to manage that
component with Chef. I would like to see the Windows piece be automated
for this space as well, since VMWare has the Windows packages out there
in their public repository.

Well, having tools unmanaged block
the update from Virtual hardware 4 to 7, in this case you have to
uninstall the repo tools and isntall manually before upgrading.

I also saw problems with tools being updated before the esx has been
patched, causing latency into guest and even freeze.

So I also
prefer to stick on the local package usually instead of using repo.

Interesting problem. We haven't made the jump yet to ESX 5, which will
have the same problem of upgrading from Virtual hardware version 7 to 8.
From the node's perspective there isn't really any way to detect the
VMWare host version within the VM guest and make an intelligent decision
in the recipe, is there?

I'd like to see the recipes others have
come up with for vmware-tools. Here is what I have:
https://github.com/atomic-penguin/cookbooks/blob/yumrepo/yumrepo/attributes/vmware-tools.rb
[1],
https://github.com/atomic-penguin/cookbooks/blob/yumrepo/yumrepo/recipes/vmware-tools.rb
[2]

I'm still working on mine, I plan to do something like what the
puppet module do (using a .tar.gz on the chef-server).

I'll have to
find a way to avoid copying the .tar.gz and untaring on the nodes (it
tooks too much space for some of my lighweight boxes)

Links:

[1]

[2]