Improving ohai's awareness of OS 'flavors'


#1

At my day job (Wanelo http://wanelo.com), we do almost all of our work in
SmartOS zones. This means that, while ‘uname’ and similar commands will
detect the global zone properly, this isn’t actually that useful for
automation, as it’s the local zone configuration that’s more important.
(For more on zones in SmartOS, see this
documenthttp://wiki.smartos.org/display/DOC/Zones
).

For example, Joyent is working on a ‘flavor’ of SmartOS called
SNGLhttp://www.joyent.com/blog/jonathan-perkins-on-why-smartos-is-not-gnu-linux,
which is meant to encourage a platform-agnostic approach to systems choice
(basically, userland in SNGL looks a lot more like Debian-flavored Linux
than mainline SmartOS). While this is useful insofar as it should allow
better cross-platform behavior for community cookbooks, it’s also a pain
for Chef users because ohai can only identify platform by kernel-specific
information, which all zones on a piece of physical hardware, SNGL or
otherwise, share.

Wrestling with these problems, it occurred to us that there are broader
implications to the way that ohai reports platform information. For
example:

  • It’s hard to tell from ohai-generated attributes if you are on a
    custom-built AWS AMI that might need different Chef behavior than the
    ’stock’ OS you baselined from
  • Users of other ‘container’ technologies (jails/lxc) will likely need to
    identify container ‘flavor’
  • Anyone who prefers a package manager that’s not the default for their
    system will have to monkeypatch ohai or hard-code attributes

A few extensions to ohai seem possible:

  • Make ohai container-aware (that is, aware if it’s being run inside a
    container and able to report container configuration) through additional
    attributes
  • Make ohai package manager aware (that is, don’t assume package manager X
    just because you’re on kernel Y) through additional attributes

Before doing any work on this (likely in the form of a cookbook extending
the platform attribute namespace) we’d like some feedback from people here
on the Right Way™ to do this.

best,
Blake


#2

ohai already provides several ways to detect this. “os”, “platform*”,
“kernel”, “lsb” are few places where these information is available. Ohai’s
notion of OS and platform is orthogonal, you can have multiple
platform_family that belongs to same OS. I dont know much about solaris,
but i could detect my custom kernel use node.kernel.release. In linux,
generally the platform_family is sufficient to decide the provider for a
resource, if thats not the case and you have to add conditional logic based
on kernel versions , that should go inside the provider, not ohai (you can
do the same inside a recipe as well). Ohai is responsible for giving the
system information, and its doing so correctly.

I would rather prefer to see core ohai and chef shrink in size, and
instead provide a stable api for extensibility. Event dispatchers, hint
system for cloud platform detection are examples on this direction. Ohai
and Chef’s loading time is already a pain. And with introduction of more
and more platform specific code we now have several blocking yet legitimate
bugs that are now part of core chef (which were not there before), and this
now prevents us from using upstream Chef.

I dont know exactly what additional information you would be able to fetch
from a container specific attributes, as the lxc/openvz templates already
drops the /etc/plattform*-(release| version) file, which ohai internally
uses to detect platform. The only thing i can think of is introspecting
/proc/mount and that might give a hint that we are inside a container. And
a ohai plugins are best for such staff.

best

ranjib

On Tue, Oct 1, 2013 at 10:31 AM, Blake Irvin blake@wanelo.com wrote:

At my day job (Wanelo http://wanelo.com), we do almost all of our work
in SmartOS zones. This means that, while ‘uname’ and similar commands will
detect the global zone properly, this isn’t actually that useful for
automation, as it’s the local zone configuration that’s more important.
(For more on zones in SmartOS, see this documenthttp://wiki.smartos.org/display/DOC/Zones
).

For example, Joyent is working on a ‘flavor’ of SmartOS called SNGLhttp://www.joyent.com/blog/jonathan-perkins-on-why-smartos-is-not-gnu-linux,
which is meant to encourage a platform-agnostic approach to systems choice
(basically, userland in SNGL looks a lot more like Debian-flavored Linux
than mainline SmartOS). While this is useful insofar as it should allow
better cross-platform behavior for community cookbooks, it’s also a pain
for Chef users because ohai can only identify platform by kernel-specific
information, which all zones on a piece of physical hardware, SNGL or
otherwise, share.

Wrestling with these problems, it occurred to us that there are broader
implications to the way that ohai reports platform information. For
example:

  • It’s hard to tell from ohai-generated attributes if you are on a
    custom-built AWS AMI that might need different Chef behavior than the
    ’stock’ OS you baselined from
  • Users of other ‘container’ technologies (jails/lxc) will likely need to
    identify container ‘flavor’
  • Anyone who prefers a package manager that’s not the default for their
    system will have to monkeypatch ohai or hard-code attributes

A few extensions to ohai seem possible:

  • Make ohai container-aware (that is, aware if it’s being run inside a
    container and able to report container configuration) through additional
    attributes
  • Make ohai package manager aware (that is, don’t assume package manager X
    just because you’re on kernel Y) through additional attributes

Before doing any work on this (likely in the form of a cookbook extending
the platform attribute namespace) we’d like some feedback from people here
on the Right Way™ to do this.

best,
Blake


#3

On Tuesday, October 1, 2013 at 10:31 AM, Blake Irvin wrote:

At my day job (Wanelo (http://wanelo.com)), we do almost all of our work in SmartOS zones. This means that, while ‘uname’ and similar commands will detect the global zone properly, this isn’t actually that useful for automation, as it’s the local zone configuration that’s more important. (For more on zones in SmartOS, see this document (http://wiki.smartos.org/display/DOC/Zones)).

For example, Joyent is working on a ‘flavor’ of SmartOS called SNGL (http://www.joyent.com/blog/jonathan-perkins-on-why-smartos-is-not-gnu-linux), which is meant to encourage a platform-agnostic approach to systems choice (basically, userland in SNGL looks a lot more like Debian-flavored Linux than mainline SmartOS). While this is useful insofar as it should allow better cross-platform behavior for community cookbooks, it’s also a pain for Chef users because ohai can only identify platform by kernel-specific information, which all zones on a piece of physical hardware, SNGL or otherwise, share.
Sounds like you just need to make the solaris platform plugin a little smarter? If SmartOS and SNGL are functionally different, they should be reported as different platforms. It’s up to Joyent to provide a sane way to detect this, and then you can patch the plugin to handle that case.

Wrestling with these problems, it occurred to us that there are broader implications to the way that ohai reports platform information. For example:

  • It’s hard to tell from ohai-generated attributes if you are on a custom-built AWS AMI that might need different Chef behavior than the ‘stock’ OS you baselined from
  • Users of other ‘container’ technologies (jails/lxc) will likely need to identify container ‘flavor’
  • Anyone who prefers a package manager that’s not the default for their system will have to monkeypatch ohai or hard-code attributes

Does ohai’s virtualization detection cover these cases? If not, can it be made to do so?

As for non-default package managers, this is more of a Chef issue than an ohai issue, though there is some overlap. But if ohai can detect SNGL as distinct from SmartOS, then you just need to add the right entries in the platform selection code that exists now and everything should be fine.

A few extensions to ohai seem possible:

  • Make ohai container-aware (that is, aware if it’s being run inside a container and able to report container configuration) through additional attributes
    This is what ohai’s virtualization detection is for. If you need a new plugin to detect whatever “virtualization” technology you’re using, we’d be happy to add it.
  • Make ohai package manager aware (that is, don’t assume package manager X just because you’re on kernel Y) through additional attributes

I don’t think Chef really needs this to work correctly in your use case. On the other hand, if you want ohai to collect this information, I don’t see a reason why we wouldn’t allow a plugin for it.

Before doing any work on this (likely in the form of a cookbook extending the platform attribute namespace) we’d like some feedback from people here on the Right Way™ to do this.

best,
Blake

I don’t think Ohai or Chef need big architectural changes to support your use case, just some small enhancements to things that are already there. Patches for this stuff would be welcome, feel free to ask questions here if you run into trouble.


Daniel DeLeo


#4

Comments inline:

On Tue, Oct 1, 2013 at 11:26 AM, Ranjib Dey dey.ranjib@gmail.com wrote:

ohai already provides several ways to detect this. “os”, “platform*”,
“kernel”, “lsb” are few places where these information is available…Ohai
is responsible for giving the system information, and its doing so
correctly.

In the case of SmartOS at least (which is closer to BSD/Debian in userland
than it is to Solaris) getting ‘system information’ as it relates to the
kernel isn’t very useful to Chef - (almos) everything that we care about is
in the zone/container. So we are getting to a place where community
cookbooks that make a lot of assumptions based on ‘platform’ can’t be
easily extended to support our environment. I’m not saying that the value
ohai gives for ‘platform’ is wrong, just that it’s incomplete for anyone
doing container-style virt.

I would rather prefer to see core ohai and chef shrink in size,

Yes :slight_smile: A basic converge (even where changes aren’t made) is currently
slower than provisioning a host for us (granted, provisioning a zone is a
lot faster than provisioning a Xen-style vm).

… ohai plugins are best for such staff.

I haven’t looked at ohai plugin code yet, but that seems like a convention
to strongly consider.

best

ranjib

On Tue, Oct 1, 2013 at 10:31 AM, Blake Irvin blake@wanelo.com wrote:

At my day job (Wanelo http://wanelo.com), we do almost all of our work
in SmartOS zones. This means that, while ‘uname’ and similar commands will
detect the global zone properly, this isn’t actually that useful for
automation, as it’s the local zone configuration that’s more important.
(For more on zones in SmartOS, see this documenthttp://wiki.smartos.org/display/DOC/Zones
).

For example, Joyent is working on a ‘flavor’ of SmartOS called SNGLhttp://www.joyent.com/blog/jonathan-perkins-on-why-smartos-is-not-gnu-linux,
which is meant to encourage a platform-agnostic approach to systems choice
(basically, userland in SNGL looks a lot more like Debian-flavored Linux
than mainline SmartOS). While this is useful insofar as it should allow
better cross-platform behavior for community cookbooks, it’s also a pain
for Chef users because ohai can only identify platform by kernel-specific
information, which all zones on a piece of physical hardware, SNGL or
otherwise, share.

Wrestling with these problems, it occurred to us that there are broader
implications to the way that ohai reports platform information. For
example:

  • It’s hard to tell from ohai-generated attributes if you are on a
    custom-built AWS AMI that might need different Chef behavior than the
    ’stock’ OS you baselined from
  • Users of other ‘container’ technologies (jails/lxc) will likely need to
    identify container ‘flavor’
  • Anyone who prefers a package manager that’s not the default for their
    system will have to monkeypatch ohai or hard-code attributes

A few extensions to ohai seem possible:

  • Make ohai container-aware (that is, aware if it’s being run inside a
    container and able to report container configuration) through additional
    attributes
  • Make ohai package manager aware (that is, don’t assume package manager
    X just because you’re on kernel Y) through additional attributes

Before doing any work on this (likely in the form of a cookbook extending
the platform attribute namespace) we’d like some feedback from people here
on the Right Way™ to do this.

best,
Blake


#5

Comments inline:

On Tue, Oct 1, 2013 at 11:36 AM, Daniel DeLeo dan@kallistec.com wrote:

Sounds like you just need to make the solaris platform plugin a little
smarter? If SmartOS and SNGL are functionally different, they should be
reported as different platforms. It’s up to Joyent to provide a sane way to
detect this, and then you can patch the plugin to handle that case.

I guess I’m unsure about the semantics of the ‘platform’ attribute - if we
expect it to tell us what kernel rev we are one, I cannot use it to
report SNGL vs plain SmartOS, because zones of each type may be running on
the same kernel rev, yet will be functionally different higher up the
stack. I’ve already talked to Joyent about providing means to detect
container ‘flavor’ - that’s in the works for SNGL GA release, I think.

Wrestling with these problems, it occurred to us that there are broader
implications to the way that ohai reports platform information. For
example:

  • It’s hard to tell from ohai-generated attributes if you are on a
    custom-built AWS AMI that might need different Chef behavior than the
    ’stock’ OS you baselined from
  • Users of other ‘container’ technologies (jails/lxc) will likely need to
    identify container ‘flavor’
  • Anyone who prefers a package manager that’s not the default for their
    system will have to monkeypatch ohai or hard-code attributes

Does ohai’s virtualization detection cover these cases? If not, can it be
made to do so?

That answers my question, maybe, about how to name container-specific
attributes - can we adopt the conventions used for other virtualization
flavors? (Not sure quite where to do this, so pointers would be helpful -
we don’t mind contributing features upstream.)

As for non-default package managers, this is more of a Chef issue than an
ohai issue, though there is some overlap. But if ohai can detect SNGL as
distinct from SmartOS, then you just need to add the right entries in the
platform selection code that exists now and everything should be fine.

Having the Chef pkg resource code handle this does sound like the right
plan to me, though I think that this will end up being coupled to ohai
correctly reporting container flavor.


#6

On Tuesday, October 1, 2013 at 12:17 PM, Blake Irvin wrote:

Comments inline:

On Tue, Oct 1, 2013 at 11:36 AM, Daniel DeLeo <dan@kallistec.com (mailto:dan@kallistec.com)> wrote:

Sounds like you just need to make the solaris platform plugin a little smarter? If SmartOS and SNGL are functionally different, they should be reported as different platforms. It’s up to Joyent to provide a sane way to detect this, and then you can patch the plugin to handle that case.

I guess I’m unsure about the semantics of the ‘platform’ attribute - if we expect it to tell us what kernel rev we are one, I cannot use it to report SNGL vs plain SmartOS, because zones of each type may be running on the same kernel rev, yet will be functionally different higher up the stack. I’ve already talked to Joyent about providing means to detect container ‘flavor’ - that’s in the works for SNGL GA release, I think.
For example, Ubuntu, CentOS, etc. are all on the Linux kernel, and could be on the same kernel version or not. Platform implies more than just the kernel. FWIW, kernel info goes under the kernel attribute. For example:


Example data:

ddeleo@ubuntu:~ ohai platform [ "ubuntu" ] ddeleo@ubuntu:~ ohai kernel
{
“name”: “Linux”,
“release”: “3.8.0-19-generic”,
“version”: “#29-Ubuntu SMP Wed Apr 17 18:16:28 UTC 2013”,
“machine”: “x86_64”,

The OS just needs to expose some way for us to get the info we need to detect the platform. There’s a few standards for this but they’re not really great in practice so I don’t have a preference for how this is done (not that it would matter anyway).

Wrestling with these problems, it occurred to us that there are broader implications to the way that ohai reports platform information. For example:

  • It’s hard to tell from ohai-generated attributes if you are on a custom-built AWS AMI that might need different Chef behavior than the ‘stock’ OS you baselined from
  • Users of other ‘container’ technologies (jails/lxc) will likely need to identify container ‘flavor’
  • Anyone who prefers a package manager that’s not the default for their system will have to monkeypatch ohai or hard-code attributes

Does ohai’s virtualization detection cover these cases? If not, can it be made to do so?

That answers my question, maybe, about how to name container-specific attributes - can we adopt the conventions used for other virtualization flavors? (Not sure quite where to do this, so pointers would be helpful - we don’t mind contributing features upstream.)

This is what we have so far for solaris:

As for non-default package managers, this is more of a Chef issue than an ohai issue, though there is some overlap. But if ohai can detect SNGL as distinct from SmartOS, then you just need to add the right entries in the platform selection code that exists now and everything should be fine.

Having the Chef pkg resource code handle this does sound like the right plan to me, though I think that this will end up being coupled to ohai correctly reporting container flavor.

In the current implementation, we have one big hash that says which platform gets what: https://github.com/opscode/chef/blob/master/lib/chef/platform/provider_mapping.rb


Daniel DeLeo


#7

On Thursday, October 10, 2013 at 2:37 PM, Blake Irvin wrote:

See: http://wiki.smartos.org/display/DOC/Managing+Datasets#ManagingDatasets-OSLevelVirtualizationZoneImages

That page has a deprecation notice at the top, and links to here: http://wiki.smartos.org/display/DOC/Managing+Images

In any case, it seems like any of the data in an image manifest could be handy. If possible, the terminology used in ohai should match the terms used in the vendor docs. You can copy the data to a “standard” name if there’s a use case for treating an attribute in a cross-platform way (e.g., some standard attributes get copied to node[:cloud] from the cloud-specific location like node[:ec2])

Daniel DeLeo


#8

Thanks Daniel - this is the kind of guidance I was looking for.

Blake

On Mon, Oct 14, 2013 at 3:46 PM, Daniel DeLeo dan@kallistec.com wrote:

On Thursday, October 10, 2013 at 2:37 PM, Blake Irvin wrote:

See:
http://wiki.smartos.org/display/DOC/Managing+Datasets#ManagingDatasets-OSLevelVirtualizationZoneImages

That page has a deprecation notice at the top, and links to here:
http://wiki.smartos.org/display/DOC/Managing+Images

In any case, it seems like any of the data in an image manifest could be
handy. If possible, the terminology used in ohai should match the terms
used in the vendor docs. You can copy the data to a “standard” name if
there’s a use case for treating an attribute in a cross-platform way (e.g.,
some standard attributes get copied to node[:cloud] from the cloud-specific
location like node[:ec2])

Daniel DeLeo


#9

Resurrecting this thread a bit, here’s what virt detection in ohai looks
like right now on a SmartOS host:

“virtualization”: {
“system”: “zone”,
“role”: “guest”,
“guest_uuid”: “b3ce5ff0-33a5-4fa3-ab7f-8035350bed54”
},

Would it make sense to extend this a bit, maybe like this?:

“virtualization”: {
“system”: “zone”,
“role”: “guest”,
“flavor”: "base64-13.1.2"
“guest_uuid”: “b3ce5ff0-33a5-4fa3-ab7f-8035350bed54”
},

On Tue, Oct 1, 2013 at 12:30 PM, Daniel DeLeo dan@kallistec.com wrote:

On Tuesday, October 1, 2013 at 12:17 PM, Blake Irvin wrote:

Comments inline:

On Tue, Oct 1, 2013 at 11:36 AM, Daniel DeLeo dan@kallistec.com wrote:

Sounds like you just need to make the solaris platform plugin a little
smarter? If SmartOS and SNGL are functionally different, they should be
reported as different platforms. It’s up to Joyent to provide a sane way to
detect this, and then you can patch the plugin to handle that case.

I guess I’m unsure about the semantics of the ‘platform’ attribute - if we
expect it to tell us what kernel rev we are one, I cannot use it to
report SNGL vs plain SmartOS, because zones of each type may be running on
the same kernel rev, yet will be functionally different higher up the
stack. I’ve already talked to Joyent about providing means to detect
container ‘flavor’ - that’s in the works for SNGL GA release, I think.

For example, Ubuntu, CentOS, etc. are all on the Linux kernel, and could
be on the same kernel version or not. Platform implies more than just the
kernel. FWIW, kernel info goes under the kernel attribute. For example:

https://github.com/opscode/ohai/blob/master/lib/ohai/plugins/linux/platform.rb

https://github.com/opscode/ohai/blob/master/lib/ohai/plugins/solaris2/platform.rb

Example data:

ddeleo@ubuntu:~ ohai platform [ "ubuntu" ] ddeleo@ubuntu:~ ohai kernel
{
“name”: “Linux”,
“release”: “3.8.0-19-generic”,
“version”: “#29-Ubuntu SMP Wed Apr 17 18:16:28 UTC 2013”,
“machine”: “x86_64”,

The OS just needs to expose some way for us to get the info we need to
detect the platform. There’s a few standards for this but they’re not
really great in practice so I don’t have a preference for how this is done
(not that it would matter anyway).

Wrestling with these problems, it occurred to us that there are broader
implications to the way that ohai reports platform information. For
example:

  • It’s hard to tell from ohai-generated attributes if you are on a
    custom-built AWS AMI that might need different Chef behavior than the
    ’stock’ OS you baselined from
  • Users of other ‘container’ technologies (jails/lxc) will likely need to
    identify container ‘flavor’
  • Anyone who prefers a package manager that’s not the default for their
    system will have to monkeypatch ohai or hard-code attributes

Does ohai’s virtualization detection cover these cases? If not, can it be
made to do so?

That answers my question, maybe, about how to name container-specific
attributes - can we adopt the conventions used for other virtualization
flavors? (Not sure quite where to do this, so pointers would be helpful -
we don’t mind contributing features upstream.)

This is what we have so far for solaris:

https://github.com/opscode/ohai/blob/master/lib/ohai/plugins/solaris2/virtualization.rb

As for non-default package managers, this is more of a Chef issue than an
ohai issue, though there is some overlap. But if ohai can detect SNGL as
distinct from SmartOS, then you just need to add the right entries in the
platform selection code that exists now and everything should be fine.

Having the Chef pkg resource code handle this does sound like the right
plan to me, though I think that this will end up being coupled to ohai
correctly reporting container flavor.

In the current implementation, we have one big hash that says which
platform gets what:
https://github.com/opscode/chef/blob/master/lib/chef/platform/provider_mapping.rb


Daniel DeLeo


#10

On Wednesday, October 9, 2013 at 5:02 PM, Blake Irvin wrote:

Resurrecting this thread a bit, here’s what virt detection in ohai looks like right now on a SmartOS host:

“virtualization”: {
“system”: “zone”,
“role”: “guest”,
“guest_uuid”: “b3ce5ff0-33a5-4fa3-ab7f-8035350bed54”
},

Would it make sense to extend this a bit, maybe like this?:

“virtualization”: {
“system”: “zone”,
“role”: “guest”,
“flavor”: "base64-13.1.2"
“guest_uuid”: “b3ce5ff0-33a5-4fa3-ab7f-8035350bed54”
},

I’m not familiar, what is the “flavor” here describing?


Daniel DeLeo


#11

“Flavor” is arbitratry. It could also (and perhaps more accurately) be
called “dataset”.

On Thu, Oct 10, 2013 at 7:33 AM, Daniel DeLeo dan@kallistec.com wrote:

On Wednesday, October 9, 2013 at 5:02 PM, Blake Irvin wrote:

Resurrecting this thread a bit, here’s what virt detection in ohai looks
like right now on a SmartOS host:

“virtualization”: {
“system”: “zone”,
“role”: “guest”,
“guest_uuid”: “b3ce5ff0-33a5-4fa3-ab7f-8035350bed54”
},

Would it make sense to extend this a bit, maybe like this?:

“virtualization”: {
“system”: “zone”,
“role”: “guest”,
“flavor”: "base64-13.1.2"
“guest_uuid”: “b3ce5ff0-33a5-4fa3-ab7f-8035350bed54”
},

I’m not familiar, what is the “flavor” here describing?


Daniel DeLeo


#12

See:
http://wiki.smartos.org/display/DOC/Managing+Datasets#ManagingDatasets-OSLevelVirtualizationZoneImages

On Thu, Oct 10, 2013 at 2:36 PM, Blake Irvin blake@wanelo.com wrote:

“Flavor” is arbitratry. It could also (and perhaps more accurately) be
called “dataset”.

On Thu, Oct 10, 2013 at 7:33 AM, Daniel DeLeo dan@kallistec.com wrote:

On Wednesday, October 9, 2013 at 5:02 PM, Blake Irvin wrote:

Resurrecting this thread a bit, here’s what virt detection in ohai looks
like right now on a SmartOS host:

“virtualization”: {
“system”: “zone”,
“role”: “guest”,
“guest_uuid”: “b3ce5ff0-33a5-4fa3-ab7f-8035350bed54”
},

Would it make sense to extend this a bit, maybe like this?:

“virtualization”: {
“system”: “zone”,
“role”: “guest”,
“flavor”: "base64-13.1.2"
“guest_uuid”: “b3ce5ff0-33a5-4fa3-ab7f-8035350bed54”
},

I’m not familiar, what is the “flavor” here describing?


Daniel DeLeo