Mixlib::Versioning 1.0.0 released


#1

Ohai,
We just open-sourced mixlib-versioning which is general purpose Ruby library that allows you to parse, compare version strings in multiple formats. This library was born out of work done in support of Omnibus, Omnitruck and our internal Jenkins build pipelines.

The library has been published to Rubygems.org and source code is available on GitHub:

https://rubygems.org/gems/mixlib-versioning

We hope some people in the community find it useful!


Seth Chisamore
Software Development Engineer, Opscode, Inc.
IRC, Skype, Twitter, Github: schisamo


#2

Seth,

This looks awesome! Can we expect this library to supplement cookbook
versioning in Chef Server/Client/Solo so cookbooks can be SemVer
compliant with prerelease and build numbers?

-M

On Sat, Mar 30, 2013 at 1:24 AM, Seth Chisamore schisamo@opscode.com wrote:

Ohai,
We just open-sourced mixlib-versioning which is general purpose Ruby library that allows you to parse, compare version strings in multiple formats. This library was born out of work done in support of Omnibus, Omnitruck and our internal Jenkins build pipelines.

The library has been published to Rubygems.org and source code is available on GitHub:

https://rubygems.org/gems/mixlib-versioning
https://github.com/opscode/mixlib-versioning

We hope some people in the community find it useful!


Seth Chisamore
Software Development Engineer, Opscode, Inc.
IRC, Skype, Twitter, Github: schisamo


#3

Chef 10.x won’t be getting any new features, and there’d have to be an erlang version of the library to get support in the chef 11 server.

Honestly, though, I’d much rather remove dependency solving from the server entirely. If we radically simplify environments to support only exact cookbook versions dependencies then its relatively straightforward to add versioning of roles to environments, which would solve a lot of wailing and gnashing of teeth.

Here’s my proposal so far: https://gist.github.com/danielsdeleo/7c55ebe39639928134df

In this scenario, you’d still have x.y.z (+ etc) version numbers, but they’d be used for communication between developers (the “semantic” part of semver) and by tools such as berkshelf and librarian when resolving dependencies. Since the server wouldn’t use the version numbers, we could relax the restrictions on them to support all the prerelease stuff without adding complexity to the dependency solver algorithm.


Daniel DeLeo

On Saturday, March 30, 2013 at 11:13 AM, Mike wrote:

Seth,

This looks awesome! Can we expect this library to supplement cookbook
versioning in Chef Server/Client/Solo so cookbooks can be SemVer
compliant with prerelease and build numbers?

-M

On Sat, Mar 30, 2013 at 1:24 AM, Seth Chisamore schisamo@opscode.com wrote:

Ohai,
We just open-sourced mixlib-versioning which is general purpose Ruby library that allows you to parse, compare version strings in multiple formats. This library was born out of work done in support of Omnibus, Omnitruck and our internal Jenkins build pipelines.

The library has been published to Rubygems.org and source code is available on GitHub:

https://rubygems.org/gems/mixlib-versioning
https://github.com/opscode/mixlib-versioning

We hope some people in the community find it useful!


Seth Chisamore
Software Development Engineer, Opscode, Inc.
IRC, Skype, Twitter, Github: schisamo


#4

Daniel,

The chef-client could also do the dependency resolution itself rather than
asking the chef-server to do it. The only new API the chef-server would
need to provide is a batch API to fetch the full metadatas of all versions
of all cookbooks uploaded to the chef-server, or at least as much of the
metadatas as is necessary for the chef-client perform the
dependency-resolution. The chef-client can then perform its own
dependency-resolution on that data and the chef-server wouldn’t need to be
involved.

In fact, perhaps this should be done anyway. Dependency-resolution can take
exponential time. Nothing on the chef-server should ever be permitted to
take exponential time. While it is a problem for a given chef-client if the
dependency-resolution takes too long on the chef-client, it’s a problem for
all clients in an infrastructure if that knocks out the whole chef-server
rather than just the one chef-client.

Regarding environments, I’ve added my thoughts here:

Seth,

Regarding Mixlib::Versioning - cool! I’ve added my thoughts here:
https://github.com/opscode/mixlib-versioning/issues/2

Cheers,
Jay

On Sat, Mar 30, 2013 at 3:28 PM, Daniel DeLeo dan@kallistec.com wrote:

Chef 10.x won’t be getting any new features, and there’d have to be an
erlang version of the library to get support in the chef 11 server.

Honestly, though, I’d much rather remove dependency solving from the
server entirely. If we radically simplify environments to support only
exact cookbook versions dependencies then its relatively straightforward to
add versioning of roles to environments, which would solve a lot of wailing
and gnashing of teeth.

Here’s my proposal so far:
https://gist.github.com/danielsdeleo/7c55ebe39639928134df

In this scenario, you’d still have x.y.z (+ etc) version numbers, but
they’d be used for communication between developers (the “semantic” part of
semver) and by tools such as berkshelf and librarian when resolving
dependencies. Since the server wouldn’t use the version numbers, we could
relax the restrictions on them to support all the prerelease stuff without
adding complexity to the dependency solver algorithm.


Daniel DeLeo

On Saturday, March 30, 2013 at 11:13 AM, Mike wrote:

Seth,

This looks awesome! Can we expect this library to supplement cookbook
versioning in Chef Server/Client/Solo so cookbooks can be SemVer
compliant with prerelease and build numbers?

-M

On Sat, Mar 30, 2013 at 1:24 AM, Seth Chisamore schisamo@opscode.com
wrote:

Ohai,
We just open-sourced mixlib-versioning which is general purpose Ruby
library that allows you to parse, compare version strings in multiple
formats. This library was born out of work done in support of Omnibus,
Omnitruck and our internal Jenkins build pipelines.

The library has been published to Rubygems.org and source code is
available on GitHub:

https://rubygems.org/gems/mixlib-versioning
https://github.com/opscode/mixlib-versioning

We hope some people in the community find it useful!


Seth Chisamore
Software Development Engineer, Opscode, Inc.
IRC, Skype, Twitter, Github: schisamo


#5

On Saturday, March 30, 2013 at 1:55 PM, Jay Feldblum wrote:

Daniel,

The chef-client could also do the dependency resolution itself rather than asking the chef-server to do it. The only new API the chef-server would need to provide is a batch API to fetch the full metadatas of all versions of all cookbooks uploaded to the chef-server, or at least as much of the metadatas as is necessary for the chef-client perform the dependency-resolution. The chef-client can then perform its own dependency-resolution on that data and the chef-server wouldn’t need to be involved.
I dislike this approach because it requires an ever-growing amount of data to be shipped to the client on every run, while not solving the problem of version clobbering. With hosted chef, we see that the cookbook version API call is slower than all the others by quite a wide margin, but with the gecode-based solver (I have less personal experience with the pure-erlang replacement) the constraint solution usually only takes a few milliseconds–the call time of the request is dominated by the large amount of disk and network IO required to get the necessary information into the constraint solver. If you were to automate patch-version bumps to avoid clobbering, then you will automatically exacerbate this issue.

Adding another (potentially much slower) network link in this chain feels like a move in the wrong direction to me.

In fact, perhaps this should be done anyway. Dependency-resolution can take exponential time. Nothing on the chef-server should ever be permitted to take exponential time. While it is a problem for a given chef-client if the dependency-resolution takes too long on the chef-client, it’s a problem for all clients in an infrastructure if that knocks out the whole chef-server rather than just the one chef-client.
Part of the point of my proposal is that dependency resolution is moved to the workstation: compile a list of compatible cookbooks by hand or automatically, then upload (if necessary) and use environments to lock some set of nodes to the pre-computed solution. This feels more elegant because it requires the least computation overall and (more importantly) moves it off of production systems–no worries about all your hosts suddenly chewing up CPU due to a gnarly dependency graph.

Regarding environments, I’ve added my thoughts here:
https://gist.github.com/danielsdeleo/7c55ebe39639928134df/#comment-808117

If roles become environment-version-able then the only thing that’s left are data bags (and clients, but in practice these are tied pretty closely to nodes, so I’m not clear about the use-case). I’ve heard many people say that they’d like data bags to be environment-version-able as well; I’ve always used environment as the id for data bag items where the contents differ per-environment, so the need for this is a bit foreign to me (not to say it’s invalid, I just don’t understand the use case).

Seth,

Regarding Mixlib::Versioning - cool! I’ve added my thoughts here:
https://github.com/opscode/mixlib-versioning/issues/2

Cheers,
Jay


Daniel DeLeo