CHEF-3788: More restrictive gem dependencies?


#1

Grégory has suggested in CHEF-3788 that the gem dependencies be more restrictive after the recent incident where Moneta 0.7.0 was released and the API was not backward compatible (CHEF-3721).

In the past we’ve been restrictive about gems that have had similar issues, particularly with slow responses, like JSON, but overall we have been more optimistic to get the benefits of new releases of libraries without having to make a new release of Chef. In the case of JSON, we occasionally have tickets where people want to bump the version because we’re starting to cause dependency resolution failures with other tools that use Chef as a library.

But maybe it would help if we were at least pessmistic about major version changes, e.g. ~> x.y. Anyone have other opinions to add?


Bryan McLellan | opscode | technical program manager, open source
© 206.607.7108 | (t) @btmspox | (b) http://blog.loftninjas.org

http://tickets.opscode.com/browse/CHEF-3788
http://tickets.opscode.com/browse/CHEF-3721


#2

I’m a big fan of looking at each dependency and seeing if they
explicitly declare that:
a) they follow SemVer
b) they are past 1.0

In that case, using ‘~> 1.0’ is great. Using anything under 1.0, i.e.
moneta, should probably be nailed to the working version, until it’s a
problem and deserves revisiting.

-M

On Thu, Jan 24, 2013 at 5:46 PM, Bryan McLellan btm@opscode.com wrote:

Grégory has suggested in CHEF-3788 that the gem dependencies be more restrictive after the recent incident where Moneta 0.7.0 was released and the API was not backward compatible (CHEF-3721).

In the past we’ve been restrictive about gems that have had similar issues, particularly with slow responses, like JSON, but overall we have been more optimistic to get the benefits of new releases of libraries without having to make a new release of Chef. In the case of JSON, we occasionally have tickets where people want to bump the version because we’re starting to cause dependency resolution failures with other tools that use Chef as a library.

But maybe it would help if we were at least pessmistic about major version changes, e.g. ~> x.y. Anyone have other opinions to add?


Bryan McLellan | opscode | technical program manager, open source
© 206.607.7108 | (t) @btmspox | (b) http://blog.loftninjas.org

http://tickets.opscode.com/browse/CHEF-3788
http://tickets.opscode.com/browse/CHEF-3721


#3

On Thu, Jan 24, 2013 at 5:46 PM, Bryan McLellan btm@opscode.com wrote:

Grégory has suggested in CHEF-3788 that the gem dependencies be more
restrictive after the recent incident where Moneta 0.7.0 was released and
the API was not backward compatible (CHEF-3721).

In the past we’ve been restrictive about gems that have had similar
issues, particularly with slow responses, like JSON, but overall we have
been more optimistic to get the benefits of new releases of libraries
without having to make a new release of Chef. In the case of JSON, we
occasionally have tickets where people want to bump the version because
we’re starting to cause dependency resolution failures with other tools
that use Chef as a library.

But maybe it would help if we were at least pessmistic about major version
changes, e.g. ~> x.y. Anyone have other opinions to add?

I was actually just working my way up to an e-mail about this, because I’m
in the middle of packaging Chef for Fedora and they have far newer versions
of Gems – like JSON – than Chef’s gemspec would suggest work.

My problem with being so restrictive about Gem versions is that few people
actually go back periodically and see if such restrictions are still
necessary.

I’m pretty sure the extremely pessimistic pinning on JSON can be removed,
but how can I be sure? Is it just a matter of running Chef’s unit tests and
if they pass on the new Gem, I can submit a pull request to remove it?

  • Julian

#4

On Thu, Jan 24, 2013 at 6:19 PM, Julian C. Dunn jdunn@aquezada.com wrote:

My problem with being so restrictive about Gem versions is that few people
actually go back periodically and see if such restrictions are still
necessary.

Right. It’s a bunch of work, which is why we’ve hesitated. Chef
releases, even if we’re just including a couple commits, take a lot of
work and time to get everything tested and all the packages built.

I was actually just working my way up to an e-mail about this, because I’m
in the middle of packaging Chef for Fedora and they have far newer versions
of Gems – like JSON – than Chef’s gemspec would suggest work.

I’m pretty sure the extremely pessimistic pinning on JSON can be removed,
but how can I be sure? Is it just a matter of running Chef’s unit tests and
if they pass on the new Gem, I can submit a pull request to remove it?

Well, to increase the version it’s just been a matter of opening a
ticket with a PR that says “I’ve tested a chef-client run with this
version and it seemed fine”

For example: http://tickets.opscode.com/browse/CHEF-1548
or for example: http://tickets.opscode.com/browse/CHEF-2031

Here is the most recent un-finished JSON dependency ticket:
http://tickets.opscode.com/browse/CHEF-3712
This is probably a dupe provided we get up to at least 1.6.5:
http://tickets.opscode.com/browse/CHEF-2960

Bryan


#5

On Thu, Jan 24, 2013 at 6:05 PM, Mike miketheman@gmail.com wrote:

I’m a big fan of looking at each dependency and seeing if they
explicitly declare that:
a) they follow SemVer
b) they are past 1.0

In that case, using ‘~> 1.0’ is great. Using anything under 1.0, i.e.
moneta, should probably be nailed to the working version, until it’s a
problem and deserves revisiting.

If you wanted to do all that research and document your findings in
comments in the gemspec, that would probably be useful, being such a
fan of research that you are. :slight_smile:

Bryan


#6

I guess I just volunteered? Should we target Chef 10.18.2, or Chef 11?
i.e. which branch is the “best” one?

On Thu, Jan 24, 2013 at 6:49 PM, Bryan McLellan btm@loftninjas.org wrote:

On Thu, Jan 24, 2013 at 6:05 PM, Mike miketheman@gmail.com wrote:

I’m a big fan of looking at each dependency and seeing if they
explicitly declare that:
a) they follow SemVer
b) they are past 1.0

In that case, using ‘~> 1.0’ is great. Using anything under 1.0, i.e.
moneta, should probably be nailed to the working version, until it’s a
problem and deserves revisiting.

If you wanted to do all that research and document your findings in
comments in the gemspec, that would probably be useful, being such a
fan of research that you are. :slight_smile:

Bryan


#7

I know my vote is to target Chef 11. Here’s to hoping that Chef 10.18 stays very stable for a long time and we can move forward. Introducing changes this late is probably asking for trouble.

Thanks,
Matt Ray
Senior Technical Evangelist | Opscode Inc.
matt@opscode.com | (512) 731-2218
Twitter, IRC, GitHub: mattray


From: Mike [miketheman@gmail.com]
Sent: Thursday, January 24, 2013 7:25 PM
To: Bryan McLellan
Cc: chef-dev@lists.opscode.com
Subject: [chef-dev] Re: Re: CHEF-3788: More restrictive gem dependencies?

I guess I just volunteered? Should we target Chef 10.18.2, or Chef 11?
i.e. which branch is the “best” one?

On Thu, Jan 24, 2013 at 6:49 PM, Bryan McLellan btm@loftninjas.org wrote:

On Thu, Jan 24, 2013 at 6:05 PM, Mike miketheman@gmail.com wrote:

I’m a big fan of looking at each dependency and seeing if they
explicitly declare that:
a) they follow SemVer
b) they are past 1.0

In that case, using ‘~> 1.0’ is great. Using anything under 1.0, i.e.
moneta, should probably be nailed to the working version, until it’s a
problem and deserves revisiting.

If you wanted to do all that research and document your findings in
comments in the gemspec, that would probably be useful, being such a
fan of research that you are. :slight_smile:

Bryan


#8

So it looks like CHEF-3788 has been closed recently, and I’m not
disagreeing with the closure.

I am interested in knowing if dependency management is a thing that
Chef 11 is going to stay on top of, i.e. has someone watching
dependency updates (I recommend Gemnasium 1), and running
updates/tests when needed to validate that the dependency didn’t Break
The World.

-Mike

On Fri, Jan 25, 2013 at 11:26 AM, Matt Ray matt@opscode.com wrote:

I know my vote is to target Chef 11. Here’s to hoping that Chef 10.18 stays very stable for a long time and we can move forward. Introducing changes this late is probably asking for trouble.

Thanks,
Matt Ray
Senior Technical Evangelist | Opscode Inc.
matt@opscode.com | (512) 731-2218
Twitter, IRC, GitHub: mattray


From: Mike [miketheman@gmail.com]
Sent: Thursday, January 24, 2013 7:25 PM
To: Bryan McLellan
Cc: chef-dev@lists.opscode.com
Subject: [chef-dev] Re: Re: CHEF-3788: More restrictive gem dependencies?

I guess I just volunteered? Should we target Chef 10.18.2, or Chef 11?
i.e. which branch is the “best” one?

On Thu, Jan 24, 2013 at 6:49 PM, Bryan McLellan btm@loftninjas.org wrote:

On Thu, Jan 24, 2013 at 6:05 PM, Mike miketheman@gmail.com wrote:

I’m a big fan of looking at each dependency and seeing if they
explicitly declare that:
a) they follow SemVer
b) they are past 1.0

In that case, using ‘~> 1.0’ is great. Using anything under 1.0, i.e.
moneta, should probably be nailed to the working version, until it’s a
problem and deserves revisiting.

If you wanted to do all that research and document your findings in
comments in the gemspec, that would probably be useful, being such a
fan of research that you are. :slight_smile:

Bryan


#9

On Feb 6, 2013, at 11:52 PM, Mike miketheman@gmail.com wrote:

So it looks like CHEF-3788 has been closed recently, and I’m not
disagreeing with the closure.

I think it’s a great thing, plus it’s awesome that Chef is no longer pinned to an old version of JSON. Helps with the packaging for Fedora.

BTW, does the pinning in master make any sense?

s.add_dependency “json”, “>= 1.4.4”, “~> 1.7.6”

Isn’t the “>= 1.4.4” redundant?

  • Julian