Moving forward with the JSON gem


#1

Chef initially required json without a version constraint, but json
1.4.3 was pretty broken so Chef 0.8.12 got a constraint of <= 1.4.2
the next day. We became explicitly cautious of changes to the json gem
as a result.

Three months later json 1.4.4 shipped and had some fixes. As part of
CHEF-1548 [1] json 1.4.6 was tested and the constraint was switched to

= 1.4.4, <= 1.4.6.

FreeBSD ports now uses json 1.5.1, and we’re surely falling behind in
other distributions as well (although debian/ubuntu appear to be
packaging a version from the stone age). The plan, in CHEF-2031 [2],
is to test out json 1.5.1 and if we’re happy with it we will set the
contraints to >= 1.4.4 <= 1.5.1 for now. It would be nice to not do
this for 0.9-stable, because of our fear of the json gem causing
rampant instability, but we’re committed to supporting the stable tree
for a few more months and I we’re behind the distributions already. So
I think we have to test json 1.5.1 on 0.9 and 0.10.

We’ve had issues in the past with Chef and Ohai having different
constraints around the json gem. Since we’re planning to do an ohai
release soon, we would need to start testing json 1.5.1 against ohai
and chef now, and release the next version of ohai with the upped
constraint to match the next 0.9-stable release.

There is strong desire in the community to switch from json to
yajl.There are a couple tickets [3][4] for that switch. To fix
CHEF-1292 [5] we added a Chef::JSONCompat (originally Chef::JSON [6])
wrapper for ::JSON which set some values. Akzhan Abdulin noted [7]
that if we switched to always using that class it would be easier to
shoe in yajl.

Does anyone have additional thoughts about the priority of this or
preferred alternatives?

Bryan McLellan

[1] http://tickets.opscode.com/browse/CHEF-1548
[2] http://tickets.opscode.com/browse/CHEF-2031
[3] http://tickets.opscode.com/browse/CHEF-1948
[4] http://tickets.opscode.com/browse/OHAI-244
[5] http://tickets.opscode.com/browse/CHEF-1292
[6] http://tickets.opscode.com/browse/CHEF-2057
[7] http://tickets.opscode.com/browse/CHEF-2059


#2

I suppose that we need to

  1. Move Chef::JSONCompat to ohai gem.
  2. Use Ohai::JSONCompat everywhere we need to.
  3. Switch to yajl-ruby as last step.

2011/3/5 Bryan McLellan btm@loftninjas.org

Chef initially required json without a version constraint, but json
1.4.3 was pretty broken so Chef 0.8.12 got a constraint of <= 1.4.2
the next day. We became explicitly cautious of changes to the json gem
as a result.

Three months later json 1.4.4 shipped and had some fixes. As part of
CHEF-1548 [1] json 1.4.6 was tested and the constraint was switched to

= 1.4.4, <= 1.4.6.

FreeBSD ports now uses json 1.5.1, and we’re surely falling behind in
other distributions as well (although debian/ubuntu appear to be
packaging a version from the stone age). The plan, in CHEF-2031 [2],
is to test out json 1.5.1 and if we’re happy with it we will set the
contraints to >= 1.4.4 <= 1.5.1 for now. It would be nice to not do
this for 0.9-stable, because of our fear of the json gem causing
rampant instability, but we’re committed to supporting the stable tree
for a few more months and I we’re behind the distributions already. So
I think we have to test json 1.5.1 on 0.9 and 0.10.

We’ve had issues in the past with Chef and Ohai having different
constraints around the json gem. Since we’re planning to do an ohai
release soon, we would need to start testing json 1.5.1 against ohai
and chef now, and release the next version of ohai with the upped
constraint to match the next 0.9-stable release.

There is strong desire in the community to switch from json to
yajl.There are a couple tickets [3][4] for that switch. To fix
CHEF-1292 [5] we added a Chef::JSONCompat (originally Chef::JSON [6])
wrapper for ::JSON which set some values. Akzhan Abdulin noted [7]
that if we switched to always using that class it would be easier to
shoe in yajl.

Does anyone have additional thoughts about the priority of this or
preferred alternatives?

Bryan McLellan

[1] http://tickets.opscode.com/browse/CHEF-1548
[2] http://tickets.opscode.com/browse/CHEF-2031
[3] http://tickets.opscode.com/browse/CHEF-1948
[4] http://tickets.opscode.com/browse/OHAI-244
[5] http://tickets.opscode.com/browse/CHEF-1292
[6] http://tickets.opscode.com/browse/CHEF-2057
[7] http://tickets.opscode.com/browse/CHEF-2059


#3

Hi all,

On Sat, Mar 5, 2011 at 1:18 AM, Akzhan Abdulin akzhan.abdulin@gmail.com wrote:

I suppose that we need to

  1. Move Chef::JSONCompat to ohai gem.
  2. Use Ohai::JSONCompat everywhere we need to.
  3. Switch to yajl-ruby as last step.

Ohai does not currently depend on Chef and I suspect there is a fair
bit of weight behind keeping it that way (I’m not even sure from
the above whether “move Chef::JSONCompat to ohai” means introducing
such a dependency or not).

I’d like to see both Ohai and Chef move to using yajl-ruby as a
wholesale replacement for the json gem (not using a compatability mode
and not offering to fall-back to json gem). My impression is that
yajl-ruby is more stable in terms of API and I have observed
reasonable speed and memory-use improvements over json gem when
testing it.

The tricky part of such a move is the interaction with Merb in the
Chef server, where Merb currently uses the json gem and does JSON
serialization auto-magically.

I’d put the priority fairly high since we believe that switching to
yajl will remove compatibility/version issues that we’ve seen with the
json gem and will provide performance improvements.

  • seth


Seth Falcon | Senior Software Design Engineer | Opscode | @sfalcon


#4

Looks correct, but:

  1. Ohai currently depends on json gem because generates it.
  2. Chef depends on Ohai.
  3. Moving of JSONCompat layer to ohai will free both ohai and chef from json
    dependency.

ohai will never depend on chef, of course.

Your idea with full rewriting to Yajl::Parser and Yajl::Encoder like like
more rosust.

By the way, after release of Rails 3 we can move away from Merb (and, of
course, yajl-ruby is supported directly by Rails 3).

2011/3/7 Seth Falcon seth@opscode.com

Hi all,

On Sat, Mar 5, 2011 at 1:18 AM, Akzhan Abdulin akzhan.abdulin@gmail.com
wrote:

I suppose that we need to

  1. Move Chef::JSONCompat to ohai gem.
  2. Use Ohai::JSONCompat everywhere we need to.
  3. Switch to yajl-ruby as last step.

Ohai does not currently depend on Chef and I suspect there is a fair
bit of weight behind keeping it that way (I’m not even sure from
the above whether “move Chef::JSONCompat to ohai” means introducing
such a dependency or not).

I’d like to see both Ohai and Chef move to using yajl-ruby as a
wholesale replacement for the json gem (not using a compatability mode
and not offering to fall-back to json gem). My impression is that
yajl-ruby is more stable in terms of API and I have observed
reasonable speed and memory-use improvements over json gem when
testing it.

The tricky part of such a move is the interaction with Merb in the
Chef server, where Merb currently uses the json gem and does JSON
serialization auto-magically.

I’d put the priority fairly high since we believe that switching to
yajl will remove compatibility/version issues that we’ve seen with the
json gem and will provide performance improvements.

  • seth


Seth Falcon | Senior Software Design Engineer | Opscode | @sfalcon


#5

typos. please read as “looks like more robust”.

2011/3/8 Akzhan Abdulin akzhan.abdulin@gmail.com

Your idea with full rewriting to Yajl::Parser and Yajl::Encoder like like
more rosust.


#6

On Mon, Mar 7, 2011 at 11:24 PM, Akzhan Abdulin
akzhan.abdulin@gmail.com wrote:

Your idea with full rewriting to Yajl::Parser and Yajl::Encoder like like
more rosust.

It sounds like we’re all in agreement that we should move wholesale to
yajl and drop our JSONCompat wrapper rather than try to maintain an
API compatibility layer ourselves.

Bryan