Decrypted data bag missing data, ruby 1.8.7 vs rvm 1.9.3

hi. anyone know what’s going on here? i was updating an encrypted data bag
item today. when i decrypted it in order to edit the json, i found entries
missing in the decrypted form when using rvm 1.9.3.

the entries missing were all “_cert” entries in the decrypted form. in the
decrypted form all the “_key” entries were intact. when displaying the data
bag item in encrypted form, all entires were intact. see below.

anyone know what’s going on?

thanks,
kallen

[zahadum chef-repo] rvm use 1.9.3 Using /home/kallen/.rvm/gems/ruby-1.9.3-p194 Running /home/kallen/.rvm/hooks/after_use [zahadum chef-repo] knife data bag show secrets webproxy-vacaynow-x509 --secret-file ~/.chef/encrypted_data_bag_secret -Fj > decrypted.omfg.json

decrypted.omfg.json looks like this, entries missing:

{
“wildcard_mysite_key”: “-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKCA\nxxxxSNIP_GOOD_DATA\n”,
“wildcard_mysite_cert”: “”,
“wildcard_anothersite-admin_key”: “-----BEGIN PRIVATE KEY-----\nMIIEvQIB\nxxxSNIP_GOOD_DATA\n”,
“wildcard_anothersite_cert”: “”,
“id”: “webproxy-vacaynow-x509”
}

if i don’t decrypt, the “_cert” entries are intact:

[zahadum chef-repo]$ knife data bag show secrets webproxy-vacaynow-x509 -Fj | grep _cert
"wildcard_mysite_cert": “Vcd646sJ/Cy8xxxx we’re cool”,
“wildcard_anothersite_cert”: “Bg8xjDjVLh2oIOz8h we’re cool”,

if i use the system ruby, 1.8.7, this problem does not occur.

[zahadum chef-repo] which ruby /usr/bin/ruby [zahadum chef-repo] ruby -v
ruby 1.8.7 (2010-08-16 patchlevel 302) [i686-linux]
[zahadum chef-repo]$ knife data bag show secrets webproxy-vacaynow-x509 --secret-file ~/.chef/encrypted_data_bag_secret -Fj > decrypted.good.json

{
“wildcard_mysite_key”: “-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKCA\nxxxxSNIP_GOOD_DATA\n”,
“wildcard_mysite_cert”: “MIIFhTCCBG2gAw\nxxxxSNIP_GOOD_DATA\n”,
“wildcard_anothersite-admin_key”: “-----BEGIN PRIVATE KEY-----\nMIIEvQIB\nxxxSNIP_GOOD_DATA\n”,
“wildcard_mysite_cert”: “MIIFYTCCBEmgAw\nxxxxSNIP_GOOD_DATA\n”,
“id”: “webproxy-vacaynow-x509”
}

[zahadum chef-repo]$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 11.04
Release: 11.04
Codename: natty

We were affected by this in an installation yesterday.

I believe it is CHEF-3393 [0] but am still gathering details.

Cheers,

AJ

[0] http://tickets.opscode.com/browse/CHEF-3393

On 31 August 2012 09:35, kallen@groknaut.net wrote:

hi. anyone know what's going on here? i was updating an encrypted data bag
item today. when i decrypted it in order to edit the json, i found entries
missing in the decrypted form when using rvm 1.9.3.

the entries missing were all "_cert" entries in the decrypted form. in the
decrypted form all the "_key" entries were intact. when displaying the data
bag item in encrypted form, all entires were intact. see below.

anyone know what's going on?

thanks,
kallen

[zahadum chef-repo]$ rvm use 1.9.3
Using /home/kallen/.rvm/gems/ruby-1.9.3-p194
Running /home/kallen/.rvm/hooks/after_use
[zahadum chef-repo]$ knife data bag show secrets webproxy-vacaynow-x509 --secret-file ~/.chef/encrypted_data_bag_secret -Fj > decrypted.omfg.json

decrypted.omfg.json looks like this, entries missing:

{
"wildcard_mysite_key": "-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKCA\nxxxxSNIP_GOOD_DATA\n",
"wildcard_mysite_cert": "",
"wildcard_anothersite-admin_key": "-----BEGIN PRIVATE KEY-----\nMIIEvQIB\nxxxSNIP_GOOD_DATA\n",
"wildcard_anothersite_cert": "",
"id": "webproxy-vacaynow-x509"
}

if i don't decrypt, the "_cert" entries are intact:

[zahadum chef-repo]$ knife data bag show secrets webproxy-vacaynow-x509 -Fj | grep _cert
"wildcard_mysite_cert": "Vcd646sJ/Cy8xxxx we're cool",
"wildcard_anothersite_cert": "Bg8xjDjVLh2oIOz8h we're cool",

if i use the system ruby, 1.8.7, this problem does not occur.

[zahadum chef-repo]$ which ruby
/usr/bin/ruby
[zahadum chef-repo]$ ruby -v
ruby 1.8.7 (2010-08-16 patchlevel 302) [i686-linux]
[zahadum chef-repo]$ knife data bag show secrets webproxy-vacaynow-x509 --secret-file ~/.chef/encrypted_data_bag_secret -Fj > decrypted.good.json

{
"wildcard_mysite_key": "-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKCA\nxxxxSNIP_GOOD_DATA\n",
"wildcard_mysite_cert": "MIIFhTCCBG2gAw\nxxxxSNIP_GOOD_DATA\n",
"wildcard_anothersite-admin_key": "-----BEGIN PRIVATE KEY-----\nMIIEvQIB\nxxxSNIP_GOOD_DATA\n",
"wildcard_mysite_cert": "MIIFYTCCBEmgAw\nxxxxSNIP_GOOD_DATA\n",
"id": "webproxy-vacaynow-x509"
}

[zahadum chef-repo]$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 11.04
Release: 11.04
Codename: natty

On Thursday, August 30, 2012 at 2:35 PM, kallen@groknaut.net wrote:

hi. anyone know what's going on here? i was updating an encrypted data bag
item today. when i decrypted it in order to edit the json, i found entries
missing in the decrypted form when using rvm 1.9.3.

the entries missing were all "_cert" entries in the decrypted form. in the
decrypted form all the "_key" entries were intact. when displaying the data
bag item in encrypted form, all entires were intact. see below.

anyone know what's going on?

thanks,
kallen

[zahadum chef-repo]$ rvm use 1.9.3
Using /home/kallen/.rvm/gems/ruby-1.9.3-p194
Running /home/kallen/.rvm/hooks/after_use
[zahadum chef-repo]$ knife data bag show secrets webproxy-vacaynow-x509 --secret-file ~/.chef/encrypted_data_bag_secret -Fj > decrypted.omfg.json

decrypted.omfg.json looks like this, entries missing:

{
"wildcard_mysite_key": "-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKCA\nxxxxSNIP_GOOD_DATA\n",
"wildcard_mysite_cert": "",
"wildcard_anothersite-admin_key": "-----BEGIN PRIVATE KEY-----\nMIIEvQIB\nxxxSNIP_GOOD_DATA\n",
"wildcard_anothersite_cert": "",
"id": "webproxy-vacaynow-x509"
}

if i don't decrypt, the "_cert" entries are intact:

[zahadum chef-repo]$ knife data bag show secrets webproxy-vacaynow-x509 -Fj | grep _cert
"wildcard_mysite_cert": "Vcd646sJ/Cy8xxxx we're cool",
"wildcard_anothersite_cert": "Bg8xjDjVLh2oIOz8h we're cool",

if i use the system ruby, 1.8.7, this problem does not occur.

[zahadum chef-repo]$ which ruby
/usr/bin/ruby
[zahadum chef-repo]$ ruby -v
ruby 1.8.7 (2010-08-16 patchlevel 302) [i686-linux]
[zahadum chef-repo]$ knife data bag show secrets webproxy-vacaynow-x509 --secret-file ~/.chef/encrypted_data_bag_secret -Fj > decrypted.good.json

{
"wildcard_mysite_key": "-----BEGIN RSA PRIVATE KEY-----\nMIIEpAIBAAKCA\nxxxxSNIP_GOOD_DATA\n",
"wildcard_mysite_cert": "MIIFhTCCBG2gAw\nxxxxSNIP_GOOD_DATA\n",
"wildcard_anothersite-admin_key": "-----BEGIN PRIVATE KEY-----\nMIIEvQIB\nxxxSNIP_GOOD_DATA\n",
"wildcard_mysite_cert": "MIIFYTCCBEmgAw\nxxxxSNIP_GOOD_DATA\n",
"id": "webproxy-vacaynow-x509"
}

[zahadum chef-repo]$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 11.04
Release: 11.04
Codename: natty

http://tickets.opscode.com/browse/CHEF-3393

Workaround is:
YAML::ENGINE.yamler = 'syck' if RUBY_VERSION > '1.9'

...until we figure out a fix that doesn't break people or just kick the can down the road.

--
Daniel DeLeo

Dan,

Where do you chuck that? knife.rb (encryption) or client.rb
(decryption) or bolth?

--AJ

On 31 August 2012 09:40, Daniel DeLeo dan@kallistec.com wrote:

On Thursday, August 30, 2012 at 2:35 PM, kallen@groknaut.net wrote:

hi. anyone know what's going on here? i was updating an encrypted data bag
item today. when i decrypted it in order to edit the json, i found entries
missing in the decrypted form when using rvm 1.9.3.

the entries missing were all "_cert" entries in the decrypted form. in the
decrypted form all the "_key" entries were intact. when displaying the data
bag item in encrypted form, all entires were intact. see below.

anyone know what's going on?

thanks,
kallen

[zahadum chef-repo]$ rvm use 1.9.3
Using /home/kallen/.rvm/gems/ruby-1.9.3-p194
Running /home/kallen/.rvm/hooks/after_use
[zahadum chef-repo]$ knife data bag show secrets webproxy-vacaynow-x509
--secret-file ~/.chef/encrypted_data_bag_secret -Fj > decrypted.omfg.json

decrypted.omfg.json looks like this, entries missing:

{
"wildcard_mysite_key": "-----BEGIN RSA PRIVATE
KEY-----\nMIIEpAIBAAKCA\nxxxxSNIP_GOOD_DATA\n",
"wildcard_mysite_cert": "",
"wildcard_anothersite-admin_key": "-----BEGIN PRIVATE
KEY-----\nMIIEvQIB\nxxxSNIP_GOOD_DATA\n",
"wildcard_anothersite_cert": "",
"id": "webproxy-vacaynow-x509"
}

if i don't decrypt, the "_cert" entries are intact:

[zahadum chef-repo]$ knife data bag show secrets webproxy-vacaynow-x509 -Fj
| grep _cert
"wildcard_mysite_cert": "Vcd646sJ/Cy8xxxx we're cool",
"wildcard_anothersite_cert": "Bg8xjDjVLh2oIOz8h we're cool",

if i use the system ruby, 1.8.7, this problem does not occur.

[zahadum chef-repo]$ which ruby
/usr/bin/ruby
[zahadum chef-repo]$ ruby -v
ruby 1.8.7 (2010-08-16 patchlevel 302) [i686-linux]
[zahadum chef-repo]$ knife data bag show secrets webproxy-vacaynow-x509
--secret-file ~/.chef/encrypted_data_bag_secret -Fj > decrypted.good.json

{
"wildcard_mysite_key": "-----BEGIN RSA PRIVATE
KEY-----\nMIIEpAIBAAKCA\nxxxxSNIP_GOOD_DATA\n",
"wildcard_mysite_cert": "MIIFhTCCBG2gAw\nxxxxSNIP_GOOD_DATA\n",
"wildcard_anothersite-admin_key": "-----BEGIN PRIVATE
KEY-----\nMIIEvQIB\nxxxSNIP_GOOD_DATA\n",
"wildcard_mysite_cert": "MIIFYTCCBEmgAw\nxxxxSNIP_GOOD_DATA\n",
"id": "webproxy-vacaynow-x509"
}

[zahadum chef-repo]$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 11.04
Release: 11.04
Codename: natty

http://tickets.opscode.com/browse/CHEF-3393

Workaround is:
YAML::ENGINE.yamler = 'syck' if RUBY_VERSION > '1.9'

...until we figure out a fix that doesn't break people or just kick the can
down the road.

--
Daniel DeLeo

woop! thanks y'all.

On Thu, 30 Aug 2012, Daniel DeLeo wrote:

On Thursday, August 30, 2012 at 2:35 PM, kallen@groknaut.net wrote:

hi. anyone know what's going on here? i was updating an encrypted data bag
item today. when i decrypted it in order to edit the json, i found entries
missing in the decrypted form when using rvm 1.9.3.

http://tickets.opscode.com/browse/CHEF-3393

Workaround is:
YAML::ENGINE.yamler = 'syck' if RUBY_VERSION > '1.9'

...until we figure out a fix that doesn't break people or just kick the can down the road.

--
Daniel DeLeo

On Thursday, August 30, 2012 at 2:42 PM, AJ Christensen wrote:

Dan,

Where do you chuck that? knife.rb (encryption) or client.rb
(decryption) or bolth?

--AJ
Put it everywhere you're using Ruby 1.9.3, which as of Chef 10.14 includes omnibus chef-client installs.

One thing we're trying to figure out to determine the best solution is if the YAML engine in Ruby 1.9.2 and below (syck) will correctly parse YAML from the Ruby 1.9.3 engine (psych).

--
Daniel DeLeo

http://tickets.opscode.com/browse/CHEF-3393

Workaround is:
YAML::ENGINE.yamler = 'syck' if RUBY_VERSION > '1.9'

...until we figure out a fix that doesn't break people or just kick the can
down the road.

--
Daniel DeLeo