Knife diff is failing with a lexical error


#1

I just started trying to use knife diff, and it will (very) often fail with an error like the following.

ERROR: Yajl::ParseError: lexical error: invalid char in json text.
# this is very hard coded for a
(right here) ------^

It appears that something is trying to parse a ruby file as if were json.

It doesn’t happen for every ruby file. I have started doing some specific test cases, and it seems to mostly occur with .rb files that I have created, but not all the ones that I have created. I have also seen it happen on a few .erb files as well.

Has anyone else seen this, and, more importantly, is there a fix or work around?


Stephen Corbesero, PAC
Synchronoss Technologies, Inc., Bethlehem, PA, USA


#2

I’ve seen this error before on a chef client run, but have never been able to reproduce or find the root cause. There was an issue opened: http://tickets.opscode.com/browse/CHEF-4032

Sorry I don’t have a fix, but you’re not the first one to see that particular exception.

Nathan L Smith
smith@opscode.com
(319) 339-0466

From: Stephen Corbesero <Stephen.Corbesero@synchronoss.commailto:Stephen.Corbesero@synchronoss.com>
Reply-To: "chef@lists.opscode.commailto:chef@lists.opscode.com" <chef@lists.opscode.commailto:chef@lists.opscode.com>
Date: Friday, July 12, 2013 8:51 AM
To: “‘chef@lists.opscode.commailto:'chef@lists.opscode.com’” <chef@lists.opscode.commailto:chef@lists.opscode.com>
Subject: [chef] knife diff is failing with a lexical error

I just started trying to use knife diff, and it will (very) often fail with an error like the following.

ERROR: Yajl::ParseError: lexical error: invalid char in json text.
# this is very hard coded for a
(right here) ------^

It appears that something is trying to parse a ruby file as if were json.

It doesn’t happen for every ruby file. I have started doing some specific test cases, and it seems to mostly occur with .rb files that I have created, but not all the ones that I have created. I have also seen it happen on a few .erb files as well.

Has anyone else seen this, and, more importantly, is there a fix or work around?


Stephen Corbesero, PAC
Synchronoss Technologies, Inc., Bethlehem, PA, USA


#3

I had this exact problem the other day. It turned out that I had a
backslash in a data bag, which choked the JSON parser. How that data bag
ever got created with that character in the first place, I have no idea…

What I did to find the problematic file is to delete my local repository
and use knife download .
This causes knife to download each file, one-by-one. When you get the
error, look at the last file that was successfully download. Compare this
with what you see on the Chef server - the next item alphabetically is your
problem child.

Ex:

Downloading data_bags/animals/camel.json
Downloading data_bags/animals/donkey.json

ERROR: Yajl::ParseError: lexical error: invalid char in json text.****

                                   # this is very hard coded for a****

      (right here) ------^

So, whatever your next data bag is would be the problem (not saying this is
necessarily one of your data-bags - just an example)

Good luck!

On Fri, Jul 12, 2013 at 9:55 AM, Nathan Smith smith@opscode.com wrote:

I’ve seen this error before on a chef client run, but have never been
able to reproduce or find the root cause. There was an issue opened:
http://tickets.opscode.com/browse/CHEF-4032

Sorry I don’t have a fix, but you’re not the first one to see that
particular exception.

Nathan L Smith
smith@opscode.com
(319) 339-0466

From: Stephen Corbesero Stephen.Corbesero@synchronoss.com
Reply-To: "chef@lists.opscode.com" chef@lists.opscode.com
Date: Friday, July 12, 2013 8:51 AM
To: “‘chef@lists.opscode.com’” chef@lists.opscode.com
Subject: [chef] knife diff is failing with a lexical error

I just started trying to use knife diff, and it will (very) often fail
with an error like the following.****


ERROR: Yajl::ParseError: lexical error: invalid char in json text.****

                                   # this is very hard coded for a****

      (right here) ------^****

It appears that something is trying to parse a ruby file as if were json.*



It doesn’t happen for every ruby file. I have started doing some specific
test cases, and it seems to mostly occur with .rb files that I have
created, but not all the ones that I have created. I have also seen it
happen on a few .erb files as well.****


Has anyone else seen this, and, more importantly, is there a fix or work
around?****



–****

Stephen Corbesero, PAC****

Synchronoss Technologies, Inc., Bethlehem, PA, USA****



#4

The source of this bug is that knife diff only supports json format for roles/environments, rather than .rb files. It should not be happening for .erb files or files under cookbooks/ in general.

Which files specifically are you having the issue with? Does it still happen after you “gem install knife-essentials”? The problem might go away, since there have been many fixes to this sort of thing (which are coming to a Chef 11.6 near you).

–John


From: Stephen Corbesero Stephen.Corbesero@synchronoss.com
Sent: Friday, July 12, 2013 6:51 AM
To: 'chef@lists.opscode.com’
Subject: [chef] knife diff is failing with a lexical error

I just started trying to use knife diff, and it will (very) often fail with an error like the following.

ERROR: Yajl::ParseError: lexical error: invalid char in json text.
# this is very hard coded for a
(right here) ------^

It appears that something is trying to parse a ruby file as if were json.

It doesn’t happen for every ruby file. I have started doing some specific test cases, and it seems to mostly occur with .rb files that I have created, but not all the ones that I have created. I have also seen it happen on a few .erb files as well.

Has anyone else seen this, and, more importantly, is there a fix or work around?


Stephen Corbesero, PAC
Synchronoss Technologies, Inc., Bethlehem, PA, USA


#5

Would you mind trying the latest knife-essentials to see if the error message is better? I added a bunch of error handling to make the output better in cases like this, so that it would be easier to diagnose exactly what files are problematic.

gem install knife-essentials to get the very latest (Chef 11.6 will have it too).

–John


From: Christopher Armstrong chris@chrisarmstrong.me
Sent: Saturday, July 13, 2013 10:44 PM
To: chef@lists.opscode.com
Subject: [chef] Re: Re: knife diff is failing with a lexical error

I had this exact problem the other day. It turned out that I had a backslash in a data bag, which choked the JSON parser. How that data bag ever got created with that character in the first place, I have no idea…

What I did to find the problematic file is to delete my local repository and use knife download .
This causes knife to download each file, one-by-one. When you get the error, look at the last file that was successfully download. Compare this with what you see on the Chef server - the next item alphabetically is your problem child.

Ex:

Downloading data_bags/animals/camel.json
Downloading data_bags/animals/donkey.json

ERROR: Yajl::ParseError: lexical error: invalid char in json text.

                                   # this is very hard coded for a

      (right here) ------^

So, whatever your next data bag is would be the problem (not saying this is necessarily one of your data-bags - just an example)

Good luck!

On Fri, Jul 12, 2013 at 9:55 AM, Nathan Smith <smith@opscode.commailto:smith@opscode.com> wrote:
I’ve seen this error before on a chef client run, but have never been able to reproduce or find the root cause. There was an issue opened: http://tickets.opscode.com/browse/CHEF-4032

Sorry I don’t have a fix, but you’re not the first one to see that particular exception.

Nathan L Smith
smith@opscode.commailto:smith@opscode.com
(319) 339-0466tel:(319)%20339-0466

From: Stephen Corbesero <Stephen.Corbesero@synchronoss.commailto:Stephen.Corbesero@synchronoss.com>
Reply-To: "chef@lists.opscode.commailto:chef@lists.opscode.com" <chef@lists.opscode.commailto:chef@lists.opscode.com>
Date: Friday, July 12, 2013 8:51 AM
To: “‘chef@lists.opscode.commailto:'chef@lists.opscode.com’” <chef@lists.opscode.commailto:chef@lists.opscode.com>
Subject: [chef] knife diff is failing with a lexical error

I just started trying to use knife diff, and it will (very) often fail with an error like the following.

ERROR: Yajl::ParseError: lexical error: invalid char in json text.
# this is very hard coded for a
(right here) ------^

It appears that something is trying to parse a ruby file as if were json.

It doesn’t happen for every ruby file. I have started doing some specific test cases, and it seems to mostly occur with .rb files that I have created, but not all the ones that I have created. I have also seen it happen on a few .erb files as well.

Has anyone else seen this, and, more importantly, is there a fix or work around?


Stephen Corbesero, PAC
Synchronoss Technologies, Inc., Bethlehem, PA, USA