Reading two JSON objects with Gson

[Kent Beck]

I successfully started using Gson to read JSON objects. Now, however I
am having a problem and I hope someone here can help.

I would like to read two JSON objects from the same stream in
succession. Using Jetty this works as I expect--two calls to the
reading method result in two (in this case) Strings:

@Test public void jettyReadWriteTwoStrings() throws IOException {
String one= JSON.toString("one");
String two= JSON.toString("two");
CharArrayWriter writer= new CharArrayWriter();
writer.write(one.toCharArray());
writer.write(two.toCharArray());
CharArrayReader reader = new CharArrayReader(writer.toCharArray());
String actualOne = (String) JSON.parse(reader);
assertEquals("one", actualOne);
String actualTwo = (String) JSON.parse(reader);
assertEquals("two", actualTwo);
}

However, the equivalent code does not work with Gson:

@Test public void readWriteTwoStrings() throws IOException {
Gson gson= new Gson();
CharArrayWriter writer= new CharArrayWriter();
writer.write(gson.toJson("one").toCharArray());
writer.write(gson.toJson("two").toCharArray());
CharArrayReader reader = new CharArrayReader(writer.toCharArray());
String actualOne = gson.fromJson(reader, String.class);
assertEquals("one", actualOne);
String actualTwo = gson.fromJson(reader, String.class); //
JsonParseException...
assertEquals("two", actualTwo);
}

Here is the stack trace:

com.google.gson.JsonParseException: Failed parsing JSON source:
java.io.CharArrayReader@8965fb to Json
at com.google.gson.JsonParser.parse(JsonParser.java:57)
at com.google.gson.Gson.fromJson(Gson.java:376)
at com.google.gson.Gson.fromJson(Gson.java:352)
at GsonTest.readWriteTwoStrings(GsonTest.java:29)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke
(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke
(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.junit.internal.runners.TestMethodRunner.executeMethodBody
(TestMethodRunner.java:99)
at org.junit.internal.runners.TestMethodRunner.runUnprotected
(TestMethodRunner.java:81)
at org.junit.internal.runners.BeforeAndAfterRunner.runProtected
(BeforeAndAfterRunner.java:34)
at org.junit.internal.runners.TestMethodRunner.runMethod
(TestMethodRunner.java:75)
at org.junit.internal.runners.TestMethodRunner.run
(TestMethodRunner.java:45)
at org.junit.internal.runners.TestClassMethodsRunner.invokeTestMethod
(TestClassMethodsRunner.java:66)
at org.junit.internal.runners.TestClassMethodsRunner.run
(TestClassMethodsRunner.java:35)
at org.junit.internal.runners.TestClassRunner$1.runUnprotected
(TestClassRunner.java:42)
at org.junit.internal.runners.BeforeAndAfterRunner.runProtected
(BeforeAndAfterRunner.java:34)
at org.junit.internal.runners.TestClassRunner.run
(TestClassRunner.java:52)
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run
(JUnit4TestReference.java:45)
at org.eclipse.jdt.internal.junit.runner.TestExecution.run
(TestExecution.java:38)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests
(RemoteTestRunner.java:460)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests
(RemoteTestRunner.java:673)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run
(RemoteTestRunner.java:386)
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main
(RemoteTestRunner.java:196)
Caused by: com.google.gson.ParseException: Encountered "<EOF>" at line
0, column 0.
Was expecting one of:
<DIGITS> ...
"null" ...
"NaN" ...
"Infinity" ...
<BOOLEAN> ...
<SINGLE_QUOTE_LITERAL> ...
<DOUBLE_QUOTE_LITERAL> ...
")]}\'\n" ...
"{" ...
"[" ...
"-" ...

at com.google.gson.JsonParserJavacc.generateParseException
(JsonParserJavacc.java:658)
at com.google.gson.JsonParserJavacc.jj_consume_token
(JsonParserJavacc.java:540)
at com.google.gson.JsonParserJavacc.parse(JsonParserJavacc.java:38)
at com.google.gson.JsonParser.parse(JsonParser.java:53)
... 23 more



Does anyone have a suggestion for reading two items with Gson?

Regards,

Kent Beck
Three Rivers Institute

[inder]

The reason it doesn't work is because if you concatenate two JSON
strings, the output is not valid JSON.
Here is a test I wrote (borrowing from your code) to illustrate this:

public void testReadWriteTwoObjects() throws IOException {

Gson gson= new Gson();
CharArrayWriter writer= new CharArrayWriter();

BagOfPrimitives expectedOne = new BagOfPrimitives(1, 1, true,
"one");
writer.write(gson.toJson(expectedOne).toCharArray());
BagOfPrimitives expectedTwo = new BagOfPrimitives(2, 2, false,
"two");
writer.write(gson.toJson(expectedTwo).toCharArray());
System.out.println(writer.toCharArray());
CharArrayReader reader = new CharArrayReader(writer.toCharArray
());
BagOfPrimitives actualOne = gson.fromJson(reader,
BagOfPrimitives.class);
assertEquals("one", actualOne.stringValue);
BagOfPrimitives actualTwo = gson.fromJson(reader,
BagOfPrimitives.class);
assertEquals("two", actualTwo.stringValue);
}

The output JSON is:
{"longValue":1,"intValue":1,"booleanValue":true,"stringValue":"one"}
{"longValue":2,"intValue":2,"booleanValue":false,"stringValue":"two"}

Now when a JSON parser is parsing this input, it would see an invalid
start of an Object where the second { starts. I dont know how Jetty
handles it, but it probably is parsing the input selectively. Gson
uses a real parser and has a parsing phase which will reject any
invalid input.

I will look into modifying Gson so that its parser stops parsing when
the initial production finishes.

Inder




ps. The BagOfPrimitives is defined to be:
public static class BagOfPrimitives {
public static final long DEFAULT_VALUE = 0;
public long longValue;
public int intValue;
public boolean booleanValue;
public String stringValue;
public BagOfPrimitives(long longValue, int intValue, boolean
booleanValue, String stringValue) {
this.longValue = longValue;
this.intValue = intValue;
this.booleanValue = booleanValue;
this.stringValue = stringValue;

[inder]

On further investigations, I realized that it is not really a parser
issue. What is happening is that parser is consuming more input than
it should while parsing the first object.
However, the second fromJson() invocation creates a new parser
throwing away the old one (alongwith its stream that read additional
data). This new parser doesn't really find valid input, and throws
back an error.

This is a somewhat deeper problem that may require us to expose the
parser directly that you can use with Gson.fromJson(JsonElement, Type)
method. We will investigate it further and see how we can support it.

Thanks
Inder

[inder]

I filed a bug on us for this issue:
http://code.google.com/p/google-gson/issues/detail?id=123

[inder]

I fixed this issue in Gson by adding a new method in JsonParser class
parseAsync() that allows the user to read multiple JSON elements off
the stream asynchronously.

See: http://code.google.com/p/google-gson/source/detail?r=422

[Kent Beck]

Thank you for your rapid attention to this issue.

Best regards,

Kent