Handling multiple objects with one ObjectReader?

[John Baima]

ObjectReader's "hasNext()" and "next()" don't appear to do anything on the object scope, only on the object detail scope and deeper. How are we intended to iterate through multiple objects in one file?

In order to do this I had to implement two hacks; one to insert line breaks between every object in ObjectWriter:

objectwriter.flush(); printwriter.write("\n");

And one to calculate the # of lines in a saved file via a BufferedReader, so that I could do

while (objectreader.row() < linecount)

to finally gain the ability to iterate through multiple objects.

I probably would have done the line-break hack anyway just to make it more readable (surprised this isn't either the default or an option), but this seems like a kind of silly/hackish solution to something that I assumed would be dead simple.

Am I just missing something here or is this just a strange omission in functionality?

[Eugen Cepoi]

Hi John,

2014-11-14 17:25 GMT+01:00 John Baima <malu...@gmail.com>:

ObjectReader's "hasNext()" and "next()" don't appear to do anything on the object scope, only on the object detail scope and deeper.

I am not sure to understand what you mean by "object scope" and "object detail scope".
hasNext will consume non "token" such as white space, line feeds etc until it reaches a valid or invalid character in a json stream. Next(), will change the ObjectReader state as it will advance/parse (key)/value (and return the type of the value being readen).

 

How are we intended to iterate through multiple objects in one file?

If it is a json array containing json values (obj, array, or litterals) you can use hasNext and next like you would do with an iterator.

But I guess you are not talking about that, but rather appending json values to a file (not enclosed in a valid json structure) and want to iterate through them, right?

It looks like a regression that I introduced, I will have a look on how to fix it for next release (soon).

 

In order to do this I had to implement two hacks; one to insert line breaks between every object in ObjectWriter:

objectwriter.flush(); printwriter.write("\n");

And one to calculate the # of lines in a saved file via a BufferedReader, so that I could do

while (objectreader.row() < linecount)

to finally gain the ability to iterate through multiple objects.

I probably would have done the line-break hack anyway just to make it more readable (surprised this isn't either the default or an option), but this seems like a kind of silly/hackish solution to something that I assumed would be dead simple.

Yep indeed it is hackish (sorry you have to do that to solve this problem).

 

Am I just missing something here or is this just a strange omission in functionality?

Would you mind providing more details on how you are using Genson in this context?

For example I wonder if you are doing manual parsing of the json or iterating and mapping each root object to some Pojo (using genson databinding)?

And in general could you describe (words or pseudo code) what kind of behaviour/api you would like to see for that?

For example:

// that would work with a root json array or just a sequence of json values

// note that here it will map every root object to the same type Foo

Iterator<Foo> fooIterator = genson.iterate(inputStream, Foo.class);

// or more low level

ObjectReader reader = genson.createReader(inputStream);

while(reader.hasNext()) {

  reader.beginObject();

  // read key/values

  reader.enbObject();

}

This would help me provide a solution that answers to concrete use and expectations.

Thanks for reporting it.

Eugen

 

--
You received this message because you are subscribed to the Google Groups "Genson user group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to genson+un...@googlegroups.com.
To post to this group, send email to gen...@googlegroups.com.
Visit this group at http://groups.google.com/group/genson.
To view this discussion on the web visit https://groups.google.com/d/msgid/genson/b73359bf-db1f-4c6d-b846-3ac4c26be192%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Eugen Cepoi]

Hi John,

I’ve made a fix for this + implementing both options, I have also pushed a snapshot to sonatype, you can get it here.

Is this kind of API/behaviour that you were expecting? So for example to bind a a sequence of object (not enclosed in an array) you would do the following:

Genson genson = new GensonBuilder().usePermissiveParsing(true).create();
ObjectReader reader = genson.createReader(json);

for (Iterator<LogEntry> it = genson.deserializeValues(reader, GenericType.of(LogEntry.class));
  it.hasNext(); ) {
  // do something
  LogEntry p = it.next();
}

You can also use directly the ObjectReader:

for (ObjectReader reader = genson.createReader(new StringReader("{\"a\":1}{\"a\":2}"));
  reader.hasNext(); reader.next()) {
  reader.beginObject();
  // advances to the key value pair and consumes it.
  reader.next();
  System.out.println(reader.name()+"="+reader.valueAsInt());
  reader.enbObject();    
}

Eugen

​

[John Baima]

Hi eugen,

I am admittedly somewhat of a noob when it comes to this stuff, so apparently this issue is mostly borne of a confusion between both understanding how the JSON is supposed to be formatted (an array of objects) and understanding that ObjectReader will only parse a single JSON string. Indeed, instead of having a JSON array of objects I simply have a bunch of objects (multiple JSON strings) listed independently one after the other. Like

{Object blah details} {Object blah details} {Object blah details}

and then I would iterate through each of those and deserialize them manually via the ObjectReader. Without any coaxing (the hacks I mentioned earlier), it just reads the first one and ignores the rest.

The latter code example is exactly what I am looking for, though; a for-loop that uses hasNext() and next() to iterate through each of those objects. However, I tried the snapshot and got the same result of it only ever reading/using the first object.

That said, now I know I can fix this simply by serializing it correctly, as in an array of objects, so I guess this isn't necessary. Would be nice to have anyway, though.

[Eugen Cepoi]

In fact this usage is not unwise, but it depends on the scenario.

For example assume you have some web app with an apache or nginx that will log the http calls as json. They will not be enclosed in an array, but appended as json objects. This can also be common if you are writting your data as json in hadoop jobs.

Enclosing them in an array and having a valid json or just appending them depends on your use case. Without more details I can't say much.

Anyway I still would like to fix it. Are you sure you created your Genson instance with this option enabled:

new GensonBuilder().usePermissiveParsing(true).create();

 Could you please provide some small example (json+code that reads it) that does not work with the snapshot version?

Thanks,

Eugen

--

You received this message because you are subscribed to the Google Groups "Genson user group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to genson+un...@googlegroups.com.
To post to this group, send email to gen...@googlegroups.com.
Visit this group at http://groups.google.com/group/genson.

To view this discussion on the web visit https://groups.google.com/d/msgid/genson/604da4a0-0458-47ef-906c-8f180a398d00%40googlegroups.com.

[John Baima]

(Sorry if I am accidentally multi-posting, my posts seem to keep not appearing?)

Ah! Wonderful. Your build works perfectly. I simply had not enabled permissive parsing. Didn't realize it was a prerequisite. Thank you very much, both for your prompt responses and implementing exactly what I wanted! :)

[Eugen Cepoi]

Ah strange, can't see any other attempt on the google group...

Great, I will make a release next week.

You are welcome :)
Eugen

--

You received this message because you are subscribed to the Google Groups "Genson user group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to genson+un...@googlegroups.com.
To post to this group, send email to gen...@googlegroups.com.
Visit this group at http://groups.google.com/group/genson.

To view this discussion on the web visit https://groups.google.com/d/msgid/genson/dec51f99-49a5-4e71-97e6-c8a6336f899b%40googlegroups.com.