I need to override some elements and change the way they are
serialized depending on other fields within the object. Below the type
of object in the list of lists (data) depends on the column object
within the same object. How can I do this?
I could use just the low level expression stuff which I can then
traverse to build my own structure but can't find a tutorial on using
this without extras of the serializer.
Any tips?
public class PyTable
{
String id;
String name;
List<PyColumn> columns = new List<PyColumn>();
List<List<Object>> data = new List<List<Object>>();
}
public class PyColumn
{
const int TEXT = 0;
const int INTEGER = 1;
const int FLOAT = 2;
const int DATE = 3;
Here's an example of an Expression handler to get you started. It
sounds like maybe you just need to tell the serializer what the data
types are during deserialization so I've left GetExpression (called
during serialization) not doing much. Evaluate is called during
deserialization and I've put in some code there to get you started.
To set the type you basically need to wrap an expression in a
CastExpression passing in the appropriate data type. Here is the wiki
page for the different expression types: http://code.google.com/p/jsonexserializer/wiki/Expressions,
sorry there's nothing yet on the handlers themselves.
public class PyTableExpressionHandler : ObjectExpressionHandler
{
public override bool CanHandle(Type objectType)
{
return typeof(PyTable).IsAssignableFrom(objectType);
}
public override object Evaluate(Expression expression, object
existingObject, IDeserializerHandler deserializer)
{
ObjectExpression objExpr = (ObjectExpression) expression;
ArrayExpression columns = (ArrayExpression) objExpr
["columns"]; //List<PyColumn> columns
ArrayExpression data = (ArrayExpression) objExpr
["data"]; //List<List<Object>> data
// Add type info to data via Cast
for (int i = 0; i < columns.Items.Count; i++) {
// assume columns and data have same length
Type dataType = null;
// TODO: figure out data type from column info:
((ValueExpression)((ObjectExpression)columns[i])
["typeId"]).StringValue
ArrayExpression dataRow = (ArrayExpression) data.Items
[i];
for (int j = 0; j < dataRow.Items.Count; j++) {
dataRow.Items[j] = new CastExpression(dataType,
dataRow.Items[i]);
}
}
return base.Evaluate(expression, existingObject,
deserializer);
}
}
Here is how to set it up in the serializer:
Serializer s = new Serializer(typeof(List<PyTable>));
s.Context.ExpressionHandlers.Insert(0, new
PyTableExpressionHandler());
If your object is really as simple as you've shown you could
potentially evaluate some of it on your own and just put it in the
final expression. You can evaluate any expression by passing it to
the IDeserializerHandler instance that gets passed into the Evaluate
method. You just need to set the ResultType property on the
expression first. If you do go this route, just make sure you remove
any child expressions from the main expression that you've already
evaluated before passing them to the base method, and then update the
object that gets passed back to you.
On Sep 10, 11:11 am, John <j...@jdiligence.com> wrote:
> I need to override some elements and change the way they are
> serialized depending on other fields within the object. Below the type
> of object in the list of lists (data) depends on the column object
> within the same object. How can I do this?
> I could use just the low level expression stuff which I can then
> traverse to build my own structure but can't find a tutorial on using
> this without extras of the serializer.
> Any tips?
> public class PyTable
> {
> String id;
> String name;
> List<PyColumn> columns = new List<PyColumn>();
> List<List<Object>> data = new List<List<Object>>();
> }
> public class PyColumn
> {
> const int TEXT = 0;
> const int INTEGER = 1;
> const int FLOAT = 2;
> const int DATE = 3;
> Here's an example of an Expression handler to get you started. It
> sounds like maybe you just need to tell the serializer what the data
> types are during deserialization so I've left GetExpression (called
> during serialization) not doing much. Evaluate is called during
> deserialization and I've put in some code there to get you started.
> To set the type you basically need to wrap an expression in a
> CastExpression passing in the appropriate data type. Here is the wiki
> page for the different expression types:http://code.google.com/p/jsonexserializer/wiki/Expressions,
> sorry there's nothing yet on the handlers themselves.
> public class PyTableExpressionHandler : ObjectExpressionHandler
> {
> public override object Evaluate(Expression expression, object
> existingObject, IDeserializerHandler deserializer)
> {
> ObjectExpression objExpr = (ObjectExpression) expression;
> ArrayExpression columns = (ArrayExpression) objExpr
> ["columns"]; //List<PyColumn> columns
> ArrayExpression data = (ArrayExpression) objExpr
> ["data"]; //List<List<Object>> data
> // Add type info to data via Cast
> for (int i = 0; i < columns.Items.Count; i++) {
> // assume columns and data have same length
> Type dataType = null;
> // TODO: figure out data type from column info:
> ((ValueExpression)((ObjectExpression)columns[i])
> ["typeId"]).StringValue
> ArrayExpression dataRow = (ArrayExpression) data.Items
> [i];
> for (int j = 0; j < dataRow.Items.Count; j++) {
> dataRow.Items[j] = new CastExpression(dataType,
> dataRow.Items[i]);
> }
> }
> return base.Evaluate(expression, existingObject,
> deserializer);
> }
> }
> Here is how to set it up in the serializer:
> Serializer s = new Serializer(typeof(List<PyTable>));
> s.Context.ExpressionHandlers.Insert(0, new
> PyTableExpressionHandler());
> If your object is really as simple as you've shown you could
> potentially evaluate some of it on your own and just put it in the
> final expression. You can evaluate any expression by passing it to
> the IDeserializerHandler instance that gets passed into the Evaluate
> method. You just need to set the ResultType property on the
> expression first. If you do go this route, just make sure you remove
> any child expressions from the main expression that you've already
> evaluated before passing them to the base method, and then update the
> object that gets passed back to you.
> On Sep 10, 11:11 am, John <j...@jdiligence.com> wrote:
> > I need to override some elements and change the way they are
> > serialized depending on other fields within the object. Below the type
> > of object in the list of lists (data) depends on the column object
> > within the same object. How can I do this?
> > I could use just the low level expression stuff which I can then
> > traverse to build my own structure but can't find a tutorial on using
> > this without extras of the serializer.
> > Any tips?
> > public class PyTable
> > {
> > String id;
> > String name;
> > List<PyColumn> columns = new List<PyColumn>();
> > List<List<Object>> data = new List<List<Object>>();
> > }
> > public class PyColumn
> > {
> > const int TEXT = 0;
> > const int INTEGER = 1;
> > const int FLOAT = 2;
> > const int DATE = 3;
Also note, I've been doing a little refactoring in the trunk so the
types are slightly different than what's in the release version:
New to Old
IConfiguration --> SerializationContext
IExpressionBuilder -> ISerializerHandler
On Sep 13, 1:12 am, Ted Elliott <elliott....@gmail.com> wrote:
> I've been thinking about writing an expression handler for DataTable
> and DataSet for a while now, so I decided to go ahead and code one
> since it is very similar to your issue. The expression handler is
> here:http://code.google.com/p/jsonexserializer/source/browse/trunk/JsonExS...
> On Sep 12, 11:33 pm, Ted Elliott <elliott....@gmail.com> wrote:
> > Here's an example of an Expression handler to get you started. It
> > sounds like maybe you just need to tell the serializer what the data
> > types are during deserialization so I've left GetExpression (called
> > during serialization) not doing much. Evaluate is called during
> > deserialization and I've put in some code there to get you started.
> > To set the type you basically need to wrap an expression in a
> > CastExpression passing in the appropriate data type. Here is the wiki
> > page for the different expression types:http://code.google.com/p/jsonexserializer/wiki/Expressions,
> > sorry there's nothing yet on the handlers themselves.
> > public class PyTableExpressionHandler : ObjectExpressionHandler
> > {
> > public override object Evaluate(Expression expression, object
> > existingObject, IDeserializerHandler deserializer)
> > {
> > ObjectExpression objExpr = (ObjectExpression) expression;
> > ArrayExpression columns = (ArrayExpression) objExpr
> > ["columns"]; //List<PyColumn> columns
> > ArrayExpression data = (ArrayExpression) objExpr
> > ["data"]; //List<List<Object>> data
> > // Add type info to data via Cast
> > for (int i = 0; i < columns.Items.Count; i++) {
> > // assume columns and data have same length
> > Type dataType = null;
> > // TODO: figure out data type from column info:
> > ((ValueExpression)((ObjectExpression)columns[i])
> > ["typeId"]).StringValue
> > ArrayExpression dataRow = (ArrayExpression) data.Items
> > [i];
> > for (int j = 0; j < dataRow.Items.Count; j++) {
> > dataRow.Items[j] = new CastExpression(dataType,
> > dataRow.Items[i]);
> > }
> > }
> > return base.Evaluate(expression, existingObject,
> > deserializer);
> > }
> > }
> > Here is how to set it up in the serializer:
> > Serializer s = new Serializer(typeof(List<PyTable>));
> > s.Context.ExpressionHandlers.Insert(0, new
> > PyTableExpressionHandler());
> > If your object is really as simple as you've shown you could
> > potentially evaluate some of it on your own and just put it in the
> > final expression. You can evaluate any expression by passing it to
> > the IDeserializerHandler instance that gets passed into the Evaluate
> > method. You just need to set the ResultType property on the
> > expression first. If you do go this route, just make sure you remove
> > any child expressions from the main expression that you've already
> > evaluated before passing them to the base method, and then update the
> > object that gets passed back to you.
> > On Sep 10, 11:11 am, John <j...@jdiligence.com> wrote:
> > > I need to override some elements and change the way they are
> > > serialized depending on other fields within the object. Below the type
> > > of object in the list of lists (data) depends on the column object
> > > within the same object. How can I do this?
> > > I could use just the low level expression stuff which I can then
> > > traverse to build my own structure but can't find a tutorial on using
> > > this without extras of the serializer.
> > > Any tips?
> > > public class PyTable
> > > {
> > > String id;
> > > String name;
> > > List<PyColumn> columns = new List<PyColumn>();
> > > List<List<Object>> data = new List<List<Object>>();
> > > }
> > > public class PyColumn
> > > {
> > > const int TEXT = 0;
> > > const int INTEGER = 1;
> > > const int FLOAT = 2;
> > > const int DATE = 3;
Thank you for your informative responses I will try them out and see
what happens.
The only concern I have is that when deserializing I cannot guarantee
the order of the fields, so the data fields may appear before the
column definitions that they rely on in the stream.
Would this be a show stopper before I get stuck in later today?
On Tue, Sep 15, 2009 at 9:36 AM, John <j...@jdiligence.com> wrote:
> Thank you for your informative responses I will try them out and see
> what happens.
> The only concern I have is that when deserializing I cannot guarantee
> the order of the fields, so the data fields may appear before the
> column definitions that they rely on in the stream.
> Would this be a show stopper before I get stuck in later today?
Thanks a lot. I got the desrialization working in the end. Will work
on the other side of the coin today.
In doing this I noticed it would be really useful to decouple the
property resolution as between different languages there are different
naming conventions for properties.
I ended up renaming all my c sharp properties in ways that are not c
sharp convention, in order to match the services supplying the json.
If there was a pluggable property name resolver, this wouldn't be a
problem. So for instance it could captialize the first letter if
required etc
Yes and No. It's possible by overriding a couple of classes (TypeData,
PropertyData), but nothing that's really pluggable. It's a good idea
though. Could you give some examples of what you have in mind? I should
probably add an option to match case-insensitive too when looking up a
property.
On Wed, Sep 16, 2009 at 7:11 AM, John <j...@jdiligence.com> wrote:
> Thanks a lot. I got the desrialization working in the end. Will work
> on the other side of the coin today.
> In doing this I noticed it would be really useful to decouple the
> property resolution as between different languages there are different
> naming conventions for properties.
> I ended up renaming all my c sharp properties in ways that are not c
> sharp convention, in order to match the services supplying the json.
> If there was a pluggable property name resolver, this wouldn't be a
> problem. So for instance it could captialize the first letter if
> required etc
JSON is used a lot to interchange between systems in different
languages.
XML is too bloat if the messages are big. In our system they can be
huge.
This is also a good argument for making JSON parsers work on streams
rather
than loading strings into memory and building intermediary objects.
When
you have over 100,000 records garbage collection can start to be an
issue.
Its a bitlate to take this approach with your parser but the property
naming thing
is pretty easy to change.
The project I am working on is written in VBA, C sharp, java and
python!
Programmers in each have different conventions for naming properties
and typcially follow them.
Typically in java is camel case i.e. customerName
Typically in python its underscored lower case i.e. customer_name
Typically in C sharp its CustomerName
By being able to plug your own strategy you can translate say
customer_name into CustomerName
So if you could add an interface IPropertyNameResolver or similar then
people could override the
default strategy when talking via JSON to another system written by
developers using their own conventions.
This is more flexible than simple case insensitivity but you could add
a CaseInsensitiveProperyResolverImpl
for people who want it.
Ok, I've implemented a property naming strategy. There are a few out
of the box strategies (PascalCase, CamelCase, Underscore) as well as
the ability to create your own. I also added the ability to modify a
property's "Alias" directly. The Alias is a new concept introduced
with this change and is what is used in the output. It defaults to
being the same as the property name, but you can change it either by
using a different naming strategy or setting it directly. I've
released a 3.1 beta build with these changes. FYI: there are a few
breaking changes between 3.0 and 3.1 within the
JsonExSerializer.Framework namespace which is where the
ExpressionHandlers are. But it's mainly just renaming some classes or
switching to an interface in the ExpressionHandler method signatures.
You can find the release on the downloads page: 3.1.0.258.beta1. I
would include a link, but project hosting is down at the moment.
As far as working on streams, that was my original implementation in
the first release. I would construct the object as I was parsing the
JSON. There were a couple of issues with that. 1) Serialization and
deserialization are very different beasts making customization much
harder. You probably wouldn't have been able to do the scenario you
posed in your original question. 2) A customization at a given level
would leave you with the burden of doing your own serialization of all
your child objects. On Deserialization you would probably end up
needing to control the parser and making sure you get that right.
With the approach that I have now, logic in the expression handlers is
pretty symmetrical between serialization and deserialization. Plus
you have the benefits of extending the base logic and altering the
object that gets returned to you. The performance on the new approach
is a little slower, but my tests show it to be a pretty neglible loss
(average 5-6 ms per serialization/deserialization). I could
potentially expose a SAX-like parser, but that would complicate my
parser logic. Right now it's more of a recursive approach, with a SAX-
style interface I would need to use stacks and different handlers to
keep track of state.
On Sep 16, 11:03 am, John <j...@jdiligence.com> wrote:
> JSON is used a lot to interchange between systems in different
> languages.
> XML is too bloat if the messages are big. In our system they can be
> huge.
> This is also a good argument for making JSON parsers work on streams
> rather
> than loading strings into memory and building intermediary objects.
> When
> you have over 100,000 records garbage collection can start to be an
> issue.
> Its a bitlate to take this approach with your parser but the property
> naming thing
> is pretty easy to change.
> The project I am working on is written in VBA, C sharp, java and
> python!
> Programmers in each have different conventions for naming properties
> and typcially follow them.
> Typically in java is camel case i.e. customerName
> Typically in python its underscored lower case i.e. customer_name
> Typically in C sharp its CustomerName
> By being able to plug your own strategy you can translate say
> customer_name into CustomerName
> So if you could add an interface IPropertyNameResolver or similar then
> people could override the
> default strategy when talking via JSON to another system written by
> developers using their own conventions.
> This is more flexible than simple case insensitivity but you could add
> a CaseInsensitiveProperyResolverImpl
> for people who want it.