Contributing to Elda project.

[Piret L]

Hello.

We are using Elda framework for linked-data-api. Most part it satisfies all our needs. 

Would like to make a small change in the code, that is used for RDF to JSON rendering. For that I would like to get more information regarding how can I contribute to Elda and what is the correct process for that?

The part we would like to improve is the date and dateTime encoding to follow ISO 8601 standard for date and dateTime format and use 'UTC' as timeZone. 

The need for this change for us comes from a problem with using SimpleDateFormat with default locale, as the month names differ in machines with different locales. And we would like the response dates to be independent from the machine configuration.

Sincerely.

Piret L

[chris dollin]

Hi Pirent (or should I say Hi Lattikas?)

> We are using Elda framework for linked-data-api. Most part it satisfies all
> our needs.

We're very pleased to hear that.

> Would like to make a small change in the code, that is used for
> RDF to JSON rendering. For that I would like to get more information
> regarding how can I contribute to Elda and what is the correct process
> for that?

You've already begun -- you've start a conversation here. We explore
what you want to change and how and how it will be tested and then
when there's working code we decide if and how its going to go into
Elda. (We may need some licencing arrangement depending on how much
change there is.)

> The part we would like to improve is the date and dateTime encoding to
> follow ISO 8601 standard for date and dateTime format and use 'UTC' as
> timeZone.

Ah. So the first thing to note is that the serialisation of Date[Time]
objects in the JSON renderer is fixed by the LDA spec (see
Serialising Literals on page

https://github.com/UKGovLD/linked-data-api/blob/wiki/API_Formatting_Graphs.md
)

We can't just change it because existing code that GET's a
.json rendering may well rely on the current serialisation,
eg it could be a Javascript program that uses Date.parse
to deserialise the date.

We usually deal with problems like this by extending the
Elda config with new properties or (more easily) by giving
special meaning to some reserved variables.

> The need for this change for us comes from a problem with using
> SimpleDateFormat with default locale, as the month names differ in machines
> with different locales.

I see. That suggests we'll either need to pick a standard locale
for Elda or have a way of configuring a locale in the LDA config.
Perhaps a variable that names the required locale and maybe
a format string for SimpleFormat rather than the current one.

> And we would like the response dates to be
> independent from the machine configuration.

Yes, that sounds like a good idea to me.

Any comments or suggestions from anyone? (I do tend to go
over the top with features like this, it may not need to be
as complex as I'm suggesting.)

Chris

--
Chris "allusive" Dollin

[Piret L]

Hi, Chris.

Thank you, for your thorough answer.

Hi Pirent (or should I say Hi Lattikas?)

Piret is fine.

 

I see. That suggests we'll either need to pick a standard locale
for Elda or have a way of configuring a locale in the LDA config.
Perhaps a variable that names the required locale and maybe
a format string for SimpleFormat rather than the current one.

I like the idea of having a variable where you could specify the locale and format string for SimpleDateFormat.

The default format could be the one used right now, to match LDA spec you referenced to. 

Perhaps also add a possibility to specify the timeZone in variables for dateTime formatter.

This probably only affects JSON rendering. Do you have suggestions, where these variables could be positioned? 

Or what should be the next step?

 

Sincerely.

Piret L

[Stuart Williams]

Hello Chris, Piret,

I'd be inclined to keep things as simple as possible. It is already possible to set the locale and timezones for the JVM.

        http://stackoverflow.com/questions/8809098/how-do-i-set-the-default-locale-for-my-jvm
        http://www.oracle.com/technetwork/articles/javase/locale-140624.html#using

I think elda should use and respect those mechanisms rather than recreated them.

I do think it worth providing a mechanism to have dates and datetimes expressed in ISO8601 (xsd:date and/or xsd:dateTime lexical forms) in a JSON response instead of the format required by older javascript Date.parse() implementations (I think more modern JS Date.parse() implementations will parse ISO8601 formatted strings).

One possible way to control this from an elda configuration would be through the use of api:structured on a property definition, where the property takes a date or dateTime value. If api:structured is absent or 'false' format as now. If api:structured is present and 'true' format as in ISO8601. System locale/timezone can also affect the 8601 presentation of a date/dateTime.

Stuart
--

--
You received this message because you are subscribed to the Google Groups "linked-data-api-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to linked-data-api-d...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

-- 
Epimorphics Ltd                        www.epimorphics.com 
Court Lodge, 105 High Street, Portishead, Bristol BS20 6PT
Tel: 01275 399069

Epimorphics Ltd. is a limited company registered in England (number 7016688)
Registered address: Court Lodge, 105 High Street, Portishead, Bristol BS20 6PT, UK

[Piret L]

Hello Stuart, Chris.

From elda source code I see that api:structured on a property definition already has it's own meaning and use. I would not like to change the use of that.

Also I would like to use some global variable on formatter definition instead of describing the same variable for every property that takes date or dateTime value. For example name it "api:useISO". In case api:useISO is absent or 'false' format as it is now. In case api:useISO is present and 'true' on api:JsonFormatter, then use ISO8601 for date and dateTime formatting in a JSON response.

What do you think about that? Does anybody have any more ideas or suggestions?

Sincerely.

Piret

[Stuart Williams]

Hello Piret,

On 06/07/2015 11:18, Piret L wrote:
> From elda source code I see that api:structured on a property definition
> already has it's own meaning and use. I would not like to change the use of that.

What would you expect the JSON output (for it is only JSON ouput that is at
issue) to look like now for a date or dateTime value if api:structured is
present and set to true for a property? And is that currently a useful mode in
which to operate?

FWIW my main point as more about *not* trying to re-engineer the whole
locale/timezone selection mechanisms within the elda config that already exist
through JVM property settings.

BR

Stuart

[Piret L]

Hello Stuart.

To maybe explain better, what I would like to achieve, I would bring an example.

For example for one endpoint we have a property named hasDate with a value of xsd:date. With the current elda version and without api:structured, the JSON response looks like this:

hasDate: "Fri, 01 May 2015"

When I change configuration file to include api:structured on property hasDate, the result with current elda version looks like this:

hasDate: {

  _value: "2015-05-01Z",

  _datatype: "date"

}

I would like to implement a variable or some other solution that would allow me to specify that the xsd:date value would be returned in a format "yyyy-MM-dd" and xsd:dateTime in format "yyyy-MM-dd'T'HH:mm:ss'Z'". So in my example the returned value for hasDate would be:

hasDate: "2015-05-01"

Hope this makes it more clear, what I would like to achieve. The api:structured actually already let's us return the value as it is returned from the database in JSON and that would be ok, but we wouldn't like to add extra fields of "_value" and "_datatype" to keep the response as short and clear as possible.

Sincerely.

Piret

[Stuart Williams]

Hello Piret,

On 06/07/2015 11:59, Piret L wrote:

Hello Stuart.

To maybe explain better, what I would like to achieve, I would bring an example.

For example for one endpoint we have a property named hasDate with a value of xsd:date. With the current elda version and without api:structured, the JSON response looks like this:

hasDate: "Fri, 01 May 2015"

When I change configuration file to include api:structured on property hasDate, the result with current elda version looks like this:

hasDate: {

  _value: "2015-05-01Z",

  _datatype: "date"

}

I would like to implement a variable or some other solution that would allow me to specify that the xsd:date value would be returned in a format "yyyy-MM-dd" and xsd:dateTime in format "yyyy-MM-dd'T'HH:mm:ss'Z'". So in my example the returned value for hasDate would be:

hasDate: "2015-05-01"

Ok... elda already does more in that respect than I'd expected. I'm curious about your locale and time-offset setup. Is the date in the data truely expressed as with a time offet of  'Z', and if so is that also how you want it expressed. Or would you expect the date to be expressed with your local offset (whatever that may be). Or is it in the data as an un-zoned local date in which case I'd not expect the trailing 'Z' offset.

Also, are you getting the correct offsets that you want with dateTimes (given appropriate setup of the JVM properties).


Hope this makes it more clear, what I would like to achieve. The api:structured actually already let's us return the value as it is returned from the database in JSON and that would be ok, but we wouldn't like to add extra fields of "_value" and "_datatype" to keep the response as short and clear as possible.

I think that I'd argue that the explicit data type gives clarity at the expense of compactness.

Sincerely.

Piret

--

Best,

Stuart

[Piret L]

Hello Stuart.

I'm curious about your locale and time-offset setup. Is the date in the data truely expressed as with a time offet of  'Z', and if so is that also how you want it expressed. Or would you expect the date to be expressed with your local offset (whatever that may be). Or is it in the data as an un-zoned local date in which case I'd not expect the trailing 'Z' offset.

Also, are you getting the correct offsets that you want with dateTimes (given appropriate setup of the JVM properties).

 The locale of my machine is "et_EE", in the example I posted previously I used the JVM property user.language=en, as proposed by you previously, to get month name in english. Date is inserted to virtuoso actually without the timezone, so as a un-zoned date.

With dateTime I get the following result without api:structured:

created: "Tue, 23 Apr 2013 08:29:29 GMT+0000"

And with api:structured set to true on property created:

created: {

  _value: "2013-04-23T08:29:29Z",

  _datatype: "dateTime"

}

In case of dateTime _value reflects exactly what we wish to achieve.

Sincerely.

Piret

[Stuart Williams]

Hello Piret,

On 06/07/2015 13:11, Piret L wrote:

Hello Stuart.

I'm curious about your locale and time-offset setup. Is the date in the data truely expressed as with a time offet of  'Z', and if so is that also how you want it expressed. Or would you expect the date to be expressed with your local offset (whatever that may be). Or is it in the data as an un-zoned local date in which case I'd not expect the trailing 'Z' offset.

Also, are you getting the correct offsets that you want with dateTimes (given appropriate setup of the JVM properties).

 The locale of my machine is "et_EE", in the example I posted previously I used the JVM property user.language=en, as proposed by you previously, to get month name in english. Date is inserted to virtuoso actually without the timezone, so as a un-zoned date.

Hmmm.... in that case, if the intention is the dates and dateTimes are unzoned, and they are unzoned in the RDF data, they should be coming through without a time zone offset showing in their serialisation. That's a possible bug tha we may have to look at (and how elda internals interacts with the JVM settings).

With dateTime I get the following result without api:structured:

created: "Tue, 23 Apr 2013 08:29:29 GMT+0000"

And with api:structured set to true on property created:

created: {

  _value: "2013-04-23T08:29:29Z",

  _datatype: "dateTime"

}

In case of dateTime _value reflects exactly what we wish to achieve.

If there is no-zone offset in the data, I'd expect the output to be unzoned.

Sincerely.

Piret

I'll stand aside on the question of how to configure an unstructured 8601 format date/dateTime within an Elda configuration.

BTW: I use a SPARQL query to generate short-name configurations [1] and that could easily be modified to decorate data/dateTime valued properties with extra configuration values.

Stuart
[1] https://github.com/epimorphics/elda-starter/blob/master/sparql/bootstrap-shortnames.rq

[Piret L]

Hello Stuart.

 

If there is no-zone offset in the data, I'd expect the output to be unzoned.

The xsd:dateTime values are inserted with timeZone set to UTC, but xsd:date values are inserted without timeZone.

 

I'll stand aside on the question of how to configure an unstructured 8601 format date/dateTime within an Elda configuration.

I would propose as a simple solution to add then a variable (for example named api:useISO) on a property configuration, what takes xsd:date or xsd:dateTime value. This would require adding a new variable to ContextPropertyInfo. In case the variable is missing or 'false' format as it is currently. When the variable is present and set to 'true', then use a separate method in com.epimorphics.jsonrdf.RDFUtil to get DateFormat for date/dateTime with full format string as "yyyy-MM-dd'T'HH:mm:ss'Z'" and use timeZone UTC.

Sincerely.

Piret

[Piret L]

Hello.

I have implemented an example in my forked version of elda and committed to different branch that is reachable from https://github.com/piretjanson/elda/tree/json-datetime-formatting.

Could you give me some feedback if this could be acceptable solution? Would appreciate any feedback.

Sincerely.

Piret

[chris dollin]

I'll look later on today. Thanks.

[chris dollin]

Some brief comments.

Tests, yay! But (assuming we push on) I'd like to see
some integration tests that set up a JSON renderer,
feed it a config/model, and make sure the expected JSON
comes out.

I think the property name useISO isn't sufficiently specific.
Something like jsonUsesISOdate would be better.

We've put new config properties in the ELDA_API class rather
than the API class to avoid misleading developers into thinking
that these proeprties would be available in other LDA implementations.

ELDA_API is auto-generated using schemagen from vocabs/elda_api.ttl,
so the path is to edit elda_api.ttl and then `mvn compile` will force
the change through to ELDA_API.

You've chosen to attach the configuration property to individual
LDA properties rather than the renderer itself. I think we should
think about this a bit more.

Are these changes that much of an improvement over using
the existing api:structured and having the consumer of the JSON
just fetch the _value member from the date/time object?

I'm a bit worried about forcing the English locale into Elda
as opposed to using the default locale and setting it for
Tomcat. (Although I haven't been able yet to find a way
of setting the default locate for /a specific Tomcat webapp/.)

I'd appreciate Stuart's comments.

Chris

--
Chris "allusive" Dollin

[Piret L]

Hi all.

I have now made some improvements and changes based on the feedback. The code examples are still available in the same branch.

I renamed the property from api:useISO to elda:jsonUsesISOdate and added it under ELDA_API class as proposed by Chris.

Also moved the property from individual LDA properties to formatter itself to enable this encoding for all properties. In addition added tests that demonstrate using the property with JSONFormatter and check that the results returned are in expected format.

Regarding the English locale I removed that from default formatting. So without the elda:jsonUsesISOdate set on JSONFormatter the format of the dates will be the same as it is right now.

Could you please give additional feedback if the implementation is moving in the right direction? I am open to suggestions.

Sincerely.

Piret

[chris dollin]

Hi Piret

Sounds good; I will have a proper look tomorrow.

Chris

--
Chris "allusive" Dollin