querying for date ranges BC

[Ian Millard]

Hi,

I'm trying to FILTER on dates, and getting some rather unexpected
behaviour, particularly with negative (BC) values. Sample data below.

SELECT ?person WHERE { ?person eg:born ?birth . FILTER( ?birth >
"1000-01-01T00:00:00"^^xsd:dateTime ) }
should return: jfk, henry8, granny
I get: jesus, sparticus, alexander

SELECT * WHERE { ?person eg:born ?birth . FILTER( ?birth <
"1000-01-01T00:00:00"^^xsd:dateTime ) }
should return: jesus, sparticus, alexander
I get: granny, jfk

this is kind of the inverse of what you would expect, however note
neither query returned henry8. If you do <= 1000 then henry8 does make
an appearance?!

Entering a negative date, eg "-0100-01-01T00:00:00"^^xsd:dateTime
gives
<!-- warning: cannot convert value to xsd:datetime -->

So I think something wacky is going on with the evaluation or
processing of the dateTime values?

Finally, it would be neat if 4store would support filters on xsd:date
as well as dateTime, so that unknown values (month/day) could be
omitted

Any thoughts, comments?

Thanks,


Ian



--- test data ---

id:sparticus
rdfs:label "109 -- 71 BC" ;
eg:born "-0109-12-12T00:00:00"^^xsd:dateTime ;
eg:died "-0071-01-01T00:00:00"^^xsd:dateTime .

id:jesus
rdfs:label "5 BC -- 30 AD" ;
eg:born "-0005-12-25T00:00:00"^^xsd:dateTime ;
eg:died "0030-01-01T00:00:00"^^xsd:dateTime .

id:alexander
rdfs:label "20 Jul 356 -- 10 Jun 323 BC" ;
eg:born "-0356-03-24T00:00:00"^^xsd:dateTime ;
eg:died "-0323-06-10T00:00:00"^^xsd:dateTime .

id:henry8
rdfs:label "henry 8th, 1491 - 1547" ;
eg:born "1491-06-14T00:00:00"^^xsd:dateTime ;
eg:died "1547-01-28T00:00:00"^^xsd:dateTime .

id:jfk
rdfs:label "May 29, 1917 -- Nov 22, 1963" ;
eg:born "1917-05-29T00:00:00"^^xsd:dateTime ;
eg:died "1963-11-22T00:00:00"^^xsd:dateTime .

id:granny
rdfs:label "31 Jan 1906 -- 28 Aug 1999" ;
eg:born "1906-01-31T00:00:00"^^xsd:dateTime ;
eg:died "1999-08-28T00:00:00"^^xsd:dateTime .

[Steve Harris]

Hi Ian,

Hm, interesting.

I suspect that the problems are due to the missing timezone, try just adding a Z to the end.

4store should degrade more gracefully though, I'll have a look at it. From what I remember it falls back to a lexical comparison if there's no timezone, which will work for all CE data, but not if there's any BCE dates involved.

Also, I don't think 4store was ever tested with BCE dates, so they might just not work, should be easy to fix though.

- Steve

> --
> You received this message because you are subscribed to the Google Groups "4store-support" group.
> To post to this group, send email to 4store-...@googlegroups.com.
> To unsubscribe from this group, send email to 4store-suppor...@googlegroups.com.
> For more options, visit this group at http://groups.google.com/group/4store-support?hl=en.
>

[Ian Millard]

Hi,

unfortunately adding a Z timezone doesn't seem to help (neither does
+00:00)

BCE dates just don't seem to work

you "should be easy to fix" ... I've not looked at any C in anger for
many years, but from a naive bit of grepping I see that frontend/
filter.c has the function fn_greater_than()

spitting out a bit of debug, I've observed the following:

in parsing the xsd:dateTime from the query to a (64bit?) integer,
values are +ve after 1970-01-01T00:00:00+00:00 and -ve before then.
winding the clock back, you get more negative until around December
1901, where the value becomes -1 ... an overflow?!

in addition, BCE dates such as -0005-12-25T00:00:00+00:00 in a FILTER
query give warning "cannot convert value to xsd:datetime", however
those asserted have a lexical value of whatever they were and 0 int
value.

think I'm a bit stuck now, other than to say I fear filter-test.c
could do with some more test cases :)

btw -- if it helps, my requirements are actually only to represent
yyyymmdd CE, and -yyyy BCE

cheers,

I.

[Steve Harris]

When I said, easy to fix, I meant easy for me to fix :) but good investigation.

I guess the UNIXy time functions stop at 1901, I'l see if there's something in glib or so that can be used to compute the timeline value.

I'll look at it probably tomorrow morning.

Cheers,
Steve

[Ian Millard]

Thanks Steve.

for the moment I've tried "YYYYMMDD"^^xsd:integer, prepending with -ve as
appropriate, which I think will achieve what I want (but is a bit hacky!)

Ian

[Steve Harris]

Hi Ian,

Was harder to fix than I though, but HEAD (ecde79d) has a fix which should work.

Turns out almost no C library handles ISO 8601 dates before 1900.

- Steve