OAL 2.0 status check

[Phyllis]

Has anyone finished with OAL 2.0 support yet? I'd like to see whether
a file created by Deep-Sky Planner can be imported by another
application. The files appear correct and pass several XML validators
at this point.

Thanks!
- Phyllis

[Tom]

Hi Phyllis,

Wim and I have fixed a lot of issues. Meanwhile it's possible to
import my log to deepskylog.org/test and everything is fine. Export
works and produces valid files. I can import the DSL export file in
E&T back again, yielding no duplicate observations. So from my
standpoint, we are done. But I will continue testing, of course.

I have updated the "reference samples" file in the files section.

Best wishes,
Tom

[Dirk Lehmann]

Hi,

I'm done with the <OAL> 2.0 support in Observation Manager 0.820.
A first release candidate can be downloaded at the projects website
at:
http://observation.sourceforge.net
I'd happy to receive feedback.

Also the website was been reworked. The COMAST information now simply
refers to this group.

Regards

Dirk

[Phyllis]

All,

@Tom: The file you sent is processing apparently as expected. I am
delighted that things are working.

@Dirk: I'll work with OM today and tomorrow. Thanks for posting an
update.

I would like to mention a few things to get your thoughts before I get
too happy and consider OAL 2.0 finished.

1) DSP is aware of timezones and the Daylight saving time rules
defined for each. For this reason, any observation defined in an OAL
document with no session or no site data is omitted from import. In
the case of the file Tom sent, my processor reported 454 observations
and actually imported 360. I hope those numbers sound correct.

2) Dates/times as usual present a challenge. DSP allows date/times to
be local or universal (but not mixed in a single observing session.)
Date/times in an OAL document are converted to UTC (if timezone
information appears in the element) on import into DSP. Once again,
DSP insists on timezone information, so converting date/times to UTC
and referring to the Greenwich Standard Time timezone makes import
possible. DSP does not change date/time from local to UTC when
exporting to OAL. I don't like the asymmetry in this, and I may change
it. At least a date/time exported by DSP includes the timezone
information in the element.

It seems that we should state in the documentation something like "it
is preferred that date/times be recorded in UT in OAL documents to
avoid confusion and/or loss of data."

3) The rating element defined in the deepsky extension has concerned
me for a while. While I feel it is absolutely reasonable to include a
rating of some type in an observation, I feel that <oal:rating> is too
narrowly defined - representing only the DSL system. Any number of
rating systems might be used by observers (or institutions.) To that
end, <oal:rating> in the deep-sky extension could be an abstract
element or just an integer that is further defined by another
extension. Perhaps the most expedient solution would be to make it a
non-negative integer with an optional attribute indicating what rating
system that the value represents. For example <rating system="DSL">5</
rating> or <rating>5</rating>.

DSP imports the rating element in an OAL document but it doesn't
export it back. This is because DSP has an ambiguous rating system
(1-10) and OAL has the more specifically defined one. A '2' to DSP
does not necessarily mean the same thing as a '2' to DSL.

Should we hold up v2.0 release to work this out? I would like to
resolve it, but I definitely wouldn't *require* a change. What do you
think?

The more I work with the schema, the more impressed I am with the
amount of work that has gone into it, and how comprehensive it is. To
those of you who have authored the schema - you've done quite a nice
job.

- Phyllis

[Tom]

Hi Phyllis,

Dates, times and timezones:

Whether you have a site (or a session referring to a site) or not, the
canonical representation of date/time in XML Schema contains a
timezone part. So you can transport the timezone information without
reference to a concrete site.

I once decided to not take care of daylight saving times rules in E&T.
In Germany, there is a law on time related subjects (the "Zeitgesetz")
like the daylight savings time. Albeit there are some "daylight
savings time rules" out there, basically beginning and end of the DST
interval is not given by a rule but (at least in Germany) by a
decision of the legislative.

After World War II, the offset for daylight savings time in Germany
was not one, but two hours. And there even are two years where we have
no historical information whether DST was used and if, with what
offset.

You might argue that it is highly unlikely that someone who has own
observations made in 1946 or 1947 wants to record this in OAL (must be
a very seasoned observer!). But I dislike taking care of DST because
it's a political concept and not an astronomical or observational one.
DST rules can and probably will change and I simply do not want to be
forced to make changes if a new president of country XY decides to
change the way DST is defined there.

As DST rules are not constant over time, a "serious" implementation
would require historical tracking. To me, this simply seemed to much.

So due to the fact that we can convey a timezone (disregarding DST) in
each and every observation without direct or indirect reference to a
site, there is no obstacle by OAL against importing siteless or
sessionless observations. Whether you do or do not import siteless/
sessionless observations is up to you as an application vendor. It's
not the responsibility of OAL.

My personal opinion is that an application should be able to import
"siteless" observations, using the timezone given in the <begin> or
<end> elements of an observation. Why? Because the information exists
- you can make use of it.


Now some thoughts on "ratings".

We use ratings scales, because we are dealing with something that is
not to be measured and carries no physical unit. Apart from the
"rating", there also is a seeing scale in OAL. But besides the 1 to 5
scale OAL uses, there are alternative, finer stepped scales, too. And
by the nature of the matter, it would be easy (and perhaps even
useful) to invent new scales. Think of the Bortle scale for rating sky
quality. I find it too coarse, but with the advent of the SQM, I feel
no need to introduce a finer scale with half steps or sth. like that.

Of course, different scales for a property could be used and the
schema could be augmented with an attribute telling you or the code
what scale is used. But I always wanted COMAST and OAL to be a
semantic schema, not only a descriptive one. This means that code
should be able to make clever use of the data, for example
investigating how the "contrast above threshold" calculated from the
target's data, optics and sky quality matches/correlates with the
rating the observers found. Or to find better extinction models, etc.
You see the point: if it's "only descriptive" and not tied to
something rather strict, it will be hard or impossible to yield new
insights from the data.

If we introduced a "what scale is used" attribute, we had to update
our schema for each new scale proposed by somebody. And if that wasn't
bad enough, we had to figure out conversions between the different
scales - or run in danger to lose the expressional power of the
schema.

It's right that the rating scale came from the national "Deep Sky
Liste" collection, but by the time I started out designing the schema,
there was nothing that made more sense to me. Later I heard from Steve
Coe that he has almost an equal number of observations as the whole
DSL collection and of course he did use another scale. That's a pity,
but neither Steve nor me can change it now.

So it comes down to conversion of existing ratings to the OAL
definition on import.

If you wanted to use OAL to exchange observations between users of
Deep Sky Planner, you could use processing instructions to convey your
original values without any impact on the schema.

E&T's import logic uses some undocumented processing instructions that
were required to build an update of the app's object database. There
is a PI, for example, telling the code to delete a target from the
database. Another PI contains the serial number of the user. This
allows the code to detect that a user wants to re-import his/her own
observations. I want to be able to detect this situation, because re-
importing is not the recommended procedure to restore your data
because some information not covered by OAL might get lost.

So from my point of view we are done with OAL2.0 ;-)

Best wishes,
Tom

[Phyllis]

Tom et al,

> My personal opinion is that an application should be able to import
> "siteless" observations, using the timezone given in the <begin> or
> <end> elements of an observation. Why? Because the information exists
> - you can make use of it.

There are other reasons that an application may need location and date/
time/timezone information. Importing a 'siteless' observations must
remain the application developer's choice. Fortunately, the schema
permits this.

> So it comes down to conversion of existing ratings to the OAL
> definition on import.

Actually, exporting a rating value that has meaning exclusive of DSL
is the problem that concerns me. Since the context of rating in an OAL
document only carries context for DSL, no other type of rating can be
inserted without implying the DSL context. I've elected not to do
that.

I don't recommend holding up release for either of these issues,
although solving the rating issue seems more important. Would some of
the other developers care to comment?

- Phyllis

[Tom]

Hi Phyllis,

> Actually, exporting a rating value that has meaning exclusive of DSL
> is the problem that concerns me. Since the context of rating in an OAL
> document only carries context for DSL, no other type of rating can be
> inserted without implying the DSL context. I've elected not to do
> that.
>

just an idea: the rating given in OAL might be converted to something
in another scale by using a XSL transformation.

- Tom

[Dirk Lehmann]

Hi *,

sry for the late reply.

On the date/timezone/DST topic:
I can basically only echo what Tom said. I was quiet surprised to read
that
DSP is aware of timezones and the Daylight saving time rules, as to
the fact (which Tom already mentioned)
at least in Germany, DST is defined by the secretary of the interior.
So a software reflecting DST needs to be
adoptable in this point (for future and past observations)
In OM I propose to the enduser to create two sites objects for one
physical site. One with a timezone including DST and one without.
Or simply enter only data without DST reflected.

For the rating: tough stuff.
I understand the narrowing factor which implementing only the DSL
scale has for <OAL>
On the other side, there is no lingua franca for deep sky ratings. So
making the rating abstract, would open the door to make <OAL>
implementing applications
incompatible with each other. Application A uses rating X, application
B uses rating Y, ...
So this could only be handled by given/proposing mappings between each
rating scale, which is
a) not doable (as everyone can create his/her own rating)
b) not in the scope of the <OAL> project IMHO

So my suggestion would be:
- Add a comment to the rating element and tell that <OAL> applications
could make a mapping to a custom rating scale is appropriate during
runtime.
- If they do not want to use the <OAL> rating (basically the DSL
rating), they should inform the enduser by bringing an information
(pop-up window, ...) during <OAL> import/export making
the enduser aware of this fact. Or in "best case": let the enduser do
the mapping, by bringing him/her a tiny mapping tool before import/
export and ask him to do the mapping (e.g. after given him a proposed
default mapping)

I know this is all not satisfying, but I
a) truly hold a rating scale for something very valuable, which should
be reflected in <OAL>
b) as long as there is no lingua franca for deep sky ratings, we
either need to skip the rating completly (see argument a)) or simply
pick one and use it (as we did with DSL scale)

My two cents...

Best

Dirk

[Wim De Meester]

On Tuesday 22 September 2009 11:10:32 Dirk Lehmann wrote:
> Hi *,
>
> sry for the late reply.
>
> On the date/timezone/DST topic:
> I can basically only echo what Tom said. I was quiet surprised to read
> that
> DSP is aware of timezones and the Daylight saving time rules, as to
> the fact (which Tom already mentioned)
> at least in Germany, DST is defined by the secretary of the interior.
> So a software reflecting DST needs to be
> adoptable in this point (for future and past observations)
> In OM I propose to the enduser to create two sites objects for one
> physical site. One with a timezone including DST and one without.
> Or simply enter only data without DST reflected.

Hi all,

For DeepskyLog, we also use the timezone. The nice thing is that (at least in
Linux, but probably also in Windows), the information about the timezones is
kept in a system library. I think this includes all changes that happened in
the past. When there is a change in a timezone somewhere in the world, this
system library is updated in an official security update. This way, the system
is always up to date. I think a similar thing is also possible in Windows
(because when the change from winter to summer time happens, windows
automatically sets your clock).

Cheers,

Wim
--
Wim De Meester
Katholieke Universiteit Leuven
Instituut voor sterrenkunde tel: +32 (0)16 327914
Celestijnenlaan 200 D fax: +32 (0)16 327999
B - 3001 Leuven email: wim.de...@ster.kuleuven.be
Belgium http://www.ster.kuleuven.be/~wim

[Tom]

Hi all,

Thanks to Dirk for commenting on the ratings topic. You expressed it
better than I did and I wholeheartedly agree with your points.

Timezones: of course, there is system or OS support for timezones. But
please separate the "world of the application(s)" from the world of
the schema. While an application might query a library or the
operating system for the timezone and/or DST rule applicable, this
affects the representation of observations in the GUI at first hand.
But this is totally decoupled from the way a date/time is expressed in
the schema - and I'm happy for this separation! I think that the way
an application presents a date/time to the user is totally up to the
developer. It's just a matter of comfort whether you see a timezone/
DST indication in the windows or the reports your application shows.

In E&T all times in the list are in UT, because if you have imported
international observations from various timezones with or without DST
at work, you could not sort consistently otherwise. When displaying an
observation's detail, the timezone is used, but no DST rule is
applied. Users may find this less comfortable than possible and it is.
But this "lack of DST awareness" is not a problem for the schema, it's
just a missing feature of my software. I could implement the DST rules
somehow, but whether this is done or not, the schema does not need to
know about.

Clear skies,
Tom

[Phyllis]

All,

I think my statement about time zone information may have launched us
into an unintended discussion. I realize that the xsd:dateTime can
include the time zone specifier so that a precise instant is well
defined. Time zone data appears in my native document format for other
reasons. I have no problem with leaving xsd:dateTimes specified as
they are.

Regarding the rating, I am happy enough to go forward with the DSL
rating in a document. I'm not aware of another accepted rating system
(I'm researching), so I'm not sure how I will handle the rating item
on my end. I can say that I won't require my users to use the DSL
system of ratings because it's not in general use in my largest
market, the US.

I have found that DeepSkyLog does not include a session definition in
a document exported in OAL format. Since session is a required data
item for Deep-Sky Planner, documents produced by DeepSkyLog will not
be able to import into DSP. I think it is fair to say that both Wim
and I are aware of this and one or both of us will try to address it
in the future.

I have worked with ObservationManager 0.82 with success.

I believe we are done with v2.0.
- Phyllis

[Dirk Lehmann]

Hi,

good to hear that OM 0.820 works for you, Phyllis.
For me <OAL> 2.0 is also ready for release.

What about KStars? Anyone from that team listening? How's your
implementation going?

Regards

Dirk

[Wim De Meester]

I also tested OM 0.820 with the observations from DeepskyLog. Works very nice!
Do we aim for a release of different software with <OAL> 2.0 support at the
same time? Any idea when to release <OAL> 2.0 final?

Cheers,

Wim

[Phyllis]

Wim,

It seems like we are all saying it is ready. May I suggest official
release on Thursday (Oct 1) unless anyone objects?

- Phyllis

[Wim De Meester]

fine for me!

Wim

[Dirk Lehmann]

Good, idea!

Best

Dirk

BTW: There's a astro logging discussion currently going on here:
http://www.astronomyforum.net/astronomy-software-forum/
I'm trying to point these guys to <OAL> as well....


On 29 Sep., 12:25, Wim De Meester <wim.demees...@ster.kuleuven.be>
wrote:

> fine for me!
>
> Wim
>
> On Tuesday 29 September 2009 12:22:36 Phyllis wrote:
>
> > Wim,
>
> > It seems like we are all saying it is ready. May I suggest official
> > release on Thursday (Oct 1) unless anyone objects?
>
> > - Phyllis
>
> --
> Wim De Meester
> Katholieke Universiteit Leuven
> Instituut voor sterrenkunde         tel: +32 (0)16 327914
> Celestijnenlaan 200 D               fax: +32 (0)16 327999

> B - 3001 Leuven                     email: wim.demees...@ster.kuleuven.be
> Belgium                            http://www.ster.kuleuven.be/~wim

[Tom]

An official release on October, 1st is fine for me, too. But I will
take some further time to update documentation and conduct some field
testing with E&T before 3.1 will be released. The OAL 2.0 support is
OK, but there's much more...

Best wishes,
Tom