Status of the TOMA implementation

7 views
Skip to first unread message

Thomas Neidhart

unread,
Oct 15, 2009, 5:10:27 AM10/15/09
to ont...@googlegroups.com
Dear all,

I just wanted to update you on the status of the TOMA implementation:

- implementation checked into sandbox
- specification can be found under sandbox/toma/doc
- almost feature complete (square brackets are not yet fully supported)
- ~80 unit tests up to now (still growing), btw nice test suite @ lmg
- documentation almost complete
- maven project descriptor
- cmdline tool for querying topic maps using TOMA

Open things:

- finish square brackets
- finish documentation
- get full test coverage
- finish QueryOptimizer
- implement a QueryTracer

A question that still remains open, is how somebody can try out the TOMA
language within Ontopia. Actually I was thinking about integrating it
into the query plugin of ontopoly. Making this plugin a bit more
generic, and search for available QueryProcessorIF implementations in
the classpath. Then you can choose the QueryProcessor to be used with
something like a checkbox or drop-down list. The builtin query language
will be Tolog, if the toma.jar is in the classpath, it will be visible.

Tell me what you think about it, or if you have other ideas.

br,

Thomas
--
Thomas Neidhart, Software Engineer
Space Applications Services, www.spaceapplications.com
Leuvensesteenweg 325, B-1932 Zaventem, Belgium
Phone: +32-(0)2-721.54.84, Fax: +32-(0)2-721.54.44

Lars Marius Garshol

unread,
Oct 15, 2009, 9:40:09 AM10/15/09
to ont...@googlegroups.com

* Thomas Neidhart

>
> I just wanted to update you on the status of the TOMA implementation:
>
> - implementation checked into sandbox
> - specification can be found under sandbox/toma/doc
> - almost feature complete (square brackets are not yet fully
> supported)
> - ~80 unit tests up to now (still growing), btw nice test suite @ lmg
> - documentation almost complete
> - maven project descriptor
> - cmdline tool for querying topic maps using TOMA

This is excellent! Perhaps you should make a wiki page for the Toma
support where you give a little background, point to the spec, and
describe how to build it and use the command-line tool? That would
give people a convenient entry point.

> A question that still remains open, is how somebody can try out the
> TOMA
> language within Ontopia. Actually I was thinking about integrating it
> into the query plugin of ontopoly. Making this plugin a bit more
> generic, and search for available QueryProcessorIF implementations in
> the classpath. Then you can choose the QueryProcessor to be used with
> something like a checkbox or drop-down list. The builtin query
> language
> will be Tolog, if the toma.jar is in the classpath, it will be
> visible.

I think this is a good idea. The TMLab is working on an implementation
of the current TMQL draft, and they also want to hook into the current
plug-in without necessarily being distributed with Ontopia.

In addition, I'm finishing the implementation of the tolog update
language, which is also going to need some sort of user interface. At
the moment these statements are run by calling
QueryProcessorIF.update, as opposed to queries, which are run by
calling QueryProcessorIF.execute.

So I have a similar, but different, issue.

It looks like we'll have four different kinds of QueryProcessorIF
implementation (basic, rdbms, toma, tmql), plus the updates. I think
your suggestion works well for the first four. I'm not sure yet what
to do about the last. Ideas?

--Lars M.
http://www.garshol.priv.no/tmphoto/
http://www.garshol.priv.no/blog/

Thomas Neidhart

unread,
Oct 15, 2009, 11:56:36 AM10/15/09
to ont...@googlegroups.com, Rani Pinchuk
Lars Marius Garshol wrote:

> This is excellent! Perhaps you should make a wiki page for the Toma
> support where you give a little background, point to the spec, and
> describe how to build it and use the command-line tool? That would
> give people a convenient entry point.

Good idea, the information is available at:
http://code.google.com/p/ontopia/wiki/TomaImplementation

> I think this is a good idea. The TMLab is working on an implementation
> of the current TMQL draft, and they also want to hook into the current
> plug-in without necessarily being distributed with Ontopia.
>
> In addition, I'm finishing the implementation of the tolog update
> language, which is also going to need some sort of user interface. At
> the moment these statements are run by calling
> QueryProcessorIF.update, as opposed to queries, which are run by
> calling QueryProcessorIF.execute.
>
> So I have a similar, but different, issue.
>
> It looks like we'll have four different kinds of QueryProcessorIF
> implementation (basic, rdbms, toma, tmql), plus the updates. I think
> your suggestion works well for the first four. I'm not sure yet what
> to do about the last. Ideas?

The Toma implementation is actually also designed in a way, to allow
optimized implementations for different backends (currently only the
basic implementation for in-memory backends is implemented).

Regarding the updates. If I remember correctly, they are implemented by
the same QueryProcessor, so two buttons (Execute, Update) would be
sufficient, at least for now. It should also be quite easy to implement
the QueryProcessor in a way to automatically guess whether its an select
or update.

Lars Marius Garshol

unread,
Oct 16, 2009, 4:15:04 AM10/16/09
to ont...@googlegroups.com, Rani Pinchuk

* Thomas Neidhart

>
> Good idea, the information is available at:
> http://code.google.com/p/ontopia/wiki/TomaImplementation

Excellent, thank you! That gave me enough info to write up a simple
blog entry about this:
http://ontopia.wordpress.com/2009/10/15/toma-implementation-in-ontopia/

> The Toma implementation is actually also designed in a way, to allow
> optimized implementations for different backends (currently only the
> basic implementation for in-memory backends is implemented).

Good. As I understand it, this would be done using different
QueryProcessorIF implementations. Is that right? If so, your classpath
detection scheme would work just fine.

> Regarding the updates. If I remember correctly, they are implemented
> by
> the same QueryProcessor, so two buttons (Execute, Update) would be
> sufficient, at least for now.

I thought about a radio button myself, but two buttons would be much
more elegant. Maybe we should just go for that. Any other ideas?

> It should also be quite easy to implement the QueryProcessor in a
> way to automatically guess whether its an select or update.

Yes, it would. That would be the other approach. We'd need to add a
method like

QueryProcessorIF.isUpdate(String statement)

but it doesn't seem very ... clean.

Thoughts welcome.

Thomas Neidhart

unread,
Oct 16, 2009, 1:16:56 PM10/16/09
to ont...@googlegroups.com
Hi all,

I checked in a generic query plugin for omnigator, which can be used to
try out different query language implementations (see
src/webapps/omnigator/plugins/query).

This is just a proof-of-concept, and not polished. What have I done:

- created description files for toma and tolog
- added a selectlist for the query processor to form.jsp
- the selectlist is dynamically loaded: tolog as builtin, toma if it is
found in the classpath.
- based on the selection, a description for the query language is
loaded, together with example queries that can be selected.

To use this plugin together with toma, you just have to add the toma jar
(as result of mvn package) to the classpath of the ontopia distribution
(place into common/lib).

I also tried to create appropriate toma queries for the existing
examples of tolog, but they are not complete yet (some bugs in
complicated queries, grr)

Lars Marius Garshol

unread,
Oct 19, 2009, 6:46:19 AM10/19/09
to ont...@googlegroups.com

* Thomas Neidhart

>
> I checked in a generic query plugin for omnigator, which can be used
> to
> try out different query language implementations (see
> src/webapps/omnigator/plugins/query).

Excellent work! As far as I can see this is just a copy of the
existing plugin, and then an extension of it to also support Toma by
dynamically looking for it on the classpath.

It looks good to me. I was also able to successfully build and install
the Toma implementation, so everything appears to work just fine.
However, I think we should probably replace the old plugin with this
one, instead of having two different ones. I guess the support for
tolog updates should also go into this plugin, using the two-button
interface you proposed.

I added a label to the language choice dropdown to make the user
interface a little easier to understand. In the longer term we should
probably make the dropdown go away when there is only one language
available.

Thomas Neidhart

unread,
Oct 19, 2009, 7:20:58 AM10/19/09
to ont...@googlegroups.com
Lars Marius Garshol wrote:
>
> Excellent work! As far as I can see this is just a copy of the
> existing plugin, and then an extension of it to also support Toma by
> dynamically looking for it on the classpath.

Exactly, I did not want to touch the original plugin, but if the new
plugin is stable, it could remove the old one.

The check for the toma implementation is not nice, I was thinking of a
more automatic registration of each language implementation with the
omnigator plugins, something like a configuration file that is placed
into each jar at a specific package, so that the query plugin can find
available implementations, but probably there are some other ideas.

> It looks good to me. I was also able to successfully build and install
> the Toma implementation, so everything appears to work just fine.
> However, I think we should probably replace the old plugin with this
> one, instead of having two different ones. I guess the support for
> tolog updates should also go into this plugin, using the two-button
> interface you proposed.

Yes, the update mechanism should also be included.

> I added a label to the language choice dropdown to make the user
> interface a little easier to understand. In the longer term we should
> probably make the dropdown go away when there is only one language
> available.

sounds good.

Steve Pepper

unread,
Oct 19, 2009, 9:54:09 AM10/19/09
to ont...@googlegroups.com
While work is being done on the query plug-in...

Would there be any chance of adding the ability to export the result table to
CSV?

Steve

Lars Marius Garshol

unread,
Oct 19, 2009, 10:06:02 AM10/19/09
to ont...@googlegroups.com

* Steve Pepper

>
> While work is being done on the query plug-in...
>
> Would there be any chance of adding the ability to export the result
> table to CSV?

We might be able to get that in there. How would you do it? That is,
how, in the user interface, would you indicate that you want CSV to be
the output, instead of HTML? Also, how would you represent topic map
objects in the CSV?

Steve Pepper

unread,
Oct 19, 2009, 12:35:25 PM10/19/09
to ont...@googlegroups.com
* Lars Marius Garshol

|
| > Would there be any chance of adding the ability to export the result
| > table to CSV?
|
| We might be able to get that in there. How would you do it? That is,
| how, in the user interface, would you indicate that you want CSV to be
| the output, instead of HTML?

A button on the query results page. In other words: always display the HTML
table, but give an option to then export the table as CSV, with a Save dialog
box to choose the location.

| Also, how would you represent topic map objects in the CSV?

Good question. Use the same strategy as in the HTML table, I think. That is:

- (default) names for topics
- values for occurrences and names
- object ID for associations and roles

(Does that cover everything?)

If the user needs an ID for the topic (object ID, subject-identifier, whatever),
or the object ID of occurrences and/or names, this can be specified in the
query.

Steve


Lars Marius Garshol

unread,
Oct 19, 2009, 2:56:39 PM10/19/09
to ont...@googlegroups.com

* Steve Pepper

>
> A button on the query results page. In other words: always display
> the HTML table, but give an option to then export the table as CSV,
> with a Save dialog box to choose the location.

Now done: http://code.google.com/p/ontopia/source/detail?r=580

I didn't do the save dialog, but that can easily be added. The code
should be cleaned up a little to avoid so much code duplication with
query.jsp, and the query language name should be passed on, but other
than that it's now there.

Note that this is the new query plugin, not the old tolog plugin.

Steve Pepper

unread,
Oct 19, 2009, 3:14:55 PM10/19/09
to ont...@googlegroups.com
* Lars Marius Garshol

|
| * Steve Pepper
|
| > A button on the query results page. In other words: always display
| > the HTML table, but give an option to then export the table as CSV,
| > with a Save dialog box to choose the location.
|
| Now done: http://code.google.com/p/ontopia/source/detail?r=580

Cool. This will be very useful. Thank you.

Steve


Reply all
Reply to author
Forward
0 new messages