JEQL Roadmap

[Martin Davis]

Now that the JEQL language is getting mature and is powerful enough for a wide variety of uses,  the focus of development direction is going to start changing towards adding useful functions and commands. 

Some of the planned enhancements are:

- charting (courtesy of open-source Java charting APIs such as JFreeChart
- geospatial reprojection (courtesy of JHLabs PROJ4 port)
- further geometry functionality (possibilities include noding, dissolving, line merging, polygon coverage cleaning, polygon overlay, Delaunay triangulation, and Constrained Delaunay Triangulation, Warping)
- more readers & writers (possibilities include: KML,. JSON, GeoJSON, GeoRSS, AtomPub)
- database connectivity (PostgisReader already exists in the lab; others are relatively simply to develop)

Potential language enhancements include:
- UNION function
- first-class syntax for geometry
- foreach looping

Suggestions for other improvements are welcome!

Martin

[Yao Cui]

Hi Martin, are you working on adding data connection to Oracle and SQL Server as well? Thanks. Yao

[Martin Davis]

Hi, Yao.

In general, yes.  This will happen in two phases.

The first phase will cover non-spatial data.  This is pretty easy to do, by taking advantage of the existing JDBC support for these databases.  In fact, I'm planning to make a generic JDBCReader which will handle any database (or other data source) for which a JDBC driver exists.

For spatial data it's a bit more complicated.  For this I plan on providing specific readers for each database (e,g. PostgisReader, OracleReader, etc).  Postgis is close to being done.  Oracle won't be too hard to provide as well.  SQLServer won't be hard, but I will have to find a testbed for this development.

I'd be interested in getting feedback on this design, if anyone is interested in being a beta tester.

Martin

[Yao Cui]

I'd be interested in beta test for both SQL Server and Oracle. I've installed SQL Server 2008 RC0 and configured a test instance.

[Martin Davis]

Ok, Yao, I'll work on getting a test version set up for you. 

Martin

[Keith]

I am very interested in the Geometry Functionality you described. The
"foreach looping" might help me build scripts as I'm more familiar
with that construct.

Keith

[Martin Davis]

Keith, can you provide any more details or ideas about how you might use looping? 

I have a rough design in mind, which I'll write up and post when I can.  It will help me fine-tune the design to get other input on what looping might be used for.

Martin

[Keith]

Martin,
After some thought, the foreach construct might not be needed. It is
challenging to make a complex SQL statement work correctly. So I
sometimes create SQL ResultSets and then use If or Case statements for
finer grain comparisons, even joining Resultsets. This way I'm able to
test each part of the solution while I build it. Maybe I need to
buckle down and learn more about Subqueries, correlated and non-
correlated. The PostGIS Nabble forum & wiki has many geometry
examples.

On Jul 21, 11:51 am, "Martin Davis" <mtncl...@gmail.com> wrote:
> Keith, can you provide any more details or ideas about how you might use
> looping?
>
> I have a rough design in mind, which I'll write up and post when I can.  It
> will help me fine-tune the design to get other input on what looping might
> be used for.
>
> Martin
>

[Martin Davis]

Yes, I know what you're saying - it can be quite tricky to get the right logic to express a complex operation in SQL.  It's a bit frustrating that SQL can be so obtuse - sometimes it feels like you're working against the language, and that just writing some loops would be simpler. 

One thing I'm trying to do with JEQL is identify patterns which lead to complex SQL statements, and try and think of ways to simplify them.  Sometimes this involves introducing operators and concepts which aren't strictly "relational" - but I think that is a pragmatic decision and a small price to pay for clarity.

An example would be selecting values from the row(s) which contain the maximum (or minimum) value in a given column.  In standard SQL this requires a hard-to-understand sub-select or self-join.  In JEQl you can use a simple grouped select and use the first() aggregate function to pick out the value of another column on the chosen row(s).

In general it's good to try and express as much as possible as standard SQL queries - these essentially correspond to iterating over an entire table.  This set or table-oriented processing is what gives SQL (and JEQL) much of its elegance, I think.

However, I still think that foreach is useful for some situations.  The idea is to allow iterating over each row of a table, and execute a series of statements for each row, using the row values as "parameters" to the statement block.  This will allow doing things like expressing a process to be carried out for each file in a directory.  (That's actually the main use case I see so far - but I suspect that others will crop up).

So carry on fighting the good fight with SQL - but stay tuned for foreach!

Martin