Reenvisioning GeoScript Scala

[David Winslow]

Hey all,

I've talked to some of you already about this, but I've been rethinking the Scala port of GeoScript.  The current implementation is kind of modeled on the Python implementation, and kind of not. It's also something that I started while pretty new to the Scala language and perhaps not the best representation of idiomatic Scala.  In the meantime, some new language features have been added and I've learned a lot, so I've been thinking about a GeoScript Scala 2.0* (*: Well actually there is no GeoScript Scala 1.0, so I'll probably just bump to 0.8 ;)

The big thing on my mind is that while Scala can be terse and "script-y", it is not a dynamically typed language and perhaps simply trying to match the Python API 1:1 is not the best way to design a GeoTools wrapper.  In particular, I think that having static types means that the Scala version of GeoScript can have even tighter integration than other GeoScript languages with existing GeoTools code, but its compiled nature means that it's not the most natural fit for (for example) a JSR223 based GeoServer extension module.  Dynamic extensions to GeoTools software are of course possible with Scala, but I think in the immediate term it's going to be a better fit to focus on the ways in which Scala is a "better Java" including a compile/static analysis phase before deployment.

So over the next while (and this may take a while as GeoScript is pretty much a back-burner project for me) I'll be reviewing and rewriting the GeoScript Scala module.  In the interest of having a consistent design philosophy, I've tried to enumerate here the principles I'm trying to stick to, this time around :)

1. Use GeoTools types.  It should be pretty rare to need to define custom types, and GeoScript users should very rarely find themselves declaring variables or methods whose types aren't in the GeoTools API.  Scala provides the following features that help us "enrich" the interface of existing types, adding methods and abstractions without needing to modify third-party code:
• type aliases http://docs.scala-lang.org/tutorials/tour/abstract-types.html
• implicit conversions to effectively add methods to existing types http://www.codecommit.com/blog/scala/scala-for-java-refugees-part-6
• implicit conversions to emulate Haskell typeclasses http://debasishg.blogspot.com/2010/06/scala-implicits-type-classes-here-i.html . There was also a good talk on this topic at this year's NEScala conference: http://nescala.org/#t-10646327 (video and slides)
2. Provide immutable alternatives to the various mutable APIs in GeoTools.  The Scala collections library does an excellent job of demonstrating how mutable and immutable alternatives can be provided with common APIs.  With GeoTools the implementation is often fundamentally mutable so things will probably not be flawless here, but we can provide immutable alternatives to most operations based on copying.  The Style Combinators API (which has already been adopted by other GeoScript dialects due to its terseness) is an excellent example of how this sort of thing can look.
3. Embrace Scala supporting libraries.  For example, I'm planning on implementing Arbitrary instances for Filter, Expression, Schema, and Feature to support testing with ScalaCheck. https://github.com/rickynils/scalacheck/wiki/User-Guide .  These could also be used to test Java code (potentially even core GeoTools code.)

I've taken some steps in this direction on the "tutorial" branch already (https://github.com/dwins/geoscript.scala/tree/tutorial going through the Python/JavaScript tutorials for inspiration on the functionality that should be covered.) Feel free to let me know if you have any questions or ideas.

--

David Winslow

[Juan Marín Otero]

David, this is very interesting. Some comments below

On Wed, May 2, 2012 at 8:25 PM, David Winslow <cdwi...@gmail.com> wrote:

Hey all,

I've talked to some of you already about this, but I've been rethinking the Scala port of GeoScript.  The current implementation is kind of modeled on the Python implementation, and kind of not. It's also something that I started while pretty new to the Scala language and perhaps not the best representation of idiomatic Scala.  In the meantime, some new language features have been added and I've learned a lot, so I've been thinking about a GeoScript Scala 2.0* (*: Well actually there is no GeoScript Scala 1.0, so I'll probably just bump to 0.8 ;)

You might want to bump it up to 1.0 since there will be significant changes in the implementation (and potentially in how it's used). This is probably not a big deal though. Are you considering upgrading to latest Scala and SBT versions? 

 

The big thing on my mind is that while Scala can be terse and "script-y", it is not a dynamically typed language and perhaps simply trying to match the Python API 1:1 is not the best way to design a GeoTools wrapper.  In particular, I think that having static types means that the Scala version of GeoScript can have even tighter integration than other GeoScript languages with existing GeoTools code, but its compiled nature means that it's not the most natural fit for (for example) a JSR223 based GeoServer extension module.  Dynamic extensions to GeoTools software are of course possible with Scala, but I think in the immediate term it's going to be a better fit to focus on the ways in which Scala is a "better Java" including a compile/static analysis phase before deployment.

So over the next while (and this may take a while as GeoScript is pretty much a back-burner project for me) I'll be reviewing and rewriting the GeoScript Scala module.  

I've already expressed to you my interest in helping out. But it's even more back-burner for me, so we'll see how that goes :-)

 

In the interest of having a consistent design philosophy, I've tried to enumerate here the principles I'm trying to stick to, this time around :)

1. Use GeoTools types.  It should be pretty rare to need to define custom types, and GeoScript users should very rarely find themselves declaring variables or methods whose types aren't in the GeoTools API.  Scala provides the following features that help us "enrich" the interface of existing types, adding methods and abstractions without needing to modify third-party code:
• type aliases http://docs.scala-lang.org/tutorials/tour/abstract-types.html
• implicit conversions to effectively add methods to existing types http://www.codecommit.com/blog/scala/scala-for-java-refugees-part-6
• implicit conversions to emulate Haskell typeclasses http://debasishg.blogspot.com/2010/06/scala-implicits-type-classes-here-i.html . There was also a good talk on this topic at this year's NEScala conference: http://nescala.org/#t-10646327 (video and slides)

A big +1 on using GeoTools types. GeoTools wrapper as you also expressed above is key imho. I would suggest to only use Scala's "magic" when necessary, to keep the audience as broad as possible and some sanity in the learning curve.

 

1. Provide immutable alternatives to the various mutable APIs in GeoTools.  The Scala collections library does an excellent job of demonstrating how mutable and immutable alternatives can be provided with common APIs.  With GeoTools the implementation is often fundamentally mutable so things will probably not be flawless here, but we can provide immutable alternatives to most operations based on copying.  The Style Combinators API (which has already been adopted by other GeoScript dialects due to its terseness) is an excellent example of how this sort of thing can look.

Interesting. Is the Style Combinators API what's included in geocss, or something else?

 

1. Embrace Scala supporting libraries.  For example, I'm planning on implementing Arbitrary instances for Filter, Expression, Schema, and Feature to support testing with ScalaCheck. https://github.com/rickynils/scalacheck/wiki/User-Guide .  These could also be used to test Java code (potentially even core GeoTools code.)

I've taken some steps in this direction on the "tutorial" branch already (https://github.com/dwins/geoscript.scala/tree/tutorial going through the Python/JavaScript tutorials for inspiration on the functionality that should be covered.) Feel free to let me know if you have any questions or ideas.

I'd like to see a lightweight example of geo web application / service done with one of the usual Scala lightweight http libraries, kind of like this example in the Python tutorials[1]. 

I also have a question about how you envision this being used. Is the main intent to use this within sbt / console mainly? Or more like a library, a true GeoTools for Scala?

[1] http://geoscript.org/tutorials/web-flask-py/index.html#tutorials-web-flask-py

 

--

David Winslow

--
You received this message because you are subscribed to the GeoScript mailing list.
To post to this group, send email to geos...@googlegroups.com
To unsubscribe from this group, send email to geoscript+...@googlegroups.com
Visit this group at http://groups.google.com/group/geoscript or see http://geoscript.org

--
Juan Marín Otero
GIS Consultant

-------Visita mi blog en---------------------
http://guachintoneando.blogspot.com
---------------------------------------------------

[David Winslow]

Sure, 1.0 makes sense.

Maybe reviewing the API to help police against overuse of magic would be a good way for you to contribute ;)

The Style Combinators API is just called "style" in other GeoScript implementations.  I haven't gotten very far in documenting it :( but it basically looks like this: https://github.com/dwins/geoscript.scala/blob/tutorial/geoscript/src/test/scala/tutorial/StylingAndRendering.scala . The Python documentation is a little better and works pretty similarly (although it introduces some mutability where the Scala version is totally immutable.) http://geoscript.org/tutorials/style-basic/index.html#tutorials-style-basic (don't forget to click the 'Python' button at the top right.)

I think that it's totally possible to have a library that's suitable for REPL exploration, embedding into existing GeoTools applications, and building lightweight apps, so I would say that all of these are within the range of things I'd like to see GeoScript scala useful for.  My own use tends toward somewhere in the middle - little one-off tools like this: https://github.com/dwins/geoscript.scala/blob/tutorial/examples/src/main/scala/example/Render.scala

--

David Winslow

[Jesse Eichar]

I will help review the API as well since this is a topic of interest for me as well.

Jesse

--

David Winslow

--

[Justin Deoliveira]

So what are you thoughts on trying to maintain some kind of correlation to the "common" geoscript api? Like at least at the module/package level?

--
Justin Deoliveira

OpenGeo - http://opengeo.org

Enterprise support for open source geospatial.

[David Winslow]

I plan to keep at least module names in common - there will still be geom, proj, etc. (Actually GeoScript Scala has always spelled out those words - org.geoscript.geometry, org.geoscript.projection, etc.  So actually an API revision is in order to bring things back in line anyway.)

And I think we can do a decent job of keeping type names and constructors in the Scala GeoScript in line with other GeoScripts as well.  For example, talking about Layers:

package org.geoscript

package object layer {
  type Layer = org.geotools.feature.simple.SimpleFeatureType
  def Layer(schema: Schema) = createMemoryDataStoreAndFeatureTypeSomehow()
}

would allow similar organization to what's in GeoScript Python, while still avoiding defining custom classes and needing to bridge them back to GeoTools when working with existing code (or just leveraging parts of GeoTools that aren't integrated with GeoScript yet.)  A REPL session might look like this:

import org.geoscript.layer._

val l: Layer = Layer(Schema(...))

So I don't think that what I'm talking about necessarily entails a big division between GeoScript Scala and the other GeoScript implementations.

--

David Winslow

[Justin Deoliveira]

Cool, sounds good. I think that is really all we can ask for in trying to keep a common geoscript api model. Package names and perhaps class and method names where it is convenient to do so.