Notes on NANOOS Xenia and OOSTethys Perl SOS server implementation

[Emilio Mayorga]

Hi there,

Jeremy: I'm happy to report that I've *finally* created my first
contribution to the Xenia Google Codes/wiki pages! It's some notes on
my still-internal implementation of the OOSTethys Perl SOS server code
originally from Eric:
http://code.google.com/p/xenia/wiki/nanoos_xenia_sos
Soon enough, I'll get around to posting notes on the Xenia
implementation itself.

Eric: These comments may or may not be of interest to you, but since
you created the SOS server code and have helped me with it, letting
you know is the least I could do.

Rick: This is the document on open issues that I said I'd put
together. As I was about to start up a Word document, I decided that
doing it this way makes more sense and gives us more mileage.

Alex: You haven't been in the loop for these SOS server discussions
(mostly between Rick and I), but I wanted you to be aware of what I've
been up to. We'll talk more about this in the future.

Cheers,

-Emilio Mayorga
NANOOS

[Eric Bridger]

This is excellent Emilio.
I'm glad you did this.
I think, when you go public with the service, perhaps you could post
the code on the OOSTethys plone site (like the current Perl code is
done) as a Xenia based reference implementation.
Great work!
I am currently seriously contemplating (and looking for funds from
NERACOOS) to update the Perl Cookbook to output the IOOS DIF
GetObservation response. I don't think it would be all that
difficult. I can steal Jeremy's example. A new GetObservation.xml
template, a new data type lookup table to map to the DMAC variable
names (WaterTemperature, AirTemperature, etc.)

Eric

[Jeremy Cothran]

Great work Emilio and thanks for the documentation, it will be good to have as a reference with some of the moving-point datasets we'll be working with.  Glad where links and info cross-posted on the oostethys website,etc and glad for Eric to adapt my earlier query resultset to IOOS DIF script as well as needed.  That resultset matching technique seems to be fairly useful - creating a database view or expected resultset which is scripted to produce various formats or services.

My original flows for some of the output products could be adapted better to the above more general target resultset technique - for some I go from xenia RDB->latest resultset->latest ObsKML->latest products(GeoRSS,html tables) - so removing the intermediate ObsKML generation or layer might be more generally useful (flatten ObsKML into a simple resultset with repeated column listings for higher repeated elements like platform,etc).

Rob Cermak pointed out some possible incosistencies with my IOOS DIF output versus NDBC&CO-OPS(Rob, feel free to reference those from earlier), I'm not sure if there is a compatibility/validation testing list/suite for either DIF or Oostethys SOS at this time but that might be a good diagnostic to have to help address service consistency across providers and to inform development efforts.

Thanks

Jeremy

[Rob Cermak]

Great work Emilio!

Jeremy, not sure if there is a compatibility problem. I have not read
the SOS spec in grave detail. Is it valid to require an
OBSERVEDPROPERTY with a GetCapability request?

I can see why you do it though. The general response would be HUGE!
I haven't seen any discussion on how this can be handled in a standard
way. One possible solution is to provide this as part of the SOS
ExceptionReports. The checker/dohicky that I've got going can then
adjust to changes to the Secoora SOS as you add/remove observed
properties.

<ExceptionReport xsi:schemaLocation="http://www.opengis.net/ows
owsExceptionReport.xsd" version="1.0.0" language="en">
<Exception exceptionCode="InvalidParameterValue"
locator="OBSERVEDPROPERTY" ExceptionText="OBSERVEDPROPERTY() requires
a value">
<ValidResponses>
<Value>windspeed</Value>
</ValidResponses>
</Exception>
</ExceptionReport>

If there is a more DIF or SWE acceptable way to do the above, I'm all
for it. For now, the Secoora SOS is the only SOS that operates this
way. I can definitely implement this as an optional feature on the
DIF server I'm rolling together. It makes sense that if someone wants
just WaterTemperature that they can get a short concise response
rather than having to parse a larger overall GetCapabilities message.

Rob

[Eric Bridger]

On Jun 2, 10:54 pm, Rob Cermak <rob.cer...@gmail.com> wrote:
> Great work Emilio!
>
> Jeremy, not sure if there is a compatibility problem.  I have not read
> the SOS spec in grave detail.  Is it valid to require an
> OBSERVEDPROPERTY with a GetCapability request?

My reading of the SOS spec is that this should not be required. The
whole idea of the GetCapability request is to be able to call it while
knowing nothing about the service besides the endpoint. For a similar
reasons the spec allows a GetCapabilities request without a version
number and to use GET vs. POST. The current SOS spec, I think,
specifies POST requests only except for GetCapabilities. But I too
understand Jeremy's logic. Are there other parameters you support
Jeremy? E.g. a way to specify just SECOORA observations? I think
that would be helpful to me.

>
> I can see why you do it though.  The general response would be HUGE!
> I haven't seen any discussion on how this can be handled in a standard
> way.  One possible solution is to provide this as part of the SOS
> ExceptionReports.  The checker/dohicky that I've got going can then
> adjust to changes to the Secoora SOS as you add/remove observed
> properties.
>
> <ExceptionReport xsi:schemaLocation="http://www.opengis.net/ows
> owsExceptionReport.xsd" version="1.0.0" language="en">
> <Exception exceptionCode="InvalidParameterValue"
> locator="OBSERVEDPROPERTY" ExceptionText="OBSERVEDPROPERTY() requires
> a value">
>   <ValidResponses>
>     <Value>windspeed</Value>
>   </ValidResponses>
> </Exception>
> </ExceptionReport>
>
> If there is a more DIF or SWE acceptable way to do the above, I'm all
> for it.  For now, the Secoora SOS is the only SOS that operates this
> way.  I can definitely implement this as an optional feature on the
> DIF server I'm rolling together.  It makes sense that if someone wants
> just WaterTemperature that they can get a short concise response
> rather than having to parse a larger overall GetCapabilities message.
>
> Rob
>

> On Tue, Jun 2, 2009 at 6:19 PM, Jeremy Cothran <jeremy.coth...@gmail.com> wrote:
> > Great work Emilio and thanks for the documentation, it will be good to have
> > as a reference with some of the moving-point datasets we'll be working with.
> >  Glad where links and info cross-posted on the oostethys website,etc and
> > glad for Eric to adapt my earlier query resultset to IOOS DIF script as well
> > as needed.  That resultset matching technique seems to be fairly useful -
> > creating a database view or expected resultset which is scripted to produce
> > various formats or services.
> > My original flows for some of the output products could be adapted better to
> > the above more general target resultset technique - for some I go from xenia
> > RDB->latest resultset->latest ObsKML->latest products(GeoRSS,html tables) -
> > so removing the intermediate ObsKML generation or layer might be more
> > generally useful (flatten ObsKML into a simple resultset with repeated
> > column listings for higher repeated elements like platform,etc).
> > Rob Cermak pointed out some possible incosistencies with my IOOS DIF output
> > versus NDBC&CO-OPS(Rob, feel free to reference those from earlier), I'm not
> > sure if there is a compatibility/validation testing list/suite for either
> > DIF or Oostethys SOS at this time but that might be a good diagnostic to
> > have to help address service consistency across providers and to inform
> > development efforts.
> > Thanks
> > Jeremy

[Emilio Mayorga]

Thanks, everyone! Your responses to what I considered a pretty
straightforward contribution gives me more motivation to document my
work here rather than internally or off on our own space.

To reply to Eric's comments:

> I think, when you go public with the service, perhaps you could post the
> code on the OOSTethys plone site (like the current Perl code is done) as a
> Xenia based reference implementation.

I'll be happy to post my code once we go public. However, I don't want
to contribute to a proliferation of versions of your code that are
just slightly different; Jeremy already has two versions on his xenia
site, that are tapping xenia databases. When the time comes, I'll ask
for suggestions here and in the oostethys list.

I found Jeremy's use of his xenia Google Code's svn to highlight
differences between versions to be very, very useful.

> I am currently seriously contemplating (and looking for funds from NERACOOS)
> to update the Perl Cookbook to output the IOOS DIF GetObservation response.
> I don't think it would be all that difficult. I can steal Jeremy's example.
> A new GetObservation.xml template, a new data type lookup table to map to
> the DMAC variable names (WaterTemperature, AirTemperature, etc.)

It'd be terrific to have a single code base to output either OOSTethys
or DIF SOS. I'm not yet following very closely the evolution of the
two and the distinct motivations for the two paths (eg, is the
OOSTethys SOS now a dead end with no future or value?).

Cheers,

-Emilio

[Eric Bridger]

On Jun 3, 2009, at 3:28 PM, Emilio Mayorga wrote:

> It'd be terrific to have a single code base to output either OOSTethys
> or DIF SOS. I'm not yet following very closely the evolution of the
> two and the distinct motivations for the two paths (eg, is the
> OOSTethys SOS now a dead end with no future or value?).

On the contrary. If anything the OOSTethys SOS is in line with the
OGC's Sensor Web Enablement (SWE Common) standards and best
practices. The DIF SOS has diverged from the SWE path and I fear may
cause issues down the line and in international interoperability
efforts.

[Emilio Mayorga]

Ah. Thanks. Then it's even better to be able to support both responses
from one code base and one data-source definition.

-Emilio

[Jeremy Cothran]

Hi Rob and Eric,

Thanks for the feedback/exchanges today, got busy with some other things today and responding just now.

I'm not sure if I follow Rob's example talking about GetCapabilities and the observedproperty parameter.  For the DIF SOS there isn't a GetCapabilities available yet that I remember and the Oostethys SOS should be the same as what is supported by Eric's earlier cgi script - so I'm not intending by design to do something outside of whats been done already with the existing DIF and Oostethys SOS methods.

Also trying to point out that my Xenia svn scripts might trail the leading groups specs/scripts by several months depending on the number of changes and my time to adapt or incorporate them.

If there's a link to one of the existing services that someone wants to point me to as incorrect, I can try to fix it or highlight it as something that needs to be fixed or lined up with other group efforts.

Thanks

Jeremy

[Jeremy Cothran]

As long as the code, cookbooks and documentation are persistently available online someplace where I can bookmark them, I can mix and match those pieces to whatever end needs.  The aspect of the google code stuff that are pluses for me is that its free hosted on someone elses servers and I don't have to worry about maintenance, security or performance issues on my servers, and the wiki documentation is part of the svn so that its possible to keep a backup of not only contributed source code but wiki documentation as well.  In regards to site organization, its always a never ending task of providing historical context of how some parts of the code are still applicable in some ways, where other stuff should be ignored in favor of something new.

So I'm ok with a more disorganized collection of websites, svn's approach using Q&A or further master documents to help point out the latest development paths between online cross-site documentation.

The virtualized vmware image is where I really want to go http://code.google.com/p/xenia/wiki/VMwareTest , more just like turnkey appliances/black boxes that are open for modification with the documentation on-hand but hopefully as long as the config and inputs are setup correctly it'll run the latest set of scripts,etc correctly.  

Thanks

Jeremy