State of INSPIRE with FOSS (long)
Just van den Broecke
This mail (sorry for cross-posting) is to provoke some thinking with
regards to INSPIRE with FOSS, its products, the INSPIRE timeschedule and
the competition with proprietary INSPIRE solutions. Also since I have no
immediate answers I am seeking some wisdom from you clever FOSS folks.
The issues below are somewhat colored for the Dutch situation, the
questions I get, and focus on Download and View Services (viz WFS/WMS).
Metadata/CSW for INSPIRE through GeoNetwork is in quite good shape IMO.
- FOSS products: deegree v3, GeoServer (GS) and recently MapServer are
all providing INSPIRE extensions. These are however in various stages of
development and priorities (e.g. GS WMS first and deegree WFS first),
using very different architectures (deegree Blob storage upcoming
hybrid, GS on-the-fly relational mapping) and different focus.
I think this diversity is good (where is Python?) but the situation is
currently that neither is INSPIRE-ready for both WMS (1.3) and WFS
(2.0?) with GML 3.2.1.
- INSPIRE timeline: Discovery (CSW) and View (WMS) services (Annex I)
have to be operational by november 2011, Download (and Transformation)
Services (WFS) by dec 2012. See also
http://inspire.jrc.ec.europa.eu/index.cfm/pageid/44
- Proprietary solutions: apart from some smaller vendors like Snowflake,
and larger ones for FME, also ESRI has entered the INSPIRE market.
ESRI's solution is provided by the German firm Conterra:
http://www.conterra.de/en/products/esri-software/afi.shtm
As usual, dressing is sharp, brochures glossy and maybe their solution
actually works.
- FOSS success: in the past two years National Mapping and Cadastral
Agencies (NMCA's) have been prototyping/testing various FOSS and
Proprietary solutions and architectures, mainly for the entire "INSPIRE
chain", i.e. Transformation (ETL), services (WMS/WFS/CSW). Like for
example within ESDIN (www.esdin.eu) where the focus was on
Transformation and WFS, about 13 out 15 NMCAs finally used FOSS tools
like PostGIS, GDAL, deegree3 because these actually worked.
See also http://2010.foss4g.org/presentations_show.php?id=3416 ;-)
- More FOSS success: Planetek with lat/lon (deegree) have been chosen to
build the European INSPIRE Geoportal. See
http://www.planetek.it/eng/projects/inspire_geoportal and GeoServer
is/has contracts with various NMCAs (UK OS and IGN FR?) to develop
INSPIRE extensions.
- The problem: currently the situation has come that NMCAs are actually
choosing "an INSPIRE solution" and the ESRI's of this world are entering
the game at the same time. The smart NMCAs like IGN and OS may just
acknowledge that FOSS will bring them long term benefits. The problem
IMHO at this moment is that there is no complete "FOSS INSPIRE Stack".
For example, the solution used in the inspire-foss project
(http://code.google.com/p/inspire-foss/) with
GDAL/XSLT/PostGIS/deegree3/GeoExt lacks a compliant WMS 1.3. GeoServer
is also still not there for INSPIRE WMS 1.3 and lacks a robust WFS/data
mapping solution (and as a warning to GS-devs, getting this right,
INSPIRE-compliant and performant is much much harder than you think).
Diversity is good, but for the NMCAs to which I have been presenting the
situation is unclear. Maybe we are too honest in the FOSS world as many
proprietary products may not be ready either.
- The way out: the inspire-foss project has always been geared on
multiple INSPIRE solutions, e.g. not just deegree-only, though this is
still a very strong solution for ETL+fast WFS serving. As said, part of
this mail is to clarify what I think is the current situation but also
to possibly get ideas from you. Others may think that INSPIRE is just a
nasty heavy beast designed top-down without any rational software
engineering constraints and will just fade away as so many
all-encompassing standards' frameworks that had no prior proven
implementation (remember the ISO 7-Layer OSI stack?). We may find
INSPIRE back in another format (OSM?). Basically this mail comes back to
what Jeroen Ticheler, Arnulf, Jo Walsh and others had already formulated
2/3 years ago:
http://wiki.osgeo.org/wiki/INSPIRE and
http://wiki.osgeo.org/wiki/INSPIRE_data_experiments
I think we have come a long way since then and have proven/diverse
INSPIRE FOSS solutions though in various stages. I realize I have maybe
been too much focused on technique/software, loosing the bigger picture,
and that we need to have something more integrated both in organization
(OSGeo?) and in the software ("INSPIRE Stack" with component options).
If you got as far as here thanks for reading !
best regards,
--Just
Just van den Broecke
The Netherlands
www.justobjects.nl
Markus Schneider
Hash: SHA1
Hi Just,
sorry for the slow reply.
Am 21.04.2011 12:20, schrieb Just van den Broecke:
> - The way out: the inspire-foss project has always been geared on
> multiple INSPIRE solutions, e.g. not just deegree-only, though this is
> still a very strong solution for ETL+fast WFS serving. As said, part of
> this mail is to clarify what I think is the current situation but also
> to possibly get ideas from you. Others may think that INSPIRE is just a
> nasty heavy beast designed top-down without any rational software
> engineering constraints and will just fade away as so many
> all-encompassing standards' frameworks that had no prior proven
> implementation (remember the ISO 7-Layer OSI stack?). We may find
> INSPIRE back in another format (OSM?). Basically this mail comes back to
> what Jeroen Ticheler, Arnulf, Jo Walsh and others had already formulated
> 2/3 years ago:
> http://wiki.osgeo.org/wiki/INSPIRE and
> http://wiki.osgeo.org/wiki/INSPIRE_data_experiments
>
> I think we have come a long way since then and have proven/diverse
> INSPIRE FOSS solutions though in various stages. I realize I have maybe
> been too much focused on technique/software, loosing the bigger picture,
> and that we need to have something more integrated both in organization
> (OSGeo?) and in the software ("INSPIRE Stack" with component options).
> If you got as far as here thanks for reading !
I would like to comment on the status of deegree 3 components for
INSPIRE and what to expect in the next months:
1. Download Services: In order to get a complete INSPIRE solution based
on deegree 3, the following things need to be addressed:
- - Relational/hybrid storage: Is happening right now. In the last weeks,
I got AU, CP and AD themes (automatically) mapped to a relational schema
(with every optional element and attribute). Both queries and
transactions are working. I hope to be able to provide a demo (and
possibly a screencast) next week. Note that the relational mapping can
also be configured manually, so using deegree 3 WFS with custom
relational schemas (e.g. the one created by ESRI) will be possible.
- - WFS 2.0 / INSPIRE Download Service protocol extensions: No timeframe
yet. Will probably be implemented this year, because of lat/lon's
involvment in the INSPIRE portal.
2. View Services: deegree 3 WMS supports WMS 1.3.0 for quite some time.
Original INSPIRE styles (Symbology Encoding) can be used out of the box
to style the unmodified INSPIRE features. Things to be done:
- - INSPIRE View Service protocol extensions: Not much to be done (as WMS
1.3.0 is already implemented). Probably a few weeks of work. Should also
happen during the INSPIRE portal tasks.
- - Performance: In order to compete with other solutions, profiling is
needed. However, the WMS is already pretty scalable as it's based on a
streaming architecture.
- - Tiling: Support for tile caching needs to be done.
3. Discovery Services: deegree 3 CSW supports the ISO application
profile together with INSPIRE extensions. The (BLOB-based) storage has
recently been improved and some customer projects are currently being
worked on that prove that the CSW scales to dozens of millions of
records. Things to be done:
- - Integrate CSW records with metadata configuration of deegree WFS / WMS
and WPS
4. Transformation Services: deegree 3 WPS implements the WPS 1.0.0
protocol and should be ready for INSPIRE. Things to be done:
- - Add INSPIRE protocol extensions
- - Implement required processes
If all the above is finished, deegree inspireNode will provide a full
suite of INSPIRE services. Alternatively, there's the option to combine
deegree 3 with other (FOSS) products, such as GeoServer / MapServer and
GeoNetworks. Once deegree 3 WFS can be configured to work on a
relational schema usable by GeoServer, a possible scenario is to use
deegree 3 for INSPIRE data access / management and GeoServer for fast
map rendering.
Any feedback / cooperation is very much welcome.
Best regards,
Markus
P.S.: Just, feel free to post this mail to inspire-data (I am still not
a member).
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iEYEARECAAYFAk25xqkACgkQLM5hjXxU/E5UjwCgm/pW/JtRjBNOZPf+MwLN9GiD
ursAni4/GvSu7SmbjN0gc0M//EE2IEqe
=CMav
-----END PGP SIGNATURE-----
Just van den Broecke
Markus, thanks for your extensive answer ! And good to see deegree
moving forward. In hinsight I may have sounded too negative in my
original mail. That was certainly not my intention. I see so many
opportunities for FOSS within INSPIRE, not only for the obvious reasons,
but also since INSPIRE is a lot about integration and standards in
constant flux, any product will need to be agile and reactive. So I feel
the various FOSS projects should stick together on this.
I have experienced that INSPIRE is very knowledge-intense, with lots of
nasty details that come up during implementation, in particular when
testing. So while we may have a diversity in products, there is a great
deal of commonality we can have at least in knowledge sharing and
user/integrator-involvement, maybe even at the code/config level like
common relational schemas and SLDs. Another area is "CITE-like" test
automation which was started in ESDIN.
I plan to be in Bolsena next month, and hope some of you are there as
well, so maybe we can move things further under the Italian sun.
best,
Just
Markus Schneider
Hash: SHA1
Hi Just,
Am 02.05.2011 14:42, schrieb Just van den Broecke:
> Markus, thanks for your extensive answer ! And good to see deegree
> moving forward. In hinsight I may have sounded too negative in my
> original mail. That was certainly not my intention. I see so many
> opportunities for FOSS within INSPIRE, not only for the obvious reasons,
> but also since INSPIRE is a lot about integration and standards in
> constant flux, any product will need to be agile and reactive. So I feel
> the various FOSS projects should stick together on this.
>
> I have experienced that INSPIRE is very knowledge-intense, with lots of
> nasty details that come up during implementation, in particular when
> testing. So while we may have a diversity in products, there is a great
> deal of commonality we can have at least in knowledge sharing and
> user/integrator-involvement, maybe even at the code/config level like
> common relational schemas and SLDs. Another area is "CITE-like" test
> automation which was started in ESDIN.
I couldn't agree more. In my opinion, mastering INSPIRE is mostly a
technical / knowledge challenge. The combination of lots of bleeding
edge standards together with a very complex data model is just very ...
complex. Mastering all involved concepts / standards simply takes a
while. And this is what makes the idea of gathering best practices
guidelines for doing INSPIRE with FOSS extremely worthwhile.
> I plan to be in Bolsena next month, and hope some of you are there as
> well, so maybe we can move things further under the Italian sun.
Definitely. Andreas and me will be there and both think that it's a
great chance for knowledge transfer / experimentation and discussing how
to move things forward.
Best regards,
Markus
iEYEARECAAYFAk2+7+4ACgkQLM5hjXxU/E6XpwCgvtCgwrfuI+qW3RQuOID3wtUV
TrAAnieFxncFno7RqfFL5Cfg8oj568ok
=yDvO
-----END PGP SIGNATURE-----