Strange problem with blob store for nilReason

[Just van den Broecke]

Hi,

This issue also comes from ESDIN tests.

It is a strange problem when inserting a GeographicalName with
FSLoader an unpopulated "pronunciation" element like:

<GN:pronunciation xsi:nil="true" nilReason="UNPOPULATED"/>

The insert into blob store is ok, but within the DB this element is
stored as:

<GN:pronunciation nilReason="UNPOPULATED"/>

The xsi:nil="true" has disappeared during the insert ? Why ?
I also tried nilReason="UNKNOWN" but this has the same effect.
Ditto for just using xsi:nil="true" without nilReason. Then I get
<GN:pronunciation></GN:pronunciation>

Other nil values are unmodified like:
<GN:beginLifespanVersion xsi:nil="true" nilReason="UNKNOWN"/>

Maybe there the issue is related to the GeographicalName definition in
the XSD which is

<element name="pronunciation" nillable="true">
<annotation>
<documentation>-- Definition -bla bla
</documentation>
</annotation>
<complexType>
<sequence>
<element ref="gn:PronunciationOfName"/>
</sequence>
<attribute name="nilReason" type="gml:NilReasonType"/>
</complexType>
</element>

But still: why ?

best,

Just

[Markus Schneider]

Hi Just,

can you provide a minimal dataset to reproduce the problem here?

Best regards,
Markus

[Just van den Broecke]

Yes:
http://code.google.com/p/inspire-foss/source/browse/trunk/etl/NL.Kadaster/GeographicalNames/test/geografisch_gebied-gn.gml

btw: I was using a very old 3.0-SNAPSHOT (sept 15) of the deegree-tools.
When upgrading to 3.0-rc1 using the script:
http://code.google.com/p/inspire-foss/source/browse/trunk/tools/loader/bin/fsloader.sh
I can't load at all and get the error :

/Users/just/project/customers/kadaster/svn/project/inspire-foss/trunk/tools/loader/../../webapps/deegree3/src/main/webapp/WEB-INF/workspace/datasources/feature/inspire-postgis.xml
/Users/just/project/customers/kadaster/svn/project/inspire-foss/trunk/tools/loader/../../webapps/deegree3/src/main/webapp/WEB-INF/workspace/jdbc/inspire_blob.xml
GML_32 USE_EXISTING geografisch_gebied-gn.gml
[18:38:27] INFO: [TempFileManager] Using
'/var/folders/uK/uKEvpdprEO0L5VAJ4Gfcak+++TI/-Tmp-/deegree-eb4be6fe-2d33-4f9e-88b5-7aa9041751a5'
for storing temporary files.
[18:38:27] INFO: [ConnectionManager] Using
'/var/folders/uK/uKEvpdprEO0L5VAJ4Gfcak+++TI/-Tmp-/deegree-eb4be6fe-2d33-4f9e-88b5-7aa9041751a5/lockdb'
for derby lock database.
Initializing feature store...done.
- Reading dataset: 'geografisch_gebied-gn.gml'...Exception in thread
"main" java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.deegree.tools.ToolBox$ToolInfo.invoke(ToolBox.java:464)
at org.deegree.tools.ToolBox.main(ToolBox.java:253)
Caused by: class org.deegree.commons.xml.XMLParsingException: Error in
XML document (file
'file:/Users/just/project/customers/kadaster/svn/project/inspire-foss/trunk/etl/NL.Kadaster/GeographicalNames/test/geografisch_gebied-gn.gml',
line: 2, column: 711, character offset: 749): Feature type
"{urn:x-inspire:specification:gmlas:BaseTypes:3.2}SpatialDataSet" is
unknown.
<< is empty >>
at
org.deegree.gml.feature.GMLFeatureReader.lookupFeatureType(GMLFeatureReader.java:848)
at
org.deegree.gml.feature.GMLFeatureReader.parseFeature(GMLFeatureReader.java:269)
at
org.deegree.gml.GMLStreamReader.readFeatureCollection(GMLStreamReader.java:233)
at
org.deegree.tools.feature.persistence.FeatureStoreLoader.insert(FeatureStoreLoader.java:124)
at
org.deegree.tools.feature.persistence.FeatureStoreLoader.main(FeatureStoreLoader.java:216)
... 6 more

But, Markus, also remember that your weekend has started! We can also
investigate this next week. best,

Just

--
kind regards / met vriendelijke groet,

--Just

Just van den Broecke ju...@justobjects.nl
Just Objects B.V. tel +31 65 4268627 Skype: justb4
The Netherlands http://www.justobjects.nl

[Just van den Broecke]

Some further investigation shows that this issue is IMO not related to
the container type SpatialDataSet. If a use a gml:featureCollection with
gml:featureMember-s then I get a similar error for application schema
types such as GN:NamedPlace:

Caused by: class org.deegree.commons.xml.XMLParsingException: Error in
XML document (file
'file:/Users/just/project/customers/kadaster/svn/project/inspire-foss/trunk/etl/NL.Kadaster/GeographicalNames/test/geografisch_gebied-gn.gml',

line: 4, column: 60, character offset: 842): Feature type
"{urn:x-inspire:specification:gmlas:GeographicalNames:3.0}NamedPlace" is

unknown.
<< is empty >>
at
org.deegree.gml.feature.GMLFeatureReader.lookupFeatureType(GMLFeatureReader.java:848)

I think that in general the FT lookup from the ApplicationSchema fails.
The reason could be that the ApplicationSchema object is not
parsed/populated. Although I have xsi:schemaLocation's I don't see any
network activity that the URLs for the XSDs are ever fetched.

best,

Just

[Markus Schneider]

Hi Just,

did you update the PostGISFeatureStore config as described? The error
could simply be caused by the feature store not getting initialized
properly (and a missing error message).

Best regards,
Markus

[Markus Schneider]

Fixed in trunk. Error was caused by not handling the XML nillability
correctly in *generic XML content*. However, for the property elements,
it was coped with, which is why it sometimes occurred and sometimes it
didn't.

Thanks for spotting this problem.

Best regards,
Markus

[Just van den Broecke]

Hi Markus,

Yes I think so. It was actually modified by Andreas on okt 22, to this:

<PostGISFeatureStore configVersion="0.6.0"
xmlns="http://www.deegree.org/datasource/feature/postgis"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.deegree.org/datasource/feature/postgis
http://schemas.deegree.org/datasource/feature/postgis/0.6.0/postgis.xsd">
<NamespaceHint prefix="ad"
namespaceURI="urn:x-inspire:specification:gmlas:Addresses:3.0" />
<NamespaceHint prefix="au"
namespaceURI="urn:x-inspire:specification:gmlas:AdministrativeUnits:3.0" />
<NamespaceHint prefix="base"
namespaceURI="urn:x-inspire:specification:gmlas:BaseTypes:3.2" />
<NamespaceHint prefix="bui"
namespaceURI="urn:x-inspire:specification:gmlas:Buildings:0.0" />
<NamespaceHint prefix="cp"
namespaceURI="urn:x-inspire:specification:gmlas:CadastralParcels:3.0" />
<NamespaceHint prefix="gn"
namespaceURI="urn:x-inspire:specification:gmlas:GeographicalNames:3.0" />
<NamespaceHint prefix="net"
namespaceURI="urn:x-inspire:specification:gmlas:Network:3.2" />
<NamespaceHint prefix="tn"
namespaceURI="urn:x-inspire:specification:gmlas:CommonTransportElements:3.0"
/>
<NamespaceHint prefix="tn-a"
namespaceURI="urn:x-inspire:specification:gmlas:AirTransportNetwork:3.0" />
<StorageCRS>EPSG:4258</StorageCRS>
<JDBCConnId>inspire_blob</JDBCConnId>
<GMLSchemaFileURL
gmlVersion="GML_32">../../schemas/inspire/annex1/</GMLSchemaFileURL>
</PostGISFeatureStore>

I also tried setting an absolute file path in GMLSchemaFileURL. But I
think the FS loader inits a FeatureStore for reading from the input file
I think.

best,

Just

[Markus Schneider]

Hi Just,

the current version of the schema is 0.6.1, see [1]. I am sorry for this
chaos, but as I wrote in an earlier mail we're currently reworking the
configuration -- and we're not finished with it yet, especially due to
changes needed for relational mapping / hybrid storage.

Up-to-date configs for INSPIRE can be found here: [2]. But be warned
that for the next days, configuration options may change anytime...

With that said, you may want to try this config:

<PostGISFeatureStore configVersion="0.6.1"

xmlns="http://www.deegree.org/datasource/feature/postgis"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.deegree.org/datasource/feature/postgis

http://schemas.deegree.org/datasource/feature/postgis/0.6.1/postgis.xsd">

<StorageCRS>EPSG:4258</StorageCRS>
<JDBCConnId>inspire_blob</JDBCConnId>

<GMLSchema version="GML_32">../../schemas/inspire/annex1</GMLSchema>
</PostGISFeatureStore>

Best regards,
Markus

[1] http://schemas.deegree.org/datasource/feature/postgis/
[2]
http://svn.wald.intevation.org/svn/deegree/applications/deegree-inspire-node/trunk/deegree-inspire-node/src/main/webapp/WEB-INF/workspace

[Just van den Broecke]

Ok thanks as well. Will test coming days. best, Just

[Markus Schneider]

A suggestion to deal with the current configuration situation:

As deegree 3.0 trunk is currently changing the configuration format,
it's probably not a good idea to switch back to 3.0-SNAPSHOT as
dependency. On the other hand, we're currently integrating bug-fixes for
INSPIRE. We plan to have 3.0-rc2 available on Friday. I could offer to
switch the dependencies and update the configs then.

What do you think?

Best regards,
Markus

[Just van den Broecke]

Hi Markus,

Yes, looks we are (almost?) there. Using 3.0-SNAPSHOT of deegree-tools
with your config gives

1) "nilReason problem" fixed
2) BLOB store loader error: fixed almost.

Only the stored namespace in the gml_objects table looks a bit strange
but is compliant I think:

<zdef-600975816:NamedPlace
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:ogc="http://www.opengis.net/ogc"
xmlns:gml="http://www.opengis.net/gml/3.2"
xmlns:xlink="http://www.w3.org/1999/xlink"

xmlns:zdef-600975816="urn:x-inspire:specification:gmlas:GeographicalNames:3.0"
gml:id="NL.KAD.GN.NL.TOP10NL.103075142">
<zdef-600975816:beginLifespanVersion xsi:nil="true"
nilReason="UNKNOWN"></zdef-600975816:beginLifespanVersion>
<zdef-600975816:endLifespanVersion xsi:nil="true"
nilReason="UNKNOWN"></zdef-600975816:endLifespanVersion>
<zdef-600975816:geometry><!--Inlined geometry
''NL.KAD.GN.NL.TOP10NL.103075142.PT''-->
<gml:Point gml:id="NL.KAD.GN.NL.TOP10NL.103075142.PT" srsName="EPSG:4258">
<gml:pos>6.12383115 52.01823089</gml:pos>
</gml:Point>
</zdef-600975816:geometry>
<zdef-600975816:inspireId>
<zdef-13819722:Identifier
xmlns:zdef-13819722="urn:x-inspire:specification:gmlas:BaseTypes:3.2">
<zdef-13819722:localId>NL.TOP10NL.103075142</zdef-13819722:localId>
<zdef-13819722:namespace>NL.KAD.GN</zdef-13819722:namespace>
</zdef-13819722:Identifier>
</zdef-600975816:inspireId>
<zdef-600975816:localType>
<zdef1853282383:LocalisedCharacterString
xmlns:zdef1853282383="http://www.isotc211.org/2005/gmd" locale="nl-NL">
streek, veld
</zdef1853282383:LocalisedCharacterString>
</zdef-600975816:localType>
<zdef-600975816:name>
<zdef-600975816:GeographicalName>
<zdef-600975816:language xsi:nil="true"></zdef-600975816:language>
<zdef-600975816:nativeness>endonym</zdef-600975816:nativeness>
<zdef-600975816:nameStatus>official</zdef-600975816:nameStatus>
<zdef-600975816:sourceOfName>Het Kadaster</zdef-600975816:sourceOfName>
<zdef-600975816:pronunciation nilReason="UNPOPULATED"
xsi:nil="true"></zdef-600975816:pronunciation>
<zdef-600975816:spelling>
<zdef-600975816:SpellingOfName>
<zdef-600975816:text>Stadsweide</zdef-600975816:text>
<zdef-600975816:script>Latn</zdef-600975816:script>
</zdef-600975816:SpellingOfName>
</zdef-600975816:spelling>
<zdef-600975816:grammaticalGender
xsi:nil="true"></zdef-600975816:grammaticalGender>
<zdef-600975816:grammaticalNumber
xsi:nil="true"></zdef-600975816:grammaticalNumber>
</zdef-600975816:GeographicalName>
</zdef-600975816:name>
<zdef-600975816:relatedSpatialObject xsi:nil="true"
nilReason="UNKNOWN"></zdef-600975816:relatedSpatialObject>
<zdef-600975816:type>landcover</zdef-600975816:type>
</zdef-600975816:NamedPlace>

Tried testing with WFS (deegree-services 3.0-rc1) but got error:
"No subcontroller for request namespace 'http://www.opengis.net/wfs'
available". But I am using now different versions (could not fetch
3.0-SNAPSHOT for deegree-services from mvn repo). I can try when all
trunk code/config is available as 3.0-SNAPSHOT or better 3.0-rc2 I think.

best,

Just

[Just van den Broecke]

Crossing mails. Yes was my suggestion as well. Ok, and let me know.
Configs are in both

http://code.google.com/p/inspire-foss/source/browse/#svn/trunk/webapps/deegree3

and

http://code.google.com/p/inspire-foss/source/browse/#svn/trunk/demos/deegree-inspire-demo

best,

Just

[Markus Schneider]

Hi,

Am 01.11.2010 14:22, schrieb Just van den Broecke:
> Yes, looks we are (almost?) there. Using 3.0-SNAPSHOT of deegree-tools
> with your config gives
>
> 1) "nilReason problem" fixed
> 2) BLOB store loader error: fixed almost.
>
> Only the stored namespace in the gml_objects table looks a bit strange
> but is compliant I think:
>
> <zdef-600975816:NamedPlace

...
> </zdef-600975816:NamedPlace>

Will look into this as soon as I find the time.

> Tried testing with WFS (deegree-services 3.0-rc1) but got error:
> "No subcontroller for request namespace 'http://www.opengis.net/wfs'
> available". But I am using now different versions (could not fetch
> 3.0-SNAPSHOT for deegree-services from mvn repo). I can try when all
> trunk code/config is available as 3.0-SNAPSHOT or better 3.0-rc2 I think.

Very well. Did you check if the WFS config has changed as well (I think
it did)?

Best regards,
Markus

[Just van den Broecke]

On 01-11-10 22:45, Markus Schneider wrote:
> Hi,
>
> Am 01.11.2010 14:22, schrieb Just van den Broecke:
>> Yes, looks we are (almost?) there. Using 3.0-SNAPSHOT of deegree-tools
>> with your config gives
>>
>> 1) "nilReason problem" fixed
>> 2) BLOB store loader error: fixed almost.
>>
>> Only the stored namespace in the gml_objects table looks a bit strange
>> but is compliant I think:
>>
>> <zdef-600975816:NamedPlace
> ...
>> </zdef-600975816:NamedPlace>
>
> Will look into this as soon as I find the time.

Ok

>
>> Tried testing with WFS (deegree-services 3.0-rc1) but got error:
>> "No subcontroller for request namespace 'http://www.opengis.net/wfs'
>> available". But I am using now different versions (could not fetch
>> 3.0-SNAPSHOT for deegree-services from mvn repo). I can try when all
>> trunk code/config is available as 3.0-SNAPSHOT or better 3.0-rc2 I think.
>
> Very well. Did you check if the WFS config has changed as well (I think
> it did)?

Yes I saw later. At some point I had an incompatible combination of
deegree-services/tools versions and WFS config versions. I have now
switched back to the versions I had previously (3.0-pre9) and will await
3.0-rc2. If there are 3.0-SNAPSHOT versions inbetween I can always test,
keeping an eye on http://schemas.deegree.org.

>
> Best regards,
> Markus
>
best,

Just

[Just van den Broecke]

Hi Markus,

Just to close off this thread.

I have just tested with 3.0-rc2 for all deegree components (services,
core, tools/loader etc): Loader works, WFS query responses validate.
All issues below have been fixed. Thanks, great work!

best,

Just