Requirements exploration - metadata and manifest

[Brad Hards]

The requirements document talks about metadata in the form of a manifest, with
reference to ows:Manifest. For the avoidance of doubt, I'm assuming that
matches the content in OGC 06-121r9 version 2.0.0 Section 13.3.

There are a few ways we could do this. The referenced document presents the
manifest as XML, and we could just dump that XML into a single field in a
metadata table. That would be trivially interoperable, but may not be very
useful. At the other extreme, we could construct a set of tables and
relationships that closely resemble the UML in Figure 19 of the reference
document.

That chain-of-thought led me to ask what (and who) is the manifest for? Is any
of that content intended to be presented to the typical end-user, or is it
mainly intended for management systems?

In terms of a database implementation, we may not need much of it for any
software implementation, because the database tables may provide sufficient
information to allow access.

Is any metadata likely to be applicable to the composite package, and be
mandatory (e.g. security classification or releasability information)?

Brad

[Jeff Yutzler]

See responses inline...

On Sun, Mar 4, 2012 at 4:35 AM, Brad Hards <br...@frogmouth.net> wrote:

The requirements document talks about metadata in the form of a manifest, with
reference to ows:Manifest. For the avoidance of doubt, I'm assuming that
matches the content in OGC 06-121r9 version 2.0.0 Section 13.3.

That is correct. 

There are a few ways we could do this. The referenced document presents the
manifest as XML, and we could just dump that XML into a single field in a
metadata table. That would be trivially interoperable, but may not be very
useful. At the other extreme, we could construct a set of tables and
relationships that closely resemble the UML in Figure 19 of the reference
document.

It is a little early for this kind of design decision, but in some side discussions we were talking about the former if for no other reason that we have an XML encoding and don't want to invent a table schema unless absolutely necessary.

That chain-of-thought led me to ask what (and who) is the manifest for? Is any
of that content intended to be presented to the typical end-user, or is it
mainly intended for management systems?

The point of any manifest is to enable the recipient to figure out what is in the package and where it can be found in the package. I wouldn't expect the manifest itself to be presented to the end-user, but that the system accessing the GeoPackage would read the manifest and use it to aid the end-user in utilizing the contents. 

In terms of a database implementation, we may not need much of it for any
software implementation, because the database tables may provide sufficient
information to allow access.

 How?  If a system can traverse database metadata tables to figure out the structure, it can just as easily parse an XML block stored in a record of a specified table. 

Is any metadata likely to be applicable to the composite package, and be
mandatory (e.g. security classification or releasability information)?

My reading of the attendees at the kickoff meeting is that there will be resistance to mandating specific metadata elements beyond the manifest as part of this standard.  Communities of Interest would be welcome to create their own profiles but I do not see how this initiative will lead to a solution that enforces classification or releasability constraints.

 

Regards,

Jeff

--
Jeff Yutzler
Image Matters LLC
Mobile: (703) 981-8753
Office: (703) 428-6731

[Brad Hards]

On Monday 05 March 2012 17:09:42 Jeff Yutzler wrote:
> > That chain-of-thought led me to ask what (and who) is the manifest for?
> > Is any
> > of that content intended to be presented to the typical end-user, or is
> > it mainly intended for management systems?
>
> The point of any manifest is to enable the recipient to figure out what is
> in the package and where it can be found in the package. I wouldn't expect
> the manifest itself to be presented to the end-user, but that the system
> accessing the GeoPackage would read the manifest and use it to aid the
> end-user in utilizing the contents.

OK.

> > In terms of a database implementation, we may not need much of it for any
> > software implementation, because the database tables may provide
> > sufficient information to allow access.
>
> How? If a system can traverse database metadata tables to figure out the
> structure, it can just as easily parse an XML block stored in a record of a
> specified table.

My thinking was more about how to avoid redundant (and possibly not-quite-the-
same) data. The database metadata has to be there (whether used as the
manifest or not), and the database design ensures integrity. That may cover
the content, structure and format requirements described by the requirements
document.

I recognise that there may be metadata requirements that aren't encompassed
within the database internal tables. In particular, I note the requirements
document refers to accuracy and lineage metadata, which wouldn't normally be
found in database-level metadata.

However if we aren't showing that information to the end user, then I'm
interested to know who it is for, so we can ensure that we get the right
information into the right place.

Brad

[Jeff Yutzler]

On 3/5/12, Brad Hards <br...@frogmouth.net> wrote:
> On Monday 05 March 2012 17:09:42 Jeff Yutzler wrote:
>> > In terms of a database implementation, we may not need much of it for
>> > any
>> > software implementation, because the database tables may provide
>> > sufficient information to allow access.
>>
>> How? If a system can traverse database metadata tables to figure out the
>> structure, it can just as easily parse an XML block stored in a record of
>> a
>> specified table.
> My thinking was more about how to avoid redundant (and possibly
> not-quite-the-
> same) data. The database metadata has to be there (whether used as the
> manifest or not), and the database design ensures integrity. That may cover
> the content, structure and format requirements described by the requirements
> document.

Using the Spatialite schemas as a strawman implementation, the master,
geometry_columns, and spatial_ref_sys tables provide the database
metadata required to maintain integrity. We can do whatever we want
here, but I don't want to reinvent the wheel if not necessary.

A manifest would allow a consuming system to answer the following questions:
1. Is this file an OGC Geopackage? In other words, is there a
manifest in the ows_manifest table?
2. What vector layers does it have? This would include title,
abstract, and any other metadata (e.g. ISO 19115/19139) the producer
has provided to allow the consumer to determine whether it is likely
to contain pertinent information.
3. What f_table_name corresponds to this layer? This would provide
the mapping from the logical to the physical.

IMHO it is reasonable to keep the answers to these questions in an XML
block as per ows-manifest as opposed to creating a (somewhat complex)
table structure to handle it all.
-Jeff

[Paul Daisey]

Another advantage of using an XML document for the manifest is that it offers a variety of extension mechanisms, from XML/Schema type inheritance to xs:any for optional contents.  We can add "hooks" for manifest extensions without having to completely specify them.  Possible examples include ISO 19139 or NGA metadata, OGC Web Map Context documents, OGC Context documents, etc. The purpose is to enable existing applications and services to discover and/or describe / specify what is in a container using their own terms.  This would enable for example e.g. OGC WMS, WCS, WFS, WMTS and Open Street Maps to both produce and consume GeoPackages.