[Open Rosa Meta Data Schema - Version 0.1]
Ian Lawrence
I would like to try to push the work on the OpenRosa API's into
accepting a formal specification agreement.
Easiest was to first cherry pick the low hanging fruit so lets start
with the MetaData schema which according to [0] is 99% complete. This
will be used by XForms to communicate with OpenRosa compliant
servers. The metadata is used to ensure that critical or extremely
useful data pertaining to XForms and submissions made by client
devices are captured and stored. The current text of the MetaData
schema is here [1] (slightly modified for clarity and pasted below)
and I have named this [Open Rosa Meta Data Schema - Version 0.1]. I
suggest we allow a grace period of a month or so for any comments/
modifications to this MetaData schema and then move towards signing at
the start of March with hosting in a publicly available space soon
after.
In terms of the actual agreement we could make use of the current Open
Web Foundation Agreement 0.9 which is presented here [2] The Open Web
Foundation (OWF) is an independent non-profit dedicated to the
development and protection of open, non-proprietary specifications for
web technologies.
There are already cases [3] of a Draft 1.0 OWFa being used out in the
wild so we can optionally use this later draft if there is general
agreement that this is a good thing. Advice obviously accepted on
that.
Regards
[0] https://bitbucket.org/javarosa/javarosa/wiki/OpenRosaAPI
[1] https://bitbucket.org/javarosa/javarosa/wiki/OpenRosaMetaDataSchema
[2] http://www.openwebfoundation.org/legal/agreement-09
[3] https://github.com/michiel-unhosted/unhosted
*********************[Open Rosa Meta Data Schema - Version
0.1]**********************************
= MetaData Scheme =
This MetaData schema will be used by XForms to communicate with
OpenRosa compliant servers. This metadata is used to ensure that
critical or extremely useful data pertaining to XForms and submissions
made by client devices is captured and stored.
= MetaData =
There are two types of metadata in a form submission. The first
provides metadata about the identity and version of the XForm used to
create the data being submitted. The second captures useful
information about the submission itself.
=== Location of XForms MetaData ===
The metadata describing the form identity and versioning is provided
as required attributes within the top-level group of the default model
instance of the form. The three required elements are:
* {{{version}}} an integer-valued major version number for the
XForm. This **MUST** be incremented when a change to the XForm alters
the structure of the data model, and therefore the submission data,
being transmitted to the server. One example of this is adding a new
element to the model or changing a {{{select1}}} into a {{{select}}}
(multi-valued) element.
* {{{uiVersion}}} an integer-valued minor version number for the
XForm. This **SHOULD** be incremented when a change in the XForm
alters the presentation of the questions to the user. I.e, if the
questions are re-arranged, the wording of the questions are changed,
etc. Such changes can alter a survey's outcomes. By incrementing
this number, data analysts can detect and investigate the impacts of
these changes.
* Either {{{id}}} or {{{xmlns}}} identifying the form id. These
values should be in the form of {{{scheme:value}}}. For the {{{id}}},
the implementor's registered domain name should be used as part of the
scheme (e.g., {{{opendatakit.org:widgetForm}}}). If specified, the
{{{id}}} value takes precedence over any explicit {{{xmlns}}}
declaration.
=== Location of Submission MetaData ===
The MetaData block is located within the Model block of an XForm. It
consists of a group of elements that describe the required information
that should be included in every xml submission made to the server.
=== Required and Optional elements ===
There are two types of fields to be used in the metadata block:
required and optional fields.
Required fields are used primarily to facilitate in the storage/
sorting of submissions on the server and are useful during analysis of
the data included in submissions.
Optional fields, as the name implies, are not required for server
operation but are fields that are commonly used for data analysis and
can also be used for administration of submissions. All optional
fields are recommended to be consumable by OpenRosa Servers.
**Required elements**
The following fields (elements) are required in the metablock:
* **<timeStart/>**- A timestamp~* of when form entry was started
* **<timeEnd/>** - A timestamp~* of when form entry ended
* **<instanceID/>** - See the 'ID Formatting' Section
**~***Timestamps should follow the format specified by [[http://
en.wikipedia.org/wiki/ISO_8601|ISO 8601]]
**Optional elements**
The following is a list of elements that are generally useful to
include in the metablock:
* **<userID />** - An ID linking a specific user to the xml
submission. See the 'ID Formatting' section.
* **<deviceID />** - The ID of the specific device used to generate
the submission. See the 'ID Formatting' section.
* **<deprecatedID />** - the {{{<instanceID/>}}} of the submission for
which this is a revision. This revision will have been given a newly
generated {{{<instanceID/>}}} and this field is populated by the prior
value. Server software can use this field to unify multiple revisions
to a submission into a consolidated submission record.
=== ID Formatting ===
ID's, such as InstanceID, deprecatedID, userID and, deviceID, must
follow the format:
Scheme:value
For example:
<instanceID>uuid:97267f98c989461995c6e47800fcde5d</instanceID>
or
<userID>openid:http://www.google.com/profiles/odkuser</userID>
a scheme:value pair format is used to allow a wide variety of
identification schemes to be used and so avoids 'lock-in' into one
particular identification system. This format will also be useful in
the event of merging large data sets from different organisations, as
the scheme:value pair should be treated as an opaque string and will
therefore ensure that very few collisions occur.
**Recommended Prefixes for IDs**
Recommended Prefixes for <deviceID />
imei:
mac:
uuid:
\\
Recommended Prefixes for <userID />
mailto:
openid:
\\
Recommended Prefixes for <instanceID /> (and <deprecatedID />)
uuid:
If the above prefixes are //not// utilized for an ID, the implementor
**MUST** use their registered domain name in the prefix. For example:
<userID>user.datahq.org:someuser</userID>
** Additional ID Constraints **
*The combined scheme:value keypair should be no longer than 80
characters (ie, so that varchar(80) can be used)
*Only ONE type of each ID can be sent to the server. That is, only
one deviceID, one userID, one instanceID, and one deprecatedID, etc.
(with one scheme:value pair in each) can be sent in a submission.
\\
=== Metadata is Defined in the Model Definition ===
The elements that are being submitted to the server **MUST** be
defined in the model within a {{{<meta/>}}} tag having the {{{http://
openrosa.org/xforms/metadata}}} namespace. I.e., the model must look
something like this:
{{{
<h:head>
<h:title>Metablock example</h:title>
<meta jr:name="myexample"/>
<model>
<instance>
<data xmlns:orx="http://openrosa.org/xforms/metadata"
id="example.org:myFormId"
version="1" uiVersion="1" >
<orx:meta>
<orx:timeStart/>
<orx:timeEnd/>
<orx:instanceID/>
</orx:meta>
....
</data>
</instance>
....
}}}
A{{{<meta/>}}} tag, if present, can appear anywhere in the model. Only
the __first__ tag appearing in the model (in the OpenRosa XForms
namespace) is automatically populated by the device. I.e., if you
have a complex form and want to transmit a composed subset of fields
to your server, you will need to set up calculations to populate the
nested fragment's {{{<meta/>}}} tag from the values in those in the
first instance. Note also that the nested group being transmitted to
the server will need to have attributes specifying the version,
uiVersion and id so that the server can properly handle the
submission.
If a {{{<meta/>}}} tag is present in the model, it **MUST** contain
within it all the required fields, each appearing exactly once. The
ordering of those fields can vary. If they are to be included in the
form submission, the optional fields ({{{<userID/>}}}, {{{<deviceID/
>}}} and {{{<deprecatedID/>}}}) defined above and any other optional
fields **MUST** also be specified in this data block. i.e., all data
elements that might be sent to the server must be explicitly specified
in this data block.
The optional fields {{{<userID/>}}}, {{{<deviceID/>}}} and
{{{<deprecatedID/>}}} may or may not be populated by the device;
however, if they are present, the device **SHOULD** make a best effort
at setting these values. **NOTE:** except for the {{{<timeEnd/>}}}
field, these fields are expected to be set at the start of a form.
There is no guarantee that the submitted form will have been completed
entirely by the same user or even on the same device on which it was
started.
If additional optional fields are present, or if the fields should be
populated in a non-default manner of some kind, the {{{<meta/>}}} tag
**MAY** have a {{{fillMechanismUri}}} attribute that communicates to
the device how it should fill the values. If used, the implementor
**MUST** use their registered domain name in the scheme prefix of this
attribute value (e.g., {{{mydomain.org:funky-fill-mechanism}}}. All
optional fields will be empty values if the device does not understand
how to populate them. For interoperability, servers **MUST NOT**
require non-empty values in these fields and **MUST** handle forms
populated via the default mechanisms even when an alternative fill
mechanism is requested via the {{{fillMechanismUri}}} attribute.
**NOTE:** the form discovery mechanism ({{{formList}}} API) sends the
{{{deviceID}}} to the server as a query parameter. Therefore, it is
likely to not be useful to override the fill mechanism for the
{{{deviceID}}} value, as it will then diverge from the identifier
supplied on the {{{formList}}} API.
**NOTE:** bind entries are not required for these values. If omitted,
the data types are assumed to be string values. If you need the times
to be treated as datetimes within your xform (when you, e.g.,
reference them in a calculation), you will need to include a bind
entry identifying them as a datetime type.
=== Notes ===
\\
\\
\\
= Example =
This is example is taken from a hypothetical xform and does not
represent a submission ready to be submitted to the server.
{{{
#!xml
<h:head>
<h:title>Metablock example</h:title>
<model>
<instance>
<data xmlns:orx="http://openrosa.org/xforms/metadata"
xmlns="http://example.org/meta"
version="1" uiVersion="1" >
<orx:meta>
<orx:deviceID/>
<orx:timeStart/>
<orx:timeEnd/>
<orx:instanceID/>
</orx:meta>
....
</data>
</instance>
....
</h:head>
....
}}}
\\
Below is an example of an instance meta block (as part of a full
instance) ready to be submitted to the server.
{{{
#!xml
<?xml version='1.0'?>
<data version="1" uiVersion="1" xmlns:orx="http://openrosa.org/xforms/
metadata" xmlns="http://example.org/meta">
<orx:meta>
<orx:deviceID>uuid:38DN0236SAKWJOJNQB3XJI9RW</orx:deviceID>
<orx:timeStart>2010-08-12T04:08:29.765-5:00</orx:timeStart>
<orx:timeEnd>2010-08-12T04:10:23.062-5:00</orx:timeEnd>
<orx:instanceID>uuid.dimagi.org:GEPSJLOGH13TY8L77066GEJJW</
orx:instanceID>
<orx:userID>chwid.dimagi.org:Akende</
orx:userID>
</orx:meta>
<!-- Example instance data follows -->
<TextSection0_data>yes</TextSection0_data>
<ENUMID_data>V -14-4</ENUMID_data>
<TextSection1_data>yes</TextSection1_data>
<PROV_data>Zambezia</PROV_data>
<DIST_data>Milange</DIST_data>
<DIST_COD_data>10</DIST_COD_data>
<POST_data>Milange</POST_data>
<POST_COD_data>1</POST_COD_data>
<LOC_data>Milange</LOC_data>
<LOCTYPE_data>Urban</LOCTYPE_data>
<VILLAGE_data>Samora-Machel</VILLAGE_data>
<VILlAGE_COD_data>2</VILlAGE_COD_data>
<ENUM_data>66</ENUM_data>
<SURVEYLOC_data>-16.9089355 36.75921051 635.0 4.0</SURVEYLOC_data>
...
</data>
}}}
*********************[End of Open Rosa Meta Data Schema - Version
0.1]**********************************
Lawrence Rosen
> There are already cases [3] of a Draft 1.0 OWFa being used out in the
> wild so we can optionally use this later draft if there is general
> agreement that this is a good thing. Advice obviously accepted on
> that.
If you have a choice I'd encourage you to use the 1.0 versions, just because
they're better, even though they're not yet formally approved.
/Larry
> --
> You received this message because you are subscribed to the Google
> Groups "Open Web Foundation Discussion" group.
>
> To post to this group, send email to open-web...@googlegroups.com
> To unsubscribe from this group, send email to
> open-web-discu...@googlegroups.com
>
> For more options, visit this group at
> http://groups.google.com/group/open-web-discuss?hl=en?hl=en