Re: fdsnsws-station and stationXML1.1

1 view
Skip to first unread message

Chad Trabant

unread,
Dec 5, 2019, 6:16:22 PM12/5/19
to fdsn-wg3...@fdsn.org

Dear Philipp,

I am in agreement with the sentiments of Jean-Marie and Philip C. From the perspective of a reader/parser, StationXML 1.1 can be read as if it were 1.0, with the small exceptions already noted. The issue you raise appears to be limited to validation of new documents against the older schema(s) and not with reading the metadata per-se.

As noted, the version is encoded in the XML instance. Any software that wishes to validate can almost trivially determine and use the appropriate schema. So the worst case scenario at the moment is: systems that wish to validate need to be updated to use the latest major-version schema as extensions are created. Or be a bit more clever and check for a declared namespace of "http://www.fdsn.org/xml/station/1" and trust the schemaLocation if it is included in the instance document.

You raise a validate point, we should be clear on the scale of the problem before considering if and how it should be addressed.

> Using this definition, an fdsn/station service returning StationXML 1.1 documents must have a different major version than its precedecessor returning StationXML 1.0, *if* a StationXML 1.1 *document* does *not* validate against the StationXML 1.0 *schema* definition (the one used by the update-unaware client to validate the response). Not vice versa.


It was not the intention to define backwards compatible as "must validate against the oldest schema (for the major version)". That would be nice if possible. Unless we find a different way to encode the extension, such a requirement, as Jean-Marie speculated, would appear to make it impossible to extend StationXML without breaking it's backwards capability. If we extend this logic to the fdsnws-station service, this would mean a new major version for any change to StationXML no matter how small.

Keep in mind that creating a new fdsnws-station major version has significant implications, new service endpoints by each center (probably running in parallel with the old ones for an extended time), updates to a lot of software that is coded with these URLs and more wrinkles. This is not something we should do trivially.

Perhaps there is a clever way to modify the schema (e.g. new elements would have a different namespace), such that extensions can be created while allowing the oldest schema to validate documents with the extensions. I will look into it next year and encourage others to investigate as well. Such a change may have implications for generators of documents, but as 1.1 is not yet widespread and could theoretically be deprecated and replaced with 1.2, this would be a much easier change than making a new service endpoint standard.

regards,
Chad


> On Dec 5, 2019, at 10:14 AM, Philipp Kästli @fdsn.org <kae...@sed.ethz.ch> wrote:
>
> Dear colleagues,
>
> First I apologize to WG3 members which are also on the EIDA technical mailing lists for cross-posting - the mail below is an implicit answer to the previous mail of Jean-Marie.
>
> Summarizing:
> I am absolutely in favor of quickly adopting stationXML 1.1, and of introducing it in fdsn/station web services.
>
> My only point is that ...
> - in order to comply with the versioning scheme defined in the FDSNWS standard,
> - in order not to break any fdsnws/station/1/ clients,
> - (thus, in order to increase trust in fdsn standards and procedures),
> ... station metadata in stationXML 1.1 format should be distributed over a fdsn station service v. 2.0, thus under the URL fdsnws/station/2/ .
>
> I suggest that the FDSN as holder of the standard quickly declares this new version and avoids all ambiguity before heterogeneous treatment of the issue starts to creep into the worldwide population of fdsn station services.
>
> Kind regards,
>
> Philipp
>
>
>
>
> ********************
>
> Dear Jean-Marie,
>
> With https://www.xml.com/pub/a/2003/12/03/versioning.html (1), you actually point to an excellent read.
>
> However before discussing its implication, I have to clarify on the term *compatibility*, as the document (1) uses it in a different way.
> (1) describes a SOAP context, where a customer sends an XML instruction to a service, which in return does something. Here, a backwards compatible service means that the new service version (implemented for digesting the new version of the instruction xml) is still capable to interpret the old one. Technically it means that the old xml fulfills also the new schema. Usability-wise (and I think this is the key point) it means that the new service version does not break the user expectation of a customer who is not aware of the service version change.
> (1) fdsnws (https://www.fdsn.org/webservices/FDSN-WS-Specification-Commonalities-1.2.pdf) describes a REST context, where a customer sends a GET request and, in return, receives a stationXML. A backwards compatible service, from a customer's perspective, means that a change of the service fulfills both requirements a) the service can understand an unchanged get request (a get request of a customer which is not aware of the new minor version, or implementation of the service), and b) the service returns a result a client unaware of the new service minor version or implementation "can understand". "can understand", in the context of a standard defined in xsd, means: it fulfills (validates against) the unchanged (old) schema.
> Only if a) and b) are met, a customer can go on using the updated service without being aware of the update. (thus without the need from the service provider's side to announce the change, and give the customer time for adaptation.
>
> Now I admit that this definition of compatibility relies more on common sense than on a literature source. However it seems in line with the intention of the authors of the FDSNws standard: they requested that a not compatible = major service upgrade should be made transparent to the customer by a change of the URL (more precisely, the major version number in the URL). A changed URL is a very powerful way to tell the user of the old URL that things are not as they were.
>
> Using this definition, an fdsn/station service returning StationXML 1.1 documents must have a different major version than its precedecessor returning StationXML 1.0, *if* a StationXML 1.1 *document* does *not* validate against the StationXML 1.0 *schema* definition (the one used by the update-unaware client to validate the response). Not vice versa.
>
> Now, stationXML 1.0 and 1.1 have the same namespace, which makes the same schemata at least applicable, and the changing version number does not per se break validation. So the question why a stationXML 1.1 document is not, at the same time, valid stationXML 1.0 is a justified one.
>
> It is not. XML works not in a way that you can, by default, just add anything anywhere, and if it is unknown to the schema, it is ignored. But you are required to define where "something unknown" can be added, and you can restrict what "unknown" means. This has been done in stationXML 1.0 - look in the schema for <xs:any> for any element, and xs:anyattribute for any attribute. However, you will see that all these extension points have the namespace="##other" attribute, thus these random add-ons to stationXML documents can be *anything but* elemnts defined in the stationXML scheme (why? To avoid ambiguity**; see below). As the new elements added in stationXML 1.1 are (obviously) part of stationXML, they do not qualify as "any unknown add-on" as allowed according to the 1.0 scheme.
>
> So all this is no argument against stationXML 1.1, or fdsn station web services. It is just an argument to offer stationXML 1.1 only on an fdsnws/station/2/query? service url, in order to make sure the customer is aware of the fact that he gets something he/she has to treat technically differently from what he gets from an fdsnws/station/1/query? service URL.
> I posted this issue on the FDSN WG 3 mailing group first as they are holders of both the stationXML and the fdsnws specifications, and should provide a fdsnws/station/2/ specification document with clarification on the topic. Although the practical implications are probably largest in Europe (aka EIDA), where scientists have to deal with many different service endpoints, potentially updated at a different pace...
>
> Best regards,
>
> Philipp
>
>
> ***********
> ** on ambiguity: imagine you have an staxml:station element type with a sequence of staxml:latitude (minOccurs: 1, maxOccurs: 1), a staxml:longitude (minOccurs:1, maxOccurs: 1), and xs:any (minOccurs:0, maxOccurs: indefinite) child elements. Now, if you allow the xs:any to be *really* anything (including elements from the staxml scheme), a user could do this:
> <station code="BLA">
> <latitude>47.0</latitude>
> <longitude> 7.0</longitude>
> <latitude>48.0</latitude>
> </station>
> , which is, at the same time valid and not valid, and it is definitely not what you would like to have passed to a staxml-aware software code...
>
> -----Original Message-----
> From: eida_...@gfz-potsdam.de [mailto:eida_...@gfz-potsdam.de]
> Sent: Donnerstag, 5. Dezember 2019 11:52
> To: eida_...@gfz-potsdam.de
> Subject: Re: fdsnsws-station and stationXML1.1
>
> Dear Philipp,
>
> Thanks for those clarifications.
>
> I'm also working on stationXML outside RESIF, I let them answer from their perspective.
> And since I will have, among other things, polynomial responses, I must use 1.1 version.
>
> * Anyway, you might have understood I'm far from an XML expert.
> I previously understood that any tag addition in an XML file (I'm not speaking specifically about stationXML here) was not a problem as clients can be made to only read a subset of them and ignore any unknown tag.
> Which was perfectly fine for me.
>
> And from your answer, I'm puzzled because I understand that any tag addition will break the backward compatibility.
> This would mean we would never be able to do any revision of an XML standard without breaking the compatibility.
>
>
> * Could it be because you are trying to validate a 1.1 document with regards to a 1.0 standard ?
> This obviously will break because, as it is written in the stationXML header, it's a 1.1 version, not a 1.0 version.
>
>
> * Quickly browsing through the web, the XML standard has an "Ignore Rule" that says precisely that a client "Must ignore" an unknown tag.
> https://www.xml.com/pub/a/2003/12/03/versioning.html
> https://www.goland.org/xmlschemabackwardscompat/
>
> Is that wrong ?
>
>
>
> Regards.
>
> Jean-Marie.
>
> Le 05/12/2019 à 09:35, Kästli Philipp kae...@sed.ethz.ch a écrit :
>> Dear Jean-Marie,
>>
>>
>> that stationXML 1.1 breaks compatibility with stationXML 1.1 you can
>> verify easily by actually validating an 1.1 document against the
>> stationXML 1.0 schem:
>>
>>
>> <https://www.fdsn.org/xml/station/fdsn-station-1.0.xsd>
>>
>> nonix:~/eida/stationxml$ curl
>> 'https://www.fdsn.org/xml/station/fdsn-station-1.0.xsd' >
>> fdsn-station-1.0.xsd nonix:~/eida/stationxml$ curl
>> 'https://ws.resif.fr/fdsnws/station/1/query?network=G&station=ECH&leve
>> l=station&format=xml'>
>> resif.xml
>> xmllint --schema fdsn-station-1.0.xsd resif.xml <?xml version="1.0"
>> encoding="UTF-8"?> <FDSNStationXML
>> xmlns="http://www.fdsn.org/xml/station/1"
>> xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
>> xmlns:resif="http://www.fdsn.org/xml/station/1/resif"
>> schemaVersion="1.1"
>> xsi:schemaLocation="http://www.fdsn.org/xml/station/1
>> http://www.fdsn.org/xml/station/fdsn-station-1.1.xsd">
>> [...]
>> </FDSNStationXML>
>> resif.xml:7: element Identifier: Schemas validity error : Element
>> '{http://www.fdsn.org/xml/station/1}Identifier': This element is not
>> expected. Expected is one of (
>> {http://www.fdsn.org/xml/station/1}Comment,
>> ##other{http://www.fdsn.org/xml/station/1}*,
>> {http://www.fdsn.org/xml/station/1}TotalNumberStations,
>> {http://www.fdsn.org/xml/station/1}SelectedNumberStations,
>> {http://www.fdsn.org/xml/station/1}Station ).
>> resif.xml fails to validate
>> nonix:~/eida/stationxml$
>>
>> XML as a format family is generally chosen because it is defined which
>> tags must, must can be present (and which tag contents are valid), and
>> bhus, because an xml document can easily be verified for validity
>> against the respective schema.In an xml-based environment, the
>> possibility to validate is a must.
>>
>> Even if you try to cheat (e.g. by declaring the 1.1 document as 1.0
>> and/or skipping validation), any client software generated against the
>> stationXML 1.0 scheme will fail to handle it, e.g. as the XML document
>> makes use of the stationXML 1.1 option that any BaseNode type (station
>> and channel) can have a persistent identifier (which was given in
>> stationXML 1.0), or if it lacks a creationDate (which is allowed to
>> lack in 1.1, but forbidden to lack in 1.0), if it has an operator at
>> the network level (allowed in 1.1, not allowed in 1.0), if it has a
>> subject or lacks an id in a comment (both not allowed in 1.0), or similar cases.
>>
>> For the full changes list, see
>> https://www.fdsn.org/xml/station/fdsn-station-changes-1.0-to-1.1.txt
>>
>> Kind regards, Philipp
>>
>>
>>
>> ----------------------------------------------------------------------
>> --
>> *From:* eida_...@gfz-potsdam.de <eida_...@gfz-potsdam.de> on
>> behalf of Jean-Marie SAUREL sau...@ipgp.fr <eida_...@gfz-potsdam.de>
>> *Sent:* 05 December 2019 09:47
>> *To:* eida_...@gfz-potsdam.de
>> *Subject:* Re: fdsnsws-station and stationXML1.1 Good morning
>> everyone,
>>
>> I also don't clearly understand why the stationXML 1.1 breaks
>> compatibility with stationXML 1.0.
>> Do you have any more details on that ?
>>
>> The only item that stationXML 1.1 removed is the StorageFormat tag,
>> which, as it was pointed in FDSN WG3, is very unlikely to have any use.
>> Furthermore, this tag is not a mandatory tag according to stationXML
>> 1.0 xsd (minoccurs=0).
>>
>> The other main modification is the correction of an error in
>> stationXML
>> 1.0 polynomial response representation.
>> The 1.0 version request a StageGain to be present event in a
>> polynomial response, which simply makes no sense and is against SEED manual 2.4.
>>
>> The only way I imagine a 1.1 stationXML could break a client coded for
>> 1.0 would be because of this missing StageGain in polynomial response.
>>
>>
>> Otherwise, a correctly coded stationXML 1.0 should not break if there
>> are optional or non-mandatory missing tags.
>>
>> Regards.
>>
>> Jean-Marie.
>>
>> Le 05/12/2019 à 08:30, Jonathan Schaeffer
>> jonathan....@univ-grenoble-alpes.fr a écrit :
>>> Hi John and all,
>>>
>>> I'd like to emphasize that RESIF does not break any specification of
>>> fdsnws:station1.0 (as you said).
>>>
>>> If we (ETC) are to identify the best options, we need to understand
>>> why
>>> stationXML1.1 "breaks any clients".
>>>
>>> We did not any complaint from users using Obspy as far as I know.
>>>
>>> The suggestion (having fdsnws:station2.0 and fdsnws:station1.2
>>> specifications and implementations) for this minor change will be a
>>> real pain for everyone, and in particular for the end users. I see a
>>> lot of usage drawbacks and won't elaborate here.
>>>
>>> If this issue is an emergency, I suggest we plan a visio-conference
>>> of ETC on this point. If it can wait untill the next ETC meeting,
>>> let's put it on the agenda.
>>>
>>> Cheers,
>>>
>>> Jonathan
>>>
>>> On 04/12/2019 22:07, Clinton John jcli...@sed.ethz.ch wrote:
>>>>
>>>> Dear EMB and ETC,
>>>>
>>>> You may recall a new version of stationXML was released by FDSN in
>>>> May last year, a minor version increase from 1.0 to 1.1.
>>>>
>>>> A few days ago, most of you will have noticed an email from Philipp
>>>> and Daniel to the FDSN WG3. They spotted that the RESIF node updated
>>>> their fdsnws-station to provide XML in stationXMl1.1 format.
>>>> Currently, we do not find any other fdsnws-event service that
>>>> returns stationXML1.1, even outside of EIDA. Inside EIDA, this
>>>> inconsistency, and lack of control on the format received from
>>>> fdsnws-station, breaks any clients, including the federator, and
>>>> will cause downstream issues for other users. For example, now a
>>>> simple request for metadata from AlpArray Z3 network will provide information in both formats.
>>>>
>>>> Unfortunately, there was no migration plan for the corresponding
>>>> fdsnws-station service - the specification is loose and 'allows'
>>>> distribution of different minor versions. As I understand, the
>>>> situation with different EIDA nodes providing different versions of
>>>> stationXML from the same service is not acceptable in the long term.
>>>>
>>>> A proposed plan that 1) ensures immediate re-compliance across EIDA
>>>> and
>>>> 2) provides a definite path to all migrate to the latest stationXML
>>>> version could be:
>>>> - RESIF reverts back to releasing stationXML1.0 from
>>>> fdsnws-station1.1 as soon as possible.
>>>> */(this unblocks the Federator developments)/*
>>>> - EIDA requests to FDSN WG3 that
>>>> a minor version fdsnws-station1.2 is released, clearly specifies it
>>>> returns stationXML1.0 format a major version fdsnws:station2.0, is
>>>> released, clearly specifies it returns stationXML1.1 format
>>>> - An engineering phase is required to ensure all nodes can provide
>>>> both
>>>> fdsnws:station1.2 and 2.0. For nodes that provide fdsnws via SC3,
>>>> this requires updating the software, most reasonably by gempa,
>>>> followed by installation of the new SC3 version at all nodes.
>>>> - all EIDA nodes would then support both
>>>> fdsnws:station1.2 to deliver stationXML1.0
>>>> fdsnws:station2.0 to deliver stationXML1.1 */(then the Federator can
>>>> also support stationXML1.0 or 1.1, depending on configuration of the
>>>> routing service)/*
>>>>
>>>>
>>>> I have discussed the issue with Helle today. She notes RESIF is not
>>>> currently in a position to provide feedback or take actions for a
>>>> couple of weeks and may not actually be able to revert to stationXML1.0.
>>>> Further, she notes this plan is relatively complex and may take
>>>> time - which is true. On the other hand, this is an opportunity for
>>>> EIDA to insist the fdsnws definitions become tighter and more
>>>> precise, and to point out future format changes need to be carefully
>>>> coupled with sufficient and appropriate changes to exchange formats.
>>>>
>>>> In the meantime, I would really appreciate opinions from the EIDA nodes.
>>>> Would others agree to the proposed plan, or suggest another plan or
>>>> even an entirely different strategy? Ideally, we would be able to
>>>> quickly come to a consensus, approach WG3 and speak on behalf of
>>>> EIDA on this issue If needed, we at ETH are happy to organise a
>>>> conference call with interested parties on this issue.
>>>>
>>>> Regards,
>>>>
>>>> John
>>>>
>>>>
>>>>> *From:* Kästli Philipp
>>>>> *Sent:* Donnerstag, 28. November 2019 16:01
>>>>> *To:* FDSN Working Group III <fdsn-wg3...@lists.fdsn.org
>>>>> <mailto:fdsn-wg3...@lists.fdsn.org>>
>>>>> *Subject:* stationXML version 1.1 and fdsnws/station service
>>>>> version 1.x
>>>>>
>>>>> Dear colleagues of FDSN WG3,
>>>>>
>>>>> Our last fdsnws/station specification is 1.2 and dates
>>>>> (content-wise) of April 2, 2019 (an update in June refers to the
>>>>> formatting of the standards document only).
>>>>> According to this document, the fdsnws/station response format is
>>>>> "StationXML" (or text), without indication of the version, and
>>>>> thus, implicitly, referring to StationXML 1.0
>>>>>
>>>>> On May 3, 2019, FDSNwg 3 published a new version 1.1 of StationXML.
>>>>> John and Chad describe the the compatibility situation as follows:
>>>>> "Additionally, a number of backwards-compatible changes suggested
>>>>> at previous meetings and on the project site were incorporated.
>>>>> Conforming to the versioning rules, all 1.0 version documents are
>>>>> also compliant with schema version 1.1, with one exception: the
>>>>> <StorageFormat> element was removed from the schema, but is not
>>>>> widely used and not expected to cause incompatibility."
>>>>> We would summarize this as follows:
>>>>> - stationXML 1.0 remains forward-compatible with 1.0, thus
>>>>> an "old" xml document is still readable for software implemented
>>>>> for the
>>>>> 1.1 format (with the exception mentioned)
>>>>> - stationXML 1-1 is *not* backward-compatible with 1.1,
>>>>> thus a new document violates the old specification and is not
>>>>> readable for software implemented against the 1.0 specification.
>>>>>
>>>>> This has implications on the fdsnws/station/1/ web service. The
>>>>> standards document v. 1.2 says the following on service versioning:
>>>>>
>>>>> "Web service implementations are versioned according the following
>>>>> three-digit (x.y.z) pattern:
>>>>> SpecMajor . SpecMinor . Implementation Where the fields have the
>>>>> following meaning:
>>>>> SpecMajor : The major specification version*, all implementations
>>>>> sharing this SpecMajor value will be backwards compatible with all
>>>>> prior releases.* Values start at 1. " And "The following base URI
>>>>> pattern is to be used at each data center implementing FDSN
>>>>> services: <site>/fdsnws/<service>/*<majorversion>*/"
>>>>>
>>>>> Thus, an FDSNWS user can expect that if she/he has implemented an
>>>>> fdsnws-station client according to the standard (thus, capable of
>>>>> digesting the stationXML of the time, e.g. 1.0), this client will
>>>>> never break, as long as she/he addresses the service under the same
>>>>> url ./fdsnws/station/*1*/. .
>>>>> An updated service with a non-backward-compatible response should
>>>>> have a different url ./fdsnws/station/2/. So, at the time being, it
>>>>> violates the fdsnws/station v. 1.x standard, and the justified
>>>>> expectation of some users, if a service fdsnws/station/1/ 1.x
>>>>> service returns the new stationXML 1.1 format.
>>>>> (at the same time, an fdsnws/station service provider of 2020 may,
>>>>> when reading the fdsnws/station/1/ specification, may not pay
>>>>> attention that it pre-dates the stationXML 1.1 version, and
>>>>> consider returning that very format)
>>>>>
>>>>> Proposed way out:
>>>>> - have a new fdsnws/station/1/ specification version
>>>>> 1.2(.1)
>>>>> with the only change that it explicitly says that it is referring
>>>>> to stationXML 1.0 as its response format
>>>>> - have a fdsnws/station/2/ specification 2.0 saying that
>>>>> the response format is 1.1 or any *backward* compatible upper
>>>>> version
>>>>>
>>>>> Kind regards,
>>>>>
>>>>> Daniel Armbruster
>>>>> Philipp Kästli
>>>>> Swiss Seismological Service @ ETHZ
>>>
>>
>>
>> --
>> --------------------------------------
>> Institut de Physique du Globe de Paris Observatoires Volcanologiques
>> et Sismologiques
>> +33 1 83 95 74 37
>> +33 6 20 35 28 04 (portable)
>> 1 rue Jussieu
>> 75238 Paris cédex 5
>>
>>
>> #############################################################
>> This message is sent to you because you are subscribed to
>> the mailing list <eida_...@gfz-potsdam.de>.
>> To unsubscribe, E-mail to: <eida_ma...@gfz-potsdam.de> To switch
>> to the DIGEST mode, E-mail to <eida_mai...@gfz-potsdam.de>
>> To switch to the INDEX mode, E-mail to
>> <eida_mai...@gfz-potsdam.de> Send administrative queries to
>> <eida_main...@gfz-potsdam.de>
>>
>
>
> --
> --------------------------------------
> Institut de Physique du Globe de Paris
> Observatoires Volcanologiques et Sismologiques
> +33 1 83 95 74 37
> +33 6 20 35 28 04 (portable)
> 1 rue Jussieu
> 75238 Paris cédex 5
>
>
> #############################################################
> This message is sent to you because you are subscribed to
> the mailing list <eida_...@gfz-potsdam.de>.
> To unsubscribe, E-mail to: <eida_ma...@gfz-potsdam.de> To switch to the DIGEST mode, E-mail to <eida_mai...@gfz-potsdam.de>
> To switch to the INDEX mode, E-mail to <eida_mai...@gfz-potsdam.de> Send administrative queries to <eida_main...@gfz-potsdam.de>
>
>
>
> ----------------------
> FDSN Working Group III
> Topic home: http://www.fdsn.org/message-center/topic/fdsn-wg3-products/ | Unsubscribe: fdsn-wg3-produ...@lists.fdsn.org
>
> Sent from the FDSN Message Center (http://www.fdsn.org/message-center/)
> Update subscription preferences at http://www.fdsn.org/account/profile/


Philipp Kästli

unread,
Dec 5, 2019, 9:13:14 PM12/5/19
to fdsn-wg3...@fdsn.org

Philip Crotwell

unread,
Dec 5, 2019, 9:43:34 PM12/5/19
to fdsn-wg3...@fdsn.org

Hi

Just want to point out that the stationxml contains the version of the schema it corresponds to, in the schemaVersion attribute and in the location of the schema for validation in the schemaLocation attribute, so a well-coded client can tell a 1.0 from a 1.1 xml file and act appropriately.

Are you aware of a specific client and service/url combination that breaks because of this? My memory is that the reason this exception to the versioning rule was allowed was that there were no known instances where the items changed were used, and so it was felt safe to allow it.

Before creating a new version of the web services, it is probably worth investigating whether or not the concern is an actual problem occurring in the real world or not. In general I am in favor of following semver, but the large disruption of a fdsn station url change has to be weighed against the benefits.

Philip

Philipp Kästli

unread,
Dec 6, 2019, 11:30:58 AM12/6/19
to fdsn-wg3...@fdsn.org
Dear Philip

> Are you aware of a specific client and service/url combination that breaks because of this? My memory is
> that the reason this exception to the versioning rule was allowed was that there were no known instances > where the items changed were used, and so it was felt safe to allow it.

I guess any decent client of a service providing a document in a format defined by an xsd would validate the document against that xsd in order to make sure that the subsequent processing software can cope with its content - it is probably the strongest reason why to use xml formats at all.

To point to a specific one: In Europe, we have many "small" data centers providing data of one or a few of our small countries (often entire networks, but some networks are also distributed over multiple datacenters). In order to make the life of the scientists easier, EIDA develops a "federator", i.e. a software which offers an fdsnws at its frontend from where customers can request (meta)data e.g. of areas covering many countries. The federator transparently & "intelligently" redistributes this request to all national or regional data centers, collects and merges the responses, and forwards the joint result to the customer. In a service landscape where the national services can return on the same request /fdsnws/station/1/query?...&format=xml technically (on the level of validation) different formats, and the format of a data center can change anytime without prior note, the "federator" needs to analyze any response first to detect its format of the day, then fully parse the response in order to convert to one homogeneous format before merging and returning. Steps which are not required if the responses of all services of the same type and version are (technically) compatible.

Now you can say, with some good reason: let the developers of the "federator" add some money and time and cpu power, and solve that problem, so we don't need to deal with it at the FDSN.

But the world is more than IRIS plus Europe, and not all data centers are federated.
So in all other places where scientists use data from multiple sources, they have to do the same task and check again and again: Do all my data sources still provide the same format? If not: how do I convert/unify? What are the impacts on my subsequent processing?
My fear is that overall, for the entire seismological community, this is more of a hassle than the one of the data centers configuring a new service URL for a service version providing a new response format (a response technically not fulfilling the previous specification).

Kind regards, Philipp

Chad Trabant

unread,
Feb 3, 2020, 7:00:06 PM2/3/20
to fdsn-wg3...@fdsn.org

Dear WG 3 members,

To move this issue forward I ask that members share their opinions and suggested solutions by the end of February. This will allow us to determine if quick action by the WG is needed.

To summarize the issue as understand it: StationXML 1.1 documents that contain elements/attributes not defined in the 1.0 schema cannot be validated using the 1.0 schema as they (purposely) use the same namespace and 1.0 does not allow additions in the same namespace. A conflict may occur, for example, when a client using an fdsnws-station 1.x service receives StationXML 1.1 that contains an <Identifier> element (new to 1.1) and tries to validate the response against the StationXML 1.0 schema.

Note: StationXML 1.1 was designed to be readable and usable by clients expecting version 1.0 (ignoring the new elements/attributes), which works in practice. Validation of a 1.1 document using the 1.0 schema is the case that does not work.

I see a couple of options to consider:

1) Define a new, major version 2, specification of the fdsnws-station service that would have a new endpoint. This would remove the ambiguity of which version of StationXML is returned (by default or via the format="xml") selection.

2) The FDSN could declare that fdsnws-station 1.x returns StationXML 1.x, and that any clients that wish to validate the responses must use the matching, or latest 1.x, schema specification. Note: the schema specification version is declared in the document itself, often with a link to the appropriate schema.

3) The next minor version of the StationXML schema (1.2) could be released with modifications that allow the documents to be validated with the 1.0 schema; then deprecate schema version 1.1 (not yet widely adopted) and move quickly to 1.2. I do not know of a technical solution for this, but it may be possible.

If you know of other options please share them to be considered. If we determine that the issue is best solved in a StationXML schema change, it will be referred to WG 2.

We should be mindful that future, minor releases of the StationXML and fdsnws-station specifications are likely. I recommend that we determine a clear path that allows for these releases with minimal disruption and invite suggestions for how to accomplish this.

regards,
Chad

> On Dec 6, 2019, at 12:32 AM, Philipp Kästli @fdsn.org <kae...@sed.ethz.ch> wrote:
>
> Dear Philip
>
>> Are you aware of a specific client and service/url combination that breaks because of this? My memory is
>> that the reason this exception to the versioning rule was allowed was that there were no known instances > where the items changed were used, and so it was felt safe to allow it.
>
> I guess any decent client of a service providing a document in a format defined by an xsd would validate the document against that xsd in order to make sure that the subsequent processing software can cope with its content - it is probably the strongest reason why to use xml formats at all.
>

> To point to a specific one: In Europe, we have many "small" data centers providing data of one or a few of our small countries (often entire networks, but some networks are also distributed over multiple datacenters). In order to make the life of the scientists easier, EIDA develops a "federator", i.e. a software which offers an fdsnws at its frontend from where customers can request (meta)data e.g. of areas covering many countries. The federator transparently & "intelligently" redistributes this request to all national or regional data centers, collects and merges the responses, and forwards the joint result to the customer. In a service landscape where the national services can return on the same request /fdsnws/station/1/query?...&format=xml technically (on the level of validation) different formats, and the format of a data center can change anytime without prior note, the "federator" needs to analyze any response first to detect its format of the day, then fully parse !

Robert Casey

unread,
Feb 4, 2020, 1:43:34 PM2/4/20
to fdsn-wg3...@fdsn.org

If schema validators were able to follow XML Schema 1.1 conventions, there's the notion of a Schema Versioning Workspace that allows for selective version ranges on individual elements. This practice should probably only be followed for minor version updates, as you can continue to use the same StationXML namespace (/1) with progressive updates. A major version is not meant to be backward or forward compatible, so the StationXML namespace would change major versions...and I would think the service endpoint as well.

Here's the W3C doc on the vc namespace:

https://www.w3.org/TR/xmlschema11-1/#vc-namespace <https://www.w3.org/TR/xmlschema11-1/#vc-namespace>

If this were something the working group wanted to pursue, the vc namespace could be introduced to a 1.2 version of the doc, with the focus of 'forward-compatibility' for older clients to be compatible on 1.0, 1.1, and 1.2 docs.

-Rob


> On Feb 4, 2020, at 8:48 AM, Philip Crotwell <crot...@seis.sc.edu> wrote:
>
> Hi
>
> I am not sure how I feel about this issue forcing a version 2 of the
> web services spec, but if there is a new major version, ie
> /fdsnws/station/2, perhaps adding a "schema" endpoint would prevent
> this issue in the future. The idea being that a version >=2 client is
> expected to validate the stationxml returned from
> /fdsnws/station/2/query against the schema returned from
> /fdsnws/station/2/schema at the same datacenter. The client should not
> validate against a predefined schema. This provides an easy way to get
> the correct schema, even though in practice it is also available at
> the URL in the stationxml document.
>
> Allowing an optional "format=" query parameter on this service would
> also allow schema (if they exist) for other formats, like geojson or
> kml that are returned by some data centers.
>
> At this point we can then define non-breaking change, which allows a
> minor version bump, as "designed to be readable and usable by clients
> expecting version x.0 so long as they ignore the new
> elements/attributes" but specifically does not expect validation
> against the older x.0 schema. Validation should only occur against the
> schema that corresponds to the version, from either the schema
> endpoint, or the corresponding schema url in the xml document (both
> assumed to be equivalent).
>
> Without something like this, I suspect that no change can ever be
> considered applicable for a minor version, which implies only major
> versions and hence there is never any backward compatibility.
>
> Philipp's federator example, would then validate the stationxml from
> the various datacenters using the schema that they expect to be used.
> The merging would of course require intelligence to know how to
> combine 1.0 and 1.1 stationxml documents, but that likely is needed
> regardless. I would guess that the federator should use the highest
> version known and be able to map earlier versions.
>
> Philip


>
> On Mon, Feb 3, 2020 at 6:58 PM Chad Trabant <ch...@iris.washington.edu> wrote:
>>
>>
>> Dear WG 3 members,
>>
>> To move this issue forward I ask that members share their opinions and suggested solutions by the end of February. This will allow us to determine if quick action by the WG is needed.
>>
>> To summarize the issue as understand it: StationXML 1.1 documents that contain elements/attributes not defined in the 1.0 schema cannot be validated using the 1.0 schema as they (purposely) use the same namespace and 1.0 does not allow additions in the same namespace. A conflict may occur, for example, when a client using an fdsnws-station 1.x service receives StationXML 1.1 that contains an <Identifier> element (new to 1.1) and tries to validate the response against the StationXML 1.0 schema.
>>
>> Note: StationXML 1.1 was designed to be readable and usable by clients expecting version 1.0 (ignoring the new elements/attributes), which works in practice. Validation of a 1.1 document using the 1.0 schema is the case that does not work.
>>
>> I see a couple of options to consider:
>>
>> 1) Define a new, major version 2, specification of the fdsnws-station service that would have a new endpoint. This would remove the ambiguity of which version of StationXML is returned (by default or via the format="xml") selection.
>>
>> 2) The FDSN could declare that fdsnws-station 1.x returns StationXML 1.x, and that any clients that wish to validate the responses must use the matching, or latest 1.x, schema specification. Note: the schema specification version is declared in the document itself, often with a link to the appropriate schema.
>>
>> 3) The next minor version of the StationXML schema (1.2) could be released with modifications that allow the documents to be validated with the 1.0 schema; then deprecate schema version 1.1 (not yet widely adopted) and move quickly to 1.2. I do not know of a technical solution for this, but it may be possible.
>>
>> If you know of other options please share them to be considered. If we determine that the issue is best solved in a StationXML schema change, it will be referred to WG 2.
>>
>> We should be mindful that future, minor releases of the StationXML and fdsnws-station specifications are likely. I recommend that we determine a clear path that allows for these releases with minimal disruption and invite suggestions for how to accomplish this.
>>
>> regards,
>> Chad
>>
>> On Dec 6, 2019, at 12:32 AM, Philipp Kästli @fdsn.org <kae...@sed.ethz.ch> wrote:
>>
>> Dear Philip
>>
>> Are you aware of a specific client and service/url combination that breaks because of this? My memory is
>> that the reason this exception to the versioning rule was allowed was that there were no known instances > where the items changed were used, and so it was felt safe to allow it.
>>
>>
>> I guess any decent client of a service providing a document in a format defined by an xsd would validate the document against that xsd in order to make sure that the subsequent processing software can cope with its content - it is probably the strongest reason why to use xml formats at all.
>>

>> To point to a specific one: In Europe, we have many "small" data centers providing data of one or a few of our small countries (often entire networks, but some networks are also distributed over multiple datacenters). In order to make the life of the scientists easier, EIDA develops a "federator", i.e. a software which offers an fdsnws at its frontend from where customers can request (meta)data e.g. of areas covering many countries. The federator transparently & "intelligently" redistributes this request to all national or regional data centers, collects and merges the responses, and forwards the joint result to the customer. In a service landscape where the national services can return on the same request /fdsnws/station/1/query?...&format=xml technically (on the level of validation) different formats, and the format of a data center can change anytime without prior note, the "federator" needs to analyze any response first to detect its format of the day, then fully pars!

Philip Crotwell

unread,
Feb 4, 2020, 2:47:31 PM2/4/20
to fdsn-wg3...@fdsn.org
Hi

Philip

Jeremy Fee

unread,
Feb 4, 2020, 6:57:49 PM2/4/20
to fdsn-wg3...@fdsn.org
The Station XML 1.0 schema appears to allow elements and attributes from any namespace.

One option is to use the existing namespace where elements are unchanged, and use a different namespace for Station XML 1.x elements. This maintains backward compatibility for existing 1.0 parsers, while allowing 1.x aware parsers to read the additional information.


Thanks,

Jeremy Fee
Computer Scientist
USGS Geologic Hazards Science Center, Golden, CO


________________________________
From: fdsn-wg3-pro...@fdsn.org <fdsn-wg3-pro...@fdsn.org> on behalf of Chad Trabant <ch...@iris.washington.edu>
Sent: Monday, February 3, 2020 16:57
To: FDSN Working Group III <fdsn-wg3...@lists.fdsn.org>
Subject: [EXTERNAL] Re: [fdsn-wg3-products] fdsnsws-station and stationXML1.1


Dear WG 3 members,

To move this issue forward I ask that members share their opinions and suggested solutions by the end of February. This will allow us to determine if quick action by the WG is needed.

To summarize the issue as understand it: StationXML 1.1 documents that contain elements/attributes not defined in the 1.0 schema cannot be validated using the 1.0 schema as they (purposely) use the same namespace and 1.0 does not allow additions in the same namespace. A conflict may occur, for example, when a client using an fdsnws-station 1.x service receives StationXML 1.1 that contains an <Identifier> element (new to 1.1) and tries to validate the response against the StationXML 1.0 schema.

Note: StationXML 1.1 was designed to be readable and usable by clients expecting version 1.0 (ignoring the new elements/attributes), which works in practice. Validation of a 1.1 document using the 1.0 schema is the case that does not work.

I see a couple of options to consider:

1) Define a new, major version 2, specification of the fdsnws-station service that would have a new endpoint. This would remove the ambiguity of which version of StationXML is returned (by default or via the format="xml") selection.

2) The FDSN could declare that fdsnws-station 1.x returns StationXML 1.x, and that any clients that wish to validate the responses must use the matching, or latest 1.x, schema specification. Note: the schema specification version is declared in the document itself, often with a link to the appropriate schema.

3) The next minor version of the StationXML schema (1.2) could be released with modifications that allow the documents to be validated with the 1.0 schema; then deprecate schema version 1.1 (not yet widely adopted) and move quickly to 1.2. I do not know of a technical solution for this, but it may be possible.

If you know of other options please share them to be considered. If we determine that the issue is best solved in a StationXML schema change, it will be referred to WG 2.

We should be mindful that future, minor releases of the StationXML and fdsnws-station specifications are likely. I recommend that we determine a clear path that allows for these releases with minimal disruption and invite suggestions for how to accomplish this.

regards,
Chad

On Dec 6, 2019, at 12:32 AM, Philipp Kästli @fdsn.org <kae...@sed.ethz.ch<mailto:kae...@sed.ethz.ch>> wrote:

Dear Philip

Are you aware of a specific client and service/url combination that breaks because of this? My memory is
that the reason this exception to the versioning rule was allowed was that there were no known instances > where the items changed were used, and so it was felt safe to allow it.

I guess any decent client of a service providing a document in a format defined by an xsd would validate the document against that xsd in order to make sure that the subsequent processing software can cope with its content - it is probably the strongest reason why to use xml formats at all.

To point to a specific one: In Europe, we have many "small" data centers providing data of one or a few of our small countries (often entire networks, but some networks are also distributed over multiple datacenters). In order to make the life of the scientists easier, EIDA develops a "federator", i.e. a software which offers an fdsnws at its frontend from where customers can request (meta)data e.g. of areas covering many countries. The federator transparently & "intelligently" redistributes this request to all national or regional data centers, collects and merges the responses, and forwards the joint result to the customer. In a service landscape where the national services can return on the same request /fdsnws/station/1/query?...&format=xml technically (on the level of validation) different formats, and the format of a data center can change anytime without prior note, the "federator" needs to analyze any response first to detect its format of the day, then fully parse !
the response in order to convert to one homogeneous format before merging and returning. Steps which are not required if the responses of all services of the same type and version are (technically) compatible.

Now you can say, with some good reason: let the developers of the "federator" add some money and time and cpu power, and solve that problem, so we don't need to deal with it at the FDSN.

But the world is more than IRIS plus Europe, and not all data centers are federated.
So in all other places where scientists use data from multiple sources, they have to do the same task and check again and again: Do all my data sources still provide the same format? If not: how do I convert/unify? What are the impacts on my subsequent processing?
My fear is that overall, for the entire seismological community, this is more of a hassle than the one of the data centers configuring a new service URL for a service version providing a new response format (a response technically not fulfilling the previous specification).

Kind regards, Philipp


----------------------
FDSN Working Group III

Topic home: http://www.fdsn.org/message-center/topic/fdsn-wg3-products/ | Unsubscribe: fdsn-wg3-produ...@lists.fdsn.org<mailto:fdsn-wg3-produ...@lists.fdsn.org>

Sent from the FDSN Message Center (http://www.fdsn.org/message-center/)
Update subscription preferences at http://www.fdsn.org/account/profile/


----------------------
FDSN Working Group III

Topic home: http://www.fdsn.org/message-center/topic/fdsn-wg3-products/ | Unsubscribe: fdsn-wg3-produ...@lists.fdsn.org<mailto:fdsn-wg3-produ...@lists.fdsn.org>

Philipp Kästli

unread,
Feb 4, 2020, 7:18:59 PM2/4/20
to fdsn-wg3...@fdsn.org
> The Station XML 1.0 schema appears to allow elements and attributes from any namespace.
Yes, any additional element or attribute from any namespace except its own.

> One option is to use the existing namespace where elements are unchanged, and use a different namespace for Station XML 1.x elements.
> This maintains backward compatibility for existing 1.0 parsers, while allowing 1.x aware parsers to read the additional information.

So, technically, all add-ons of current stationXML 1.1 compared to 1.0 could be implemented with such an extension to stationxml 1(.0) in a separate namespace. However, as far as I know, you cannot introduce restrictions to elements in the stationXML namespace from a different namespace.

Thus, the following stationXML 1.1 features / restrictions, could probably not be enforced with an extension in a different namespace:

* disallow "unit" attribute to <Numerator> and <Denominator>
* Allow <CreationDate> to be optional (change from required)
* Use xs:double for <ApproximationLowerBound>, <ApproximationUpperBound> and <MaximumError>
* Include data availability elements described in the fdsn-station+availability-1.0.xsd extension schema as optional elements of the main schema (they would stay in a different namespace)
* Remove <StorageFormat> from <Channel> (i.e. somebody can decide not to add the storage format, but an xml containing it remains valid)
* Limit each <Operator> to a single <Agency>
* Allow more than a single <Equipment> occurrence in <Channel>, same as in <Station>
* Do not require and disallow <StageGain> and <Decimation> for <Polynomial> response stages

So in my understanding, and referring to Chad's proposal #3, such an stationXML 1.2 with 1.1 features, but 1.0 compatibility is only partly possible.

Cheers, Philipp


________________________________
From: fdsn-wg3-pro...@fdsn.org<mailto:fdsn-wg3-pro...@fdsn.org> <fdsn-wg3-pro...@fdsn.org<mailto:fdsn-wg3-pro...@fdsn.org>> on behalf of Chad Trabant <ch...@iris.washington.edu<mailto:ch...@iris.washington.edu>>


Sent: Monday, February 3, 2020 16:57

To: FDSN Working Group III <fdsn-wg3...@lists.fdsn.org<mailto:fdsn-wg3...@lists.fdsn.org>>

Jonathan Schaeffer

unread,
Feb 6, 2020, 5:07:30 PM2/6/20
to fdsn-wg3...@fdsn.org

On 04/02/2020 00:57, Chad Trabant wrote:
>
> Dear WG 3 members,
>

> To move this issue forward I ask that *members share their opinions*
> *and suggested solutions* *by the end of February*. This will allow us


> to determine if quick action by the WG is needed.
>
> To summarize the issue as understand it: StationXML 1.1 documents that
> contain elements/attributes not defined in the 1.0 schema cannot be
> validated using the 1.0 schema as they (purposely) use the same
> namespace and 1.0 does not allow additions in the same namespace.  A
> conflict may occur, for example, when a client using an fdsnws-station
> 1.x service receives StationXML 1.1 that contains an <Identifier>
> element (new to 1.1) and tries to validate the response against the
> StationXML 1.0 schema.

Hi everyone,

sorry if my suggestion is actually stupid, but I'm not aware of all the
possible use-cases. So please don't hurt mie, I still wanted to suggest
this :

And what about taking another approach : let the datacenters be
responsible for the syntaxic validity of the metadata delivered instead
of the clients.
We (datacenters) only need to announce in the metadata wich version it
is (stationXML1.x), and the clients work with that. They don't need to
validate, they just have to handle the differences between the minor
versions *if they need to*.
Thus, no need for a new major version of stationXML service.

Does this solve anything ?

regards,
--
Jonathan Schaeffer
Observatoire des Sciences de l'Univers de Grenoble
Responsable Infrastructure RESIF-DC
+33 4 76 63 51 37 ou +33 4 76 63 52 48
🏢 Isterre, bureau 035
1681 rue de la Piscine
38400 Grenoble Cedex

Helle PEDERSEN

unread,
Feb 28, 2020, 12:27:15 AM2/28/20
to fdsn-wg3...@fdsn.org
cc: EIDA Management Board


Dear Chad, dear all,

Taking into account potential costs and delays, the 11 EIDA nodes
(ODC/KNMI, GFZ, RESIF, INGV, ETHZ, BGR, NIEP, KOERI, NOA, UIB, LMU) do
not support the immediate creation of new versions for either stationXML
or fdsnws-station. Our standpoint is motivated by concerns that any
additional format/specification changes will lead to undue delays that
will  compound current issues within EIDA where we have different
stationXML output across EIDA nodes.

Our plan is therefore to migrate all EIDA nodes to deliver stationXML1.1
as soon as possible.

Note that although all EIDA nodes intend to migrate fast to an internal
solution that has all nodes supporting a single version of stationXML,
the issues concerning ambiguous fdsnws specifications as well as other
fdsnws-station implementations providing different XML versions will
persist. In the long run, this should be addressed within future format
/ specification changes.

Please notify us if the FDSN needs individual statements from each node

Best regards

Helle Pedersen

Chair of the EIDA Management Board.


Le 04/02/2020 à 00:57, Chad Trabant a écrit :
>
> Dear WG 3 members,
>

> To move this issue forward I ask that *members share their opinions*

> *and suggested solutions* *by the end of February*. This will allow us

--
Helle Pedersen

ISTerre
Université Grenoble Alpes
CS 40700
38058 GRENOBLE Cedex 9
FRANCE
Tel +33 (0)4 76 63 52 59
Fax +33 (0)4 76 63 52 01

Chad Trabant

unread,
Apr 2, 2020, 7:48:47 PM4/2/20
to fdsn-wg3...@fdsn.org
Dear WG III members,

Thank you to those that contributed your thoughts on this issue. Based on responses, consensus among members is not to release a new specification for the fdsnws-station service and, instead to encourage the service providers to migrate their implementations to offer StationXML 1.1 as quickly as possible.

We also agree with the sentiment expressed by multiple members that future versions of the web service specifications and the StationXML format should be designed to avoid the ambiguity identified in this case moving forward.

regards,
Chad Trabant (WG III Chair)
Mark Chadwick (WG III Vice Chair)


> On Feb 3, 2020, at 4:00 PM, Chad Trabant <ch...@iris.washington.edu> wrote:
>
>
> Dear WG 3 members,
>
> To move this issue forward I ask that members share their opinions and suggested solutions by the end of February. This will allow us to determine if quick action by the WG is needed.
>
> To summarize the issue as understand it: StationXML 1.1 documents that contain elements/attributes not defined in the 1.0 schema cannot be validated using the 1.0 schema as they (purposely) use the same namespace and 1.0 does not allow additions in the same namespace. A conflict may occur, for example, when a client using an fdsnws-station 1.x service receives StationXML 1.1 that contains an <Identifier> element (new to 1.1) and tries to validate the response against the StationXML 1.0 schema.
>
> Note: StationXML 1.1 was designed to be readable and usable by clients expecting version 1.0 (ignoring the new elements/attributes), which works in practice. Validation of a 1.1 document using the 1.0 schema is the case that does not work.
>
> I see a couple of options to consider:
>
> 1) Define a new, major version 2, specification of the fdsnws-station service that would have a new endpoint. This would remove the ambiguity of which version of StationXML is returned (by default or via the format="xml") selection.
>
> 2) The FDSN could declare that fdsnws-station 1.x returns StationXML 1.x, and that any clients that wish to validate the responses must use the matching, or latest 1.x, schema specification. Note: the schema specification version is declared in the document itself, often with a link to the appropriate schema.
>
> 3) The next minor version of the StationXML schema (1.2) could be released with modifications that allow the documents to be validated with the 1.0 schema; then deprecate schema version 1.1 (not yet widely adopted) and move quickly to 1.2. I do not know of a technical solution for this, but it may be possible.
>
> If you know of other options please share them to be considered. If we determine that the issue is best solved in a StationXML schema change, it will be referred to WG 2.
>
> We should be mindful that future, minor releases of the StationXML and fdsnws-station specifications are likely. I recommend that we determine a clear path that allows for these releases with minimal disruption and invite suggestions for how to accomplish this.
>
> regards,
> Chad
>

>> On Dec 6, 2019, at 12:32 AM, Philipp Kästli @fdsn.org <kae...@sed.ethz.ch <mailto:kae...@sed.ethz.ch>> wrote:
>>
>> Dear Philip
>>
>>> Are you aware of a specific client and service/url combination that breaks because of this? My memory is
>>> that the reason this exception to the versioning rule was allowed was that there were no known instances > where the items changed were used, and so it was felt safe to allow it.
>>
>> I guess any decent client of a service providing a document in a format defined by an xsd would validate the document against that xsd in order to make sure that the subsequent processing software can cope with its content - it is probably the strongest reason why to use xml formats at all.
>>
>> To point to a specific one: In Europe, we have many "small" data centers providing data of one or a few of our small countries (often entire networks, but some networks are also distributed over multiple datacenters). In order to make the life of the scientists easier, EIDA develops a "federator", i.e. a software which offers an fdsnws at its frontend from where customers can request (meta)data e.g. of areas covering many countries. The federator transparently & "intelligently" redistributes this request to all national or regional data centers, collects and merges the responses, and forwards the joint result to the customer. In a service landscape where the national services can return on the same request /fdsnws/station/1/query?...&format=xml technically (on the level of validation) different formats, and the format of a data center can change anytime without prior note, the "federator" needs to analyze any response first to detect its format of the day, then fully parse !
>> the response in order to convert to one homogeneous format before merging and returning. Steps which are not required if the responses of all services of the same type and version are (technically) compatible.
>>
>> Now you can say, with some good reason: let the developers of the "federator" add some money and time and cpu power, and solve that problem, so we don't need to deal with it at the FDSN.
>>
>> But the world is more than IRIS plus Europe, and not all data centers are federated.
>> So in all other places where scientists use data from multiple sources, they have to do the same task and check again and again: Do all my data sources still provide the same format? If not: how do I convert/unify? What are the impacts on my subsequent processing?
>> My fear is that overall, for the entire seismological community, this is more of a hassle than the one of the data centers configuring a new service URL for a service version providing a new response format (a response technically not fulfilling the previous specification).
>>
>> Kind regards, Philipp
>>
>>
>> ----------------------
>> FDSN Working Group III

>> Topic home: http://www.fdsn.org/message-center/topic/fdsn-wg3-products/ <http://www.fdsn.org/message-center/topic/fdsn-wg3-products/> | Unsubscribe: fdsn-wg3-produ...@lists.fdsn.org <mailto:fdsn-wg3-produ...@lists.fdsn.org>
>>
>> Sent from the FDSN Message Center (http://www.fdsn.org/message-center/ <http://www.fdsn.org/message-center/>)
>> Update subscription preferences at http://www.fdsn.org/account/profile/ <http://www.fdsn.org/account/profile/>


>>
>>
>> ----------------------
>> FDSN Working Group III

>> Topic home: http://www.fdsn.org/message-center/topic/fdsn-wg3-products/ <http://www.fdsn.org/message-center/topic/fdsn-wg3-products/> | Unsubscribe: fdsn-wg3-produ...@lists.fdsn.org <mailto:fdsn-wg3-produ...@lists.fdsn.org>
>>
>> Sent from the FDSN Message Center (http://www.fdsn.org/message-center/ <http://www.fdsn.org/message-center/>)
>> Update subscription preferences at http://www.fdsn.org/account/profile/ <http://www.fdsn.org/account/profile/>
>

Reply all
Reply to author
Forward
0 new messages