[Standards] disco#info Results Ordering
Mike Wacker
ordered like this:
1. identities
2. features
3. extensions
However, is that specific order required, or should I expect the info to
come in any order? Here's an example disco#info result I've seen from an
ejabbered server, where the extensions are instead between the
identities and features:
<query xmlns='http://jabber.org/protocol/disco#info'><identity
category='pubsub' type='pep'/><identity category='server' type='im'
name='ejabberd'/><x xmlns='jabber:x:data'
type='result'>...[extensions]...</x><feature
var='http://jabber.org/protocol/commands'/><feature
var='http://jabber.org/protocol/disco#info'/>...[more features]...</query>
The schema in XEP-0030 would at least imply an order for identities and
features, but it has always been the case that those schema are
descriptive, not normative.
Ralph Meijer
> In all the examples I've seen in XEPs, disco#info results have been
> ordered like this:
>
> The schema in XEP-0030 would at least imply an order for identities
> and features, but it has always been the case that those schema are
> descriptive, not normative.
I believe this is a mistake in the schema. It should probably use xs:all
instead. In general we try to prevent dependence on order if at all
possible.
--
ralphm
Kim Alvefur
> However, is that specific order required, or should I expect the info
> to come in any order?
I think the answer to this is
> Be liberal in what you accept, and conservative in what you send.
Unless the schema says that it has to be in a specific order.
--
Kim Alvefur <za...@zash.se>
Mike Wacker
So the main takeaway, if I understand this correctly, would be that if
even the schema implies an order, I shouldn't assume any particular
order unless the RFC or XEP explicitly calls it out.
As another example, for RFC 6120, the client namespace in A.5 implies
that while the "normal" content (<show>, <status>, and <priority>) can
come in any order for presence stanzas, all extended content must come
after all "normal" content. But unless RFC 6120 specifically calls out
that ordering constraint (and it doesn't as far as I can tell), the
server should be able to handle, for example:
<show>xa</show><c
xmlns='http://jabber.org/protocol/caps'...><status>Vacation!</status>
Mike
Ralph Meijer
> [..]
>
> So the main takeaway, if I understand this correctly, would be that if
> even the schema implies an order, I shouldn't assume any particular
> order unless the RFC or XEP explicitly calls it out.
Yeah, that sounds right. Expecting stuff in a particular order makes
processing unreasonably more difficult, especially since there is the
expectation that you can ignore everything you don't understand.
> As another example, for RFC 6120, the client namespace in A.5 implies
> that while the "normal" content (<show>, <status>, and <priority>) can
> come in any order for presence stanzas, all extended content must come
> after all "normal" content. But unless RFC 6120 specifically calls out
> that ordering constraint (and it doesn't as far as I can tell), the
> server should be able to handle, for example:
>
> <show>xa</show><c
> xmlns='http://jabber.org/protocol/caps'...><status>Vacation!</status>
Indeed, those elements could be in any order.
From the top of my head, the only specification that I know to rely on
a specific order is Publish Subscribe (XEP-0060), for the
create/configure and subscribe/options combination. This is written out
explicitly in the prose. I didn't enjoy implementing that in Wokkel. In
retrospect it would have been so much easier to have those data forms be
a child element of the create and subscribe elements respectively.
--
ralphm
Matthew A. Miller
I think XEP-0004 also says something about the order of elements (as in, maintain the order of <field/>s and <item/>s as they were received).
I'd like to point out that all of our XML Schemas are non-normative. They're provided for informational use, and ought not be considered the absolute record of authority.
- m&m
<http://goo.gl/LK55L>
Dave Cridland
> I'd like to point out that all of our XML Schemas are
> non-normative. They're provided for informational use, and ought
> not be considered the absolute record of authority.
What follows is my understanding; we should probably have this
documented somewhere (a Tao Of XSF XEP?):
- The schemas in XEPs are not normative.
- We do, however, try to keep them aligned properly with the text,
and will accept bug reports with gratitude.
- The schemas in RFCs *are* normative.
- The IETF does, however, accept errata should they not match the
text or the intent.
So in both cases, we'd expect the schemas to be right, and welcome
fixes; technically, though, there's a distinction in normativeness
(normativity?) between RFC and XEP.
Dave.
--
Dave Cridland - mailto:da...@cridland.net - xmpp:d...@dave.cridland.net
- acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
- http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade
Peter Saint-Andre
> On Thu Dec 8 23:13:38 2011, Matthew A. Miller wrote:
>> I'd like to point out that all of our XML Schemas are non-normative.
>> They're provided for informational use, and ought not be considered
>> the absolute record of authority.
>
> What follows is my understanding; we should probably have this
> documented somewhere (a Tao Of XSF XEP?):
>
> - The schemas in XEPs are not normative.
> - We do, however, try to keep them aligned properly with the text, and
> will accept bug reports with gratitude.
> - The schemas in RFCs *are* normative.
> - The IETF does, however, accept errata should they not match the text
> or the intent.
>
> So in both cases, we'd expect the schemas to be right, and welcome
> fixes; technically, though, there's a distinction in normativeness
> (normativity?) between RFC and XEP.
RFC 6120 says:
The following schemas formally define various namespaces used in this
document, in conformance with [XML‑SCHEMA]. Because validation of XML
streams and stanzas is optional, these schemas are not normative and are
provided for descriptive purposes only.
/psa
Dave Cridland
I sit corrected - I'd assumed that the normal status quo of the IETF
held with those.
Peter Saint-Andre
I don't think there is a normal status quo at the IETF regarding the
normativity of schemas.
Peter
--
Peter Saint-Andre
https://stpeter.im/
Hannes Tschofenig
Over the time I have gotten the impression that an XML schema is really a waste of time. It creates the illusion that there is something that provides help (to implementers and to those who read the specification) but in reality it doesn't.
Working on different specifications I later thought that the problem is with the readability and extensibility of the XML schema and then we switched to Relax NG in some IETF working groups. That turned to be a mistake as well. When it comes to extensibility a Relax NG schema is equally bad.
The extensibility mechanism of XML would prevent you from getting any meaningful validation anyway. So, validation isn't useful because more or less everything validates (after you add the extension points everywhere).
So, I believe we are doing fine without XML schema but with lots of examples. Implementers just look at examples.
Maybe you could therefore recommend not to use XML schemas (or Relax NG schemas).
Ciao
Hannes
Dave Cridland
> Over the time I have gotten the impression that an XML schema is
> really a waste of time. It creates the illusion that there is
> something that provides help (to implementers and to those who read
> the specification) but in reality it doesn't.
>
> Working on different specifications I later thought that the
> problem is with the readability and extensibility of the XML schema
> and then we switched to Relax NG in some IETF working groups. That
> turned to be a mistake as well. When it comes to extensibility a
> Relax NG schema is equally bad.
>
> The extensibility mechanism of XML would prevent you from getting
> any meaningful validation anyway. So, validation isn't useful
> because more or less everything validates (after you add the
> extension points everywhere).
>
> So, I believe we are doing fine without XML schema but with lots of
> examples. Implementers just look at examples.
>
> Maybe you could therefore recommend not to use XML schemas (or
> Relax NG schemas).
Or at least move them out of the XEP itself, perhaps?
I think we vaguely require them, at present. I'd be happy with
hosting them out of the XEP itself, which'd make them more obviously
informative.
The XSF Board chair and the XMPP Council chair have been trying to
figure out how we go about such a decision, and decided the best
thing to do was seek consensus on the lists as a first step.
Peter Saint-Andre
In my experience, writing the schema forces you to think through the XML
design, and that alone is a good thing. We do host the schemas
separately, but they are included in the spec as well:
http://xmpp.org/resources/schemas/
Another option: migrate to JSON. XML is *so* second-millennium!
Bala Pitchandi
Without a schema, all we have is the text and examples which could be interpreted in many ways by how they are written and how they are understood by the reader.
Requiring a schema also forces the XEP authors to think hard and come up with a design that's structured, extensible (Yes, XSD schemas can be made extensible).
-- Bala
Peter Saint-Andre
> I strongly disagree that XSD (RelaxNG or any other way of specifying
> the structure of the XML messages) is a "waste of time". If you don't
> have a schema, you'll either end up implementing all the validation
> manually or worse crash on invalid input. XSD parser-validators (they
> exist) do all that work for you, and are optimized to do that.
>
> Without a schema, all we have is the text and examples which could be
> interpreted in many ways by how they are written and how they are
> understood by the reader.
>
> Requiring a schema also forces the XEP authors to think hard and come
> up with a design that's structured, extensible (Yes, XSD schemas can
> be made extensible).
No argument here.
In addition, I know of at least one large deployment community that
performs active schema-checking for XMPP traffic. Those folks would like
it if our schemas were normative, I'd bet. :)
Kevin Smith
> I strongly disagree that XSD (RelaxNG or any other way of specifying the structure of the XML messages) is a "waste of time". If you don't have a schema, you'll either end up implementing all the validation manually or worse crash on invalid input. XSD parser-validators (they exist) do all that work for you, and are optimized to do that.
While this may be true, the schemas are not normative, and so can't be
reliably used in this way.
> Without a schema, all we have is the text and examples which could be interpreted in many ways by how they are written and how they are understood by the reader.
Yes - but these are what's normative, not the schemas.
> Requiring a schema also forces the XEP authors to think hard and come up with a design that's structured, extensible (Yes, XSD schemas can be made extensible).
I'll ponder this argument. I'm not entirely convinced one way or the
other at the moment.
/K
Mike Wacker
Since I seemed to have re-opened this rather large can of worms, I'll
offer my own thoughts. I have actually found the schemas to be useful,
even though I understand their limitations. You just need to put their
value in the proper context.
And even though they are not normative, there is still some sort of
normative constraint for the XML, and to some extent the schemas do
capture that. Here's an example from XEP-0045, 6.4 Discovering Room Items:
"These <item/> elements are qualified by the disco#items namespace, not
the muc namespace; this means that they cannot possess 'affiliation' or
'role' attributes, for example."
One direction we could go with the schemas is to build schemas that will
not capture every normative constraint, but will catch most violations.
Not all invalid XML would be rejected by the schema, but valid XML would
never be rejected by the schema.
This would assign some normative value to the schemas and help us from
both a standards and interopability perspective. However, there will
always be some rules that schemas cannot cleanly capture, and for those
people will still need to read the RFCs and XEPs. (However, the one
major obstacle to that approach would be extension attributes [see RFC
6120, 8.4. Extended Content], though I don't know if there are any XEPs
which actually use those.)
Peter Saint-Andre
Yes, that all seems reasonable.
> However, there will
> always be some rules that schemas cannot cleanly capture, and for those
> people will still need to read the RFCs and XEPs. (However, the one
> major obstacle to that approach would be extension attributes [see RFC
> 6120, 8.4. Extended Content], though I don't know if there are any XEPs
> which actually use those.)
So far we have avoided those almost entirely, although I started to
consider them again when thinking about XEP-0300 recently.
Adam Nemeth
Before the anarchy ensues, I'd like to propose the following plan:
1) define every XEP in a suite of (unit) tests
- unit tests should be written in a way that equals to formal checking(!!!)
- the test framework can be anything, but it's likely that a custom test language would be needed
- optional parts should be defined as "config" variables for XEP test suites on a per XEP basis (as, MAY (default false), SHOULD (default true), etc)
- whatever passes the test is said to be conform to the given XEP
2) Find sponsors to finance building this testsuite and find test developers who will actually do it
- vendor-independency should be ensured
- fosdem summit is a perfect time to actually start buildi the test suite
- it's a lot of work, people won't volunteer to do it properly enough (as it never happened here)
3) Ask major XSF-related services to collect statistics on which XEPs are used most of the time
- Ask major server vendors to build this possibility in a way that doesn't disrupt network traffic and is privacy-compliant
- Ask service providers to enable this on their most used services
- Implement these XEP test suites first
Perhaps the world would be a better place if standards wouldn't be about "do as you wish and pray that the global network doesn't get even more fractured because of that", but they would be _standards_ which are actually explicitly enforced. While it's great to have liberal views, when it comes to a standard, I want conformance to be a question of Yes or No, not "well, sort of".
Udv:
Aadaam
Bala Pitchandi
I can't agree more with this:
"Perhaps the world would be a better place if standards wouldn't be about "do as you wish and pray that the global network doesn't get even more fractured because of that", but they would be _standards_ which are actually explicitly enforced. While it's great to have liberal views, when it comes to a standard, I want conformance to be a question of Yes or No, not "well, sort of"."
Example of a standard in disarray: SDP, for the stated reason.
-- Bala
-----Original Message-----
From: standard...@xmpp.org [mailto:standard...@xmpp.org] On Behalf Of Adam Nemeth
Sent: Friday, December 09, 2011 1:37 PM
To: XMPP Standards
Hannes Tschofenig
Here is my view:
1) the schema never expresses all constraints since it is not powerful enough to describe them. For simpler tasks it is not useful to describe all constraints in a schema since otherwise the schema becomes unreadable.
2) nobody! validates instance documents (in a protocol) when receiving messages against the schema (since you have to do the input validation anyway).
2) when you extend the schema, which is a core concept in protocol design to design them with extension points, then in almost all of the cases I have seen validation does not work anymore because the extended schema is not validated, i.e. it verifies as correct.
Even worse, since most people are not super skilled with writing XML schemas their protocol design is essentially guided with what they are able to express in the XML schema.
Ciao
Hannes
Hannes Tschofenig
And, explain what you mean by schema checking?
Ciao
Hannes
Peter Saint-Andre
error (or perhaps quarantine it somehow). These are military deployments
in which they don't like to have rogue XML sent around.
Bala Pitchandi
On 12/10/11 8:55 AM, "Hannes Tschofenig" <hannes.t...@gmx.net> wrote:
>Hi Bala,
>
>Here is my view:
>
>1) the schema never expresses all constraints since it is not powerful
>enough to describe them. For simpler tasks it is not useful to describe
>all constraints in a schema since otherwise the schema becomes unreadable.
[BP] Please point to an example constraint a schema can't express. I know
of a few but those are extremely rare. Schemas don't have to be "human
readable". Schemas are meant to be read by machines and processed by
machines. And machines are pretty good at processing "complex" algorithms.
>
>2) nobody! validates instance documents (in a protocol) when receiving
>messages against the schema (since you have to do the input validation
>anyway).
[BP] That's too general of a statement. I know of a few implementations
(including our own) that do very strict validation of incoming XML
messages against their schemas. Otherwise, you are setting yourselves up
against rogue attacks/crashes.
>
>
>2) when you extend the schema, which is a core concept in protocol design
>to design them with extension points, then in almost all of the cases I
>have seen validation does not work anymore because the extended schema is
>not validated, i.e. it verifies as correct.
[BP] When you extend schemas, the "old" deployments by definition cannot
validate the new elements/attributes but the "new" deployments (that were
written after the schema extensions) do validate against the schema.
>
>
>Even worse, since most people are not super skilled with writing XML
>schemas their protocol design is essentially guided with what they are
>able to express in the XML schema.
[BP] All of the XEPs are peer-reviewed and hopefully even if the authors
are not "XML/Schema experts" (It isn't that hard to learn it) they can get
help from others in the community for writing the schemas. I'd gladly
volunteer.