Post sanity checking in buddycloud-server

[Denis Washington]

Hi,

I am currently in the process of implementing POST on channel nodes in
the HTTP API server, and I was wondering how much sanity checking I'll
have to do before I send an Atom entry to the buddycloud server.

- Does the buddycloud server check if the <author> tag actually matches
the JID of the publisher? If not, should it? Or does the server even
fill in the <author> itself if none is provided?

- What about the <published> date? Could an XMPP client write anything
in there, or does the server fill in / replace <published> tags with its
own versions? (It really should, because we cannot trust the timestamps
of the bc clients and server, or even the HTTP and bc server, to be in
sync.)

- Given that the post-to-channel example on the "XMPP XEP" wiki page [1]
doesn't contain an <id>, I guess this is filled in to match the node
item ID?

If you agree that they are a good idea, I could hack on the buddycloud
server code to add them if they are not there yet. This would be a much
better solution than to put this into the HTTP server (and, thus, still
allow clients to circumvent them by simply using XMPP).

Regards,
Denis

[1] https://buddycloud.org/wiki/XMPP_XEP#Post_to_a_channel
[1]

[Tuomas Koski]

Hi,

(warning, I don't know the node channel server so just guessing here.)

On 1 June 2012 10:00, Denis Washington <den...@online.de> wrote:
> Hi,
>
> I am currently in the process of implementing POST on channel nodes in the
> HTTP API server, and I was wondering how much sanity checking I'll have to
> do before I send an Atom entry to the buddycloud server.
>
> - Does the buddycloud server check if the <author> tag actually matches the
> JID of the publisher? If not, should it? Or does the server even fill in the
> <author> itself if none is provided?

I hope it does :-) It has to be the job of the "inbox". The
experimental java server does this check.
And I think the server should fill the author if it's missing.

On 1 June 2012 10:00, Denis Washington <den...@online.de> wrote:
> - What about the <published> date? Could an XMPP client write anything in
> there, or does the server fill in / replace <published> tags with its own
> versions? (It really should, because we cannot trust the timestamps of the
> bc clients and server, or even the HTTP and bc server, to be in sync.)
>
> - Given that the post-to-channel example on the "XMPP XEP" wiki page [1]
> doesn't contain an <id>, I guess this is filled in to match the node item
> ID?

ID is mandatory element of valid atom, right?

Again, in the experimental java server the ID was filled by the server to match
the correct ID from the servers data system.

On 1 June 2012 10:00, Denis Washington <den...@online.de> wrote:
> If you agree that they are a good idea, I could hack on the buddycloud
> server code to add them if they are not there yet. This would be a much
> better solution than to put this into the HTTP server (and, thus, still
> allow clients to circumvent them by simply using XMPP).

+1 from me to check that the validation is done correctly on the "node
channel server".
+1 also to implement any new features needed on the node server too. :-)

Cheers,
--
Tuomas

[Thomas Jost]

Le 1 juin 2012 à 10:36 CEST, Tuomas Koski a écrit :
> Hi,
>
> (warning, I don't know the node channel server so just guessing here.)
>
> On 1 June 2012 10:00, Denis Washington <den...@online.de> wrote:
>> Hi,
>>
>> I am currently in the process of implementing POST on channel nodes in the
>> HTTP API server, and I was wondering how much sanity checking I'll have to
>> do before I send an Atom entry to the buddycloud server.
>>
>> - Does the buddycloud server check if the <author> tag actually matches the
>> JID of the publisher? If not, should it? Or does the server even fill in the
>> <author> itself if none is provided?
>
> I hope it does :-) It has to be the job of the "inbox". The
> experimental java server does this check.
> And I think the server should fill the author if it's missing.

I don't know if the server checks that the author matches the JID, but I
doubt it: there are posts without an author in Lloyd's channel (and some
older ones in dodo's channel).

Actually I had to add checks for that in bccc (see Atom.author() in
bccc/client/atom.py): in Lloyd's channel there are posts with no
<author> at all, in dodo's there are some with empty <author></author>.

Besides, adding validity checks on the server would be tricky: if a.com
reject invalid Atoms from cli...@a.com, what should it do with invalid
Atoms coming from another server?

> On 1 June 2012 10:00, Denis Washington <den...@online.de> wrote:
>> - What about the <published> date? Could an XMPP client write anything in
>> there, or does the server fill in / replace <published> tags with its own
>> versions? (It really should, because we cannot trust the timestamps of the
>> bc clients and server, or even the HTTP and bc server, to be in sync.)
>>
>> - Given that the post-to-channel example on the "XMPP XEP" wiki page [1]
>> doesn't contain an <id>, I guess this is filled in to match the node item
>> ID?
>
> ID is mandatory element of valid atom, right?
>
> Again, in the experimental java server the ID was filled by the server to match
> the correct ID from the servers data system.

Same in the Node server: bccc doesn't add any ID before publishing to a
channel, it's done on the server.

> On 1 June 2012 10:00, Denis Washington <den...@online.de> wrote:
>> If you agree that they are a good idea, I could hack on the buddycloud
>> server code to add them if they are not there yet. This would be a much
>> better solution than to put this into the HTTP server (and, thus, still
>> allow clients to circumvent them by simply using XMPP).
>
> +1 from me to check that the validation is done correctly on the "node
> channel server".
> +1 also to implement any new features needed on the node server too. :-)
>
> Cheers,
> --
> Tuomas

--
Thomas/Schnouki

[Denis Washington]

Am 01.06.2012 10:54, schrieb Thomas Jost:
> Besides, adding validity checks on the server would be tricky: if a.com
> reject invalid Atoms from cli...@a.com, what should it do with invalid
> Atoms coming from another server?

Why should the domain matter? The buddycloud server would just return an
error stanza if the sent Atom entry is not valid, regardless of where
the client comes from. What do you mean exactly?

Denis

[Denis Washington]

Am 01.06.2012 11:40, schrieb Thomas Jost:

> Sorry if I wasn't clear. Let's consider 2 buddycloud instances, a.com
> running the Node server and b.org running some other server.
>
> Imagine the Node server does strict Atom validation. When j...@a.com
> tries to publish an invalid Atom on his channel, it is rejected by the
> server.
>
> If the server on b.org does not do the same, bad...@b.org *can* publish
> invalid Atoms on his channel. But if j...@a.com follows bad...@b.org's
> channel, b.org will forward this invalid Atom to a.com.
>
> What should happen then? Should the server on a.com accept or reject
> this Atom? How should the robustness principle ("be conservative in what
> you send, liberal in what you accept") be applied in such a case?

Which validations / fill-ins are done should be part of the buddycloud
protocol spec. So if b.org accepts posts that are invalid by the words
of the spec, it is simply no compliant buddycloud server. I don't think
that a.com's server should bend the rules here - the guarantee that all
information in the posts are well-formed, correct and verified is far
more important than fussing with buggy server implementations.

Denis

[Thomas Jost]

Le 1 juin 2012 à 11:52 CEST, Denis Washington a écrit :
>> What should happen then? Should the server on a.com accept or reject
>> this Atom? How should the robustness principle ("be conservative in what
>> you send, liberal in what you accept") be applied in such a case?
>
> Which validations / fill-ins are done should be part of the buddycloud
> protocol spec. So if b.org accepts posts that are invalid by the words
> of the spec, it is simply no compliant buddycloud server.

I fully agree :)

(There's already something about content normalization in the protocol:
https://buddycloud.org/wiki/XMPP_XEP#Content_normalization)

> I don't think that a.com's server should bend the rules here - the
> guarantee that all information in the posts are well-formed, correct
> and verified is far more important than fussing with buggy server
> implementations.

Again, I agree. But what worries me is that if b.org can communicate
with c.net and d.info (because they use the same buggy server) but not
with a.com, in the end users will think that a.com is wrong.

Does anyone here know how XMPP servers deal with these kind of problems?

--
Thomas/Schnouki

[Kevin Smith]

Illegal XML is rejected at the first hop (and every hop). Other than
that it's the processor's responsibility to validate input.

e.g. your server has no responsibility to check that forms you submit
to a remote MUC room's config match the form that the MUC sent you to
fill out - only that they're legal XML (and in some cases that they're
also valid XML).

/K

[Astro]

Denis Washington wrote:
> I am currently in the process of implementing POST on channel nodes
> in the HTTP API server, and I was wondering how much sanity checking
> I'll have to do before I send an Atom entry to the buddycloud server.

Yes to all your questions:

* https://buddycloud.org/wiki/XMPP_XEP#Content_normalization
* https://github.com/buddycloud/buddycloud-server/blob/master/src/normalize.coffee#L36

Note that all this applies just to ATOM items, not Geo Location or
anything with another xmlns.


Astro

[Denis Washington]

"Yes" means these things are merely checked? Or are they also filled in
if not provided?

Denis