Hi,
On 24.12.2013, at 15:21, Tobias Oberstein <
tobias.o...@gmail.com> wrote:
> thanks for your detailed comments. Lets fold that into WAMPv2 ..
more will follow - i just started ;-)
>> parameter. In the read/write it's more or less clear that I'd use the
>> argument for that [CALL, id, {}, "com.paroga.write", [],
>> {"path":"/folder/subfolder/file.ext", "content":"new_content"}], but for
>
> This is one option. What I find somewhat unsatisfying is that if you would use above topic URIs, the URIs address application level objects (resources = files/dirs). But with "com.paroga.write", the URI addresses an operation, and the resource the operation works on is within the CALL.Args.
>
> An alternative would be
>
> CALL.Procedure|uri == "com.paroga.filesystem.append.dir1.dir2.file1/ext" and
> CALL.Args|list = ["new_content"]
>
> Your filesystem implementation (the Callee that implements the append operation) would then probably need to do pattern-based REGISTERs:
>
> [REGISTER, Request|id, Options|dict, Procedure|uri]
>
> Procedure == "com.paroga.filesystem.append"
> Options == {"match": "prefix"}
>
> The actual invocation of the "append" endpoint would then need to provide the Procedure|uri
>
> [INVOCATION, Request|id, REGISTERED.Registration|id, Options|dict, CALL.Procedure|uri, CALL.Arguments|list, CALL.ArgumentsKw|dict]
>
> where CALL.Procedure == "com.paroga.filesystem.append.dir1.dir2.file1/ext"
>
> Your Callee implementation would then "cut out" everything that follows the prefix "com.paroga.filesystem.append":
>
> ".dir1.dir2.file1/ext"
>
> do the replacing '.' <=> '/':
>
> "/dir1/dir2/file1.ext"
>
> and perform the append with the supplied data in INVOCATION.Args to the respective file.
>
> I have added:
>
https://github.com/tavendo/WAMP/issues/28
>
> Please leave your comments there. My vote is +1 for this.
Ok, pattern based registration is for sure an important point, but I have to solve an other problem. I must be able to set two files at the _same time_. If one fails, the whole call fails. Encoding a list of files then into the topic gets really ugly, so I still prefer this information in the payload.
>> the subscribe there the Options need to be abused for that. So I suggest
>> adding arguments to the subscribe to allow [SUBSCRIBE, id, {},
>> "com.paroga.filechange", [], {"path":"/folder/subfolder/file.ext"}].
>
> If you subscribe to "com.paroga.filechange", the Broker will dispatch any file change event on any file to subscribers.
>
> Brokers are unaware of a "path" attribute, whether that is in SUBSCRIBE.Options or a new SUBSCRIBE.Args element.
>
> If you come up with a Broker that processes events based on that custom behavior, you won't be able to use any other WAMPv2 broker.
>
> In general, the idea is to specify core routing behavior for WAMPv2 so that Broker and Dealer implementations are interchangable.
Ok, so I need to add additional requirements. ;-) I must be able to add additional "how to subscribe" stuff into the SUBSCRIBE. E.g. a maximum update rate, or filter for a specific user. Encoding this into the topic isn't a clean solution for me. Extending the options with all possible stuff, so that the broker is able to handle it, wouldn't work with very specific stuff.
This use-case will only work without a generic broker, so every subscriber has a publisher (which is then the WebSocket server directly). Do you have any idea for a clean solution where we can skip the broker and have a 1:1 matching for subscribers to be able to handle very specific subscribe requirements without violating the specification?
>> This would also avoid the "x_" prefix workaround for the Options.
>> If you think about a very frequently changing file I see an unnecessary
>> overhead in sending the Topic all the time, since it's already clear
>> from the Subscription. Since it's required for prefix/wildcard
>> subscribe, I'd suggest to allow null or "" if it provides no additional
>> information.
>
> Yes, we could do that optimization: in case the Subscription or Registration is for "match" == "exact", transmit
>
> EVENT.Topic == ""
> INVOCATION.Procedure == ""
>
> In case the Subscription/Registration is using "match" != "exact", the fields would be filled.
>
> My vote is +1 for this optimization (since it is also for a probably very frequent case):
>
https://github.com/tavendo/WAMP/issues/29
Cool!
>> I somehow miss the option for PREFIX and suggest an ALIAS and maybe
>> UNALIAS procedure as an replacement. I creates a simple mapping between
>> the full URI and an usually shorter alias. This alias will then be valid
>> everywhere for the type uri. This can also help for the previous point,
>> since the server would be able to replace the long Topic in the EVENT
>> message with the shorter alias.
>
> There are multiple problems with PREFIX, one of which is: PREFIX in WAMPv1 is per session. So a Broker cannot serialize an event once and dispatch to all receivers. In fact, Autobahn will not use CURIEs in EVENTs at all - even for WAMPv1.
>
> There has been a longer discussion here:
>
https://github.com/tavendo/WAMP/issues/8
>
> I don't think PREFIX or ALIAS brings any value, but introduces multiple problems.
>
> If you worry about wire level traffic volume: WebSocket has Deflate compression. And this will compress not only URIs in WAMP messages, but also URIs (and strings in general) within application payloads.
It's not only about the traffic, it's also about the CPU performance. Believe it or not, but I had an setup where the string lookup was an important factor to performance: Imaging the filesystem server from above, which sends an high amount of com.paroga.filechange events. The client needs to lookup the "client dispatch function" in a map (e.g. {"com.paroga.filechange":function(){}, "com.paroga.otherstuff":function(){}}). The length of the string made a noticeable difference.
If you now suggest to encode the path in the topic too, then the client dispatching gets even much more complicated.
>> an META(UN)SUBSCRIBE instead? Or if it's only because of the list: What
>
> My thinking is this: the subscription to a topic and associated metatopics should happen in one request, and result in one subscription.
>
> Say I have subscribed to "com.foo.bar" with "match" == "prefix".
>
> How would I subscribe to metaevents for that subscription after the initial "normal" subscription is already in place. That would need to provide the SUBSCRIBED.Subscription|id. So it would need a new message type.
>
> But I am open to have a discussion on METAEVENTs in general. We don't have practical experience .. and people might have different expectations. Up to now, I was only thinking of the
>
> wamp.metatopic.subscriber.add
> wamp.metatopic.subscriber.remove
>
> metaevents. In fact, I can't come up with other useful "metaevents". What do you think? What should be cover?
I think I understand now why you put them there: They will only create metaevents when "com.foo.bar" has been subscribed by some other peer. My initial understanding was that when I subscribe to the add metatopic I'll get metaevent when a peer subscribes to "com.foo" and "com.bar" and the "normal" topic is useless in the SUBSCRIBE message. Maybe that should be made more clear in the spec...
>> about a MULTI(UN)SUBSCRIBE, which is exactly like SUBSCRIBE, but allows
>> a list for Topic.
>
> But what's the advantage?
With your idea of metaevents: None :-)
> Also: How would errors be handled? Say I multisubscribe 3 topics, and I am not authorized for 1 of the 3.
Is subscribing to an metaevent always possible? Any security concerns about exposing this information?
>> Do you think PUBLISH.Options.rkey|string and PUBLISH.Options.nkey|string
>> really belong into this spec? They seam quite specialized to me.
>
> They are part of the call/event routing flexibility that WAMP provides - so yes, I think they are core.
Can you describe your use-case a little bit more? Routing flexibility is for sure something I agree with, but I didn't get the connection between EVENT and CALL. "nkey" is only mentioned for the EVENT message. What's the exact difference between a peer, node and resource?
>> The names of the "good return" messages don't look very consistent to
>> me. Some end to "ED" and some to "_RESULT". What about making all end to
>> "_OK"? ("CALL_OK", "SUBSCRIBE_OK"). I'd also reorder some ids, so that
>> the "good return" is always the id of the request incremented by one and
>> the "bad return" incremented by two (CALL=70, CALL_OK=71,CALL_ERROR=72,
>> CALL_PROGRESS=73, CANCEL_CALL=74)
>
> If that systematics would make it easier for people to grasp the message flows, then yes. Any opinions by others? I am mostly indifferent on this one.
I don't think that this is a big issue and have no strong opinion on it. Just wanted to mention it.
>> Is there a good reason to use 2^53 instead of 2^32 for the upper bound
>> of id? If the valid range would be 0 to 2^32-1 instead, CPUs could use
>> simple registers to store the id.
>
> Modern CPUs have 64 bit register banks. The upper bound is carefully chosen to be representable in 64 bit integers and IEEE doubles, and at the same time allow to make the probability of collisions practically zero.
As you might be able to imagine already I come from the embedded world, where 32bit is already a large register. For modern desktop PCs it's for sure nothing to think about. Do you think this additional 21bit will really reduce the probability of collision for real world applications? IMHO 32bit should be enough. If you create an event every millisecond, your application needs to run more than 3 years to reuse an id (if they are dumb incrementing numbers).
>> Additionally I'd merge "Arguments|list, ArgumentsKw|dict" to
>> "Arguments|any", because this looks very python-ish to me and WAMP is
>> not python only. Providing a documentation for the WAMP->Python mapping
>> would be better IMHO.
>
> No, please see here:
https://github.com/tavendo/WAMP/issues/21
>
> In fact, WAMPv2 allows positional and keyword based call results, which isn't even in Python, but for example in PostgreSQL and Oracle.
>
> And I was first wary of this, but beatgammit convinced me that we should take the "superset of all" approach in WAMPv2. With WAMPv1, we tool the "smallest common denominator" (positional args, 1 result) approach.
>
> But, regarding arguments (and in general, all application payloads), there is 1 major outstanding question:
>
> All application payloads are now at the end within the lists forming WAMP messages. The idea is, that Dealers and Brokers actually don't have any need to parse or touch those app payloads, and might skip parsing altogether.
>
> Taking this idea further, I was thinking of making all app payloads just strings (JSON) or binaries (MsgPack). So Dealers/Brokers can be even more efficient and _probably_ we could even allow to transmit end-to-end encrypted payloads.
>
> Only the Subscriber/Publisher and Caller/Callee needs to perform a 2nd parse on the WAMP level payload to get at the actual payload.
>
> I would highly appreciate any input on this!
At which level do you see WAMP? Do you want to specify how functions have to look like in the protocol, or isn't it something for the application which builds on WAMP? If you go the first way then the spec should include the mapping for all programming languages, which might be out of scope.
Is there an advantage of having a defined number of elements in an message? If not, we can use
[CALL, Request|id, Options|dict, Procedure|uri, ...]
[CALL_RESULT, CALL.Request|id, ...]
with an variable length payload. This would reduce the size for simple calls and allow any combination for the implementation of applications. For the other case I'd suggest
[CALL, Request|id, Options|dict, Procedure|uri, Arguments|any]
[CALL_RESULT, CALL.Request|id, INVOCATION_RESULT.Result|any]
where the application has to encode everything into the any object/list/string.
What about allowing an optional payload (maybe with a maximum count?) for all messages? E.g.:
[SUBSCRIBE, Request|id, Options|dict, Topic|uri, ...]
[PUBLISH, Request|id, Options|dict, Topic|uri, ...]
[EVENT, SUBSCRIBED.Subscription|id, PUBLISHED.Publication|id, Details|dict, PUBLISH.Topic|uri, PUBLISH....]
Please don't remove the possibility for objects in the payload, since encoding everything as a string doesn't seam like a clean solution for me. If some application wants to do that an payload type of any would allow that anyway.
>> Do you accept pull request for changes in the spec?
>
> Yes. It would be good to have small PRs though (eg branch the spec per issue/feature you would like to get merged). Git doesn't cope well with cherry picking ..
I'll try my best. :-)
Two additional points I forgot in my last mail:
1) Can we introduce the requirement that all clients get messages in the exactly same order, when they sent the same messages to the broker:
E.g:
C->S: [SUBSCRIBE, 1, {}, "topic1"]
C->S: [SUBSCRIBE, 2, {}, "topic2"]
S->C: [SUBSCRIBED, 1]
S->C: [SUBSCRIBED, 2]
S->C: [EVENT, ..., "topic1", "A"]
S->C: [EVENT, ..., "topic2", null]
S->C: [EVENT, ..., "topic1", "B"]
can be also;
C->S: [SUBSCRIBE, 1, {}, "topic1"]
S->C: [SUBSCRIBED, 1]
C->S: [SUBSCRIBE, 2, {}, "topic2"]
S->C: [SUBSCRIBED, 2]
S->C: [EVENT, ..., "topic1", "A"]
S->C: [EVENT, ..., "topic2", null]
S->C: [EVENT, ..., "topic1", "B"]
but not:
C->S: [SUBSCRIBE, 1, {}, "topic1"]
C->S: [SUBSCRIBE, 2, {}, "topic2"]
S->C: [SUBSCRIBED, 2] <== Second subscribe returned first
S->C: [SUBSCRIBED, 1] <==
S->C: [EVENT, ..., "topic1", "A"]
S->C: [EVENT, ..., "topic2", null]
S->C: [EVENT, ..., "topic1", "B"]
and not:
C->S: [SUBSCRIBE, 1, {}, "topic1"]
C->S: [SUBSCRIBE, 2, {}, "topic2"]
S->C: [SUBSCRIBED, 1]
S->C: [SUBSCRIBED, 2]
S->C: [EVENT, ..., "topic1", "A"]
S->C: [EVENT, ..., "topic1", "B"] <== Second "topic1" before "topic2"
S->C: [EVENT, ..., "topic2", null] <==
2) Did you ever thought about high availability for WAMP? A client might want to connect to two broker at the same time to make sure it can get data even if one of them dies. I think about something like the following:
* Client connects to first broker B1 and does HELO
* Client connects to second broker B2 and does HELO
* Client tells B1 to "unify" with Session from B2 HELLO as active part
* Client tells B2 to "unify" with Session from B1 HELLO as passive part
* B1 und B2 are allowed to "unify" since they got matching Session ids from each other
* Client subscribes to "topic1" at B1 (since he is the active part)
* B1 tells B2 that "topic1" has been subscribed
* someone publishes "topic1"
* B1 sends "topic1" to client (since he is the active part)
* B2 ignores "topic1" (since he is the passive part)
* B1 dies
* Client recognizes dead B1
* Client switches B2 from passive to active
* someone publishes "topic1"
* B2 sends "topic1" to client (since he is the active part now)
Is that something you might consider to be specified directly at WAMP or is it overly application specific?
> Thanks!! And Merry Christmas;)
Thanks for the good basis to build upon. And Merry Christmas back, but maybe already too late. ;-)
-- Patrick