On Performance Evaluation and Scalability of OT and OCW
28 views
Skip to first unread message
nvd_ai
Jul 20, 2011, 5:06:19 AM7/20/11
to opencoweb
Hi,
I am wondering what happens if clients make a lot of sendSync()
calls?
By 'a lot' I mean something in the neighborhood of [1000,10000] calls
per second!! I know this seems a bit crazy or maybe expecting too much
but I think the question worth exploring at least for the sake of
improving OCW performance.
To make the question simpler, let's think only one client makes this
huge number of sendSync calls and all the other clients are listeners.
If you are interested in a real-world application, think about a real-
time gaming application in which one client is playing the role of the
server, executing the game logic and broadcasting all the changes of
game objects to other clients. Is not this interesting?
Searching for an answer, I did some little experience, having a client
generate 5000 operations in a period of 5 seconds (1000 operations per
second) and I tracked the network POSTs sent to cometd and their
latency. I ran the test on my localhost, ignoring the real-world
latency for the sake of simplicity for the moment. I ran the test few
times to get a sense of mean and variance.
observations:
1- OCW generates one post for every single sendSync call.
2- Based on what the browser (chrome) network monitoring shows, each
post takes somewhere between 2ms and 7ms to be returned from
localhost. So, 5000*2= 10s should be the lowest amount of time and
5000*7=35s should be the highest amount of time that the whole
operation takes.
3- In reality, generating all the posts takes about 65(+|-)2 seconds.
I measured this personally using a watch! Any idea where the minimum
65-35=30s and maximum 65-10=55ms are spent for 5000 operations? on
JavaScript engine? in the minimum case of 30s, it translates to
5000/30=166ms per operation. This is a lot of time to be spent on V8.
I account some few milliseconds for updating the UI, and calling the
setTimeout for generating operations every millisecond. But I am
wondering where the rest of time is being spent? we can either assume
that the network monitoring values are not reliable, or the time is
being spent in OCW. For algorithms that happen in memory, to generate
the outbound operations (so I assume no transformation is being done
on the operations) what could be the bottleneck? Is there some other
places out of OCW, in which time is being spent? Am I wrong?
Based on the overall time of sending 5000 operations, I concluded that
the posts are sent to the server sequentially, with no overlapping.
So, the posts are being buffered before sending to the server.
Therefore, even with the lowest latency and fast algorithms,
sequential issuing of posts take the scalability down to say
5000/65=77op/s. It made me wondering if there is a solution to this
burden? Is OCW buffering the operations or the browser? is it possible
for OCW to collect all the outbound operations and send them
periodically to the server in one batch? if so, can this option become
a coWeb config option so that applications can adjust it according to
their needs?
Long post and lots of questions. Now this makes me wondering if anyone
manage to read the whole post!
Navid
I am wondering what happens if clients make a lot of sendSync()
calls?
By 'a lot' I mean something in the neighborhood of [1000,10000] calls
per second!! I know this seems a bit crazy or maybe expecting too much
but I think the question worth exploring at least for the sake of
improving OCW performance.
To make the question simpler, let's think only one client makes this
huge number of sendSync calls and all the other clients are listeners.
If you are interested in a real-world application, think about a real-
time gaming application in which one client is playing the role of the
server, executing the game logic and broadcasting all the changes of
game objects to other clients. Is not this interesting?
Searching for an answer, I did some little experience, having a client
generate 5000 operations in a period of 5 seconds (1000 operations per
second) and I tracked the network POSTs sent to cometd and their
latency. I ran the test on my localhost, ignoring the real-world
latency for the sake of simplicity for the moment. I ran the test few
times to get a sense of mean and variance.
observations:
1- OCW generates one post for every single sendSync call.
2- Based on what the browser (chrome) network monitoring shows, each
post takes somewhere between 2ms and 7ms to be returned from
localhost. So, 5000*2= 10s should be the lowest amount of time and
5000*7=35s should be the highest amount of time that the whole
operation takes.
3- In reality, generating all the posts takes about 65(+|-)2 seconds.
I measured this personally using a watch! Any idea where the minimum
65-35=30s and maximum 65-10=55ms are spent for 5000 operations? on
JavaScript engine? in the minimum case of 30s, it translates to
5000/30=166ms per operation. This is a lot of time to be spent on V8.
I account some few milliseconds for updating the UI, and calling the
setTimeout for generating operations every millisecond. But I am
wondering where the rest of time is being spent? we can either assume
that the network monitoring values are not reliable, or the time is
being spent in OCW. For algorithms that happen in memory, to generate
the outbound operations (so I assume no transformation is being done
on the operations) what could be the bottleneck? Is there some other
places out of OCW, in which time is being spent? Am I wrong?
Based on the overall time of sending 5000 operations, I concluded that
the posts are sent to the server sequentially, with no overlapping.
So, the posts are being buffered before sending to the server.
Therefore, even with the lowest latency and fast algorithms,
sequential issuing of posts take the scalability down to say
5000/65=77op/s. It made me wondering if there is a solution to this
burden? Is OCW buffering the operations or the browser? is it possible
for OCW to collect all the outbound operations and send them
periodically to the server in one batch? if so, can this option become
a coWeb config option so that applications can adjust it according to
their needs?
Long post and lots of questions. Now this makes me wondering if anyone
manage to read the whole post!
Navid
nvd_ai
Jul 20, 2011, 5:29:12 AM7/20/11
to opencoweb
The first part of section 3, and all the computations about 166ms is
invalid. It is actually 30000ms/5000ops=6ms/op time being spent in
javascript, which sounds reasonable.
However, the main question regarding Buffering and batch operations
still is valid.
To myself: Don't write long posts 3 in the morning!
Navid
invalid. It is actually 30000ms/5000ops=6ms/op time being spent in
javascript, which sounds reasonable.
However, the main question regarding Buffering and batch operations
still is valid.
To myself: Don't write long posts 3 in the morning!
Navid
Peter Parente
Jul 21, 2011, 10:42:22 PM7/21/11
to open...@googlegroups.com
Navid,
The cometd layer we use does support batching. The configuration option should be set to deliver multiple syncs to the server within a single connection.
See:
Pete
Reply all
Reply to author
Forward
0 new messages