Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
[off-topic] Re: Periodic message transfers using asio
32 views
Skip to first unread message
Message has been deleted
Alf P. Steinbach
Aug 4, 2019, 12:02:06 AM8/4/19
to
On 04.08.2019 04:53, M Powell wrote:
>
> For starters I apologize if I’m in the wrong forum.
> A colleague is using asio IO to transfer messages between two applications. The message size is 300 bytes and one of the apps is leveraging a periodic timer predicated on chrono so the message is sent on local host at intervals (1 hertz, 10 hertz ...up to 10K hertz)
>
> For example:
> app 1 is set to transfer the message @ 100 Hz
> app 1 @T0 increments a counter within the message then transfers the message
>
> app 2 receives the message, verifies counter was incremented before incrementing the counter and sending the message
>
> The process repeats periodically at the interval specified. app1 has metrics that tracks transfer rate and dropped messages.
>
>
> At issue. The 10k hertz was flagged as questionable by a few members during a review. The code executes on Windows and Linux and there was added skepticism WRT windows. The outputs didn’t revel any performance issues with 10K Herz transfers on either windows or Linux.
>
> My question. Is there a limitation in TCP transactions or asio that would preclude 10K hertz transfers ? The OS windows ?
>
> It’s unclear to me what the potential issues are/could be and I can’t seem to find anything from online search about threshold and metrics on tcp transactions so thought I’d throw it out here to get perspectives.
You are indeed in the wrong forum.
Anyway...
If the network has sufficient bandwidth then I don't see what the
problem is.
However, the old ADSL connection I'm using to post this is pretty slow,
like 12 Mbps for downloads. That's too slow for your 10 kHz => 10 000 *
300 * ~10 = 30 Mbps minimum requirement. It has nothing to do with the
OS and all to do with the connection.
Disclaimer: I haven't done any low level network things since the 1990's
so the ~10 factor instead of 8 bits per byte is roughly the overhead of
serial cable communication, and the overhead of a modern network
connection, its speed requirement, is probably waaaaaaay higher.
Cheers!,
- Alf
>
> For starters I apologize if I’m in the wrong forum.
> A colleague is using asio IO to transfer messages between two applications. The message size is 300 bytes and one of the apps is leveraging a periodic timer predicated on chrono so the message is sent on local host at intervals (1 hertz, 10 hertz ...up to 10K hertz)
>
> For example:
> app 1 is set to transfer the message @ 100 Hz
> app 1 @T0 increments a counter within the message then transfers the message
>
> app 2 receives the message, verifies counter was incremented before incrementing the counter and sending the message
>
> The process repeats periodically at the interval specified. app1 has metrics that tracks transfer rate and dropped messages.
>
>
> At issue. The 10k hertz was flagged as questionable by a few members during a review. The code executes on Windows and Linux and there was added skepticism WRT windows. The outputs didn’t revel any performance issues with 10K Herz transfers on either windows or Linux.
>
> My question. Is there a limitation in TCP transactions or asio that would preclude 10K hertz transfers ? The OS windows ?
>
> It’s unclear to me what the potential issues are/could be and I can’t seem to find anything from online search about threshold and metrics on tcp transactions so thought I’d throw it out here to get perspectives.
You are indeed in the wrong forum.
Anyway...
If the network has sufficient bandwidth then I don't see what the
problem is.
However, the old ADSL connection I'm using to post this is pretty slow,
like 12 Mbps for downloads. That's too slow for your 10 kHz => 10 000 *
300 * ~10 = 30 Mbps minimum requirement. It has nothing to do with the
OS and all to do with the connection.
Disclaimer: I haven't done any low level network things since the 1990's
so the ~10 factor instead of 8 bits per byte is roughly the overhead of
serial cable communication, and the overhead of a modern network
connection, its speed requirement, is probably waaaaaaay higher.
Cheers!,
- Alf
Melzzzzz
Aug 4, 2019, 12:12:24 AM8/4/19
to
On 2019-08-04, M Powell <foru...@gmail.com> wrote:
>
> For starters I apologize if I’m in the wrong forum. A colleague is
> using asio IO to transfer messages between two applications.
Whats asio IO?
>
> For starters I apologize if I’m in the wrong forum. A colleague is
> using asio IO to transfer messages between two applications.
>
> At issue. The 10k hertz was flagged as questionable by a few members
> during a review. The code executes on Windows and Linux and there was
> added skepticism WRT windows. The outputs didn’t revel any
> performance issues with 10K Herz transfers on either windows or Linux.
>
> My question. Is there a limitation in TCP transactions or asio that
> would preclude 10K hertz transfers ? The OS windows ?
>
> It’s unclear to me what the potential issues are/could be and I can’t
> seem to find anything from online search about threshold and metrics
> on tcp transactions so thought I’d throw it out here to get
> perspectives.
>
> Thanks in advance.
You are welcome.
--
press any key to continue or any other to quit...
U ničemu ja ne uživam kao u svom statusu INVALIDA -- Zli Zec
Na divljem zapadu i nije bilo tako puno nasilja, upravo zato jer su svi
bili naoruzani. -- Mladen Gogala
Cholo Lennon
Aug 4, 2019, 12:47:45 AM8/4/19
to
On 8/3/19 11:53 PM, M Powell wrote:
>
> For starters I apologize if I’m in the wrong forum.
>
> For starters I apologize if I’m in the wrong forum.
> A colleague is using asio IO to transfer messages between two applications. The message size is 300 bytes and one of the apps is leveraging a periodic timer predicated on chrono so the message is sent on local host at intervals (1 hertz, 10 hertz ...up to 10K hertz)
I assume that your are talking about Boost.Asio. The right forum for
that is news://news.gmane.org/gmane.comp.lib.boost.user
More information here: https://www.boost.org/community/groups.html
>
> For example:
> app 1 is set to transfer the message @ 100 Hz
> app 1 @T0 increments a counter within the message then transfers the message
>
> app 2 receives the message, verifies counter was incremented before incrementing the counter and sending the message
>
> The process repeats periodically at the interval specified. app1 has metrics that tracks transfer rate and dropped messages.
>
>
> At issue. The 10k hertz was flagged as questionable by a few members during a review. The code executes on Windows and Linux and there was added skepticism WRT windows. The outputs didn’t revel any performance issues with 10K Herz transfers on either windows or Linux.
>
> My question. Is there a limitation in TCP transactions or asio that would preclude 10K hertz transfers ? The OS windows ?
>
>
> My question. Is there a limitation in TCP transactions or asio that would preclude 10K hertz transfers ? The OS windows ?
>
> It’s unclear to me what the potential issues are/could be and I can’t seem to find anything from online search about threshold and metrics on tcp transactions so thought I’d throw it out here to get perspectives.
>
> Thanks in advance.
>
>
Best Regards
--
Cholo Lennon
Bs.As.
ARG
Paavo Helde
Aug 4, 2019, 8:57:40 AM8/4/19
to
On 4.08.2019 5:53, M Powell wrote:
>
> For starters I apologize if I’m in the wrong forum.
> A colleague is using asio IO to transfer messages between two applications. The message size is 300 bytes and one of the apps is leveraging a periodic timer predicated on chrono so the message is sent on local host at intervals (1 hertz, 10 hertz ...up to 10K hertz)
>
>
> For starters I apologize if I’m in the wrong forum.
> A colleague is using asio IO to transfer messages between two applications. The message size is 300 bytes and one of the apps is leveraging a periodic timer predicated on chrono so the message is sent on local host at intervals (1 hertz, 10 hertz ...up to 10K hertz)
>
> For example:
> app 1 is set to transfer the message @ 100 Hz
> app 1 @T0 increments a counter within the message then transfers the message
>
> app 2 receives the message, verifies counter was incremented before incrementing the counter and sending the message
>
> The process repeats periodically at the interval specified. app1 has metrics that tracks transfer rate and dropped messages.
>
>
> At issue. The 10k hertz was flagged as questionable by a few members during a review. The code executes on Windows and Linux and there was added skepticism WRT windows. The outputs didn’t revel any performance issues with 10K Herz transfers on either windows or Linux.
>
> My question. Is there a limitation in TCP transactions or asio that would preclude 10K hertz transfers ? The OS windows ?
I'm pretty sure the problematic points would not be the TCP or network
> app 1 is set to transfer the message @ 100 Hz
> app 1 @T0 increments a counter within the message then transfers the message
>
> app 2 receives the message, verifies counter was incremented before incrementing the counter and sending the message
>
> The process repeats periodically at the interval specified. app1 has metrics that tracks transfer rate and dropped messages.
>
>
> At issue. The 10k hertz was flagged as questionable by a few members during a review. The code executes on Windows and Linux and there was added skepticism WRT windows. The outputs didn’t revel any performance issues with 10K Herz transfers on either windows or Linux.
>
> My question. Is there a limitation in TCP transactions or asio that would preclude 10K hertz transfers ? The OS windows ?
stack, but rather the overall performance of the computer, and
especially the variations in its performance. And the 10 kHz number is
not something special, similar problems are there always when you
require something to happen during some fixed time.
A consumer-grade OS like Windows or Linux does not guarantee your
program gets a timeslice for running during each 0.1 ms (i.e. 10 kHz).
For hard guarantees one should use some real-time OS instead. In
Windows/Linux one must accept the possibility of occasional slowdowns,
and code accordingly.
The slowdowns may sometimes be severe. If the computer gets overloaded
with too many tasks or too large memory consumption, it may slow to
crawl. Your program might not get a timeslice even during a whole
second, not to speak about milliseconds. There are some ways to mitigate
this by playing with thread/process priorities and locking pages in RAM,
which might help to an extent.
Experience shows Windows tends to be more prone to this "slowing to
crawl" behavior than Linux, and what's worse, it appears to recover from
such situations much worse, if at ever. After there has been a memory
exhaustion event, often the only way to get Windows to function normally
again is a restart. A typical Windows box might need a restart after
every few weeks anyway, to restore its original performance.
In Windows, you also do not know when an antivirus or another too snoopy
spyware decides to install itself in the network loopback interface and
start monitoring all the traffic, causing unpredictable delays. I recall
in one Windows product we replaced the loopback TCP connection by a
solution using shared memory, because there appeared to be random delays
in TCP which we were not able to explain or eliminate. YMMV, of course.
In short, if you have full control over the hardware and installed
software on the machine where your program is supposed to work, and have
verified it by prolonged testing, and you can accept occasional loss of
traffic, and can ensure daily restarts of Windows, then you should
probably be OK.
Another way is to accept there might be functionality loss and tell the
user to fix any problems. Your task is a bit similar to audio playing,
although audio is a bit easier as important frequencies are way below 10
kHz, and for audio there is special hardware support as well (which I do
not know much about though). Now let's say if an .mp3 file does not play
well because of computer overload, the user can try to close other apps
or restart the machine. If this would be OK for your users you should be
fine as well.
Jorgen Grahn
Aug 5, 2019, 8:06:06 AM8/5/19
to
On Sun, 2019-08-04, Paavo Helde wrote:
> On 4.08.2019 5:53, M Powell wrote:
>>
>> For starters I apologize if I’m in the wrong forum. A colleague is
>> using asio IO
You later write about TCP so I assume that's what it really is, TCP
> On 4.08.2019 5:53, M Powell wrote:
>>
>> For starters I apologize if I’m in the wrong forum. A colleague is
>> using asio IO
I/O implemented using some part of the huge and varied Asio library.
>> to transfer messages between two applications. The
>> message size is 300 bytes and one of the apps is leveraging a
>> periodic timer predicated on chrono so the message is sent on local
>> host at intervals (1 hertz, 10 hertz ...up to 10K hertz)
>>
>> For example:
>> app 1 is set to transfer the message @ 100 Hz
>> app 1 @T0 increments a counter within the message then transfers the message
>>
>> app 2 receives the message, verifies counter was incremented
>> before incrementing the counter and sending the message
>>
>> The process repeats periodically at the interval specified. app1
>> has metrics that tracks transfer rate and dropped messages.
>> At issue. The 10k hertz was flagged as questionable by a few
>> members during a review. The code executes on Windows and Linux
>> and there was added skepticism WRT windows. The outputs didn’t
>> revel any performance issues with 10K Herz transfers on either
>> windows or Linux.
>>
>> My question. Is there a limitation in TCP transactions or asio
>> that would preclude 10K hertz transfers ? The OS windows ?
>
> I'm pretty sure the problematic points would not be the TCP or network
> stack, but rather the overall performance of the computer, and
> especially the variations in its performance. And the 10 kHz number is
> not something special, similar problems are there always when you
> require something to happen during some fixed time.
with no real-time properties; if the stack sees you do tiny writes to
the stream at a high rate it might (and /should/) start lumping them
together to avoid wasting computer and network resources.
Sounds to me the protocol is misdesigned, if it expects heartbeats
to work over TCP in a timely fashion, even at 10 or 100 Hz.
> A consumer-grade OS like Windows or Linux does not guarantee your
> program gets a timeslice for running during each 0.1 ms (i.e. 10 kHz).
> For hard guarantees one should use some real-time OS instead. In
> Windows/Linux one must accept the possibility of occasional slowdowns,
> and code accordingly.
/Jorgen
--
// Jorgen Grahn <grahn@ Oo o. . .
\X/ snipabacken.se> O o .
M Powell
Aug 5, 2019, 8:22:54 PM8/5/19
to
On Sunday, August 4, 2019 at 12:02:06 AM UTC-4, Alf P. Steinbach wrote:
> On 04.08.2019 04:53, M Powell wrote:
> >
> > For starters I apologize if I’m in the wrong forum.
> > A colleague is using asio IO to transfer messages between two applications. The message size is 300 bytes and one of the apps is leveraging a periodic timer predicated on chrono so the message is sent on local host at intervals (1 hertz, 10 hertz ...up to 10K hertz)
> >
> > For example:
> > app 1 is set to transfer the message @ 100 Hz
> > app 1 @T0 increments a counter within the message then transfers the message
> >
> > app 2 receives the message, verifies counter was incremented before incrementing the counter and sending the message
> >
> > The process repeats periodically at the interval specified. app1 has metrics that tracks transfer rate and dropped messages.
> >
> >
> > At issue. The 10k hertz was flagged as questionable by a few members during a review. The code executes on Windows and Linux and there was added skepticism WRT windows. The outputs didn’t revel any performance issues with 10K Herz transfers on either windows or Linux.
> >
> > My question. Is there a limitation in TCP transactions or asio that would preclude 10K hertz transfers ? The OS windows ?
> >
> > It’s unclear to me what the potential issues are/could be and I can’t seem to find anything from online search about threshold and metrics on tcp transactions so thought I’d throw it out here to get perspectives.
>
> You are indeed in the wrong forum.
>
> Anyway...
>
> If the network has sufficient bandwidth then I don't see what the
> problem is.
>
Got it. Thanks Alf
> On 04.08.2019 04:53, M Powell wrote:
> >
> > For starters I apologize if I’m in the wrong forum.
> > A colleague is using asio IO to transfer messages between two applications. The message size is 300 bytes and one of the apps is leveraging a periodic timer predicated on chrono so the message is sent on local host at intervals (1 hertz, 10 hertz ...up to 10K hertz)
> >
> > For example:
> > app 1 is set to transfer the message @ 100 Hz
> > app 1 @T0 increments a counter within the message then transfers the message
> >
> > app 2 receives the message, verifies counter was incremented before incrementing the counter and sending the message
> >
> > The process repeats periodically at the interval specified. app1 has metrics that tracks transfer rate and dropped messages.
> >
> >
> > At issue. The 10k hertz was flagged as questionable by a few members during a review. The code executes on Windows and Linux and there was added skepticism WRT windows. The outputs didn’t revel any performance issues with 10K Herz transfers on either windows or Linux.
> >
> > My question. Is there a limitation in TCP transactions or asio that would preclude 10K hertz transfers ? The OS windows ?
> >
> > It’s unclear to me what the potential issues are/could be and I can’t seem to find anything from online search about threshold and metrics on tcp transactions so thought I’d throw it out here to get perspectives.
>
> You are indeed in the wrong forum.
>
> Anyway...
>
> If the network has sufficient bandwidth then I don't see what the
> problem is.
>
M Powell
Aug 5, 2019, 8:24:20 PM8/5/19
to
On Sunday, August 4, 2019 at 8:57:40 AM UTC-4, Paavo Helde wrote:
> On 4.08.2019 5:53, M Powell wrote:
> >
> > For starters I apologize if I’m in the wrong forum.
> > A colleague is using asio IO to transfer messages between two applications. The message size is 300 bytes and one of the apps is leveraging a periodic timer predicated on chrono so the message is sent on local host at intervals (1 hertz, 10 hertz ...up to 10K hertz)
> >
> > For example:
> > app 1 is set to transfer the message @ 100 Hz
> > app 1 @T0 increments a counter within the message then transfers the message
> >
> > app 2 receives the message, verifies counter was incremented before incrementing the counter and sending the message
> >
> > The process repeats periodically at the interval specified. app1 has metrics that tracks transfer rate and dropped messages.
> >
> >
> > At issue. The 10k hertz was flagged as questionable by a few members during a review. The code executes on Windows and Linux and there was added skepticism WRT windows. The outputs didn’t revel any performance issues with 10K Herz transfers on either windows or Linux.
> >
> > My question. Is there a limitation in TCP transactions or asio that would preclude 10K hertz transfers ? The OS windows ?
>
> I'm pretty sure the problematic points would not be the TCP or network
> stack, but rather the overall performance of the computer, and
> especially the variations in its performance. And the 10 kHz number is
> not something special, similar problems are there always when you
> require something to happen during some fixed time.
>
Got it. Thanks Paavo
> On 4.08.2019 5:53, M Powell wrote:
> >
> > For starters I apologize if I’m in the wrong forum.
> > A colleague is using asio IO to transfer messages between two applications. The message size is 300 bytes and one of the apps is leveraging a periodic timer predicated on chrono so the message is sent on local host at intervals (1 hertz, 10 hertz ...up to 10K hertz)
> >
> > For example:
> > app 1 is set to transfer the message @ 100 Hz
> > app 1 @T0 increments a counter within the message then transfers the message
> >
> > app 2 receives the message, verifies counter was incremented before incrementing the counter and sending the message
> >
> > The process repeats periodically at the interval specified. app1 has metrics that tracks transfer rate and dropped messages.
> >
> >
> > At issue. The 10k hertz was flagged as questionable by a few members during a review. The code executes on Windows and Linux and there was added skepticism WRT windows. The outputs didn’t revel any performance issues with 10K Herz transfers on either windows or Linux.
> >
> > My question. Is there a limitation in TCP transactions or asio that would preclude 10K hertz transfers ? The OS windows ?
>
> I'm pretty sure the problematic points would not be the TCP or network
> stack, but rather the overall performance of the computer, and
> especially the variations in its performance. And the 10 kHz number is
> not something special, similar problems are there always when you
> require something to happen during some fixed time.
>
0 new messages