I need a CPU core exclusively for one thread

[Frederick Virchanza Gotham]

I'm writing a network analysis program, and just today I've found out that it's too slow and missing packets.

Ideally, I want the thread that reads from the network card to have exclusive use of one of the CPU cores.

I'm looking at the following two functions:
sched_setaffinity
pthread_setaffinity_np

And I note the following in the Linux manual:
"For example, by dedicating one CPU to a particular process (i.e., setting the affinity mask of that process to specify a single CPU, and setting the affinity mask of all other processes to exclude that CPU), it is possible to ensure maximum execution speed for that process"

I can find lots and lots of code examples to set the affinity for one thread or for one process -- however I cannot find one code example showing me how to exclude all other threads.

There's absolutely no point in me restricting my thread to the 1st CPU core if other threads can run on that core too -- in fact that will make my program _slower_ because my thread can't be scheduled on the other cores. Every code sample I can find online seems to just restrict the thread in question without placing any restriction on other threads.

Anyone know how this is supposed to work?

[David Brown]

"cpu isolation" is, I believe, the term you want to look for on Google.
I don't know if this can only be set from boot parameters or configured
at run-time. You could also search for real-time linux, as that is
where these features are often used.

You will probably also want to disable some or all power-down modes and
clock speed switching, as these can give significant variations in
latencies. And you may need to pin the network card interrupts to a
processor core too.

[Scott Lurndal]

Frederick Virchanza Gotham <cauldwel...@gmail.com> writes:
>
>I'm writing a network analysis program, and just today I've found out that =

>it's too slow and missing packets.
>

>Ideally, I want the thread that reads from the network card to have exclusi=

>ve use of one of the CPU cores.
>
>I'm looking at the following two functions:
> sched_setaffinity
> pthread_setaffinity_np

This is A C++ group. A simple google search should be more productive,
and the answer will be dependent upon which operating system you are
targetting.

[Chris M. Thomasson]

On 7/3/2023 3:42 AM, Frederick Virchanza Gotham wrote:
>
> I'm writing a network analysis program, and just today I've found out that it's too slow and missing packets.
>
> Ideally, I want the thread that reads from the network card to have exclusive use of one of the CPU cores.
>
> I'm looking at the following two functions:
> sched_setaffinity
> pthread_setaffinity_np
>
> And I note the following in the Linux manual:
> "For example, by dedicating one CPU to a particular process (i.e., setting the affinity mask of that process to specify a single CPU, and setting the affinity mask of all other processes to exclude that CPU), it is possible to ensure maximum execution speed for that process"
>
> I can find lots and lots of code examples to set the affinity for one thread or for one process -- however I cannot find one code example showing me how to exclude all other threads.

Not sure you can do that. The OS can do whatever it wants. It can pin
your thread to a CPU, but that does not mean it cannot use said CPU for
other programs threads...

[Scott Lurndal]

"Chris M. Thomasson" <chris.m.t...@gmail.com> writes:
>On 7/3/2023 3:42 AM, Frederick Virchanza Gotham wrote:
>>
>> I'm writing a network analysis program, and just today I've found out that it's too slow and missing packets.
>>
>> Ideally, I want the thread that reads from the network card to have exclusive use of one of the CPU cores.
>>
>> I'm looking at the following two functions:
>> sched_setaffinity
>> pthread_setaffinity_np
>>
>> And I note the following in the Linux manual:
>> "For example, by dedicating one CPU to a particular process (i.e., setting the affinity mask of that process to specify a single CPU, and setting the affinity mask of all other processes to exclude that CPU), it is possible to ensure maximum execution speed for that process"
>>
>> I can find lots and lots of code examples to set the affinity for one thread or for one process -- however I cannot find one code example showing me how to exclude all other threads.
>
>Not sure you can do that. The OS can do whatever it wants. It can pin
>your thread to a CPU, but that does not mean it cannot use said CPU for
>other programs threads...

Linux provides mechanisms that allow dedication of compute resources
(e.g. hardware thread/core) to a specific set of processes. So do
most other legacy Unix operating systems.

The mechanisms vary based on the host operating system. cgroups is
the current favored mechanism on linux.

[Bonita Montero]

Am 03.07.2023 um 12:42 schrieb Frederick Virchanza Gotham:
>
> I'm writing a network analysis program, and just today I've found out that it's too slow and missing packets.
>
> Ideally, I want the thread that reads from the network card to have exclusive use of one of the CPU cores.
>
> I'm looking at the following two functions:
> sched_setaffinity
> pthread_setaffinity_np

I think the packets are likely to be processed at the core where the
interrupt of the NIC hits if the core isn't currently running another
thread. The interrupt runs at an aribtrary core, so I think it's un-
likely that it will hit the core you reserve for the procesing.
And at all I think the system has enough buffering capabilities for
the packets which aren't delivered to your thread yet. So there must
sth. different that you're missing packets.
If you like you could try io_uring, which has the shortest path from
kernel to userspace if the frequency of packets is high. If you have
one packet in the receive queue the processing is as usual with a
kernel notification. If you have further packets in the queue and the
kernel notices that theres an unprocessed packed in the ring it pushes
further packets into the ring without any notifications form kernel-
to user-space.

[Scott Lurndal]

Bonita Montero <Bonita....@gmail.com> writes:
>Am 03.07.2023 um 12:42 schrieb Frederick Virchanza Gotham:
>>
>> I'm writing a network analysis program, and just today I've found out that it's too slow and missing packets.
>>
>> Ideally, I want the thread that reads from the network card to have exclusive use of one of the CPU cores.
>>
>> I'm looking at the following two functions:
>> sched_setaffinity
>> pthread_setaffinity_np
>
>I think the packets are likely to be processed at the core where the
>interrupt of the NIC hits if the core isn't currently running another
>thread. The interrupt runs at an aribtrary core,

This is not accurate. RSS is used to direct NIC interrupts to specific
cores, often depending on the contents of header fields (e.g. TCP packets
destined for port 80 may be sent to a different core than UDP packets or
RDMA packets). Rules are programmed into the NIC to designate the target
core in the packet parsing logic.

Most modern NICs support some form of RSS (receive side scaling) and the
parsing rules can be arbitrarily complex in advanced networking chips
such as the OcteonTx2 processors (parsing MPLS, various tunnelling
protocols, IPsec, higher-level protocols like TCP et cetera et alia).

[Bonita Montero]

Am 04.07.2023 um 15:53 schrieb Scott Lurndal:

> This is not accurate. RSS is used to direct NIC interrupts to

> specific cores, often depending on the contents of header fields ...

And which NIC supports that ?

> Most modern NICs support some form of RSS ...

I don't believe that and Linux doesn't use task-offloading.

[Scott Lurndal]

Bonita Montero <Bonita....@gmail.com> writes:
>Am 04.07.2023 um 15:53 schrieb Scott Lurndal:
>
>> This is not accurate. RSS is used to direct NIC interrupts to
>> specific cores, often depending on the contents of header fields ...
>
>And which NIC supports that ?

Start with the ubiquitous Intel E-1000 and pretty much
every other Intel pro NIC.

https://www.intel.com/content/www/us/en/support/articles/000006703/ethernet-products.html

>
>> Most modern NICs support some form of RSS ...
>

Most modern server NICs support some form of RSS. Cheap-ass consumer
gear (realtek, etc) - you get what you pay for.

[Frederick Virchanza Gotham]

On Tuesday, July 4, 2023 at 1:50:34 PM UTC+1, Bonita Montero wrote:
>
> So there must sth. different that you're missing packets.

I figured out what's going on.

In my code, when an Ethernet frame is read from the network card, I check the checksum of the IP header, and also the checksum of the TCP segment. If either is wrong, the entire Ethernet frame is discarded.

In my code, I calculate what the checksum should be for the IP header, and I saw that in cases where the frame is discarded, the incoming packet had the 'total packet length' in the IP header as 44, but in my code I was calculating the checksum of 46 bytes. I didn't know where the other 2 bytes were coming from, whether they were 'options' in the TCP header or whatever, but then in Wireshark I saw that the packet had a mysterious "00 00" at the end. I selected those two bytes and Wireshark told me "Ethernet padding bytes".

Turns out that if an Ethernet frame is less than 60 bytes, then it's padded with zeroes to bring it to 60 bytes.

Ethernet header is 14 bytes, and the IP packet was 44 bytes, bringing the incoming frame to 58 bytes. So the operating system / network card tagged two zero bytes on the end.

I was debugging it for approx 8 hours before I figured out what was wrong. Now I'm lost losing any packets.

Still though I just set the 'priority' of my network reading thread to maximum.

[Frederick Virchanza Gotham]

On Tuesday, July 4, 2023 at 6:25:16 PM UTC+1, Frederick Virchanza Gotham wrote:

> I was debugging it for approx 8 hours before I figured out what was wrong. Now I'm lost losing any packets.

Typo:

Now I'm ***not*** losing any packets.

[Chris M. Thomasson]

Thanks Scott. Actually, I remember using affinity masks to try to
emulate per-cpu data in user mode as an experiment.

[Chris M. Thomasson]

Iirc, I think I used the ALOM to check a cpu temperature when I pinned a
hot thread with a high priority to a CPU.

[Bonita Montero]

Am 04.07.2023 um 19:08 schrieb Scott Lurndal:
> Bonita Montero <Bonita....@gmail.com> writes:
>> Am 04.07.2023 um 15:53 schrieb Scott Lurndal:
>>
>>> This is not accurate. RSS is used to direct NIC interrupts to
>>> specific cores, often depending on the contents of header fields ...
>>
>> And which NIC supports that ?
>
> Start with the ubiquitous Intel E-1000 and pretty much
> every other Intel pro NIC.

The NICs have Task offloading for IP- and TCP Checksum calculation.
Bu they for sure don't manage streams ! And interrupt routing isn't
done by the NIC.

> Most modern server NICs support some form of RSS.
> Cheap-ass consumer gear (realtek, etc) - you get what you pay for.

Linux doesn't use task offloading according to this:

https://en.wikipedia.org/wiki/TCP_offload_engine

[Bonita Montero]

Why do you do the checksum-stuff yourself ?

[Frederick Virchanza Gotham]

On Wednesday, July 5, 2023 at 4:28:22 AM UTC+1, Bonita Montero wrote:
>
> Why do you do the checksum-stuff yourself ?

My program opens a raw socket and does port scanning. I last released a version about 14 years ago: ( http://www.virjacode.com/projects/dynamo/ ), and I hope to release a new version with way more features this year.

[Vir Campestris]

Thanks for letting us know. You've also just shown me how out of date I
am in HW knowledge - last time I looked a device only had one interrupt
line :(

So in fact you never were losing packets at the HW level, you were
throwing them away because you thought they were bad!

Andy

[Bonita Montero]

Am 05.07.2023 um 10:54 schrieb Frederick Virchanza Gotham:

> My program opens a raw socket and does port scanning. I last released a version about 14 years ago: ( http://www.virjacode.com/projects/dynamo/ ), and I hope to release a new version with way more features this year.

I don't see much sense in that. If you scan a external
port range you need your own IP as the sender's address
to get replies. And if you do it that way you could use
normal sockets.

[Frederick Virchanza Gotham]

I suppose I could use normal sockets but I don't want the overhead of the operating system managing tens of thousands (yes, tens of thousands) of sockets and managing the 3-way handshakes for all those sockets. It's a lot more efficient to use a raw socket to send a byte-by-byte hand-crafted TCP SYN packet to each host, from port numbers 1 through 49151.

[Scott Lurndal]

Bonita Montero <Bonita....@gmail.com> writes:
>Am 04.07.2023 um 19:08 schrieb Scott Lurndal:
>> Bonita Montero <Bonita....@gmail.com> writes:
>>> Am 04.07.2023 um 15:53 schrieb Scott Lurndal:
>>>
>>>> This is not accurate. RSS is used to direct NIC interrupts to
>>>> specific cores, often depending on the contents of header fields ...
>>>
>>> And which NIC supports that ?
>>
>> Start with the ubiquitous Intel E-1000 and pretty much
>> every other Intel pro NIC.
>
>The NICs have Task offloading for IP- and TCP Checksum calculation.
>Bu they for sure don't manage streams ! And interrupt routing isn't
>done by the NIC.

Your ignorance is showing. My company makes NICs. Very powerful
NICs. Which manage streams (the programmable TCAMs allow software
to select the packet fields that get CAMd when selecting the target CPU
for the interrupt).

Most use PCI/PCIe MSI-X interrupts, which have a 32-bit payload field
allowing (depending on the host interrupt controller) up to
2^32 different interrupt vectors. Our system support 21 bit
payloads, supporting 2^21 physical interrupts _per_ virtual device
using the ARM GIC-700 interrupt controller.

Using SR-IOV allows direct shared, but mediated, access to the NIC
from guest operating systems.

>
>> Most modern server NICs support some form of RSS.
>> Cheap-ass consumer gear (realtek, etc) - you get what you pay for.
>
>Linux doesn't use task offloading according to this:
>
>https://en.wikipedia.org/wiki/TCP_offload_engine

That's also incorrect. Take a look at DPDK or ODP.

[Scott Lurndal]

Most modern high throughput devices use message signaled interrupts (MSI or MSI-X); MSI-X
has a 32-bit payload field, allowing up to four billion distinct interrupt
"lines". With interrupt virtualization, each guest operating system has
its own 32-bit interrupt namespace, allowing an infinite number of interrupt
"lines". The days of sharing level sensitive interrupts are long gone,
except for supporting legacy hardware.

>
>So in fact you never were losing packets at the HW level, you were
>throwing them away because you thought they were bad!

Or RED kicked in.

https://en.wikipedia.org/wiki/Random_early_detection

[Bonita Montero]

Am 05.07.2023 um 15:28 schrieb Frederick Virchanza Gotham:

> I suppose I could use normal sockets but I don't want the overhead of the operating system managing tens of thousands (yes, tens of thousands) of sockets and managing the 3-way handshakes for all those sockets. ...

These information is hashed and the time for the lookup is O(1).

[Frederick Virchanza Gotham]

Yeah but we're talking 40,000 file descriptors for one process.

[Bonita Montero]

Am 05.07.2023 um 15:49 schrieb Scott Lurndal:

> Your ignorance is showing. My company makes NICs. Very powerful
> NICs. Which manage streams (the programmable TCAMs allow software
> to select the packet fields that get CAMd when selecting the target
> CPU for the interrupt).

Show me the documentation.

[Bonita Montero]

Even with a billion file descriptors the access time would
be the same, i.e. O(1).

[David Brown]

I have no idea what NIC's Scott's company makes, but it should not be
too hard to google a bit about "intelligent" NIC's. They range from
relatively simple acceleration such as RSS and TCP/IP checksums to
devices that combine switching, routing, filtering and server facilities
on the NIC card, running their own Linux system on the card.

The simpler TCP/IP offload engines are rarely used with Linux, because
they significantly complicate the network stacks, are much more limited
than networking in software, and modern processors can do the job faster
in software than the devices can do it in hardware. (It's a bit like
hardware RAID cards in that aspect.)

But when you have 100 Gbps and faster NICs, with large multi-core
processors, then it is not at all uncommon to use RSS (and similar) to
hold multiple queues in the NIC, each with their own interrupt that is
then routed to specific CPU cores. The aim is to maximise cache reuse
and reduce latency. More sophisticated cards will do filtering, NAT,
transformations, etc., as well as queueing and interrupt routing based
on content. And these kinds of cards are almost always used with Linux.

[Bonita Montero]

Am 06.07.2023 um 13:49 schrieb David Brown:

> I have no idea what NIC's Scott's company makes, but it should
> not be too hard to google a bit about "intelligent" NIC's.

Google for NIC task-offloading and you don't find anything that
supports his statements at least for Linux.

> The simpler TCP/IP offload engines are rarely used with Linux, ...

And where are they actually supportet ?
With Windows, that supports that kind of task-offloading ?

> But when you have 100 Gbps and faster NICs, ...

100GbE is rather a backbone technology.

[Scott Lurndal]

This is an older generation:

https://doc.dpdk.org/guides-20.11/platform/octeontx2.html

And the current, which builds on the prior generations, going
all the way back to the original MIPS-based Octeons more than a decade ago.

https://www.marvell.com/company/media-kit/marvell-octeon-tx2-and-octeon-fusion-press-kit.html

[Michael S]

That's too high end for just about everybody, and I don't think that it really used in any
Marvell-made NIC, not even in the ultra-high-end FastLinQ 45000 Series.

This one more relevant:
marvell-fastLinq-edge-aqc113-aqc113c-aqc113cs-aqc114cs-aqc115c-aqc116c-product-brief.pdf

[Scott Lurndal]

Michael S <already...@yahoo.com> writes:
>On Thursday, July 6, 2023 at 5:32:16=E2=80=AFPM UTC+3, Scott Lurndal wrote:
>> Bonita Montero <Bonita....@gmail.com> writes:=20
>> >Am 05.07.2023 um 15:49 schrieb Scott Lurndal:=20
>> >=20
>> >> Your ignorance is showing. My company makes NICs. Very powerful=20
>> >> NICs. Which manage streams (the programmable TCAMs allow software=20
>> >> to select the packet fields that get CAMd when selecting the target=20
>> >> CPU for the interrupt).=20
>> >=20
>> >Show me the documentation.
>> This is an older generation:=20
>>=20
>> https://doc.dpdk.org/guides-20.11/platform/octeontx2.html=20
>>=20
>> And the current, which builds on the prior generations, going=20
>> all the way back to the original MIPS-based Octeons more than a decade ag=
>o.=20
>>=20
>> https://www.marvell.com/company/media-kit/marvell-octeon-tx2-and-octeon-f=
>usion-press-kit.html
>
>
>That's too high end for just about everybody, and I don't think that it rea=

>lly used in any
>Marvell-made NIC, not even in the ultra-high-end FastLinQ 45000 Series.

Marvell has a wide range of network interface chips. The OcteonTX
series have both low-end and high-end entries. The low-end (MIPS and ARM64)
Octeons are used in home routers, SOHO routers and some low-end enterprise
routers. The high-end OcteonTx are used in high-end enterprise routers,
cellular and backbone infrastructure.

However, the feature that started this conversation, recieve-side scaling
is widely used and supported by many different NIC vendors, including Intel's
PRO line of network interfaces.

[Bonita Montero]

Am 06.07.2023 um 16:31 schrieb Scott Lurndal:

> This is an older generation:
>
> https://doc.dpdk.org/guides-20.11/platform/octeontx2.html
>
> And the current, which builds on the prior generations, going
> all the way back to the original MIPS-based Octeons more than a decade ago.
>
> https://www.marvell.com/company/media-kit/marvell-octeon-tx2-and-octeon-fusion-press-kit.html

That are infrastructure processors. I don't see much sense in
having such >= 25GbE networking in a server since the processing
behind such flows of any server application cost magnitudes more
CPU time than flow segmentation. The devices shown in the PDF
I've checkt confirm that.

[Michael S]

Exactly. My point is that Octeon chips are not used in NICs.

> However, the feature that started this conversation, recieve-side scaling
> is widely used and supported by many different NIC vendors, including Intel's
> PRO line of network interfaces.

That's, I think, is obvious to everybody except Bonita.
In the past RSS was supported even on non-pro or semi-pro Intel NICs like
82574L that I use to post this message. I didn't check if it's still supported
in this range. Practically, it's no longer needed for 1Gbe since even cheap
today's CPU will handle everything without breaking a sweet. But if they had
in the past, may be they retain it just for sake of check box checked.

[David Brown]

On 06/07/2023 16:28, Bonita Montero wrote:
> Am 06.07.2023 um 13:49 schrieb David Brown:
>
>> I have no idea what NIC's Scott's company makes, but it should
>> not be  too hard to google a bit about "intelligent" NIC's.
>
> Google for NIC task-offloading and you don't find anything that
> supports his statements at least for Linux.
>

Which statements, /exactly/, do you disagree with? Try to quote his
words, then perhaps explain in your own words what you think he meant.

(Note that Scott did not mention TOE - /you/ did. He talked about RSS.)

>> The simpler TCP/IP offload engines are rarely used with Linux, ...
>
> And where are they actually supportet ?

They /are/ supported in Linux - they are just not /used/ with Linux,
because they TOE is a crappy solution. Generally you get faster
throughput by disabling TOE.

> With Windows, that supports that kind of task-offloading ?

TOE is used with Windows, because it is a crappy OS for server work.
(Windows is fine for many other aspects - all systems have their good
points and bad points.) Windows' network stack is very limited - it can
only do simple tasks, so it does no harm to have a simple accelerators.

It's like RAID. Windows - even the server versions - does not have
software RAID worth mentioning. So you have to use hardware RAID
solutions. Linux has several different types of software RAID to suit
different needs, with a great deal more flexibility than you get on any
hardware RAID devices, and usually a lot faster.

You only bother with accelerators on Linux systems when you are dealing
with massive amounts of traffic on fast (and expensive) ports. TOE is
not really a consideration here - when RSS is not enough, you have more
advanced pre-filtering and queuing on the NICs. And these all involve
interrupt direction to specific cpus.

I can't imagine anyone using Windows for such systems.

>
>> But when you have 100 Gbps and faster NICs, ...
>
> 100GbE is rather a backbone technology.
>

A dual-port 100 Gb network card is perhaps $800. That is well within
the budget for a high-end server.

[David Brown]

There is steadily more work being done on network cards, in data centres
and wherever you need to handle a lot of traffic. You might not call
them "NICs", since they have a full OS on board, but they are cards that
you put in a server to connect to the network. I don't know the parts
Scott is talking about (I don't know much about /any/ of these, except
what I read about on the net and dream of having an excuse to use them).

<https://www.servethehome.com/what-is-a-dpu-a-data-processing-unit-quick-primer/>

[Scott Lurndal]

Michael S <already...@yahoo.com> writes:

>On Thursday, July 6, 2023 at 6:01:06=E2=80=AFPM UTC+3, Scott Lurndal wrote:

>> >Marvell-made NIC, not even in the ultra-high-end FastLinQ 45000 Series.

>> Marvell has a wide range of network interface chips. The OcteonTX=20
>> series have both low-end and high-end entries. The low-end (MIPS and ARM6=
>4)=20
>> Octeons are used in home routers, SOHO routers and some low-end enterpris=
>e=20
>> routers. The high-end OcteonTx are used in high-end enterprise routers,=
>=20
>> cellular and backbone infrastructure.=20
>>=20

>
>Exactly. My point is that Octeon chips are not used in NICs.

Actually, Octeon chips _are_ used in NICs. See "LiquidIO", very
useful in large data centers.

https://www.marvell.com/products/infrastructure-processors/liquidio-smart-nics/liquidio-ii-smart-nics.html

[Scott Lurndal]

David Brown <david...@hesbynett.no> writes:
>On 06/07/2023 17:52, Michael S wrote:
>> On Thursday, July 6, 2023 at 6:01:06 PM UTC+3, Scott Lurndal wrote:
>>> Michael S <already...@yahoo.com> writes:
>>>> Marvell-made NIC, not even in the ultra-high-end FastLinQ 45000 Series.
>>> Marvell has a wide range of network interface chips. The OcteonTX
>>> series have both low-end and high-end entries. The low-end (MIPS and ARM64)
>>> Octeons are used in home routers, SOHO routers and some low-end enterprise
>>> routers. The high-end OcteonTx are used in high-end enterprise routers,
>>> cellular and backbone infrastructure.
>>>
>>
>> Exactly. My point is that Octeon chips are not used in NICs.
>>
>
>There is steadily more work being done on network cards, in data centres
>and wherever you need to handle a lot of traffic. You might not call
>them "NICs", since they have a full OS on board, but they are cards that
>you put in a server to connect to the network. I don't know the parts
>Scott is talking about (I don't know much about /any/ of these, except
>what I read about on the net and dream of having an excuse to use them).
>
><https://www.servethehome.com/what-is-a-dpu-a-data-processing-unit-quick-primer/>

Here's the list:

https://www.marvell.com/products/data-processing-units.html

[Scott Lurndal]

Or one of those 96-core AMD systems. Run 96 guest operating systems
and guarantee each one 1Gb/sec. Bandwidth allocations controlled by the
hypervisor and data center management software dynamically. Perfect for
cloud operators.

[David Brown]

Or virtual machine hosts with SAN storage for the disk images - you can
never have too much bandwidth for that kind of thing. 100 Gb network is
about 10 GB per second - two good NVM disks could saturate it.

[Bonita Montero]

Am 06.07.2023 um 18:37 schrieb David Brown:

> A dual-port 100 Gb network card is perhaps $800.
> That is well within the budget for a high-end server.

And which server application on a server of any size can saturate
a 100GbE-link ?

[Chris M. Thomasson]

For some reason this is reminding me of the 50,000 concurrent connection
scenario back on nt 4.0 wrt IOCP vs events.

[Bonita Montero]

Am 06.07.2023 um 09:25 schrieb Bonita Montero:

> Even with a billion file descriptors the access time would
> be the same, i.e. O(1).

In theory, but actually there's some cacching effect if the
hashtable fits into the cache:

#include <iostream>
#include <unordered_map>
#include <random>
#include <chrono>

using namespace std;
using namespace chrono;

atomic<size_t> aSum( 0 );

int main()
{
constexpr size_t
TO = 0x100000000,
ROUNDS = 10'000'000;
unordered_map<size_t, size_t> map;
mt19937_64 mt;
for( size_t n = 1, b = 0; n <= TO; n *= 2, b = n )
{
for( size_t i = b; i != n; ++i )
map.emplace( i, i );
uniform_int_distribution<size_t> uid( 0, n - 1 );
size_t sum = 0;
auto start = high_resolution_clock::now();
for( size_t r = ROUNDS; r--; )
sum += map[uid( mt )];
double ns = duration_cast<nanoseconds>( high_resolution_clock::now() -
start ).count() / (double)ROUNDS;
::aSum.fetch_add( sum );
cout << hex << n << ": " << ns << endl;
}
}

This are the results on a AMD 7950X:

1: 6.82776
2: 6.82223
4: 6.82411
8: 6.82552
10: 6.82525
20: 6.82547
40: 6.82349
80: 6.84252
100: 6.84796
200: 10.5177
400: 9.12652
800: 12.9653
1000: 15.0953
2000: 16.1944
4000: 17.4853
8000: 11.3426
10000: 13.4169
20000: 14.9261
40000: 18.6082
80000: 31.3227
100000: 63.0259
200000: 87.1134
400000: 103.188
800000: 141.211
1000000: 200.26
2000000: 308.339
4000000: 178.041
8000000: 225.446
10000000: 514.778
20000000: 223.162
40000000: 230.186
80000000: 834.93
100000000: 202.915

The steps to get to the value are always the same.
But the access time of the memory largely differs
according to the size of the hashtable.

[David Brown]

A file server with a couple of fast SSD's could do it. Remember, these
are 100 Gbps links, not 100 GBps.

If you have a 32 core server, that's an average 3 Gbps per core.

Usually, however, saturation of the link is not the issue - just like
cpu clock speeds, it is often the peaks that matter. You don't
(typically) buy a 100 Gb link because you want to send 36 TB over the
next hour, you buy it so that you can send 1 GB in a tenth of a second.


AMD's latest server chips have 128 cores, and Ampere One's have 192
cores - per socket. Do you think people with a 192 core cpu are going
to be happy with a 10 Gb link? And do you think people would be making
and selling such processors, and servers containing them, if they were
not useful?

Note that I said /high-end/ server. Most servers won't need anything
like that for their application sides. But if you have a cluster, and
storage external to the server, you'll easily find such speeds to be
useful. And plenty of systems in data centres, HPC systems, cloud
hosting, etc., will have links like that - or faster.

[Bonita Montero]

Am 07.07.2023 um 10:03 schrieb David Brown:

> A file server with a couple of fast SSD's could do it. ...

No one uses a fileserver with 12,5GB/s.

> If you have a 32 core server, that's an average 3 Gbps per core.

LOL. You have been bathed to hot by your mother.

[Bonita Montero]

Am 07.07.2023 um 10:03 schrieb David Brown:

> A file server with a couple of fast SSD's could do it.  Remember, these
> are 100 Gbps links, not 100 GBps.

And one thing which just in my mind: file server's don't have any
additional load beyond I/O, so separating the flows wouldn't hurt.

[Scott Lurndal]

Bonita Montero <Bonita....@gmail.com> writes:
>Am 07.07.2023 um 10:03 schrieb David Brown:
>
>> A file server with a couple of fast SSD's could do it. ...
>
>No one uses a fileserver with 12,5GB/s.

Your computing experiences seem very limited.

Our lab fileservers (using 25gb, 40gb and 100gb network adapters)
serve hundreds of high-end multicore servers performing RTL simulations
24x7 using NFS.

<Die kindische Beleidigung verschwand>

[Bonita Montero]

Am 07.07.2023 um 16:05 schrieb Scott Lurndal:

> Our lab fileservers (using 25gb, 40gb and 100gb network adapters)
> serve hundreds of high-end multicore servers performing RTL simulations
> 24x7 using NFS.

I don't believe you at all, that's pure phantasy.
But fileservers don't have high CPU-load anyway.
So manual segementation wouldn't hurt, even more
because in a LAN-segment you have jumbo frames.

[Kalevi Kolttonen]

Bonita Montero <Bonita....@gmail.com> wrote:
> Am 07.07.2023 um 16:05 schrieb Scott Lurndal:
>
>> Our lab fileservers (using 25gb, 40gb and 100gb network adapters)
>> serve hundreds of high-end multicore servers performing RTL simulations
>> 24x7 using NFS.
>
> I don't believe you at all, that's pure phantasy.

Do you happen to have any good reasons why he would lie
about their lab?

br,
KK

[Bonita Montero]

Am 07.07.2023 um 17:53 schrieb Kalevi Kolttonen:

> Do you happen to have any good reasons why he would lie
> about their lab?

100GbE is a backbone-technology, maybe for linking switches
with further lower speed links. For which application would
you need a fileserver which can supply 12,5GB/s ?

[David Brown]

On 07/07/2023 17:30, Bonita Montero wrote:
> Am 07.07.2023 um 16:05 schrieb Scott Lurndal:
>
>> Our lab fileservers (using 25gb, 40gb and 100gb network adapters)
>> serve hundreds of high-end multicore servers performing RTL simulations
>> 24x7 using NFS.
>
> I don't believe you at all, that's pure phantasy.

Do you think Scott is deliberately lying here? That's quite the accusation.

It would appear you know practically nothing about servers or
networking, especially for more demanding uses. That's fine, of course
- no one knows about everything, and most people have little interest in
such things unless they actually need to know about them. However, it
is absurd to suggest that just because /you/ can't imagine what how such
systems might be used, they don't exist. And it is arrogant and
obnoxious to accuse those who /do/ know, and /do/ use such systems, of
lying about it.

[Kalevi Kolttonen]

He did describe his lab setup by saying that there are

"hundreds of high-end multicore servers performing RTL
simulations 24x7 using NFS".

I have no idea what RTL is, it could perhaps be something
to do with CPUs, I don't know. But it is obvious that what
they are doing is no joke.

br,
KK

[James Kuyper]

On 7/7/23 12:14, Kalevi Kolttonen wrote:
...

> He did describe his lab setup by saying that there are
> "hundreds of high-end multicore servers performing RTL
> simulations 24x7 using NFS".
>

> I have no idea what RTL is, ...
Scott can tell us, of course, but it might be one of these:
<https://en.wikipedia.org/wiki/RTL#Electronics>

[Scott Lurndal]

kal...@kolttonen.fi (Kalevi Kolttonen) writes:
>Bonita Montero <Bonita....@gmail.com> wrote:
>> Am 07.07.2023 um 17:53 schrieb Kalevi Kolttonen:
>>
>>> Do you happen to have any good reasons why he would lie
>>> about their lab?
>>
>> 100GbE is a backbone-technology, maybe for linking switches
>> with further lower speed links. For which application would
>> you need a fileserver which can supply 12,5GB/s ?
>
>He did describe his lab setup by saying that there are
>"hundreds of high-end multicore servers performing RTL
>simulations 24x7 using NFS".

RTL (Register Transfer Language) aka HDL (Hardware Description Language)
aka Verilog. A hardware description language used
to design advanced (3nm process in our case) processor chips. With
lots of cores (36 or more) and the aforementioned hardware accelerator
blocks. The simulations actually simulate every gate clock by clock under various
directed and randomized test cases to ensure that the very expensive 3nm
process masks create a working chip. Each block is verified (simulated)
individual, groups of blocks are simulated together and various
full-chip simulations (the entire design) are also run. Then there are
the back-end tasks - floorplanning and pnr (Place and Route) to locate each
RTL block totaling many billion transistors. Then there is timing closure[*],
often with margins in the 10s of picosecond range.

[*] Ensuring that signal propogation in any particular combinatorial
path doesn't exceed the clock interval.

I'd also point out that the major customers for server grade processors
are AWS, Azure, Google and Facebook; all of which use 25/40Gbe(past) or 100Gbe(present)
rack to rack and rack to egress/ingress pipes and at least 10g rack to each server. As
David pointed out, with 192 core processors now available, 192 guest VMs can
easly saturate a 100Gbe link on a server.

Meanwhile, 400Gbe optical PHY modules became available a couple years ago
signaling that 200Gbe and 400Gbe are soon to follow.

[Bonita Montero]

Am 07.07.2023 um 19:00 schrieb Scott Lurndal:

> RTL (Register Transfer Language) aka HDL (Hardware Description Language)
> aka Verilog. A hardware description language used
> to design advanced (3nm process in our case) processor chips. With
> lots of cores (36 or more) and the aforementioned hardware accelerator
> blocks. The simulations actually simulate every gate clock by clock under various
> directed and randomized test cases to ensure that the very expensive 3nm
> process masks create a working chip. Each block is verified (simulated)
> individual, groups of blocks are simulated together and various
> full-chip simulations (the entire design) are also run. Then there are
> the back-end tasks - floorplanning and pnr (Place and Route) to locate each
> RTL block totaling many billion transistors. Then there is timing closure[*],
> often with margins in the 10s of picosecond range.

And why should that need such large and fast transfers ?

[Bonita Montero]

Am 07.07.2023 um 18:02 schrieb David Brown:
> On 07/07/2023 17:30, Bonita Montero wrote:
>> Am 07.07.2023 um 16:05 schrieb Scott Lurndal:
>>
>>> Our lab fileservers (using 25gb, 40gb and 100gb network adapters)
>>> serve hundreds of high-end multicore servers performing RTL simulations
>>> 24x7 using NFS.
>>
>> I don't believe you at all, that's pure phantasy.
>

> Do you think Scott is deliberately lying here? ...

Yes.

[Bonita Montero]

Am 07.07.2023 um 18:14 schrieb Kalevi Kolttonen:

> He did describe his lab setup by saying that there are
> "hundreds of high-end multicore servers performing RTL
> simulations 24x7 using NFS".

Simulations needing constant I/Os with such a speed ???

[Scott Lurndal]

David Brown <david...@hesbynett.no> writes:
>On 07/07/2023 17:30, Bonita Montero wrote:
>> Am 07.07.2023 um 16:05 schrieb Scott Lurndal:
>>
>>> Our lab fileservers (using 25gb, 40gb and 100gb network adapters)
>>> serve hundreds of high-end multicore servers performing RTL simulations
>>> 24x7 using NFS.
>>
>> I don't believe you at all, that's pure phantasy.
>
>Do you think Scott is deliberately lying here? That's quite the accusation.

Indeed.

>
>It would appear you know practically nothing about servers or
>networking, especially for more demanding uses. That's fine, of course
>- no one knows about everything, and most people have little interest in
>such things unless they actually need to know about them. However, it
>is absurd to suggest that just because /you/ can't imagine what how such
>systems might be used, they don't exist. And it is arrogant and
>obnoxious to accuse those who /do/ know, and /do/ use such systems, of
>lying about it.
>
>
>
>> But fileservers don't have high CPU-load anyway.

I would argue that this isn't actually the case. Consider,
for example, file servers that support deduplication.

https://www.netapp.com/data-management/what-is-data-deduplication/

[Kalevi Kolttonen]

Scott Lurndal <sc...@slp53.sl.home> wrote:
> RTL (Register Transfer Language)

When reading GCC documentation many years ago, I came
across this RTL:

https://gcc.gnu.org/onlinedocs/gcc-7.1.0/gccint/RTL.html

Does it have anything to do with the RTL you guys use?

br,
KK

[David Brown]

Since you haven't a clue what RTL simulation involves, while Scott
clearly does, why do you feel qualified to doubt him?

RTL simulation for these kinds of chips are /massive/ tasks. (Note that
the "36 or more cores" refers to the chips designed at Scott's company -
not the servers used for simulation, which will have as many cores as
economically feasible.) These chips will have billions of gates, which
need to be simulated. My guess is that they will be using many
thousands of processor cores spread across hundreds of machines for
doing the work, each machine having perhaps half a terabyte of ram.
Since you are simulating a single chip, each server can only simulate
part of it (in time and space), and you need a great deal of
communication between the machines to accurately model the interactions
between parts. So there will be very fast networking between the servers.

Then there are the files and databases involved, and output files. The
design files will be measured in gigabytes, as will the output files,
and there are hundreds of simulation servers accessing these files from
central repositories. Of course they will need massive bandwidths on
the file servers. And you will want the lowest practical latencies for
the file serving - even though file serving takes relatively little
processing power, you want to spread the different clients around the
cores of the server (with RSS) to minimise latencies. You do not want
everything bottlenecked through one core.

All this is, of course, very expensive. But if it saves a single failed
tape-out and prototype run of the chips, it will pay for itself.

[Bonita Montero]

Am 07.07.2023 um 19:32 schrieb David Brown:

> RTL simulation for these kinds of chips are /massive/ tasks.  (Note that
> the "36 or more cores" refers to the chips designed at Scott's company -
> not the servers used for simulation, which will have as many cores as
> economically feasible.)  These chips will have billions of gates, which
> need to be simulated.  My guess is that they will be using many
> thousands of processor cores spread across hundreds of machines for
> doing the work, each machine having perhaps half a terabyte of ram.
> Since you are simulating a single chip, each server can only simulate
> part of it (in time and space), and you need a great deal of
> communication between the machines to accurately model the interactions
> between parts.  So there will be very fast networking between the servers.

If you would do that with constant I/Os on the data and not in RAM
you would simulate almost nothing. Scott is lying.

[Kalevi Kolttonen]

Bonita Montero <Bonita....@gmail.com> wrote:
> Scott is lying.

I am beginning to understand why some people have
killfiled you. I have never killfiled anybody and
never will, but please stop those ridiculous
accusations immediately. It is utter foolishness.

Based on several wise comments, there is absolutely
no reason to doubt Scott Lurndal's sincerity.
I am sure you are the only one who thinks he is lying.

br,
KK

[Bonita Montero]

Am 07.07.2023 um 20:09 schrieb Kalevi Kolttonen:

> Bonita Montero <Bonita....@gmail.com> wrote:

>> Scott is lying.

> I am beginning to understand why some people have
> killfiled you. I have never killfiled anybody and
> never will, but please stop those ridiculous
> accusations immediately. It is utter foolishness.

Think about the lag between I/O latency and cache- / memory-latency.
Imagine what a calculation's speed on a data set constantly fetched
from I/O would be.

[Kalevi Kolttonen]

No, I will not. Instead I will freely admit that
those demanding CPU simulations involving high-end
hardware devices are totally outside my very small
set of knowledge. It is specialized expert knowlegde
that only a handful of people have. Ordinary folks
never get access to labs like that.

It is obvious that Scott is one of those experts.
Calling him a liar is stone cold crazy... Oh no!

Again, why would he lie? To show off his skills,
or to brag about the workings of his advanced lab?

I seem to remember that Scott has been working in
the CS field ever since the big Burroughs machines
were hot. It means he has *decades of experience*.

This is my last comment concerning this matter.

br,
KK

[Bonita Montero]

Am 07.07.2023 um 20:34 schrieb Kalevi Kolttonen:

> No, I will not. Instead I will freely admit that
> those demanding CPU simulations involving high-end
> hardware devices are totally outside my very small
> set of knowledge. It is specialized expert knowlegde
> that only a handful of people have. Ordinary folks
> never get access to labs like that.

LOL

[Scott Lurndal]

All correct. The output files consist of log files (debug information
from the test bench and test driver - when simulating a CPU, the log
also contains each disassembled instruction, all input and output registers
and any state changed as a result of instruction execution. This state is
compared against a golden reference model each instruction. There are also
optional 'wave' files, which contain a
clock-by-clock list of signal state changes for post-analysis and debugging
using visualization tools.

Simulating one real-time second of a 2Ghz design results in 2 billion
clocks, for which all the signal state data needs to be stored on disk.

>and there are hundreds of simulation servers accessing these files from
>central repositories. Of course they will need massive bandwidths on
>the file servers. And you will want the lowest practical latencies for
>the file serving - even though file serving takes relatively little
>processing power, you want to spread the different clients around the
>cores of the server (with RSS) to minimise latencies. You do not want
>everything bottlenecked through one core.
>
>All this is, of course, very expensive. But if it saves a single failed
>tape-out and prototype run of the chips, it will pay for itself.

Indeed.

"There are many clever tricks that are being used to lower costs
of designing that chip, but the biggest hinderances to bringing
that design to market is the cost of a mask set. On a foundry
process node, at 90nm to 45nm, mask sets cost on the order of
hundreds of thousands of dollars. At 28nm it moves beyond $1M. With
7nm, the cost increases beyond $10M, and now, as we cross the 3nm
barrier, mask sets will begin to push into the $40M range."

https://www.semianalysis.com/p/the-dark-side-of-the-semiconductor

You don't want to have to spin your design.

[Scott Lurndal]

Completely unrelated. Here's a sample:

logic [7:0] smb_ctl;
logic [7:0] smb_sta;
logic [31:0] grb_rdata;
logic csr_inf_rd_err;
logic [31:0] elided;

always_ff @(posedge cclk or negedge crst_l) begin
if (!crst_l) begin
smb_re <= 1'b0;
smb_we <= 1'b0;
csr_blk_rd_en <= 1'b0;
csr_blk_wr_en <= 1'b0;
end else if (inf_csr_we &&
(inf_csr_cmd_code == `SMB_CMD_CTL)) begin
smb_re <= inf_csr_wdata[`SMB_CTL_RD_START];
smb_we <= inf_csr_wdata[`SMB_CTL_WR_START];
csr_blk_rd_en <= inf_csr_wdata[`SMB_CTL_BLK_RD_EN];
csr_blk_wr_en <= inf_csr_wdata[`SMB_CTL_BLK_WR_EN];
end else begin
smb_re <= 1'b0;
smb_we <= 1'b0;
csr_blk_rd_en <= csr_blk_rd_en;
csr_blk_wr_en <= csr_blk_wr_en;
end
end

assign smb_ctl = {4'b0, csr_blk_wr_en, csr_blk_rd_en, 2'b0};
assign smb_sta = {2'b0, csr_inf_rd_err, csr_inf_rd_vld,
last_prcl_st[2:0], smb_prcl_err};

// Write Address
always_ff @(posedge cclk or negedge crst_l) begin
if (!crst_l) begin
smb_waddr <= 32'b0;
end else if (inf_csr_we) begin
if (inf_csr_cmd_code == `SMB_CMD_WADDR0)
smb_waddr[7:0] <= inf_csr_wdata;
if (inf_csr_cmd_code == `SMB_CMD_WADDR1)
smb_waddr[15:8] <= inf_csr_wdata;
if (inf_csr_cmd_code == `SMB_CMD_WADDR2)
smb_waddr[23:16] <= inf_csr_wdata;
if (inf_csr_cmd_code == `SMB_CMD_WADDR3)
smb_waddr[31:24] <= inf_csr_wdata;
end
end

[Chris M. Thomasson]

On 7/7/2023 8:30 AM, Bonita Montero wrote:
> Am 07.07.2023 um 16:05 schrieb Scott Lurndal:
>
>> Our lab fileservers (using 25gb, 40gb and 100gb network adapters)
>> serve hundreds of high-end multicore servers performing RTL simulations
>> 24x7 using NFS.
>
> I don't believe you at all, that's pure phantasy.

Show me your proof that Scott is lying to us all?

;^/

[Chris M. Thomasson]

Humm... Wow, Bonita, wow... ;^o

[Bonita Montero]

Am 07.07.2023 um 21:25 schrieb Chris M. Thomasson:

> Show me your proof that Scott is lying to us all?

It's unplausible that such a simulation isn't done
entirely in RAM but with constant I/O over the network.

[David Brown]

I've tried to explain things to you, as has Scott. Clearly, it is all
so far beyond you that the Dunning Kruger effect has taken over.

Do you think /everyone/ here is lying, with some grand conspiracy theory
designed to annoy you personally? Or could it simply be that others
know more than you?

I don't have anything like these systems in our company. We have a
couple of servers for which 10 Gb is worth the cost and effort - nothing
like Scott's setup. But I have done a lot of weird things in network
setups, and I have designed electronics with a fair bit of networking -
we buy chips from the same kinds of companies that make devices like
Scott's, albeit much more mundane chips at lower speeds. And I have
done programmable logic design - again, on far smaller scales.

What Scott describes is entirely realistic for labs used in designing
devices like these.

[Bonita Montero]

Am 08.07.2023 um 14:51 schrieb David Brown:

> I've tried to explain things to you, as has Scott.  Clearly, it is
> all so far beyond you that the Dunning Kruger effect has taken over.

His reasoning is really 180 degrees opposite of mine,
to an extent that is easily recognizable as fantasy.

[Mut...@dastardlyhq.com]

How do you think online multiplayer RPGs work?

[Bonita Montero]

Am 08.07.2023 um 16:59 schrieb Mut...@dastardlyhq.com:
> How do you think online multiplayer RPGs work?

The only transfer a small amout of state about the player over the
network. This data needs to be small so that it will reach the other
players very soon. So this is totally different than what was claimed.

[Mut...@dastardlyhq.com]

On Sat, 8 Jul 2023 18:01:12 +0200
Bonita Montero <Bonita....@gmail.com> wrote:
>Am 08.07.2023 um 16:59 schrieb Mut...@dastardlyhq.com:
>> How do you think online multiplayer RPGs work?
>
>The only transfer a small amout of state about the player over the

There's also a large amount of state about everything else going on
in the simulated world that has to be uploaded to each player which has to be
kept in sync.

[Chris M. Thomasson]

I think a lot of them (games) use UDP instead of TCP.

[Chris M. Thomasson]

Is this a projection?

[V]

Your face almost impossibly similar to Rando Tomson from city with name Viljandi.




On Monday, July 3, 2023 at 4:09:48 PM UTC+3, David Brown wrote:
> On 03/07/2023 12:42, Frederick Virchanza Gotham wrote:
> >
> > I'm writing a network analysis program, and just today I've found out that it's too slow and missing packets.
> >
> > Ideally, I want the thread that reads from the network card to have exclusive use of one of the CPU cores.
> >
> > I'm looking at the following two functions:
> > sched_setaffinity
> > pthread_setaffinity_np
> >
> > And I note the following in the Linux manual:
> > "For example, by dedicating one CPU to a particular process (i.e., setting the affinity mask of that process to specify a single CPU, and setting the affinity mask of all other processes to exclude that CPU), it is possible to ensure maximum execution speed for that process"
> >
> > I can find lots and lots of code examples to set the affinity for one thread or for one process -- however I cannot find one code example showing me how to exclude all other threads.
> >
> > There's absolutely no point in me restricting my thread to the 1st CPU core if other threads can run on that core too -- in fact that will make my program _slower_ because my thread can't be scheduled on the other cores. Every code sample I can find online seems to just restrict the thread in question without placing any restriction on other threads.
> >
> > Anyone know how this is supposed to work?
> "cpu isolation" is, I believe, the term you want to look for on Google.
> I don't know if this can only be set from boot parameters or configured
> at run-time. You could also search for real-time linux, as that is
> where these features are often used.
>
> You will probably also want to disable some or all power-down modes and
> clock speed switching, as these can give significant variations in
> latencies. And you may need to pin the network card interrupts to a
> processor core too.

[Bonita Montero]

No, a very good knowledge of human nature.

[Bonita Montero]

I'm pretty sure you've never worked on things like that.

[Bonita Montero]

Am 08.07.2023 um 20:45 schrieb Chris M. Thomasson:

> I think a lot of them (games) use UDP instead of TCP.

Of course. That's while you don't need re-transmits because
they would reach the other parties too late anyway.

[Chris M. Thomasson]

Are you some sort of oracle? ;^)

[Keith Thompson]

V <vvvvvvvvvvvvvvvvvvvvvvvvv...@hotmail.com>
writes:

> Your face almost impossibly similar to Rando Tomson from city with name Viljandi.

[...]

"V", I've seen several posts from you here in comp.lang.c++ lately.
None of them have anything to do with C++.

Are you planning to participate in any meaningful way?

--
Keith Thompson (The_Other_Keith) Keith.S.T...@gmail.com
Will write code for food.
void Void(void) { Void(); } /* The recursive call of the void */

[Richard Harnden]

On 08/07/2023 22:57, Keith Thompson wrote:
> V <vvvvvvvvvvvvvvvvvvvvvvvvv...@hotmail.com>
> writes:
>> Your face almost impossibly similar to Rando Tomson from city with name Viljandi.
> [...]
>
> "V", I've seen several posts from you here in comp.lang.c++ lately.
> None of them have anything to do with C++.
>
> Are you planning to participate in any meaningful way?
>

'V' is Kristjan Robam - and no he isn't. Time to update your killfile.

[Chris M. Thomasson]

Actually, I remember way back on my nt 4.0 days, this might of been
around 2001-2002 ish... I would have a single TCP connection for control
then a bunch on UDP connections to send datagrams.

[Bonita Montero]

Am 09.07.2023 um 07:47 schrieb Chris M. Thomasson:

> Actually, I remember way back on my nt 4.0 days, this might of
> been around 2001-2002 ish... I would have a single TCP connection
> for control then a bunch on UDP connections to send datagrams.

I'm pettifogging: you never had UDP-_connections_.

[Chris M. Thomasson]

It was many years ago. Iirc, I would have a single TCP connection for
status and send out datagrams via UDP. It was kind of like the zmodem
protocol. I remember using a multiplex in the TCP connection to embedded
multiple status messages for different sockets. UDP was the main usage.
TCP was there to maintain a lossless comm. UDP datagrams can get lost
and/or duplicated, iirc.

[Chris M. Thomasson]

I remember putting in some state in the datagrams. So, if they arrived
out of order, I could put them back together. Or, they just knew where
to be written to a memory mapped file.

[Chris M. Thomasson]

On 7/9/2023 1:44 AM, Bonita Montero wrote:

Basically, I would try to send a file offset in the datagrams. The
receiver of a single datagram knew where to write it in a memory mapped
file.

[Chris M. Thomasson]

Damn Bonita! You are making my brain think of things I did long ago...

This is why I had a TCP connection to be able to reliably request more
data until the memory mapped file was full via UDP datagrams filling it
up, so to speak...

:^D

[David Brown]

On 08/07/2023 16:19, Bonita Montero wrote:
> Am 08.07.2023 um 14:51 schrieb David Brown:
>
>> I've tried to explain things to you, as has Scott.  Clearly, it is
>> all  so far beyond you that the Dunning Kruger effect has taken over.
>
> His reasoning is really 180 degrees opposite of mine,

Yes - because he knows what he is talking about - he has the computers
there, in his lab - and you haven't a clue. Can you really not see that?

[Chris M. Thomasson]

YIKES! ;^/

[David Brown]

Yes, that would be another type of application where the server would
need very fast network card(s). It's a different case from Scott's,
since online games generally aim for smaller amounts of data at a time
but handle vast numbers of connections (hundreds of thousands, or more)
and also want lower latencies. For Scott's file servers, there are only
hundreds of client machines, but the data files are very large and the
simulations generate massive quantities of data as they go along. But
it is certainly an application for which you want as fast networking as
you can afford on the server.

[Chris M. Thomasson]

Fwiw, I am wondering of anybody else got pissed off at the nt 4.0 client
version only allowing for two concurrent TransmitFile IOCP functions to
be allowed at any one time? The nt 4 server allowed for enough
concurrent TransmitFile's to blow the non-paged memory. Grrr!

[Bonita Montero]

Am 09.07.2023 um 11:41 schrieb Chris M. Thomasson:

> On 7/9/2023 1:44 AM, Bonita Montero wrote:

>> I'm pettifogging: you never had UDP-_connections_.

> It was many years ago. Iirc, ...

UDP is connectionless and stateless.

[Mut...@dastardlyhq.com]

On Sat, 8 Jul 2023 21:25:16 +0200

Bonita Montero <Bonita....@gmail.com> wrote:
>Am 08.07.2023 um 20:28 schrieb Mut...@dastardlyhq.com:
>> On Sat, 8 Jul 2023 18:01:12 +0200
>> Bonita Montero <Bonita....@gmail.com> wrote:
>>> Am 08.07.2023 um 16:59 schrieb Mut...@dastardlyhq.com:
>>>> How do you think online multiplayer RPGs work?
>>>
>>> The only transfer a small amout of state about the player over the
>>
>> There's also a large amount of state about everything else going on
>> in the simulated world that has to be uploaded to each player which
>> has to be kept in sync.
>
>I'm pretty sure you've never worked on things like that.
>

Bonita playbook rule #1: If proven wrong accuse other person of knowing
nothing/being an idiot/lying etc.

Perhaps look in the mirror occasionally.

[Bonita Montero]

If you make realtime transfers you can't afford to make
large transfers because they'd have a large latency until
being received by other parties.

[Scott Lurndal]

Can't say I've ever actually used NT4 for anything. Or any windows
release for that matter. In the NT4 timeframe, I was still using large
IRIX machines and NT was considered a toy. We did have a source license
for NT4 and modified it to use as one of our supported guest OS' in early
hypervisor skunkwork (1998/1999) at SGI, so I was, at the time, quite familiar
with the NTOS portions of windows (which were strikingly similar to the
VAX VMS internals that I worked with in the early 80's).

[David Brown]

On 07/07/2023 19:31, Kalevi Kolttonen wrote:
> Scott Lurndal <sc...@slp53.sl.home> wrote:
>> RTL (Register Transfer Language)
>
> When reading GCC documentation many years ago, I came
> across this RTL:
>
> https://gcc.gnu.org/onlinedocs/gcc-7.1.0/gccint/RTL.html
>
> Does it have anything to do with the RTL you guys use?
>

In both cases, the abbreviation means "Register transfer level". But
they are different kinds of register.

For the compiler, you are talking about processor registers (typically
matching a 32-bit or 64-bit integer, double precision floating point, or
possibly SIMD vector). RTL is then about transforming the program into
virtual operations that are register to register, or register op
register to register.

In logic design, whether it be programmable logic (FPGA's), ASICs, or
full-blown digital logic chips, a "register" is a single storage bit,
and RTL covers input to the register as combinational logic of signals,
clocking, enables, reset lines, and possible asynchronous overrides.
There's a lot that goes on for each single bit here, to describe it and
how it works, to simulate it, and to record its state and output. It's
not just the logical states - timing has to be taken into account too.
And while you can do bulk simulation for things like caches and static
rams, there are still hundreds of millions of gates to simulate in these
big chips.

(There are also other kinds of simulations needed for power usage,
timing closure, fanouts, etc.)

[Chris M. Thomasson]

Just read datagrams from a bound socket.

[Keith Thompson]

David Brown <david...@hesbynett.no> writes:
> On 07/07/2023 19:31, Kalevi Kolttonen wrote:
>> Scott Lurndal <sc...@slp53.sl.home> wrote:
>>> RTL (Register Transfer Language)
>> When reading GCC documentation many years ago, I came
>> across this RTL:
>> https://gcc.gnu.org/onlinedocs/gcc-7.1.0/gccint/RTL.html
>> Does it have anything to do with the RTL you guys use?
>>
>
> In both cases, the abbreviation means "Register transfer level". But
> they are different kinds of register.

[...]

In the gcc documentation, RTL means "Register Transfer Language".

[David Brown]

On 10/07/2023 01:09, Keith Thompson wrote:
> David Brown <david...@hesbynett.no> writes:
>> On 07/07/2023 19:31, Kalevi Kolttonen wrote:
>>> Scott Lurndal <sc...@slp53.sl.home> wrote:
>>>> RTL (Register Transfer Language)
>>> When reading GCC documentation many years ago, I came
>>> across this RTL:
>>> https://gcc.gnu.org/onlinedocs/gcc-7.1.0/gccint/RTL.html
>>> Does it have anything to do with the RTL you guys use?
>>>
>>
>> In both cases, the abbreviation means "Register transfer level". But
>> they are different kinds of register.
> [...]
>
> In the gcc documentation, RTL means "Register Transfer Language".
>

Thanks for the correction.

GCC's RTL is the internal language / notation / data structure at the
register transfer level. In either case, we are talking about
registers, and transfer of data into and out of them - the same name,
and related concepts, but major differences nonetheless when talking
about the inner workings of compilers, and digital hardware designs.
They are close enough for potential confusion, and I hope I've made
things a little clearer for folks here unfamiliar with hardware design
languages.