GO on multi-processor systems

[joe mcguckin]

I read somewhere that the default # of GO threads is the number of cores of the cpu.

What about where there are multiple cpus?  My servers have 2, 6 core Xeons. With hyper threading, it looks like 24 cores available to Linux.

Will the GO scheduler schedule GO routines on both CPU's?

If the scheduler is running on one core, how does it manage to put GO routines on other cores and/or CPU's? Does it create system threads (or pthreads) on each of the other 

cores and CPU's? Can it pin a thread to a specific core, or are you at the mercy of the OS thread scheduler?

Inquiring minds want to know...

Thanks,

Joe

[Tamás Gulácsi]

AFAIK it will use all cores, that's 24 on you.

It will use 24 system threads to run goroutines (and many more for syscalls),

and it's  at the mercy of the OS thread scheduler :)

[Sam Mortimer]

FYI - Go on linux uses the result of a sched_getaffinity() system call to determine cpu count:

https://github.com/golang/go/blob/e92be18fd8b525b642ca25bdb3e2056b35d9d73c/src/runtime/os_linux.go#L82

[simon place]

seems to me whats relevant is that the core count is 'below' in the software stack, so transparent, so here it will be 24.

but, like all progs, go progs use what they're told about, so you could 'see' less or you COULD be running inside an emulator that mimics, potentially very slowly, any number.

also since the OS is 'just' a program, it too could be running with reserved cores or inside any emulator. (would guess windows will detect and do licence/pricing related restrictions on you though.)

[Kevin Chadwick]

>> What about where there are multiple cpus? My servers have 2, 6 core
>> Xeons. With hyper threading, it looks like 24 cores available to
>Linux.

I know the latest issues also affect hyper threading/SMT but you shoukld consider switching it off or using AMD, if you care about security. OpenBSD was proven right in that regard and yet Linux still kept it insecure/enabled, by default!!!

[joe mcguckin]

Actually, I'd like to turn off all the cpu bug fixes (e.g. row hammer). It's my understanding that there is a significant performance penalty and I don't share my machines

with anyone else, so I'm not concerned with information leaking between processes.

Joe

[Brian Candler]

On Thursday, 11 June 2020 01:45:53 UTC+1, joe mcguckin wrote:

Actually, I'd like to turn off all the cpu bug fixes (e.g. row hammer). It's my understanding that there is a significant performance penalty and I don't share my machines

with anyone else, so I'm not concerned with information leaking between processes.

For Linux, add mitigations=off to the kernel command line. e.g. for Ubuntu put this in /etc/default/grub:

GRUB_CMDLINE_LINUX="mitigations=off"

then run "update-grub" and reboot.

For virtualization-heavy workloads (network simulations) I've seen in excess of 30% loss in performance from the mitigations.

[Kevin Chadwick]

>Actually, I'd like to turn off all the cpu bug fixes (e.g. row hammer).
>
>It's my understanding that there is a significant performance penalty
>and I
>don't share my machines
>with anyone else, so I'm not concerned with information leaking between
>

It is likely more dangerous, than you realise. So they don't receive any data from the Internet?

[joe mcguckin]

Yes, of course they have internet connections, but I don't run virtualization software. It's my understanding that most of these 

bugs have to do with information leaking from one process or VM to another or with a process trying to escape from it's user process into a higher 

privileged one.  If simply being connected to the internet is risky enough that your machine can be compromised, there's more of a problem here than a cpu bug.

[Kevin Chadwick]

On 2020-06-12 00:47, joe mcguckin wrote:
> Yes, of course they have internet connections, but I don't run virtualization
> software. It's my understanding that most of these 
> bugs have to do with information leaking from one process or VM to another or
> with a process trying to escape from it's user process into a higher 
> privileged one.  If simply being connected to the internet is risky enough that
> your machine can be compromised, there's more of a problem here than a cpu bug.

Yes, the original play down by Linus proved to be a bad move, like the SMT
default has/will! Just like embargos measured in months (latest is something
like a year or more!) while smaller players are kept in the dark for no good
reason. Who then have to work under pressure before POC that bad guys have
probably acquired for ages, are being released.

An OS uses privileges for everything!

"https://www.redhat.com/en/blog/thoughts-netspectre"

"NetSpectre proves the danger of jumping to conclusions (although the current
observed access rates to unauthorized data are slow as noted below). The
brilliant team of researchers at TU Graz in Austria have once again demonstrated
a completely novel attack."

"https://misc0110.net/web/files/netspectre.pdf"

[Kevin Chadwick]

>  If simply being connected to the internet is risky enough that
> your machine can be compromised

Incidentally, as the leakage is bits over time. OpenSSH added this defence.

Add protection for private keys at rest in RAM against speculation and memory
sidechannel attacks like Spectre, Meltdown, Rowhammer and Rambleed. This change
encrypts private keys when they are not in use with a symmetic key that is
derived from a relatively large "prekey" consisting of random data (currently 16KB).

"https://www.undeadly.org/cgi?action=article;sid=20190621081455"