Temporary prevent preemption?

[TH]

Hey,

I'm writing some latency & speed sensitive routines that are processing large amount of signal data in batches. When I run this in a standalone program the speed is fine. However when I combine this to the rest of software (>1000 goroutines, mainly network code) the speed gets reduced by 2x.

Are there any low-level ways to temporary prevent preemption of a function or a goroutine?

Alternatively I tried to run two piece of software in parallel, via shm+IPC synchronization methods, but futex, posix mutex, pipes, all produced latency jitter from 20us - 200us. If there's lower latency methods or more stable ones, please do recommend.

Thanks

[TH]

Btw.

I forgot to mention that CPU load is typically <2-5% on a 32 core server.

- Running in stand-alone as testing purposes speed is let say 100%

- Running two pieces code separately and connected via IPC methods, speed is also 100%, but IPC latency jitter is a problem

- But running two pieces code together (data processing part as 5 additional goroutines) speed decreases to <50% (No lock contention between the two pieces of code, by analyzing more carefully it seems that processing routine gets interrupted in midway)

Thanks

[Kurtis Rader]

The "asyncpreemptoff" debug option may be useful. See https://pkg.go.dev/runtime. Specifically, try running your program after doing "export GODEBUG=asyncpreemptoff=1". I'll leave it to others to explain why this can be risky; especially in a context like you described.

--
You received this message because you are subscribed to the Google Groups "golang-nuts" group.
To unsubscribe from this group and stop receiving emails from it, send an email to golang-nuts...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/golang-nuts/60220814-5763-4be4-a811-a9a84f64f12dn%40googlegroups.com.

--

Kurtis Rader

Caretaker of the exceptional canines Junior and Hank

[TH]

Hey Kurtis,

Is asynchronous preemption only the ones that are waiting for netpolls? So when a call is waiting on a network receive or send, scheduler will context switch? And when... lets say...  for { i++ } -loop get's preemptied, this would be called synchronous preemption? Or is it vice versa?? Bit confused.

Thanks

[Brian Candler]

Vice-versa. Asynchronous preemption is when the goroutine is *not* waiting in a syscall, but is performing computations.  If it spends too much time working, it will be kicked off and another goroutine given a chance to run on the same thread.