Howto - or - Feature Request - Process to Core locking

[John S]

My project is currently developing a DSP application that will run
on a dual-cpu X5670 system - having a total of 12 cores/24 threads.

We have an issue we have yet to find a solution for. Our Signal
Processing engines are locked to specific hyperthreads (a total of 8
of the 24 hyperthreads) in an attempt to minimize conflict with other
tasking. Each engine typically uses about 80% of that threads CPU
time.
The problem is, during the 20% time that engine is idle, the
operating system schedule sometimes places other tasks on one of the 8
hyperthreads the engines are using - because the current cpu usage is
approx 0%. This is a problem if the scheduler places another process
on one of the engine's core/hyperthreads while that engine is idle
because when the engine comes out of suspend, it will need the
undivided attention of that core - least it not be able to keep up
with the data stream.
We have tried cpuset and taskset to control affinity, but these
haven't fully resolved the issues. The CPU scheduler will still
utilize an engine's core even though other cores are either unused or
have low useage.
Is there a way to have the CPU scheduler assign processes based on
average CPU usage? If so, this would keep the scheduler from using an
engine's core.

Any suggestions or ideas would be greatly appreciated.

Thanks,
John

[Ersek, Laszlo]

On Wed, 7 Mar 2012, John S wrote:

> We have tried cpuset and taskset to control affinity, but these
> haven't fully resolved the issues. The CPU scheduler will still
> utilize an engine's core even though other cores are either unused or
> have low useage.

I may be misunderstanding this: have you tried restricting everything else
to the complement CPU set?

(Eg. find the one script that's used to start "everything" in the system
when it reaches the preferred multi-user runlevel. (On Debian
/etc/init.d/rc seems possible.) Use "taskset -p complement_mask $$" in it;
it's children will inherit the affinity. Use "taskset mask_i engine i"
(like now) to start a given engine instance.)

Laszlo

[John S]

Will give it a try. Thanks

John