[9fans] Interrupt saturation

[lu...@proxima.alt.za]

The workstation I'm using presently seems to trigger as many
interrupts as stats(1) can display, while the syscall graph is also
extremely busy.

I presume this is anomalous. Killing the single instance of
timesync(1) does not seem to make any difference at all.

Any suggestions to where I should look? I'm not running anything that
would make me suspicious, but I'm going to restart the system just to
check.

++L

[erik quanstrom]

the exact number would be interesting. i've added a field
to '#P/irqalloc' on my kernels that is the count of interrupts
per vector:

; cat '#P/irqalloc'
3 0 0 debugpt
7 0 0 mathemu
8 0 0 doublefault
9 0 0 mathover
14 0 0 fault386
15 0 0 unexpected
16 0 0 matherror
50 18 18822033321 clock
51 19 0 lapicerror
63 31 0 lapicspurious
65 1 2 kbd
73 9 772061679 ether0
81 10 8689592 ether1
89 11 11583013 ether2
97 4 5092 COM1
105 14 5 sdC (ata)

perhaps this would be useful. you can pick it out
of ftp://ftp.quanstro.net/other/kernel.mkfs.bz2
in pc/trap.c.

- erik

[ron minnich]

On Sun, Feb 28, 2010 at 1:55 AM, <lu...@proxima.alt.za> wrote:
> The workstation I'm using presently seems to trigger as many
> interrupts as stats(1) can display, while the syscall graph is also
> extremely busy.

workstation only? I.e. it is not running venti?

ron

[lu...@proxima.alt.za]

> workstation only? I.e. it is not running venti?

Thanks for the chance to follow up. I implemented quanstro's
enhancement to /proc, and this is what I get right now, some
twenty-four hours after starting up:

3 0 0 debugpt
7 0 0 mathemu
8 0 0 doublefault
9 0 0 mathover
14 0 0 fault386
15 0 0 unexpected
16 0 0 matherror

50 18 76160483 clock

51 19 0 lapicerror
63 31 0 lapicspurious

65 1 15761 kbd
73 11 9799735 ether0
81 6 0 floppy
89 3 124777 ac97audio
97 5 0 usbuhci
105 9 0 usbuhci
113 11 0 usbuhci
121 10 0 usbehci
129 15 82 sdD (ata)
137 7 0 lpt
145 12 213171 kbdaux

The stuff Erik displayed looked a lot worse, so maybe it's just that
I'm not used to having stats(1) display a solid block for the
Interrupts. Still, I wish cpu cycles could be better utilised.

And, tes, it's effectively a diskless terminal.

++L

[erik quanstrom]

On Mon Mar 1 12:53:59 EST 2010, lu...@proxima.alt.za wrote:
> > workstation only? I.e. it is not running venti?
>
> Thanks for the chance to follow up. I implemented quanstro's
> enhancement to /proc, and this is what I get right now, some
> twenty-four hours after starting up:
>

> 50 18 76160483 clock

> 65 1 15761 kbd
> 73 11 9799735 ether0

> 89 3 124777 ac97audio

> 145 12 213171 kbdaux
>
> The stuff Erik displayed looked a lot worse, so maybe it's just that
> I'm not used to having stats(1) display a solid block for the
> Interrupts. Still, I wish cpu cycles could be better utilised.
>
> And, tes, it's effectively a diskless terminal.

my machine has been up for 90+ days.

looks like you just have HZ set to 1000.
stats isn't counting on that.

also, 113 ethernet interrupts/sec is a pretty
good clip.

- erik

[lu...@proxima.alt.za]

> my machine has been up for 90+ days.
>

That sure makes a difference :-)

> looks like you just have HZ set to 1000.
> stats isn't counting on that.
>

How does one change that?

> also, 113 ethernet interrupts/sec is a pretty
> good clip.

Yes, that's also a bit steep. Intel chip, maybe I should install the
3Com card I had on the old hardware. I changed from a 900MHz board to
a newer, 2.6GHz device with some (additional) peripherals built in.
Didn't expect a weird stats display.

++L

[erik quanstrom]

> > looks like you just have HZ set to 1000.
> > stats isn't counting on that.
> >
> How does one change that?

i wouldn't suspect that one would want to.
the higher clock rate allows more precise
scheduling. the extra overhead should be
unnoticable on a 2.6ghz processor, except via
stats.

> > also, 113 ethernet interrupts/sec is a pretty
> > good clip.
>
> Yes, that's also a bit steep. Intel chip, maybe I should install the
> 3Com card I had on the old hardware. I changed from a 900MHz board to
> a newer, 2.6GHz device with some (additional) peripherals built in.
> Didn't expect a weird stats display.

i would imagine that you're getting this kind of
interrupt load from the nic because you're getting
a lot of packets.

do you suspect the interrupt load is a problem, or
are you just bothered by stats?

- erik

[lu...@proxima.alt.za]

> do you suspect the interrupt load is a problem, or
> are you just bothered by stats?

The latter, I replaced the workstation hardware bacause of a power
supply fan failure and the most noticeable difference was stats'
behaviour.

In passing, has anyone looked at fixing the uneven width of the last
column in a multi-host stats display? I tried, but it seemed such a
convoluted computation I couldn't figure out how the last column's
width turned out a good few pixels wider than the others. This is
particularly visible with many narrow columns (I have had up to eight
of them).

++L