Storm High CPU on Idle (0.9.0)

[Michael Ritsema]

I'm seeing high CPU (around 50%) usage in storm when things should be mostly idle. I'm using a JmsSpout off of ActiveMQ with a sleep of 30 ms. However, we are currently testing these inside Virtual Machines that tend to have disk contentions.

A dump from top:

Cpu0  :  2.2%us,  4.1%sy,  0.0%ni, 50.6%id, 43.1%wa,  0.0%hi,  0.0%si,  0.0%st

Cpu1  :  1.1%us,  4.5%sy,  0.0%ni, 34.2%id, 59.9%wa,  0.0%hi,  0.4%si,  0.0%st

It looks like most of the time the CPU is waiting on disk. I'm thinking the situation is that perhaps the 30ms ping just can't keep up with the slow disks very well and this is causing high CPU. Does anyone have any comments or seen similair situations?

I'm using Storm 0.9.0.

Thanks,

-Michael

[Michael Ritsema]

So appears my issue was having too many DRPCStreams going on. I had 24 instances. This took up 50% CPU even in an idle scenario.

-Micael

[Michael Ritsema]

I'm also running LocalDRPC. But it appears that every DRPCStream I add I get another 4-5% CPU hit. I'm not sure if it is a bug or not. It doesn't seem like just adding a stream should take much processing.

I'm limited to VisualVM sampler mode for a profiler, but what I can tell is that it's a factory method in a nio zoo keeper connection class that is taking up all the CPU. If anyone is interested in knowing more about this issue I can provide more info. As for me, I think I'll just punt on this and know that DRPCStreams are known to cost me a big CPU hit.

-Michael