Performance in Retlang 0.4.0.4

[Robert M.]

Hi,

I've attached a sample Retlang app (RetTests.zip) conceived 100% for
learning based on the sample app for solving quadratic equations. I've
tried doing some performance measurement against it using a simple
Stopwatch and I found quite surprising results considering the claims
on the project wiki saying that Retlang can process millions of
messages per second using IChannel.
For example, processing 2500000 messages using the attached
application produces a max of 170 000 msg/s.

Since at some point in time I've played with older versions of
Retlang, I've checkout from svn "PRE-Fiber-API" tag and I've run
PerfTests unit tests which indeed performed ~2600000 messages/sec on
my machine using IProcessContext instead of IChannel.

I know that tests are not at all similar but the difference is quite
large. Do you have any ideas why this performance gap between the
current version and the older versions? Is there something in my
sample app which can be optimized to scale up the speed?

I've also attached RetTestsANTSProfilerResults.zip containing a
profiling of the app using ANTS Profiler for performance measurements.

Thanks,
Robert

PS. Both tests where done in release mode. The machine is a dual core
machine at 2.4 Ghz for each core.

[Mike Rettig]

Are you throttling your fiber threads?

A tight publish loop will cause the queues to expand rapidly and the clr will thrash as it allocates memory. You can fix this by placing some bounds on the queues.

            ActionExecutor executor = new ActionExecutor();

            executor.MaxDepth = 10000;

            executor.MaxEnqueueWaitTime = 1000;

            IFiber bus = new ThreadFiber(executor);

I just ran same perf tests against trunk and the numbers look fine.

Mike

[Mike Rettig]

I also ported the old perf test to trunk.

http://code.google.com/p/retlang/source/browse/trunk/src/RetlangTests/PerfTests.cs

Mike

[Robert M.]

I've modified the source code (and re-attached it as
RetTestsWithConstrainedActionExecutor.zip) using bounds for
ActionExecutor as you suggested. Although the performance improved - I
am now obtaining 320000 msg/s - it is not even near to the speed
achieved in your updated perf test. Below you can notice that on my
machine your test achieves 2mil msg/s for 2500000 events.

------ Test started: Assembly: RetlangTests.dll ------

Elapsed: 1227 Events: 2500000
Avg/S: 2037489.81255094

1 passed, 0 failed, 0 skipped, took 1.66 seconds.


Could you please check/run my app on your machine and compare the
results?

Regards,
Robert

[Robert M.]

I found something new: if you use another data type beside int for
message type, event in your PerfTests.cs PointToPointPerfTest, the
average speed will drop from 2 mil msg/s to around 320 000 msg/s as in
my application. Take a look at the code and try run it for a "string"
message type.

ActionExecutor executor = new ActionExecutor();

executor.BatchExecutor = new PerfExecutor();

executor.MaxDepth = 10000;
executor.MaxEnqueueWaitTime = 1000;

using (IFiber bus = new ThreadFiber(executor))
{
bus.Start();
IChannel<string> channel = new Channel<string>();


AutoResetEvent reset = new AutoResetEvent(false);
Action<string> onMsg = delegate(string msg)
{
if (msg == "finish")
{
reset.Set();
}
};
channel.Subscribe(bus, onMsg);
using (new PerfTimer(max))
{
for (int i = 0; i < max; i++)
{
channel.Publish("msg");
}
channel.Publish("finish");
reset.WaitOne(30000, false);

[Mike Rettig]

I don't see anything attached. I don't think this list allows attachments. 

Yes, I do see a performance degradation after switching to string. I'm getting about 500k per second on a dual core. With these high message rates, performance bottlenecks can be very subtle. With c# generics, it is possible that the CLR optimizes differently for the value types. I added another perf test using a struct and the performance was similar to the original int test. 

I haven't found the exact bottleneck in this case, but I don't think it is Retlang. The message type makes no difference to the framework. Do you have a particular use case that requires 1 million messages per second?

Mike

[Robert M.]

In an effort to learn Erlang and put it side by side with .NET
capabilities, I was comparing a small Erlang program doing the same
thing with this Retlang test app. This was my use case.

I've read an article about Jetlang capabilities compared with another
Actor frameworks. In the article, Jetlang performed extremely well
with other data types than int and I thought to Retlang could be
alike.

500K msg/s is very good, no doubt about it, but I assume that probably
there is just a glitch somewhere which doesn't allow full speed as in
the value types case.

I understand that the message rate in real-world apps might vary
drastically from tests rates due to more elaborated message processing
and locking/waiting with other threads, but is good to know that we
are dropping from 1-2M msg/s to a lower rate than 500K msg/s. :-)

I might be far quite far fetched, but this is my *feeling* of course
(and a *feeling* must not be considered as a replacement for real
world measurements)

Regards,
Robert

[Robert M.]

By the way, you cannot attach files per message; only at group level.
All files are showing up in group's home page or if you switch to
"Files" tab on the top right of each thread.

[mrettig]

After some research I found the bottleneck. For some reason, clr
performance is terrible for methods that combine generics and
delegates. I posted code to MSDN that reproduces the problem. Others
have reproduced the problem and agree that the performance is not
acceptable.

Bug Forum thread:
http://social.msdn.microsoft.com/Forums/en-US/clr/thread/dd66d965-73cc-4d7d-9fe6-3b9bd3c55c57

Support Issue with Microsoft
https://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=402282

Unfortunately, I doubt this issue will be fixed anytime soon. If the
issue is critical for you, please add your validation and rating to
the support ticket.

Thanks,

Mike Rettig