Something cool

[Nahum Wild]

Hi,

In trying to get things started here, just wondering if anyone had
done anything cool recently to solve speed problems or to just make
things even faster?

Nahum.

[Jun Yamog]

I did something a few days ago, I can't classify it as cool though.
Essentially the problem was not about speed, but more of limiting
resources if there is a sudden surge of hits. I did get a side effect
of making things faster.

Here are more details about it.

Backgrounder (apologies if its not much details, not sure how much I can state)

- we have a web application that and there is a particular operation
that is slow and resource hungry. This operation is not normally
performed on each request and is cached.
- there is a possibility that such concurrent requests be made, but
the chances are slim and its able to recover from it. Using ab or
jmeter reveals that some request will just simply be not completed,
e.g. lack of resource.
- application is not yet production and I still had some time to tweak it a bit.
- application is done using the environment of java, spring, etc.
- the big operation can't be async.

Solution

The idea is to limit the number of threads that does the processing of
the operation. So the tomcat threads of 200 will only have 20
available threads for the heavy operation.

- Use java package java.util.concurrent (ExecutorService, Executors,
Future, TimeoutException, etc.)
- I just used the simple fixed thread pool by:

threadExecutor = Executors.newFixedThreadPool(processMax);

- Then create a Runnable and submit them to the threadExecutor

Future future = threadExecutor.submit(bigTask);
try {
future.get(TIMEOUT, TimeUnit.MILLISECONDS);
} catch (InterruptedException e) {
// Stop the process from running
bigTask.halt();
}

With the above, we where able to make the operation chug along if such
a massive need of concurrent operation is done. As a side effect of
lowering the maximum allowed big operations, more resource is
available to the server per operation which makes things a bit faster.
Also since it was blocking some threads, the blocked threads is able
to pickup the cached results making things overall faster.

I guess it would have been better if some queue mechanism was made and
operation is done when the queue is filled in a particular threshold.
The speed was not a requirement and I was only worried about getting
an un-intentional DoS. I was looking for a none complex solution in
terms of additional code. So by making some parts slower, I got an
overall faster operation. I am not a java guru (I am language
agnostic, whichever fits the bill), so hopefully I am not doing
something stupid here.

Hopefully this gets things started. Thanks.


Jun