JEE Containers

[Russell Hart]

Does anyone have any experience of using Hystrix with WebSphere or any JEE container??  I'm wondering how the thread-pool strategy would work in this case as in my experience these containers usually don't like threads being created outside of their own thread pools.

[Ben Christensen]

That depends on what the reason for "not liking" the threads is. I'm aware of 3 considerations:

1) thread safety

... but that's always the case with threading

2) thread context and state

If threads in the container are expected to pass around a particular state in order for functionality, then it may perhaps be a valid concern and you may need to customize the HystrixConcurrencyStrategy for spawning threads (http://netflix.github.io/Hystrix/javadoc/com/netflix/hystrix/strategy/concurrency/HystrixConcurrencyStrategy.html).

I have never run across an application container that does this by default though, typically that is only application logic which would dictate this. I can't speak to how WebSphere and current generation JEE containers do this as I have used nothing beyond a servlet container for several years.

3) module or application lifecycle

This is I believe the most common reason for the guidance. Launching threads within a container makes it easy to get a module or application in a state that it can not be gracefully shutdown and unloaded. Enterprise containers are often intended for dynamic deployment of one EAR/WAR module without restarting the entire container. Not being able to shut one down would prevent this or cause resource leaks.

To accommodate this scenario, Hystrix added functionality to enable graceful shutdown so a EAR/WAR module in a container can be unloaded. This is the Hystrix.reset() command found here: http://netflix.github.io/Hystrix/javadoc/com/netflix/hystrix/Hystrix.html#reset()

If you call that in your Servlet destroy or application shutdown hook the EAR/WAR module should be able to cleanly shutdown.

I don't know if this fully answers the question but hopefully it helps.

-- 
Ben Christensen
+1.310.782.5511 @benjchristensen

On September 6, 2013 at 2:09:38 PM, Russell Hart (russel...@googlemail.com) wrote:

Does anyone have any experience of using Hystrix with WebSphere or any JEE container??  I'm wondering how the thread-pool strategy would work in this case as in my experience these containers usually don't like threads being created outside of their own thread pools.

--
You received this message because you are subscribed to the Google Groups "HystrixOSS" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hystrixoss+...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Russell Hart]

Thanks for your response, that's very helpful.  I'm basically looking for reassurance about using Hystrix within a JEE container :)  I'll give it a go with just the shutdown hook and see how it runs.

My apps are Spring apps and are already set up with commonj task executors which delegate to the container so maybe changing the HystrixConcurrencyStrategy to spawn threads via them may not be too hard actually.  Might give this a go too.

Thanks

[Baargav Raghavan]

Hey Russell,

Did you get hystrix to work on websphere? I'm currently thinking of implementing the same and was wondering if you ran into any issues. 

[Russell Hart]

I had to pause this work unfortunately.  It's still on my list though.  I came to the conclusion that using the shutdown hook would be fine and didn't need to go as far as creating a custom HystrixConcurrencyStrategy.  But I didn't get as far as trying it properly.

[Jay Boyd]

I realize this is an old thread, but believe this will be of interest to others researching integration with WebSphere:  I found it necessary to use a custom thread pool to ensure J2EE Subject and context is maintained on the worker threads.  Although written years ago, the J2SE5 version of this implementation's java.util.concurrent.ThreadPoolExecutor works seamlessly and plugs in via HystrixConcurrencyStrategy.   I've used this on WebSphere 8.0 and 8.5.5.

[lumi lumich]

Hi Jay, 

Thanx for your note. I'm really interested in your work because we are also thinking about using it in WAS. Could you provide me with some further suggestions/details? 

[Jay Boyd]

Download the linked zip and extract ABConcurrencyTester_JDK5.ear.  From here extract ABConcurrencyUtils.jar and ABConcurrencyUtilsJDK5.jar.   These two jars have the source you need.  From ABConcurrencyUtilsJDK5 take ContextExecutor, ContextExecutorService and WASThreadFactory.  From the ABConcurrencyUtils take IThread, RunnableWrapper, ThreadWrapper, UnableToStartException, WASThreadFactoryBase.   Ensure these classes compile.  This will provide the thread factory required to create a ThreadPoolExecutor for use with Hystrix.   Create your Hystrix Concurrency Strategy:

public class MyHystrixConcurrencyStrategy extends HystrixConcurrencyStrategy {

    /*
     * @see com.netflix.hystrix.strategy.concurrency.HystrixConcurrencyStrategy#wrapCallable(java.util.concurrent.Callable)
     */
    @SuppressWarnings({ "rawtypes", "unchecked" })
    @Override
    public Callable wrapCallable(Callable callable) {
        return new MyHystrixContextCallable(callable);
    }


    /* Set custom ThreadPoolExecutor based on WebSphere Async Bean framework which is used to
     * propagate a user's J2EE Subject onto the worker thread.
     *
     * See  http://www.ibm.com/developerworks/websphere/techjournal/0606_johnson/0606_johnson.html
     * and the classes in lc.util/base  under the com.ibm.lconn.core.concurrent package.
     *
     */
    public ThreadPoolExecutor getThreadPool(final HystrixThreadPoolKey threadPoolKey, HystrixProperty<Integer> corePoolSize, HystrixProperty<Integer> maximumPoolSize,
                                            HystrixProperty<Integer> keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue) {

        InitialContext ctx;
        ThreadPoolExecutor tpe=null;
       
        try {
            ctx = new InitialContext();
            WorkManager wm = (WorkManager)ctx.lookup("wm/default");
            ThreadFactory tf = new WASThreadFactory(wm);
            tpe = new ThreadPoolExecutor(corePoolSize.get(), maximumPoolSize.get(), keepAliveTime.get(), unit, workQueue, tf);
        } catch (NamingException e) {
            LOGGER.error("err.failed.to.set.threadpool", e);
        }
        return tpe;
    }
}

and then you need to register the strategy in your initialization code:

    HystrixPlugins.getInstance().registerConcurrencyStrategy(new MyHystrixConcurrencyStrategy());

Hope this helps.

- Jay

[lumi lumich]

Hi Jay,

Thanx for your quick answer. I will try it and provide you feedback! :)

Bye,
Rocco

[lumi lumich]

Hi Jay,

in the meantime I followed your instructions and placed given classes in my app. I also registered the concurrency strategy within my servlet filter.
My system logs show me that indeed workmanager threadpool is used to provide the worker threads. :) Thank you for your support.

For me there's one question left. If I configure my Hystrix Command with a timeout value according to this Hystrix aborts the application thread and returnes my fallback implementation. In the meanwhile my workerthreads are continuing to work and work and work. I wanna interrupt their work. Do you have experiences/suggestions for me ? 

I tried to call boolean issuccess = workmanager.join(myItems,JOIN_OR, myTimeout) after calling the startWork(...) method... 

And afterwards called release(); which sends interrupt() to the worker:

if (!issucces) {

 workItem.release();

}

I regognized that my worker got interruption signals everytime a new thread was created via ThreadFactory. 

Then I noticed that once created the pool reuses the threads and so my join-method and release() methods were not called anymore.

What do you suggest? As I know one can configure the workmanager in generell with a timeout via admin console. Maybe this is sufficient ? 

I'm also interested in your experiences with WAS 8.5.x and Hystrix in ND Environment. (At the moment I'm testing on a local WAS)

Greetings, Rocco

Am Donnerstag, 30. Oktober 2014 03:30:22 UTC+1 schrieb Jay Boyd:

[Philipp Rossberger]

Hi Rocco,

I know this thread is really old. Still, did you manage to get it working?

Greetings, Philipp

[David G.]

Hi!

Few years later we are also using Hystrix inside apps that run on IBM WAS (mainly 8.5.5). Has anyone got any advice on this or can share their experiences?

We have had no issues, and in fact didn't even use WAS thread factory so far. Recently, we have found some problems with transactions while using JDBC from Hystrix commands running asynchronously (via the .queue() method). It seems the server kills connections (created on the main thread, outside the command) while the command is still using them...

Has anyone found similar problems? Any advice?

Thanks and greetings!