threading issue in cascading.util.ShutdownUtil

5 views
Skip to first unread message

Ezra Epstein

unread,
Jul 23, 2015, 5:13:49 PM7/23/15
to cascading-user
I'm getting an intermittent error during job termination:


ERROR 2015-07-23 13:55:13.924 26125 --- [ shutdown hooks] cascading.util.ShutdownUtil              : failed executing hook: null, with exception: {}


java.lang.NullPointerException: null

at cascading.util.ShutdownUtil$2.run(ShutdownUtil.java:106)




The class is multithreaded but uses a java.util.PriorityQueue  I think consider using  PriorityBlockingQueue  instead.

Per the javadocs from PriorityQueue:

Note that this implementation is not synchronized. Multiple threads should not access a PriorityQueue instance concurrently if any of the threads modifies the queue. Instead, use the thread-safe PriorityBlockingQueue class.


Or mark the outer static methods:  addHook(), removeHook(), etc. synchronized.


HOW / WHERE does one open a bug report for cascading?

Chris K Wensel

unread,
Jul 23, 2015, 5:52:51 PM7/23/15
to cascadi...@googlegroups.com
would like to understand more about the environment that’s causing this, and the Cascading version

There is only one thread for the shutdown hook and its stored on a static field, and there is only one priority queue, and its stored on a static field. so multi-threading shouldn’t be an issue. 

regardless, poll() seems to be returning a null. 


so it seems there is a race condition on register and run. this would be fixed with proper synchronized blocks, or the Blocking queue and a null check.

let me put something together on this in 3.0

but to answer your other question:


ckw


--
You received this message because you are subscribed to the Google Groups "cascading-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cascading-use...@googlegroups.com.
To post to this group, send email to cascadi...@googlegroups.com.
Visit this group at http://groups.google.com/group/cascading-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/cascading-user/65165ff3-5942-409b-9463-cc75fcc8450f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

—
Chris K Wensel




Chris K Wensel

unread,
Jul 23, 2015, 6:09:36 PM7/23/15
to cascadi...@googlegroups.com

For more options, visit https://groups.google.com/d/optout.

—
Chris K Wensel




Ezra Epstein

unread,
Jul 23, 2015, 6:36:40 PM7/23/15
to cascading-user, ezra.e...@gmail.com
That should fix it.

This issue is on the 2.7 release.   Is that open to updates?

Chris K Wensel

unread,
Jul 24, 2015, 1:23:26 PM7/24/15
to cascadi...@googlegroups.com, ezra.e...@gmail.com
I cherry-picked the fix into wip-2.7, there will be a published wip-32 in 3 hours or so.

i’ll make a 2.7.1 release in a couple/few weeks since this is a minor issue unless more fixes enter the pipeline.

-- 
You received this message because you are subscribed to the Google Groups "cascading-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cascading-use...@googlegroups.com.
To post to this group, send email to cascadi...@googlegroups.com.
Visit this group at http://groups.google.com/group/cascading-user.

For more options, visit https://groups.google.com/d/optout.

—
Chris K Wensel




Reply all
Reply to author
Forward
0 new messages