threading issue in cascading.util.ShutdownUtil

[Ezra Epstein]

I'm getting an intermittent error during job termination:

ERROR 2015-07-23 13:55:13.924 26125 --- [ shutdown hooks] cascading.util.ShutdownUtil              : failed executing hook: null, with exception: {}

java.lang.NullPointerException: null

at cascading.util.ShutdownUtil$2.run(ShutdownUtil.java:106)

The class is multithreaded but uses a java.util.PriorityQueue  I think consider using  PriorityBlockingQueue  instead.

Per the javadocs from PriorityQueue:

Note that this implementation is not synchronized. Multiple threads should not access a PriorityQueue instance concurrently if any of the threads modifies the queue. Instead, use the thread-safe PriorityBlockingQueue class.

Or mark the outer static methods:  addHook(), removeHook(), etc. synchronized.

HOW / WHERE does one open a bug report for cascading?

[Chris K Wensel]

would like to understand more about the environment that’s causing this, and the Cascading version

There is only one thread for the shutdown hook and its stored on a static field, and there is only one priority queue, and its stored on a static field. so multi-threading shouldn’t be an issue. 

regardless, poll() seems to be returning a null. 

https://github.com/Cascading/cascading/blob/2.7/cascading-core/src/main/java/cascading/util/ShutdownUtil.java#L106

so it seems there is a race condition on register and run. this would be fixed with proper synchronized blocks, or the Blocking queue and a null check.

let me put something together on this in 3.0

but to answer your other question:

https://github.com/Cascading/cascading/blob/3.0/README.md#contributing-and-reporting-issues

ckw

--
You received this message because you are subscribed to the Google Groups "cascading-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cascading-use...@googlegroups.com.
To post to this group, send email to cascadi...@googlegroups.com.
Visit this group at http://groups.google.com/group/cascading-user.
To view this discussion on the web visit https://groups.google.com/d/msgid/cascading-user/65165ff3-5942-409b-9463-cc75fcc8450f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

—

Chris K Wensel

ch...@wensel.net

[Chris K Wensel]

should see this show up in the next 3.0 wip

https://github.com/cwensel/cascading/commit/a97f8f3685133f6c83fb4c39a2cd048cce392dbb

To view this discussion on the web visit https://groups.google.com/d/msgid/cascading-user/6724A014-7F22-4640-A6E5-7CCBEC003E03%40wensel.net.

For more options, visit https://groups.google.com/d/optout.

—

Chris K Wensel

ch...@wensel.net

[Ezra Epstein]

That should fix it.

This issue is on the 2.7 release.   Is that open to updates?

[Chris K Wensel]

I cherry-picked the fix into wip-2.7, there will be a published wip-32 in 3 hours or so.

i’ll make a 2.7.1 release in a couple/few weeks since this is a minor issue unless more fixes enter the pipeline.

-- 
You received this message because you are subscribed to the Google Groups "cascading-user" group.
To unsubscribe from this group and stop receiving emails from it, send an email to cascading-use...@googlegroups.com.
To post to this group, send email to cascadi...@googlegroups.com.
Visit this group at http://groups.google.com/group/cascading-user.

To view this discussion on the web visit https://groups.google.com/d/msgid/cascading-user/67d94609-dc1c-4e1f-ab40-2843367b6a33%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

—

Chris K Wensel

ch...@wensel.net