parallelStream within DAO Transactional method

[Alex Chevelev]

Hi, 

I have onDemand DAO object and Transactional method in it. In this method, first I create in DB parent object and then several child objects. Parent object ID is generated via sequence and then used to creare children. I have collection of children to be insterted and I tried to have something like this to insert them in parallel:

childrenData.parallelStream().

    map(each -> childFactory.create(each)) // creates child objects for insertion

    forEach(each -> anotherDao.insertChild(each)) // actually inserts them

Now I have referential integrity error when insert child saying that id of parent object id is unknown. I suppose it is related with parallel execution and the way jdbi stores the transaction (thread local?).

Am I correct?

Is there any way this parallel create case can work?

Thanks,

Alex

[Matthew Hall]

You cannot combine onDemand with parallel streams.

On demand uses thread locals to keep track of each thread's handle / transaction state.

Parallel stream operations run by default within a shared thread pool owned by the JVM.

So the moment you do something with an on-demand DAO within another thread, you get a brand new handle with no transaction started.

You'll have to either use a sequential stream, or use a normal DBI.open(Class) call to get a standalone dao.

-Matthew

--
You received this message because you are subscribed to the Google Groups "jDBI" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jdbi+unsubscribe@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Alex Chevelev]

Hi

It will be good to mention this in javadocs and docs.

Also I suppose, I can easily get memory leaks if I use onDemand DAO in spawned threads or futures. Good to mention this also.

понедельник, 13 февраля 2017 г., 19:30:08 UTC+4 пользователь Matthew Hall написал:

-Matthew

To unsubscribe from this group and stop receiving emails from it, send an email to jdbi+uns...@googlegroups.com.

[Matthew Hall]

You shouldn't have to worry about memory leaks. The on-demand extension generator is pretty careful about closing things out after each call.

I've been working on the developer guide--ill make sure to mention the threading issues we've discussed.

-Matthew

To unsubscribe from this group and stop receiving emails from it, send an email to jdbi+unsubscribe@googlegroups.com.

[Alex Chevelev]

OK. I've got occasional connection limit hits in tests when I switched one feature from sync execution to async tasks in Futures. I decided it might be related to onDemand(). I continue investigating this and write if I'll catch something.

Right because you're working on the guide I come up with this again ) Thanks

понедельник, 17 апреля 2017 г., 18:01:14 UTC+4 пользователь Matthew Hall написал:

[Steven Schlansker]

Just make sure to always close any returned Closeable resources -- for example
if you return a ResultIterator<T> type it must be closed otherwise you leak
a database connection.