A lot of transactions in one unit of work

[Gösta Jonasson]

Hi,

I'm using Hibernate and Warp Persistence and I am running many
"stand-alone" transactions (>20.000) in one unit of work.
This causes performance problems since it seems like the Hibernate
session isn't cleared after each transaction so all updated entities
remain in the session.

How am I supposed to solve this by using Warp Persistence?
Is there a way to clear the session for each transaction or something like that?

Pseudo code:
final WorkManager unitOfWork;
runJob() {
unitOfWork.beginWork();
orderUpdates = readOrderUpdatesFromFile();
for (OrderUpdate orderUpdate : orderUpdates) {
updateOrder(orderUpdate);
}
unitOfWork.endWork();
}

@Transactional
updateOrder(OrderUpdate orderUpdate) {
// Get order from db, update and store it
};

Very thankful for help.

Best regards,
Gösta

[Gösta Jonasson]

Seems like I found a solution to my own problem, even though it
doesn't feel as clean as only using Warp Persistence annotations.
I'm injecting an EntityManager and calling clear() after each transaction.
I would rather have a specific @Transactional annotation
(@TransactionalStandAlone/@Transactional(clearAfter = true)?) that
called clear() on the EntityManager for me after the commit.

Pseudo code (I'm using Guice) of my solution:

@Inject
public JobRunner(EntityManager entityManager) {};

final WorkManager unitOfWork;
runJob() {
unitOfWork.beginWork();
orderUpdates = readOrderUpdatesFromFile();
for (OrderUpdate orderUpdate : orderUpdates) {
updateOrder(orderUpdate);

entityManager.clear();
}
unitOfWork.endWork();
}

@Transactional
updateOrder(OrderUpdate orderUpdate) {
// Get order from db, update and store it
};

2010/11/26 Gösta Jonasson <gos...@gmail.com>:

[Dhanji R. Prasanna]

Hmm, Im not sure I understand the problem clearly enough--is it that the L1 cache is getting too big? Or is it that the data is not being flushed to the database? The latter can be fixed with a periodic call to flush().

For the former, I guess if you're calling clear() that seems to work? The greater question is are you sure that this is the designed use of Hibernate/EntityManager? It may be that you're trying to do something that it's just not quite designed for, and closing and reopening the session periodically may be a valid solution (or making your transactions more coarse grained).

Anyway, I'd say these things are better answered by the Hibernate/JPA folk.

Dhanji.

2010/11/27 Gösta Jonasson <gos...@gmail.com>

--
You received this message because you are subscribed to the Google Groups "warp-core" group.
To post to this group, send email to warp...@googlegroups.com.
To unsubscribe from this group, send email to warp-core+...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/warp-core?hl=en.

[Gösta Jonasson]

Thanks for your reply.

Yes, the L1 cache is getting too big, that's the problem.
I guess the @Transactional does a call to flush() after the method so
that's not a problem.
Clearing the persistence context/session after each transaction works
perfect but it would be nicer if it were an optional part of Warp.

It seems like this use case, one job that does a lot of transactions,
would be quite common.
Just one last question: How would you solve this?
* By creating a new unit of work/session for each order that should be
processed (with one transaction inside).
* "Batching" let's say 1000 order changes per unit of work/session
(with one transaction per order).
* Having one long unit of work/session and clearing it after each
transaction (with one transaction per order) as I'm doing now.

Thanks for your help.
/Gösta

[Dhanji R. Prasanna]

I'd probably have a bunch of sessions. They are not expensive to create and destroy.