entity framework saga persistence

[Jeff Posey]

I'm trying to use the EntityFrameworkSagaRepository and when I run single instances of the saga I can see the sagas persisting and completing. If I do many I start getting the following error:

System.Data.Entity.Core.EntityException: The underlying provider failed on Rollback. ---> System.ArgumentNullException: Value cannot be null.

Parameter name: connection

any ideas why?

[Sergii K]

I had similar exception message. In my case it was because of transaction deadlock. It may happen, for example, when you using saga which CorrelationId correlates with some message property (like EntityId) and two message with same property value were received simultaneously.
 
четверг, 3 марта 2016 г., 20:34:04 UTC+2 пользователь Jeff Posey написал:

[Chris Patterson]

I actually thought this was fixed, because of the way EF discards "unsaved changes." Are you on a current version of MT?

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/8503ac10-3fae-468c-9f6e-6705fb5870bd%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Dmitry Mikhaylov]

Hi,

I see same issue running saga consumer with 10 items in queue. It is actually topic not a queue with Azure Service Bus. When saga start processing those messages in happened to persist 3 of 10.

MassTransit version 3.2.4.

Please advise for solution here.

On Monday, 21 March 2016 23:24:39 UTC+2, Chris Patterson wrote:

I actually thought this was fixed, because of the way EF discards "unsaved changes." Are you on a current version of MT?

On Mon, Mar 21, 2016 at 1:32 PM, Sergii K <serg.c...@gmail.com> wrote:

I had similar exception message. In my case it was because of transaction deadlock. It may happen, for example, when you using saga which CorrelationId correlates with some message property (like EntityId) and two message with same property value were received simultaneously.
 
четверг, 3 марта 2016 г., 20:34:04 UTC+2 пользователь Jeff Posey написал:

I'm trying to use the EntityFrameworkSagaRepository and when I run single instances of the saga I can see the sagas persisting and completing. If I do many I start getting the following error:

System.Data.Entity.Core.EntityException: The underlying provider failed on Rollback. ---> System.ArgumentNullException: Value cannot be null.

Parameter name: connection

any ideas why?

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

[Dmitry Mikhaylov]

Looks like if I specify isolation level creating repository the issue is gone. I set Isolation Level RepeatableRead for now. Will check would Serializable do the same thing.

[Chris Patterson]

For sagas, serializable should always be used. This needs to be documented, as it's apparently only shown in the samples.

On Tue, Apr 5, 2016 at 9:21 AM, Dmitry Mikhaylov <dmitry.pyroj...@gmail.com> wrote:

Looks like if I specify isolation level creating repository the issue is gone. I set Isolation Level RepeatableRead for now. Will check would Serializable do the same thing.

On Tuesday, 5 April 2016 16:26:35 UTC+3, Dmitry Mikhaylov wrote:

Hi,

I see same issue running saga consumer with 10 items in queue. It is actually topic not a queue with Azure Service Bus. When saga start processing those messages in happened to persist 3 of 10.

MassTransit version 3.2.4.

Please advise for solution here.

On Monday, 21 March 2016 23:24:39 UTC+2, Chris Patterson wrote:

I actually thought this was fixed, because of the way EF discards "unsaved changes." Are you on a current version of MT?

On Mon, Mar 21, 2016 at 1:32 PM, Sergii K <serg.c...@gmail.com> wrote:

I had similar exception message. In my case it was because of transaction deadlock. It may happen, for example, when you using saga which CorrelationId correlates with some message property (like EntityId) and two message with same property value were received simultaneously.
 
четверг, 3 марта 2016 г., 20:34:04 UTC+2 пользователь Jeff Posey написал:

I'm trying to use the EntityFrameworkSagaRepository and when I run single instances of the saga I can see the sagas persisting and completing. If I do many I start getting the following error:

System.Data.Entity.Core.EntityException: The underlying provider failed on Rollback. ---> System.ArgumentNullException: Value cannot be null.

Parameter name: connection

any ideas why?

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/8503ac10-3fae-468c-9f6e-6705fb5870bd%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/3e2eb7a8-81cc-4140-bd52-a006ee17468a%40googlegroups.com.

[Dmitry Mikhaylov]

With serializable I get the rollback error. Looks like EF transaction is shared across concurrent calls and gets messed up. Trying to find solution.

On Tuesday, 5 April 2016 20:29:15 UTC+3, Chris Patterson wrote:

For sagas, serializable should always be used. This needs to be documented, as it's apparently only shown in the samples.

On Tue, Apr 5, 2016 at 9:21 AM, Dmitry Mikhaylov <dmitry.pyroj...@gmail.com> wrote:

Looks like if I specify isolation level creating repository the issue is gone. I set Isolation Level RepeatableRead for now. Will check would Serializable do the same thing.

On Tuesday, 5 April 2016 16:26:35 UTC+3, Dmitry Mikhaylov wrote:

Hi,

I see same issue running saga consumer with 10 items in queue. It is actually topic not a queue with Azure Service Bus. When saga start processing those messages in happened to persist 3 of 10.

MassTransit version 3.2.4.

Please advise for solution here.

On Monday, 21 March 2016 23:24:39 UTC+2, Chris Patterson wrote:

I actually thought this was fixed, because of the way EF discards "unsaved changes." Are you on a current version of MT?

On Mon, Mar 21, 2016 at 1:32 PM, Sergii K <serg.c...@gmail.com> wrote:

I had similar exception message. In my case it was because of transaction deadlock. It may happen, for example, when you using saga which CorrelationId correlates with some message property (like EntityId) and two message with same property value were received simultaneously.
 
четверг, 3 марта 2016 г., 20:34:04 UTC+2 пользователь Jeff Posey написал:

I'm trying to use the EntityFrameworkSagaRepository and when I run single instances of the saga I can see the sagas persisting and completing. If I do many I start getting the following error:

System.Data.Entity.Core.EntityException: The underlying provider failed on Rollback. ---> System.ArgumentNullException: Value cannot be null.

Parameter name: connection

any ideas why?

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/8503ac10-3fae-468c-9f6e-6705fb5870bd%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

[Chris Patterson]

That would be a strange case of sharing a DbContext, versus providing a separate DbContext for each message. The DbContext cannot be shared, if I recall my understanding of how EF works.

On Tue, Apr 5, 2016 at 1:24 PM, Dmitry Mikhaylov <dmitry.pyroj...@gmail.com> wrote:

With serializable I get the rollback error. Looks like EF transaction is shared across concurrent calls and gets messed up. Trying to find solution.

On Tuesday, 5 April 2016 20:29:15 UTC+3, Chris Patterson wrote:

For sagas, serializable should always be used. This needs to be documented, as it's apparently only shown in the samples.

On Tue, Apr 5, 2016 at 9:21 AM, Dmitry Mikhaylov <dmitry.pyroj...@gmail.com> wrote:

Looks like if I specify isolation level creating repository the issue is gone. I set Isolation Level RepeatableRead for now. Will check would Serializable do the same thing.

On Tuesday, 5 April 2016 16:26:35 UTC+3, Dmitry Mikhaylov wrote:

Hi,

I see same issue running saga consumer with 10 items in queue. It is actually topic not a queue with Azure Service Bus. When saga start processing those messages in happened to persist 3 of 10.

MassTransit version 3.2.4.

Please advise for solution here.

On Monday, 21 March 2016 23:24:39 UTC+2, Chris Patterson wrote:

I actually thought this was fixed, because of the way EF discards "unsaved changes." Are you on a current version of MT?

On Mon, Mar 21, 2016 at 1:32 PM, Sergii K <serg.c...@gmail.com> wrote:

I had similar exception message. In my case it was because of transaction deadlock. It may happen, for example, when you using saga which CorrelationId correlates with some message property (like EntityId) and two message with same property value were received simultaneously.
 
четверг, 3 марта 2016 г., 20:34:04 UTC+2 пользователь Jeff Posey написал:

I'm trying to use the EntityFrameworkSagaRepository and when I run single instances of the saga I can see the sagas persisting and completing. If I do many I start getting the following error:

System.Data.Entity.Core.EntityException: The underlying provider failed on Rollback. ---> System.ArgumentNullException: Value cannot be null.

Parameter name: connection

any ideas why?

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/8503ac10-3fae-468c-9f6e-6705fb5870bd%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/3e2eb7a8-81cc-4140-bd52-a006ee17468a%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/17310070-ce11-46dc-b3b3-6d16a2ac14b1%40googlegroups.com.

[Dmitry Mikhaylov]

You're right. DbContext must not be shared in multi threaded environment. And I see that db context created by from SagaDbFactory, it is wrapped by using statement.

However transaction scope that is created out of the context might be shared by local DTC. Not sure is it the case, though. 

BTW, I'm using localdb at this moment for persistence.

[Dmitry Mikhaylov]

Don't think that my idea from previous post is correct...

I took EntityFrameworkSagaRepository, added as custom one to my project for debugging purpose, customized it not to be generic, and found the the issue is no longer reproducible.

Chris, you said that you fixed that previous, can you please give a hint where is the code that you changed for the fix.

Thank you,

Dmytro

[Dmitry Mikhaylov]

Hello,

Apparently I have not got to the bottom of this. My latest progress, that with same code and IsolationLevel.Serializable I have connection closed in underlying transaction on Rollback.

Playing around with code, changing awaiting there found that if I get rid of await on policy.Missing and use "policy.Missing(context, missingSagaPipe).Wait()" (EntityFrameworkSagaRepository.cs Line: 166), I can keep isolation level and don't have error on rollback.

That gives performance penalty, but I get more reliable code now.

Will keep you posted on my research here.

If I get a free minute, I'll do sample isolated app for reproducing. Let me know if you're interesting in it.

Regards,

Dmytro

[Chris Patterson]

It really sounds like you're sharing a DbContext, and not creating a unique one for the saga repository to use for each message.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/be1018fb-e765-4803-986c-9731c21f1359%40googlegroups.com.

[Dmitry Mikhaylov]

I'm using SagaDbFactory and EntityFrameworkSagaRepository. I see that the repository creates new context using the factory.

On Thursday, 7 April 2016 01:21:33 UTC+3, Chris Patterson wrote:

It really sounds like you're sharing a DbContext, and not creating a unique one for the saga repository to use for each message.

On Wed, Apr 6, 2016 at 5:57 AM, Dmitry Mikhaylov <dmitry.pyroj...@gmail.com> wrote:

Hello,

Apparently I have not got to the bottom of this. My latest progress, that with same code and IsolationLevel.Serializable I have connection closed in underlying transaction on Rollback.

Playing around with code, changing awaiting there found that if I get rid of await on policy.Missing and use "policy.Missing(context, missingSagaPipe).Wait()" (EntityFrameworkSagaRepository.cs Line: 166), I can keep isolation level and don't have error on rollback.

That gives performance penalty, but I get more reliable code now.

Will keep you posted on my research here.

If I get a free minute, I'll do sample isolated app for reproducing. Let me know if you're interesting in it.

Regards,

Dmytro

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

[Chris Patterson]

A standalone reproducible test would really help here, as I can't seem to recreate it myself.

On Thu, Apr 7, 2016 at 11:18 AM, Dmitry Mikhaylov <dmitry.pyroj...@gmail.com> wrote:

I'm using SagaDbFactory and EntityFrameworkSagaRepository. I see that the repository creates new context using the factory.

On Thursday, 7 April 2016 01:21:33 UTC+3, Chris Patterson wrote:

It really sounds like you're sharing a DbContext, and not creating a unique one for the saga repository to use for each message.

On Wed, Apr 6, 2016 at 5:57 AM, Dmitry Mikhaylov <dmitry.pyroj...@gmail.com> wrote:

Hello,

Apparently I have not got to the bottom of this. My latest progress, that with same code and IsolationLevel.Serializable I have connection closed in underlying transaction on Rollback.

Playing around with code, changing awaiting there found that if I get rid of await on policy.Missing and use "policy.Missing(context, missingSagaPipe).Wait()" (EntityFrameworkSagaRepository.cs Line: 166), I can keep isolation level and don't have error on rollback.

That gives performance penalty, but I get more reliable code now.

Will keep you posted on my research here.

If I get a free minute, I'll do sample isolated app for reproducing. Let me know if you're interesting in it.

Regards,

Dmytro

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/be1018fb-e765-4803-986c-9731c21f1359%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/549933eb-1e46-4e55-9242-4646c74bacdc%40googlegroups.com.

[Dmitry Mikhaylov]

Hello Chris,

I've put together some test code that I can reproduce the issue with. You can find it here: https://github.com/PyroJoke/mt-ef-issue

The code pre-populates queue with 50 messages and then starts receiving endpoint.

With that setup, DB actually get's about 20 saga instances created others fails on rollback.

Hope this would help.

Best Regards,

Dmytro

[Chris Patterson]

Perfect, that was the trick, queuing them up. I'll pull it down and try to fix it shortly.

--

You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.

To post to this group, send email to masstrans...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/db4699cd-ebf2-4a6c-8124-378746a001c3%40googlegroups.com.

[Dmitry Mikhaylov]

Hi,

Have you got chance to check the issue?

[Chris Patterson]

Yeah, and I have a fix that works. I think I pushed it to develop, I'll have to verify.

Yeah, this was the fix: https://github.com/MassTransit/MassTransit/commit/12eaf9aed9325c301253c3637042f372e9734a59

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/a31c0613-1071-4a06-91c1-8c96ea305923%40googlegroups.com.

[Dmitry Mikhaylov]

Cool! Thanks!

I will check it with our code and let you know about the result.

[Chris Patterson]

That would be helpful to make sure it's resolved. 

__
Chris Patterson

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-dis...@googlegroups.com.
To post to this group, send email to masstrans...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/masstransit-discuss/1ef10ded-4235-475a-9cdb-4d23236c0df9%40googlegroups.com.

[Jonathan Wong]

I built the develop branch (commit 9af2226) and still got the rollback issue. Can anyone else reproduce or verify the fix?

On Saturday, April 16, 2016 at 12:55:43 PM UTC-7, Chris Patterson wrote:

That would be helpful to make sure it's resolved. 

__
Chris Patterson

On Sat, Apr 16, 2016 at 9:30 AM -0700, "Dmitry Mikhaylov" <dmitry.pyroj...@gmail.com> wrote:

Cool! Thanks!

I will check it with our code and let you know about the result.

On Friday, 15 April 2016 21:21:03 UTC+3, Chris Patterson wrote:

Yeah, and I have a fix that works. I think I pushed it to develop, I'll have to verify.

Yeah, this was the fix: https://github.com/MassTransit/MassTransit/commit/12eaf9aed9325c301253c3637042f372e9734a59

On Fri, Apr 15, 2016 at 7:03 AM, Dmitry Mikhaylov <dmitry.pyroj...@gmail.com> wrote:

Hi,

Have you got chance to check the issue?

On Friday, 8 April 2016 01:51:00 UTC+3, Chris Patterson wrote:

Perfect, that was the trick, queuing them up. I'll pull it down and try to fix it shortly.

-- 

--
You received this message because you are subscribed to the Google Groups "masstransit-discuss" group.

To unsubscribe from this group and stop receiving emails from it, send an email to masstransit-discuss+unsub...@googlegroups.com.