Embbeded DB initialization hanging in WPF app

[Derek Heiser]

Hey all,

I am trying to use RavenDB Embedded for a WPF application and am having issues with initializing the Store. I am currently running v2.5.2750, the latest (stable) from NuGet, and am using Ninject to pass the EmbeddableDocumentStore (as an IDocumentStore) into my ViewModel:

          

  NonAdminHttp.EnsureCanListenToWhenInNonAdminContext(8080);

            Bind<IDocumentStore>().ToMethod(context =>
            {
                var documentStore = new EmbeddableDocumentStore
                {
                    DataDirectory = "Data",
                    UseEmbeddedHttpServer = true
                };
                return documentStore.Initialize();
            }).InSingletonScope();

You can see the issue yourself from my latest commit here: http://goo.gl/EZVQ1l (commit 7209c674e58a68259a0df8e48a18c972a73b8409). The application opens, but if you navigate to Administration and then the Machines menu item, it will freeze.

The only warning in the console is the following:

       

 System.ComponentModel.Composition Warning: 1 : The ComposablePartDefinition 'Raven.Database.Server.Security.OAuth.OAuthClientCredentialsTokenResponder' has been rejected. The composition remains unchanged. The changes were rejected because of the following error(s): The composition produced a single composition error. The root cause is provided below. Review the CompositionException.Errors property for more detailed information.

        1) No exports were found that match the constraint:
            ContractName    Raven.Database.Server.Security.OAuth.IAuthenticateClient
            RequiredTypeIdentity    Raven.Database.Server.Security.OAuth.IAuthenticateClient

        Resulting in: Cannot set import 'Raven.Database.Server.Security.OAuth.OAuthClientCredentialsTokenResponder.AuthenticateClient (ContractName="Raven.Database.Server.Security.OAuth.IAuthenticateClient")' on part 'Raven.Database.Server.Security.OAuth.OAuthClientCredentialsTokenResponder'.
        Element: Raven.Database.Server.Security.OAuth.OAuthClientCredentialsTokenResponder.AuthenticateClient (ContractName="Raven.Database.Server.Security.OAuth.IAuthenticateClient") -->  Raven.Database.Server.Security.OAuth.OAuthClientCredentialsTokenResponder -->  AssemblyCatalog (Assembly="Raven.Database, Version=2.5.0.0, Culture=neutral, PublicKeyToken=37f41c7f99471593")

I've tried constructor-injection, property-injection, creating the EmbeddableDataStore manually inside the constructor w/o injection. Each makes the UI unresponsive after initialization. Without trying to create an EmbeddableDocumentStore, everything works fine. Regardless of whether the putting the Store in the class is a good idea, I am stumped as to why it's completely deadlocked.

Any thoughts?

[Mauro Servienti]

I’m doing exactly the same thing, using Windsor, in a WPF application without any issue. really strange.

The error logged by MEF is an expected error, I’m getting it too, and is te MEF standard behavior.

.m

--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Oren Eini (Ayende Rahien)]

Stop in the debugger, where is it hanging?

Oren Eini

CEO

Hibernating Rhinos

Cellular: +972-52-548-6969

Office:    +972-4-674-7811

Fax:       +972-153-4622-7811

--

[Derek Heiser]

When I call Initialize() it seems to hang there. For instance, see two examples:

FIRST
//Autofac registration of Embedded DB instance
container.RegisterInstance(new EmbeddableDocumentStore() {

DataDirectory = "Data",
UseEmbeddedHttpServer = true

 }.Initialize()).As<IDocumentStore>().SingleInstance();

This hangs right away on application startup.

SECOND
//Autofac registration of Embedded DB instance
container.RegisterInstance(new EmbeddableDocumentStore() {

DataDirectory = "Data",
UseEmbeddedHttpServer = true

 }).As<IDocumentStore>().SingleInstance();

//Constructor for ViewModel
public SystemsViewModel(IDocumentStore documentStore)
{

_documentStore = documentStore;
_documentStore.Initialize();

if(!IsInDesignMode)
{

//some code

}
else
{

//some other code

}

}

In this scenario, I can break on the Initialize() line, but any break points after that never hit (even though the UI does pop back up as if it's released the debugger).


I have checked in a new version of my application that uses Autofac instead of Ninject (commit: e84bae61e3093f0122cb412f52bd7726bff9759f). I was concerned I wasn't using the IoC correctly but it still exhibits the same behavior. Also, the HttpServer isn't starting up so I am pretty confident that something checks out during initialization. I've had someone check this out and run it on their machine and it exhibits the same behavior that I'm seeing.

Thanks for taking a look!

[Derek Heiser]

Tracing it further, it looks like the call below is where things sorta disappear into the ether:

//IndexStorage class

BackgroundTaskExecuter.Instance.ExecuteAllInterleaved(documentDatabase.WorkContext, indexDefinitionStorage.IndexNames, OpenIndexOnStartup);

Basically, nothing after that catches on break points.

[Oren Eini (Ayende Rahien)]

Look at the other threads, Use Debug > Windows > Parallel Stacks

Oren Eini

CEO

Hibernating Rhinos

Cellular: +972-52-548-6969

Office:    +972-4-674-7811

Fax:       +972-153-4622-7811

[Derek Heiser]

I think this might be where the hang is coming into play:

Task.WaitAll(tasks);

This happens in ExecuteAllInterleaved<T> method within DefaultBackgroundTaskExecutor. I took a screenshot of the tasks its trying to execute. What's odd is that the methods are {null}.

See attached screenshot

[Oren Eini (Ayende Rahien)]

What happens in the parallel stack window?

Oren Eini

CEO

Hibernating Rhinos

Cellular: +972-52-548-6969

Office:    +972-4-674-7811

Fax:       +972-153-4622-7811

[Derek Heiser]

It doesn't look like it spins off a bunch of tasks or threads. Here's two screenshots of the Parallel Stack Window. I'll be honest, I've never used it before in my debugging.

[Oren Eini (Ayende Rahien)]

Oh, hell.

That is probably the WPF timer happening.

Can you try initializing RavenDB inside the following code:

 var old = SynchronizationContext.Current;

 SynchronizationContext.SetSynchronizationContext(null);

try

{

   // initialize RavenDB

}

finally

{

 SynchronizationContext.SetSynchronizationContext(old)

}

Oren Eini

CEO

Hibernating Rhinos

Cellular: +972-52-548-6969

Office:    +972-4-674-7811

Fax:       +972-153-4622-7811

[Derek Heiser]

Let me start off by giving a big thanks for investigating this issue. I really appreciate everyone's time and recommendations.

Oren, that did get me past the initialization process, but it's still hanging at some point. The application, when I just F10 through the code, jumps back out to the UI and then breaks within AbstractIndexingExecutor but it appears like it's not finding any work to do (I am not calling anything on Raven, so this is expected) and exits the method. After that, the UI is just frozen again.

One other strange thing I notice is that, after I quit the application, sometimes I can't restart it again because my View.vshost.exe is locked by another process. It's as if some thread or something is holding onto the process after I try and exit.

Lastly, localhost:8080 is not responsive. So I don't feel like it's something unique to the UI, but almost like the UI and Raven are stuck in a deadlock.

I don't want to waste too much more of your time. I'd like this to work as a nice embedded data store, but it seems like ModernUI and Raven aren't meant to be. Or, more likely, something about my specific implementation is causing a conflict between the packages.

Thanks again

[Oren Eini (Ayende Rahien)]

Can you handle initialization in a background thread? 

Oren Eini

CEO

Hibernating Rhinos

Cellular: +972-52-548-6969

Office:    +972-4-674-7811

Fax:       +972-153-4622-7811

[Tobi]

Am 06.02.2014 06:20, schrieb Derek Heiser:

> I don't want to waste too much more of your time. I'd like this to work
> as a nice embedded data store, but it seems like ModernUI and Raven
> aren't meant to be. Or, more likely, something about my specific
> implementation is causing a conflict between the packages.

I'm using RavenDB within a WPF application from the very beginning and
never encountered such problems.

Can you share a piece of code to reproduce this behaviour?

Tobias

[Derek Heiser]

@Tobi: Yes, I posted my GitHub repo (MvvmTest) here -- goo.gl/EZVQ1l. The last two commits (e84bae61e3093f0122cb412f52bd7726bff9759f and 7209c674e58a68259a0df8e48a18c972a73b8409) are built with AutoFac and Ninject, respectively to generate the EmbeddableDataStore. Both crash when you navigate to the "Systems" page, under "Administration". It's a very simple test application.

[Tobi]

Ok. I can reproduce this.

You inject IDocumentStore into SystemsViewModel and there you call
_documentStore.Initialize();

So the documentstore basically gets initialized, when this WPF binding
expression is resolved:

DataContext="{Binding Source={StaticResource Locator},
Path=SystemsViewModel}"

I'm not quite sure, why this makes the WPF thread hang (still trying to
figure this out).

But in general I would advice to simply initialize the document store
when starting up the application. If I do this e.g. in
Application.OnStartup() it works just fine.

Tobias

Am 06.02.2014 15:10, schrieb Derek Heiser:

> @Tobi: Yes, I posted my GitHub repo (MvvmTest) here -- goo.gl/EZVQ1l

> <http://goo.gl/EZVQ1l>. The last two commits

[Tobi]

Am 06.02.2014 16:23, schrieb Tobi:

> So the documentstore basically gets initialized, when this WPF binding
> expression is resolved:
>
> DataContext="{Binding Source={StaticResource Locator},
> Path=SystemsViewModel}"
>
> I'm not quite sure, why this makes the WPF thread hang (still trying to
> figure this out).

Ok. The reason this is happening is, that ModernUI in
LoadContentAsync() starts a Task on the UI scheduler, which triggers
the creation of the SystemsViewModel and the DocumentStore initialization.

So for documentStore.Initialize() the current task scheduler is the UI
scheduler. If a Task is created with:

Task.Factory.StartNew(() =>
{
while (true)
{
Thread.Sleep(100);
}
});

Then the default scheduler chossen will be TaskScheduler.Current. And I
guess somewhere in RavenDB a Task is started on the UI scheduler like
this, causing the deadlock.

Simply explicitly choosing TaskScheduler.Default in RavenDB should fix
this issue too.

I'm trying to figure out, where this happens in RavenDB.

Tobias

[Tobi]

Ok, seems there it is:

DocumentStore.cs:

backgroundTaskScheduler = configuration.CustomTaskScheduler ??
TaskScheduler.Current;

To proof I'm, right, just try this:

Task.Factory.StartNew(() =>
{
_documentStore.Initialize();
}, CancellationToken.None, TaskCreationOptions.None, TaskScheduler.Default)
.Wait();
or this:


var embeddableDocumentStore = new EmbeddableDocumentStore()
{
DataDirectory = "Data",
UseEmbeddedHttpServer = true,
};
embeddableDocumentStore.Configuration.CustomTaskScheduler =
TaskScheduler.Default;
containerBuilder.RegisterInstance(embeddableDocumentStore).As<IDocumentStore>().SingleInstance();


I've created an issue:

http://issues.hibernatingrhinos.com/issue/RavenDB-1691

Tobias

[Tobi]

Fixed here:

https://github.com/ayende/ravendb/pull/579

Anyways: You probably should intialize RavenDB at the application
start, not when loading a view - DocumentStore.Initialize() might take
a while.

Tobias

[Kijana Woodard]

Wow. Just wow.

This is awesome work, even if you're not "forum tobi". 😁

--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+unsubscribe@googlegroups.com.

[Tobi]

On 06.02.2014 18:13, Kijana Woodard wrote:

> Wow. Just wow.
>
> This is awesome work, even if you're not "forum tobi". 😁

Thanks, but I've only changed 7 characters in one line of code - probably
the tiniest pull request ever made :-)

Tobias

[Kijana Woodard]

Ummmmm. Doesn't matter. Knowing which 7 characters to change, issuing the pull, AND giving the OP a workaround plus a better direction [init on app start] is pretty cool.

http://usinghumor.com/2008/10/knowing-where-to-hit-the-value-of-expertise/

There is an old story about a multi-million dollar power plant that had mysteriously ground to a halt. All efforts to restart it had failed and an expert was brought in. After studying the problem for a few minutes he took a hammer and hit one of the valves. With a rumble, the plant came back to life. Incredulous glances were shared, grateful cries and high-fives were exchanged. Later, the expert’s bill arrived for the amount of $10,000.00. The outraged executive in charge thought “All he did was hit a valve with a hammer, this bill is ridiculous.” he asked for an itemized breakdown and the consultant responded with a bill that read: “Hitting valve with hammer $10.00. Knowing which valve to hit: $9,990.00.”

The value of expertise.

Tobias

--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.

[Derek Heiser]

Tobias, I am incredibly grateful. In fact, to all that partook in the discussion. Not only did I get my issue resolved, but I learned quite a bit, and perhaps RavenDB has been slightly improved in the process. 

Really excited to get developing! 

Cheers!

[Justin A]

Wow Tobi! Excellent work :)

RavenDb Dance time!

[Ryan Heath]

+1!!!1!@!!😄

// Ryan

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.

[Flavien Montagné]

Hi there !

I was playing around with the new RavenDb version (Build 3528) and some WPF / ModernUI / MvvmLight, basicly the same project structure than Derek (https://github.com/killnine/MvvmTest). 

And it appears that i have the same kind of freeze that Derek got. When i switch to the 2.5 version it works fine. 

Do you have any clue ? 

(For information i just took derek project and upgraded the ravendb version to 3.0  and i got the same freeze)

Thanks.

[Oren Eini (Ayende Rahien)]

Can you try with the latest unstable? I'm pretty sure we already fixed that.

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

On Tue, Dec 30, 2014 at 1:08 PM, Flavien Montagné <fmon...@plasmasoft.fr> wrote:

Hi there !

I was playing around with the new RavenDb version (Build 3528) and some WPF / ModernUI / MvvmLight, basicly the same project structure than Derek (https://github.com/killnine/MvvmTest). 

And it appears that i have the same kind of freeze that Derek got. When i switch to the 2.5 version it works fine. 

Do you have any clue ? 

(For information i just took derek project and upgraded the ravendb version to 3.0  and i got the same freeze)

Thanks.

Le samedi 8 février 2014 00:04:38 UTC+1, Ryan Heath a écrit :

+1!!!1!@!!

// Ryan

On Thursday, February 6, 2014, Kijana Woodard <kijana....@gmail.com> wrote:

Wow. Just wow.

This is awesome work, even if you're not "forum tobi".

To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

[Flavien Montagné]

Hi thanks for the reply, let me wish you a happy new year !

I just tried with the RavenDB Unstable Build 3561 and the problem is not solved 

[Oren Eini (Ayende Rahien)]

Can you show the exact code that you are using?

[Oren Eini (Ayende Rahien)]

Sample project would be great

[Flavien Montagné]

Sure, i join derek's project with RavenDB 3, if you want to switch to RavenDb 2.5 it's in Dependencies/Raven 2.5 (The DocumentStore initiation is in the App.xaml.cs)

if you go to the administration tab, you'll see the freeze.

https://mega.co.nz/#!3J0wkQIY!xtg1ZjbwyRblKiQGSkag0YaDoNSACaNeNMpxCCO9sJA

i hope it'll help, 

thanks

[Oren Eini (Ayende Rahien)]

The hang happens here, and it is very strange.

https://github.com/ayende/ravendb/blob/master/Raven.Client.Lightweight/Connection/Implementation/HttpJsonRequest.cs#L191

The problem is that we are calling ConfigureAwait(false), as we should, so there shouldn't be anything there that would cause this to do this.

The issue is here:

  WindowsBase.dll!System.Windows.Threading.DispatcherSynchronizationContext.Wait(System.IntPtr[] waitHandles, bool waitAll, int millisecondsTimeout) Unknown

  mscorlib.dll!System.Threading.SynchronizationContext.InvokeWaitMethodHelper(System.Threading.SynchronizationContext syncContext, System.IntPtr[] waitHandles, bool waitAll, int millisecondsTimeout) Unknown

  [Native to Managed Transition]

  mscorlib.dll!System.Threading.Monitor.Wait(object obj, int millisecondsTimeout, bool exitContext) Unknown

  mscorlib.dll!System.Threading.Monitor.Wait(object obj, int millisecondsTimeout) Unknown

  mscorlib.dll!System.Threading.ManualResetEventSlim.Wait(int millisecondsTimeout, System.Threading.CancellationToken cancellationToken) Unknown

  mscorlib.dll!System.Threading.Tasks.Task.SpinThenBlockingWait(int millisecondsTimeout, System.Threading.CancellationToken cancellationToken) Unknown

  mscorlib.dll!System.Threading.Tasks.Task.InternalWait(int millisecondsTimeout, System.Threading.CancellationToken cancellationToken) Unknown

  mscorlib.dll!System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task task) Unknown

This is even though ConfigureAwait was called.

I tried several ways to go around it, and it seems like it is just stuck, and I'm not sure what can be done about it.

Do you have any ideas?

[Flavien Montagné]

I'm not an expert with threads issues, but if in understand well the problem seem's to come from the SynchronisationContext dispatcher that is called from await.

And the deadlock come from the second await which is trying to execute his method on a thread that is already blocked from the first await ?

[Oren Eini (Ayende Rahien)]

Yes, that is pretty much it.

The issue is that we are aware of this issue, and have taken steps to correct it, but it doesn't work, I'm not sure why

[Flavien Montagné]

And i guess it's only a WPF problem ? due to it's way to manage thread (one thread for the UI and one for the rendering) 

[Oren Eini (Ayende Rahien)]

Yes, that is correct, but again, we _are_ handling it, and I wasn't able to reproduce in a smaller sample app doing the exact same thing.

[Flavien Montagné]

It seems to be linked to ModernUI, i've done a TestProject without MvvmLight / Autofac, only ModernUi and i can reproduce it, i've done the same with MvvmLight and Autofac without ModernUI and RavenDB 3 works like a charm.

Here the ModernUi sample project, hope it can help

https://mega.co.nz/#!7cNlnT6L!9WhRSw3OSYsyEDBSI72kAC8otTMQTWRo0J4ziS4kkqI

[Oren Eini (Ayende Rahien)]

I want able to reproduce with no RavenDB code using:

  public partial class Home : UserControl

  {

      public Home()

      {

          InitializeComponent();

          TestObject lTest = new TestObject();

          lTest.Test = String.Format("Test - {0}", DateTime.Now);

        var client = new HttpClient();

        HttpResponseMessage httpResponseMessage = client.GetAsync("http://google.com").ConfigureAwait(false).GetAwaiter().GetResult();

        Console.WriteLine(httpResponseMessage.ReasonPhrase);

      }

  }

Which is effectively what is going on. I don't know what ModernUI is doing to mess up ConfigureAwait(false).