RSB - Esent error when browsing web app. File locked or in use.

[Daventry]

Hi,

 

We're getting below error when accessing a web app. The error disappears after refreshing the page a couple of times. It happens from time to time, not just after a deployment.

 

Has anyone seen this behavior before? BTW we're not deleting the queue folders on startup.

 

And so that you know, these queues are created on a shared drive because we're not allowed to create files/folders on the machine the web app is runnig on. We've already made sure the credentials are correctly set up. As I said before, after refreshing everything works out well.

 

Thanks.

 

Error FileAccessDenied (JET_errFileAccessDenied, Cannot access file, the file is locked or in use)

Description: An unhandled exception occurred during the execution of the current web request. Please review the stack trace for more information about the error and where it originated in the code.

Exception Details: Microsoft.Isam.Esent.Interop.EsentErrorException: Error FileAccessDenied (JET_errFileAccessDenied, Cannot access file, the file is locked or in use)

Source Error:

An unhandled exception was generated during the execution of the current web request. Information regarding the origin and location of the exception can be identified using the exception stack trace below.

Stack Trace:

[EsentErrorException: Error FileAccessDenied (JET_errFileAccessDenied, Cannot access file, the file is locked or in use)]    Microsoft.Isam.Esent.Interop.Api.Fail(Int32 err) +57    Microsoft.Isam.Esent.Interop.Api.JetInit(JET_INSTANCE& instance) +28    Rhino.PersistentHashTable.PersistentHashTable.Initialize() +53   [InvalidOperationException: Could not open cache: \\#####\ICEBus\client_subscriptions.esent\client_subscriptions.esent]    Rhino.PersistentHashTable.PersistentHashTable.Initialize() +207    Rhino.ServiceBus.RhinoQueues.PhtSubscriptionStorage.Initialize() +35    Rhino.ServiceBus.Impl.DefaultServiceBus.Start() +430    Rhino.ServiceBus.Hosting.DefaultHost.Start(String asmName) +21    Rhino.ServiceBus.Hosting.DefaultHost.Start() +110    CS.RAR.ICE.Web.Global.Application_Start(Object sender, EventArgs e) in g:\D\CS.RAR.ICE\CS.RAR.ICE.Web\Global.asax.cs:37   [HttpException (0x80004005): Could not open cache: \\###\ICEBus\client_subscriptions.esent\client_subscriptions.esent]    System.Web.HttpApplicationFactory.EnsureAppStartCalledForIntegratedMode(HttpContext context, HttpApplication app) +9168509    System.Web.HttpApplication.RegisterEventSubscriptionsWithIIS(IntPtr appContext, HttpContext context, MethodInfo[] handlers) +131    System.Web.HttpApplication.InitSpecial(HttpApplicationState state, MethodInfo[] handlers, IntPtr appContext, HttpContext context) +194    System.Web.HttpApplicationFactory.GetSpecialApplicationInstance(IntPtr appContext, HttpContext context) +339    System.Web.Hosting.PipelineRuntime.InitializeApplication(IntPtr appContext) +253   [HttpException (0x80004005): Could not open cache: \\#####\ICEBus\client_subscriptions.esent\client_subscriptions.esent]    System.Web.HttpRuntime.FirstRequestInit(HttpContext context) +9087676    System.Web.HttpRuntime.EnsureFirstRequestInit(HttpContext context) +97    System.Web.HttpRuntime.ProcessRequestNotificationPrivate(IIS7WorkerRequest wr, HttpContext context) +256

[Corey Kaylor]

ASP.NET spins up a second app pool before the first is finished shutting down, therefore it still has a lock on those files.

--
You received this message because you are subscribed to the Google Groups "Rhino Tools Dev" group.
To view this discussion on the web visit https://groups.google.com/d/msg/rhino-tools-dev/-/ae4ZqpYd10cJ.
To post to this group, send email to rhino-t...@googlegroups.com.
To unsubscribe from this group, send email to rhino-tools-d...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/rhino-tools-dev?hl=en.

[Daventry]

So in order to avoid this, should the DefaultHost object be wrapped by a lock and also be disposed in Application_End?

 

Thanks.

[Corey Kaylor]

A mutex may be needed. I don't remember off the top of my head but I believe a lock only applies to a single AppDomain.

To view this discussion on the web visit https://groups.google.com/d/msg/rhino-tools-dev/-/x4NMZV28XnAJ.

[Daventry]

Our bad - looks like we don't need the mutex, we've forgotten about disposing DefaultHost in Application_End. The file handles to the esent files weren't released.

[Daventry]

Hi,
 
We thought we had fix it, still having problems.
 
Looks like it happens when running in IIS with load balance enabled. It is all right when running off our workstations. Have you seen this before?
 
Thanks.

[Daventry]

Hi,

 

Would it be possible that first request comes to a node in the web farm which initialised the Service Bus - i.e creates Rhino Queue folders, etc. - then a second request comes through and it's forwarded to a second node. The second node's AppPool will initialise the Bus again and try to get access to the Rhino Queues ESENT files.

 

Apologies if the scenario above doesn't make any sense - haven't used neither RSB nor web load balance before.

 

If this was the problem, would there be any workaround?

 

Another problem we have here is that our "corporative" platform doesn't allow us to write on the web server file system, hence we have had to create the client queue folders on a shared. Our web.config RSB section looks like below:

 

<rhino.esb>
    <bus threadCount="1" numberOfRetries="1" endpoint="rhino.queues://#web_hostname#/ICE_WEB" name="client"
         path="\\#shared_drive#\ICEBus" />
    <messages>
      <add name="CS.RAR.ICE.Common.Messages" endpoint="rhino.queues://#backend_hostname#/ICE_ETL"   />
    </messages>
  </rhino.esb>

 

Any thoughts?

 

Thanks.

[Corey Kaylor]

Rhino queues path cannot be shared across multiple processes. You would need to have a different share for each one.

--

You received this message because you are subscribed to the Google Groups "Rhino Tools Dev" group.

To view this discussion on the web visit https://groups.google.com/d/msg/rhino-tools-dev/-/_TTp5h4IerQJ.

[Daventry]

Hi,

 

Do you think that setting up the RSB config by C# code and using the machine name as part of the storage path would be a good workaround?

Therefore each node would create its own "client" queues.

 

And just to confirm this, in the Global.asax we have to:

 - 1. Create and start the DefaultHost in Application_Start.

 - 2. Dispose DefaultHost in Application_End.

 

Thanks.

[Miles Waller]

Hi,

BTW, the config files support environment variables, so you can set up an environment variable in global.asax that contains the hostname, and then use it in the config file (I have a personal preference for putting all this stuff into config).

We have this as we wanted to get the queue db in the App_Data folder for the specific web application:

In global.asax:

Environment.SetEnvironmentVariable("WebConsoleAppData", HttpContext.Current.Server.MapPath("~/App_Data/"), EnvironmentVariableTarget.Process);

In web.config:

<rhino.esb>

    <bus path="%WebConsoleAppData%" />

    <messages>...</messages>

</rhino.esb>

Miles

To view this discussion on the web visit https://groups.google.com/d/msg/rhino-tools-dev/-/QT3XeBhnRdQJ.

[Daventry]

Thanks a lot, this will be very handy to us.