Microsoft.Isam.Esent.Interop.EsentTooManyInstancesException

[Christophe Dubray]

While populating a new production server with tenant data, I encountered a EsentTooManyInstancesException exception (see below). 

Using RavenDB 2.5.2962 running as a service.

Is there a limit of having 1024 databases loaded at the same time? I stopped the population process and waited a few minutes and I noticed that there were about 900 tenants loaded.

I lowered the amount of population processes and it seems to be working fine now.

System.AggregateException: One or more errors occurred. ---> System.InvalidOperationException: Url: "/databases/o3.cbguide-4066.cbischannel-47.culture-en/stats" 

System.AggregateException: One or more errors occurred. ---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> Microsoft.Isam.Esent.Interop.EsentTooManyInstancesException: Cannot start any more database instances 

at Raven.Storage.Esent.TransactionalStorage.CreateInstance(JET_INSTANCE& compactInstance, String name) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Storage\Esent\TransactionalStorage.cs:line 346 

at Raven.Storage.Esent.TransactionalStorage..ctor(InMemoryRavenConfiguration configuration, Action onCommit) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Storage\Esent\TransactionalStorage.cs:line 99 

--- End of inner exception stack trace --- 

at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor) 

at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) 

at System.RuntimeType.CreateInstanceImpl(BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, StackCrawlMark& stackMark) 

at System.Activator.CreateInstance(Type type, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes) 

at System.Activator.CreateInstance(Type type, Object[] args) 

at Raven.Database.Config.InMemoryRavenConfiguration.CreateTransactionalStorage(Action notifyAboutWork) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Config\InMemoryRavenConfiguration.cs:line 958 

at Raven.Database.DocumentDatabase..ctor(InMemoryRavenConfiguration configuration, TransportState transportState) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\DocumentDatabase.cs:line 205 

at Raven.Database.Server.HttpServer.<>c__DisplayClass45.<TryGetOrCreateResourceStore>b__43() in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Server\HttpServer.cs:line 1136 

at System.Threading.Tasks.Task`1.InnerInvoke() 

at System.Threading.Tasks.Task.Execute() 

--- End of inner exception stack trace --- 

at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification) 

at Raven.Database.Server.HttpServer.SetupRequestToProperDatabase(IHttpContext ctx) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Server\HttpServer.cs:line 1038 

at Raven.Database.Server.HttpServer.DispatchRequest(IHttpContext ctx) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Server\HttpServer.cs:line 805 

at Raven.Database.Server.HttpServer.HandleActualRequest(IHttpContext ctx) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Server\HttpServer.cs:line 700 

---> (Inner Exception #0) System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> Microsoft.Isam.Esent.Interop.EsentTooManyInstancesException: Cannot start any more database instances 

at Raven.Storage.Esent.TransactionalStorage.CreateInstance(JET_INSTANCE& compactInstance, String name) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Storage\Esent\TransactionalStorage.cs:line 346 

at Raven.Storage.Esent.TransactionalStorage..ctor(InMemoryRavenConfiguration configuration, Action onCommit) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Storage\Esent\TransactionalStorage.cs:line 99 

--- End of inner exception stack trace --- 

at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor) 

at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) 

at System.RuntimeType.CreateInstanceImpl(BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, StackCrawlMark& stackMark) 

at System.Activator.CreateInstance(Type type, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes) 

at System.Activator.CreateInstance(Type type, Object[] args) 

at Raven.Database.Config.InMemoryRavenConfiguration.CreateTransactionalStorage(Action notifyAboutWork) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Config\InMemoryRavenConfiguration.cs:line 958 

at Raven.Database.DocumentDatabase..ctor(InMemoryRavenConfiguration configuration, TransportState transportState) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\DocumentDatabase.cs:line 205 

at Raven.Database.Server.HttpServer.<>c__DisplayClass45.<TryGetOrCreateResourceStore>b__43() in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Server\HttpServer.cs:line 1136 

at System.Threading.Tasks.Task`1.InnerInvoke() 

at System.Threading.Tasks.Task.Execute()<--- 

---> System.Net.WebException: The remote server returned an error: (500) Internal Server Error. 

at System.Net.HttpWebRequest.GetResponse() 

at Raven.Client.Connection.HttpJsonRequest.ReadJsonInternal(Func`1 getResponse) in c:\Builds\RavenDB-Stable-2.5\Raven.Client.Lightweight\Connection\HttpJsonRequest.cs:line 349 

--- End of inner exception stack trace --- 

at Raven.Client.Connection.HttpJsonRequest.HandleErrors(WebException e) in c:\Builds\RavenDB-Stable-2.5\Raven.Client.Lightweight\Connection\HttpJsonRequest.cs:line 543 

at Raven.Client.Connection.HttpJsonRequest.ReadJsonInternal(Func`1 getResponse) in c:\Builds\RavenDB-Stable-2.5\Raven.Client.Lightweight\Connection\HttpJsonRequest.cs:line 355 

at Raven.Client.Connection.HttpJsonRequest.ReadResponseJson() in c:\Builds\RavenDB-Stable-2.5\Raven.Client.Lightweight\Connection\HttpJsonRequest.cs:line 239 

at Raven.Client.Connection.ServerClient.GetStatistics() in c:\Builds\RavenDB-Stable-2.5\Raven.Client.Lightweight\Connection\ServerClient.cs:line 1855 

at Online3.Infrastructure.RavenDB.DocumentStoreExtensions.EnsureDatabaseIsLoaded(IDocumentStore store, String databaseName, TimeSpan timeout, ILog log, CancellationToken cancellationToken) in c:\BuildAgent\work\35de53147eba9030\Online3\Online3\Infrastructure\RavenDB\DocumentStoreExtensions.cs:line 219 

at Online3.Infrastructure.RavenDB.EnsureDatabase.RavenHelpers.InitializeMasterForStore(CultureInfo languageCulture, String dbName, ReplicationDestination replicationDestination, IDocumentStore masterStore, CancellationToken cancellationToken, ILog log) in c:\BuildAgent\work\35de53147eba9030\Online3\Online3\Infrastructure\RavenDB\EnsureDatabase\RavenHelpers.cs:line 119 

at Online3.RavenPopulator.Commands.EnsureDatabaseCommandHandler.Execute(EnsureDatabaseCommand context) 

at Online3.RavenPopulator.EnsureDatabase.EnsureDatabaseWorker.Work(CancellationToken cancellationToken, IPersistentConsumerConnection rabbit, Nullable`1 timeoutMilliseconds) 

--- End of inner exception stack trace --- 

at Online3.RavenPopulator.EnsureDatabase.EnsureDatabaseWorker.Work(CancellationToken cancellationToken, IPersistentConsumerConnection rabbit, Nullable`1 timeoutMilliseconds) 

---> (Inner Exception #0) System.InvalidOperationException: Url: "/databases/o3.cbguide-4066.cbischannel-47.culture-en/stats" 

System.AggregateException: One or more errors occurred. ---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> Microsoft.Isam.Esent.Interop.EsentTooManyInstancesException: Cannot start any more database instances 

at Raven.Storage.Esent.TransactionalStorage.CreateInstance(JET_INSTANCE& compactInstance, String name) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Storage\Esent\TransactionalStorage.cs:line 346 

at Raven.Storage.Esent.TransactionalStorage..ctor(InMemoryRavenConfiguration configuration, Action onCommit) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Storage\Esent\TransactionalStorage.cs:line 99 

--- End of inner exception stack trace --- 

at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor) 

at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) 

at System.RuntimeType.CreateInstanceImpl(BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, StackCrawlMark& stackMark) 

at System.Activator.CreateInstance(Type type, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes) 

at System.Activator.CreateInstance(Type type, Object[] args) 

at Raven.Database.Config.InMemoryRavenConfiguration.CreateTransactionalStorage(Action notifyAboutWork) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Config\InMemoryRavenConfiguration.cs:line 958 

at Raven.Database.DocumentDatabase..ctor(InMemoryRavenConfiguration configuration, TransportState transportState) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\DocumentDatabase.cs:line 205 

at Raven.Database.Server.HttpServer.<>c__DisplayClass45.<TryGetOrCreateResourceStore>b__43() in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Server\HttpServer.cs:line 1136 

at System.Threading.Tasks.Task`1.InnerInvoke() 

at System.Threading.Tasks.Task.Execute() 

--- End of inner exception stack trace --- 

at System.Threading.Tasks.Task`1.GetResultCore(Boolean waitCompletionNotification) 

at Raven.Database.Server.HttpServer.SetupRequestToProperDatabase(IHttpContext ctx) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Server\HttpServer.cs:line 1038 

at Raven.Database.Server.HttpServer.DispatchRequest(IHttpContext ctx) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Server\HttpServer.cs:line 805 

at Raven.Database.Server.HttpServer.HandleActualRequest(IHttpContext ctx) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Server\HttpServer.cs:line 700 

---> (Inner Exception #0) System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> Microsoft.Isam.Esent.Interop.EsentTooManyInstancesException: Cannot start any more database instances 

at Raven.Storage.Esent.TransactionalStorage.CreateInstance(JET_INSTANCE& compactInstance, String name) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Storage\Esent\TransactionalStorage.cs:line 346 

at Raven.Storage.Esent.TransactionalStorage..ctor(InMemoryRavenConfiguration configuration, Action onCommit) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Storage\Esent\TransactionalStorage.cs:line 99 

--- End of inner exception stack trace --- 

at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor) 

at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture) 

at System.RuntimeType.CreateInstanceImpl(BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes, StackCrawlMark& stackMark) 

at System.Activator.CreateInstance(Type type, BindingFlags bindingAttr, Binder binder, Object[] args, CultureInfo culture, Object[] activationAttributes) 

at System.Activator.CreateInstance(Type type, Object[] args) 

at Raven.Database.Config.InMemoryRavenConfiguration.CreateTransactionalStorage(Action notifyAboutWork) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Config\InMemoryRavenConfiguration.cs:line 958 

at Raven.Database.DocumentDatabase..ctor(InMemoryRavenConfiguration configuration, TransportState transportState) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\DocumentDatabase.cs:line 205 

at Raven.Database.Server.HttpServer.<>c__DisplayClass45.<TryGetOrCreateResourceStore>b__43() in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Server\HttpServer.cs:line 1136 

at System.Threading.Tasks.Task`1.InnerInvoke() 

at System.Threading.Tasks.Task.Execute()<--- 

---> System.Net.WebException: The remote server returned an error: (500) Internal Server Error. 

at System.Net.HttpWebRequest.GetResponse() 

at Raven.Client.Connection.HttpJsonRequest.ReadJsonInternal(Func`1 getResponse) in c:\Builds\RavenDB-Stable-2.5\Raven.Client.Lightweight\Connection\HttpJsonRequest.cs:line 349 

--- End of inner exception stack trace --- 

at Raven.Client.Connection.HttpJsonRequest.HandleErrors(WebException e) in c:\Builds\RavenDB-Stable-2.5\Raven.Client.Lightweight\Connection\HttpJsonRequest.cs:line 543 

at Raven.Client.Connection.HttpJsonRequest.ReadJsonInternal(Func`1 getResponse) in c:\Builds\RavenDB-Stable-2.5\Raven.Client.Lightweight\Connection\HttpJsonRequest.cs:line 355 

at Raven.Client.Connection.HttpJsonRequest.ReadResponseJson() in c:\Builds\RavenDB-Stable-2.5\Raven.Client.Lightweight\Connection\HttpJsonRequest.cs:line 239 

at Raven.Client.Connection.ServerClient.GetStatistics() in c:\Builds\RavenDB-Stable-2.5\Raven.Client.Lightweight\Connection\ServerClient.cs:line 1855 

at Online3.Infrastructure.RavenDB.DocumentStoreExtensions.EnsureDatabaseIsLoaded(IDocumentStore store, String databaseName, TimeSpan timeout, ILog log, CancellationToken cancellationToken) in c:\BuildAgent\work\35de53147eba9030\Online3\Online3\Infrastructure\RavenDB\DocumentStoreExtensions.cs:line 219 

at Online3.Infrastructure.RavenDB.EnsureDatabase.RavenHelpers.InitializeMasterForStore(CultureInfo languageCulture, String dbName, ReplicationDestination replicationDestination, IDocumentStore masterStore, CancellationToken cancellationToken, ILog log) in c:\BuildAgent\work\35de53147eba9030\Online3\Online3\Infrastructure\RavenDB\EnsureDatabase\RavenHelpers.cs:line 119 

at Online3.RavenPopulator.Commands.EnsureDatabaseCommandHandler.Execute(EnsureDatabaseCommand context) 

at Online3.RavenPopulator.EnsureDatabase.EnsureDatabaseWorker.Work(CancellationToken cancellationToken, IPersistentConsumerConnection rabbit, Nullable`1 timeoutMilliseconds)<---

While accessing /admin/stats I encountered the following exception:

{

• Url: "/admin/stats",
• Error: "Microsoft.Isam.Esent.Interop.EsentInvalidInstanceException: Invalid instance handle at Microsoft.Isam.Esent.Interop.Session..ctor(JET_INSTANCE instance) in C:\Work\ravendb\SharedLibs\Sources\managedesent-61618\EsentInterop\Session.cs:line 30 at Raven.Storage.Esent.StorageActions.DocumentStorageActions..ctor(JET_INSTANCE instance, String database, TableColumnsCache tableColumnsCache, OrderedPartCollection`1 documentCodecs, IUuidGenerator uuidGenerator, IDocumentCacher cacher, EsentTransactionContext transactionContext, TransactionalStorage transactionalStorage) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Storage\Esent\StorageActions\General.cs:line 106 at Raven.Storage.Esent.TransactionalStorage.GetDatabaseSizeInBytes() in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Storage\Esent\TransactionalStorage.cs:line 205 at Raven.Database.Server.HttpServer.<get_Statistics>b__14(<>f__AnonymousType18`2 <>h__TransparentIdentifiera) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Server\HttpServer.cs:line 240 at System.Linq.Enumerable.<>c__DisplayClass12`3.<CombineSelectors>b__11(TSource x) at System.Linq.Enumerable.<>c__DisplayClass12`3.<CombineSelectors>b__11(TSource x) at System.Linq.Enumerable.<>c__DisplayClass12`3.<CombineSelectors>b__11(TSource x) at System.Linq.Enumerable.WhereSelectArrayIterator`2.MoveNext() at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection) at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source) at Raven.Imports.Newtonsoft.Json.Serialization.JsonArrayContract.CreateWrapper(Object list) in c:\Builds\RavenDB-Stable-2.5\Imports\Newtonsoft.Json\Src\Newtonsoft.Json\Serialization\JsonArrayContract.cs:line 110 at Raven.Imports.Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeValue(JsonWriter writer, Object value, JsonContract valueContract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerProperty) in c:\Builds\RavenDB-Stable-2.5\Imports\Newtonsoft.Json\Src\Newtonsoft.Json\Serialization\JsonSerializerInternalWriter.cs:line 129 at Raven.Imports.Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.SerializeObject(JsonWriter writer, Object value, JsonObjectContract contract, JsonProperty member, JsonContainerContract collectionContract, JsonProperty containerProperty) in c:\Builds\RavenDB-Stable-2.5\Imports\Newtonsoft.Json\Src\Newtonsoft.Json\Serialization\JsonSerializerInternalWriter.cs:line 383 at Raven.Imports.Newtonsoft.Json.Serialization.JsonSerializerInternalWriter.Serialize(JsonWriter jsonWriter, Object value) in c:\Builds\RavenDB-Stable-2.5\Imports\Newtonsoft.Json\Src\Newtonsoft.Json\Serialization\JsonSerializerInternalWriter.cs:line 62 at Raven.Imports.Newtonsoft.Json.JsonSerializer.SerializeInternal(JsonWriter jsonWriter, Object value) in c:\Builds\RavenDB-Stable-2.5\Imports\Newtonsoft.Json\Src\Newtonsoft.Json\JsonSerializer.cs:line 627 at Raven.Json.Linq.RavenJToken.FromObjectInternal(Object o, JsonSerializer jsonSerializer) in c:\Builds\RavenDB-Stable-2.5\Raven.Abstractions\Json\Linq\RavenJToken.cs:line 84 at Raven.Database.Extensions.HttpExtensions.WriteJson(IHttpContext context, Object obj) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Extensions\HttpExtensions.cs:line 97 at Raven.Database.Server.HttpServer.DispatchRequest(IHttpContext ctx) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Server\HttpServer.cs:line 838 at Raven.Database.Server.HttpServer.HandleActualRequest(IHttpContext ctx) in c:\Builds\RavenDB-Stable-2.5\Raven.Database\Server\HttpServer.cs:line 700"

}

[Oren Eini (Ayende Rahien)]

Yes, you are limited to 1,024 running instances on a single server.

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.