System.TimeoutException when doing BulkInsert.

[Christian Hansen]

I am receiving the following error when doing a BulkInsert using the client API and the Voron engine:

   at Raven.Client.Document.RemoteBulkInsertOperation.Write(String id, RavenJObject metadata, RavenJObject data, Nullable`1 dataSize) in c:\Builds\RavenDB-Stable-3.0\Raven.Client.Lightweight\Document\RemoteBulkInsertOperation.cs:line 275
   at Raven.Client.Document.BulkInsertOperation.Store(Object entity, String id) in c:\Builds\RavenDB-Stable-3.0\Raven.Client.Lightweight\Document\BulkInsertOperation.cs:line 99
   at Raven.Client.Document.BulkInsertOperation.Store(Object entity) in c:\Builds\RavenDB-Stable-3.0\Raven.Client.Lightweight\Document\BulkInsertOperation.cs:line 82
   at CAL_Upload_to_RavenDB.Program.BulkInsertCAL(IEnumerable`1 cals, IDocumentStore documentStore) in c:\TFS\CAL2\CAL_Upload_to_RavenDB\CAL_Upload_to_RavenDB\Program.cs:line 1184
   at CAL_Upload_to_RavenDB.Program.ProcessCALS(FileInfo f, IDocumentStore documentStore) in c:\TFS\CAL2\CAL_Upload_to_RavenDB\CAL_Upload_to_RavenDB\Program.cs:line 1146
   at CAL_Upload_to_RavenDB.Program.LoadCAL() in c:\TFS\CAL2\CAL_Upload_to_RavenDB\CAL_Upload_to_RavenDB\Program.cs:line 78
   at CAL_Upload_to_RavenDB.Program.Main(String[] args) in c:\TFS\CAL2\CAL_Upload_to_RavenDB\CAL_Upload_to_RavenDB\Program.cs:line 23

Below is my code that calls the BulkInsert:

private static void BulkInsertCAL(IEnumerable<CAL> cals, IDocumentStore documentStore)

{

}

[Christian Hansen]

Here is the code that is inside the method above.  I inadvertently saved the post.

var bulkInsertOptions = new BulkInsertOptions {CheckReferencesInIndexes = true, WriteTimeoutMilliseconds = 100000, OverwriteExisting = true, SkipOverwriteIfUnchanged = true, BatchSize = 256};

using (var bulkInsert = documentStore.BulkInsert(options: bulkInsertOptions))

{

   foreach (var cal in cals)

   {

       bulkInsert.Store(cal);

   }

}

I am using Build 3660.

[Michael Yarichuk]

Hi,

Can you post the exception itself? (not just the stacktrace..)

--
You received this message because you are subscribed to the Google Groups "RavenDB - 2nd generation document database" group.
To unsubscribe from this group and stop receiving emails from it, send an email to ravendb+u...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Best regards,

 

Michael Yarichuk

RavenDB Core Team

Tel: 972-4-6227811

Fax:972-153-4-6227811

Email : michael....@hibernatingrhinos.com

 

RavenDB paving the way to "Data Made Simple" http://ravendb.net/  

[Michael Yarichuk]

Also, how large are the documents in the operation?

[Christian Hansen]

The documents are between 1.2 k and 2.0 k in size and I am trying to bulk insert 100000 documents at a time.  I closed my command prompt with the error message so I ran the process again but I got this message instead:

Unhandled Exception: System.AggregateException: One or more errors occurred. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled.

I guess that error gets thrown because of the first error?

Christian

[Christian Hansen]

I'm also noticing in the Event Viewer this warning that happens just before the error occurs:

A process serving application pool 'RavenApplicationPool' failed to respond to a ping. The process id was '5208'.

Don't know if this is normal or not.

Christian

[Oren Eini (Ayende Rahien)]

You running this in IIS, do you have a request size limit that might be cutting this off?

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

[Christian Hansen]

I am running Ravendb in IIS.  Pardon my ignorance but where would I set the request size limit?

Thanks for all your help!

Christian Hanse

On Tuesday, May 26, 2015 at 12:53:20 PM UTC-6, Christian Hansen wrote:

[Christian Hansen]

I found where the request size limit is set to approximately 2 GB.  So that might not be the problem.

Once again thanks so much for your help!

Christian Hansen

On Tuesday, May 26, 2015 at 12:53:20 PM UTC-6, Christian Hansen wrote:

[Oren Eini (Ayende Rahien)]

    <system.web>
        <httpRuntime
            maxRequestLength="1048576"
            executionTimeout="600" />
    </system.web>
<system.webServer> 
    <security> 
        <requestFiltering> 
            <requestLimits maxAllowedContentLength="1073741824"/> 
        </requestFiltering> 
    </security> 
</system.webServer>

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

[Christian Hansen]

I increased the request size limit to approximately 3 GB and saved the web.config.  I have restarted my import and hope that fixes my problem.

It is so cool to have the creator of RavenDB answering my stupid questions!

Thanks Oren!

On Tuesday, May 26, 2015 at 12:53:20 PM UTC-6, Christian Hansen wrote:

[Christian Hansen]

It crashed again with this error:  Unhandled Exception: System.AggregateException: One or more errors occurred. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled.

I got about 500,000 documents inserted then the error occurred.  It ran for about 30 minutes but the amount of time is not the same nor the amount of documents inserted before the error occurs.

Anything else this could be?  Running Server/Client #3660 with encryption and compression turned on.  Running on Windows 8.1.

Christian Hansen

On Tuesday, May 26, 2015 at 12:53:20 PM UTC-6, Christian Hansen wrote:

[Oren Eini (Ayende Rahien)]

Did you update the reuqest _time_ ?

Can you run this trhough fiddler?

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

[Christian Hansen]

Haven't run it through fiddler but will do it right now.  I added executionTimeout="1000" to the httpRuntime tag.  Is there anywhere else that I need to change?

Thanks again!

On Tuesday, May 26, 2015 at 12:53:20 PM UTC-6, Christian Hansen wrote:

[Christian Hansen]

Ran my import process and logged it with Fiddler.  Below is the Fiddler trace where it failed:

760   504  HTTP    w482.mbc.com:8080  /databases/CALr4/changes/events?id=1/juPFQ    TextView:  [Fiddler] ReadResponse() failed: The server did not return a complete response for this request. Server returned 0 bytes.   Response Headers:   504 Fiddler - Receive Failure  Cache-Control: no-cache, must-revalidate   Transport  Connection: close                                                                                                                                                                                                                                                                                                                                                                                                    

761   504  HTTP    w482.mbc.com:8080  /databases/CALr4/changes/events?id=1/juPFQ    TextView:  [Fiddler] ReadResponse() failed: The server did not return a complete response for this request. Server returned 0 bytes.   Response Headers:   504 Fiddler - Receive Failure  Cache-Control: no-cache, must-revalidate   Transport  Connection: close                                                                                                                                                                                                     

I have attached a text file with the HTTP Headers only.  If you need the full trace let me know.

Thanks!

Below is the error my console window displays (of course it displays a totally different error):

Unhandled Exception: Raven.Abstractions.Connection.ErrorResponseException: Failed request at Raven.Client.Connection.HttpJsonRequest.

On Tuesday, May 26, 2015 at 12:53:20 PM UTC-6, Christian Hansen wrote:

[Christian Hansen]

After digging through the Fiddler trace I found that this command:

POST http://w482.mbc.com:8080/databases/CALr4/bulkInsert?overwriteExisting=true&checkReferencesInIndexes=true&skipOverwriteIfUnchanged=true&operationId=65620b4e-ce6b-41f0-a98a-1d64e0f76df4 HTTP/1.1

caused a 403 error.  In Fiddler under TextView it has this:

40

{"Error":"Unknown single use token, maybe it was already used?"}

0

The previous command returned 200 and had this under TextView:

12

{"OperationId":28}

0

So it appears to me that something happened when trying to execute the command above.

Thanks again!!!

On Tuesday, May 26, 2015 at 12:53:20 PM UTC-6, Christian Hansen wrote:

[Oren Eini (Ayende Rahien)]

Can you send the SAZ file? It will give us the chance to look at everything that happened there.

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

[Christian Hansen]

Here is the link to the saz file:  https://drive.google.com/file/d/0B4N80vLClCemUTltZ2NBNDl0ODQ/view?usp=sharing

Christian Hansen

On Tuesday, May 26, 2015 at 12:53:20 PM UTC-6, Christian Hansen wrote:

[Oren Eini (Ayende Rahien)]

Hm...

Can you check the event log? About the only thing that should cause the single use auth token to fail is if you restarted the db midway through.

Is there anything relevant from the event log there?

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

[Christian Hansen]

At the time the error occurred there was several entries like this in the event log:

Windows detected your registry file is still in use by other applications or services. The file will be unloaded now. The applications or services that hold your registry file may not function properly afterwards. No user action is required.  

 DETAIL - 

 4 user registry handles leaked from \Registry\User\S-1-5-82-379933376-2520954161-2096689016-66779327-733263802_Classes:

Process 556 (\Device\HarddiskVolume2\Windows\System32\svchost.exe) has opened key \REGISTRY\USER\S-1-5-82-379933376-2520954161-2096689016-66779327-733263802_CLASSES

Process 4604 (\Device\HarddiskVolume2\Windows\System32\inetsrv\w3wp.exe) has opened key \REGISTRY\USER\S-1-5-82-379933376-2520954161-2096689016-66779327-733263802_CLASSES

Process 4604 (\Device\HarddiskVolume2\Windows\System32\inetsrv\w3wp.exe) has opened key \REGISTRY\USER\S-1-5-82-379933376-2520954161-2096689016-66779327-733263802_CLASSES

Process 4604 (\Device\HarddiskVolume2\Windows\System32\inetsrv\w3wp.exe) has opened key \REGISTRY\USER\S-1-5-82-379933376-2520954161-2096689016-66779327-733263802_CLASSES

Windows detected your registry file is still in use by other applications or services. The file will be unloaded now. The applications or services that hold your registry file may not function properly afterwards. No user action is required.  

 DETAIL - 

 5 user registry handles leaked from \Registry\User\S-1-5-82-379933376-2520954161-2096689016-66779327-733263802:

Process 556 (\Device\HarddiskVolume2\Windows\System32\svchost.exe) has opened key \REGISTRY\USER\S-1-5-82-379933376-2520954161-2096689016-66779327-733263802

Process 4604 (\Device\HarddiskVolume2\Windows\System32\inetsrv\w3wp.exe) has opened key \REGISTRY\USER\S-1-5-82-379933376-2520954161-2096689016-66779327-733263802

Process 4604 (\Device\HarddiskVolume2\Windows\System32\inetsrv\w3wp.exe) has opened key \REGISTRY\USER\S-1-5-82-379933376-2520954161-2096689016-66779327-733263802\Control Panel\International

Process 4604 (\Device\HarddiskVolume2\Windows\System32\inetsrv\w3wp.exe) has opened key \REGISTRY\USER\S-1-5-82-379933376-2520954161-2096689016-66779327-733263802\Software\Microsoft\Windows\CurrentVersion\Explorer

Process 4604 (\Device\HarddiskVolume2\Windows\System32\inetsrv\w3wp.exe) has opened key \REGISTRY\USER\S-1-5-82-379933376-2520954161-2096689016-66779327-733263802\Software\Microsoft\.NETFramework

Raven (2724) 1-6K1eUKqUNsy/4pI7oikbyoz-E:\Raven\Data\System\Data: The database engine (6.03.9600.0000) is starting a new instance (0).

Raven (2724) 1-6K1eUKqUNsy/4pI7oikbyoz-E:\Raven\Data\System\Data: The database engine is initiating recovery steps.

Raven (2724) 1-6K1eUKqUNsy/4pI7oikbyoz-E:\Raven\Data\System\Data: The database engine has begun replaying logfile E:\Raven\Data\System\logs\RVN.log.

Raven (2724) 1-6K1eUKqUNsy/4pI7oikbyoz-E:\Raven\Data\System\Data: The database engine has successfully completed recovery steps.

Raven (2724) 1-6K1eUKqUNsy/4pI7oikbyoz-E:\Raven\Data\System\Data: The database engine started a new instance (0). (Time=0 seconds) 

Internal Timing Sequence: [1] 0.031, [2] 0.000, [3] 0.016, [4] 0.031, [5] 0.047, [6] 0.125, [7] 0.031, [8] 0.000, [9] 0.000, [10] 0.016.

Raven (2724) 1-6K1eUKqUNsy/4pI7oikbyoz-E:\Raven\Data\System\Data: The database engine attached a database (1, E:\Raven\Data\System\Data). (Time=0 seconds) 

Internal Timing Sequence: [1] 0.000, [2] 0.000, [3] 0.031, [4] 0.000, [5] 0.000, [6] 0.000, [7] 0.000, [8] 0.000, [9] 0.000, [10] 0.000, [11] 0.000, [12] 0.000. 

Saved Cache: 1 0

Here are 2 errors that occurred in the event log:

.NET Runtime  Event ID: 1026

Application: CAL_Upload_to_RavenDB.exe

Framework Version: v4.0.30319

Description: The process was terminated due to an unhandled exception.

Exception Info: Raven.Abstractions.Connection.ErrorResponseException

Stack:

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(System.Threading.Tasks.Task)

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(System.Threading.Tasks.Task)

   at Raven.Client.Document.RemoteBulkInsertOperation.Dispose()

   at Raven.Client.Document.BulkInsertOperation.Dispose()

   at CAL_Upload_to_RavenDB.Program.BulkInsertCAL(System.Collections.Generic.IEnumerable`1<CAL_Upload_to_RavenDB.Model.CAL>, Raven.Client.IDocumentStore)

   at CAL_Upload_to_RavenDB.Program.ProcessCALS(System.IO.FileInfo, Raven.Client.IDocumentStore)

   at CAL_Upload_to_RavenDB.Program.LoadCAL()

   at CAL_Upload_to_RavenDB.Program.Main(System.String[])

Application Error  Event ID: 1000

Faulting application name: CAL_Upload_to_RavenDB.exe, version: 1.0.0.0, time stamp: 0x556504b8

Faulting module name: KERNELBASE.dll, version: 6.3.9600.17415, time stamp: 0x54504ade

Exception code: 0xe0434352

Fault offset: 0x00014598

Faulting process id: 0x1708

Faulting application start time: 0x01d0988ed7cbc377

Faulting application path: C:\TFS\CAL2\CAL_Upload_to_RavenDB\CAL_Upload_to_RavenDB\bin\Debug\CAL_Upload_to_RavenDB.exe

Faulting module path: C:\Windows\SYSTEM32\KERNELBASE.dll

Report Id: 9f9a8553-049c-11e5-829f-4437e6880808

Faulting package full name: 

Faulting package-relative application ID: 

Hope this info helps!

Christian Hansen

On Tuesday, May 26, 2015 at 12:53:20 PM UTC-6, Christian Hansen wrote:

[Grisha Kotler]

Can you try this without fiddler?

Also, go to traffic watch http://ravendb.net/docs/article-page/3.0/Csharp/studio/management/traffic-watch.

What do you see after the exception happens?

Hibernating Rhinos Ltd  

Grisha Kotler l RavenDB Core Team Developer l Mobile: +972-54-586-8647

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

RavenDB paving the way to "Data Made Simple" http://ravendb.net/

[Christian Hansen]

Running import again with Traffic Watch enabled.  I will let you know when the exception occurs.

Christian Hansen

On Tuesday, May 26, 2015 at 12:53:20 PM UTC-6, Christian Hansen wrote:

[Christian Hansen]

Alright, I have attached the export of the Traffic Watch log.   The error I got this time was:  Unhandled Exception: System.AggregateException: One or more errors occurred. ---> System.Threading.Tasks.TaskCanceledException: A task was canceled.

Here are the errors and such from the event log:

Event code: 3005 

Event message: An unhandled exception has occurred. 

Event time: 5/28/2015 8:03:33 AM 

Event time (UTC): 5/28/2015 2:03:33 PM 

Event ID: 6f417542aea643189b30664b5bf5889c 

Event sequence: 8 

Event occurrence: 1 

Event detail code: 0 

 

Application information: 

    Application domain: /LM/W3SVC/2/ROOT-1-130772954107642297 

    Trust level: Full 

    Application Virtual Path: / 

    Application Path: E:\RavenSite\ 

    Machine name: W482 

 

Process information: 

    Process ID: 3888 

    Process name: w3wp.exe 

    Account name: IIS APPPOOL\RavenApplicationPool 

 

Exception information: 

    Exception type: OperationCanceledException 

    Exception message: The operation was canceled.

   at System.Threading.CancellationToken.ThrowOperationCanceledException()

   at System.Web.Http.Owin.HttpMessageHandlerAdapter.<SendResponseContentAsync>d__20.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

   at System.Web.Http.Owin.HttpMessageHandlerAdapter.<InvokeCore>d__0.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

   at Owin.AppBuilderExtensions.<UpgradeToWebSockets>d__3.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

   at Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.IntegratedPipelineContextStage.<RunApp>d__5.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

   at Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.IntegratedPipelineContext.<DoFinalWork>d__2.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

   at Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.IntegratedPipelineContext.EndFinalWork(IAsyncResult ar)

   at System.Web.HttpApplication.AsyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()

   at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

 

 

Request information: 

    Request URL: http://localhost:8080/ 

    Request path: / 

    User host address: ::1 

    User:  

    Is authenticated: False 

    Authentication Type:  

    Thread account name: IIS APPPOOL\RavenApplicationPool 

 

Thread information: 

    Thread ID: 16 

    Thread account name: IIS APPPOOL\RavenApplicationPool 

    Is impersonating: False 

    Stack trace:    at System.Threading.CancellationToken.ThrowOperationCanceledException()

   at System.Web.Http.Owin.HttpMessageHandlerAdapter.<SendResponseContentAsync>d__20.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

   at System.Web.Http.Owin.HttpMessageHandlerAdapter.<InvokeCore>d__0.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

   at Owin.AppBuilderExtensions.<UpgradeToWebSockets>d__3.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

   at Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.IntegratedPipelineContextStage.<RunApp>d__5.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

   at Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.IntegratedPipelineContext.<DoFinalWork>d__2.MoveNext()

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()

   at Microsoft.Owin.Host.SystemWeb.IntegratedPipeline.IntegratedPipelineContext.EndFinalWork(IAsyncResult ar)

   at System.Web.HttpApplication.AsyncEventExecutionStep.System.Web.HttpApplication.IExecutionStep.Execute()

   at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)

 

 

Custom event details: 

Application: CAL_Upload_to_RavenDB.exe

Framework Version: v4.0.30319

Description: The process was terminated due to an unhandled exception.

Exception Info: System.AggregateException

Stack:

   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean)

   at System.Threading.Tasks.Task.Wait(Int32, System.Threading.CancellationToken)

   at Raven.Client.Document.RemoteBulkInsertOperation.Write(System.String, Raven.Json.Linq.RavenJObject, Raven.Json.Linq.RavenJObject, System.Nullable`1<Int32>)

   at Raven.Client.Document.BulkInsertOperation.Store(System.Object, System.String)

   at Raven.Client.Document.BulkInsertOperation.Store(System.Object)

   at CAL_Upload_to_RavenDB.Program.BulkInsertCAL(System.Collections.Generic.IEnumerable`1<CAL_Upload_to_RavenDB.Model.CAL>, Raven.Client.IDocumentStore)

   at CAL_Upload_to_RavenDB.Program.ProcessCALS(System.IO.FileInfo, Raven.Client.IDocumentStore)

   at CAL_Upload_to_RavenDB.Program.LoadCAL()

   at CAL_Upload_to_RavenDB.Program.Main(System.String[])

Faulting application name: CAL_Upload_to_RavenDB.exe, version: 1.0.0.0, time stamp: 0x556504b8

Faulting module name: KERNELBASE.dll, version: 6.3.9600.17415, time stamp: 0x54504ade

Exception code: 0xe0434352

Fault offset: 0x00014598

Faulting process id: 0x174c

Faulting application start time: 0x01d0994f6e2bbac8

Faulting application path: C:\TFS\CAL2\CAL_Upload_to_RavenDB\CAL_Upload_to_RavenDB\bin\Debug\CAL_Upload_to_RavenDB.exe

Faulting module path: C:\Windows\SYSTEM32\KERNELBASE.dll

Report Id: db1cbc39-0543-11e5-829f-4437e6880808

Faulting package full name: 

Faulting package-relative application ID: 

On Tuesday, May 26, 2015 at 12:53:20 PM UTC-6, Christian Hansen wrote:

[Grisha Kotler]

I'd like to conference with you on skype.

My skype is grisha.kotler.

Hibernating Rhinos Ltd  

Grisha Kotler l RavenDB Core Team Developer l Mobile: +972-54-586-8647

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

RavenDB paving the way to "Data Made Simple" http://ravendb.net/

[Christian Hansen]

So I decided to update to version 3690 to see what would happen.  Well, I still get an error but at least its the same error every time.  This is the error I get now:  

Error:System.TimeoutException: Bulk insert operation did not receive new data longer than configured treshold.

Does this help any?

Thanks for your time!

Christian Hansen

On Tuesday, May 26, 2015 at 12:53:20 PM UTC-6, Christian Hansen wrote:

[Oren Eini (Ayende Rahien)]

Yes, it does.

This means that for some reason, you are showing a _very_ long time, over a minute where we got no entires from the server, so we shut down the connection.

You can configure this using: Raven/BulkImport/BatchTimeout

Or you can find why you are holding the connection open without sending anything down the line.

This can happen if you are making additional remote calls during the bulk insert operation.

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

[Christian Hansen]

Okay, I put this snippet of code into the web.config:

<add key="Raven/BulkImport/BatchTimeout" value="100" />

I now get this error:  

"Error":"System.ArgumentOutOfRangeException: Time-out interval must be less than 2^32-2. Parameter name: dueTime

Did I add the wrong code in the web.config file?  Having a hard time here....please help!

Christian

[Grisha Kotler]

This should be in milliseconds. Default: 60000.

Hibernating Rhinos Ltd  

Grisha Kotler l RavenDB Core Team Developer l Mobile: +972-54-586-8647

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

RavenDB paving the way to "Data Made Simple" http://ravendb.net/

[Christian Hansen]

Thanks for letting me know.  Trying the import again.

[Christian Hansen]

Okay, I set the timeout = 60000 and this is the error I get:

Unhandled Exception: System.InvalidOperationException: InternalServerError

{"Url":"/databases/CALr4/bulkInsert?overwriteExisting=true&checkReferencesInInde

xes=true&skipOverwriteIfUnchanged=true&operationId=29a01da6-58b9-4577-b675-3b867

730d0e5",

"Error":"System.ArgumentOutOfRangeException: Time-out interval must be

less than 2^32-2.\r\nParameter name: dueTime\r\n

at System.Threading.Timer..ct

or(TimerCallback callback, Object state, Int64 dueTime, Int64 period)\r\n   at R

aven.Database.Extensions.CancellationTimeout..ctor(CancellationTokenSource sourc

e, TimeSpan dueTime)\r\n   at Raven.Database.Server.Controllers.BulkInsertContro

ller.<BulkInsertPost>d__9.MoveNext()\r\n--- End of stack trace from previous loc

ation where exception was thrown ---\r\n   at System.Runtime.CompilerServices.Ta

skAwaiter.ThrowForNonSuccess(Task task)\r\n   at System.Runtime.CompilerServices

.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.

Threading.Tasks.TaskHelpersExtensions.<CastToObject>d__3`1.MoveNext()\r\n--- End

 of stack trace from previous location where exception was thrown ---\r\n   at S

ystem.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n   a

t System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotific

ation(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResul

t()\r\n   at System.Web.Http.Controllers.ApiControllerActionInvoker.<InvokeActio

nAsyncCore>d__0.MoveNext()\r\n--- End of stack trace from previous location wher

e exception was thrown ---\r\n   at System.Runtime.CompilerServices.TaskAwaiter.

ThrowForNonSuccess(Task task)\r\n   at System.Runtime.CompilerServices.TaskAwait

er.HandleNonSuccessAndDebuggerNotification(Task task)\r\n   at System.Runtime.Co

mpilerServices.TaskAwaiter`1.GetResult()\r\n   at System.Web.Http.Controllers.Ac

tionFilterResult.<ExecuteAsync>d__2.MoveNext()\r\n--- End of stack trace from pr

evious location where exception was thrown ---\r\n   at System.Runtime.CompilerS

ervices.TaskAwaiter.ThrowForNonSuccess(Task task)\r\n   at System.Runtime.Compil

erServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)\r\n

at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()\r\n   at System.Web

.Http.Controllers.ExceptionFilterResult.<ExecuteAsync>d__0.MoveNext()"}

   at Raven.Client.Extensions.HttpJsonRequestExtensions.<AssertNotFailingRespons

e>d__0.MoveNext() in c:\Builds\RavenDB-Stable-3.0\Raven.Client.Lightweight\Exten

sions\Internal\HttpJsonRequestExtensions.cs:line 30

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNot

ification(Task task)

   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)

   at Raven.Client.Document.RemoteBulkInsertOperation.<StartBulkInsertAsync>d__8

.MoveNext() in c:\Builds\RavenDB-Stable-3.0\Raven.Client.Lightweight\Document\Re

moteBulkInsertOperation.cs:line 128

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNot

ification(Task task)

   at Raven.Client.Document.RemoteBulkInsertOperation.<DisposeAsync>d__2a.MoveNe

xt() in c:\Builds\RavenDB-Stable-3.0\Raven.Client.Lightweight\Document\RemoteBul

kInsertOperation.cs:line 364

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNot

ification(Task task)

   at Raven.Client.Document.RemoteBulkInsertOperation.Dispose() in c:\Builds\Rav

enDB-Stable-3.0\Raven.Client.Lightweight\Document\RemoteBulkInsertOperation.cs:l

ine 377

   at Raven.Client.Document.BulkInsertOperation.Dispose() in c:\Builds\RavenDB-S

table-3.0\Raven.Client.Lightweight\Document\BulkInsertOperation.cs:line 76

   at CAL_Upload_to_RavenDB.Program.BulkInsertCAL(IEnumerable`1 cals, IDocumentS

tore documentStore) in c:\TFS\CAL2\CAL_Upload_to_RavenDB\CAL_Upload_to_RavenDB\P

rogram.cs:line 1185

   at CAL_Upload_to_RavenDB.Program.ProcessCALS(FileInfo f, IDocumentStore docum

entStore) in c:\TFS\CAL2\CAL_Upload_to_RavenDB\CAL_Upload_to_RavenDB\Program.cs:

line 1145

   at CAL_Upload_to_RavenDB.Program.LoadCAL() in c:\TFS\CAL2\CAL_Upload_to_Raven

DB\CAL_Upload_to_RavenDB\Program.cs:line 77

   at CAL_Upload_to_RavenDB.Program.Main(String[] args) in c:\TFS\CAL2\CAL_Uploa

d_to_RavenDB\CAL_Upload_to_RavenDB\Program.cs:line 22

[Grisha Kotler]

Sorry, it needs to be in timespan string format. Default 00:01:00 (1 minute).

Hibernating Rhinos Ltd  

Grisha Kotler l RavenDB Core Team Developer l Mobile: +972-54-586-8647

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

RavenDB paving the way to "Data Made Simple" http://ravendb.net/

[Christian Hansen]

Thanks Grisha!  I will adjust to 2 minutes and try my import again!

Thanks again!

[Christian Hansen]

After running for 4 hours it threw an error.  This is the error I received:

Unhandled Exception: Raven.Abstractions.Connection.ErrorResponseException: 

Failed request at Raven.Client.Connection.Implementation.HttpJsonRequest.<>c__DisplayClass6f.<<ExecuteRawRequestAsync>b__6e>d__71.MoveNext() in c:\Builds\RavenDB-Stable-3.0\Raven.Client.Lightweight\Connection\Implementation\HttpJsonRequest.cs:line 777

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

   at Raven.Client.Connection.Implementation.HttpJsonRequest.<RunWithAuthRetry>d__16`1.MoveNext() in c:\Builds\RavenDB-Stable-.0\Raven.Client.Lightweight\Connection\Implementation\HttpJsonRequest.cs:line 235

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

   at Raven.Client.Connection.Implementation.HttpJsonRequest.<ExecuteRawRequestAsync>d__75.MoveNext() in c:\Builds\RavenDB-Stable-3.0\Raven.Client.Lightweight\Connection\Implementation\HttpJsonRequest.cs:line 761

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

   at System.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)

   at Raven.Client.Document.RemoteBulkInsertOperation.<StartBulkInsertAsync>d__8.MoveNext() in c:\Builds\RavenDB-Stable-3.0\Raven.Client.Lightweight\Document\RemoteBulkInsertOperation.cs:line 115

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

   at Raven.Client.Document.RemoteBulkInsertOperation.<DisposeAsync>d__2a.MoveNext() in c:\Builds\RavenDB-Stable-3.0\Raven.Client.Lightweight\Document\RemoteBulkInsertOperation.cs:line 364

--- End of stack trace from previous location where exception was thrown ---

   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)

   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)

   at Raven.Client.Document.RemoteBulkInsertOperation.Dispose() in c:\Builds\RavenDB-Stable-3.0\Raven.Client.Lightweight\Document\RemoteBulkInsertOperation.cs:line 377

   at Raven.Client.Document.BulkInsertOperation.Dispose() in c:\Builds\RavenDB-Stable-3.0\Raven.Client.Lightweight\Document\BulkInsertOperation.cs:line 76

   at CAL_Upload_to_RavenDB.Program.BulkInsertCAL(IEnumerable`1 cals, IDocumentStore documentStore) in c:\TFS\CAL2\CAL_Upload_to_RavenDB\CAL_Upload_to_RavenDB\Program.cs:line 1185

   at CAL_Upload_to_RavenDB.Program.ProcessCALS(FileInfo f, IDocumentStore documentStore) in c:\TFS\CAL2\CAL_Upload_to_RavenDB\CAL_Upload_to_RavenDB\Program.cs:line 1145

   at CAL_Upload_to_RavenDB.Program.LoadCAL() in c:\TFS\CAL2\CAL_Upload_to_RavenDB\CAL_Upload_to_RavenDB\Program.cs:line 77

   at CAL_Upload_to_RavenDB.Program.Main(String[] args) in c:\TFS\CAL2\CAL_Upload_to_RavenDB\CAL_Upload_to_RavenDB\Program.cs:line 22

I don't know if this will be much help.  Is there anything else I can do?

Thanks!

Christian Hansen

[Oren Eini (Ayende Rahien)]

Can you go on skype?

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811

 

[Christian Hansen]

I can try.  I work at a bank and I don't know if the firewall has the port opened or not.  My id is cstenbergh.

[Oren Eini (Ayende Rahien)]

We made a fix that should take care of this, will be in the next build.

Hibernating Rhinos Ltd  

Oren Eini l CEO l Mobile: + 972-52-548-6969

Office: +972-4-622-7811 l Fax: +972-153-4-622-7811