Re: SqlBulkInsertOperation doesn't return failures

[Jason Meckley]

errors are caught and collected by each operation and the etl process. Rhino.ETL does it's best to prevent errors from throwing through as that would break the pipeline.

On Wednesday, September 5, 2012 4:13:55 PM UTC-4, oliwa wrote:

I have a simple process that pulls data from one database and is attempting to insert it into another. My process looks likes this...

protected override void Initialize()
{
    Register(new ExtractLocationData());
    Register(new LoadLocationData("ConnectionString", "TableName"));
}

protected override void OnFinishedProcessing(IOperation op)
{
    foreach (var exception in op.GetAllErrors())
    {
        throw exception;
    }
}

protected override void PostProcessing()
{
    foreach (var exception in Errors)
    {
        throw exception;
    }
}

Now I am purposely trying to insert a NULL value into a column that is NOT NULL so I can test how to handle the errors.  In my console output I get the error like so...

9/5/2012 4:06:45 PM [ERROR] Rhino.Etl.Core.Pipelines.ThreadPoolPipelineExecuter - Failed to execute operation LoadLocationData
System.Data.SqlClient.SqlException (0x80131904): Cannot insert the value NULL into column 'Id', table 'Location'; column does not allow nulls. INSERT fails.
The statement has been terminated.

So I added 2 overrides in an attempt to catch the error and throw it so my process will fail but neither override finds any errors.

It appears as if the SqlBulkInsertOperation class swallows the errors and doesn't add it to the error collection.

Am I missing something?

And on a side note, why would an error bubble up and cause the process to fail?  Having the process fail silently doesn't make any sense at all to me.

Thanks!

[oliwa]

The error is getting caught but it NOT being collected because when I interrogate the Errors property on the process I get nothing even though the error is logged to the console.  I thought it was a threading issue with a race condition so I changed the PipelineExecutor to be single threaded and it still doesn't see any errors.

Here is my full stack trace...

System.InvalidOperationException: Column 'StratRunId' does not allow DBNull.Value.
   at System.Data.SqlClient.SqlBulkCopy.ConvertValue(Object value, _SqlMetaData metadata)
   at System.Data.SqlClient.SqlBulkCopy.WriteToServerInternal()
   at System.Data.SqlClient.SqlBulkCopy.WriteRowSourceToServer(Int32 columnCount)
   at System.Data.SqlClient.SqlBulkCopy.WriteToServer(IDataReader reader)
   at Rhino.Etl.Core.Operations.SqlBulkInsertOperation.<Execute>d__0.MoveNext()
   at Rhino.Etl.Core.Enumerables.SingleRowEventRaisingEnumerator.MoveNext()
   at Rhino.Etl.Core.Enumerables.EventRaisingEnumerator.MoveNext()
   at Rhino.Etl.Core.Pipelines.ThreadPoolPipelineExecuter.<>c__DisplayClass1.<DecorateEnumerableForExecution>b__0(Object )
9/6/2012 9:08:04 AM [TRACE] Rhino.Etl.Core.Pipelines.ThreadPoolPipelineExecuter - Completed process LocationEtlProcess in 00:00:00.4080408