NServiceBus.Host cannot connect to MS DTC

[Ian Jones]

When my NServiceBus.Host endpoint starts I get a bunch of errors telling me it cannot connect to MS DTC (stack trace below). I've run the Powershell cmdlet to ensure MSDTC is configured correctly, and likewise MSMQ (my transport choice). This service has been running in the past but will now no longer start up. Our infrastructure team assure me no changes have been made, but I tend to take such statements with a pinch of salt. Any ideas?

2016-11-03 14:50:56.758 WARN  NServiceBus.ReceiveWithNativeTransaction MSMQ receive operation failed
System.Messaging.MessageQueueException (0x80004005): Failure to connect to MS DTC.
   at System.Messaging.MessageQueue.ReceiveCurrent(TimeSpan timeout, Int32 action, CursorHandle cursor, MessagePropertyFilter filter, MessageQueueTransaction internalTransaction, MessageQueueTransactionType transactionType)
   at System.Messaging.MessageQueue.Receive(TimeSpan timeout, MessageQueueTransactionType transactionType)
   at NServiceBus.ReceiveStrategy.TryReceive(MessageQueueTransactionType transactionType, Message& message) in C:\Build\src\NServiceBus.Core\Transports\Msmq\ReceiveStrategy.cs:line 42
   at NServiceBus.ReceiveWithTransactionScope.<ReceiveMessage>d__1.MoveNext() in C:\Build\src\NServiceBus.Core\Transports\Msmq\ReceiveWithTransactionScope.cs:line 59
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at NServiceBus.MessagePump.<>c.<<ReceiveMessage>b__7_0>d.MoveNext() in C:\Build\src\NServiceBus.Core\Transports\Msmq\MessagePump.cs:line 181

[ramon...@particular.net]

Hi Ian,

There can be various reasons. It can be related to the network, more transactions, duration of the transactions.

Windows Event Log

Start with verifying the Windows Event Log and check if there are any MSDTC related errors reported.

Remote MSDTC server

Maybe a remote MSDTC server is configured on the machine? This is likely if MSDTC is made high available in your network.

To verify this:

• Open component services
  
• Expand the component services tree
  
• Right click "My Computer" and select properties
  

You should now get a new dialog.

• Select the MSDTC tab
  
• Check what the default coordinator is (local or a remote machine)
  

If a remote server is listed it is very likely that there are firewall issues.

Increasing MSDTC log size

You can also increase the MSDTC log size. Maybe you have more endpoints and more transactions. The default log size is 4MB, try increasing this to 16MB is the event log mentions that the log is too small.

• Open component services
  
• Expand the component services tree
  
• Go to Component Services > Computers > My Computer > Distributed Transaction Coordinator > Local DTC
• Right click "Local DTC" and select properties
  

A dialog should now open

• Select "Logging" tab
• Enter a larger value in the field labeled "Capacity"

DTCPing

If not, this could also indicate a network firewall configuration issue. You can use a tool called DTCPing to diagnose DTC issues between two machines:

• https://docs.particular.net/nservicebus/operations/transactions-message-processing
  

Regards,

Ramon

[Ian Jones]

Hi,

I went round and round, uninstalling and reinstalling MSDTC, running the PS cmdlet, rebooting servers, and so on. Eventually I got my service running again, but it was still complaining that MSDTC was not available on the remote machine running SQL Server.

It's a pity the error messages MSDTC chucks out are not more useful. Eventually we realised it was due to the fact that in our Production, Staging and Test environments there is no domain - so the Network Service Account on one machine was not recognised on a different machine. Microsoft's answer is to run MSDTC with no authentication. Whilst this might make security auditors a little uneasy, it doesn't mean that access to SQL Server is unauthenticated - just the distributed transaction around it.

I never did identify exactly what was wrong in the first instance, but at least all my messages were queued up ready to be retried once we got the service running again!