Recently, one of my clients contacted me to assist in fixing and issue which was causing their production database to be in SUSPECT state. The learnings I am getting from various consulting engagements is awesome and the after effect of this is a number of blogs I have published in the recent past. They were using Microsoft SQL Server 2008 R2 (SP2) in a clustered environment. They had a disaster and had a hard failure of SQL cluster. Initially there were issues with DTC and they were able to fix them by installing the DTC component in the cluster. Now, once they reinstalled DTC, the production database was in SUSPECT state. Let us learn about how to detect DTC/KTM in-doubt Transaction with UOW.
I got connected to their server via desktop sharing tool and I checked SSMS and found that the database ‘IN_SELLER_MAIN’ was in SUSPECT state. I looked into ERRORLOG and found below:
2016-06-27 07:12:19.55 spid21s SQL Server detected a DTC/KTM in-doubt transaction with UOW {C36097CA-09D8-406D-B8D4-3A0200DD6A9A}.Please resolve it following the guideline for Troubleshooting DTC Transactions.
2016-06-27 07:12:19.55 spid21s Error: 3437, Severity: 21, State: 3.
2016-06-27 07:12:19.55 spid21s An error occurred while recovering database ‘SELLER’. Unable to connect to Microsoft Distributed Transaction Coordinator (MS DTC) to check the completion status of transaction (32:-1730264831). Fix MS DTC, and run recovery again.
2016-06-27 07:12:19.55 spid21s Error: 3414, Severity: 21, State: 2.
2016-06-27 07:12:19.55 spid21s An error occurred during recovery, preventing the database ‘SELLER’ (database ID 9) from restarting. Diagnose the recovery errors and fix them, or restore from a known good backup. If errors are not corrected or expected, contact Technical Support.
When I checked DTC, there were no transactions listed
SOLUTION
Error messages are pretty straight forward. When SQL was stopped, there were some transactions which need to roll-forward/rollback. Since client reinstalled DTC, there was no way to recovery in-doubt transaction by itself. So, we decided to change setting in SQL Server such that it aborts those transaction and recovery continues.
sp_configure 'in-doubt xact resolution', 2 GO RECONFIGURE GO
The value 1 or 2 determines what to be done with such in-doubt transactions. Since it would have been rolled back on other server, we have changed the value to 2. Then we restarted SQL resource and it came online. With that database also came online.
When I search for “Troubleshooting DTC Transactions” as mentioned in ERRORLOG message, I found an article (which is for SQL 2000).
If you are a DBA then you must have seen SUSPECT database at least once, please share the reason and the fix via comments to share the knowledge.
Reference: Pinal Dave (http://blog.sqlauthority.com)
First appeared on SQL SERVER – Detected a DTC/KTM in-doubt Transaction with UOW