Fatal Error 682 when accessing MS CRM 2011

Developing a CRM 2011 application, I unexpectedly received the SqlException

Warning: Fatal error 682 occurred at [date, time]. Note the error and time, and contact your system administrator.

The complete trace entry in the MS CRM log file looks like this:

[2012-06-12 11:40:43.387] Process: w3wp |Organization:00000000-0000-0000-0000-000000000000 |Thread: 15 |Category: Platform.Sql |User: 00000000-0000-0000-0000-000000000000 |Level: Error | DynamicMetadataCacheLoader.LoadMetadataContainerFromDatabase
>Exception while loading Metadata Cache from Database with LoadMasks = All, OData using new connection and new transaction. Exception: System.Data.SqlClient.SqlException (0x80131904): Warnung: Schwerwiegender Fehler 682 um Jun 12 2012 11:40AM. Notieren Sie den Fehler und den Zeitpunkt, und wenden Sie sich an den Systemadministrator.
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning()
at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
at System.Data.SqlClient.SqlDataReader.HasMoreRows()
at System.Data.SqlClient.SqlDataReader.ReadInternal(Boolean setTimeout)
at Microsoft.Crm.CrmDataReader.Read()
at Microsoft.Crm.Metadata.MultiOrgSharableMetadataCacheLoader.LoadDescriptions(Boolean isSystemDataLoaded, CrmDbConnection connection, IDbTransaction transaction, MetadataContainer systemContainer, MetadataContainer aggregateContainer, TableFillPropertiesMultiOrg properties, CounterList counter)
at Microsoft.Crm.Metadata.MultiOrgSharableMetadataCacheLoader.LoadContainerData(Boolean isSystemDataLoaded, Boolean isSystemLanguageDataLoaded, CrmDbConnection connection, IDbTransaction transaction, MetadataContainer systemContainer, MetadataContainer aggregateContainer, List`1 fillProperties, CounterList counter)
at Microsoft.Crm.Metadata.MultiOrgSharableMetadataCacheLoader.BuildMetadataContainerFromDatabase(LoadMasks masks, CrmDbConnection connection, IDbTransaction transaction, Guid organizationId, CounterList counter)
at Microsoft.Crm.Metadata.DynamicMetadataCacheLoader.LoadMetadataContainerFromDatabase(LoadMasks masks, CrmDbConnection connection, CrmTransaction transaction, Guid organizationId, CounterList counter)
[2012-06-12 11:40:43.398] Process: w3wp |Organization:00000000-0000-0000-0000-000000000000 |Thread: 15 |Category: Platform.Sdk |User: 00000000-0000-0000-0000-000000000000 |Level: Error | ServiceModelTraceRedirector.TraceData
><TraceRecord xmlns="http://schemas.microsoft.com/2004/10/E2ETraceEvent/TraceRecord" Severity="Error"><TraceIdentifier>http://msdn.microsoft.com/de-DE/library/System.ServiceModel.Diagnostics.TraceHandledException.aspx</TraceIdentifier><Description>Handling an exception.</Description><AppDomain>/LM/W3SVC/2/ROOT-1-129839675491242152</AppDomain><Exception><ExceptionType>System.Data.SqlClient.SqlException, System.Data, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</ExceptionType><Message>Warnung: Schwerwiegender Fehler 682 um Jun 12 2012 11:40AM. Notieren Sie den Fehler und den Zeitpunkt, und wenden Sie sich an den Systemadministrator.</Message><StackTrace> at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
> at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning()
> at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
> at System.Data.SqlClient.SqlDataReader.HasMoreRows()
> at System.Data.SqlClient.SqlDataReader.ReadInternal(Boolean setTimeout)
> at Microsoft.Crm.CrmDataReader.Read()
> at Microsoft.Crm.Metadata.MultiOrgSharableMetadataCacheLoader.LoadDescriptions(Boolean isSystemDataLoaded, CrmDbConnection connection, IDbTransaction transaction, MetadataContainer systemContainer, MetadataContainer aggregateContainer, TableFillPropertiesMultiOrg properties, CounterList counter)
> at Microsoft.Crm.Metadata.MultiOrgSharableMetadataCacheLoader.LoadContainerData(Boolean isSystemDataLoaded, Boolean isSystemLanguageDataLoaded, CrmDbConnection connection, IDbTransaction transaction, MetadataContainer systemContainer, MetadataContainer aggregateContainer, List`1 fillProperties, CounterList counter)
> at Microsoft.Crm.Metadata.MultiOrgSharableMetadataCacheLoader.BuildMetadataContainerFromDatabase(LoadMasks masks, CrmDbConnection connection, IDbTransaction transaction, Guid organizationId, CounterList counter)
> at Microsoft.Crm.Metadata.DynamicMetadataCacheLoader.LoadMetadataContainerFromDatabase(LoadMasks masks, CrmDbConnection connection, CrmTransaction transaction, Guid organizationId, CounterList counter)
> at Microsoft.Crm.Metadata.DynamicMetadataCacheLoader.LoadCacheFromDatabaseInternal(LoadMasks masks, CrmDbConnection connection, CrmTransaction transaction, IOrganizationContext context, CounterList counter)
> at Microsoft.Crm.Metadata.DynamicMetadataCacheLoader.LoadCacheFromDatabase(LoadMasks masks, IOrganizationContext context, CounterList counter)
> at Microsoft.Crm.Metadata.DynamicMetadataCacheFactory.LoadMetadataCache(LoadMethod method, CacheType type, IOrganizationContext context)
> at Microsoft.Crm.Metadata.MetadataCache.LoadCache(IOrganizationContext context, Boolean fileOnlyIfExists)
> at Microsoft.Crm.Metadata.MetadataCache.GetInstance(IOrganizationContext context)
> at Microsoft.Crm.BusinessEntities.BusinessEntityMoniker..ctor(Guid id, String entityName, IOrganizationContext orgContext)
> at Microsoft.Crm.Caching.UserDataCacheLoader.LoadCacheData(Guid key, ExecutionContext context)
> at Microsoft.Crm.Caching.ObjectModelCacheLoader`2.LoadCacheData(TKey key, IOrganizationContext context)
> at Microsoft.Crm.Caching.CrmMultiOrgCache`2.CreateEntry(TKey key, IOrganizationContext context)
> at Microsoft.Crm.Caching.CrmMultiOrgCache`2.LookupEntry(TKey key, IOrganizationContext context)
> at Microsoft.Crm.Authentication.UserManagementFactory.UpdateAccessTime(Guid userId, UserAuthenticationAccessType accessType, DateTime accessTime, IOrganizationContext context)
> at Microsoft.Crm.Authentication.UserManagementFactory.UpdateAccessTime(Guid userId, Guid organizationId, UserAuthenticationAccessType accessType, DateTime accessTime)
> at Microsoft.Crm.Authentication.WindowsIdentityAuthorizationManager.Authenticate(OperationContext operationContext)
> at Microsoft.Crm.Authentication.WindowsIdentityAuthorizationManager.CheckAccessCore(OperationContext operationContext)
> at System.ServiceModel.Dispatcher.AuthorizationBehavior.Authorize(MessageRpc&amp;amp; rpc)
> at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage11(MessageRpc&amp;amp; rpc)
> at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)</StackTrace><ExceptionString>System.Data.SqlClient.SqlException (0x80131904): Warnung: Schwerwiegender Fehler 682 um Jun 12 2012 11:40AM. Notieren Sie den Fehler und den Zeitpunkt, und wenden Sie sich an den Systemadministrator.
> at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
> at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning()
> at System.Data.SqlClient.TdsParser.Run(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj)
> at System.Data.SqlClient.SqlDataReader.HasMoreRows()
> at System.Data.SqlClient.SqlDataReader.ReadInternal(Boolean setTimeout)
> at Microsoft.Crm.CrmDataReader.Read()
> at Microsoft.Crm.Metadata.MultiOrgSharableMetadataCacheLoader.LoadDescriptions(Boolean isSystemDataLoaded, CrmDbConnection connection, IDbTransaction transaction, MetadataContainer systemContainer, MetadataContainer aggregateContainer, TableFillPropertiesMultiOrg properties, CounterList counter)
> at Microsoft.Crm.Metadata.MultiOrgSharableMetadataCacheLoader.LoadContainerData(Boolean isSystemDataLoaded, Boolean isSystemLanguageDataLoaded, CrmDbConnection connection, IDbTransaction transaction, MetadataContainer systemContainer, MetadataContainer aggregateContainer, List`1 fillProperties, CounterList counter)
> at Microsoft.Crm.Metadata.MultiOrgSharableMetadataCacheLoader.BuildMetadataContainerFromDatabase(LoadMasks masks, CrmDbConnection connection, IDbTransaction transaction, Guid organizationId, CounterList counter)
> at Microsoft.Crm.Metadata.DynamicMetadataCacheLoader.LoadMetadataContainerFromDatabase(LoadMasks masks, CrmDbConnection connection, CrmTransaction transaction, Guid organizationId, CounterList counter)
> at Microsoft.Crm.Metadata.DynamicMetadataCacheLoader.LoadCacheFromDatabaseInternal(LoadMasks masks, CrmDbConnection connection, CrmTransaction transaction, IOrganizationContext context, CounterList counter)
> at Microsoft.Crm.Metadata.DynamicMetadataCacheLoader.LoadCacheFromDatabase(LoadMasks masks, IOrganizationContext context, CounterList counter)
> at Microsoft.Crm.Metadata.DynamicMetadataCacheFactory.LoadMetadataCache(LoadMethod method, CacheType type, IOrganizationContext context)
> at Microsoft.Crm.Metadata.MetadataCache.LoadCache(IOrganizationContext context, Boolean fileOnlyIfExists)
> at Microsoft.Crm.Metadata.MetadataCache.GetInstance(IOrganizationContext context)
> at Microsoft.Crm.BusinessEntities.BusinessEntityMoniker..ctor(Guid id, String entityName, IOrganizationContext orgContext)
> at Microsoft.Crm.Caching.UserDataCacheLoader.LoadCacheData(Guid key, ExecutionContext context)
> at Microsoft.Crm.Caching.ObjectModelCacheLoader`2.LoadCacheData(TKey key, IOrganizationContext context)
> at Microsoft.Crm.Caching.CrmMultiOrgCache`2.CreateEntry(TKey key, IOrganizationContext context)
> at Microsoft.Crm.Caching.CrmMultiOrgCache`2.LookupEntry(TKey key, IOrganizationContext context)
> at Microsoft.Crm.Authentication.UserManagementFactory.UpdateAccessTime(Guid userId, UserAuthenticationAccessType accessType, DateTime accessTime, IOrganizationContext context)
> at Microsoft.Crm.Authentication.UserManagementFactory.UpdateAccessTime(Guid userId, Guid organizationId, UserAuthenticationAccessType accessType, DateTime accessTime)
> at Microsoft.Crm.Authentication.WindowsIdentityAuthorizationManager.Authenticate(OperationContext operationContext)
> at Microsoft.Crm.Authentication.WindowsIdentityAuthorizationManager.CheckAccessCore(OperationContext operationContext)
> at System.ServiceModel.Dispatcher.AuthorizationBehavior.Authorize(MessageRpc&amp;amp; rpc)
> at System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage11(MessageRpc&amp;amp; rpc)
> at System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean isOperationContextSet)</ExceptionString><DataItems><Data><Key>HelpLink.ProdName</Key><Value>Microsoft SQL Server</Value></Data><Data><Key>HelpLink.EvtSrc</Key><Value>MSSQLServer</Value></Data><Data><Key>HelpLink.EvtID</Key><Value>21</Value></Data><Data><Key>HelpLink.BaseHelpUrl</Key><Value>http://go.microsoft.com/fwlink</Value></Data><Data><Key>HelpLink.LinkId</Key><Value>20476</Value></Data></DataItems></Exception></TraceRecord>

I originally thought that something was wrong with the restored CRM database, when I found a similar bug report. The comments clearly indicated that there was a problem with (another) CRM database, rather than CRM itself.

Go to the Services control panel and stop the services accessing the MSCRM databases

  • Microsoft CRM Async Service (MSCRMAsyncService)
  • Microsoft Dynamics CRM Unzip Service (MSCRMUnzipService)
  • MS CRM Sandbox Service (MSCRMSandboxService)

and the corresponding IIS webs or the IIS service in IIS Manager.

In SSMS, execute the statement

EXEC sp_who

to find out whether there are open connections to the database causing the error.

Next, run

DBCC CHECKDB
WITH NO_INFOMSGS

to check the database for errors and consistency. Some of the results (message codes) I got were (original in German, English versions as found on the internet):

Meldung 8925, Ebene 16, Status 2, Zeile 1
Tabellenfehler: Objektquerverknüpfung: Seite (1:366305), Slot 0, in Objekt-ID 229575856, Index-ID 1, Partitions-ID 72057609057206272, Zuordnungseinheits-ID 72057594044350464 (LOB data-Typ) verweist auf Seite (1:366560), Slot 0, in Objekt-ID 229575856, Index-ID 1, Partitions-ID 2377900618270900224, Zuordnungseinheits-ID 72057594044350464 (LOB data-Typ).

Msg 8925, Level 16, State 1, Line 1
Table error: Cross object linkage: Page (1:5193082), slot 89, in object ID 1575168857, index ID 6, refers to page (1:5193939), slot 0, in object ID 1575168857, index ID 1.

Meldung 8928, Ebene 16, Status 1, Zeile 1
Objekt-ID 162815642, Index-ID 1, Partitions-ID 72057609051242496, Zuordnungseinheits-ID 72057594068992000 (LOB data-Typ): Seite (1:599672) konnte nicht verarbeitet werden. Einzelheiten finden Sie in anderen Fehlermeldungen.

Msg 8928, Level 16, State 1, Line 2 Object ID 645577338, index ID 0: Page (1:168576) could not be processed. See other errors for details.

Meldung 8929, Ebene 16, Status 1, Zeile 1
Objekt-ID 162815642, Index-ID 1, Partitions-ID 72057609051242496, Zuordnungseinheits-ID 72057609471328256 (In-row data-Typ): Es wurden Fehler in Daten außerhalb von Zeilen gefunden mit der ID 1288110080, im Besitz von data, Datensatz identifiziert durch RID = (1:1036135:0).

Msg 8929, Level 16, State 1, Line 1
Object ID 2: Errors found in text ID 852426752 owned by data record identified by RID = (1:110:19) id = 1925581898 and indid = 2.

Meldung 8939, Ebene 16, Status 98, Zeile 1
Tabellenfehler: Objekt-ID 162815642, Index-ID 1, Partitions-ID 72057609051242496, Zuordnungseinheits-ID 72057594068992000 (LOB data-Typ), Seite (1:599672). Fehler bei Test (IS_OFF (BUF_IOERR, pBUF->bstat)). Die Werte sind 12716041 und -4.

Msg 8939, Level 16, State 1, Line 2 Table error: Object ID 1797637899, index ID 0, page (1:168576). Test (m_headerVersion == HEADER_7_0) failed. Values are 0 and 1.

Meldung 8961, Ebene 16, Status 1, Zeile 1
Tabellenfehler: Objekt-ID 229575856, Index-ID 1, Partitions-ID 72057609057206272, Zuordnungseinheits-ID 72057594044350464 (Unknown-Typ). Der Datenknoten außerhalb von Zeilen auf Seite (1:366501), Slot 0, Text-ID 35185262002176 stimmt nicht mit seinem Verweis von Seite (1:366305), Slot 0 überein.

Msg 8961, Level 16, State 1, Line 1
Table error: Object ID 434100587, index ID 1, partition ID 72057594146521088, alloc unit ID 71804568277286912 (type LOB data). The off-row data node at page (1:2487), slot 2, text ID 341442560 does not match its reference from page (1:2487), slot 0.

Meldung 8965, Ebene 16, Status 1, Zeile 1
Tabellenfehler: Objekt-ID 162815642, Index-ID 1, Partitions-ID 72057609051242496, Zuordnungseinheits-ID 72057594068992000 (LOB data-Typ). Auf den Datenknoten außerhalb von Zeilen auf Seite (1:599672), Slot 0, Text-ID 1288110080 wird von Seite (1:1036135), Slot 0 verwiesen, er wurde jedoch im Scan nicht betrachtet.

Msg 8965, Level 16, State 1, Line 1
Table error: Object ID 2. The text, ntext, or image node at page (1:438), slot 3, text ID 852426752 is referenced by page (1:71), slot 4, but was not seen in the scan.

DBCC finished with the message

Von CHECKDB wurden 0 Zuordnungsfehler und 10 Konsistenzfehler in der My_MSCRM-Datenbank gefunden.
repair_allow_data_loss ist die minimale Reparaturstufe für die Fehler, die mit DBCC CHECKDB (My_MSCRM) gefunden wurden.

CHECKDB found 0 allocation errors and 29 consistency errors in database ‘My_MSCRM’.
repair_allow_data_loss is the minimum repair level for the errors found by DBCC CHECKDB (My_MSCRM).

This final message tells you to run DBCC with the REPAIR_ALLOW_DATA_LOSS option. This option can only be applied if the database runs in single-user mode:

ALTER DATABASE MY_MSCRM 
SET SINGLE_USER
WITH ROLLBACK IMMEDIATE;
DBCC CHECKDB ('MY_MSCRM', REPAIR_ALLOW_DATA_LOSS)
WITH NO_INFOMSGS;

If this command still lists unrecoverable errors, use the REPAIR_REBUILD option:

DBCC CHECKDB ('MY_MSCRM', REPAIR_REBUILD)
WITH NO_INFOMSGS

After completion, reset the database to multi-user mode again:

ALTER DATABASE MY_MSCRM SET MULTI_USER 
WITH ROLLBACK IMMEDIATE

Start the CRM services again, as well as the IIS web sites.

One Response to Fatal Error 682 when accessing MS CRM 2011

  1. Ashish Garg says:

    Thanks Man, this saved a day for me …

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: