Query notification processing failed due to a sql exception

We got this SCOM error last week along with a ton of the following warnings. Microsoft claims that this is a known issue that was supposed to be fixed with SCOM 2007 SP1 / SCOM 2007 R2, but I still encountered it running R2.

Event Type: Error
Event Source: DataAccessLayer
Event Category: None
Event ID: 26308
Date:  3/11/2010
Time:  11:30:01 AM
User:  N/A
Computer:
Description:
Query notification processing failed due to a sql exception.

System.Data.SqlClient.SqlException: A transport-level error has occurred when receiving results from the server. (provider: TCP Provider, error: 0 – The semaphore timeout period has expired.)
at System.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection)
at System.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj)
at System.Data.SqlClient.SqlCommand.CheckThrowSNIException()
at System.Data.SqlClient.SqlCommand.InternalEndExecuteReader(IAsyncResult asyncResult, String endMethod)
at System.Data.SqlClient.SqlCommand.EndExecuteReader(IAsyncResult asyncResult)
at Microsoft.EnterpriseManagement.Mom.DataAccess.QueryNotificationManager.HandleNotifications(Object state)

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Event Type: Error
Event Source: Health Service Modules
Event Category: Data Warehouse
Event ID: 31569
Date:  3/11/2010
Time:  11:30:03 AM
User:  N/A
Computer:
Description:
Report deployment process failed to request management pack list from Data Warehouse. The operation will be retried.Exception ‘InvalidConnectionException’: ExecuteScalar requires an open and available Connection. The connection’s current state is closed.

One or more workflows were affected by this.

Workflow name: Microsoft.SystemCenter.DataWarehouse.Deployment.Report
Instance name:
Instance ID: {A18674C9-F1A9-98DA-3B94-DD88EAE278FB}
Management group:

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Event Type: Warning
Event Source: HealthService
Event Category: None
Event ID: 2115
Date:  3/11/2010
Time:  11:30:39 AM
User:  N/A
Computer:
Description:
A Bind Data Source in Management Group has posted items to the workflow, but has not received a response in 61 seconds.  This indicates a performance or functional problem with the workflow.
Workflow Id : Microsoft.SystemCenter.CollectDiscoveryData
Instance    :
Instance Id : {A18674C9-F1A9-98DA-3B94-DD88EAE278FB}
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Event Type: Warning
Event Source: HealthService
Event Category: None
Event ID: 2115
Date:  3/11/2010
Time:  11:30:39 AM
User:  N/A
Computer:
Description:
A Bind Data Source in Management Group  has posted items to the workflow, but has not received a response in 61 seconds.  This indicates a performance or functional problem with the workflow.
Workflow Id : Microsoft.SystemCenter.CollectAlerts
Instance    :
Instance Id : {A18674C9-F1A9-98DA-3B94-DD88EAE278FB}
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Event Type: Warning
Event Source: HealthService
Event Category: None
Event ID: 2115
Date:  3/11/2010
Time:  11:30:39 AM
User:  N/A
Computer:
Description:
A Bind Data Source in Management Group  has posted items to the workflow, but has not received a response in 61 seconds.  This indicates a performance or functional problem with the workflow.
Workflow Id : Microsoft.SystemCenter.CollectPerformanceData
Instance    : yksc1s3.ehit.org
Instance Id : {A18674C9-F1A9-98DA-3B94-DD88EAE278FB}

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Event Type: Warning
Event Source: HealthService
Event Category: None
Event ID: 2115
Date:  3/11/2010
Time:  11:31:18 AM
User:  N/A
Computer:
Description:
A Bind Data Source in Management Group  has posted items to the workflow, but has not received a response in 61 seconds.  This indicates a performance or functional problem with the workflow.
Workflow Id : Microsoft.SystemCenter.CollectEventData
Instance    :
Instance Id : {A18674C9-F1A9-98DA-3B94-DD88EAE278FB}
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Event Type: Warning
Event Source: HealthService
Event Category: None
Event ID: 2115
Date:  3/11/2010
Time:  11:31:18 AM
User:  N/A
Computer:
Description:
A Bind Data Source in Management Group  has posted items to the workflow, but has not received a response in 61 seconds.  This indicates a performance or functional problem with the workflow.
Workflow Id : Microsoft.SystemCenter.CollectSignatureData
Instance    : yksc1s3.ehit.org
Instance Id : {A18674C9-F1A9-98DA-3B94-DD88EAE278FB}
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

This is caused by a high number of sql queries to the Datawarehouse database. The work around is to add an override for the Event Data Collector rule in the Data warehouse connection server scope.
The Execution Attempt Timeout Interval Seconds property was changed from 0 to 6 and the Max Execution Attempt Count was set from 0 to 10. Restart the System Center service on the RMS and any gateway servers.

 

SCOM 2007 Stopped Receiving Alerts


I noticed one day that I stopped getting any SCOM alerts in my System Center Operation Manager 2007 R2 environment. As part of my troubleshooting I found a ton of the following warnings in the Operations Manager Event Viewer. I contacted Microsoft tech support and discovered that my RMS was in maintenance mode. I had put my RMS in maintenance mode for 30 minutes when I did some windows updates, but it never come out of it. After stopping maintenance mode and re-starting the System Center Management service on the Root Management Server, alerts started coming in again. The Microsoft tech told me that you should never put your RMS in maintenance mode and when I asked to have some documentation where it said that, he said it does not exist. This a tip that they have discovered in their troubleshooting. During this time the RMS was unable to process agent requests, so the agent holds on to the notifications in a buffer until it can communicate with the RMS again. If you are getting this warning on the RMS there is a good chance that you lost any alerts that the agent was unable to store in the buffer.

Event Type:      Warning

Event Source:   OpsMgr Connector

Event Category:            None

Event ID:          20058

Date:                2/6/2010

Time:                2:00:17 PM

User:                N/A

Computer:

Description:

The Root Connector has received bad packets which have been dropped. It received 8374 such packet(s) in the last five minutes

For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.

Copyright EverythingTech 2018
Tech Nerd theme designed by Siteturner