CommServe response time seems slow affecting jobs, reports, and CommCell Console performance

Invetigation shows one of more of the folling symptoms

• Performance degrades afer OS upgrade or update - or over a period of time.  
• Slow SQL database I/O performance
• SQL Lock timeouts and deadlocks in job manager logs
• Running custom SQL queries on the CommServe database
• Repeated resource allocation failures
• Resource allocation loop takes additional time
• CommCell Console updates/screen refresh take longer to load and display

Possible causes

• System requirements for level of activity not met.
• Available disk space for SQL Transaction logs is low. This can result from Full SQL Recovery mode being enabled and transaction logs not being backed up and pruned)
• When a custom SQL query is run on the CommServe database this creates locks on SQL tables.
• Incorrect data path configuration in a Storage Policy causing resource allocation to loop and take longer than expected.
• Resource allocation for a job can fail repeatedly if there are lots of jobs running on a given Storage Policy
• Temp space is low  (Database space check warnings issued) 
• Lack of useful DB statistics. SQL Server 2008 R2 or SQL Server 2012 creates and destroys some memory structures about global statistics instead of caching them.  (Microsoft Hotfix available)
• Slow network communication.
• High number of CommCell Console or WebConsole sessions open.
• (HP Proliant) HP Power Profile not set to High Performance (default is set to Balanced Power and Performance)

• Close unnecessary CommCell Console/Web Console sessions. If persistent high number of Web Consoles are open consider moving WebConsole server and client to different host.
• Verify sufficent disk space, I/O performance, and memory on CommServe host. Review System Requirements for all Commvault compnents installed on CommServe host. 
• Run DBMaintenance to refresh SQL indexes and statistics.
• Monitor Disk I/O. Check for long Disk I/O queue/waits. If needed, move/separate high disk I/O applications on different disks. Performance may be improved by separating SQL databases, OS swap space (pagefile.sys), Commvault Index Cache, and Commvault logs on different disks.
• Check network bandwidth/performance.  Slow performance on the CommServe may be the result of network congestion or incorrect network performance settings.
• Review the following Microsoft KB articles:
  • http://support.microsoft.com/kb/2662301
  • http://support.microsoft.com/default.aspx?scid=kb;EN-US;976700
  • http://support.microsoft.com/kb/979149
• If host is HP Proliant, check BIOS Power Profile is set to High Performance.
• Check the custom SQL queries to ensure they do not cause locks on SQL tabels.  The SQL statements should be written with NO LOCKS specified where appropriate.
• Review the data path configuration for correctness
• Review the number of subclients being backed up concurrently to the same storage policy and redistribute them appropriately. 
• If performance degrade occurs after OS upgrade/update and none of the above actions improve performance, contact customer support. You may need to do a fresh install of the new OS and re-install Commvault software on the host. Make sure you have several DR Backup copies (Database dumps) and latest Commvault software and services packs before proceeding.