Hyper-V node crashes when performing a backup to a LUN that supports 4K sectors

Article ID: HPV0026 ##_DOC_OEM_INSTALL_SCREEN_PRODUCT_NAME_## Service Pack 5 includes a changed block tracking (CBT) driver with 4K LUN support. In a Hyper-V cluster environment where not all nodes have been upgraded to SP5 or later, an older driver version that attempts to write 512 byte blocks to a LUN that uses 4K sectors can cause a system crash on the Hyper-V node that is performing the backup.

Symptom

A Hyper-V node crashes when performing a backup to a LUN that supports 4K sectors. 

Cause

If a Hyper-V cluster includes some nodes that have been upgraded to Service Pack 5 or later, and some nodes that are still using an earlier service pack, a Hyper-V node that is using an older driver might attempt to write 512 byte blocks to a LUN that supports 4K sectors. This operation can cause a system crash on the Hyper-V node. 

Resolution

The best solution for this scenario is to upgrade all nodes in the cluster to SP5 or later at the same time. This solution enables all nodes in the cluster to continue backup operations using CBT.

If you cannot upgrade all nodes, perform the following actions:

  1. On the Backup Options tab of the subclient properties, disable CBT by clearing the Use Changed Block Tracking check box.
  2. On nodes that are using an older service pack, disable or uninstall the CVCBT driver.
  • To disable the CVCBT driver, run the following command on the Hyper-V node:

    sc stop cvcbt

  • To uninstall the CVCBT driver, run the following command on the Hyper-V node:

    DriverInstaller.exe -cvcbt -op unInstall -inf "path"

    Where path is the location of the ##_DOC_OEM_INSTALL_SCREEN_PRODUCT_NAME_## drivers folder. For example, you can use the following command:

    DriverInstaller.exe -cvcbt -op unInstall -inf "C:\Program Files\##_DOC_OEM_INSTALL_DESTINATION_COMPANY_FOLDER_##\##_DOC_OEM_INSTALL_DESTINATION_SOFTWARE_FOLDER_##\Drivers"

If you disable or uninstall the CBT driver, backups will use the cyclic redundancy check (CRC) mechanism to back up the parts of virtual disks that have changed since the last full backup.

Note: You do not need to uninstall the Virtual Server Agent.

After all nodes in the cluster have been upgraded to SP5 or later, you can re-enable CBT on the Backup Options tab of the subclient properties. When CBT is enabled, the CBT driver is automatically installed and enabled when a Hyper-V node performs a backup.