Safely Remove Hardware Appears on HW v7 VMs

Recently I noticed an issue with some Citrix VMs which were displaying the Safely Remove Hardware option to users logged in to that server.

These VMs were on Hardware Version 7 with the latest VMware Tools installed. Both of the above hardware components were using two of the HW7 only drivers, PVSCSI and VMXNET3.  The below KB article explains how to disable the HotPlug capability so that the ‘Safely Remove Hardware’ message no longer appears in the system tray.

KB article 1012225

This worked successfully on Windows Server 2003 VMs. On Windows Server 2008 VMs the message was only appearing when logged in with administrative privileges, again the KB article changes worked. (Actually they didn’t on 2008 - see below update)

One curiosity with the KB article was that it stated this change would not work with the VMXNET3 driver, however in both Windows 2003 and 2008 it was successful. Looking back at the KB today to make this post the article seems to have been updated (July 19th) to remove this caveat and now only states the more vague:

Note: You can disable the HotPlug capability for PCI devices such as e1000 or vmxnet2 NICs”

Update: Actually the devices.hotplug = “false” suggestion in the KB article does not seem to work on Windows Server 2008 or 2008 R2.

Even with the above setting applied the below is still available from ‘Safely Remove Hardware’ in Windows Server 2008 and 2008 R2.

I thought I would test out and see what happens if you try to eject either device. Attempting to remove the disk results in the below, i.e. it won’t actually let you do it.

However, ejecting the network card sees no warning or blocking prompts and the network is consequently disconnected.

The network card is now no longer available to be removed.

Looking at the configuration of the VM you will see that the network adapater has been removed.

However, it is possible to add it back whilst powered on and the drivers are installed.

I’ll post a further update after my support ticket with VMware is answered.

Update: 20/08/2010

VMware Support acknowledged the issue and have provided a workaround for vSphere 4.0 and said that is was already fixed in 4.1. The (long!) workaround for 4.0 is listed below and is VM specific, i.e. the change would need to be made manually on any affected VM. I have successfully tested this workaround on both Windows Server 2008 and 2008 R2. They indicated that the KB article would be updated to reflect this, but so far no change to it - this was about three weeks ago, not sure how long these changes take to filter through.

vSphere 4.0 Workaround Begin

Verifying the ID of the NIC device you want to remove from being shown as a Removable Device

1. Click Start > Run and type devmgmt.msc to launch Device Manager. 2. Expand the appropriate device category (for example, Network Adapters or Storage Controllers). 3. Right-click the device and go to Properties, then click the Details tab. 4. In Windows XP and Server 2003, the Device Instance ID is selected by default. In Windows 7 and Server 2008, select Device Instance Path from the Property drop-down list.

The ID/Path looks similar to PCI\VEN_1022&DEV_2000&SUBSYS_20001022&REV_10\4&47B7341&0&1088.

Note: The Device ID/Path may be different, depending on the device type of the virtual machine

5. Highlight the instance ID and copy it to a notepad as you need it later in this section.

Modifying the Windows registry to hide the device from the Windows Removable Devices

1. Click Start > Run and type regedit to launch Registry Editor. 2. Navigate to HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Enum. 3. Use the sections of the ID to locate the device. For example: PCI, then VEN_1022&DEV_2000&SUBSYS_20001022&REV_10 then 4&47B7341&0&1088. 4. By default, you likely do not have permission to modify the value, so right-click on the key (folder), go to Permissions and give your account Full Control. 5. Inside that key is a value named Capabilities, which has a value of 4 or 6 or higher. 6. Right-click the value and click Modify, then subtract 4 from the value and click OK. For example: 6 to 2, 4 to 0, etc.

Note: It may take Windows a moment to hide the NIC from the Safely Remove Hardware list.

In Windows 2008, the local Administrator account does not have write permissions in the section of registry this workaround edits. To workaround this, the user level can be temporarily raised to run as the Local System and the appropriate permissions then applied to the registry. To do this:

1. Click Start > Run and type cmd to launch Command Prompt. 2. Take note of the current time. In the command window, enter the following (where 16:30 is replaced with a time one or two minutes ahead of the current time): at 16:30 /interactive cmd.exe 3. This spawns a cmd window that is running as the Local System account. Type regedit.exe to open Regedit as Local System. 4. Navigate to the necessary key, apply Full Control permissions to the local Administrator account. 5. Close Regedit and the cmd window.

Jonathan’s Note: There is an easier way to achieve running regedit with local system permissions using the sysinternals tool psexec.

psexec -i -s regedit

Exporting and editing the Windows registry keys

1. In the Windows Registry, click File > Export and save the key on your computer. 2. Open the key with a text editor and remove all the extra subkeys and values that you do not want to change. In the end it should look similar to:

Windows Registry Editor Version 5.00 [HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Enum\PCI\VEN_1022&DEV_2000&SUBSYS_20001022&REV_10\4&47b7341&0&1088] “Capabilities”=dword:00000002

Creating a Windows batch script to make the registry changes

1. Open Notepad. 2. Enter the following line (update the path for where you saved the file and what you named it):

regedit /s “C:\Startup\vmnicregedit.reg”

3. Save the file with a .bat extension.

Creating a local policy on the system to run the batch script at start up

1. Click Start > Run and type gpedit.msc. 2. Expand Computer Configuration. 3. Expand Windows Settings. 4. Select Scripts (Startup/Shutdown) in the left window pane. 5. Double-click Startup in the right window pane. 6. Click Add. 7. Click Browse and browse to the .bat file created above.

Testing the workaround

After applying all the changes above, reboot the virtual machine. The network adapter should not show up as a removable device.

vSphere 4.0 Workaround End

vSphere 4.1

After I tested the above workaround I then fired up a Windows Server 2008 VM on vSphere 4.1 and without any modifications the issue was still present. This time though when I added the devices.hotplug = “false” configuration to the VM, the Safely Remove Hardware icon was completly removed from the task bar - so the support engineer was correct, it is resolved in 4.1. See below - there is no Safely Remove Hardware icon.

It’s amost worth upgrading to vSphere 4.1 just for that rather than having to follow the lengthy workaround! I did quiz the engineer as to whether we would see a fix in 4.0. He said he hoped so, but it was not guaranteed and would be fed back to engineering, so I guess we wait and see.