Strange behavior in Windows 8.1 HVM (e.g., Group Policy settings reset seemingly at random)
fshar...@gmail.com
I have a Windows 8.1 HVM that I'm running in Qubes, and I'm getting some strange behavior from it. I thought I'd write this up and see if it matches anyone else's experiences.
I created the Windows 8.1 VM as a .vhdx in Hyper-V. I then converted it to an .img file with StarWind Converter and imported it into Qubes using qvm-create with the --root-copy-from flag.
When I install the Xen PV drivers, the VM becomes unresponsive shortly afterward. It doesn't visibly freeze. The mouse cursor still moves (and has a "busy" icon), but the VM ignores keyboard input, mouse clicks, and shutdown signals from Qubes. I have to kill the VM, and after that it won't boot any more, so I have to re-create it. This isn't a showstopper - the only driver I really wanted was the storage driver so I could attach drives to the VM, and I can work around that with Samba.
I've seen some other, minor, seemingly random issues in the VM that I've never seen from Windows before. For example, sometimes when I try to run an .exe to install a Xen PV driver, Windows refuses and says "Error performing inpage operation".
When I build a Windows VM, I go through the CIS guidelines and make the recommended changes to Group Policy. The other day, the VM reset about two dozen of these GP settings to Not Configured/Not Defined. The settings were all located in Local Computer Policy/Windows Settings/Security Settings/Local Policies/Security Options. Other than that, they seemed to have been selected randomly. I had made changes to the settings in other sections of Group Policy, but all the other sections were left alone. (I used the MMC to dump all the settings to text, and used WinMerge to compare them with settings exported from the original .vhdx.)
Just to make sure the GP settings hadn't been reset earlier without my noticing, I opened the original .vhdx in Hyper-V, and the converted .img file in QEMU. In both cases, the settings were intact.
I could be wrong, but to me these things suggest issues with the integrity of the virtual disk Windows is running on. One thing I'll do today is run chkdsk on the VM in Qubes, and probably on the .vhdx and .img files as well.
A few wild guesses as to what might be going on:
- The VM might be losing something during the conversion from .vhdx to .img or during the import into Qubes?
- I believe Qubes uses QEMU to run HVMs. Maybe QEMU and Windows don't get along?
- The Windows VM runs on a virtual disk formatted with NTFS. Maybe there are issues in running that on top of an ext4 file system?
- Prior to the resetting of the GP settings, I had Qubes pause the Windows VM while I had the Group Policy editor open. When I resumed the VM, I was suddenly in a different section of the GP settings than I had been. As I remember it, it was like watching a tape being fast forwarded, as if the VM had abruptly processed a bunch of buffered input. There's no way I could have accidentally reset the GP settings, however. Anyway, maybe HVMs don't like to be paused and resumed?
Thank you for any input, and if I learn anything in my investigations I'll add it.
fshar...@gmail.com
Before:
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System]
"EnableVirtualization"=dword:00000001
"EnableInstallerDetection"=dword:00000001
"PromptOnSecureDesktop"=dword:00000001
"EnableLUA"=dword:00000001
"EnableSecureUIAPaths"=dword:00000001
"ConsentPromptBehaviorAdmin"=dword:00000002
"ValidateAdminCodeSignatures"=dword:00000000
"EnableUIADesktopToggle"=dword:00000000
"EnableCursorSuppression"=dword:00000001
"ConsentPromptBehaviorUser"=dword:00000000
"dontdisplaylastusername"=dword:00000001
"legalnoticecaption"=""
"legalnoticetext"=""
"scforceoption"=dword:00000000
"shutdownwithoutlogon"=dword:00000001
"undockwithoutlogon"=dword:00000001
"FilterAdministratorToken"=dword:00000001
"NoConnectedUser"=dword:00000003
"DontDisplayLockedUserId"=dword:00000003
"DisableCAD"=dword:00000000
"InactivityTimeoutSecs"=dword:00000258
"HideFastUserSwitching"=dword:00000001
"MSAOptional"=dword:00000001
"DisableShutdownNamedPipe"=dword:00000001
"DisableAutomaticRestartSignOn"=dword:00000001
"DisplayLastLogonInfo"=dword:00000000
After:
[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System]
"HideFastUserSwitching"=dword:00000001
"MSAOptional"=dword:00000001
"DisableShutdownNamedPipe"=dword:00000001
"DisableAutomaticRestartSignOn"=dword:00000001
"DisplayLastLogonInfo"=dword:00000000
Two subkeys under System, Kerberos and UIPI, were also deleted.
I haven't seen registry entries just disappear before, so that again makes me wonder if there's a problem with the integrity of the virtual disk. (FWIW, the physical disk I'm running Qubes on is a brand-new Samsung 860 Pro.)
I then ran chkdsk on the Win 8.1 .vhdx (Hyper-V) file, the .img (Raw) file, and the HVM in Qubes.
1. chkdsk on the Win 8.1 .vhdx, prior to the conversion to .img format.
The type of the file system is NTFS.
The volume is in use by another process. Chkdsk
might report errors when no corruption is present.
WARNING! F parameter not specified.
Running CHKDSK in read-only mode.
Stage 1: Examining basic file system structure ...
97280 file records processed.
File verification completed.
613 large file records processed.
0 bad file records processed.
Stage 2: Examining file name linkage ...
135394 index entries processed.
Index verification completed.
0 unindexed files scanned.
0 unindexed files recovered.
Stage 3: Examining security descriptors ...
Security descriptor verification completed.
19058 data files processed.
CHKDSK is verifying Usn Journal...
4964528 USN bytes processed.
Usn Journal verification completed.
Windows has scanned the file system and found no problems.
No further action is required.
12222463 KB total disk space.
10157700 KB in 74849 files.
63600 KB in 19059 indexes.
0 KB in bad sectors.
122547 KB in use by the system.
18864 KB occupied by the log file.
1878616 KB available on disk.
4096 bytes in each allocation unit.
3055615 total allocation units on disk.
469654 allocation units available on disk.
Since no problems were detected, I did not run chkdsk /f.
2. I loaded the .img file into QEMU and ran chkdsk /f on it, prior to the import into Qubes.
Checking file system on C:
The type of the file system is NTFS.
A disk check has been scheduled.
Windows will now check the disk.
Stage 1: Examining basic file system structure ...
97280 file records processed.
File verification completed.
613 large file records processed.
0 bad file records processed.
Stage 2: Examining file name linkage ...
135396 index entries processed.
Index verification completed.
0 unindexed files scanned.
0 unindexed files recovered.
Stage 3: Examining security descriptors ...
Cleaning up 376 unused index entries from index $SII of file 0x9.
Cleaning up 376 unused index entries from index $SDH of file 0x9.
Cleaning up 376 unused security descriptors.
Security descriptor verification completed.
19059 data files processed.
CHKDSK is verifying Usn Journal...
5136936 USN bytes processed.
Usn Journal verification completed.
CHKDSK discovered free space marked as allocated in the volume bitmap.
Windows has made corrections to the file system.
No further action is required.
12222463 KB total disk space.
10142220 KB in 74857 files.
63616 KB in 19060 indexes.
0 KB in bad sectors.
122543 KB in use by the system.
18864 KB occupied by the log file.
1894084 KB available on disk.
4096 bytes in each allocation unit.
3055615 total allocation units on disk.
473521 allocation units available on disk.
Internal Info:
00 7c 01 00 e8 6e 01 00 4d 14 03 00 00 00 00 00 .|...n..M.......
4f 00 00 00 2f 00 00 00 00 00 00 00 00 00 00 00 O.../...........
Windows has finished checking your disk.
Please wait while your computer restarts.
3. I ran chkdsk again (without the /f flag) on the same .img file in QEMU, just to see if there were any more problems.
The type of the file system is NTFS.
The volume is in use by another process. Chkdsk
might report errors when no corruption is present.
WARNING! F parameter not specified.
Running CHKDSK in read-only mode.
Stage 1: Examining basic file system structure ...
97280 file records processed.
File verification completed.
612 large file records processed.
0 bad file records processed.
Stage 2: Examining file name linkage ...
135404 index entries processed.
Index verification completed.
0 unindexed files scanned.
0 unindexed files recovered.
Stage 3: Examining security descriptors ...
Security descriptor verification completed.
19063 data files processed.
CHKDSK is verifying Usn Journal...
5264264 USN bytes processed.
Usn Journal verification completed.
The Volume Bitmap is incorrect.
Windows has checked the file system and found problems.
Please run chkdsk /scan to find the problems and queue them for repair.
12222463 KB total disk space.
10156144 KB in 75023 files.
63696 KB in 19064 indexes.
0 KB in bad sectors.
123055 KB in use by the system.
18864 KB occupied by the log file.
1879568 KB available on disk.
4096 bytes in each allocation unit.
3055615 total allocation units on disk.
469892 allocation units available on disk.
I don't know if those errors were caused by running chkdsk while Windows was running, but still, they weren't there when I ran chkdsk on the original .vhdx.
4. chkdsk /f on the Win 8.1 HVM in Qubes (where the Group Policy settings had been reset).
Checking file system on C:
The type of the file system is NTFS.
A disk check has been scheduled.
Windows will now check the disk.
Stage 1: Examining basic file system structure ...
97280 file records processed.
File verification completed.
613 large file records processed.
0 bad file records processed.
Stage 2: Examining file name linkage ...
135310 index entries processed.
Index verification completed.
0 unindexed files scanned.
0 unindexed files recovered.
Stage 3: Examining security descriptors ...
Cleaning up 386 unused index entries from index $SII of file 0x9.
Cleaning up 386 unused index entries from index $SDH of file 0x9.
Cleaning up 386 unused security descriptors.
Security descriptor verification completed.
19016 data files processed.
CHKDSK is verifying Usn Journal...
6960608 USN bytes processed.
Usn Journal verification completed.
CHKDSK discovered free space marked as allocated in the volume bitmap.
Write failure with status 0xc0000015 at offset 0x2e9fffe00 for 0x200 bytes.
The second NTFS boot sector is unwriteable.
Internal Info:
00 7c 01 00 df 6d 01 00 1f 12 03 00 00 00 00 00 .|...m..........
4e 00 00 00 2f 00 00 00 00 00 00 00 00 00 00 00 N.../...........
Windows has finished checking your disk.
Please wait while your computer restarts.
Looks even worse.
If I am right that my issues are caused by problems with the integrity of the virtual disk, and if this chkdsk output is indicative of these issues, it looks like the problems start when I convert the .vhdx file to .img format.
Previously, I used StarWind Converter to do the conversion. I tried using qemu-img for Windows instead. The .img files produced by both these tools had the same size, so I gather they work in similar ways.
I then ran chkdsk on the .img file produced by qemu-img. It gave me the same output as steps 2 and 3 above.
So, I guess what I need to do is find some other way to convert my .vhdx file to .img. Or find a way to convert it into some other format that Qubes can then import as an HVM.
I could just rebuild my Windows 8.1 HVM from the ground up in Qubes, but I'd really rather avoid that if possible, especially as I don't yet know how to export it/back it up to a separate disk.
Thank you again for any input.
awokd
> If I am right that my issues are caused by problems with the integrity of
> the virtual disk, and if this chkdsk output is indicative of these issues,
> it looks like the problems start when I convert the .vhdx file to .img
> format.
>
> Previously, I used StarWind Converter to do the conversion. I tried using
> qemu-img for Windows instead. The .img files produced by both these tools
> had the same size, so I gather they work in similar ways.
>
> I then ran chkdsk on the .img file produced by qemu-img. It gave me the
> same output as steps 2 and 3 above.
>
> So, I guess what I need to do is find some other way to convert my .vhdx
> file to .img. Or find a way to convert it into some other format that
> Qubes can then import as an HVM.
have the same bug, though.
fshar...@gmail.com
> have the same bug, though.
Thank you, I'll try that next. Looks like qemu-img on Linux also supports .vhdx format, which will help.
fshar...@gmail.com
Anyway, I loaded the .vhdx into QEMU and ran chkdsk /f. Result:
Checking file system on C:
The type of the file system is NTFS.
A disk check has been scheduled.
Windows will now check the disk.
Stage 1: Examining basic file system structure ...
97280 file records processed.
File verification completed.
613 large file records processed.
0 bad file records processed.
Stage 2: Examining file name linkage ...
135400 index entries processed.
Index verification completed.
0 unindexed files scanned.
0 unindexed files recovered.
Stage 3: Examining security descriptors ...
Cleaning up 377 unused index entries from index $SII of file 0x9.
Cleaning up 377 unused index entries from index $SDH of file 0x9.
Cleaning up 377 unused security descriptors.
Security descriptor verification completed.
19061 data files processed.
CHKDSK is verifying Usn Journal...
5135624 USN bytes processed.
Usn Journal verification completed.
CHKDSK discovered free space marked as allocated in the volume bitmap.
Windows has made corrections to the file system.
No further action is required.
12222463 KB total disk space.
10142472 KB in 74856 files.
63616 KB in 19062 indexes.
0 KB in bad sectors.
122543 KB in use by the system.
18864 KB occupied by the log file.
1893832 KB available on disk.
4096 bytes in each allocation unit.
3055615 total allocation units on disk.
473458 allocation units available on disk.
Internal Info:
00 7c 01 00 e9 6e 01 00 4f 14 03 00 00 00 00 00 .|...n..O.......
4f 00 00 00 2f 00 00 00 00 00 00 00 00 00 00 00 O.../...........
Windows has finished checking your disk.
Please wait while your computer restarts.
That looks better, but it still isn't as clean as running chkdsk on the same .vhdx in Hyper-V (which reports "Windows has scanned the file system and found no problems.") So it makes me wonder if QEMU just isn't very fond of .vhdx images, or converted images that start out as .vhdx images.