Some fixes for PAE mode on XP SP3
Jon Larimer
While testing out hyperdbg on XP SP3 with PAE, I ran into a few issues:
1. Building hyperdbg-pae will fail unless the project was already
built as hyperdbg. This happens because the first argument to
build_code.cmd is passed to the Build command as the directory to
build. If you pass hyperdbg-pae, it won't build the code in the
hyperdbg directory. I fixed this by adding a BUILD_DIR variable that
gets set to 'hyperdbg' when hyperdbg or hyperdbg-pae are specified to
build. When other plugins or modules or added, the build script can
easily add support for those too. I attached the fix for this as
build_code-hyperdbg-pae-fix.txt.
2. Once built, hyperdbg failed to start. There wasn't an error
message, so I added some logging and did some debugging. The problem
was in MmuInit() in mmu.c. The code was trying to read a PAGE_SIZE
chunk of memory from CR3, but on my system CR3 is set to 0x0AD40020
and the read failed because it will cross a page boundary:
> [vmm] [MMU] Error: physical region 0ad40020-00000000 crosses multiple frames
Since CR3 doesn't point to a page table in PAE mode (it points to 4
64-bit PDPTE's), this code doesn't need to read a full page of bytes.
It actually doesn't need to read anything at all, since hostpt isn't
being used yet. For now, I fixed this by only reading 4*16 bytes of
data instead of a whole page. This works on my machine.
3. Once #2 was fixed, hyperdbg bluescreened with an invalid
instruction on the VMCLEAR instruction. The problem was that hyperdbg
was passing an invalid physical memory address for the VMCS region.
This was because MmuGetPhysicalAddress() was returning a 64-bit value
with the high bit (XD, the non-exec bit) set. This happened because
PTE.PageBaseAddr in x86.h was set to use the top 52 bits of the 64 bit
PTE. It should actually be 36 for PAE mode. It would be better to get
the real MAXPHYSADDR value from the CPUID instruction, but it's always
going to be 36 for PAE.
After fixing #2 and #3, hyperdbg starts up and can break in and out
with F12. And that leads us to #4...
4. When stopping hyperdbg (with 'net stop pill') my system hangs. The
attached kernel debugger (external, on a 2nd machine over
1394/firewire) doesn't break so the hang must be in root mode and the
kernel doesn't get a chance to handle whatever's wrong. I'm still
investigating this.
5. I also fixed an issue with xpvideo.c where sometimes VideoAlloc()
would fail in video.c. I fixed it by setting the framebuffer_size
value to just the amount needed for the screen.
The patch for issues #2, #3, and #5 (along with some extra logging in
DriverEntry() and MmuInit() is attached as mmu-pae-fixes.txt.
-jon
Jon Larimer
jlar...@gmail.com