Himemx patch for proper E820h 001 memory block support.

[Rod Pemberton]

This is a Himemx patch for proper E820h 001 memory block support.

Currently, Himemx only supports one 001 E820h memory block at or
above 1MB. I added to and changed Himemx's functionality to allow
Himemx to use multiple 001 E820h memory blocks. This should help
machines that have AGP video controllers which split the memory
region above 1MB. More accurate and complete XMS mapping may also
help with apps that use XMS, e.g., CWSDPMI which doesn't have fall
back memory support for E820h.

I also added the BIOS 8Ah memory call. This should help machines
with Phoenix BIOS which support 8Ah to get the correct size of
memory above 1MB, when no E820h map is available.

I'm calling this version 3.34. Ninho's changes would be 3.33.
Ninho's changes were reported to the "DOS ain't dead" forums.
This patch is against Himemx 3.32 by Japheth. It is not against
3.33 on iBiblio which is possibly Ninho's update.

The himemx.diff is created as 'diff -Naurw'. The extra option
'-w' is because there is a bunch of extra whitespace in the 3.32
source at the end of lines... To create 3.34, place himemx.asm
from 3.32 and my patch into a new folder. Then, 'patch -p0
himemx.asm himemx.dif'. This will create a new himemx.asm. To
build, use the 3.32 make.bat. I built with jwasm v2.10. You may
wish to place a 'cls' in the .bat at the start to bypass a jwasm
crash bug. This bug occurs if compiling in a Win98/SE/ME console
window ("dosbox"). This bug affects a number of versions of
jwasm.

Although I have a machine with multiple 001 blocks above 1MB, I
haven't able to test with it. The changes have only been tested
with a _simulated_ E820h routine that reports multiple 001 blocks
in it's memory map. The simulated E820 routine was installed on a
machine that only has one large memory block at 1MB. I.e., while
I believe the patch to be correct and work correctly, this has not
been tested thoroughly on real hardware and on machine that
actually needs or can use these changes. So, use at your own
risk.

PS. The main reason I'm posting this is so Japheth or Devore can
test is and add it to the versions on his website. If you have
access to the "DOS ain't dead" forums and notify Japheth of the
availability of this patch, thank you in advance.

PPS. I increased my newsreaders line length, so I hope there
is no wrapping of lines in the diff output.


Rod Pemberton



BEGIN---cut-here--

--- himemx.asm 2008-03-11 10:24:50.000000000 +0000
+++ himemx.new 2013-06-05 21:12:52.000000000 +0000
@@ -83,11 +83,21 @@
; - Interrupt window implemented for real-mode EMB move
; - C part abandoned to simplify the make process and reduce binary size
; - test A20 changed (no more memory writes)
+;******************************************************************************
+; 386/486 PMode switch patch by Ninho, tested by RayeR on Am386SX-25
+;******************************************************************************
+; Rod Pemberton 2013 (all changes Public Domain):
+; - fixed version numbers in DRIVER_VER - should be hex - not decimal
+; - added 8Ah call to geti15mem
+; - separated E820 code from geti15mem
+; - fixed E820 code to use multiple 001 blocks
+; - changed E820 code to update handle table directly
+; - corrected E820 code to clamp at /MAX= memory limit, when used

;--- assembly time parameters

-VERSIONSTR equ <'3.32'>
-DRIVER_VER equ 300h+32
+VERSIONSTR equ <'3.34'>
+DRIVER_VER equ 300h+34h
INTERFACE_VER equ 300h

NUMHANDLES equ 48 ;std 48, default number of handles
@@ -104,7 +114,8 @@
EXECMODE_SYS equ 0 ;binary has been loaded as device
EXECMODE_EXE equ 1 ;binary has been loaded as .EXE

-XMS_START equ 1024+64 ; XMS starts at 1088k after HMA
+HMA_SIZE equ 64
+XMS_START equ 1024+HMA_SIZE ; XMS starts at 1088k after HMA

CMD_INIT equ 0 ; init command (used when installed)

@@ -1324,6 +1335,8 @@
mov cr0,eax
;--- the 80386 (and 80486?) need a short delay after switching to PM
;--- before a segment register can be set! Any instruction is sufficient.
+ jmp flsh ; flush both the instruction prefetch and predecode queues
+flsh:
dec ax ; clear PE bit
mov ds,dx
mov es,dx
@@ -1734,7 +1747,9 @@
dw xms_unlock_emb ;13
dw xms_get_handle_info ;14
dw xms_realloc_emb ;15
-
+; dw xms_request_umb ;16 unsupported
+; dw xms_release_umb ;17 unsupported
+; dw xms_realloc_umb ;18 unsupported
dw xms_ext_query_free_mem ; 88
dw xms_ext_alloc_emb ; 89
dw xms_ext_get_handle_info ; 8e
@@ -3505,19 +3520,94 @@
ret
dispmsg endp

-;-- look for extended memory, int 15h, ax/ah 0e820h -> 0e801h -> 88h
-;-- could always add other calls like 0c7h and 8ah to catch oddball cases
+;-- look for extended memory, int 15h, ax/ah 0e801h -> 8ah -> 88h
+;-- could always add other calls, like 0c7h, da88h, e881h, for odd cases
+;-- e820h moved to a separate procedure to implement multiple 001 blocks
;-- out: NC if ok, then memory in kB in eax
;-- C on errors
-;-- modifies eax, ebx, ecx, edx, esi, edi
+;-- modifies eax, ebx, ecx, edx

geti15mem proc
+
+; try 0e801h, but set up the registers to fail status because not
+; all BIOS's properly return the carry flag on failure
+@@e801_check:
+ cmp [_no_above_16],0
+ jne @@try_8Ah ; cannot use 0e801h, per user /NOABOVE16 command
+
+ @DbgOutS <"geti15mem: get extended memory with int 15, E801",13,10>
+
+ xor ax,ax
+ mov bx,ax
+ mov cx,ax
+ mov dx,ax
+ mov ax,0e801h
+ int 15h
+ jc @@try_8Ah
+ mov ax,cx
+ or ax,dx
+ je @@try_8Ah
+
+; if dx is > 0, then cx should be 3c00h since that's full 1-16M range
+; if cx != 3c00h use cx and not dx
+ cmp cx,3c00h
+ je @@e801_compute
+ cmp dx,0
+ je @@e801_compute
+ xor dx,dx
+
+@@e801_compute:
+ movzx edx,dx
+ shl edx,6 ; convert 64K blocks to 1K
+ movzx eax,cx
+ add eax,edx
+ cmp eax,64 ; only use if useful amount
+ ja @@exit
+
+; e801h didn't do the trick, fall back to 8Ah
+@@try_8Ah:
+
+ @DbgOutS <"geti15mem: get extended memory with int 15, 8A",13,10>
+
+ clc
+ mov ah,8ah
+ int 15h
+ pushf
+ shl edx,10h
+ mov dx,ax
+ mov eax,edx
+ popf
+ jnc @@exit
+
+; 8Ah didn't do the trick, fall back to old 88h with 64M max
+@@try_88h:
+
+ @DbgOutS <"geti15mem: get extended memory with int 15, 88",13,10>
+
+ clc
+ mov ah,88h
+ int 15h
+ movzx eax,ax
+@@exit:
+ ret
+geti15mem endp
+
+;-- allocate extended memory from int 15h, ax/ah 0e820h
+;-- handles multiple 001 memory blocks
+;-- updates handle table with correct blocks
+;-- out: NC if ok, then memory in kB in eax
+;-- C on errors
+;-- modifies eax, ebx, ecx, edx, esi, edi
+
+e820alloc proc
+ xor eax,eax
cmp [_x_option],0
- jne @@e801_check ; cannot use 0e820h, per user /X command
+ stc
+ jne @@exit ; cannot use 0e820h, per user /X command

@DbgOutS <"get15mem: get extended memory with int 15, E820",13,10>

-; try 0e820h first
+; try 0e820h

push ebp
xor ebx,ebx
@@ -3526,6 +3616,8 @@
push ds
pop es

+ mov bp,@word [xms_handle_table.xht_pArray]
+
; ebx offset is updated with each successive int 15h
@@e820_loop:
mov edx,SMAP
@@ -3552,91 +3644,61 @@
jne @@e820_loop
mov eax,[di].E820MAP.baselow
cmp eax,100000h ; has to live in extended memory
- setz dl
jb @@e820_loop
+ pushf
+ shr eax,10
+ mov edx,[di].E820MAP.lenlow
+ shr edx,10
+ popf
+ jne @@not1stblock
+ mov eax,XMS_START
+ sub edx,HMA_SIZE ; subtract size of HMA
+@@not1stblock:
+ push ebx
+ mov bx,bp
+ mov [bx].XMS_HANDLE.xh_baseK,eax ; init blocks
+ mov [bx].XMS_HANDLE.xh_sizeK,edx
+ mov [bx].XMS_HANDLE.xh_flags,XMSF_FREE
+ add bx,size xms_handle
+ mov bp,bx
+ pop ebx

- cmp esi,0
- jne @@e820_checkhole
+; add to the memory count
+ add esi,[di].E820MAP.lenlow
+ jc @@overflowed

-; we're not able to handle extended base start not exactly at 100000h
-; not big deal to add support later (does this happen, though?)
- cmp dl,1
- jne @@e820_done
- mov ebp,eax
- jmp @@e820_matchcrit
-
-; check that there isn't a hole in memory, stop at the hole if detected
-; this presumes the map will return contiguous addresses rather than a spray
-@@e820_checkhole:
- mov eax,ebp
- add eax,esi
- cmp eax,[di].E820MAP.baselow
- jne @@e820_done ; current base plus memory length not equal to this base
+; clamp e820 memory to /MAX= memory limit, if set, and exit
+ mov eax,[_xms_max]
+ or eax,eax
+ je @@e820_loop ; no maximum XMS set
+ shl eax,10
+ cmp esi,eax
+ jbe @@e820_loop ; at or below maximum
+
+ mov bx,bp
+ sub bx,size xms_handle
+ sub esi,eax ; amount over
+ shr esi,10
+ mov eax,[bx].XMS_HANDLE.xh_sizeK
+ sub eax,esi ; reduce to limit
+ mov [bx].XMS_HANDLE.xh_sizeK,eax ; above max, limit to maximum

-; matched all the criteria, add to the memory count
-@@e820_matchcrit:
- add esi,[di].E820MAP.lenlow
- jnc @@e820_loop
- mov esi,-1 ; wow, we overflowed a 4G counter, force a limit
+@@overflowed: ; overflowed a 4G counter or reached /MAX= limit
+ clc
jmp @@e820_done

@@e820_bad:
+ stc
xor esi,esi ; force failure

@@e820_done:
pop ebp
- mov eax,esi
- shr eax,10 ; convert from bytes to 1K blocks
- cmp eax,64 ; only use if useful amount
- ja @@exit
-
-; try 0e801h, but set up the registers to fail status because not
-; all BIOS's properly return the carry flag on failure
-@@e801_check:
- cmp [_no_above_16],0
- jne @@try_88h ; cannot use 0e801h, per user /NOABOVE16 command
-
- @DbgOutS <"geti15mem: get extended memory with int 15, E801",13,10>
-
- xor ax,ax
- mov bx,ax
- mov cx,ax
- mov dx,ax
- mov ax,0e801h
- int 15h
- jc @@try_88h
- mov ax,cx
- or ax,dx
- je @@try_88h
-
-; if dx is > 0, then cx should be 3c00h since that's full 1-16M range
-; if cx != 3c00h use cx and not dx
- cmp cx,3c00h
- je @@e801_compute
- cmp dx,0
- je @@e801_compute
- xor dx,dx
-
-@@e801_compute:
- movzx edx,dx
- shl edx,6 ; convert 64K blocks to 1K
- movzx eax,cx
- add eax,edx
- cmp eax,64 ; only use if useful amount
- ja @@exit
-
-; e801h didn't do the trick, fall back to old 88h with 64M max
-@@try_88h:

- @DbgOutS <"geti15mem: get extended memory with int 15, 88",13,10>
-
- clc
- mov ah,88h
- int 15h
- movzx eax,ax
@@exit:
ret
-geti15mem endp
+
+e820alloc endp
+

;--- driver init. this proc should be last
;--- since it initializes the handle table
@@ -3680,7 +3742,7 @@
mov dx,@offset vdisk_detected
jz @@error_exit

- call geti15mem ; look for extended memory via int 15h
+ call geti15mem ; look for non-E820 extended memory via int 15h
mov dx,@offset xms_sizeerr
jc @@error_exit

@@ -3691,7 +3753,7 @@
jbe @@save_size ; at or below maximum
mov eax,edx ; above max, limit to maximum
@@save_size:
- sub eax,64 ; subtract size of HMA
+ sub eax,HMA_SIZE ; subtract size of HMA
mov [xms_size],eax ; save size
mov dx,@offset xms_toosmall
jc @@error_exit ; if there aren't 64k, there's nothing to do
@@ -3784,6 +3846,12 @@
mov [bx].XMS_HANDLE.xh_baseK,XMS_START ; init first block and give
mov [bx].XMS_HANDLE.xh_sizeK,esi
mov [bx].XMS_HANDLE.xh_flags,XMSF_FREE
+
+; handle table has single memory block found by older memory calls
+; if e820 is present, rework handle table with e820 map
+; this supports multiple memory blocks found by e820
+ call e820alloc ; allocate e820 memory via int 15h
+
@@exit:
@DbgOutS <"initialize exit",13,10>
popad

END---cut-here--

[drives...@gmail.com]

Hi Rod,

thanks for the patch.

BUT:
this patch is by far too big and complicated to be used without *significant* testing and/or reviewing.

> Although I have a machine with multiple 001 blocks above 1MB, I
> haven't able to test with it.

could you please provide more details why you could not test ?
could you please provide more details what you tested ?

Tom

[Rod Pemberton]

<drives...@gmail.com> wrote in message
news:e4275113-e7f2-4e6b...@googlegroups.com...
> thanks for the patch.
>

You're welcome.

> BUT:
> this patch is by far too big and complicated to be used

Well, no, not really. Basically, it has a new routine for E820.
The rest is trivial changes.

I separated the existing E820 routine from geti15mem. That's
trivial. I rewrote the E820 code for multiple blocks and /MAX.
That's not so trivial, but wasn't difficult. I also added Ninho's
jmp+$2 patch, the 8Ah BIOS memory call to geti15mem, hex version
numbers, and few label name changes, etc. Those are all trivial,
IMO.

If you're trying to understand what is happening from the patch
itself, don't. The patch is confusing to me... Patch 3.32. Read
the updated code. That's far less confusing.

> without *significant* testing and/or reviewing.

I agree.

I clearly stated that I was trying to get the patch to Japheth or
Devore for testing. They apparently worked on it previously, and I
thought they should be able to test it. Rugxulo mentioned it on
the "DOS Ain't Dead" forums for me, where Japheth has said he
doesn't have a machine with multiple 001 blocks to test. Japheth
hasn't said anything about being able to do any regression testing,
i.e., checking that no existing functionality was unintentionally
broken.

I've suggested to Rugxulo that maybe it should just be packaged and
labeled as experimental for now. That would allow those who think
they might need the extra E820h functionality to use it, without
encouraging everyone to switch to it before it's been tested
enough. I don't know what situations cause the 001 block at 1MB to
be split into multiple blocks or why some machines have multiple
001 blocks above 1MB. For that machine, it seems to be the AGP
video card. If AGP is the common case, then not many people will
need this patch.

(Also, this is being discussed on comp.os.msdos.djgpp in the
unrelated thread "Rebuilding 2.04 from source." and the "DOS Ain't
Dead" forums too.)

I don't intend to sound "snippy" but did you actually read my post
or did you just reply to it without reading it...? I get the same
impression below too.

> > Although I have a machine with multiple 001 blocks above 1MB, I
> > haven't able to test with it.
>
> could you please provide more details why you could not test ?

I don't have any machines old enough that they don't have an E820h
BIOS call. These should be pre-1996 or so. However, that can be
simulated with HIMEMX's /X option on machines with E820h.

I can't test my one machine with multiple E820h 001 memory blocks
currently. It's not operable at the moment.

I've not confirmed the 8Ah BIOS call works. It's available on the
same machine. AFAICT, this call is only available on mid 1990's
BIOSes by Phoenix. 8Ah is not capped at 64MB like 88h. 8Ah can
return more than 64MB of memory when installed. So, I didn't code
an simulated BIOS routine for 8Ah. All I know is that the 8Ah call
doesn't cause problems on this machine which has a different BIOS
vendor and no 8Ah call, i.e., no lockup, no crash, use of carry
flag for not present works correctly, etc.

> could you please provide more details what you tested ?

I tested the new E820h routine with the BIOS call for this machine
which has only one 001 block at 1MB.

I tested the new E820h routine with a simulated E820h routine that
reported two 001 blocks at or above 1MB. The routine is a TSR
which I coded that replaced the original BIOS interrupt for the
E820 BIOS call with a new E820 routine. The returned E820 memory
map was exactly the same as reported by the machine with multiple
001 blocks. HIMEMX reports two XMS blocks that correspond to the
correct 001 regions in the memory map. This was checked with
Japheth's XMSSTAT from his JEMM package and XMSTAT from Franck
Uberto's XMSDSK ramdisk package FU_RD19I.ZIP.

I tested the /MAX parameter with the new E820 routine to confirm it
limited memory.

I think this was all stated previously either in my post or the
comments in the patch... If not, it was in the unreleated thread
on comp.os.msdos.djgpp with Rugxulo "Rebuilding 2.04 from source."


Rod Pemberton