Delphi 64 bit compiler still limited to 2 GB static data structures.

[Skybuck Flying]

Yesterday/Today 27/28 august 2022 I discover Delphi 10.3 is still limited to 2 gigabyte static data structures.

TstaticArray = array[0..?] of byte;

Where ? is limited to 2 giga.

This could become a problem when wanting to debug a big array, where only a pointer to the array is available.

For now I worked around the problem by converting the static array to a dynamic array as follows:

TdynamicArray= array of byte;

And then passing these arrays to a DLL library as a pointer.

vArray : TdynamicArray;

SetLength(vArray, 100000);
Pass( @vArray[0] );

The debugger is then capable of inspecting all array elements of vArray as long as it does not get to big.

On this 6 GB RAM system it was not possible to debug a 3 GB array, even with dynamic arrays.

Delphi would not visualize the content of the 3 GB array, or it simply took too long.

I hope this will be improved somehow in the future.

Bye for now,
Skybuck.

[Skybuck Flying]

Testing this was a lot more difficult than you would thing, ran into all kinds of issues, like overflows and other things:

For now I am posting the test program, the comments are a bit harsh ! ;)

I consider this test program or the comments to be incomplete, but it's a start.

*** Begin of Test Program ***

program TestProgram;

{

version 0.01 created on 27 august 2022 by Skybuck Flying

Problem 1: Delphi 64 bit compiler has a 2 gigabyte data structure limitation ?!?! WTF ?!?!

When the Platform is set to Windows 64 bit the data structures are still limited
to 2 gigabytes ?!

This is absolutely disgusting. Delphi 10.3 still cannot use constants
or data structures exceeding 2 gigabytes !

WTF ?!?!

This should be fixed for the near future.

Test program below illustrates this very serious limitation, it will not
compile producing the error message:

[dcc64 Error] TestProgram.dpr(29): E2100 Data type too large: exceeds 2 GB

Problem 2: Problem 1 could lead to problems debugging big arrays.


Problem 3: Allocating dynamic array above 2 gigabyte might fail if memory is
fragmented ? So far, after terminating some programs and waiting a little bit,
allocating more than 2 gigabytes of RAM does seem to work.

Tested on windows 7 home edition, toshiba laptop L670 with 6 GB of RAM installed.
and 4 GB of RAM free/available.



}

{$APPTYPE CONSOLE}

{$R *.res}

uses
System.SysUtils;

const
// 3 gigabyte values:

// [dcc64 Error] TestProgram.dpr(40): E2099 Overflow in conversion or arithmetic operation
// Still overflowing with Windows-64-bit platform setting !?!?!?!
// ConstArraySizeV0 : int64 = 3*1024*1024*1024;

// typecast first part of computation to int64 does solve this computation problem but leads
// to other problems further down the road/code.
ConstArraySizeV1a : int64 = int64(3)*1024*1024*1024;
ConstArraySizeV1b : int64 = 3*1024*1024*int64(1024);

// It's possible to use round in constants interesting.
// 2.2 gigabyte value:
// ConstArraySizeV1 : int64 = round(int64(1024)*1024*1024*(2.2)); // typecast int64 does solve this computation problem.

// 3 gigabyte value:
// ConstArraySizeV2 = 3221225472; // Does compile under platform 64 bit.

ConstArraySizeV2 = 100; // Does compile under platform 64 bit.

type
// *** static array tests ***

// Declarations fail but in different ways:

// Cannot use the matching size v0 so disabling this example
// TStaticMaximumArrayV0 = array[0..ConstArraySizeV0-1] of byte;

// Cannot use
// [dcc64 Error] TestProgram.dpr(52): E2026 Constant expression expected
// TStaticMaximumArrayV1a = array[0..ConstArraySizeV1a-1] of byte; // very strange
// TStaticMaximumArrayV1b = array[0..ConstArraySizeV1b-1] of byte; // very strange

// Cannot use
// [dcc64 Error] TestProgram.dpr(53): E2100 Data type too large: exceeds 2 GB
// TStaticMaximumArrayV2 = array[0..ConstArraySizeV2-1] of byte;

// *** dynamic array tests ***
TDynamicMaximumArray = array of byte;

procedure Main;
var
vDynamicMaximumArray : TDynamicMaximumArray;
begin
writeln('program started');

// So far allocating dynamic arrays seems to work,
// but only after I terminated a whole bunch of programs.
// Maybe if the memory is fragmented, it might still fail ?!?
// So far so good though.
SetLength( vDynamicMaximumArray, ConstArraySizeV2 );

vDynamicMaximumArray[0] := 66;

vDynamicMaximumArray[ConstArraySizeV2-1] := 77;

writeln( 'vDynamicMaximumArray[0]: ', vDynamicMaximumArray[0] );
writeln( 'vDynamicMaximumArray[3221225471]: ', vDynamicMaximumArray[ConstArraySizeV2-1] );

writeln( 'press enter to continue');
readln;

writeln('program finished');
end;

begin
try
Main;
except
on E: Exception do
Writeln(E.ClassName, ': ', E.Message);
end;
ReadLn;
end.

// *** End of Program ***

Bye,
Skybuck.