Problem with TFileStream.seek in Turbo Delphi

PC Pete

unread,

Dec 20, 2007, 7:06:01 AM12/20/07

to

Hi,
I've just started using Turbo Delphi after 25 years with Turbo and Delphi
products.
I'm not sure if this is the appropriate forum, but I couldn't find another
TD-specific support mechanism anywhere.

I'm reading a large data file, using a tfilestream object's seek method to
skip to a known header location, which I then read into a record consisting
of a 4-byte char array, followed by an integer (32-bit normal garden type
integer).

I use that integer value to seek forward in the file to the next header,
using the soFromCurrent constant, and so on through the file.

The problem I have is that for very large files (2G <filesize<4G), using a
normal integer with an absolute value greater than 2G as the seek offset
results in the seek not being executed at all, and the code jumps without
warning out of the statement block and to the end of the procedure, and that
triggers an integer-out-of-bounds error.

I understand that the seek parameter(offset) is a 64-bit integer value, but
typecasting the 32-bit value to either 64bit or longword does exactly the
same thing. I tried using abs(), but that just does stupid things and
returns a very strange result. In fact, abs(2147483648) returns -2147483648,
which is 'absolutely' stretching my understanding of the abs() function,
unless I'm missing somethign very obvious.

I get different results if I abs(int64(value)) than if I int64(abs(value)).

For all other seek operations, the filestream object works fine, it's only
when I try to seek past the 2G mark with a 32-bit value that it all comes
horribly unstuck. This is repeatable with any file.

The same code executed in Delphi CS8 and Delphi 6 works perfectly - on the
same file!

Here's an actual example of the figures:
Filesize : 4,126,339,152 bytes
Current stream position (offset from start of file) : 40 bytes
32-bit integer value to try to seek to, relative to the current position :
4,126,335,152
Running MyFileStream.Seek(4126335152,soFromCurrent) : code jumps out of the
block and to the end of the procedure.

If anyone can point me to a patch reference or another resource, I'd be
happy to fix this.

Thanks in advance for any suggestions.

PC Pete

Here's some examples of the values I'm getting with these functions:
abs(4126335152) = 168632144
int64(4126335152) = 4126335152
longword(4126335152) = 4126335152
int64(abs(4126335152)) = 168632144
abs(int64(4126335152)) = 4126335152

Peter Below (TeamB)

unread,

Dec 20, 2007, 11:56:42 AM12/20/07

to

PC Pete wrote:

> Hi,
> I've just started using Turbo Delphi after 25 years with Turbo and
> Delphi products. I'm not sure if this is the appropriate forum, but
> I couldn't find another TD-specific support mechanism anywhere.
>
> I'm reading a large data file, using a tfilestream object's seek
> method to skip to a known header location, which I then read into a
> record consisting of a 4-byte char array, followed by an integer
> (32-bit normal garden type integer).
>
> I use that integer value to seek forward in the file to the next
> header, using the soFromCurrent constant, and so on through the file.
>
> The problem I have is that for very large files (2G <filesize<4G),
> using a normal integer with an absolute value greater than 2G as the
> seek offset results in the seek not being executed at all, and the
> code jumps without warning out of the statement block and to the end
> of the procedure, and that triggers an integer-out-of-bounds error.
>
> I understand that the seek parameter(offset) is a 64-bit integer
> value, but typecasting the 32-bit value to either 64bit or longword
> does exactly the same thing.

Do not typecast, assign the value to a local variable of type int64.

--
Peter Below (TeamB)
Don't be a vampire (http://slash7.com/pages/vampires),
use the newsgroup archives :
http://www.tamaracka.com/search.htm
http://groups.google.com

PC Pete

unread,

Dec 21, 2007, 8:17:38 AM12/21/07

to

Peter Below (TeamB) wrote:

> Do not typecast, assign the value to a local variable of type int64.

Peter Below (TeamB) wrote:

> Do not typecast, assign the value to a local variable of type int64.

Thanks for the suggestion, Peter - and I tried it last thing last night
(independently, I had an "aha" moment), with no change in the behaviour.:(

I hope no-one minds, but I've included a simple replication of the
problem. I can even cut the actual physical code down to this most basic
of logic, and the error repeats, every time without fail.

The basic procedure skeleton is listed below - anything left out has
nothing to do with the stream, nor does anything touch the stream or
handle (mostly I'm just adding some IntToStrs from the data returned by
the filestream's read method to a debug memo). At no time is the
filestream object referenced or accessed once this procedure is entered.

RFile is a global TFileStream object variable declared in the interface
section of the unit and instantiated just prior to calling this procedure.

MyRecord is defined as a simple record :

ID : array[1..4] of byte;
JSize : integer;

... that's it.

procedure TMyForm.Parse;
var
TempPos : int64;

begin
*** RFile.Position is 20 at this point when the procedure is called.
RFile.Read(MyOtherRec,SizeOf(MyOtherRec));
*** now it's at 40...
if MyOtherRec.RSize <> -1 then
begin
... print some stuff
repeat
RFile.Read(MyRecord,SizeOf(MyRecord));
*** correctly reads 8 bytes - the char array data
*** and the 32-bit size field from the actual physical file's data.
TempPos := MyRecord.JSize;
RFile.Seek(TempPos,soFromCurrent);
*** at this point, if TempPos is > 2G and < 4G,
*** the code immediately jumps to the final end in this procedure
*** But with any other value (100k, 23, 1G, whatever),
*** and the code in the repeat block executes perfectly.
until RFile.Position >= RFile.Size;
end;
end;

If I use a typecast instead (and I know its wrong!) I get the exact same
result, and if I try to fiddle with the value using ABS I get all sorts
of wrong data returned as the stream's seek parameter is set to the
garbage produced by the abs() function for any integer >$80000000

This feels so much like it's my fault, but I can't seem to find the
logic error.

Any suggestions would be most welcome.

PC Pete

unread,

Dec 21, 2007, 8:33:53 AM12/21/07

to

Well, there's an interesting twist...

Continuing along the same lines as not typecasting, I hard-coded the
seek value into my code and ran it, and the same thing happened - the
execution point suddenly jumped to the end of the procedure and I got an
Integer out of range error. Here's the actual code:

RFile.Seek(4126335152,soFromCurrent);

RFile.Size is 4126339152.
RFile.Position when the seek is called is 40 (decimal).

Any ideas? I'm starting to think it's a problem with my integer math,
but both 4126335152 and 4126339152 + 40 are valid 32bit unsigned values.

And remember, if I feed abs() the same value, it returns an invalid
number, so I can't assign the integer field value into a longword and
then to an int64, because I get the same error!

Peter Below (TeamB)

unread,

Dec 21, 2007, 11:49:50 AM12/21/07

to

PC Pete wrote:

> MyRecord is defined as a simple record :
>
> ID : array[1..4] of byte;
> JSize : integer;

If JSize is typed as integer then the block it describes cannot be
larger than High(Integer). Any bit pattern that would correspond to a
Cardinal with values between Cardinal(High(Integer))+1 and
High(Cardinal) would be interpreted as a negative number.

Since chunk sizes in a file cannot be < 0, type JSize as LongWord.

Remy Lebeau (TeamB)

unread,

Dec 21, 2007, 1:13:33 PM12/21/07

to

"PC Pete" <PCP...@audiography.com.au> wrote in message
news:476bbd04$1...@newsgroups.borland.com...

> RFile.Seek(TempPos,soFromCurrent);

soFromCurrent is an ordinal constant:

const
...
soFromCurrent = 1;

By using soFromCurrent, you are forcing the compiler to call the 32-bit
version of Seek():

function Seek(Offset: Longint; Origin: Word): Longint; overload;
virtual;

The 64-bit version of Seek() takes a TSeekOrigin enum instead of an ordinal:

type
TSeekOrigin = (soBeginning, soCurrent, soEnd);

function Seek(const Offset: Int64; Origin: TSeekOrigin): Int64;
overload; virtual;

So, in your code, you need to specify 'soCurrent' instead of 'soFromCurrent'
so that the 64-bit version can be called:

RFile.Seek(MyRecord.JSize, soCurrent);

Gambit

Remy Lebeau (TeamB)

unread,

Dec 21, 2007, 1:15:24 PM12/21/07

to

"PC Pete" <PCP...@audiography.com.au> wrote in message

news:476bc0d2$1...@newsgroups.borland.com...

> Continuing along the same lines as not typecasting, I hard-coded
> the seek value into my code and ran it, and the same thing happened
> - the execution point suddenly jumped to the end of the procedure
> and I got an Integer out of range error.

You are calling the 32-bit version of Seek(), not the 64-bit version. See
my other reply.

Gambit

PC Pete

unread,

Dec 23, 2007, 8:28:38 PM12/23/07

to

That was it! Thank you so much for both explaining AND solving this problem.

I'm still a bit new to the x64 world, and I haven't compiled a
non-CPM/non-DOS/Non-W32 application in a long time, so understanding
which overloads need to be called was a bit of a grey area for me,
especiall with the new flavour of Delphi. But I'll look a bit more
closely at alternative parameter lists and overloaded functions in the
future.

Thanks to both of you for helping me out. I really do appreciate it!

Oh, and Merry Christmas/Happy Saturnalia/Kwanzaa/Hanukkah...
PC Pete

--
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML>
<HEAD>
<META HTTP-EQUIV="CONTENT-TYPE" CONTENT="text/html; charset=windows-1252">
<TITLE></TITLE>
<META NAME="GENERATOR" CONTENT="OpenOffice.org 2.3 (Win32)">
<META NAME="AUTHOR" CONTENT="Peter Naus">
<META NAME="CREATED" CONTENT="20071202;16483703">
<META NAME="CHANGEDBY" CONTENT="Peter Naus">
<META NAME="CHANGED" CONTENT="20071202;17003064">
<STYLE TYPE="text/css">

</STYLE>
</HEAD>
<BODY LANG="en-AU" DIR="LTR">
 

 

Peter
Naus
Audio
Engineering Manager
Audiography
20
Churinga Avenue, Mitcham. Victoria. 3132. Australia.
Freecall: 1300
78 4576
Phone:
+613
8802-4562
e-mail: <A
HREF="mailto:sup...@audiography.com.au">sup...@audiography.com.au</A>
web:
<A
HREF="http://www.audiography.com.au/">http://www.audiography.com.au</A>
 

</BODY>
</HTML>