fopen on big files fails

[Timothy S. Sullivan]

So I'm doing some scientific programming in C on a RedHat 7.1 Beowulf
cluster and my friend is working on the same problem in IDL on a Windows
box. (He's prototyping algorithms and I'm trying to parallelize the
resulting code.) I'm feeling like I can pretty much handle anything he
sends my way, because I've got Linux and he's got Windows (2000).

So we are scaling the problem up and he ftp's me a 9.8 GB file that acts
as input to the program. I run the code and it seg faults, which I trace
to the failure of the fopen statement that tried to open the file. It
gives me errno 27, file too big.

I check that my ulimit -a values are unlimited. I run the code as root. No
go.

So now this guy is "whoop whoop Windows the Man! Linux a sham!" Somebody
has got to help me save face here and uphold Linus' honor by telling me
how to get around this. I need to stick with 7.1, so please only suggest
upgrading the OS if this is a known bug and it is known that the upgrade
will fix it.

Thanks!

Tim Sullivan
Timothy....@EarthLink.Net

[Robert Grizzard]

Timothy S. Sullivan <Timothy....@earthlink.net> wrote:

[...]

> So now this guy is "whoop whoop Windows the Man! Linux a sham!" Somebody
> has got to help me save face here and uphold Linus' honor by telling me
> how to get around this. I need to stick with 7.1, so please only suggest
> upgrading the OS if this is a known bug and it is known that the upgrade
> will fix it.

http://www.daimi.au.dk/~kasperd/comp.os.linux.development.faq.html

Look for the boxed-off section in the 13th Q for the answer.

(If you're running a 2.2 kernel you may run into file system limitations
there as well. I had to upgrade to a 2.4 kernel to *write* a file
bigger than 2 GB; I don't recall if I had to play with it to get it to
read one that big or not.)

> Thanks!

HTH
--
The appearance of my E-mail address in any venue does not in and of itself
constitute a solicitation of bulk or commercial E-mail.

I don't want unsolicited commercial E-mail.