Groups keyboard shortcuts have been updated
Dismiss
See shortcuts

fread() file size limit?

3,354 views
Skip to first unread message

Michael

unread,
Jan 1, 2009, 4:58:44 AM1/1/09
to nzp...@googlegroups.com
I am trying to read a 1.9Gb text file and fread() barfs. How can I fix this
situation?

I have already upped php.ini "memory_limit" to 2500Mb.

Michael

Tony Rigden

unread,
Jan 1, 2009, 1:28:45 PM1/1/09
to NZ PHP Users Group
Are you trying to load the entire file into memory? fread will read
only a maximum of 8192 bytes at a time. See http://nz2.php.net/fread.
It is not desirable to the memory limit that high nor should it be
needed. Most people have it set to a max of about 32 Mb if they are
using graphics or less if not.

Tony

Michael

unread,
Jan 1, 2009, 4:50:21 PM1/1/09
to nzp...@googlegroups.com
On Fri, 02 Jan 2009 07:28:45 Tony Rigden wrote:
> Are you trying to load the entire file into memory? fread will read
> only a maximum of 8192 bytes at a time. See http://nz2.php.net/fread.
> It is not desirable to the memory limit that high nor should it be
> needed. Most people have it set to a max of about 32 Mb if they are
> using graphics or less if not.
>
> Tony

Do you have any idea what could be causing this then please?

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to
allocate 2023545829 bytes) in /www/scan.php on line 145

Line 145 is a call to fread()

Kind regards,

Michael

Matthew White

unread,
Jan 1, 2009, 5:01:20 PM1/1/09
to nzp...@googlegroups.com
How are you using fread?

Something like

$contents = fread($handle, filesize($filename));

Will read the whole thing into memory, which is not something you
really want to do as previously noted

Can you tell us a little more about the nature of the file?

If its textual rows, with the \n character, then you'll be wanting to
use fgets() to read one line at a time.

If its xml, then look at the xml parsing functions.

HTH,

Matt

Michael

unread,
Jan 2, 2009, 6:14:12 AM1/2/09
to nzp...@googlegroups.com
On Fri, 02 Jan 2009 11:01:20 Matthew White wrote:
> How are you using fread?
>
> Something like
>
> $contents = fread($handle, filesize($filename));

Yes, like that.

> Will read the whole thing into memory, which is not something you
> really want to do as previously noted

Well, the system does have 4Gb and I would upgrade it to 8Gb if necessary, but
even when I allow PHP to use 2.5Gb or even 3Gb of RAM, it still barfs.

> Can you tell us a little more about the nature of the file?

text file.

> If its textual rows, with the \n character, then you'll be wanting to
> use fgets() to read one line at a time.
>
> If its xml, then look at the xml parsing functions.

Even when I use fgets() I still get the same error message:

Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to
allocate 2023545829 bytes) in /www/scan.php on line 145

This file is XML though I don't want specific XML programming if I can avoid
it. It should work fine on XML files to retrieve the data, as it works on
haphazard non-XML files.

Reply all
Reply to author
Forward
0 new messages