how does one trap for out-of-memory errors?
lawrence
running Phorum. Occassionally a thread will have hundreds of entries,
perhaps a meg or two worth of data. You won't necessarily print all
that to the screen, but PHP has to hold it in memory. Run some
operations on it, or, more likely, have an array that you keep adding
things to, and very soon you run into the 8 meg limit that is the
default limit for PHP scripts.
How do you trap for out-of-memory errors? I want to do something like:
if (php is now using more than 8 megs) {
print "Sorry, but this operation requires more memory than PHP is
allowed.";
} else {
printOutEntryes($allEntries);
}
PHP has a command that checks the current memory usage of PHP, but
PHP, near as I know, has no command to estimate how much the next
operation is going to drive up memory usuage.
I'm sure this problem comes up for people using Phorum, or PostNuke,
or phpSlash, or phpMyAdmin or any of a lot of programs. How is it
normally handled?
Nikolai Chuvakhin
news:<da7e68e8.03101...@posting.google.com>...
>
> How do you trap for out-of-memory errors? I want to do something like:
>
> if (php is now using more than 8 megs) {
> print "Sorry, but this operation requires more memory than PHP is
> allowed.";
> } else {
> printOutEntryes($allEntries);
> }
Not going to work. PHP will display an out-of-memory error and stop
running BEFORE it gets to executing your if() statement.
> I'm sure this problem comes up for people using Phorum, or PostNuke,
> or phpSlash, or phpMyAdmin or any of a lot of programs. How is it
> normally handled?
By preventing it from happening. More specifically, by avoiding
sucking into memory things of unknown and potentially very large
size. Even more specifically, by writing proper database queries,
so that the resulting datasets can be processed and output strictly
one record at a time, without accumulating them in arrays on the
client side.
Cheers,
NC
Perttu Pulkkinen
"Nikolai Chuvakhin" <n...@iname.com> wrote in message
> Not going to work. PHP will display an out-of-memory error and stop
> running BEFORE it gets to executing your if() statement.
Are you also saying that memory errors are OUTSIDE of possibilities of php
error handling systems (notices, warnings, errors and fatal errors). Because
normally PHP errors CAN be handled and it is possible to execute code after
error. You just need to write error handlers and set them to use.
perttu pulkkinen, Finland
DvDmanDT
then see what happends... It's strange really, but all that happends is
slight slowness... There was some thread comparing PHP with ASP where
someone wanted to something with big XML files...
ASP+40mb XML file... Script ran for 60 mins or so unless my memory fails
me... Then it just said out of memory or something...
PHP + 100mb: 15 mins... He read entire file into memory, then did some
operations on it... PHP handled the file and the memory without problems...
--
// DvDmanDT
MSN: dvdm...@hotmail.com
Mail: dvdm...@telia.com
"lawrence" <lkru...@geocities.com> skrev i meddelandet
news:da7e68e8.03101...@posting.google.com...
DvDmanDT
happends):
<?php
/**** Memory killer 1.0 ****
* Will allocate some memory.
* Case you wonder: No, I
* don't have a life. I
* suggest using this on cgi
* version, PHP 4 or 5. Apache
* might crash otherwise.
*/
set_time_limit(0); // This WILL take some time, up to 30 mins, so disable
time limit
define("BYTES_TO_ALLOCATE",20*1024*1024); // 20 mb
ob_implicit_flush();
header("Content-Type: text/plain");
function pmem(){
global $str;
echo "Current lenght: ".strlen($str)."\r\n";
// Uncomment next line to show usage
#echo "Memory useage: ".(memory_get_usage()/1024)." kb\r\n";
}
register_tick_function('pmem');
declare(ticks=1024);
$str="";
srand(time());
for($i=0;$i<BYTES_TO_ALLOCATE;$i++)$str.=chr(rand(32,100)); // Add a random
char
?>
--
// DvDmanDT
MSN: dvdm...@hotmail.com
Mail: dvdm...@telia.com
"DvDmanDT" <dvdm...@telia.com> skrev i meddelandet
news:u7Dkb.30206$mU6....@newsb.telia.net...
Nikolai Chuvakhin
news:<1yrkb.51$BY1...@read3.inet.fi>...
> > BEFORE it gets to executing your if() statement.
>
> Are you also saying that memory errors are OUTSIDE of possibilities
> of php error handling systems (notices, warnings, errors and fatal
> errors).
No. All I'm saying is that what the original poster wanted cannot
be done the way he wants it. In particular, if my understanding
is correct, a memory allocation problem causes a fatal error
(E_ERROR), but until the error is handled (possibly, by a user-
defined error handling function), it is impossible to know whether
the error is caused by memory allocation problem or something else.
Cheers,
NC
lawrence
Yes, thank you. You understood my intention. I was asking about
trapping for errors.
If you can open a 100mb file and process it and then write it to disk,
then what does the 8 mb limit refer to?
lawrence
> > I'm sure this problem comes up for people using Phorum, or PostNuke,
> > or phpSlash, or phpMyAdmin or any of a lot of programs. How is it
> > normally handled?
>
> By preventing it from happening. More specifically, by avoiding
> sucking into memory things of unknown and potentially very large
> size. Even more specifically, by writing proper database queries,
> so that the resulting datasets can be processed and output strictly
> one record at a time, without accumulating them in arrays on the
> client side.
The right kind of queries? So something like this is bad:
SELECT * FROM content WHERE type = 'weblogEntries'
I get this back and then try to read it into an array. If there are
enough entries, then I run into the 8 meg limit. I take it that your
approach would avoid putting the whole return into an array?
Nikolai Chuvakhin
>
> > > How is it normally handled?
> >
> > By preventing it from happening. More specifically, by avoiding
> > sucking into memory things of unknown and potentially very large
> > size. Even more specifically, by writing proper database queries,
> > so that the resulting datasets can be processed and output strictly
> > one record at a time, without accumulating them in arrays on the
> > client side.
>
> The right kind of queries? So something like this is bad:
>
> SELECT * FROM content WHERE type = 'weblogEntries'
No (except for the fact that `type` is a text, and thus likely
unindexed, field), but your handling of it is definitely bad:
> I get this back and then try to read it into an array.
Why would you want to do this? With very few exceptions, any data
manipulation you are about to do with PHP is done faster and with
less overhead by MySQL. And if you are not going to manipulate
the data, why bother reading it into a huge array?
If you explain what exactly you are trying to achieve, I'll gladly
help you find a better way of getting it done.
> I take it that your approach would avoid putting the whole
> return into an array?
Of course. The usual
while ($record = mysql_fetch_row ($result)) {}
only keeps in memory one record at a time. Remember, $result
is a resource (sort of like a file pointer), so it has the same
memory footprint regardless of the number of records in the
returned dataset.
Cheers,
NC
Zurab Davitiani
> Of course. The usual
>
> while ($record = mysql_fetch_row ($result)) {}
>
> only keeps in memory one record at a time.
I'm not so sure about this. My understanding is that
$my_var = "something";
$my_var = "something else";
does not free the original memory occupied by "something", it just allocates
new memory for "something else". All memory is only freed when script
execution or PHP process ends. I agree though, maybe some garbage
collection would be great in PHP.
--
Business Web Solutions
ActiveLink, LLC
www.active-link.com/intranet/
Nikolai Chuvakhin
news:<fW4mb.3044$dv3....@newssvr14.news.prodigy.com>...
>
> Nikolai Chuvakhin wrote on Friday 24 October 2003 00:25:
>
> > The usual
> > while ($record = mysql_fetch_row ($result)) {}
> > only keeps in memory one record at a time.
>
> I'm not so sure about this. My understanding is that
>
> $my_var = "something";
> $my_var = "something else";
>
> does not free the original memory occupied by "something",
> it just allocates new memory for "something else". All memory
> is only freed when script execution or PHP process ends.
I honestly doubt it. If you were correct, one of my largest
applications couldn't possibly work, because one of its
functionalities is like this:
1. Retrieve about 10 fields per record from the entire
table (currently, ~200,000 records).
2. Based on those 10 fields and some extra inputs, compute
about 70 more fields for each record.
3. Write the results back into the table.
In other words, the
while ($record = mysql_fetch_row ($result)) {}
cycle was repeated 200,000 times. Given that $record was an
array including a date and a collection of floating-point
numbers, I'd say its memory requirement was about 100 bytes.
This would mean that the script would trash about 20 megabytes
of memory only when handling this one variable. But what about
the 70 values that were computed for each of the 200,000 records?
They (being mostly integers, with a few floats) would trash
at least another 20 megabytes. The UPDATE query string formed
for writing the output into the databse was about 100 characters
long, so it, too, would trash 20 megabytes over 200,000
reassignments. Yet the whole thing never ever hit the default
memory limit...
I am not saying you're wrong, but if you're right, I'd like to
understand how it is possible for you to be right while I see
what I see. No pun intended; I really would like to learn
something here...
Cheers,
NC
lawrence
>
> Why would you want to do this? With very few exceptions, any data
> manipulation you are about to do with PHP is done faster and with
> less overhead by MySQL. And if you are not going to manipulate
> the data, why bother reading it into a huge array?
>
> If you explain what exactly you are trying to achieve, I'll gladly
> help you find a better way of getting it done.
>
> > I take it that your approach would avoid putting the whole
> > return into an array?
>
> Of course. The usual
>
> while ($record = mysql_fetch_row ($result)) {}
>
> only keeps in memory one record at a time. Remember, $result
> is a resource (sort of like a file pointer), so it has the same
> memory footprint regardless of the number of records in the
> returned dataset.
I am putting database returns in PHP arrays to mask the client code
from the underlying database. That is, I don't want my code to know
what kind of database or datastore is being dealt with.
I am storing all actual SQL in objects that I am calling queryObjects.
But the only place the query objects are called is in what I call Get,
Update, Delete, and Insert objects. The Get, Update, Delete, and
Insert objects are meant to mask, from the rest of the code, what kind
of database or datastore is in use. In other words, if I want to get
all weblog entries out of a database, I might have a method called
getAllWeblogEntries, and this function would contain actual SQL, and
return a PHP array. So I might have a query object for MySql and a
query object for PostGre and a query object for XML streams. Each of
these query objects would have a method called getAllWeblogEntries,
and each would return a PHP array.
The Get object knows what kind of database or datastore the website
uses (this is something set in the config file for that website). All
the other code simply makes a call to the Get object. Thus, the other
code, I mean the client code, never needs to know what kind of
database or datastore is in use.
I can't imagine how I can hide the database from the client code if I
use database return resources, as you suggest above. But I now realize
how limiting these arrays can be, in the presence of PHP's 8 meg
limit. So I'm open to any graceful ideas for otherwise masking the
database from the client code.
Nikolai Chuvakhin
in message <da7e68e8.03102...@posting.google.com>:
>
> I am putting database returns in PHP arrays to mask the client
> code from the underlying database. That is, I don't want my code
> to know what kind of database or datastore is being dealt with.
Haven't you heard that abstraction doesn't work? :) I am not
trying to start a flame war here, but you need to understand that
higher level of abstraction ALWAYS implies a performance penalty.
In your case, you are shooting for capital abstraction and what
you get for it is the performance equivalent of capital
punishment.
> I am storing all actual SQL in objects that I am calling
> queryObjects. But the only place the query objects are called
> is in what I call Get, Update, Delete, and Insert objects. The
> Get, Update, Delete, and Insert objects are meant to mask, from
> the rest of the code, what kind of database or datastore is in
> use. In other words, if I want to get all weblog entries out of
> a database, I might have a method called getAllWeblogEntries,
> and this function would contain actual SQL, and return a PHP
> array. So I might have a query object for MySql and a query
> object for PostGre and a query object for XML streams. Each of
> these query objects would have a method called
> getAllWeblogEntries, and each would return a PHP array.
And that's the root of your problem. You want to return something
of potentially very large size, and you are hell-bent on storing
it in memory. But let me ask you: what will the application do
with a PHP array that contains all entries in the weblog? Output
them? Then the entries don't have to be in memory at all, they
can be output straight from the DB buffer. Sort them? Again,
they don't have to be in the memory for that, you should have
taken care of it on the database level. Search through them?
Same thing; too late to do it efficiently, had to be done on the
database level.
So my suggestion to you is not to have a getAllWeblogEntries
method at all. Rather, you should have a getWeblogEntry method
that returns a single weblog entry in an array and an entriesLeft
method that returns a boolean value. This way, you can retrieve
all entries one by one:
$qo = new queryObjectMySQL ('SELECT * FROM entries');
// the constructor establishes the connection, retrieves data,
// and keeps the resulting resource for the object's internal
// use...
while ($qo->entriesLeft ()) {
$entry = $qo->getWeblogEntry ();
// now, do what you want with the $entry...
}
> I can't imagine how I can hide the database from the client code
> if I use database return resources, as you suggest above.
You make it sound like it's a good thing... :)
Cheers,
NC
lawrence
> lkru...@geocities.com (lawrence) wrote
> in message <da7e68e8.03102...@posting.google.com>:
> >
> > I am putting database returns in PHP arrays to mask the client
> > code from the underlying database. That is, I don't want my code
> > to know what kind of database or datastore is being dealt with.
>
> Haven't you heard that abstraction doesn't work? :) I am not
> trying to start a flame war here, but you need to understand that
> higher level of abstraction ALWAYS implies a performance penalty.
> In your case, you are shooting for capital abstraction and what
> you get for it is the performance equivalent of capital
> punishment.
This runs counter to most of what I've been taught. I'm not sure what
you mean when you say abstraction doesn't work. Much software work
involves abstraction. You could argue that PHP itself is an
abstraction, hiding the complexity of the underlying C code. Some
might say that anything other than machine code is an abstraction. I
can't think of a software project that doesn't involve abstraction.
As for OO design, to hide the datastore seems like an obvious move.
Hasn't the team working on PEAR done the same?
>
> > I am storing all actual SQL in objects that I am calling
> > queryObjects. But the only place the query objects are called
> > is in what I call Get, Update, Delete, and Insert objects. The
> > Get, Update, Delete, and Insert objects are meant to mask, from
> > the rest of the code, what kind of database or datastore is in
> > use. In other words, if I want to get all weblog entries out of
> > a database, I might have a method called getAllWeblogEntries,
> > and this function would contain actual SQL, and return a PHP
> > array. So I might have a query object for MySql and a query
> > object for PostGre and a query object for XML streams. Each of
> > these query objects would have a method called
> > getAllWeblogEntries, and each would return a PHP array.
>
> And that's the root of your problem. You want to return something
> of potentially very large size, and you are hell-bent on storing
> it in memory.
No, I am not hell bent. I asked for alternatives.
> But let me ask you: what will the application do
> with a PHP array that contains all entries in the weblog? Output
> them? Then the entries don't have to be in memory at all, they
> can be output straight from the DB buffer. Sort them? Again,
> they don't have to be in the memory for that, you should have
> taken care of it on the database level. Search through them?
> Same thing; too late to do it efficiently, had to be done on the
> database level.
>
> So my suggestion to you is not to have a getAllWeblogEntries
> method at all. Rather, you should have a getWeblogEntry method
> that returns a single weblog entry in an array and an entriesLeft
> method that returns a boolean value. This way, you can retrieve
> all entries one by one:
>
> $qo = new queryObjectMySQL ('SELECT * FROM entries');
> // the constructor establishes the connection, retrieves data,
> // and keeps the resulting resource for the object's internal
> // use...
> while ($qo->entriesLeft ()) {
> $entry = $qo->getWeblogEntry ();
> // now, do what you want with the $entry...
> }
I think your suggestion is a good one. Somehow I must get just one
entry at a time. The query object also needs to remember which entry
it got last, so it can get the next one. The SELECT object would have
to wrap that interger and pass it along to the final code, especially
if the final code has a loop that needs to know how long it is
supposed to keep running.
Off hand, it sounds terribly complicated, and the whole of the code
would have to be rewritten. Every loop would have to be rewritten. But
it would eventually get the code free of working with dangerously
large arrays. It is a project that, I think, will take a few months,
but I think the direction you suggest is a good one.
> > I can't imagine how I can hide the database from the client code
> > if I use database return resources, as you suggest above.
>
> You make it sound like it's a good thing... :)
Well, this code is being written first to work with MySql, but I'm
supposed to write it in such a way that would make it easy to switch
to PostGre. So I've written it the way I have to save myself some
future work. And saving oneself future work is the aim of most OO
abstraction.
Nikolai Chuvakhin
> n...@iname.com (Nikolai Chuvakhin) wrote in message
> news:<32d7a63c.03102...@posting.google.com>...
>
> > you get for it is the performance equivalent of capital
> > punishment.
>
> This runs counter to most of what I've been taught. I'm not sure
> what you mean when you say abstraction doesn't work.
My apologies for vague language. I should have said "database
abstraction layers" rather than just "abstraction".
> As for OO design, to hide the datastore seems like an obvious move.
Which is exactly the reason I generally object to using OO design
in the first place.
> Hasn't the team working on PEAR done the same?
Yes, they have. And, according to phplens, this resulted in 150-170%
increase in time it takes to complete a series of SELECT queries:
http://phplens.com/lens/adodb/
> I am not hell bent. I asked for alternatives.
I think looking for an altermative in your situation was the right
thing to do. Note also that you didn't have a compelling reason
to do it "your" way in the first place; at least, my question to
that extent:
> > But let me ask you: what will the application do
> > with a PHP array that contains all entries in the weblog? Output
> > them? Then the entries don't have to be in memory at all, they
> > can be output straight from the DB buffer. Sort them? Again,
> > they don't have to be in the memory for that, you should have
> > taken care of it on the database level. Search through them?
> > Same thing; too late to do it efficiently, had to be done on the
> > database level.
went unanswered...
> Somehow I must get just one entry at a time.
I'd say this is the preferred way of dealing with the problem.
> The query object also needs to remember which entry it got last,
> so it can get the next one.
It already does, since it has a resource with an internal pointer
to the next available entry...
> Well, this code is being written first to work with MySql, but I'm
> supposed to write it in such a way that would make it easy to switch
> to PostGre.
This is yet another reason why database abstraction layers are
such a bad idea... PostgreSQL has a whole slew of features
that are not available in MySQL: views, rules, triggers, stored
procedures, etc. Database abstraction forces you to stick to
the lowest common denominator, which robs you of opportunities
to use the more advanced, higher-performance features...
Cheers,
NC
lawrence
>
> I think looking for an altermative in your situation was the right
> thing to do. Note also that you didn't have a compelling reason
> to do it "your" way in the first place; at least, my question to
> that extent:
>
> > > But let me ask you: what will the application do
> > > with a PHP array that contains all entries in the weblog? Output
> > > them? Then the entries don't have to be in memory at all, they
> > > can be output straight from the DB buffer. Sort them? Again,
> > > they don't have to be in the memory for that, you should have
> > > taken care of it on the database level. Search through them?
> > > Same thing; too late to do it efficiently, had to be done on the
> > > database level.
>
> went unanswered...
I'm not sure what you mean when you say I didn't answer your question.
I explained that I was trying to hide the database from the client
code. Putting the database return into a PHP array had been my answer
to that. You can, and have, argued that it is the wrong answer, but I
certainly answered your question about what it was that I was trying
to accomplish.
> > Well, this code is being written first to work with MySql, but I'm
> > supposed to write it in such a way that would make it easy to switch
> > to PostGre.
>
> This is yet another reason why database abstraction layers are
> such a bad idea... PostgreSQL has a whole slew of features
> that are not available in MySQL: views, rules, triggers, stored
> procedures, etc. Database abstraction forces you to stick to
> the lowest common denominator, which robs you of opportunities
> to use the more advanced, higher-performance features...
Engineering of any kind is often about goals and tradeoffs.
Programmers would, in general, be delighted if, by magic, there was
some way to both take advantage of all the features of any database
and yet also easily port software from one database to another.
However, there is no easy way to achieve both these goals, so one
generally has to choose one. Either software can take advantage of the
special features of one piece of software, or it can be easily ported
from one database to another.
In our case, portability trumps advanced feature use. For another
project the opposite decision might be a wise one.
lawrence
> > so it can get the next one.
>
> It already does, since it has a resource with an internal pointer
> to the next available entry...
It strikes me as breathtakingly complicated. If I want my code to do
one thing if the database has returned less than 50 rows, but
something else if the database has returned 50 or more rows, then a
request to count the number of rows returned needs to be sent down the
line, through the selectObject to the queryObject, and then returned
back up the line. And both objects need a getNextRow method, wrapping
(for MySQL) mysql_fetch_row. Or, if I want sometimes use
mysql_fetch_object, or any of the other commands for getting rows,
then I'd have to write a separate method for each of those commands.
If I want the API of every queryObject to conform to the contract of a
standard interface, then I need to write these methods with that in
mind.
This may be the best way of doing things, I can't think of anything
better, but it strikes me as a large project, especially as I have
about 100 SQL statements for MySql alone.
Zurab Davitiani
> I am not saying you're wrong, but if you're right, I'd like to
> understand how it is possible for you to be right while I see
> what I see. No pun intended; I really would like to learn
> something here...
This is OT from the original discussion. Sorry for taking it in this
direction, but I'd like to clear up my original claim.
You are right. The issue doesn't affect primitive variables and arrays
thereof. It only affects objects. Objects don't seem to be destroyed (and
memory freed) until the script execution ends. e.g., if you have:
for($i=0; $i<10000; $i++) {
$myObj = new LogParser("SomeBigLogFile.log");
}
$myObj objects are not destroyed (to free the memory) even though the
variable with the same name is being reassigned. So, total memory
consumption goes up with each pass. And unset() doesn't help either in this
case.
But, if you have:
for($i=0; $i<10000; $i++) {
$contents = file_get_contents("SomeBigLogFile.log");
}
the memory consumption will stay the same.
By going over some comments in previous posts on this topic, it seems like
PHP5 does have a better memory management with regard to objects. I will
test it when I have time, or if anyone has verified, feel free to post.
Alan Little
lawrence of comp.lang.php make plain:
> I can't imagine how I can hide the database from the client code if I
> use database return resources, as you suggest above. But I now realize
> how limiting these arrays can be, in the presence of PHP's 8 meg
> limit. So I'm open to any graceful ideas for otherwise masking the
> database from the client code.
Unless I'm misunderstanding, it seems like you're overlooking an obvious
solution. You have methods for transparently managing the queries, why
not have one for transparently retrieving the data from the resource
pointer? I have something like this in one of my applications, which
needs to be able to work with MySQL or Sybase. I simply wrote db_
functions to replace all the mysql_ and sybase_ functions, and made them
intelligent enough to handle the differences transparently.
--
Alan Little
Phorm PHP Form Processor
http://www.phorm.com/
Nikolai Chuvakhin
> Nikolai Chuvakhin wrote on Friday 24 October 2003 15:08:
>
> > I am not saying you're wrong, but if you're right, I'd like to
> > understand how it is possible for you to be right while I see
> > what I see. No pun intended; I really would like to learn
> > something here...
>
> This is OT from the original discussion.
I don't think so; the original discussion was about preventing
out-of-memory errors. The issue of how memory is allocated is,
I believe, relevant to that discussion.
> You are right. The issue doesn't affect primitive variables and arrays
> thereof. It only affects objects. Objects don't seem to be destroyed (and
> memory freed) until the script execution ends.
Thank you, this indeed clears things up.
Cheers,
NC
lawrence
Yes, exactly. Sorry if I wasn't more clear, but that was what I did in
the end. I was a little dense, and did not at first see that as a
solution, but then in the end that is what I did.
lawrence
I'm very glad you wrote this. I was just about to start a new thread,
with the title "Word of warning: garbage collection with objects not
efficient", but now that you've written this post I guess there is no
need. This is a lesson that I have learned empircally, through a
painful process. I never got out-of-memory messages until I started
using objects, but they've become more frequent as I've used more
objects. The most startling, and stark, example, came last week when I
took a function and moved it, unchanged, into an object. Suddenly,
many things in my code that had worked now got out-of-memory errors
with that class method. And mind you, I made no change to the function
except move into a class. As a function in procedural code, it brought
no memory errors, but as a class method it became a fatal bottle-neck
for the script.
So there is something to what you say. PHP's garbage collection is
much less automatic when it deals with objects. Anyone considering an
OO approach with PHP should be warned of this.
It also brings up the issue of how, exactly, to empty class variables
in PHP. Is it enough to do this:
$myVar = null;
??????
I have a class array which seems to make increasing demands on memory
with every use, even though I've started to do this at the end of each
use:
$dataTable["password"] = "";
$dataTable["username"] = "";
$dataTable["perPagePassword"] = "";
lawrence
> $myObj objects are not destroyed (to free the memory) even though the
>variable with the same name is being reassigned. So, total memory
>consumption goes up with each pass. And unset() doesn't help either
in this
>case.
This makes me nervous. If unset() does not work, what does? "null" I
hope?
I have empirically noticed what you describe. With object, in a for()
loop, memory consumption goes up with every pass. How is one to deal
with this?
Zurab Davitiani
> So there is something to what you say. PHP's garbage collection is
> much less automatic when it deals with objects. Anyone considering an
> OO approach with PHP should be warned of this.
I agree. Not only is it not automatic, but there's no way to even manually
free up the memory used by objects. Although I haven't had time to install
and test PHP5 in this regard, it'd be interesting to know if this is
implemented better.
> It also brings up the issue of how, exactly, to empty class variables
> in PHP. Is it enough to do this:
>
> $myVar = null;
>
>
> ??????
>
> I have a class array which seems to make increasing demands on memory
> with every use, even though I've started to do this at the end of each
> use:
>
> $dataTable["password"] = "";
> $dataTable["username"] = "";
> $dataTable["perPagePassword"] = "";
I am not sure whether it will make any difference at all. i.e., I am not
sure the original memory used by object property will be freed. If you do
the testing on this, please post the results.
Zurab Davitiani
Again, as I said before, there's no way to "deal" with this in PHP4 - no way
that I found anyway. Assigning to "", null, using unset(), etc. does not
free the memory.
I came accross this issue when I tried to run a long database update script
(from shell) with a lot of data using objects. I would get memory
allocation error at certain point. The way I resolved it (and still used
objects) was to launch a separate PHP process to query and update the data
for each record; so, all object references would get created inside that
process and die when that process was finished and memory would be freed
up. Now, this is not an option for most web apps, but it may work on longer
cron jobs when using objects is a necessity.
Nikolai Chuvakhin
>
> With object, in a for() loop, memory consumption goes up
> with every pass. How is one to deal with this?
Just say no to OOP... PHP (or any other development tool, for that
matter) can't be all things to all people. My understanding is that
PHP was conceived as a lightweight high-performance highly portable
scripting environment. Hence, no dependence on system objects and
OOP added as an afterthought, an option for the developer to pay for
code reusability with loss of performance. What I personally find
very strange is the number of people willing to make the trade...
Sure, OOP sounds sophisticated, but the simple truth is, PHP was not
concieved sophisticated (in fact, I find it humorous that the PHP
team chose to replace Rasmus Lerdorf's initial meaning of PHP --
"Professional Home Page" -- with their recursive "PHP: Hypertext
Preprocessor" routine). So I sincerely believe that if OOP is what
you like to do, your time is better spent working with ASP or JSP,
both of which require you to subscribe to the OOP methodology.
None of the above is meant to be a personal criticism of you; rather,
it's a general statement of (not necessarily unbiased) opinion on
application performance issues...
Cheers,
NC
lawrence
> > loop, memory consumption goes up with every pass. How is one to deal
> > with this?
>
> Again, as I said before, there's no way to "deal" with this in PHP4 - no way
> that I found anyway. Assigning to "", null, using unset(), etc. does not
> free the memory.
>
> I came accross this issue when I tried to run a long database update script
> (from shell) with a lot of data using objects. I would get memory
> allocation error at certain point. The way I resolved it (and still used
> objects) was to launch a separate PHP process to query and update the data
> for each record; so, all object references would get created inside that
> process and die when that process was finished and memory would be freed
> up. Now, this is not an option for most web apps, but it may work on longer
> cron jobs when using objects is a necessity.
Basically, OOP in PHP is broken. I am a bit stunned by this. If you
can't manually free the memory and the memory does not clear
automatically, then I think it is fair to descibe this feature as
"broken". For most applications, there is nothing the programmer can
do, except not do the application. There is no way for the programmer
to protect themself against this bug, no matter how talented they are,
and no matter how diligent.
I am very disappointed. As someone who has used PHP for most of their
work for the last 3 years, it is sad to learn that PHP has such a
major flaw. I thought it had matured and was now a robust,
sophisticated language for major web-development, but clearly, it is
still showing its roots as a scripting environment for "Personal Home
Pages."
People should be warned away from OOP in PHP. It should be advertised
as a strictly procedural language. Any large database return is likely
to give memory errors in OOP in PHP. People should be warned.
I'm surprised that I have not heard of this before. This should be a
fairly big deal.
lawrence
> solution. You have methods for transparently managing the queries, why
> not have one for transparently retrieving the data from the resource
> pointer? I have something like this in one of my applications, which
> needs to be able to work with MySQL or Sybase. I simply wrote db_
> functions to replace all the mysql_ and sybase_ functions, and made them
> intelligent enough to handle the differences transparently.
Now it seems that even this won't work. Getting one entry at a time or
getting the whole array at once uses the same memory, because the
memory garbage collection in objects in PHP seems to be broken.
Nikolai Chuvakhin
>
> Basically, OOP in PHP is broken. I am a bit stunned by this.
You shouldn't be. You have a language that has no typing to
speak of (Niklaus Wirth has long since given up screaming about
how little sense OOP makes without strong typing), which, unlike
VBscript, runs cross-platform, but, unlike JSP, does not require
a virtual machine. All interfaces to hardware and lower-level
software (such as databases) are implemented procedurally; in
other words, there are no system objects. Why do you expect OOP
to be anything but a pain in the neck in this framework?
> I thought it had matured and was now a robust, sophisticated
> language for major web-development
Robust, yes. Sophisticated, no. Think of PHP as the assembly
language of Web development, and it will immediately become clear
to you that sophistication and robustness are conflicting notions.
Robust systems have a small footprint and thus are necessarily
feature-poor. Sophisticated systems are feature-rich and thus
must have a bigger footprint.
> People should be warned away from OOP in PHP.
Amen! But then, people should be generally warned away from OOP...
Cheers,
NC
Zurab Davitiani
>> People should be warned away from OOP in PHP.
>
> Amen! But then, people should be generally warned away from OOP...
I respectfully disagree. There are times when you do need objects in your
apps. An existing example of this is DOM. Also, a lot of PHP OOP
problems/issues are being taken care of in PHP5, with major enhancements
and additions to OOP and related functionality. PHP is evolving, getting
new features, and fixing existing flaws, and resisting the use of the
available features will sometimes result in not taking the full advantage
of what they have to offer.
This does not mean that your PHP scripts have to look like Java code, but
use of objects in PHP are many times convenient and useful. As far as the
flaw pointed out in this (sub)thread is a PHP issue that will hopefully be
addressed in upcoming releases so that developers can take full advantage
of OO features that they expect from the environment to provide.
lawrence
It seems to me that I foolishly commited myself to OOP in PHP without
realizing how flawed PHP's implementation of OOP is. I've now put in
several months of work building a solid OOP framework for my content
management system. I'm frustrated to find myself in the position that
I now find myself. Going backwards to procedural code would also take
a lot of work, and would have the disappointing feel of moving
backwards. But moving forward basically means waiting for PHP 5.0 and
crossing my fingers and hoping really hard that PHP 5.0 is as good as
the hype promises. I do not like to be in this position.
I suppose as a short term hack I can go into the php.ini file and
increase the maximum memory allowed to a PHP script, from 8 megs to
something more. (I can test and see at what level I stop getting any
errors). But what an awful demanding script it becomes. With each
script taking 10 megs, then on a RH Linux machine with 256 RAM,
Apache, Mysql, and PHP, I'd probably be able to support less than 40
simultaneous visits. Which won't be an immediate problem, but it might
become a huge one in a few months. Of course, easy enough to add in a
gig of RAM, but I hate writing bad code and then trying to recover
from my error by getting a more powerful machine.
Nikolai Chuvakhin
> Nikolai Chuvakhin wrote on Tuesday 04 November 2003 23:39:
> >
> > people should be generally warned away from OOP...
>
> I respectfully disagree. There are times when you do need objects
> in your apps. An existing example of this is DOM.
You are not disagreeing at all. You are saying that there are times
when OOP is the preferred solution, and your example highlights this
point very well. However, it has been my experience that simply
assuming that OOP has to be used without considering the procedural
approach as an alternative is usually a mistake. This is especially
true in PHP, as Lawrence just realized after spending a few months
on a project that is now going to be killed or put on hold pending
the resolution of issues he discovered.
More generally, there is a great number of things that simply run
better when not polluted by OOP, regardless of language (unless
it's Java, of course). Advanced numerical algorithms seem to be
one area where this is especially true.
> resisting the use of the available features will sometimes result
> in not taking the full advantage of what they have to offer.
I agree. In a similar vein, using some of the features will sometimes
result in a major performance drag.
> This does not mean that your PHP scripts have to look like Java code
In PHP 5 (or maybe 6), it is likely to mean exactly that:
Zend is working with Sun Microsystems to develop a standard method
of integration interface. The result will give PHP programmers
the ability to treat Java objects just like PHP objects. And without
worrying about JVM resource wasting, or system stability.
http://www.zend.com/php_and_java.php
Cheers,
NC