[PHP-DEV] Re: CLI in PHP6
Peter Brodersen
("Ron Korving") wrote:
>There was once (can't remember when exactly, so it must be a long time =
ago)
>here on PHP CLI scripts in which it came forward that one should not =
rely on
>such a script to run forever. And it's true; the scripts sometimes =
magically
>and suddenly die. Now I have no clue where this instability (for lack of=
a
>better word) comes from, but could it be possible for this to be =
resolved
>for PHP6 so that PHP becomes an extremely viable solution for CLI daemon
>scripts?
Jani mentioned in http://bugs.php.net/bug.php?id=3D34483 : "Running a
PHP for 24 hours (under windows) is REALLY not supported or suggested.
It's definately nothing to do with PHP but your OS." (and "Try this on
a real OS, like Linux")
Maybe this was what you were thinking of.
I can't see any reason for that statement, though.
--=20
- Peter Brodersen
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
James Aylett
> Jani mentioned in http://bugs.php.net/bug.php?id=34483 : "Running a
> PHP for 24 hours (under windows) is REALLY not supported or suggested.
> It's definately nothing to do with PHP but your OS." (and "Try this on
> a real OS, like Linux")
>
> Maybe this was what you were thinking of.
>
> I can't see any reason for that statement, though.
For early consumers versions of Windows (95, 98, ME) I wouldn't
recommend running a process for that long if you rely on it. For
NT-based versions, including consumer editions, there should be no
problems in running scripts for as long as you like.
This particular bug report was against Windows 2000 Server, which
should have no problems. I would heavily dispute the statement that
it's definitely the OS and not PHP - there are plenty of processes
that run happily for months on W2K Server.
James
--
james aylett, chief technical architect
tangozebra
020 7535 9814
________________________________________________________________________
This e-mail has been scanned for all viruses by Star. The
service is powered by MessageLabs. For more information on a proactive
anti-virus service working around the clock, around the globe, visit:
http://www.star.net.uk
________________________________________________________________________
Ron Korving
reason, and honestly I'm a little bit puzzled on how to debug the situation.
Any suggestions?
Ron
"James Aylett" <james....@tangozebra.com> schreef in bericht
news:20051006105...@tangozebra.com...
J. Allen Dove
for the last 18-20 months with great success. That is not to say it
wasn't a challenge to get them as "stable" as a C++ daemon but we have.
PHP 4.x CLI seemed to have a mem leak somewhere, PHP 5.x seems to have
resolved that. However, a process running for a long time (yes, even on
Linux) will also just "die" for no reason. We have a few hints as to
what causes the deaths but don't know what causes the cause. To solve
the problem we created a daemon class that has a lifetime and can make
itself "eternal" if desired. The net effect is a stable daemon(s).=20
We currently process 65-70MM transactions a day with our PHP-based
daemons. We chose to try using PHP to share business logic and it paid
off.=20
-- Allen
-----Original Message-----
From: Ron Korving [mailto:r.ko...@xit.nl]=20
Sent: Thursday, October 06, 2005 2:49 AM
To: inte...@lists.php.net
Subject: [PHP-DEV] CLI in PHP6
Hi,
There was once (can't remember when exactly, so it must be a long time
ago)
here on PHP CLI scripts in which it came forward that one should not
rely on
such a script to run forever. And it's true; the scripts sometimes
magically
and suddenly die. Now I have no clue where this instability (for lack of
a
better word) comes from, but could it be possible for this to be
resolved
for PHP6 so that PHP becomes an extremely viable solution for CLI daemon
scripts?
Thanks,
Ron Korving
--=20
Ron Korving
A lot :) Thanks for the response. It's good to see it is possible.
> However, a process running for a long time (yes, even on
> Linux) will also just "die" for no reason. We have a few hints as to
> what causes the deaths but don't know what causes the cause. To solve
> the problem we created a daemon class that has a lifetime and can make
> itself "eternal" if desired. The net effect is a stable daemon(s).
Could you get into a little more detail? I don't quite understand what you
mean by this daemon class that can make itself "eternal" and how this can
turn the situation stable.
Thanks,
Ron
""J. Allen Dove"" <ad...@booyahnetworks.com> schreef in bericht
news:4E0C5C8E5F8C994F9013...@pearl.hq.booyahnetworks.com...
FWIW, we have been running a number of very high-volume PHP CLI daemons
for the last 18-20 months with great success. That is not to say it
wasn't a challenge to get them as "stable" as a C++ daemon but we have.
PHP 4.x CLI seemed to have a mem leak somewhere, PHP 5.x seems to have
resolved that. However, a process running for a long time (yes, even on
Linux) will also just "die" for no reason. We have a few hints as to
what causes the deaths but don't know what causes the cause. To solve
the problem we created a daemon class that has a lifetime and can make
itself "eternal" if desired. The net effect is a stable daemon(s).
We currently process 65-70MM transactions a day with our PHP-based
daemons. We chose to try using PHP to share business logic and it paid
off.
-- Allen
-----Original Message-----
From: Ron Korving [mailto:r.ko...@xit.nl]
Sent: Thursday, October 06, 2005 2:49 AM
To: inte...@lists.php.net
Subject: [PHP-DEV] CLI in PHP6
Hi,
There was once (can't remember when exactly, so it must be a long time
ago)
here on PHP CLI scripts in which it came forward that one should not
rely on
such a script to run forever. And it's true; the scripts sometimes
magically
and suddenly die. Now I have no clue where this instability (for lack of
a
better word) comes from, but could it be possible for this to be
resolved
for PHP6 so that PHP becomes an extremely viable solution for CLI daemon
scripts?
Thanks,
Ron Korving
--
J. Allen Dove
message had to do with the process running out of the "special" unix
file handles. The processes open and close lots of files (and yes, we
are explicitly closing files) and it seems for no rhyme or reason after
the processes run for a while the handles never seemed to really get
freed and would stack up and house the daemons. We could never
replicated in a lab, it would just happen from time to time and the
errors were things we could not explicitly control from PHP. (Note that
we found all this in the 4.x CLI and have not re-tested in the PHP 5.x
CLI. Our solution works great so why change it? Plus, if there are any
strange behaviors in the CLI they won't get exploited by dying and
re-spawning.)
So, what I mean re: "eternal" is that we can give our base daemon class
an optional "lifetime" count (e.g., the number of times the daemon
should perform its core processing before suicide) and an optional
"eternal" command. Since we have a common way to spawn our daemons this
eternal command is usually just the current script name plus some
specific command-line parameters. Once the daemon reaches its lifetime,
it requests its death, optionally re-spawning itself if there is an
eternal command given. This command can be kicked off via your method of
choice, e.g. pcntl_exec, exec, etc.=20
No real magic here, just a simple method to insure we can have clean
process spaces. :-)
-- AD
Thanks,
Ron
-- Allen
Hi,
Thanks,
Ron Korving
--=20
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
--=20
Sara Golemon
> ago)
> here on PHP CLI scripts in which it came forward that one should not rely
> on
> such a script to run forever. And it's true; the scripts sometimes
> magically
> and suddenly die. Now I have no clue where this instability (for lack of a
> better word) comes from, but could it be possible for this to be resolved
> for PHP6 so that PHP becomes an extremely viable solution for CLI daemon
> scripts?
>
it tracks because it has no idea what you plan on doing with that data. In
*theory* allocated resources get cleaned up within a request the minute they
no longer have any possible relevance (i.e. A database connection is created
within a function, then that function leaves making the variable it was
placed in no longer accessible from anywhere).
In 99% of the engine/core/exts this happens seemlessly, however there *may*
be (and most likely is) a lingering leak somewhere because a file descriptor
wasn't closed, or a couple bytes of memory wern't freed. In a "normal"
webserver environment these resources get forcibly cleaned up by the
end-of-request garbage collection process so they don't get noticed by the
bulk of PHP users out there. In a long-running daemon process this GC phase
never triggers and those piccune leaks pile up.
Things you can do to help locate these issues:
#1) Compile PHP with --enable-debug and run a version of your daemon
designed to kill itself off after a short period (how long may vary: a
minute, an hour, a day, a week.... try different periods). At the end of
execution, Zend will report what memory was leaked by the request and where
it was allocated. Though often the allocation was done by the engine but
the free was the responsibility of an ext, so this is only partially useful.
#2) Do the same as #1, but run the process through valgrind enabling
reporting of memory leaks and unclosed file descriptors and other debug
data. This is much more verbose output, but it helps track down the origin
of such leaks more precisely and efficiently.
#3) Report back what you find. The best way to do this is to post (A) Your
code, (B) Output from #1, and (C) Ouput from #2 on a webserver somewhere and
provide links to those pages in a message to php.internals. At that point,
someone here will either:
(I) identify the problem and just fix it,
(II) Ask for more information,
(III) Provide some insight but ask that you reformat the problem into a bug
report,
(IV) Shrug and say "I dunno", and/or ignore it
-Sara
J. Allen Dove
resources in PHP, especially if you do things in a lower scope that do
not get passed back/referenced to a higher scope. Since the concept of
scope gets a little gray in PHP we explicitly close the resource in the
"scope" it was opened and it still seems that we have the strange
"leak", which honestly surprised me. We even explicitly unset the vars
but that doesn't guarantee the GC kicks off for them near-time?=20
As a total aside, and being a paranoid C++ guy, I would love a "free"
method that I could call that frees what I tell it to exactly when I
tell it to... :-)
In any case, we'll apply the steps you outlined to debug the situations
and report back what we find.=20
Thanks.
- AD
-----Original Message-----
From: Sara Golemon [mailto:pol...@php.net]=20
Sent: Thursday, October 06, 2005 11:58 AM
To: "Ron Korving"
Cc: inte...@lists.php.net
Subject: [PHP-DEV] Re: CLI in PHP6
> There was once (can't remember when exactly, so it must be a long time
> ago)
> here on PHP CLI scripts in which it came forward that one should not
rely=20
> on
> such a script to run forever. And it's true; the scripts sometimes=20
> magically
> and suddenly die. Now I have no clue where this instability (for lack
of a
> better word) comes from, but could it be possible for this to be
resolved
> for PHP6 so that PHP becomes an extremely viable solution for CLI
daemon
> scripts?
>
The short answer is that PHP plays a little "fast-and-loose" with the
data=20
it tracks because it has no idea what you plan on doing with that data.
In=20
*theory* allocated resources get cleaned up within a request the minute
they=20
no longer have any possible relevance (i.e. A database connection is
created=20
within a function, then that function leaves making the variable it was=20
placed in no longer accessible from anywhere).
In 99% of the engine/core/exts this happens seemlessly, however there
*may*=20
be (and most likely is) a lingering leak somewhere because a file
descriptor=20
wasn't closed, or a couple bytes of memory wern't freed. In a "normal"
webserver environment these resources get forcibly cleaned up by the=20
end-of-request garbage collection process so they don't get noticed by
the=20
bulk of PHP users out there. In a long-running daemon process this GC
phase=20
never triggers and those piccune leaks pile up.
--
Jani Taskinen
> "leak", which honestly surprised me. We even explicitly unset the vars
> but that doesn't guarantee the GC kicks off for them near-time?
unset() != free(). The memory allocated is still freed during
the request shutdown (where GC actually kicks in).
> As a total aside, and being a paranoid C++ guy, I would love a "free"
> method that I could call that frees what I tell it to exactly when I
> tell it to... :-)
I think someone requested this before but it was shut down for
some reason..can't remember what it was again.
--Jani
J. Allen Dove
Unset() !=3D free() is the bummer in the CLI env. :-( Def could use that
to help shape the performance contour in a daemon env since the
"request" never ends unless you self-terminate. Even then it can be
tricky to get that lifetime right if your loads change, etc.=20
-- AD
> "leak", which honestly surprised me. We even explicitly unset the vars
> but that doesn't guarantee the GC kicks off for them near-time?
>
> unset() !=3D free(). The memory allocated is still freed during
> the request shutdown (where GC actually kicks in).
>
> As a total aside, and being a paranoid C++ guy, I would love a "free"
> method that I could call that frees what I tell it to exactly when I
> tell it to... :-)
>
> I think someone requested this before but it was shut down for
> some reason..can't remember what it was again.
--
Jani Taskinen
There was some mail about the memory handling some time ago on
this list..something about making it better in these cases..
Try search the archives for this. (I can't find it right now)
IIRC, there even was a patch..
--Jani
On Thu, 6 Oct 2005, J. Allen Dove wrote:
> Unset() != free() is the bummer in the CLI env. :-( Def could use that
> to help shape the performance contour in a daemon env since the
> "request" never ends unless you self-terminate. Even then it can be
> tricky to get that lifetime right if your loads change, etc.
>
> -- AD
>
>> "leak", which honestly surprised me. We even explicitly unset the vars
>> but that doesn't guarantee the GC kicks off for them near-time?
>>
>> unset() != free(). The memory allocated is still freed during
>> the request shutdown (where GC actually kicks in).
>>
>> As a total aside, and being a paranoid C++ guy, I would love a "free"
>> method that I could call that frees what I tell it to exactly when I
>> tell it to... :-)
>>
>> I think someone requested this before but it was shut down for
>> some reason..can't remember what it was again.
>
>
--
Give me your money at @ <http://pecl.php.net/wishlist.php/sniper>
Donating money may make me happier and friendlier for a limited period!
Death to all 4 letter abbreviations starting with P!
Ron Korving
freed at the end of the request, I can see where the problem is. It would be
very useful if this memory really would be freed at the moment all
references to it disappear. This would be a lot better for the CLI
environment, but also for a web environment, if you ask me, because the
memory required for handling pages might just be reduced by a big
percentage. I guess the question is: would this be a big performance hit in
processing time? (and I fear the answer is too big a "yes", otherwise I
expect it would've already been implemented).
Ron
"Jani Taskinen" <sni...@iki.fi> schreef in bericht
news:Pine.LNX.4.61.05...@arfg.argcubovn.sv...