On Thu, 26 May 2005, Dmitry Stogov wrote:
> This patch breaks binary compatibility.
> It cannot be broken in 4.3.x tree, and that doesn't make sense to release
> 4.4 just for this patch.
You mean that they think that it doesn't make sense to fix PHP in cases
where it's totally broken? I think that's totally irresponsible
behavior. Thousands of people still use PHP 4 for good reasons. If you
look correctly through the bug database you'll find atleast 10 bugs
related to strange segfaults and some have been confirmed to be related
to references that crash PHP.
I think we should definitely put this in a PHP release, although the
patch doesn't fix all segfaults yet.
(Patch is at: http://files.derickrethans.nl/patches/ze1-return-reference-20050429.diff.txt)
regards,
Derick
--
Derick Rethans
http://derickrethans.nl | http://ez.no | http://xdebug.org
--
PHP Internals - PHP Runtime Development Mailing List
To unsubscribe, visit: http://www.php.net/unsub.php
I also agree that these issues need to be fixed inside the 4.x tree.
What would be the motivation no to release a 4.4 to fix these issues?
Beating users with a stick towards PHP 5?
regards,
Lukas
In my opinion we should probably not fix this at all in the 4.x tree,
because the hassle involved in starting a new 4.4 branch - breaking module
compatibility for everyone, confusing people (imagine Apache 1.4 coming out
at this point), etc.
If it was an issue that everyone and their dog was bumping into, then I may
have thought differently - but it's an issue that is rare enough, and can
be worked around. And those that really need it to be fixed - can use the
patch.
Does everyone else think that this issue warrants starting a new version
branch?
Zeev
As we have run into segfaults with objects and references a couple of
times here I'd be interested in more details about the problem and the
implications of the patch.
I'm reading the mailing list throught the NNTP interface and it seems to
miss part of the conversation as the thread starts with Re: References
Problem Patch. Seems to be the same for the mailing list archive.
If you decide not to fix the 4.x branch then we'd minimally need an
easily accessible document describing the known problems and
work-arounds IMHO.
Thanks,
- Chris
> If it was an issue that everyone and their dog was bumping into, then I may
> have thought differently - but it's an issue that is rare enough, and can be
> worked around. And those that really need it to be fixed - can use the patch.
>
> Does everyone else think that this issue warrants starting a new version
> branch?
There are more issues besides this one. I still have segfaults here but
did not have the time to make a short script out of it. I will do that
very soon.
You know just as well that nobody that runs PHP seriously wants to patch
their version.
We would have no more maintenence tasks, as we don't have to continue
the 4.3 branch.
People don't easily encounter it because of the Zend memory manager
hiding stuff. The segfault is actually just a memory corruption which
can cause other weird results too (like objects changing class on the
fly).
(Not that it is an excuse to do again, but) We've broken binary
compatibility plenty of times in mini releases - it never was a
significant problem before for PHP.
Not fixing it is *not* an option. You fix something that's broken - you
don't leave it broken. That's called responsibility. And no, switching
to PHP 5 is not an option either.
regards,
Derick
--
Derick Rethans
http://derickrethans.nl | http://ez.no | http://xdebug.org
--
> Zeev Suraski wrote:
> > If it was an issue that everyone and their dog was bumping into, then I may
> > have thought differently - but it's an issue that is rare enough, and can be
> > worked around. And those that really need it to be fixed - can use the
> > patch.
>
> As we have run into segfaults with objects and references a couple of times
> here I'd be interested in more details about the problem and the implications
> of the patch.
See my two testcases:
http://files.derickrethans.nl/patches/reference-bug.phps
http://files.derickrethans.nl/patches/reference-test-case2.phps
But you would only spot it with valgrind and turning off Zend's memory
manager. The MM hides this kinds of errors pretty well.
> I'm reading the mailing list throught the NNTP interface and it seems to miss
> part of the conversation as the thread starts with Re: References Problem
> Patch. Seems to be the same for the mailing list archive.
It was a private discussion between Dmitry and me about the technical
aspects of the patch.
> If you decide not to fix the 4.x branch then we'd minimally need an easily
> accessible document describing the known problems and work-arounds IMHO.
As I tried to do that in our large code base, I would say that's totally
not possible to do. There is no way you know you're doing someting
wrong, as there are no warnings and go through 200k+ lines of code
without missing an instance is impossible.
> Derick Rethans wrote:
> > >If you decide not to fix the 4.x branch then we'd minimally need an easily
> > >accessible document describing the known problems and work-arounds IMHO.
> >
> > As I tried to do that in our large code base, I would say that's totally not
> > possible to do. There is no way you know you're doing someting wrong, as
> > there are no warnings and go through 200k+ lines of code without missing an
> > instance is impossible.
>
> That's a misunderstanding then. I was meaning work-arounds on the user side
> like avoiding certain constructs.
That's what I meant too. "Our large code base" is a PHP application.
Derick
Yes, that's what Derick and I are talking about too. What Derick is saying
is that finding the problematic pieces of code can be a long, agonizing
process. Then again, a lot of bugs fall in the same category...
Zeev
I think the question should really be: "why don't we want to give 4.x
users this bugfix?"
I can't think of any valid reasons not to do that.
--Wez.
On 5/30/05, Zeev Suraski <ze...@zend.com> wrote:
> At 12:00 30/05/2005, Lukas Smith wrote:
> >Derick Rethans wrote:
> >>Hi Dmitry,
> >>On Thu, 26 May 2005, Dmitry Stogov wrote:
> >>
> >>>This patch breaks binary compatibility.
> >>>It cannot be broken in 4.3.x tree, and that doesn't make sense to rele=
ase
> >>>4.4 just for this patch.
> >>
> >>You mean that they think that it doesn't make sense to fix PHP in cases
> >>where it's totally broken? I think that's totally irresponsible
> >>behavior. Thousands of people still use PHP 4 for good reasons. If you
> >>look correctly through the bug database you'll find atleast 10 bugs
> >>related to strange segfaults and some have been confirmed to be related
> >>to references that crash PHP.
> >>I think we should definitely put this in a PHP release, although the
> >>patch doesn't fix all segfaults yet.
> >>(Patch is at:
> >>http://files.derickrethans.nl/patches/ze1-return-reference-20050429.dif=
f.txt)
> >
> >I also agree that these issues need to be fixed inside the 4.x tree. Wha=
t
> >would be the motivation no to release a 4.4 to fix these issues? Beating
> >users with a stick towards PHP 5?
>=20
> In my opinion we should probably not fix this at all in the 4.x tree,
> because the hassle involved in starting a new 4.4 branch - breaking modul=
e
> compatibility for everyone, confusing people (imagine Apache 1.4 coming o=
ut
> at this point), etc.
> If it was an issue that everyone and their dog was bumping into, then I m=
ay
> have thought differently - but it's an issue that is rare enough, and can
> be worked around. And those that really need it to be fixed - can use th=
e
> patch.
>=20
> Does everyone else think that this issue warrants starting a new version
> branch?
>=20
> Zeev
>=20
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: http://www.php.net/unsub.php
>=20
Regarding the magnitude: It's pretty damn high, if you look at how
many bug reports we've got about reference issues and large (huge)
codebases. (where finding the short reproducing script is impossible)
--Jani
On Mon, 30 May 2005, Zeev Suraski wrote:
> At 17:10 30/05/2005, Derick Rethans wrote:
>> Not fixing it is *not* an option. You fix something that's broken - you
>> don't leave it broken. That's called responsibility. And no, switching
>> to PHP 5 is not an option either.
>
> Sorry Derick, but you saying that not fixing it and/or that switching to PHP 5
> are not valid options doesn't mean that they're not valid options. I think
> that with the rarity of this issue and even higher rarity of it actually
> causing problems, taking this approach is extremely valid. Breaking binary
> compatibility inside a 'PHP version family' on the other hand is not an
> option, we've decided not to do this 3 or 4 years ago, and I don't see any
> reason to break this decision at all.
>
> I for one think that (a) providing info and workarounds, and (b) pointing
> people to use PHP 5 is a completely acceptable solution to a problem of this
> magnitude, when compared to the cost of fixing it. Let's see what others
> think.
>
> Zeev
So, what's stopping the PHP project from kicking out 4.4?
--Wez.
On 5/30/05, Zeev Suraski <ze...@zend.com> wrote:
> At 18:01 30/05/2005, Wez Furlong wrote:
> >If we know the bug, and we have a fix, there shouldn't be anything
> >stopping us from making a release.
> >If this patch break binary compat, then the only logical move forward
> >is a 4.4 branch and release.
> >
> >I think the question should really be: "why don't we want to give 4.x
> >users this bugfix?"
> >I can't think of any valid reasons not to do that.
>=20
> I mentioned two, there are probably more - breaking binary compatibility
> (everyone has to rebuild everything, we'll probably have 0.00001C of glob=
al
> warming on our hands ;), and confusion.
>=20
> These reasons are not absolute. For instance, if it was a remote exploit
> we were dealing with - then obviously security takes precedence over thes=
e
> arguments. But it isn't. It's a bug that is pretty uncommon and can be
> worked around in userspace. Yes, it's annoying if you bump into it, but =
in
> the scale of severity, I don't think it rates very high.
>=20
> Zeev
>=20
Why cannot the desirable behaviour be obtained without
changing that particular structure?
- Sascha
If we have a fix, we should get it to our users.
Personally this problem has been haunting me for quite a while now and I
will be deploying the patch and happily breaking binary compatibility as
soon as Derick and Marcus tell me it is ready. A binary compatibility
break is annoying, but once I recompile all the revevant extensions, the
millions of lines of existing PHP4 code will run unchanged and with
fewer crashes.
The migration to PHP5 is a longer term effort that requires all the
domxml code to be rewritten, for one, about 140 custom extensions need
to be reviewed and some of them heavily modified, APC needs an update
and the existing PHP4 OO code needs to be reviewed. There is no way I
am going to not deploy a PHP4 crash fix in order to speed up the
transition to PHP5. I have a schedule for that which isn't going to
change and in the meantime anything I can do to improve the current
situation, I will do. And yes, while I realize my specific environment
is not exactly representative of the world at large, I also don't think
that other folks out there are all that different.
I think many people rely heavily on the packages maintained by the
various Linux distributions. A binary compatibility break is a burden
on the maintainers of these packages, but beyond needing to update every
PHP package, the end user really isn't burdened by it in any way unless
they have their own custom extensions, or if they have pecl extensions
installed, they'll need to grab them again from pecl and build against
the new dev files.
If the cost of fixing this is purely on us and a handful of distribution
maintainers with minimal cost to the end users, then I think the choice
should be simple here. Asking Joe Orton and the various other package
maintainers from the major distros might be a good idea too.
-Rasmus
Thank you! Thank you! Thank you!
The problems with references are one of the main reasons I watch this
list. As others have stated, with many tens of thousands of lines of
code, it is extremely painful to figure out which misplaced or missing
"&" has caused php to lose its mind. The fact that php loses its mind
in these cases has left me with the impression that php is sort of a
second class language. Putting the reference bugs to rest are the
most important bug fixes I can imagine. (No, I don't think it will
be feasible for us to move to php 5 anytime soon.)
Thank you!
- Todd Ruth
On Mon, 2005-05-30 at 16:44 +0200, Derick Rethans wrote:
> On Mon, 30 May 2005, Christian Schneider wrote:
>
> > Derick Rethans wrote:
> > > >If you decide not to fix the 4.x branch then we'd minimally need an easily
> > > >accessible document describing the known problems and work-arounds IMHO.
> > >
> > > As I tried to do that in our large code base, I would say that's totally not
> > > possible to do. There is no way you know you're doing someting wrong, as
> > > there are no warnings and go through 200k+ lines of code without missing an
> > > instance is impossible.
> >
> > That's a misunderstanding then. I was meaning work-arounds on the user side
> > like avoiding certain constructs.
>
> That's what I meant too. "Our large code base" is a PHP application.
>
> Derick
>
--
Monday, May 30, 2005, 5:21:00 PM, you wrote:
> At 18:01 30/05/2005, Wez Furlong wrote:
>>If we know the bug, and we have a fix, there shouldn't be anything
>>stopping us from making a release.
>>If this patch break binary compat, then the only logical move forward
>>is a 4.4 branch and release.
>>
>>I think the question should really be: "why don't we want to give 4.x
>>users this bugfix?"
>>I can't think of any valid reasons not to do that.
> I mentioned two, there are probably more - breaking binary compatibility
> (everyone has to rebuild everything, we'll probably have 0.00001C of global
> warming on our hands ;), and confusion.
We don't break BC here. We simply proceed as we did in the past. And we do
this to give the users the best PHP we can. On the other hand all you said
pretty much sounds like excuses to not do a new version. But why do you fear
that? Too much work - no since we would emmidiatley stop 4.3 support.
BC break - no since there simply is none. Everyone has to rebuild - well
this is a normal thing and all the windows users are used to simply download
the executables and for 95% of the *nix users it is only waiting for new
packages. Anybody being able to compile on his own will not have any problem
at all since he obviously is capable of building a working php anyway.
And the patch addresses some very serious problems. Unfortunatley there are
still a bunch of other issues unaddressed by now. To prevent 4.5 from
popping up to soon i think we should all take some look into fixing those
issues too. Anybody interested in those with a very deeply understanding
of the engine should contact Derick, Dmitry or me (working since several
weeks on those issues).
best regards
marcus
Monday, May 30, 2005, 7:42:27 PM, you wrote:
> Rasmus Lerdorf wrote:
>> Asking Joe Orton and the various other package maintainers from the
>> major distros might be a good idea too.
> If there were to be no PHP 4.4 release with the fix I think that we
> would integrate the patch (once it is ready, of course) into the PHP 4
> ebuilds for Gentoo Linux.
> That aside, it seems to me that releasing PHP 4.4 is not a technical but
> a psychological (another PHP 4 release might slow PHP 5 adoption) and
> political (another release series to be maintained by vendors like Zend)
> problem.
If that is true - why did espicaially companies made a rush towards php 5
release which now nearly nobody is using becuase everyone seems to consider
it as a beta version of php 5.1 with the consequence that atm we have three
php versions we need to take care of?
--
Best regards,
Marcus mailto:ma...@marcus-boerger.de
I think we need to clearly differentiate between features/limitations and
bugs. Often people on this list think that the former, especially
limitations, is something which desperately needs addressing, whereas I
think that addressing such issues in only the latest versions is fine (and
sometimes it's even better to live with it then over-design the language).
Now as far as bugs are concerned, we will never be able to resolve all
bugs, but we should try and solve the ones which can seriously bite people
especially if they are hard to detect *and* it's not clear how to work
around them.
I think in this case, the bug we are talking about falls into the category
of hard to detect and hard to work around. Therefore, I *do* think we
should release a new version of PHP 4 which resolves this bug, and I think
we should stick to the rule and not break binary compatibility, thus having
a 4.4.x version. If there are other bugs which can be addressed for 4.4.x
that's nice but don't make it the holy grail or it will never be released
(email to follow regarding 5.1 tomorrow).
Again, I think the fact that this bug has appeared a few times, has caused
a lot of pain, and is hard to understand and work around, it'd be best to
address it.
Andi
I don't think too many people consider PHP 5 as a beta of PHP 5.1. I
haven't bumped into many, the main thing I'm seeing is concern about the
ease of upgrading, which is slightly justified. I haven't bumped into any
company that has concerns about stability. If they exist it's unfortunate,
because as you know, the chances of breakage between 5.0 and 5.1 are around
the same as the breakage between 4.3 and 5.0 as far as stability is concerned.
I'm really not sure how you can consider a release cycle of just over 3
years to be rushing anything anywhere. It's not the first time I'm hearing
this, and it's utter ****. We wanted to bring the huge improvements in PHP
5 to a few people beyond the Marcus's of the world, who feel comfortable
playing with CVS's. We're already seeing quite a few PHP 5 deployments,
and PHP's proliferation wouldn't have been anywhere near where it is now if
PHP 5 wasn't released a year ago, so all I can say is thank heavens we
didn't wait for 5.1.
Anyway, that's off topic. If everyone here thinks it's a good idea to go
with 4.4, let's go ahead and do it.
Zeev
I'm tired going through the reasons again and again, and frankly it's not
the end of the world if we go with 4.4. I just thought that it's not
justified (there are downsides to it even if you guys fail to admit that),
but since everyone appears to prefer the upsides to the downsides, let's
just go ahead and do it. We just have to be very clear in our release
notes stating it's a bug fix release that just happens to break downwards
compatibility.
>And the patch addresses some very serious problems. Unfortunatley there are
>still a bunch of other issues unaddressed by now. To prevent 4.5 from
>popping up to soon i think we should all take some look into fixing those
>issues too.
I agree.
I knew there were other reasons! :D
But anyway, it's really irrelevant to this discussion.
Andi
At 09:19 PM 5/30/2005 +0200, Marcus Boerger wrote:
>Hello Sebastian,
>
>Monday, May 30, 2005, 7:42:27 PM, you wrote:
>
> > Rasmus Lerdorf wrote:
> >> Asking Joe Orton and the various other package maintainers from the
> >> major distros might be a good idea too.
>
> > If there were to be no PHP 4.4 release with the fix I think that we
> > would integrate the patch (once it is ready, of course) into the PHP 4
> > ebuilds for Gentoo Linux.
>
> > That aside, it seems to me that releasing PHP 4.4 is not a technical but
> > a psychological (another PHP 4 release might slow PHP 5 adoption) and
> > political (another release series to be maintained by vendors like Zend)
> > problem.
>
>If that is true - why did espicaially companies made a rush towards php 5
>release which now nearly nobody is using becuase everyone seems to consider
>it as a beta version of php 5.1 with the consequence that atm we have three
>php versions we need to take care of?
>
>--
>Best regards,
> Marcus mailto:ma...@marcus-boerger.de
>
>I don't think too many people consider PHP 5 as a beta of PHP 5.1. I=20
>haven't bumped into many, the main thing I'm seeing is concern about the=
=20
>ease of upgrading, which is slightly justified.
Slightly unrelated, I think this is the case where a bug like #32553
probably hits the users much, much harder than the references problem,
isn't included in an official release (or mentioned at php.net) and
has bugged users for two months now, would add to the "PHP5 is
beta"-consideration.
=46or the sake of the references-fix, I suppose it's a case of DIYDDIYD.
I agree with Zeev regarding the importance of the wording in the
release notes.
--=20
- Peter Brodersen
Anyway php-5 is not free of such refcount issues. One example is given
below. This code is a reduced form of xoops content Management
application's one activity which causes double free error.
<excerpts_of_my_mail_to_list_48_days_back>
<?php
class MyTextSanitizer
{
var $smileys=array()
function MyTextSanitizer() {}
function getSmileys()
{
return $this->smileys;
}
}
$myts = new MyTextSanitizer();
$smiles =& $myts->getSmileys(); //calling by ref alone causes improper
?>
The opcodes for the above script
ZEND_FETCH_CLASS
ZEND_NEW (Increases the refcount of smileys array from 1 to 2.
zend_declare class made it 1 from 0)
ZEND_JMP_NO_CTOR (Not executed)
ZEND_INIT_CTOR_CALL (No change in smileys refcount)
ZEND_DO_FCALL_BY_NAME(No change in smileys refcount)
ZEND_FETCH_W(No change in smileys refcount)
ZEND_ASSIGN(No change in smileys refcount)
ZEND_FETCH_R(No change in smileys refcount)
ZEND_INIT_METHOD_CALL(No change in smileys refcount)
ZEND_DO_FCALL_BY_NAME(Increases the refcount of smileys array from 2 to
3)
ZEND_FETCH_W(No change in smileys refcount)
ZEND_ASSIGN_REF(Increases the refcount of smileys array from 3 to 1.
_get_zval_ptr_ptr on &opline->op2 makes it 3 to 2.
zend_assign_to_variable_reference(&opline->result,
get_zval_ptr_ptr(&opline->op1, EX(Ts), BP_VAR_W), value_ptr_ptr, EX(Ts)
TSRMLS_CC); decreases it from 2 to 1)
ZEND_RETURN
ZEND_HANDLE_EXCEPTION
object destructor reduces the refcount from 1 to 0 and destroys the
$smileys. zend_destroy_class now attempts to destroy it again. This
causes a segfault.
With regards
Kamesh Jayachandran
</excerpts_of_my_mail_to_list_48_days_back>
On Mon, 30 May 2005 18:25:31 +0300 (EEST), "Jani Taskinen"
<sni...@iki.fi> said:
>
> Switching to PHP 5 (.1) here is not an option yet either.
> (and nobody can guarantee this same bug doesn't exist in it).
>
> Regarding the magnitude: It's pretty damn high, if you look at how
> many bug reports we've got about reference issues and large (huge)
> codebases. (where finding the short reproducing script is
> impossible)
>
> --Jani
>
>
> On Mon, 30 May 2005, Zeev Suraski wrote:
>
> > At 17:10 30/05/2005, Derick Rethans wrote:
> >> Not fixing it is *not* an option. You fix something that's broken - you
> >> don't leave it broken. That's called responsibility. And no, switching
> >> to PHP 5 is not an option either.
> >
> > Sorry Derick, but you saying that not fixing it and/or that switching to PHP 5
> > are not valid options doesn't mean that they're not valid options. I think
> > that with the rarity of this issue and even higher rarity of it actually
> > causing problems, taking this approach is extremely valid. Breaking binary
> > compatibility inside a 'PHP version family' on the other hand is not an
> > option, we've decided not to do this 3 or 4 years ago, and I don't see any
> > reason to break this decision at all.
> >
> > I for one think that (a) providing info and workarounds, and (b) pointing
> > people to use PHP 5 is a completely acceptable solution to a problem of this
> > magnitude, when compared to the cost of fixing it. Let's see what others
> > think.
> >
> > Zeev
> >
> >
>
> --
> PHP Internals - PHP Runtime Development Mailing List
> To unsubscribe, visit: http://www.php.net/unsub.php
>
--
http://www.fastmail.fm - IMAP accessible web-mail
> I'm tired going through the reasons again and again, and frankly it's not the
> end of the world if we go with 4.4. I just thought that it's not justified
> (there are downsides to it even if you guys fail to admit that), but since
> everyone appears to prefer the upsides to the downsides, let's just go ahead
> and do it. We just have to be very clear in our release notes stating it's a
> bug fix release that just happens to break downwards compatibility.
Ok, so I propose I'll make a breanch PHP_4_4 next week where we can
commit those patches too. It seems with the reference patch and the
stristr patch that I did today it seems all working just fine.
I can definitely put time in making sure this branch is as stable as we
want, before we release. I can also be RM - glad to do it.
> >And the patch addresses some very serious problems. Unfortunatley there are
> >still a bunch of other issues unaddressed by now. To prevent 4.5 from
> >popping up to soon i think we should all take some look into fixing those
> >issues too.
Now is the time to make this list :)
regards,
Derick
--
Derick Rethans
http://derickrethans.nl | http://ez.no | http://xdebug.org
--
> On Mon, May 30, 2005 at 08:58:47AM -0700, Rasmus Lerdorf wrote:
> > I think many people rely heavily on the packages maintained by the
> > various Linux distributions. A binary compatibility break is a burden
> > on the maintainers of these packages, but beyond needing to update every
> > PHP package, the end user really isn't burdened by it in any way unless
> > they have their own custom extensions, or if they have pecl extensions
> > installed, they'll need to grab them again from pecl and build against
> > the new dev files.
> >
> > If the cost of fixing this is purely on us and a handful of distribution
> > maintainers with minimal cost to the end users, then I think the choice
> > should be simple here. Asking Joe Orton and the various other package
> > maintainers from the major distros might be a good idea too.
>
> I think it's fair to say we would not ship a php update for Fedora Core
> 3 (and certainly not for RHEL) which broke third-party module
> compability unless it was a for a really critical issue i.e. remotely
> exploitable security bug. I'm not sure this bug qualifies, given that
> it can be worked around.
Actually, I don't think it can be worked around. Perhaps technically,
but not practically.