starting with 2.7
Maciej Soltysiak
I was wondering in the tram today are we close to branching
off to 2.7
Do the mighty kernel developers have solid plans, ideas, etc
to start experimental code
Regards,
Maciej
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majo...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Emmanuel Fleury
> Hi,
>
> I was wondering in the tram today are we close to branching
> off to 2.7
>
> Do the mighty kernel developers have solid plans, ideas, etc
> to start experimental code
You should read this: http://kerneltrap.org/node/3513
And this: http://kerneltrap.org/node/3522
Regards
--
Emmanuel Fleury
Computer Science Department, | Office: B1-201
Aalborg University, | Phone: +45 96 35 72 23
Fredriks Bajersvej 7E, | Fax: +45 98 15 98 89
9220 Aalborg East, Denmark | Email: fle...@cs.aau.dk
William Lee Irwin III
> I was wondering in the tram today are we close to branching
> off to 2.7
> Do the mighty kernel developers have solid plans, ideas, etc
> to start experimental code
I have a plan to never ever stop experimental code, which is to
actually move on the 2.6.x.y strategy if no one else does and these
kinds of complaints remain persistent and become more widespread.
There is a standard. Breaking things and hoping someone cleans up
later doesn't work. So it has to be stable all the time anyway, and
this is one of the observations upon which the "2.6 forever" theme is
based. Frozen "minimal fix trees" for the benefit of those terrified of
new working code (or alternatively, the astoundingly risk-averse) are a
relatively straightforward theme, which kernel maintainers should be
fully able to faithfully develop.
-- wli
Maciej Soltysiak
Sunday, January 2, 2005, 9:36:15 PM, you wrote:
> I have a plan to never ever stop experimental code, which is to
> actually move on the 2.6.x.y strategy if no one else does and these
> kinds of complaints remain persistent and become more widespread.
Well, personally I like the 2.6.x.y idea.
> There is a standard. Breaking things and hoping someone cleans up
> later doesn't work. So it has to be stable all the time anyway, and
> this is one of the observations upon which the "2.6 forever" theme is
> based.
Hm, now that I think about it, it has an advantage. eg. No problems
for other people with maintaining code for 2.4, 2.5, 2.6, ...
Anyway, starting with other trees like 2.7 sends a signal to people, that
something huge is going to happen. Like totally new ideas that require
to change the APIs, rewrite all the drivers (I know nobody likes that, really)
etc...
I was asking If something like this is about to happen.
Is a stockpile of things like this building up?
--
Maciej
Andries Brouwer
> There is a standard. Breaking things and hoping someone cleans up
> later doesn't work. So it has to be stable all the time anyway, and
> this is one of the observations upon which the "2.6 forever" theme is
> based.
You are an optimist. I think reality is different.
You change some stuff. The bad mistakes are discovered very soon.
Some subtler things or some things that occur only in special
configurations or under special conditions or just with
very low probability may not be noticed until much later.
So, your changes have a wake behind them that is wide the first
few days and becomes thinner and thinner over time. Nontrivial
changes may have bugs discovered after two or three years.
If a kernel is set apart and called "stable", then it is not,
but it will become more and more stable over time, until after
two or three years only very few unknown problems are encountered.
If you come with a new kernel every month, then you get
the stability that the "stable" kernel has after less than a month,
which is not particularly stable.
Andries
William Lee Irwin III
> You are an optimist. I think reality is different.
> You change some stuff. The bad mistakes are discovered very soon.
> Some subtler things or some things that occur only in special
> configurations or under special conditions or just with
> very low probability may not be noticed until much later.
> So, your changes have a wake behind them that is wide the first
> few days and becomes thinner and thinner over time. Nontrivial
> changes may have bugs discovered after two or three years.
> If a kernel is set apart and called "stable", then it is not,
> but it will become more and more stable over time, until after
> two or three years only very few unknown problems are encountered.
> If you come with a new kernel every month, then you get
> the stability that the "stable" kernel has after less than a month,
> which is not particularly stable.
This is not optimism. This is experience. Every ``stable'' kernel I've
seen is a pile of incredibly stale code where vi'ing any file in it
instantly reveals numerous months or years old bugs fixed upstream.
What is gained in terms of reducing the risk of regressions is more
than lost by the loss of critical examination and by a long longshot.
-- wli
Adrian Bunk
>
> This is not optimism. This is experience. Every ``stable'' kernel I've
> seen is a pile of incredibly stale code where vi'ing any file in it
> instantly reveals numerous months or years old bugs fixed upstream.
> What is gained in terms of reducing the risk of regressions is more
> than lost by the loss of critical examination and by a long longshot.
The main advantage with stable kernels in the good old days (tm) when 4
and 6 were even numbers was that you knew if something didn't work, and
upgrading to a new kernel inside this stable kernel series had a
relatively low risk of new breakages. This meant one big migration every
few years and relatively easy upgrades between stable series kernels.
Nowadays in 2.6, every new 2.6 kernel has several regressions compared
to the previous one, and additionally obsolete but used code like
ipchains and devfs is scheduled for removal making upgrades even harder
for many users.
There's the point that most users should use distribution kernels, but
consider e.g. that there are poor souls with new hardware not supported
by the 3 years old 2.4.18 kernel in the stable part of your Debian
distribution.
> -- wli
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
Bill Davidsen
> On Sun, Jan 02, 2005 at 01:42:11PM -0800, William Lee Irwin III wrote:
>
>>This is not optimism. This is experience. Every ``stable'' kernel I've
>>seen is a pile of incredibly stale code where vi'ing any file in it
>>instantly reveals numerous months or years old bugs fixed upstream.
>>What is gained in terms of reducing the risk of regressions is more
>>than lost by the loss of critical examination and by a long longshot.
>
>
> The main advantage with stable kernels in the good old days (tm) when 4
> and 6 were even numbers was that you knew if something didn't work, and
> upgrading to a new kernel inside this stable kernel series had a
> relatively low risk of new breakages. This meant one big migration every
> few years and relatively easy upgrades between stable series kernels.
>
> Nowadays in 2.6, every new 2.6 kernel has several regressions compared
> to the previous one, and additionally obsolete but used code like
> ipchains and devfs is scheduled for removal making upgrades even harder
> for many users.
And there you have my largest complaint with the new model. If 2.6 is
stable, it should not have existing features removed just because
someone has a new wet dream about a better but incompatible way to do
things. I expect working programs to be deliberately broken in a
development tree, but once existing features are removed there simply is
no stable set of features.
>
> There's the point that most users should use distribution kernels, but
> consider e.g. that there are poor souls with new hardware not supported
> by the 3 years old 2.4.18 kernel in the stable part of your Debian
> distribution.
The stable and development kernel model worked for a decade, partly
because people could build on a feature set and not have that feature
just go away, leaving the choice of running without fixes or not
running. Since we manage to support 2.2 and 2.4 (and perhaps even 2.0?)
I don't see why the definition of "stable" can't simply mean "no
deletions from the feature set" and let new features come in for those
who want them. Absent that 2.4 will be the last stable kernel, in the
sense that features won't be deliberately broken or removed.
--
bill davidsen <davi...@tmr.com>
CTO TMR Associates, Inc
Doing interesting things with small computers since 1979
Jesper Juhl
The new model has some good aspects to it, mainly that new fixes and
features make it into the "stable" kernel a lot faster these days. But, it
is not as stable/static as previous stable series.
Personally I think a lot of the problems could go away if the -mm tree was
utilized more.
How does something like this sound?
2.6.x being stable means that :
a) features are not removed
b) patches applied to 2.6 have spend a minimum of time in -mm first (one
-mm release minimum? -mm release frequency would probably need to go
up a bit)
c) interfaces do not change unless to fix a serious bug
That would make it a stable series in my book.
Development can still be rapid since -mm would be open to all the
experimental stuff, and features could be removed in -mm etc.
If *all* patches go through -mm (critical security fixes possibly being
the excetion) before ending up in mainline, then -mm won't get out of sync
with mainline with regard to bugfixes - main differences would be new
experimental stuff and removed features.
When should 2.7 open then? Well, probably when -mm has diverged so much
that it becomes a pain to maintain against 2.6 mainline, then it could be
renamed 2.7.0 and work could start on stabilizing all the experimental
stuff in it etc.
Does that sound completely insane or as something that could work?
I think it's not so far from what we have now.
--
Jesper Juhl
Diego Calleja
:
> The main advantage with stable kernels in the good old days (tm) when=
4
> and 6 were even numbers was that you knew if something didn't work, a=
nd
> upgrading to a new kernel inside this stable kernel series had a
> relatively low risk of new breakages. This meant one big migration ev=
ery
> few years and relatively easy upgrades between stable series kernels.
That's not always true, 2.4.x development has not been exactly what I'd
call "stable". IIRC 2.4.15 - the 2.6 fork I think - could corrupt your
filesystems and I don't remember right now if there were more, personal=
ly
I've suffered of "weird" behaviours until the new VM was stabilized, an=
d
I've heard of lots of reiser and ext3 problems until both filesystems g=
ot
stabilized. I've lost my filesystems 3 times with 2.4, 0 times running =
2.5
since 2.5.3x (of course that could be just good luck or bad luck but...=
)
Of course that only proves your point: that changes may cause bugs 8) b=
ut
for me 2.6 has been by far the stablest release linux has ever had, wit=
h
some minor issues in each release while at the same time incorporating =
"big"
changes which is something I can accept as "desktop user". Perhaps 2.6 =
will
become "rock stable" or "to be used only by servers not desktops" when =
2.7
forks?
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel"=
Dr. David Alan Gilbert
the kernel, I have always found the stable series useful for
two reasons:
1) It encourages me to test the kernel; if I have a kernel
that is generally thought to be stable then I will try it on
my home machine and report problems - this lets the kernel
get tested on a wide range of hardware and situations; if there
is no kernel that is liable to be stable changes will get much
less testing on a smaller range of hardware.
2) If I have a bug in a vendor kernel everyone just tells
me to go and speak to the vendor - so at least having a stable
base to go back to can let me report a bug that isn't due
to any vendors patches.
3) In some cases the commercial vendors don't seem to release
source to some of the kernels except to people who have bought
the packages, so those vendor kernel fixes aren't 'publically'
visible.
I think (1) is very important - getting large numbers of people
to test OSS is its greatest asset.
Dave
-----Open up your eyes, open up your mind, open up your code -------
/ Dr. David Alan Gilbert | Running GNU/Linux on Alpha,68K| Happy \
\ gro.gilbert @ treblig.org | MIPS,x86,ARM,SPARC,PPC & HPPA | In Hex /
\ _________________________|_____ http://www.treblig.org |_______/
William Lee Irwin III
>> This is not optimism. This is experience. Every ``stable'' kernel I've
>> seen is a pile of incredibly stale code where vi'ing any file in it
>> instantly reveals numerous months or years old bugs fixed upstream.
>> What is gained in terms of reducing the risk of regressions is more
>> than lost by the loss of critical examination and by a long longshot.
On Sun, Jan 02, 2005 at 11:15:34PM +0100, Adrian Bunk wrote:
> The main advantage with stable kernels in the good old days (tm) when 4
> and 6 were even numbers was that you knew if something didn't work, and
> upgrading to a new kernel inside this stable kernel series had a
> relatively low risk of new breakages. This meant one big migration every
> few years and relatively easy upgrades between stable series kernels.
This never saved anyone any pain. 2.4.x was not the stable kernel
you're painting it to be until 2.4.20 or later, and by the time it
became so the fixes for major regressions that occurred during 2.3.x
were deemphasized and ignored for anything prior to 2.6.x.
On Sun, Jan 02, 2005 at 11:15:34PM +0100, Adrian Bunk wrote:
> Nowadays in 2.6, every new 2.6 kernel has several regressions compared
> to the previous one, and additionally obsolete but used code like
> ipchains and devfs is scheduled for removal making upgrades even harder
> for many users.
My experience tells me that the number of regressions in 2.6.x compared
to purportedly ``far stabler'' kernels is about the same or (gasp!)
less. So the observable advantage of the ``frozen'' or ``stable'' model
is less than or equal to zero.
Frankly, kernel hacking is a difficult enough task (not that I
personally find it so) that frivolous patches are not overwhemingly
numerous. The ``barrier'' you're erecting is primarily acting as a
barrier to fixes, not bugs.
On Sun, Jan 02, 2005 at 11:15:34PM +0100, Adrian Bunk wrote:
> There's the point that most users should use distribution kernels, but
> consider e.g. that there are poor souls with new hardware not supported
> by the 3 years old 2.4.18 kernel in the stable part of your Debian
> distribution.
Again, the loss of critical examination far outweighs the purported
defense against regressions. The most typical result of playing the fix
backporting game for extended periods of time is numerous rounds of
months-long bughunts for bugs whose fixes were merged years ago upstream.
When the bugs are at long last found, they are discovered to fix the
problems of hundreds of users until the next such problem surfaces.
-- wli
William Lee Irwin III
>> The main advantage with stable kernels in the good old days (tm) when 4
>> and 6 were even numbers was that you knew if something didn't work, and
>> upgrading to a new kernel inside this stable kernel series had a
>> relatively low risk of new breakages. This meant one big migration every
>> few years and relatively easy upgrades between stable series kernels.
>> Nowadays in 2.6, every new 2.6 kernel has several regressions compared
>> to the previous one, and additionally obsolete but used code like
>> ipchains and devfs is scheduled for removal making upgrades even harder
>> for many users.
On Sun, Jan 02, 2005 at 05:49:08PM -0500, Bill Davidsen wrote:
> And there you have my largest complaint with the new model. If 2.6 is
> stable, it should not have existing features removed just because
> someone has a new wet dream about a better but incompatible way to do
> things. I expect working programs to be deliberately broken in a
> development tree, but once existing features are removed there simply is
> no stable set of features.
The presumption is that these changes are frivolous. This is false.
The removals of these features are motivated by their unsoundness,
and those removals resolve real problems. If they did not do so, they
would not pass peer review.
Adrian Bunk wrote:
>> There's the point that most users should use distribution kernels, but
>> consider e.g. that there are poor souls with new hardware not supported
>> by the 3 years old 2.4.18 kernel in the stable part of your Debian
>> distribution.
On Sun, Jan 02, 2005 at 05:49:08PM -0500, Bill Davidsen wrote:
> The stable and development kernel model worked for a decade, partly
> because people could build on a feature set and not have that feature
> just go away, leaving the choice of running without fixes or not
> running. Since we manage to support 2.2 and 2.4 (and perhaps even 2.0?)
> I don't see why the definition of "stable" can't simply mean "no
> deletions from the feature set" and let new features come in for those
> who want them. Absent that 2.4 will be the last stable kernel, in the
> sense that features won't be deliberately broken or removed.
I can't speak for anyone during the times of more ancient Linux history;
however, developers' dissatisfaction with the development model has been
aired numerous times in certain fora. It has not satisfactorily served
developers or users. Users are locked into distro kernels for
incompatible extensions, and developers are torn between multiple
codebases.
This fragmentation of programmer effort is trivially recognizable as
counterproductive. A single focal point for programmer effort is far
superior for a development model. If the standard of stability is not
passed then the code is not ready to be included in any kernel. Then
the distinction is lost, and each of the fragmented codebases gets a
third-class effort, and a spurious expenditure of effort is wasted on
porting fixes and features across numerous different codebases.
Your estimate of the impact of the feature set upon applications is
also somewhat exaggerated. The system call ABI has remained backward
compatible for extended periods of time covering major releases. In
fact, it would be my personal preference for the next major release to
signify nothing more than a system call ABI change involving the purging
of several older, deprecated system call conventions.
-- wli
Adrian Bunk
> On Sun, Jan 02, 2005 at 01:42:11PM -0800, William Lee Irwin III wrote:
> >> This is not optimism. This is experience. Every ``stable'' kernel I've
> >> seen is a pile of incredibly stale code where vi'ing any file in it
> >> instantly reveals numerous months or years old bugs fixed upstream.
> >> What is gained in terms of reducing the risk of regressions is more
> >> than lost by the loss of critical examination and by a long longshot.
>
> On Sun, Jan 02, 2005 at 11:15:34PM +0100, Adrian Bunk wrote:
> > The main advantage with stable kernels in the good old days (tm) when 4
> > and 6 were even numbers was that you knew if something didn't work, and
> > upgrading to a new kernel inside this stable kernel series had a
> > relatively low risk of new breakages. This meant one big migration every
> > few years and relatively easy upgrades between stable series kernels.
>
> This never saved anyone any pain. 2.4.x was not the stable kernel
> you're painting it to be until 2.4.20 or later, and by the time it
> became so the fixes for major regressions that occurred during 2.3.x
> were deemphasized and ignored for anything prior to 2.6.x.
I don't know which specific regressions you have in mind, but for
> 95% of the users 2.4 is a pretty usable kernel.
> On Sun, Jan 02, 2005 at 11:15:34PM +0100, Adrian Bunk wrote:
> > Nowadays in 2.6, every new 2.6 kernel has several regressions compared
> > to the previous one, and additionally obsolete but used code like
> > ipchains and devfs is scheduled for removal making upgrades even harder
> > for many users.
>
> My experience tells me that the number of regressions in 2.6.x compared
> to purportedly ``far stabler'' kernels is about the same or (gasp!)
> less. So the observable advantage of the ``frozen'' or ``stable'' model
> is less than or equal to zero.
>
> Frankly, kernel hacking is a difficult enough task (not that I
> personally find it so) that frivolous patches are not overwhemingly
> numerous. The ``barrier'' you're erecting is primarily acting as a
> barrier to fixes, not bugs.
My point is different.
Perhaps the number of fixes for bugs equals the number of new bugs
in 2.6 .
But it's not about the number of bugs alone. The question is the number
of regressions compared to a previous kernel in this series.
2.4 -> 2.6 is a major migration.
2.4.27 -> 2.4.28 is a kernel upgrade that is very unlikely to cause
problems.
Compared to this, 2.6.9 -> 2.6.10 is much more likely to break an
existing setup that worked in 2.6.9 .
> On Sun, Jan 02, 2005 at 11:15:34PM +0100, Adrian Bunk wrote:
> > There's the point that most users should use distribution kernels, but
> > consider e.g. that there are poor souls with new hardware not supported
> > by the 3 years old 2.4.18 kernel in the stable part of your Debian
> > distribution.
>
> Again, the loss of critical examination far outweighs the purported
> defense against regressions. The most typical result of playing the fix
> backporting game for extended periods of time is numerous rounds of
> months-long bughunts for bugs whose fixes were merged years ago upstream.
> When the bugs are at long last found, they are discovered to fix the
> problems of hundreds of users until the next such problem surfaces.
The main question is, whether it might be possible to make a very short
2.7 line (< 6 months).
Imagine e.g. a feature freeze for 2.6 now. Then 2.7 starts with a
feature freeze for 2.7 one or two months later. During this time, all
the changes that do now flood into 2.6 would go into 2.7, and then
there are a few months of stabilizing 2.7 .
It's quite the opposite of the current 2.6 model, but a quick 2.8 should
also avoid this problem you describe.
Basically, in this proposal (if it started today), what was expected to
be called 2.6.11 will be called 2.7.0, and 2.6.11 will be a bugfix-only
kernel (considering the amount of changes more like the current -ac than
the latest -mm).
> -- wli
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
-
Adrian Bunk
> Adrian Bunk wrote:
> >> The main advantage with stable kernels in the good old days (tm) when 4
> >> and 6 were even numbers was that you knew if something didn't work, and
> >> upgrading to a new kernel inside this stable kernel series had a
> >> relatively low risk of new breakages. This meant one big migration every
> >> few years and relatively easy upgrades between stable series kernels.
> >> Nowadays in 2.6, every new 2.6 kernel has several regressions compared
> >> to the previous one, and additionally obsolete but used code like
> >> ipchains and devfs is scheduled for removal making upgrades even harder
> >> for many users.
>
> On Sun, Jan 02, 2005 at 05:49:08PM -0500, Bill Davidsen wrote:
> > And there you have my largest complaint with the new model. If 2.6 is
> > stable, it should not have existing features removed just because
> > someone has a new wet dream about a better but incompatible way to do
> > things. I expect working programs to be deliberately broken in a
> > development tree, but once existing features are removed there simply is
> > no stable set of features.
>
> The presumption is that these changes are frivolous. This is false.
> The removals of these features are motivated by their unsoundness,
> and those removals resolve real problems. If they did not do so, they
> would not pass peer review.
The netfilter people plan to remove ipfwadm and ipchains before 2.6.11 .
This is legacy code that makes their development sometimes a bit harder,
but AFAIK ipchains in 2.6.10 doesn't suffer from any serious real
problems.
> Adrian Bunk wrote:
> >> There's the point that most users should use distribution kernels, but
> >> consider e.g. that there are poor souls with new hardware not supported
> >> by the 3 years old 2.4.18 kernel in the stable part of your Debian
> >> distribution.
>
> On Sun, Jan 02, 2005 at 05:49:08PM -0500, Bill Davidsen wrote:
> > The stable and development kernel model worked for a decade, partly
> > because people could build on a feature set and not have that feature
> > just go away, leaving the choice of running without fixes or not
> > running. Since we manage to support 2.2 and 2.4 (and perhaps even 2.0?)
> > I don't see why the definition of "stable" can't simply mean "no
> > deletions from the feature set" and let new features come in for those
> > who want them. Absent that 2.4 will be the last stable kernel, in the
> > sense that features won't be deliberately broken or removed.
>
> I can't speak for anyone during the times of more ancient Linux history;
> however, developers' dissatisfaction with the development model has been
> aired numerous times in certain fora. It has not satisfactorily served
> developers or users. Users are locked into distro kernels for
> incompatible extensions, and developers are torn between multiple
> codebases.
At least on Debian, ftp.kernel.org kernels work fine.
> This fragmentation of programmer effort is trivially recognizable as
> counterproductive. A single focal point for programmer effort is far
> superior for a development model. If the standard of stability is not
> passed then the code is not ready to be included in any kernel. Then
> the distinction is lost, and each of the fragmented codebases gets a
> third-class effort, and a spurious expenditure of effort is wasted on
> porting fixes and features across numerous different codebases.
>...
My impression is that currently 2.4 doesn't take that much time of
developers (except for Marcelo's), and that it's a quite usable and
stable kernel.
> -- wli
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
-
Adam Mercer
> 2.4.27 -> 2.4.28 is a kernel upgrade that is very unlikely to cause
> problems.
>
> Compared to this, 2.6.9 -> 2.6.10 is much more likely to break an
> existing setup that worked in 2.6.9 .
IIRC 2.4.9 -> 2.4.10 broke a few setups as well.
Cheers
Adam
William Lee Irwin III
>> 2.4.27 -> 2.4.28 is a kernel upgrade that is very unlikely to cause
>> problems.
>> Compared to this, 2.6.9 -> 2.6.10 is much more likely to break an
>> existing setup that worked in 2.6.9 .
On Mon, Jan 03, 2005 at 12:49:22AM +0000, Adam Mercer wrote:
> IIRC 2.4.9 -> 2.4.10 broke a few setups as well.
Negligible. Compare to 2.4.9 vs. 2.4.10
-- wli
William Lee Irwin III
>> The presumption is that these changes are frivolous. This is false.
>> The removals of these features are motivated by their unsoundness,
>> and those removals resolve real problems. If they did not do so, they
>> would not pass peer review.
On Mon, Jan 03, 2005 at 01:45:51AM +0100, Adrian Bunk wrote:
> The netfilter people plan to remove ipfwadm and ipchains before 2.6.11 .
> This is legacy code that makes their development sometimes a bit harder,
> but AFAIK ipchains in 2.6.10 doesn't suffer from any serious real
> problems.
They're superseded by iptables and their sole uses are by the Linux-
specific applications to manipulate this privileged aspect of system
state. This makes a much weaker case for backward compatibility than
general nonprivileged standardized system calls.
On Sun, Jan 02, 2005 at 04:30:11PM -0800, William Lee Irwin III wrote:
>> I can't speak for anyone during the times of more ancient Linux history;
>> however, developers' dissatisfaction with the development model has been
>> aired numerous times in certain fora. It has not satisfactorily served
>> developers or users. Users are locked into distro kernels for
>> incompatible extensions, and developers are torn between multiple
>> codebases.
On Mon, Jan 03, 2005 at 01:45:51AM +0100, Adrian Bunk wrote:
> At least on Debian, ftp.kernel.org kernels work fine.
This does not advance the argument.
On Sun, Jan 02, 2005 at 04:30:11PM -0800, William Lee Irwin III wrote:
>> This fragmentation of programmer effort is trivially recognizable as
>> counterproductive. A single focal point for programmer effort is far
>> superior for a development model. If the standard of stability is not
>> passed then the code is not ready to be included in any kernel. Then
>> the distinction is lost, and each of the fragmented codebases gets a
>> third-class effort, and a spurious expenditure of effort is wasted on
>> porting fixes and features across numerous different codebases.
>>...
On Mon, Jan 03, 2005 at 01:45:51AM +0100, Adrian Bunk wrote:
> My impression is that currently 2.4 doesn't take that much time of
> developers (except for Marcelo's), and that it's a quite usable and
> stable kernel.
The ``stable'' moniker and distros being based on 2.4 are horrors
beyond imagination and developers are pushed to in turn push
inappropriate features on 2.4.x and maintain them out-of-tree for
2.4.x
-- wli
William Lee Irwin III
> Compared to this, 2.6.9 -> 2.6.10 is much more likely to break an
> existing setup that worked in 2.6.9 .
c.f. the reply to Adam Mercery wrt. 2.4.9 vs. 2.4.10
-- wli
Willy Tarreau
> On Sun, Jan 02, 2005 at 04:30:11PM -0800, William Lee Irwin III wrote:
> >> The presumption is that these changes are frivolous. This is false.
> >> The removals of these features are motivated by their unsoundness,
> >> and those removals resolve real problems. If they did not do so, they
> >> would not pass peer review.
>
> On Mon, Jan 03, 2005 at 01:45:51AM +0100, Adrian Bunk wrote:
> > The netfilter people plan to remove ipfwadm and ipchains before 2.6.11 .
> > This is legacy code that makes their development sometimes a bit harder,
> > but AFAIK ipchains in 2.6.10 doesn't suffer from any serious real
> > problems.
>
> They're superseded by iptables and their sole uses are by the Linux-
> specific applications to manipulate this privileged aspect of system
> state. This makes a much weaker case for backward compatibility than
> general nonprivileged standardized system calls.
That's not the problem. There was a feature freeze by which everything which
was considered hard to maintain or not very stable should have been removed.
When 2.6 was announced, it was with a set of features. Who know, perhaps there
are a few people who could replace a kernel 2.0 by a 2.6 on some firewalls.
Even if they are only 2 or 3 people, there is no reason that suddenly a feature
should be removed in the stable series. But it should be removed in 2.7 if it's
a nightmare to maintain.
If the motivation to break backwards compatibility is not enough anymore to
justify development kernels, I don't know what will justify it anymore.
I'm particularly fed up by some developer's attitude who seem to never go
outside and see how their creations are used by people who really trust the
"stable" term... until they realize that this word is used only for marketting,
eg. help distro makers announce their new major release at the right moment.
ipfwadm had about 2 years to be removed before 2.6, wasn't that enough ? Once
the stable release is out, the developer's point of view about how is creation
*might* be used is not a justification to remove it. But of course, his
difficulties at maintaining the code is fairly enough for him to say "well, it
was a mistake to enable this, I don't want it in the future version anymore".
Why do you think that so many people are still using 2.4 (and even older
versions) ? This is because they are the only ones who don't constantly
change under your feet and from which you can build something reliable and
guaranteed maintainable. At least, I've not seen any commercial product based
on 2.6 yet !
Please, stop constantly changing the contents of the "stable" kernel.
> On Mon, Jan 03, 2005 at 01:45:51AM +0100, Adrian Bunk wrote:
> > My impression is that currently 2.4 doesn't take that much time of
> > developers (except for Marcelo's), and that it's a quite usable and
> > stable kernel.
>
> The ``stable'' moniker and distros being based on 2.4 are horrors
> beyond imagination and developers are pushed to in turn push
> inappropriate features on 2.4.x and maintain them out-of-tree for
> 2.4.x
I clearly don't agree with you, for a simple reason : those out-of-tree
features will always be, because each distro likes to add a few features,
like SquashFS, PaX, etc... And indeed, that's one of the reasons I *stay*
on 2.4. It's so simple to simply upgrade the kernel, patch and recompile
without spending days complaining "grrr... why did they change this ?".
As soon as you have at least *ONE* patch to apply to a kernel for your
distro, 2.4 is a safer bet than 2.6 if you don't want to restart everything
at each minor release. The 2.4 kernel is more what I consider stable than
2.6, eventhough it's not totally. 2.0 and 2.2 *are* stable, because I'm
certain that every future releases will only be bugfixes and will touch
only a few lines.
At the moment, the only "serious" use I've found for a 2.6 is a kexec-based
bootloader for known hardware. I've already seen that maintaining it up to
date is not simple, I wonder how distro people work with it... I wouldn't
have to do their work right now.
Regards,
Willy
L. A. Walsh
I always preferred to run a vanilla stable kernel as I did not
trust the vendors' kernels because their patches were not as well
eyed as the vanilla kernel. I prefer to compile a kernel for
my specific machines, some of which are old and do better with a
hand-configured kernel rather than a Microsoftian monolith that
is compiled with all possible options as modules.
I have one old laptop that sound just doesn't work on no matter
what the settings -- may be failed hardware, but darned if I can't
seem to easily stop the loading of sound related modules as hardware
is probed by automatic hardware probing on each bootup, and the loading
of sound modules by GUI dependencies on a memory constrained system.
With each new kernel release, I wonder if it will be satisfactory
to use for a new, base-line, stable vanilla kernel, but post release
comments seem to prove otherwise.
It seems that some developers have the opinion that the end-user base
no longer is their problem or audience and that the distros will patch
all the little boo-boo's in each unstable 2.6 release. Well, that's
just plain asking for problems. Just in SuSE's previous release of
9.1, it wouldn't boot up, for update, on any system that used xfs
disks. Redhat has officially dropped support for end-user distros,
that leaves...who looking after end users? Debian, Mandrake?
From what I've read here, stable Debian, it seems, is in the 2.4 series.
I don't know what Mandrake is up to, but I don't want to have to be
jumping distros because my distro maker has screwed up the kernel with
one of their patches. I also wouldn't want to give up reporting
kernel bugs directly to developers as I would if I am using a non-vanilla,
or worse, some tainted module.
However, all that being said, there would still be the choosing of
someone, steady and capable, of holding on to the stable release and
being it's gate-keeper. It seems like it would become quite a chore
to decide what code is let into the stable version. It's also
considered by many to be "less" fun, not only to "manage the
stable distro", but backport code into the previous distro.
Maybe no one _qualified_, wanted to manage a stable release.
It takes time and possibly enough time to qualify as a
full-time job. It takes a special person to find gainful
employment as a vendor-neutral kernel maintainer. The alternative is
to try to work 2 jobs where, in programming, each job might "like"
to have 60-80 hours of attention per week. That's a demanding
sacrifice as well.
It may be the case that no one at the last closed door kernel developer
meeting wanted to undertake the care of a stable kernel. No
volunteers...no kernel. There is less "wiggle room" in the average,
mature, developer's schedule with the advent of easy outsourcing to
cheaper labor that doesn't come from societies that breed independence
and nurture talented, more mature, or eccentric developers that love
spending spare cycles working on Open Source code.
Nevertheless, it would be nice to see a no-new-features, stable series
spun off from these development kernels, maybe .4th number releases,
like 2.6.10 also becomes a 2.6.10.0 that starts a 2.6.10.1, then 2.6.10.2,
etc...with iteritive bug fixes to the same kernel and no new features
in such a branch, it might become stable enough for users to have confidence
installing them on their desktop or stable machines.
It wouldn't have to be months upon months of diverging code, as jumps
to a new stable base can be started upon any fairly stable development
kernel, say 2.6.10 w/e100 fixed, tracing fixed, the slab bug fix, and
the capabilities bugs fixed going into a 2.6.10.1 that has no new features
or old features removed. Serious bug fixes after that could go into a
2.6.10.2, etc. Such point releases would be easier to manage and only
be updated/maintained as long as someone was interested enough to do it.
The same process would be applied to a future dev-kernel that appears to be
mostly stable after some number of weeks of alpha testing. It may be
the case that a given furture dev-kernel has no stable branch off of it
because it either a) didn't need one, or b) was too far from stable to start
one.
Anyway, just a thought for having something of the old with out as much
of a headache of kernels that diverge for a year or more before getting
sync'ed up.
-l
:-)
Dr. David Alan Gilbert wrote:
>For me, as someone who very rarely actually changes any code in
>the kernel, I have always found the stable series useful for
>two reasons:
>
> 1) It encourages me to test the kernel; if I have a kernel
> that is generally thought to be stable then I will try it on
> my home machine and report problems - this lets the kernel
> get tested on a wide range of hardware and situations; if there
> is no kernel that is liable to be stable changes will get much
> less testing on a smaller range of hardware.
>
> 2) If I have a bug in a vendor kernel everyone just tells
> me to go and speak to the vendor - so at least having a stable
> base to go back to can let me report a bug that isn't due
> to any vendors patches.
>
> 3) In some cases the commercial vendors don't seem to release
> source to some of the kernels except to people who have bought
> the packages, so those vendor kernel fixes aren't 'publically'
> visible.
>
>I think (1) is very important - getting large numbers of people
>to test OSS is its greatest asset.
>
>
Steven Rostedt
> On Mon, 3 Jan 2005 01:38:58 +0100, Adrian Bunk <bu...@stusta.de> wrote:
>
> > 2.4.27 -> 2.4.28 is a kernel upgrade that is very unlikely to cause
> > problems.
> >
> > Compared to this, 2.6.9 -> 2.6.10 is much more likely to break an
> > existing setup that worked in 2.6.9 .
Yes, it broke my NVIDIA module. I had to hack it to get it to work. Yes,
yes, I know NVIDIA bad and all that, but it is an example, and those
that have NVIDIA cards and want 3D graphics, need to bow to the evil
power that is.
>
> IIRC 2.4.9 -> 2.4.10 broke a few setups as well.
>
IIRC, there was a big argument to what was going on in the 2.4.9->2.4.10
kernel. Mainly the new VM. Alan Cox didn't want to include it because a
change like that was too big for a stable release. Actually, I thought
that 2.4.0 -> 2.4.14 was still unstable, and didn't migrate much on my
"stable" machines, until 2.4.14.
I think both approaches have their pros and cons. Maybe the problem is
that 2.x -> 2.x+1 is too slow. If it was faster, then it wouldn't be a
problem. The way I develop applications/libraries is that if I change an
interface, it changes the second number, and if I make major changes the
first is changed. Maybe keep 2.6.x just for bug fixes (like usual) and
start 2.7 for updates and jump quicker to 2.8. When a major design is
done, that should be 3.0. I believe that the kernel has settled with
the 2.6 series to not be jumping to something different as 2.4->2.6 did
any time soon. So maybe make 3.0 the next big change, and let the 2.x
rise quicker.
As to use the distribution kernels? People do that? The first thing I do
when installing a distribution, is to download and run the latests
kernel ;-)
-- Steve
Robert W. Fuller
since it's rare I get answers to my posts anyway....)
1. The distributors, such as Redhat, Mandrake, etc. ought to be
actively involved in stabilizing the kernel especially if they offer
kernel support services. (This isn't meant to imply that they aren't
currently doing so. After all, they employ a number of people who work
on the kernel.)
2. There is nothing to prevent the distributors from pooling their
resources and funding a small group of developers to maintain a "stable"
branch as their fulltime job.
3. If progress is to be made in the development model for Linux, then
people need to be less reactionary. In other words, don't criticize
changes in the development model unless you have a suggestion for
progressing the model.
L. A. Walsh wrote:
*omitted*
*more omitted*
William Lee Irwin III
>> They're superseded by iptables and their sole uses are by the Linux-
>> specific applications to manipulate this privileged aspect of system
>> state. This makes a much weaker case for backward compatibility than
>> general nonprivileged standardized system calls.
On Mon, Jan 03, 2005 at 06:33:04AM +0100, Willy Tarreau wrote:
> That's not the problem. There was a feature freeze by which
> everything which was considered hard to maintain or not very stable
> should have been removed. When 2.6 was announced, it was with a set
> of features. Who know, perhaps there are a few people who could
> replace a kernel 2.0 by a 2.6 on some firewalls. Even if they are
> only 2 or 3 people, there is no reason that suddenly a feature should
> be removed in the stable series. But it should be removed in 2.7 if
> it's a nightmare to maintain.
This is bizarre. iptables was made the de facto standard in 2.4.x and
the alternatives have issues with functionality. The 2.0/2.2 firewalling
interfaces are probably ready to go regardless. You do realize this is
what you're referring to?
2 major releases is long enough.
On Mon, Jan 03, 2005 at 06:33:04AM +0100, Willy Tarreau wrote:
> If the motivation to break backwards compatibility is not enough
> anymore to justify development kernels, I don't know what will
> justify it anymore. I'm particularly fed up by some developer's
> attitude who seem to never go outside and see how their creations are
> used by people who really trust the "stable" term... until they
> realize that this word is used only for marketting,
Who do you think is actually banging out the code on this mailing list?
Anyway, features aren't really allowed to break backward compatibility;
we've effectively got 10-year lifetimes for userspace-visible interfaces.
If this isn't good enough, well, tough.
On Mon, Jan 03, 2005 at 06:33:04AM +0100, Willy Tarreau wrote:
> eg. help distro makers announce their new major release at the right
> moment. ipfwadm had about 2 years to be removed before 2.6, wasn't
> that enough ? Once the stable release is out, the developer's point
> of view about how is creation *might* be used is not a justification
> to remove it. But of course, his difficulties at maintaining the code
> is fairly enough for him to say "well, it was a mistake to enable
> this, I don't want it in the future version anymore".
There isn't really any code behind ipfwadm, just a mocked-up interface.
I have little insight into what goes in in net/ to be honest. I do have
a firm notion that what goes on there is similar to the rest of the
kernel wrt. backward compatibility etc.
On Mon, Jan 03, 2005 at 06:33:04AM +0100, Willy Tarreau wrote:
> Why do you think that so many people are still using 2.4 (and even
> older versions) ? This is because they are the only ones who don't
> constantly change under your feet and from which you can build
> something reliable and guaranteed maintainable. At least, I've not
> seen any commercial product based on 2.6 yet !
> Please, stop constantly changing the contents of the "stable" kernel.
Either this is some kind of sick joke or you've never heard of SLES9.
On Sun, Jan 02, 2005 at 05:19:35PM -0800, William Lee Irwin III wrote:
>> The ``stable'' moniker and distros being based on 2.4 are horrors
>> beyond imagination and developers are pushed to in turn push
>> inappropriate features on 2.4.x and maintain them out-of-tree for
>> 2.4.x
On Mon, Jan 03, 2005 at 06:33:04AM +0100, Willy Tarreau wrote:
> I clearly don't agree with you, for a simple reason : those out-of-tree
> features will always be, because each distro likes to add a few features,
> like SquashFS, PaX, etc... And indeed, that's one of the reasons I *stay*
> on 2.4. It's so simple to simply upgrade the kernel, patch and recompile
> without spending days complaining "grrr... why did they change this ?".
> As soon as you have at least *ONE* patch to apply to a kernel for your
> distro, 2.4 is a safer bet than 2.6 if you don't want to restart everything
> at each minor release. The 2.4 kernel is more what I consider stable than
> 2.6, eventhough it's not totally. 2.0 and 2.2 *are* stable, because I'm
> certain that every future releases will only be bugfixes and will touch
> only a few lines.
You have ignored my entire argument in favor of reiterating your own.
One more time, since this apparently needs to be repeated in a
condensed and/or simplified form.
(1) the "stable" kernels are actually buggier because no one's looking
(2) the creation of those feature patches for stable kernels has
detracted from the efforts needed to get them actually into
the kernel, and they're not going to exist for long
On Mon, Jan 03, 2005 at 06:33:04AM +0100, Willy Tarreau wrote:
> At the moment, the only "serious" use I've found for a 2.6 is a kexec-based
> bootloader for known hardware. I've already seen that maintaining it up to
> date is not simple, I wonder how distro people work with it... I wouldn't
> have to do their work right now.
People are already using it to run the databases their paychecks rely on.
Whatever else you had in mind can't be anticipated.
-- wli
Bill Davidsen
This is exactly the type of change I meant. Anyone who has put 2.6 on an
older distro is probably still using ipchains. I can't imagine anyone
still using ipfwadm, but why didn't it go away during the 2.5 phase,
when everyone would have said that it was expected behaviour.
And there have been repeated suggestions the cryptoloop go away, which
was one of the reasons to go to 2.6 in the first place. I spent a year
during 2.5 time convincing {company} that having laptops around without
crypto was a very bad thing, and that cryptoloop was far better even if
professionals could break the security, casual theves would be less
likely to do so. They are NOT going to redo the setup on every laptop to
use {something else}, they would ignore any future security issues in
thge kenrel because they can't send out a "boot this CD" new kernel upgrade.
What's next, ext2? jfs? Features should be added in a stable tree, not
deleted. "sometimes a bit harder" hardly sounds like a great reason to
break existing systems.
Can you give an example of some feature which had to be removed because
no progress could be made while it was present? Remember that I am not
advocating "no new features," nor is anyone else AFAIK, just no removed
features. Developers may have had multiple streams for new stuff, but
the argument that this is now cured is BS. We have (major) lines
of -mm -ck, -aa and -ac, just to name the ones I've tried in the
last 3-4 months, not to mention Nick Piggin patch sets which come
and go in -mm, and Reiser_N patches.
In other words, I don't buy that keeping features is holding people
back, nor that there aren't many parallel development lines of
new patches.
>
>
> My impression is that currently 2.4 doesn't take that much time of
> developers (except for Marcelo's), and that it's a quite usable and
> stable kernel.
--
bill davidsen <davi...@tmr.com>
CTO TMR Associates, Inc
Doing interesting things with small computers since 1979
Diego Calleja
bió:
> I clearly don't agree with you, for a simple reason : those out-of-tr=
ee
> features will always be, because each distro likes to add a few featu=
res,
> like SquashFS, PaX, etc... And indeed, that's one of the reasons I *s=
tay*
> on 2.4. It's so simple to simply upgrade the kernel, patch and recomp=
ile
> without spending days complaining "grrr... why did they change this ?=
".
2.6 will stop having small issues in each release until 2.7 is forked j=
ust
like 2.4 broke things until 2.5 was forked. The difference IMO
is that linux development now avoids things like the unstability which =
the
2.4.10 changes caused and things like the fs corruption bugs we saw in =
2.4
I fully agree with WLI that the 2.4 development model and the
backporting-mania created more problems than it solved, because in the =
real
world almost everybody uses what distros ship, and what distros ship is=
n't
kernel.org but heavily modified kernels, which means that the kernel.or=
g
was not really "well-tested" or it took much longer to become "well-tes=
ted"
because it wasn't really being used.
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel"=
Adrian Bunk
> El Mon, 3 Jan 2005 06:33:04 +0100 Willy Tarreau <wi...@w.ods.org> esc=
ribió:
>
> > I clearly don't agree with you, for a simple reason : those out-of-=
tree
> > features will always be, because each distro likes to add a few fea=
tures,
> > like SquashFS, PaX, etc... And indeed, that's one of the reasons I =
*stay*
> > on 2.4. It's so simple to simply upgrade the kernel, patch and reco=
mpile
> > without spending days complaining "grrr... why did they change this=
?".
>
>
> 2.6 will stop having small issues in each release until 2.7 is forked=
just
> like 2.4 broke things until 2.5 was forked. The difference IMO
h the
> 2.4.10 changes caused and things like the fs corruption bugs we saw i=
n 2.4
>...
The 2.6.9 -> 2.6.10 patch is 28 MB, and while the changes that went int=
o
2.4 were limited since the most invasive patches were postponed for 2.5=
,
now _all_ patches go into 2.6 .
Yes, -mm gives a bit more testing coverage, but it doesn't seem to be
enough for this vast amount of changes.
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
-
Adrian Bunk
>...
> 3. If progress is to be made in the development model for Linux, then
> people need to be less reactionary. In other words, don't criticize
> changes in the development model unless you have a suggestion for
> progressing the model.
You can compare the old development model with the new development
model, and if you think the progression is a regression it's not
reactionary but simply correct to ask for going back to the old model.
It's a bit like at the height of the "new economy" hype:
"Reactionary" people who didn't believe in the wonders of the
"new economiy" but bought "old economy" stocks (or no stocks at all)
lost less money.
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
-
Horst von Brand
> I don't know about #3 below, but #1 and #2 are certainly true.
> I always preferred to run a vanilla stable kernel as I did not
> trust the vendors' kernels because their patches were not as well
> eyed as the vanilla kernel. I prefer to compile a kernel for
> my specific machines, some of which are old and do better with a
> hand-configured kernel rather than a Microsoftian monolith that
> is compiled with all possible options as modules.
The generic kernel + modules is a nice convenience for those that don't
compile their own. And the modules technology has made the need for
custom-compiled kernels all but go away. It is a _huge_ step forward (yes,
I do remember the large collection of Slackware boot disks for all sorts of
weird setups).
> I have one old laptop that sound just doesn't work on no matter
> what the settings -- may be failed hardware, but darned if I can't
> seem to easily stop the loading of sound related modules as hardware
> is probed by automatic hardware probing on each bootup, and the loading
> of sound modules by GUI dependencies on a memory constrained system.
Qualifies as "need for custom-compied kernel". Or even just custom
configured GUI.
> With each new kernel release, I wonder if it will be satisfactory
> to use for a new, base-line, stable vanilla kernel, but post release
> comments seem to prove otherwise.
Only TeX is guaranteed bug-free.
> It seems that some developers have the opinion that the end-user base
> no longer is their problem or audience and that the distros will patch
> all the little boo-boo's in each unstable 2.6 release.
AFAIU, the current development model is designed to _diminish_ the need of
custom patching by distributions. For example, RH 9 2.4 kernels were mostly
2.6 via backports and random patches. But the patches were only maintained
by RH, so it was a large duplication of effort (not even counting the other
distributions). With 2.6 everybody can work on a up-to-date code base, much
less need of distribution backports and patches (and associated random
incompatibilities) benefits every user.
> Well, that's
> just plain asking for problems.
Quite to the contrary.
> Just in SuSE's previous release of
> 9.1, it wouldn't boot up, for update, on any system that used xfs
> disks. Redhat has officially dropped support for end-user distros,
> that leaves...who looking after end users? Debian, Mandrake?
> From what I've read here, stable Debian, it seems, is in the 2.4 series.
Stable Debian is 3 years old.
> I don't know what Mandrake is up to, but I don't want to have to be
> jumping distros because my distro maker has screwed up the kernel with
> one of their patches.
You could complain to the distribution maker (so every one of their users
benefits from your problem, that is the whole point of open source!), undo
the patch, run a vanilla kernel. No need to skip around (doing so is
probably much more work than just changing kernels).
> I also wouldn't want to give up reporting
> kernel bugs directly to developers as I would if I am using a non-vanilla,
> or worse, some tainted module.
If it is a distribution kernel, you should complain to them, they will
forward your complaint to the maintainers if it warrants doing so. Only if
vanilla kernel you get to swamp the maintainers directly.
> However, all that being said, there would still be the choosing of
> someone, steady and capable, of holding on to the stable release and
> being it's gate-keeper.
The people in charge decided otherwise, for sound reasons. If you don't
like it, you are free to create you own fork off 2.6.10 (or something) and
stabilize that. That is the wonder of open source...
> It seems like it would become quite a chore
> to decide what code is let into the stable version. It's also
> considered by many to be "less" fun, not only to "manage the
> stable distro", but backport code into the previous distro.
Lots of rather pointless work. Much of it something each distribution has
to do on their own (because f.ex. vanilla 2.4 is _just fixes_, no backports
of cool (and required) new functionality), instead of cooperating in
building a better overall kernel.
> Maybe no one _qualified_, wanted to manage a stable release.
> It takes time and possibly enough time to qualify as a
> full-time job. It takes a special person to find gainful
> employment as a vendor-neutral kernel maintainer. The alternative is
> to try to work 2 jobs where, in programming, each job might "like"
> to have 60-80 hours of attention per week. That's a demanding
> sacrifice as well.
Yep. That's why nobody (and that certainly includes you) is entitled to
demand such.
> It may be the case that no one at the last closed door kernel developer
> meeting wanted to undertake the care of a stable kernel.
Andrew Morton had been designated for the job (and he accepted).
> No
> volunteers...no kernel. There is less "wiggle room" in the average,
> mature, developer's schedule with the advent of easy outsourcing to
> cheaper labor that doesn't come from societies that breed independence
> and nurture talented, more mature, or eccentric developers that love
> spending spare cycles working on Open Source code.
English, please?
> Nevertheless, it would be nice to see a no-new-features, stable series
> spun off from these development kernels, maybe .4th number releases,
> like 2.6.10 also becomes a 2.6.10.0 that starts a 2.6.10.1, then 2.6.10.2,
> etc...with iteritive bug fixes to the same kernel and no new features
> in such a branch, it might become stable enough for users to have confidence
> installing them on their desktop or stable machines.
See above. The 2.6.9-x kernels from Red Hat/Fedora are targeted to be
exactly that...
> It wouldn't have to be months upon months of diverging code, as jumps
> to a new stable base can be started upon any fairly stable development
> kernel, say 2.6.10 w/e100 fixed, tracing fixed, the slab bug fix, and
> the capabilities bugs fixed going into a 2.6.10.1 that has no new features
> or old features removed. Serious bug fixes after that could go into a
> 2.6.10.2, etc. Such point releases would be easier to manage and only
> be updated/maintained as long as someone was interested enough to do it.
That is exactly the model! Just that no vanillla 2.6.9.1 has been needed
yet.
> The same process would be applied to a future dev-kernel that appears to be
> mostly stable after some number of weeks of alpha testing.
... conveniently forgetting that it is a experimental data point that
people start using the kernel (and finding its bugs) only when it leaves
"alpha" (by whatever name), so this doesn't work.
> It may be
> the case that a given furture dev-kernel has no stable branch off of it
> because it either a) didn't need one, or b) was too far from stable to start
> one.
(a) makes no sense, (b) is a mess that Linux has been able to avoid thus
far.
> Anyway, just a thought for having something of the old with out as much
> of a headache of kernels that diverge for a year or more before getting
> sync'ed up.
That's what you get with the new development model.
--
Dr. Horst H. von Brand User #22616 counter.li.org
Departamento de Informatica Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria +56 32 654239
Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513
Rik van Riel
> You change some stuff. The bad mistakes are discovered very soon.
> Some subtler things or some things that occur only in special
> configurations or under special conditions or just with
> very low probability may not be noticed until much later.
Some of these subtle bugs are only discovered a year
after the distribution with some particular kernel has
been deployed - at which point the kernel has moved on
so far that the fix the distro does might no longer
apply (even in concept) to the upstream kernel...
This is especially true when you are talking about really
big database servers and bugs that take weeks or months
to trigger.
--
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." - Brian W. Kernighan
Rik van Riel
> The main advantage with stable kernels in the good old days (tm) when 4
> Nowadays in 2.6, every new 2.6 kernel has several regressions compared
> to the previous one, and additionally obsolete but used code like
2.2 before 2.2.20 also had this kind of problem, as did
the 2.4 kernel before 2.4.20 or thereabouts.
I'm pretty sure 2.6 is actually doing better than the
early 2.0, 2.2 and 2.4 kernels...
--
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." - Brian W. Kernighan
Adrian Bunk
> On Sun, 2 Jan 2005, Adrian Bunk wrote:
>
> >The main advantage with stable kernels in the good old days (tm) when 4
>
> >Nowadays in 2.6, every new 2.6 kernel has several regressions compared
> >to the previous one, and additionally obsolete but used code like
>
> 2.2 before 2.2.20 also had this kind of problem, as did
> the 2.4 kernel before 2.4.20 or thereabouts.
>
> I'm pretty sure 2.6 is actually doing better than the
> early 2.0, 2.2 and 2.4 kernels...
My personal impression was that even the 2.6.0-test kernels were much
better than the 2.4.0-test kernels.
But 2.6.20 will most likely still have the stability of the early
2.6 kernels instead of a greatly increased stability as observed in
2.2.20 and 2.4.20 .
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
-
Adrian Bunk
> On Sun, 2 Jan 2005, Andries Brouwer wrote:
>
> >You change some stuff. The bad mistakes are discovered very soon.
> >Some subtler things or some things that occur only in special
> >configurations or under special conditions or just with
> >very low probability may not be noticed until much later.
>
> Some of these subtle bugs are only discovered a year
> after the distribution with some particular kernel has
> been deployed - at which point the kernel has moved on
> so far that the fix the distro does might no longer
> apply (even in concept) to the upstream kernel...
>
> This is especially true when you are talking about really
> big database servers and bugs that take weeks or months
> to trigger.
If at this time 2.8 was already released, the 2.8 kernel available at
this time will be roughly what 2.6 would have been under the current
development model, and 2.6 will be a rock stable kernel.
If it was possible to get the 2.7 cycle pretty short, this would give
the advantages of the old development model without most of its'
disadvantages.
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
-
William Lee Irwin III
>> 2.2 before 2.2.20 also had this kind of problem, as did
>> the 2.4 kernel before 2.4.20 or thereabouts.
>> I'm pretty sure 2.6 is actually doing better than the
>> early 2.0, 2.2 and 2.4 kernels...
On Mon, Jan 03, 2005 at 04:29:53PM +0100, Adrian Bunk wrote:
> My personal impression was that even the 2.6.0-test kernels were much
> better than the 2.4.0-test kernels.
> But 2.6.20 will most likely still have the stability of the early
> 2.6 kernels instead of a greatly increased stability as observed in
> 2.2.20 and 2.4.20 .
This is speculation; there is no reason not to expect the process to
converge to as great of stability or greater stability than the
2.4-style process. I specuate that it will in fact do precisely that.
-- wli
William Lee Irwin III
>> This is especially true when you are talking about really
>> big database servers and bugs that take weeks or months
>> to trigger.
On Mon, Jan 03, 2005 at 04:34:38PM +0100, Adrian Bunk wrote:
> If at this time 2.8 was already released, the 2.8 kernel available at
> this time will be roughly what 2.6 would have been under the current
> development model, and 2.6 will be a rock stable kernel.
> If it was possible to get the 2.7 cycle pretty short, this would give
> the advantages of the old development model without most of its'
> disadvantages.
But that cannot be. Splitting the developer base is guaranteed to
reduce the amount of critical examination and testing given to both
series of kernel versions.
Also, .com's have finite horizons and slow response times; dev kernels
are almost universally beyond them. A dedicated dev kernel with a short
development cycle guarantees that the entire corporate side will be
left out of the development cycle. And this is not speculation; even
the long dev cycles do similar, or are only given attention after their
huge freezes. If they're always too late for a slow-moving dev cycle a
fast-moving dev cycle is guaranteed to outrun them completely.
-- wli
Arjan van de Ven
> On Mon, Jan 03, 2005 at 10:18:47AM -0500, Rik van Riel wrote:
> > On Sun, 2 Jan 2005, Andries Brouwer wrote:
> >
> > >You change some stuff. The bad mistakes are discovered very soon.
> > >Some subtler things or some things that occur only in special
> > >configurations or under special conditions or just with
> > >very low probability may not be noticed until much later.
> >
> > Some of these subtle bugs are only discovered a year
> > after the distribution with some particular kernel has
> > been deployed - at which point the kernel has moved on
> > so far that the fix the distro does might no longer
> > apply (even in concept) to the upstream kernel...
> >
> > This is especially true when you are talking about really
> > big database servers and bugs that take weeks or months
> > to trigger.
>
> If at this time 2.8 was already released, the 2.8 kernel available at
> this time will be roughly what 2.6 would have been under the current
> development model, and 2.6 will be a rock stable kernel.
as long as more things get fixed than new bugs introduced (and that
still seems to be the case) things only improve in 2.6.
The joint approach also has major advantages, even for quality:
All testing happens on the same codebase.
Previously, the testing focus was split between the stable and unstable
branch, to the detriment of *both*.
Alan Cox
need for testing. It is one factor but the amount of review, the
modularity of the code and the effectiveness of the management and
verification tools are all involved greatly.
Jeff V. Merkey
Rate of change X 3 = Rate of testing.
Seems to apply in commerical software development, provided the testing
engineers are brighter than
the developers (which in most cases they need to be).
:-)
Jeff
Felipe Alfaro Solana
> On Mon, Jan 03, 2005 at 10:20:40AM -0500, Rik van Riel wrote:
>>> 2.2 before 2.2.20 also had this kind of problem, as did
>>> the 2.4 kernel before 2.4.20 or thereabouts.
>>> I'm pretty sure 2.6 is actually doing better than the
>>> early 2.0, 2.2 and 2.4 kernels...
>
> On Mon, Jan 03, 2005 at 04:29:53PM +0100, Adrian Bunk wrote:
>> My personal impression was that even the 2.6.0-test kernels were much
>> better than the 2.4.0-test kernels.
>> But 2.6.20 will most likely still have the stability of the early
>> 2.6 kernels instead of a greatly increased stability as observed in
>> 2.2.20 and 2.4.20 .
>
> This is speculation; there is no reason not to expect the process to
> converge to as great of stability or greater stability than the
> 2.4-style process. I specuate that it will in fact do precisely that.
I would like to comment in that the issue is not exclusively targeted
to stability, but the ability to keep up with kernel development. For
example, it was pretty common for older versions of VMWare and NVidia
driver to break up whenever a new kernel version was released.
I think it's a PITA for developers to rework some of the closed-source
code to adopt the new changes in the Linux kernel.
Bill Davidsen
> On Mon, Jan 03, 2005 at 02:24:12PM +0100, Diego Calleja wrote:
> > El Mon, 3 Jan 2005 06:33:04 +0100 Willy Tarreau <wi...@w.ods.org> e=
scribió:
> >
> > > I clearly don't agree with you, for a simple reason : those out-o=
f-tree
> > > features will always be, because each distro likes to add a few f=
eatures,
> > > like SquashFS, PaX, etc... And indeed, that's one of the reasons =
I *stay*
> > > on 2.4. It's so simple to simply upgrade the kernel, patch and re=
compile
> > > without spending days complaining "grrr... why did they change th=
is ?".
> >
> >
> > 2.6 will stop having small issues in each release until 2.7 is fork=
ed just
> > like 2.4 broke things until 2.5 was forked. The difference IMO
> > is that linux development now avoids things like the unstability wh=
ich the
> > 2.4.10 changes caused and things like the fs corruption bugs we saw=
in 2.4
> >...
>
> The 2.6.9 -> 2.6.10 patch is 28 MB, and while the changes that went i=
nto
> 2.4 were limited since the most invasive patches were postponed for 2=
.5,
> now _all_ patches go into 2.6 .
>
> Yes, -mm gives a bit more testing coverage, but it doesn't seem to be=
> enough for this vast amount of changes.
I have to say that with a few minor exceptions the introduction of new
features hasn't created long term (more than a few days) of problems. A=
nd
we have had that in previous stable versions as well. New features
themselves may not be totally stable, but in most cases they don't brea=
k
existing features, or are fixed in bk1 or bk2. What worries me is remov=
ing
features deliberately, and I won't beat that dead horse again, I've sai=
d
my piece.
The "few minor exceptions:"
SCSI command filtering - while I totally support the idea (and always
have), I miss running cdrecord as a normal user. Multisession doesn't w=
ork
as a normal user (at least if you follow the man page) because only roo=
t
can use -msinfo. There's also some raw mode which got a permission deni=
ed,
don't remember as I was trying something not doing production stuff.
APM vs. ACPI - shutdown doesn't reliably power down about half of the
machines I use, and all five laptops have working suspend and non-worki=
ng
resume. APM seems to be pretty unsupported beyond "use ACPI for that."
None of these would prevent using 2.6 if there were some feature not in
2.4 which gave a reason to switch.
--
bill davidsen <davi...@tmr.com>
CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.
Adrian Bunk
>...
> The "few minor exceptions:"
>
> SCSI command filtering - while I totally support the idea (and always
> as a normal user (at least if you follow the man page) because only root
> can use -msinfo. There's also some raw mode which got a permission denied,
> don't remember as I was trying something not doing production stuff.
>
> APM vs. ACPI - shutdown doesn't reliably power down about half of the
> resume. APM seems to be pretty unsupported beyond "use ACPI for that."
More serious were other problems like e.g. the problems XFS has (had?)
with the 4kB stacks option on i386 that was introduced in 2.6
after 2.6.0 .
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
Wakko Warner
>
> My personal impression was that even the 2.6.0-test kernels were much
> better than the 2.4.0-test kernels.
>
> But 2.6.20 will most likely still have the stability of the early
> 2.6 kernels instead of a greatly increased stability as observed in
> 2.2.20 and 2.4.20 .
In my experiences, 2.6.8 and above have been quite unstable for me. I was
able to crash 2.6.8.1 as a normal user over nfs (I thought that was fixed?)
2.6.9 gave me problems with USB and random lockups. 2.6.7 has been fairly
stable for me, but I honestly do not see 2.6 as a stable kernel. From my
past experiences, 2.4.x (low numbered) was just as stable as 2.6 has been or
more so.
--
Lab tests show that use of micro$oft causes cancer in lab animals
Theodore Ts'o
> we have had that in previous stable versions as well. New features
> existing features, or are fixed in bk1 or bk2. What worries me is removing
> features deliberately, and I won't beat that dead horse again, I've said
> my piece.
Indeed. Part of the problem is that we don't get that much testing
with the rc* releases, so there are a lot of problems that don't get
noticed until after 2.6.x ships. This has been true for both 2.6.9
and 2.6.10. My personal practice is to never run with 2.6.x release,
but wait for 2.6.x plus one or 2 days (i.e. bk1 or bk2). The problems
with this approach are that (1) out-of-tree patches against official
versions of the kernel (i.e., things like the mppc/mppe patch) don't
necessarly apply cleanly, and (2) other more destablizing patches get
folded in right after 2.6.x ships, so there is a chance bk1 or bk2 may
not be stable.
We could delay the destablizing changes until after rc1 ships, and
ship rc1 about 2-3 days after 2.6.x is released, so that the really
obvious/critical regressions can be addressed immediately. The
problem with this approach though is that some people will just wait
until rc1 ships before they start using a new kernel version, and we
lose the testing we need to stablize the release.
The real key, as always, is getting users to download and test a
release. So another approach might be to shorten the time between
2.6.x and 2.6.x+1 releases, so as to recreate more testing points,
without training people to wait for -bk1, -bk2, -rc1, etc. before
trying out the kernel code. This is the model that we used with the
2.3.x series, where the time between releases was often quite short.
That worked fairly well, but we stopped doing it when the introduction
of BitKeeper eliminated the developer synch-up problem. But perhaps
we've gone too far between 2.6.x releases, and should shorten the time
in order to force more testing.
- Ted
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
Russell King
> This is the model that we used with the
> 2.3.x series, where the time between releases was often quite short.
> That worked fairly well, but we stopped doing it when the introduction
> of BitKeeper eliminated the developer synch-up problem. But perhaps
> we've gone too far between 2.6.x releases, and should shorten the time
> in order to force more testing.
It is also the model we used until OLS this year - there was a 2.6
release about once a month prior to OLS. Post OLS, it's now once
every three months or there abouts, which, IMO is far too long.
I really liked the way pre-OLS 2.6 was working... it means I don't
have to twiddle my fingers getting completely bored waiting for the
next 2.6 release to happen. Can we return to that methodology please?
--
Russell King
Linux kernel 2.6 ARM Linux - http://www.arm.linux.org.uk/
maintainer of: 2.6 PCMCIA - http://pcmcia.arm.linux.org.uk/
2.6 Serial core
Bill Davidsen
> On Mon, Jan 03, 2005 at 12:18:36PM -0500, Bill Davidsen wrote:
> >...
> > The "few minor exceptions:"
> >
> > SCSI command filtering - while I totally support the idea (and always
> > have), I miss running cdrecord as a normal user. Multisession doesn't work
> > as a normal user (at least if you follow the man page) because only root
> > can use -msinfo. There's also some raw mode which got a permission denied,
> > don't remember as I was trying something not doing production stuff.
> >
> > APM vs. ACPI - shutdown doesn't reliably power down about half of the
> > machines I use, and all five laptops have working suspend and non-working
> > resume. APM seems to be pretty unsupported beyond "use ACPI for that."
> >...
>
> More serious were other problems like e.g. the problems XFS has (had?)
> with the 4kB stacks option on i386 that was introduced in 2.6
> after 2.6.0 .
Can't speak for XFS, but the 4k stack issue was an option, and could be
investigated with care. I only found one case where 4k would repeatably
cause a problem, and the fix was in the -bk before I had a decent test
case. I am going to try XFS in a month or so, I have a chance to bench JFS
vs. XFS for an application with 40-50k files in a directory. Hope it works
by then if it doesn't now, I'm told BSD works well.
--
bill davidsen <davi...@tmr.com>
CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.
-
William Lee Irwin III
>> This is the model that we used with the
>> 2.3.x series, where the time between releases was often quite short.
>> That worked fairly well, but we stopped doing it when the introduction
>> of BitKeeper eliminated the developer synch-up problem. But perhaps
>> we've gone too far between 2.6.x releases, and should shorten the time
>> in order to force more testing.
On Mon, Jan 03, 2005 at 06:59:27PM +0000, Russell King wrote:
> It is also the model we used until OLS this year - there was a 2.6
> release about once a month prior to OLS. Post OLS, it's now once
> every three months or there abouts, which, IMO is far too long.
> I really liked the way pre-OLS 2.6 was working... it means I don't
> have to twiddle my fingers getting completely bored waiting for the
> next 2.6 release to happen. Can we return to that methodology please?
Seconded.
-- wli
Jens Axboe
> SCSI command filtering - while I totally support the idea (and always
> as a normal user (at least if you follow the man page) because only root
> can use -msinfo. There's also some raw mode which got a permission denied,
> don't remember as I was trying something not doing production stuff.
So look at dmesg, the kernel will dump the failed command. Send the
result here and we can add that command, done deal. 2.6.10 will do this
by default.
--
Jens Axboe
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
Randy.Dunlap
> On Mon, Jan 03, 2005 at 01:36:21PM -0500, Theodore Ts'o wrote:
>
>>This is the model that we used with the
>>2.3.x series, where the time between releases was often quite short.
>>That worked fairly well, but we stopped doing it when the introduction
>>of BitKeeper eliminated the developer synch-up problem. But perhaps
>>we've gone too far between 2.6.x releases, and should shorten the time
>>in order to force more testing.
>
>
> It is also the model we used until OLS this year - there was a 2.6
> release about once a month prior to OLS. Post OLS, it's now once
> every three months or there abouts, which, IMO is far too long.
>
> I really liked the way pre-OLS 2.6 was working... it means I don't
> have to twiddle my fingers getting completely bored waiting for the
> next 2.6 release to happen. Can we return to that methodology please?
Agreed. We (whoever "we" are) have erred too much on longer
cycles for stability, but it's not working out as hoped IMO.
--
~Randy
Horst von Brand
[...]
> I have to say that with a few minor exceptions the introduction of new
> features hasn't created long term (more than a few days) of problems. And
> we have had that in previous stable versions as well. New features
> themselves may not be totally stable, but in most cases they don't break
> existing features, or are fixed in bk1 or bk2. What worries me is removing
> features deliberately, and I won't beat that dead horse again, I've said
> my piece.
>
> The "few minor exceptions:"
>
> SCSI command filtering - while I totally support the idea (and always
> have), I miss running cdrecord as a normal user. Multisession doesn't work
> as a normal user (at least if you follow the man page) because only root
> can use -msinfo. There's also some raw mode which got a permission denied,
> don't remember as I was trying something not doing production stuff.
It had very nasty security problems. After a short discussion here, it was
deemed much more important to have a secure system than a (very minor)
convenience. AFAIU, the patch was backported to 2.4 (or should be ASAP).
> APM vs. ACPI - shutdown doesn't reliably power down about half of the
> machines I use, and all five laptops have working suspend and non-working
> resume. APM seems to be pretty unsupported beyond "use ACPI for that."
Many never machines just don't have APM.
> None of these would prevent using 2.6 if there were some feature not in
> 2.4 which gave a reason to switch.
Like 2.6 works fine, 2.4 has no chance on some machines?
--
Dr. Horst H. von Brand User #22616 counter.li.org
Departamento de Informatica Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria +56 32 654239
Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
Horst von Brand
[...]
> I would like to comment in that the issue is not exclusively targeted
> to stability, but the ability to keep up with kernel development. For
> example, it was pretty common for older versions of VMWare and NVidia
> driver to break up whenever a new kernel version was released.
That is the price for closed-source drivers.
> I think it's a PITA for developers to rework some of the closed-source
> code to adopt the new changes in the Linux kernel.
Open up the code. Most of the changes will then be done as a matter of
course by others.
--
Dr. Horst H. von Brand User #22616 counter.li.org
Departamento de Informatica Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria +56 32 654239
Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513
Horst von Brand
[...]
> The real key, as always, is getting users to download and test a
> release. So another approach might be to shorten the time between
> 2.6.x and 2.6.x+1 releases, so as to recreate more testing points,
> without training people to wait for -bk1, -bk2, -rc1, etc. before
> trying out the kernel code. This is the model that we used with the
> 2.3.x series, where the time between releases was often quite short.
> That worked fairly well, but we stopped doing it when the introduction
> of BitKeeper eliminated the developer synch-up problem. But perhaps
> we've gone too far between 2.6.x releases, and should shorten the time
> in order to force more testing.
Is there any estimate of the number of daily-straight-from-BK users? I'm
one, haven't seen any trouble (thus silent up to here).
--
Dr. Horst H. von Brand User #22616 counter.li.org
Departamento de Informatica Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria +56 32 654239
Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513
Jesper Juhl
> "Theodore Ts'o" <ty...@mit.edu> said:
>
> [...]
>
> > The real key, as always, is getting users to download and test a
> > release. So another approach might be to shorten the time between
> > 2.6.x and 2.6.x+1 releases, so as to recreate more testing points,
> > without training people to wait for -bk1, -bk2, -rc1, etc. before
> > trying out the kernel code. This is the model that we used with the
> > 2.3.x series, where the time between releases was often quite short.
> > That worked fairly well, but we stopped doing it when the introduction
> > of BitKeeper eliminated the developer synch-up problem. But perhaps
> > we've gone too far between 2.6.x releases, and should shorten the time
> > in order to force more testing.
>
> Is there any estimate of the number of daily-straight-from-BK users? I'm
> one, haven't seen any trouble (thus silent up to here).
I'm another. Every morning when I turn on my machine I grab the latest
-bk, build it with my usual config, install that kernel and reboot, then
use that as my "kernel of the day". I do this on both my home and work
box (well, the work box only does this on mondays) and I've had very
little trouble so far.
--
Jesper Juhl
Willy Tarreau
> This is bizarre. iptables was made the de facto standard in 2.4.x and
> the alternatives have issues with functionality. The 2.0/2.2 firewalling
> interfaces are probably ready to go regardless. You do realize this is
> what you're referring to?
>
> 2 major releases is long enough.
if it's long enough, ipfwadm should not have entered 2.6 at all. It's not
because you don't see any use to that particular feature that you can guarantee
that it is not used at all. At least, a large public call to get in touch with
the potentially unique user of this feature would be a start, but generally we
should not remove a feature from a stable kernel. What will go next ? minix,
because someone will decide that there have been many better filesystems for a
long time, so that's long enough ? Revert modules to modutils because someone
will think it's simpler for everyone to use a single toolset ? I have no
problem removing numerous feature between major releases, even breaking APIs,
but I really hate it when something which is called "stable" constantly
changes.
> On Mon, Jan 03, 2005 at 06:33:04AM +0100, Willy Tarreau wrote:
> > If the motivation to break backwards compatibility is not enough
> > anymore to justify development kernels, I don't know what will
> > justify it anymore. I'm particularly fed up by some developer's
> > attitude who seem to never go outside and see how their creations are
> > used by people who really trust the "stable" term... until they
> > realize that this word is used only for marketting,
>
> Who do you think is actually banging out the code on this mailing list?
Frankly, sometimes I'm really wondering. We see lots of very clever ideas,
and sometimes people come up with concepts which can break existing apps, and
simply justify by "this should not have been done in the first time." (eg:
unexport syscall_table). But I'm certain that all these mistakes are caused
by those too long development cycles. Some developpers get bored by things that
irritate them, and prefer to fix the stable tree to stop what they believe is
an error, instead of waiting for the next release to fix it there.
> Anyway, features aren't really allowed to break backward compatibility;
> we've effectively got 10-year lifetimes for userspace-visible interfaces.
> If this isn't good enough, well, tough.
All in all, I agree with you. The small differences lie in /proc files or
oops syntax, etc... But even old syscalls are still supported and that's fine.
I appreciate it when I read the packet(7) man page to find that even the
interface from linux 2.0 is still supported.
The problem is that nowadays, the userspace-visible code is not only in
userspace anymore, but also involves modules interfaces sometimes because
some commercial apps rely on modules (firewalls, virtual machines, etc...),
and their userspace is nuts without those modules, so in a certain way,
breaking some kernel internals within a stable release does break some apps.
> On Mon, Jan 03, 2005 at 06:33:04AM +0100, Willy Tarreau wrote:
> > Why do you think that so many people are still using 2.4 (and even
> > older versions) ? This is because they are the only ones who don't
> > constantly change under your feet and from which you can build
> > something reliable and guaranteed maintainable. At least, I've not
> > seen any commercial product based on 2.6 yet !
> > Please, stop constantly changing the contents of the "stable" kernel.
>
> Either this is some kind of sick joke or you've never heard of SLES9.
the later :-)
> On Sun, Jan 02, 2005 at 05:19:35PM -0800, William Lee Irwin III wrote:
(...)
> You have ignored my entire argument in favor of reiterating your own.
>
> One more time, since this apparently needs to be repeated in a
> condensed and/or simplified form.
>
> (1) the "stable" kernels are actually buggier because no one's looking
I don't agree with you. "known" bugs become features of this particular
release and people learn how to play with them. The MM beahaviour when one
single user can crash the whole machine just accidentely playing with malloc()
would be called a bug on any other decent OS. For us it's a feature we have
been used to live with.
It's possible that 2.6 has fewer of those known bugs, but it still has many
yet-to-discover bugs (the first ones being all those 'my machine does not
boot anymore' reported here and caused by those too long release cycles).
> (2) the creation of those feature patches for stable kernels has
> detracted from the efforts needed to get them actually into
> the kernel, and they're not going to exist for long
I agree with you on the first part, but not on the second one, because as a
stable kernel implies it, it will still be possible to apply current patches
to new releases with very few efforts. Indeed, I have already sent rediffed
patches to different maintainers because they were easy to do. For a while
now, on 2.4, you can easily apply jiffies64, epoll, netdev-random, preempt,
lowlat, bme, squashfs, tux, etc... The list is long and demonstrantes what
stable code looks like. "stable" does not mean it will not crash, but it means
"it will not change much", eventhough this tends to imply the former.
> On Mon, Jan 03, 2005 at 06:33:04AM +0100, Willy Tarreau wrote:
> > At the moment, the only "serious" use I've found for a 2.6 is a kexec-based
> > bootloader for known hardware. I've already seen that maintaining it up to
> > date is not simple, I wonder how distro people work with it... I wouldn't
> > have to do their work right now.
>
> People are already using it to run the databases their paychecks rely on.
I feel they're brave. I know several other people who went back, either because
they didn't feel comfortable with upgrades these size, which sometimes did not
boot because of random patches, or simply because of the scheduler which didn't
let them type normally in an SSH session on a CPU-bound system, or even a
proxy which performance dropped by a factor of 5 between 2.4 and 2.6. I know
they don't report it, but they are not developpers. They see that 2.6 is not
ready yet, and turn back to stable 2.4.
> Whatever else you had in mind can't be anticipated.
agreed.
Willy
Felipe Alfaro Solana
> Felipe Alfaro Solana <lk...@mac.com> said:
>
> [...]
>
>> I would like to comment in that the issue is not exclusively targeted
>> to stability, but the ability to keep up with kernel development. For
>> example, it was pretty common for older versions of VMWare and NVidia
>> driver to break up whenever a new kernel version was released.
>
> That is the price for closed-source drivers.
>
>> I think it's a PITA for developers to rework some of the closed-source
>> code to adopt the new changes in the Linux kernel.
>
> Open up the code. Most of the changes will then be done as a matter of
> course by others.
Unfortunately, you can't force the entire hardware industry to open up
their drivers.
Rik van Riel
> On 3 Jan 2005, at 21:59, Horst von Brand wrote:
>> Open up the code. Most of the changes will then be done as a matter of
>> course by others.
>
> Unfortunately, you can't force the entire hardware industry to open up
> their drivers.
That's ok. I don't have to buy that hardware.
--
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." - Brian W. Kernighan
Sean
> On 3 Jan 2005, at 21:59, Horst von Brand wrote:
>
>> Felipe Alfaro Solana <lk...@mac.com> said:
>>
>> [...]
>>
>>> I would like to comment in that the issue is not exclusively targeted
>>> to stability, but the ability to keep up with kernel development. For
>>> example, it was pretty common for older versions of VMWare and NVidia
>>> driver to break up whenever a new kernel version was released.
>>
>> That is the price for closed-source drivers.
>>
>>> I think it's a PITA for developers to rework some of the closed-source
>>> code to adopt the new changes in the Linux kernel.
>>
>> Open up the code. Most of the changes will then be done as a matter of
>> course by others.
>
> Unfortunately, you can't force the entire hardware industry to open up
> their drivers.
>
The point is, they'll have to deal with the burrden of any extra work
created as a result. It's not the responsibility of the open source
developers. Smart users pick hardware that is well supported by Linux
and doesn't run the risk of becoming obsolete the day the manufacturer
decides they don't want to provide drivers any longer.
Sean
Felipe Alfaro Solana
> On Mon, 3 Jan 2005, Felipe Alfaro Solana wrote:
>> On 3 Jan 2005, at 21:59, Horst von Brand wrote:
>
>>> Open up the code. Most of the changes will then be done as a matter
>>> of
>>> course by others.
>>
>> Unfortunately, you can't force the entire hardware industry to open
>> up their drivers.
>
> That's ok. I don't have to buy that hardware.
Gosh! I bought an ATI video card, I bought a VMware license, etc.... I
want to keep using them. Changing a "stable" kernel will continuously
annoy users and vendors.
I think new developments will force a 2.7 branch: when 2.6 feature set
stabilizes, people will keep more time testing a stable, relatively
static kernel base, finding bugs, instead of trying to keep up with
changes.
Rik van Riel
> On 3 Jan 2005, at 22:48, Rik van Riel wrote:
>> On Mon, 3 Jan 2005, Felipe Alfaro Solana wrote:
>>> Unfortunately, you can't force the entire hardware industry to open up
>>> their drivers.
>>
>> That's ok. I don't have to buy that hardware.
>
> Gosh! I bought an ATI video card, I bought a VMware license, etc.... I
> want to keep using them.
That's your choice, and you (and your vendors) will have to
deal with the issues. I don't see why we should hold back
the development of Linux for it...
> Changing a "stable" kernel will continuously annoy users and vendors.
On the other hand, not having a feature users need available
in a stable kernel also annoys users and vendors. During the
2.4 days this lead to every distribution needing to do a bunch
of backports from the 2.5 kernel, and the availability of 2.4
kernels with every possible subset of 2.5 features - but none
with all the features. Arguably, that is a much worse situation
for users and vendors than a faster-changing 2.6 tree, where at
least everybody is using the same tree.
You can't have your cake and eat it, too.
--
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." - Brian W. Kernighan
William Lee Irwin III
>> This is bizarre. iptables was made the de facto standard in 2.4.x and
>> the alternatives have issues with functionality. The 2.0/2.2 firewalling
>> interfaces are probably ready to go regardless. You do realize this is
>> what you're referring to?
>> 2 major releases is long enough.
On Mon, Jan 03, 2005 at 10:38:45PM +0100, Willy Tarreau wrote:
> if it's long enough, ipfwadm should not have entered 2.6 at all. It's
> not because you don't see any use to that particular feature that you
> can guarantee that it is not used at all. At least, a large public
> call to get in touch with the potentially unique user of this feature
> would be a start, but generally we should not remove a feature from a
> stable kernel. What will go next ? minix, because someone will decide
> that there have been many better filesystems for a long time, so
> that's long enough ? Revert modules to modutils because someone
> will think it's simpler for everyone to use a single toolset ? I have
> no problem removing numerous feature between major releases, even
> breaking APIs, but I really hate it when something which is called
> "stable" constantly changes.
Unfortunately you're never going to find a rock to build your house
on. there is no rock, there is only shifting sand.
On Mon, Jan 03, 2005 at 04:33:25AM -0800, William Lee Irwin III wrote:
>> Who do you think is actually banging out the code on this mailing list?
On Mon, Jan 03, 2005 at 10:38:45PM +0100, Willy Tarreau wrote:
> Frankly, sometimes I'm really wondering. We see lots of very clever
> ideas, and sometimes people come up with concepts which can break
> existing apps, and simply justify by "this should not have been done
> in the first time." (eg: unexport syscall_table). But I'm certain
> that all these mistakes are caused by those too long development
> cycles. Some developpers get bored by things that irritate them, and
> prefer to fix the stable tree to stop what they believe is an error,
> instead of waiting for the next release to fix it there.
These things are harder and colder than "boredom" and the like can
influence. What you are dredging up does not actually exist.
On Mon, Jan 03, 2005 at 04:33:25AM -0800, William Lee Irwin III wrote:
>> Anyway, features aren't really allowed to break backward compatibility;
>> we've effectively got 10-year lifetimes for userspace-visible interfaces.
>> If this isn't good enough, well, tough.
On Mon, Jan 03, 2005 at 10:38:45PM +0100, Willy Tarreau wrote:
> All in all, I agree with you. The small differences lie in /proc files or
> oops syntax, etc... But even old syscalls are still supported and that's fine.
> I appreciate it when I read the packet(7) man page to find that even the
> interface from linux 2.0 is still supported.
Despite your appreciation the net effect is actually irrelevance or
less.
On Mon, Jan 03, 2005 at 10:38:45PM +0100, Willy Tarreau wrote:
> The problem is that nowadays, the userspace-visible code is not only in
> userspace anymore, but also involves modules interfaces sometimes because
> some commercial apps rely on modules (firewalls, virtual machines, etc...),
> and their userspace is nuts without those modules, so in a certain way,
> breaking some kernel internals within a stable release does break some apps.
On Sun, Jan 02, 2005 at 05:19:35PM -0800, William Lee Irwin III wrote:
>> Either this is some kind of sick joke or you've never heard of SLES9.
On Mon, Jan 03, 2005 at 10:38:45PM +0100, Willy Tarreau wrote:
> the later :-)
SLES9 is a 2.6.x-based distro release from SuSE. It is in widespread
use by customers and customers are migrating to it en masse on account
of its 2.6.x basis due to 2.6.x' superior performance and stability
characteristics. No "glowing praise" is necessary. This story tells itself.
On Sun, Jan 02, 2005 at 05:19:35PM -0800, William Lee Irwin III wrote:
> (...)
>> You have ignored my entire argument in favor of reiterating your own.
>> One more time, since this apparently needs to be repeated in a
>> condensed and/or simplified form.
On Sun, Jan 02, 2005 at 05:19:35PM -0800, William Lee Irwin III wrote:
>> (1) the "stable" kernels are actually buggier because no one's looking
On Mon, Jan 03, 2005 at 10:38:45PM +0100, Willy Tarreau wrote:
> I don't agree with you. "known" bugs become features of this particular
> release and people learn how to play with them. The MM beahaviour when one
> single user can crash the whole machine just accidentely playing with malloc()
> would be called a bug on any other decent OS. For us it's a feature we have
> been used to live with.
Patterns of userspace usage that take down boxen are not "features" and
are never going to be considered valid aspects of "stable" releases.
On Mon, Jan 03, 2005 at 10:38:45PM +0100, Willy Tarreau wrote:
> It's possible that 2.6 has fewer of those known bugs, but it still has many
> yet-to-discover bugs (the first ones being all those 'my machine does not
> boot anymore' reported here and caused by those too long release cycles).
Ockham. QED.
On Mon, Jan 03, 2005 at 04:33:25AM -0800, William Lee Irwin III wrote:
>> (2) the creation of those feature patches for stable kernels has
>> detracted from the efforts needed to get them actually into
>> the kernel, and they're not going to exist for long
On Mon, Jan 03, 2005 at 10:38:45PM +0100, Willy Tarreau wrote:
> I agree with you on the first part, but not on the second one, because
> as a stable kernel implies it, it will still be possible to apply
> current patches to new releases with very few efforts. Indeed, I have
> already sent rediffed > patches to different maintainers because they
> were easy to do. For a while now, on 2.4, you can easily apply
> jiffies64, epoll, netdev-random, preempt, lowlat, bme, squashfs, tux,
> etc... The list is long and demonstrantes what stable code looks like.
> "stable" does not mean it will not crash, but it means "it will not
> change much", eventhough this tends to imply the former.
Now the everrever-changing definition of "stable" comes into play.
If you can't handle the baseline changing, freeze on a specific release.
On Mon, Jan 03, 2005 at 04:33:25AM -0800, William Lee Irwin III wrote:
>> People are already using it to run the databases their paychecks rely on.
On Mon, Jan 03, 2005 at 10:38:45PM +0100, Willy Tarreau wrote:
> I feel they're brave. I know several other people who went back,
> either because they didn't feel comfortable with upgrades these size,
> which sometimes did not boot because of random patches, or simply
> because of the scheduler which didn't let them type normally in an
> SSH session on a CPU-bound system, or even a proxy which performance
> dropped by a factor of 5 between 2.4 and 2.6. I know they don't
> report it, but they are not developpers. They see that 2.6 is not
> ready yet, and turn back to stable 2.4.
They are not brave. They are the most cowardly people in existence.
They don't dare go on using 2.4.x because it can't handle the load,
because it can't hadle the bigger and newer machines, and because it
just doesn't work for them. It's not like they moved frivolously;
they're more terrified of migrating than even the most horrible bugs.
-- wli
Alan Cox
> Agreed. We (whoever "we" are) have erred too much on longer
> cycles for stability, but it's not working out as hoped IMO.
After 2.6.9-ac its clear that the long 2.6.9 process worked very badly.
While 2.6.10 is looking much better its long period meant the allegedly
"official" base kernel was a complete pile of insecure donkey turd for
months. That doesn't hurt most vendor users but it does hurt those
trying to do stuff on the base kernels very badly.
Alan
Christoph Hellwig
> want to keep using them. Changing a "stable" kernel will continuously
> annoy users and vendors.
So buy some Operating System that supports the propritary software of
your choice but stop annoying us.
Bill Davidsen
> On Mon, Jan 03 2005, Bill Davidsen wrote:
>
>>SCSI command filtering - while I totally support the idea (and always
>>have), I miss running cdrecord as a normal user. Multisession doesn't work
>>as a normal user (at least if you follow the man page) because only root
>>can use -msinfo. There's also some raw mode which got a permission denied,
>>don't remember as I was trying something not doing production stuff.
>
>
> So look at dmesg, the kernel will dump the failed command. Send the
> result here and we can add that command, done deal. 2.6.10 will do this
> by default.
>
Is this enough? I'm building 2.6.10-bk6 on a spare machine to try this
on a system with a "scsi" CD interface via USB. The commands appear to
go through the same process, but I'll know in an hour or so.
I was going to look these up before suggesting that they were
trustworthy, but I'll take this as a offer to do that and accept!
Obviously security comes first, if these are not trustworthy I won't
argue for their inclusion.
kjournald starting. Commit interval 5 seconds
EXT3 FS on hdb1, internal journal
EXT3-fs: mounted filesystem with ordered data mode.
scsi: unknown opcode 0x01
scsi: unknown opcode 0x55
scsi: unknown opcode 0x1e
scsi: unknown opcode 0x35
--
-bill davidsen (davi...@tmr.com)
"The secret to procrastination is to put things off until the
last possible moment - but no longer" -me
Bill Davidsen
> I don't know about #3 below, but #1 and #2 are certainly true.
> I always preferred to run a vanilla stable kernel as I did not
> trust the vendors' kernels because their patches were not as well
> eyed as the vanilla kernel. I prefer to compile a kernel for
> my specific machines, some of which are old and do better with a
> hand-configured kernel rather than a Microsoftian monolith that
> is compiled with all possible options as modules.
Same conclusion from another direction. If I make a patch which I know
can't (or won't) be accepted into the mainline, it's easier for me to
carry it forward on a mainline kernel. I'm happy to say I haven't had to
do that for a while, although I will probably rehack the network code a
little this summer.
> Nevertheless, it would be nice to see a no-new-features, stable series
> spun off from these development kernels, maybe .4th number releases,
> like 2.6.10 also becomes a 2.6.10.0 that starts a 2.6.10.1, then 2.6.10.2,
> etc...with iteritive bug fixes to the same kernel and no new features
> in such a branch, it might become stable enough for users to have
> confidence
> installing them on their desktop or stable machines.
>
> It wouldn't have to be months upon months of diverging code, as jumps
> to a new stable base can be started upon any fairly stable development
> kernel, say 2.6.10 w/e100 fixed, tracing fixed, the slab bug fix, and
> the capabilities bugs fixed going into a 2.6.10.1 that has no new features
> or old features removed. Serious bug fixes after that could go into a
> 2.6.10.2, etc. Such point releases would be easier to manage and only
> be updated/maintained as long as someone was interested enough to do it.
>
> The same process would be applied to a future dev-kernel that appears to be
> mostly stable after some number of weeks of alpha testing. It may be
> the case that a given furture dev-kernel has no stable branch off of it
> because it either a) didn't need one, or b) was too far from stable to
> start
> one.
If the -rc process were in place, new feature freeze until the big green
bugs were fixed just before the next release, that actually might be
most of a solution.
No one bug akpm can accurately asses how well fixes come back from
vendors, but I suspect that the kernel is moving too fast and vendors
"pick one" and stabilize that, by which time the kernel.org is
generations down the road. It's possible that some fixes are then
rediffed against the current kernel and fed, but I have zero information
on that happening or not.
>> 3) In some cases the commercial vendors don't seem to release
>> source to some of the kernels except to people who have bought
>> the packages, so those vendor kernel fixes aren't 'publically'
>> visible.
That shouldn't happen, and in practice it's rare. But you may have to
search a bit to find the sources...
Bill Davidsen
> On Sun, 2 Jan 2005, Andries Brouwer wrote:
>
>> You change some stuff. The bad mistakes are discovered very soon.
>> Some subtler things or some things that occur only in special
>> configurations or under special conditions or just with
>> very low probability may not be noticed until much later.
>
>
> Some of these subtle bugs are only discovered a year
> after the distribution with some particular kernel has
> been deployed - at which point the kernel has moved on
> so far that the fix the distro does might no longer
> apply (even in concept) to the upstream kernel...
>
> This is especially true when you are talking about really
> big database servers and bugs that take weeks or months
> to trigger.
>
There is a reason why people pay big bucks to Redhat (and others) for a
five year contract to back port the bug fixes to the original kernel and
software. Barring some huge change I need, I expect to run AS3.0 for
four more years for one application, "learning experiences" are not a
good thing.
Bill Davidsen
> On Sun, Jan 02, 2005 at 04:19:17PM -0800, William Lee Irwin III wrote:
>
>>On Sun, Jan 02, 2005 at 01:42:11PM -0800, William Lee Irwin III wrote:
>>
>>>>This is not optimism. This is experience. Every ``stable'' kernel I've
>>>>seen is a pile of incredibly stale code where vi'ing any file in it
>>>>instantly reveals numerous months or years old bugs fixed upstream.
>>>>What is gained in terms of reducing the risk of regressions is more
>>>>than lost by the loss of critical examination and by a long longshot.
>>
>>On Sun, Jan 02, 2005 at 11:15:34PM +0100, Adrian Bunk wrote:
>>
>>>The main advantage with stable kernels in the good old days (tm) when 4
>>>and 6 were even numbers was that you knew if something didn't work, and
>>>upgrading to a new kernel inside this stable kernel series had a
>>>relatively low risk of new breakages. This meant one big migration every
>>>few years and relatively easy upgrades between stable series kernels.
>>
>>This never saved anyone any pain. 2.4.x was not the stable kernel
>>you're painting it to be until 2.4.20 or later, and by the time it
>>became so the fixes for major regressions that occurred during 2.3.x
>>were deemphasized and ignored for anything prior to 2.6.x.
>
>
> I don't know which specific regressions you have in mind, but for
>
>>95% of the users 2.4 is a pretty usable kernel.
>
>
>>On Sun, Jan 02, 2005 at 11:15:34PM +0100, Adrian Bunk wrote:
>>
>>>Nowadays in 2.6, every new 2.6 kernel has several regressions compared
>>>to the previous one, and additionally obsolete but used code like
>>>ipchains and devfs is scheduled for removal making upgrades even harder
>>>for many users.
>>
>>My experience tells me that the number of regressions in 2.6.x compared
>>to purportedly ``far stabler'' kernels is about the same or (gasp!)
>>less. So the observable advantage of the ``frozen'' or ``stable'' model
>>is less than or equal to zero.
>>
>>Frankly, kernel hacking is a difficult enough task (not that I
>>personally find it so) that frivolous patches are not overwhemingly
>>numerous. The ``barrier'' you're erecting is primarily acting as a
>>barrier to fixes, not bugs.
>
>
> My point is different.
>
> Perhaps the number of fixes for bugs equals the number of new bugs
> in 2.6 .
>
> But it's not about the number of bugs alone. The question is the number
> of regressions compared to a previous kernel in this series.
>
> 2.4 -> 2.6 is a major migration.
>
> 2.4.27 -> 2.4.28 is a kernel upgrade that is very unlikely to cause
> problems.
>
> Compared to this, 2.6.9 -> 2.6.10 is much more likely to break an
> existing setup that worked in 2.6.9 .
>
>
>>On Sun, Jan 02, 2005 at 11:15:34PM +0100, Adrian Bunk wrote:
>>
>>>There's the point that most users should use distribution kernels, but
>>>consider e.g. that there are poor souls with new hardware not supported
>>>by the 3 years old 2.4.18 kernel in the stable part of your Debian
>>>distribution.
>>
>>Again, the loss of critical examination far outweighs the purported
>>defense against regressions. The most typical result of playing the fix
>>backporting game for extended periods of time is numerous rounds of
>>months-long bughunts for bugs whose fixes were merged years ago upstream.
>>When the bugs are at long last found, they are discovered to fix the
>>problems of hundreds of users until the next such problem surfaces.
>
>
> The main question is, whether it might be possible to make a very short
> 2.7 line (< 6 months).
>
> Imagine e.g. a feature freeze for 2.6 now. Then 2.7 starts with a
> feature freeze for 2.7 one or two months later. During this time, all
> the changes that do now flood into 2.6 would go into 2.7, and then
> there are a few months of stabilizing 2.7 .
>
> It's quite the opposite of the current 2.6 model, but a quick 2.8 should
> also avoid this problem you describe.
>
> Basically, in this proposal (if it started today), what was expected to
> be called 2.6.11 will be called 2.7.0, and 2.6.11 will be a bugfix-only
> kernel (considering the amount of changes more like the current -ac than
> the latest -mm).
The development policy is set by majority vote on a regular basis.
However, since only one vote counts and Linus prefers it the way it is,
we live with it. In my opinion the stable series is -ac, Alan actually
runs the kernels.
Bill Davidsen
> On Sun, 2 Jan 2005, Adrian Bunk wrote:
>
>> The main advantage with stable kernels in the good old days (tm) when 4
>
>
>> to the previous one, and additionally obsolete but used code like
>
>
> the 2.4 kernel before 2.4.20 or thereabouts.
>
> I'm pretty sure 2.6 is actually doing better than the
> early 2.0, 2.2 and 2.4 kernels...
>
2.6 is doing better in terms of staying up, not eating my files, etc.
I'm less sure about the things being 'changed' (by design) vs. 'broken'
(by unintended bug introduction). My sense is that there are people who
want to remove features which are not broken nor causing huge overhead
or developer effort.
Bill Davidsen
> On 3 Jan 2005, at 16:37, William Lee Irwin III wrote:
>
>> On Mon, Jan 03, 2005 at 10:20:40AM -0500, Rik van Riel wrote:
>>
>>>> 2.2 before 2.2.20 also had this kind of problem, as did
>>>> the 2.4 kernel before 2.4.20 or thereabouts.
>>>> I'm pretty sure 2.6 is actually doing better than the
>>>> early 2.0, 2.2 and 2.4 kernels...
>>
>>
>>
>>> My personal impression was that even the 2.6.0-test kernels were much
>>> better than the 2.4.0-test kernels.
>>> But 2.6.20 will most likely still have the stability of the early
>>> 2.6 kernels instead of a greatly increased stability as observed in
>>> 2.2.20 and 2.4.20 .
>>
>>
>> converge to as great of stability or greater stability than the
>> 2.4-style process. I specuate that it will in fact do precisely that.
>
>
> I would like to comment in that the issue is not exclusively targeted to
> stability, but the ability to keep up with kernel development. For
> example, it was pretty common for older versions of VMWare and NVidia
> driver to break up whenever a new kernel version was released.
>
> code to adopt the new changes in the Linux kernel.
Different can, different worms... there are a number of developers who
dislike closed source drivers and software and make no effort to avoid
breaking them. I'm happy to see a vendor care enough about Linux to have
the driver, there are legal reasons why some source isn't open. You
can't always fund using politically correct hardware. To paraphrase, "we
don't always compute with the hardware we'd LIKE to have, we have to
compute with the hardware we DO have."
Bill Davidsen
> On Mon, 2005-01-03 at 16:34 +0100, Adrian Bunk wrote:
>
>>On Mon, Jan 03, 2005 at 10:18:47AM -0500, Rik van Riel wrote:
>>
>>>On Sun, 2 Jan 2005, Andries Brouwer wrote:
>>>
>>>
>>>>You change some stuff. The bad mistakes are discovered very soon.
>>>>Some subtler things or some things that occur only in special
>>>>configurations or under special conditions or just with
>>>>very low probability may not be noticed until much later.
>>>
>>>Some of these subtle bugs are only discovered a year
>>>after the distribution with some particular kernel has
>>>been deployed - at which point the kernel has moved on
>>>so far that the fix the distro does might no longer
>>>apply (even in concept) to the upstream kernel...
>>>
>>>This is especially true when you are talking about really
>>>big database servers and bugs that take weeks or months
>>>to trigger.
>>
>>this time will be roughly what 2.6 would have been under the current
>>development model, and 2.6 will be a rock stable kernel.
>
>
>
> as long as more things get fixed than new bugs introduced (and that
> still seems to be the case) things only improve in 2.6.
>
> The joint approach also has major advantages, even for quality:
> All testing happens on the same codebase.
> Previously, the testing focus was split between the stable and unstable
> branch, to the detriment of *both*.
You think so? I think the number of people testing the 2.4.xx-rc
versions AND the 2.6.xx-bkN versions is a small (nonzero) percentage of
total people trying any new release. I think people test what they plan
to use, so there's less competition for testers than you suggest. People
staying with 2.4 test that, people wanting or needing to move forward
test 2.6.
Bill Davidsen
> Bill Davidsen <davi...@tmr.com> said:
>
> [...]
>
> > I have to say that with a few minor exceptions the introduction of new
> > features hasn't created long term (more than a few days) of problems. And
> > we have had that in previous stable versions as well. New features
> > themselves may not be totally stable, but in most cases they don't break
> > existing features, or are fixed in bk1 or bk2. What worries me is removing
> > features deliberately, and I won't beat that dead horse again, I've said
> > my piece.
> >
> > The "few minor exceptions:"
> >
> > SCSI command filtering - while I totally support the idea (and always
> > have), I miss running cdrecord as a normal user. Multisession doesn't work
> > as a normal user (at least if you follow the man page) because only root
> > can use -msinfo. There's also some raw mode which got a permission denied,
> > don't remember as I was trying something not doing production stuff.
>
> It had very nasty security problems. After a short discussion here, it was
> deemed much more important to have a secure system than a (very minor)
> convenience. AFAIU, the patch was backported to 2.4 (or should be ASAP).
As I said, I supported that, but the check is done in such a way that not
even making the application setuid helps, so users can't burn multisession
(and some other obscure forms of) CDs.
>
> > APM vs. ACPI - shutdown doesn't reliably power down about half of the
> > machines I use, and all five laptops have working suspend and non-working
> > resume. APM seems to be pretty unsupported beyond "use ACPI for that."
>
> Many never machines just don't have APM.
What's your point? I'm damn sure there are more machines with APM than 386
CPUs, AHA1540 SCSI controllers, or a lot of other supported stuff. Most
machines which have APM at all have a functional power off capability,
which is a desirable thing for most people.
>
> > None of these would prevent using 2.6 if there were some feature not in
> > 2.4 which gave a reason to switch.
>
> Like 2.6 works fine, 2.4 has no chance on some machines?
Haven't hit one of those yet, although after you get away from Intel I'm
sure there are some.
--
bill davidsen <davi...@tmr.com>
CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.
-
Bill Davidsen
> On Mon, Jan 03, 2005 at 04:33:25AM -0800, William Lee Irwin III wrote:
>
> > This is bizarre. iptables was made the de facto standard in 2.4.x and
> > the alternatives have issues with functionality. The 2.0/2.2 firewalling
> > interfaces are probably ready to go regardless. You do realize this is
> > what you're referring to?
> >
> > 2 major releases is long enough.
>
> if it's long enough, ipfwadm should not have entered 2.6 at all.
That is the truth! No one would have complained if they went away in 2.5.
But now people are using ipchains (it was the default Redhat firewall for
a while even after iptables came out).
> > On Mon, Jan 03, 2005 at 06:33:04AM +0100, Willy Tarreau wrote:
> > > At the moment, the only "serious" use I've found for a 2.6 is a kexec-based
> > > bootloader for known hardware. I've already seen that maintaining it up to
> > > date is not simple, I wonder how distro people work with it... I wouldn't
> > > have to do their work right now.
> >
> > People are already using it to run the databases their paychecks rely on.
>
> I feel they're brave. I know several other people who went back, either because
> they didn't feel comfortable with upgrades these size, which sometimes did not
> boot because of random patches, or simply because of the scheduler which didn't
> let them type normally in an SSH session on a CPU-bound system, or even a
> proxy which performance dropped by a factor of 5 between 2.4 and 2.6. I know
> they don't report it, but they are not developpers. They see that 2.6 is not
> ready yet, and turn back to stable 2.4.
Actually one of the reasons I use 2.6, in many cases it resists hangs
which occur in 2.4. Thank Ingo, Nick and Con for that (and others) making
the scheduler far better at "plays well with others." Between the O(1)
scheduler and the HT fixes, and the various large and small tweaks of
everyone, 2.6 runs well on both large and small machines compared to 2.4.
Yes, there are exceptions.
--
bill davidsen <davi...@tmr.com>
CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.
-
Felipe Alfaro Solana
On 3 Jan 2005, at 23:14, Christoph Hellwig wrote:
>> Gosh! I bought an ATI video card, I bought a VMware license, etc.... I
>> want to keep using them. Changing a "stable" kernel will continuously
>> annoy users and vendors.
>
> So buy some Operating System that supports the propritary software of
> your choice but stop annoying us.
I'm sorry if my comments annoy you. I thought this was an open list,
open for discussion. I see I was wrong.
Bill Davidsen
> On Mon, 3 Jan 2005, Horst von Brand wrote:
>
> > "Theodore Ts'o" <ty...@mit.edu> said:
> >
> > [...]
> >
> > > The real key, as always, is getting users to download and test a
> > > release. So another approach might be to shorten the time between
> > > 2.6.x and 2.6.x+1 releases, so as to recreate more testing points,
> > > without training people to wait for -bk1, -bk2, -rc1, etc. before
> > > trying out the kernel code. This is the model that we used with the
> > > 2.3.x series, where the time between releases was often quite short.
> > > That worked fairly well, but we stopped doing it when the introduction
> > > of BitKeeper eliminated the developer synch-up problem. But perhaps
> > > we've gone too far between 2.6.x releases, and should shorten the time
> > > in order to force more testing.
> >
> > Is there any estimate of the number of daily-straight-from-BK users? I'm
> > one, haven't seen any trouble (thus silent up to here).
>
> I'm another. Every morning when I turn on my machine I grab the latest
> -bk, build it with my usual config, install that kernel and reboot, then
> use that as my "kernel of the day". I do this on both my home and work
> box (well, the work box only does this on mondays) and I've had very
> little trouble so far.
Somewhere there is a pawn shop with only one big brass ball, and I know
where the other two are...
--
bill davidsen <davi...@tmr.com>
CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.
-
Theodore Ts'o
> On Mon, Jan 03, 2005 at 01:36:21PM -0500, Theodore Ts'o wrote:
> > 2.3.x series, where the time between releases was often quite short.
> > That worked fairly well, but we stopped doing it when the introduction
> > of BitKeeper eliminated the developer synch-up problem. But perhaps
> > we've gone too far between 2.6.x releases, and should shorten the time
> > in order to force more testing.
>
> release about once a month prior to OLS. Post OLS, it's now once
> every three months or there abouts, which, IMO is far too long.
I was thinking more about every week or two (ok, two releases in a day
like we used to do in the 2.3 days was probably too freequent :-), but
sure, even going to a once-a-month release cycle would be better than
the current 3 months between 2.6.x releases.
- Ted
Roman Zippel
On Monday 03 January 2005 14:24, Diego Calleja wrote:
> I fully agree with WLI that the 2.4 development model and the
> backporting-mania created more problems than it solved, because in the real
> world almost everybody uses what distros ship, and what distros ship isn't
> kernel.org but heavily modified kernels, which means that the kernel.org
> was not really "well-tested" or it took much longer to become "well-tested"
> because it wasn't really being used.
Backporting isn't the primary problem. The real problem were the huge time
intervals between stable releases. A new stable release brings a huge amount
of changes which got different levels of testing, which makes upgrading quite
an experience.
What we need are regular releases of stable kernels with a manageable amount
of changes and a development tree to pull these changes from. It's a bit
comparable to Debian testing/unstable. Changes go only from one tree to the
other if they fulfil certain criteria. The job of the stable tree maintainer
wouldn't be anymore to apply random patches sent to him, but to select
instead which patches to pull from the development tree.
This doesn't of course guarantees perfectly stable kernels, but it would
encourage more people to run recent stable kernels and avoids the huge steps
in kernel upgrades. The only problem is that I don't know of any source code
management system which supports this kind of development reasonably easy...
bye, Roman
Paolo Ciarrocchi
> Hi,
>
> On Monday 03 January 2005 14:24, Diego Calleja wrote:
>
> > I fully agree with WLI that the 2.4 development model and the
> > backporting-mania created more problems than it solved, because in the real
> > world almost everybody uses what distros ship, and what distros ship isn't
> > kernel.org but heavily modified kernels, which means that the kernel.org
> > was not really "well-tested" or it took much longer to become "well-tested"
> > because it wasn't really being used.
>
> Backporting isn't the primary problem. The real problem were the huge time
> intervals between stable releases. A new stable release brings a huge amount
> of changes which got different levels of testing, which makes upgrading quite
> an experience.
> What we need are regular releases of stable kernels with a manageable amount
> of changes and a development tree to pull these changes from. It's a bit
> comparable to Debian testing/unstable. Changes go only from one tree to the
> other if they fulfil certain criteria. The job of the stable tree maintainer
> wouldn't be anymore to apply random patches sent to him, but to select
> instead which patches to pull from the development tree.
> This doesn't of course guarantees perfectly stable kernels, but it would
> encourage more people to run recent stable kernels and avoids the huge steps
> in kernel upgrades. The only problem is that I don't know of any source code
> management system which supports this kind of development reasonably easy...
It really makes sense.
vanilla and -mm are already a kind of stable/unstale tree though.
--
Paolo
Thomas Graf
> On Mon, Jan 03, 2005 at 06:59:27PM +0000, Russell King wrote:
> > It is also the model we used until OLS this year - there was a 2.6
> > release about once a month prior to OLS. Post OLS, it's now once
> > every three months or there abouts, which, IMO is far too long.
>
> I was thinking more about every week or two (ok, two releases in a day
> like we used to do in the 2.3 days was probably too freequent :-), but
> sure, even going to a once-a-month release cycle would be better than
> the current 3 months between 2.6.x releases.
It definitely satifies many of the impatients but it doesn't solve the
stability problem. Many bugs do not show up on developer machines until
just right after the release (as you pointed out already). rc releases
don't work out as expected due to various reasons, i think one of them
is that rc releases don't get announced on the newstickers, extra work
is required to patch the kernel etc. What about doing a test release
just before releasing the final version. I'm not talking about yet
another 2 weeks period but rather just 2-3 days and at most 2 bk
releases in between. Full tarball must be available to make it as
easy as possible. I'm quite sure there are a lot of willing testers
simply too lazy to take a shot at every single rc release. If things
get really worse and huge fixes are required the final release could
be defered in favour of another rc cycle.
Gene Heskett
[...]
>Somewhere there is a pawn shop with only one big brass ball, and I
> know where the other two are...
Yeah, well, one does get used to carrying them around after a while
Bill. Not quite in this context, but I have been asked how in hell I
can sit so comfortably by witnesses, after just having torn some $10k
piece of broadcast gear down, and then put it back together again,
and it works when I'm done, something it didn't do whan I started...
And thats what it does take sometimes, big (brass?) balls. And thats
what keeps me lurking here and playing with new kernels all the time
at age 70. Currently running 2.6.10-ac2.
But, I have to agree with the general tone of this thread, we do not
IMO have, as 2005 opens up, a kernel code base that runs on
everything its supposed to run on, not by a long shot. And to apply
the 'stable' label to this is stretching the point like a used car
salesman selling a 49 nash. Don't get me wrong either, I choose to
do this and generally speaking I'm having a lot of fun trying to keep
up with the various new kernels. And if something doesn't work, you
all hear from me fairly quick, and thats how stability is achieved,
by folks like me taking the chance and getting burnt. I may not know
how to fix it cause this ain't an amiga anymore, but I can be the
remote hands to furnish the clues those of you who do code in your
sleep can fix.
Its moving way too fast in terms of new features to ever get to a
'stable' point, and I think it is now time to fork things off into a
2.7 tree, while 2.6 continues on till the individual distros don't
have the huge menu of patches they are now applying to their own
kernels, as everything worth doing in 2.6 has made it to the
kernel.org downloadable code by the time it gets to 2.6.20 or so.
And thats what I'd call stable, stable like the
2.4.20-sthg-or-other-ck6 I've been running on my firewall box for
years. It 'just works' in between hardware glitches...
--
Cheers, Gene
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
99.31% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com attorneys please note, additions to this message
by Gene Heskett are:
Copyright 2004 by Maurice Eugene Heskett, all rights reserved.
Alexander E. Patrakov
> I feel they're brave. I know several other people who went back, either
> because they didn't feel comfortable with upgrades these size, which
> sometimes did not boot because of random patches, or simply because of the
> scheduler which didn't let them type normally in an SSH session on a
> CPU-bound system, or even a proxy which performance dropped by a factor of
> 5 between 2.4 and 2.6. I know they don't report it, but they are not
> developpers. They see that 2.6 is not ready yet, and turn back to stable
> 2.4.
Here is one more regression report.
My /home was on reiserfs some time ago (migrated to ext3 using convertfs due
to this regression). I read my mail with KMail. I am also subscribed to
several mailing lists. I have a separate Maildir-formatted folder for each
mailing list. Some of such folders are more than a year old and contain
thousands of messages. With linux-2.4, I could click on such folder and the
list of messages sorted by subject will appear in KMail almost instantly.
With linux-2.6, this process takes much longer.
--
Alexander E. Patrakov
David Lang
> "L. A. Walsh" <l...@tlinx.org> said:
>
>> From what I've read here, stable Debian, it seems, is in the 2.4 series.
>
> Stable Debian is 3 years old.
Stable Debian uses the 2.2 kernel
--
There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies.
-- C.A.R. Hoare
Sean
> Willy Tarreau wrote:
>
>> I feel they're brave. I know several other people who went back, either
>> because they didn't feel comfortable with upgrades these size, which
>> sometimes did not boot because of random patches, or simply because of
>> the
>> scheduler which didn't let them type normally in an SSH session on a
>> CPU-bound system, or even a proxy which performance dropped by a factor
>> of
>> 5 between 2.4 and 2.6. I know they don't report it, but they are not
>> developpers. They see that 2.6 is not ready yet, and turn back to stable
>> 2.4.
>
> Here is one more regression report.
>
> My /home was on reiserfs some time ago (migrated to ext3 using convertfs
> due
> to this regression). I read my mail with KMail. I am also subscribed to
> several mailing lists. I have a separate Maildir-formatted folder for each
> mailing list. Some of such folders are more than a year old and contain
> thousands of messages. With linux-2.4, I could click on such folder and
> the
> list of messages sorted by subject will appear in KMail almost instantly.
> With linux-2.6, this process takes much longer.
>
Which might just indicate that the old development model under which the
2.6 kernel was developed wasn't ideal. Perhaps you wouldn't be having
this problem had the new development model been in place sooner.
Sean
Willy Tarreau
> * Theodore Ts'o <2005010400...@thunk.org> 2005-01-03 19:24
> > On Mon, Jan 03, 2005 at 06:59:27PM +0000, Russell King wrote:
> > > It is also the model we used until OLS this year - there was a 2.6
> > > release about once a month prior to OLS. Post OLS, it's now once
> > > every three months or there abouts, which, IMO is far too long.
> >
> > I was thinking more about every week or two (ok, two releases in a day
> > like we used to do in the 2.3 days was probably too freequent :-), but
> > sure, even going to a once-a-month release cycle would be better than
> > the current 3 months between 2.6.x releases.
>
> It definitely satifies many of the impatients but it doesn't solve the
> stability problem. Many bugs do not show up on developer machines until
> just right after the release (as you pointed out already). rc releases
> don't work out as expected due to various reasons, i think one of them
> is that rc releases don't get announced on the newstickers, extra work
> is required to patch the kernel etc.
The problem with -rc is that there are two types of people :
- the optimists who think "good, it's already rc. I'll download it and
run it as soon as it's released"
- the pessimists who think "I killed my machine twice with rc, let's leave
it to other brave testers".
These two problems are solvable with the same solution : no rc anymore.
I agree with Ted. A version every week or 2 weeks is good. People will
run random versions, some will report problems, others not. After that,
you know the differences between exact releases, you don't have to parse
28 MB changes. And you can also ask them to upgrade or downgrade and
quickly find where the bug entered.
Others will find stable versions they will want to stick to for some time,
which would also improve the bugs detection. At the time of 2.1, there were
many people using it because there were known stable versions (thanks to
Alan, btw). I remember having run an NFS server on 2.1.131-ac13 which was
fast an rock solid at this time.
Today's -rc system slows down testing. I also look at 2.4 : people only
test 2.4 when there is a new release. Between releases, very few people
such Adrian and a few others recompile a full kernel and report problems.
When you don't have much free time, you don't want to spend it testing
pre-releases which you think did not change from the previous one.
> What about doing a test release
> just before releasing the final version. I'm not talking about yet
> another 2 weeks period but rather just 2-3 days and at most 2 bk
> releases in between. Full tarball must be available to make it as
> easy as possible. I'm quite sure there are a lot of willing testers
> simply too lazy to take a shot at every single rc release. If things
> get really worse and huge fixes are required the final release could
> be defered in favour of another rc cycle.
if it's 2-3 days, it's reasonnable I think. It should let some people
report build problems just before the real one.
Regards,
Willy
Willy Tarreau
> Felipe Alfaro Solana <lk...@mac.com> said:
>
> [...]
>
> > I would like to comment in that the issue is not exclusively targeted
> > to stability, but the ability to keep up with kernel development. For
> > example, it was pretty common for older versions of VMWare and NVidia
> > driver to break up whenever a new kernel version was released.
>
>
> > I think it's a PITA for developers to rework some of the closed-source
> > code to adopt the new changes in the Linux kernel.
>
> course by others.
it will not solve the problem : if a driver or any glue logic breaks, it's
because interface has changed again. When you will have 3000 open drivers,
you'll have to find people to make the changes every week. The solution in
the first place is to respect some code stability and not to break thinks
every week.
Willy Tarreau
> > Gosh! I bought an ATI video card, I bought a VMware license, etc.... I
> > want to keep using them. Changing a "stable" kernel will continuously
> > annoy users and vendors.
>
> So buy some Operating System that supports the propritary software of
> your choice but stop annoying us.
That's what he did. But it was not written in the notice that it could stop
working at any time :-)
Willy
Al Viro
> On Mon, Jan 03, 2005 at 10:14:42PM +0000, Christoph Hellwig wrote:
> > > Gosh! I bought an ATI video card, I bought a VMware license, etc.... I
> > > want to keep using them. Changing a "stable" kernel will continuously
> > > annoy users and vendors.
> >
> > So buy some Operating System that supports the propritary software of
> > your choice but stop annoying us.
>
> That's what he did. But it was not written in the notice that it could stop
> working at any time :-)
Do you want a long list of message-IDs going way, way back? Ones of Linus'
postings saying that there never had been any promise whatsoever of in-kernel
interfaces staying unchanged...
For fsck sake, people, give it a rest already. 3rd-party kernel modules
are and had always been responsibility of their maintainers, regardless
of licensing, commercial status, etc. Exported functions and data structures
can and do change; doing out-of-tree development means taking a calculated
risk and being ready to follow these changes.
So yes, it *had* been written. Many times. In details. With feeling.
Eric W. Biederman
> "L. A. Walsh" <l...@tlinx.org> said:
>
> > It seems that some developers have the opinion that the end-user base
> > no longer is their problem or audience and that the distros will patch
> > all the little boo-boo's in each unstable 2.6 release.
>
> AFAIU, the current development model is designed to _diminish_ the need of
> custom patching by distributions. For example, RH 9 2.4 kernels were mostly
> 2.6 via backports and random patches. But the patches were only maintained
> by RH, so it was a large duplication of effort (not even counting the other
> distributions). With 2.6 everybody can work on a up-to-date code base, much
> less need of distribution backports and patches (and associated random
> incompatibilities) benefits every user.
And that idea I really appreciate it. From the looks of things though
it does not feel like the distros have caught on. I know at least that
it has been painful working with SuSE's 2.6.ancient fork when I have
perfectly good code that runs in 2.6.latest.
If the distros will update their base kernel once a year or so I can
seem some benefits to the new dev model. But so far I have not seen
the updates and when you have to use a distro kernel is seems to
be the same old same old.
> > It seems like it would become quite a chore
> > to decide what code is let into the stable version. It's also
> > considered by many to be "less" fun, not only to "manage the
> > stable distro", but backport code into the previous distro.
>
> Lots of rather pointless work. Much of it something each distribution has
> to do on their own (because f.ex. vanilla 2.4 is _just fixes_, no backports
> of cool (and required) new functionality), instead of cooperating in
> building a better overall kernel.
Except some features did make it into 2.4.x like native pci-express support.
That is certainly more than just fixes.
> > Nevertheless, it would be nice to see a no-new-features, stable series
> > spun off from these development kernels, maybe .4th number releases,
> > like 2.6.10 also becomes a 2.6.10.0 that starts a 2.6.10.1, then 2.6.10.2,
> > etc...with iteritive bug fixes to the same kernel and no new features
> > in such a branch, it might become stable enough for users to have confidence
> > installing them on their desktop or stable machines.
>
> See above. The 2.6.9-x kernels from Red Hat/Fedora are targeted to be
> exactly that...
Ah another fork that makes support from third parties a pain. So it
appears Red Hat is going the same way I have observed with SuSE.
I do believe a model where we stabilize features and let them shake out
independently. Is where we need to go for Linux. But we seem still
to be at the teething stage and I am frustrated.
Eric
Arjan van de Ven
> > as long as more things get fixed than new bugs introduced (and that
> > still seems to be the case) things only improve in 2.6.
> >
> > The joint approach also has major advantages, even for quality:
> > All testing happens on the same codebase.
> > Previously, the testing focus was split between the stable and unstable
> > branch, to the detriment of *both*.
>
> You think so? I think the number of people testing the 2.4.xx-rc
> versions AND the 2.6.xx-bkN versions is a small (nonzero) percentage of
> total people trying any new release. I think people test what they plan
> to use, so there's less competition for testers than you suggest. People
> staying with 2.4 test that, people wanting or needing to move forward
> test 2.6.
>
Actually I suspect the number of people testing 2.4.xx-rc is *really*
small now. My point however was more towards a 2.6 / 2.7 split, where
the people who want to test newest do 2.7 while people who want to test
stable test 2.6; right now those two groups test basically the same
codebase.
Jens Axboe
You don't have write permissions on the device.
--
Jens Axboe
Bernd Petrovitsch
[...]
> Gosh! I bought an ATI video card, I bought a VMware license, etc.... I
> want to keep using them. Changing a "stable" kernel will continuously
> annoy users and vendors.
Then avoid a "changing kernel" and do not upgrade.
Bernd
--
Firmix Software GmbH http://www.firmix.at/
mobil: +43 664 4416156 fax: +43 1 7890849-55
Embedded Linux Development and Services
Felipe Alfaro Solana
> On Tue, Jan 04, 2005 at 06:46:49AM +0100, Willy Tarreau wrote:
>> On Mon, Jan 03, 2005 at 10:14:42PM +0000, Christoph Hellwig wrote:
>>>> Gosh! I bought an ATI video card, I bought a VMware license,
>>>> etc.... I
>>>> want to keep using them. Changing a "stable" kernel will
>>>> continuously
>>>> annoy users and vendors.
>>>
>>> So buy some Operating System that supports the propritary software of
>>> your choice but stop annoying us.
>>
>> That's what he did. But it was not written in the notice that it
>> could stop
>> working at any time :-)
>
> Do you want a long list of message-IDs going way, way back? Ones of
> Linus'
> postings saying that there never had been any promise whatsoever of
> in-kernel
> interfaces staying unchanged...
I don't pretend that kernel interfaces stay written in stone, for
ages. What I would like is that, at least, those interfaces were stable
enough, let's say for a few months for a stable kernel series, so I
don't have to keep bothering my propietary VMWare vendor to fix the
problems for me, since the new kernel interface broke VMWare. Yeah, I
know I could decide not to upgrade kernels in last instance, but that's
not always possible.
If kernel interfaces need to be changed for whatever reason, change
them in 2.7, -mm, -ac or whatever tree first, and let the community
know beforehand what those changes will be, and be prepared to adapt.
Meanwhile, try to leave 2.6 as stable as possible.
Rik van Riel
> I don't pretend that kernel interfaces stay written in stone, for ages. What
> I would like is that, at least, those interfaces were stable enough, let's
> say for a few months for a stable kernel series, so I don't have to keep
> bothering my propietary VMWare vendor to fix the problems for me, since the
How much work are you willing to do to make this happen ? ;)
It would be easy enough for you to take 2.6.9 and add only
security fixes and critical bugfixes to it for the next 6
months - that would give your binary vendors a stable
source base to work with...
--
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it." - Brian W. Kernighan
Felipe Alfaro Solana
> On Tue, 4 Jan 2005, Felipe Alfaro Solana wrote:
>
>> I don't pretend that kernel interfaces stay written in stone, for
>> ages. What I would like is that, at least, those interfaces were
>> stable enough, let's say for a few months for a stable kernel series,
>> so I don't have to keep bothering my propietary VMWare vendor to fix
>> the problems for me, since the
>
> How much work are you willing to do to make this happen ? ;)
As much as needed :-)
> It would be easy enough for you to take 2.6.9 and add only
> security fixes and critical bugfixes to it for the next 6
> months - that would give your binary vendors a stable
> source base to work with...
I would... if it was easy enough to find some form of a security
patches pool. It's usually difficult to find a site where I can
download security patches for older versions of vanilla kernels. I have
the feeling that this security fixes go mainstream onto the latest
kernel versions, leaving users in hands of their distribution (either
to upgrade to a new distribution kernel, or waiting for the
distribution vendor to backport).
Thus, sometimes people are forced to upgrade to a new kernel version as
such security patches either don't exist for older kernel versions, are
difficult to find, or need backporting (and I'm not knowledgeable
enough to backport nearly half of them), and since the new kernel
version introduces new features -- which sometimes do break existing
propietary software -- users starts complaining.
However, it's true that distributions, like Red Hat or Fedora, try at
its best to keep the kernel as stable as possible. For example, FC3
seems to sport something like a 2.6.9 kernel, but sometimes those
kernels are so heavily patched that some closed-source software doesn't
work.
I know I can choose open software and hardware vendors compatible with
Linux, but sometimes I cannot. I like VMware, and I use it a lot. I'm
not willing to sacrifice it, and that's the reason I think 2.6 must
fork as soon as possible into 2.7.
William Lee Irwin III
>> 2.6 will stop having small issues in each release until 2.7 is forked just
>> like 2.4 broke things until 2.5 was forked. The difference IMO
>> is that linux development now avoids things like the unstability which the
>> 2.4.10 changes caused and things like the fs corruption bugs we saw in 2.4
On Mon, Jan 03, 2005 at 02:47:27PM +0100, Adrian Bunk wrote:
> The 2.6.9 -> 2.6.10 patch is 28 MB, and while the changes that went into
> 2.4 were limited since the most invasive patches were postponed for 2.5,
> now _all_ patches go into 2.6 .
> Yes, -mm gives a bit more testing coverage, but it doesn't seem to be
> enough for this vast amount of changes.
No amount of testing coverage will ever suffice. You're trying to
empirically establish the nonexistence of something, which is only
possible to repudiate, and never to verify.
-- wli
William Lee Irwin III
>> Open up the code. Most of the changes will then be done as a matter of
>> course by others.
On Tue, Jan 04, 2005 at 06:44:58AM +0100, Willy Tarreau wrote:
> it will not solve the problem : if a driver or any glue logic breaks, it's
> because interface has changed again. When you will have 3000 open drivers,
> you'll have to find people to make the changes every week. The solution in
> the first place is to respect some code stability and not to break thinks
> every week.
Tihs is not entirely true. I'd like to point to remap_pfn_range() as a
smoothly-executed API change. All in-tree drivers were swept. Out-of-
tree open-source drivers got by with nothing more than warnings. Even
binary-only drivers had no trouble with mere recompiles of glue layers.
Many other API changes are also executed with similar smoothness.
-- wli
William Lee Irwin III
> If the -rc process were in place, new feature freeze until the big green
> bugs were fixed just before the next release, that actually might be
> most of a solution.
> No one bug akpm can accurately asses how well fixes come back from
> vendors, but I suspect that the kernel is moving too fast and vendors
> "pick one" and stabilize that, by which time the kernel.org is
> generations down the road. It's possible that some fixes are then
> rediffed against the current kernel and fed, but I have zero information
> on that happening or not.
It does happen. I can't give a good estimate of how often. Someone at a
distro may be able to help here, though it's unclear what this specific
point is useful for.
What is a useful observation is that the 2.6-style development model is
not in use in these instances, which instead use the older "frozen"
model. This means that using frozen models in mainline is redundant. The
function and service are available elsewhere and numerous simultaneously
frozen trees guarantees no forward progress during such syzygys.
Horst von Brand
> On Mon, Jan 03, 2005 at 04:33:25AM -0800, William Lee Irwin III wrote:
> > This is bizarre. iptables was made the de facto standard in 2.4.x and
> > the alternatives have issues with functionality. The 2.0/2.2 firewalling
> > interfaces are probably ready to go regardless. You do realize this is
> > what you're referring to?
> > 2 major releases is long enough.
When iptables went in, they computed reasonable dates for the final demise
of ipfwadm and ipchains compatibility. IIRC, both came out as March, 2004.
> if it's long enough, ipfwadm should not have entered 2.6 at all.
Backward compatibility is important. Sure, under the old development model
it would have been "deprecate <feature> in 2.<2n>, axe it at the start of
2.<2n+1>, by 2.<2n+2> it is gone".
> It's not
> because you don't see any use to that particular feature that you can
> guarantee that it is not used at all. At least, a large public call to
> get in touch with the potentially unique user of this feature would be a
> start, but generally we should not remove a feature from a stable
> kernel.
A public call won't get to that single user, or she is simply too lazy to
answer. Axe it, and they scream. That is much more effective. Besides,
that way she looks for help (and surely finds a kind soul helping out with
migrating). Or stays put (2.4 is still alivve...).
> What will go next ? minix, because someone will decide that there
> have been many better filesystems for a long time, so that's long enough
> ?
Minix is a nice example, doesn't interfere with other work and is no great
maintainance burden. I guess it'll stay.
> Revert modules to modutils because someone will think it's simpler for
> everyone to use a single toolset ?
Nope. The change had its very good reasons.
> I have no problem removing numerous
> feature between major releases,
What's the beef then?
> even breaking APIs,
AFAIR, the only public APIs broken have been for exclusive use by utilities
tightly coupled to the kernel (module handling, firewall), and those during
development series. If you fool around with development kernels, you are
supposed to be prepared for such breakage. Normal users get insulated by
their respective distributions.
> but I really hate it
> when something which is called "stable" constantly changes.
List the offending changes for us all to contemplate, please.
> > On Mon, Jan 03, 2005 at 06:33:04AM +0100, Willy Tarreau wrote:
[...]
> > Who do you think is actually banging out the code on this mailing list?
> Frankly, sometimes I'm really wondering. We see lots of very clever
> ideas, and sometimes people come up with concepts which can break
> existing apps, and simply justify by "this should not have been done in
> the first time." (eg: unexport syscall_table). But I'm certain that all
> these mistakes are caused by those too long development cycles. Some
> developpers get bored by things that irritate them, and prefer to fix the
> stable tree to stop what they believe is an error, instead of waiting for
> the next release to fix it there.
Distributions (or final users) are free to undo offending changes.
[...]
> The problem is that nowadays, the userspace-visible code is not only in
> userspace anymore, but also involves modules interfaces sometimes because
> some commercial apps rely on modules (firewalls, virtual machines,
> etc...), and their userspace is nuts without those modules, so in a
> certain way, breaking some kernel internals within a stable release does
> break some apps.
Tough luck. If it is closed source, whoever builds it is responsible for
making it work. If they don't keep it working for your preferred setup,
it's the price _you_ pay for closed source + bleeding edge. LKML has no way
of helping you there.
--
Dr. Horst H. von Brand User #22616 counter.li.org
Departamento de Informatica Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria +56 32 654239
Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513
William Lee Irwin III
>>> The joint approach also has major advantages, even for quality:
>>> All testing happens on the same codebase.
>>> Previously, the testing focus was split between the stable and unstable
>>> branch, to the detriment of *both*.
At some point in the past, someone else wrote:
>> You think so? I think the number of people testing the 2.4.xx-rc
>> versions AND the 2.6.xx-bkN versions is a small (nonzero) percentage of
>> total people trying any new release. I think people test what they plan
>> to use, so there's less competition for testers than you suggest. People
>> staying with 2.4 test that, people wanting or needing to move forward
>> test 2.6.
On Tue, Jan 04, 2005 at 08:42:36AM +0100, Arjan van de Ven wrote:
> Actually I suspect the number of people testing 2.4.xx-rc is *really*
> small now. My point however was more towards a 2.6 / 2.7 split, where
> the people who want to test newest do 2.7 while people who want to test
> stable test 2.6; right now those two groups test basically the same
> codebase.
But this is a good thing; new code should meet the prior standards
of stability and correctness as should the tree at all times. Efforts
to recover it once it is lost to a large degree are doomed.
-- wli
Horst von Brand
[...]
> Gosh! I bought an ATI video card, I bought a VMware license, etc.... I
> want to keep using them. Changing a "stable" kernel will continuously
> annoy users and vendors.
If you are sooo attached to this, just keep a distribution for which
vendors give you drivers. But when the vendor decides the product has to
die to get you to buy the next "completely redone" (== minor fixes and
updates) version, you are stranded for good.
> I think new developments will force a 2.7 branch: when 2.6 feature set
> stabilizes, people will keep more time testing a stable, relatively
> static kernel base, finding bugs, instead of trying to keep up with
> changes.
And when 2.7 opens, very few developers will tend 2.6; and as 2.7 diverges
from it, fewer and fewer fixes will find their way back. And so you finally
get a rock-stable (== unchanging) 2.6, but hopelessly out of date and thus
unfixable (if nothing else because there are no people around who remember
how it worked).
--
Dr. Horst H. von Brand User #22616 counter.li.org
Departamento de Informatica Fono: +56 32 654431
Universidad Tecnica Federico Santa Maria +56 32 654239
Casilla 110-V, Valparaiso, Chile Fax: +56 32 797513
Felipe Alfaro Solana
> Felipe Alfaro Solana <lk...@mac.com> said:
>
> [...]
>
>> Gosh! I bought an ATI video card, I bought a VMware license, etc.... I
>> want to keep using them. Changing a "stable" kernel will continuously
>> annoy users and vendors.
>
> If you are sooo attached to this, just keep a distribution for which
> vendors give you drivers. But when the vendor decides the product has
> to
> die to get you to buy the next "completely redone" (== minor fixes and
> updates) version, you are stranded for good.
>
>> I think new developments will force a 2.7 branch: when 2.6 feature set
>> stabilizes, people will keep more time testing a stable, relatively
>> static kernel base, finding bugs, instead of trying to keep up with
>> changes.
>
> And when 2.7 opens, very few developers will tend 2.6; and as 2.7
> diverges
> from it, fewer and fewer fixes will find their way back. And so you
> finally
> get a rock-stable (== unchanging) 2.6, but hopelessly out of date and
> thus
> unfixable (if nothing else because there are no people around who
> remember
> how it worked).
I can see no easy solution for this... If Linus decides to fork off
2.7, development efforts will go into 2.7 and fixes should get
backported to 2.6. If Linus decides to stay with 2.6, new development
will have to be "conservative" enough not to break things that were
working.
I tend to prefer forking off 2.7: more agressive features can be
implemented and tested without bothering disrupting the stable 2.6
branch.
Adrian Bunk
> On Mon, 3 Jan 2005, Horst von Brand wrote:
>
> >"L. A. Walsh" <l...@tlinx.org> said:
> >
> >> From what I've read here, stable Debian, it seems, is in the 2.4 series.
> >
> >Stable Debian is 3 years old.
>
> Stable Debian uses the 2.2 kernel
2.2 is the default kernel, but kernel 2.4 is supported and 2.4.18 is
shipped with Debian stable.
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
Adrian Bunk
> On Mon, Jan 03, 2005 at 02:24:12PM +0100, Diego Calleja wrote:
> >> 2.6 will stop having small issues in each release until 2.7 is forked just
> >> like 2.4 broke things until 2.5 was forked. The difference IMO
> >> is that linux development now avoids things like the unstability which the
> >> 2.4.10 changes caused and things like the fs corruption bugs we saw in 2.4
>
> On Mon, Jan 03, 2005 at 02:47:27PM +0100, Adrian Bunk wrote:
> > The 2.6.9 -> 2.6.10 patch is 28 MB, and while the changes that went into
> > 2.4 were limited since the most invasive patches were postponed for 2.5,
> > now _all_ patches go into 2.6 .
> > Yes, -mm gives a bit more testing coverage, but it doesn't seem to be
> > enough for this vast amount of changes.
>
> No amount of testing coverage will ever suffice. You're trying to
> empirically establish the nonexistence of something, which is only
> possible to repudiate, and never to verify.
I claim:
The less and the less invasive patches go into the kernel, the less
likely are breakages.
"enough" shouldn't say "mathematically exactly proven that no
regressions exist" but more something like the pretty small number of
regressions in recent 2.4 kernels.
> -- wli
cu
Adrian
--
"Is there not promise of rain?" Ling Tan asked suddenly out
of the darkness. There had been need of rain for many days.
"Only a promise," Lao Er said.
Pearl S. Buck - Dragon Seed
-