RFD: Kernel release numbering
Linus Torvalds
This is an idea that has been brewing for some time: Andrew has mentioned
it a couple of times, I've talked to some people about it, and today Davem
sent a suggestion along similar lines to me for 2.6.12.
Namely that we could adopt the even/odd numbering scheme that we used to
do on a minor number basis, and instead of dropping it entirely like we
did, we could have just moved it to the release number, as an indication
of what was the intent of the release.
The problem with major development trees like 2.4.x vs 2.5.x was that the
release cycles were too long, and that people hated the back- and
forward-porting. That said, it did serve a purpose - people kind of knew
where they stood, even though we always ended up having to have big
changes in the stable tree too, just to keep up with a changing landscape.
So the suggestion on the table would be to go back to even/odd, but do it
at the "micro-level" of single releases, rather than make it a two- or
three-year release cycle.
In this setup, all kernels would still be _stable_, in the sense that we
don't anticipate any real breakage (if we end up having to rip up so much
basic stuff that we have to break anything, we'd go back to the 2.7.x kind
of numbering scheme). So we should fear odd releases, but track them, to
make sure that they are good (if you don't track them, and problems won't
be fixed in the even version either)
But we'd basically have stricter concerns for an even release, and in
particular the plan would be that the diff files would alternate between
bigger ones (the 2.6.10->11 full diff was almost 5MB) and smaller ones (a
2.6.11->12 release would be a "stability only" thing, and hopefully the
diff file would be much smaller).
We'd still do the -rcX candidates as we go along in either case, so as a
user you wouldn't even _need_ to know, but the numbering would be a rough
guide to intentions. Ie I'd expect that distributions would always try to
base their stuff off a 2.6.<even> release.
It seems like a sensible approach, and it's not like the 2.4.x vs 2.5.x
kind of even/odd thing didn't _work_, the problems really were an issue of
too big granularity making it hard for user and developers alike. So I see
this as a tweak of the "let's drop the notion althogether for now"
decision, and just modify it to "even/odd is meaningful at all levels".
In other words, we'd have an increasing level of instability with an odd
release number, depending on how long-term the instability is.
- 2.6.<even>: even at all levels, aim for having had minimally intrusive
patches leading up to it (timeframe: a week or two)
with the odd numbers going like:
- 2.6.<odd>: still a stable kernel, but accept bigger changes leading up
to it (timeframe: a month or two).
- 2.<odd>.x: aim for big changes that may destabilize the kernel for
several releases (timeframe: a year or two)
- <odd>.x.x: Linus went crazy, broke absolutely _everything_, and rewrote
the kernel to be a microkernel using a special message-passing version
of Visual Basic. (timeframe: "we expect that he will be released from
the mental institution in a decade or two").
The reason I put a shorter timeframe on the "all-even" kernel is because I
don't want developers to be too itchy and sitting on stuff for too long if
they did something slightly bigger. In theory, the longer the better
there, but in practice this release numbering is still nothing but a hint
of the _intent_ of the developers - it's still not a guarantee of "we
fixed all bugs", and anybody who expects that (and tries to avoid all odd
release entirely) is just setting himself up for not testing - and thus
bugs.
Comments?
Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majo...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Andrew Morton
>
> The reason I put a shorter timeframe on the "all-even" kernel is because I
> don't want developers to be too itchy and sitting on stuff for too long if
> they did something slightly bigger.
If they're feeling itchy they should dig in and help fix the bugs in 2.6.<odd>.
Jeff V. Merkey
__Stable__ would be a good thing. The entire 2.6 development has been a
disaster from
a stability viewpoint. I have to maintain a huge tree of patches in
order to ship appliance
builds due to the lack of stability for 2.6. I think that the even
number releases will take longer
but it's worth the wait.
Jeff
Lars Marowsky-Bree
> We'd still do the -rcX candidates as we go along in either case, so a=
s a
> user you wouldn't even _need_ to know, but the numbering would be a r=
ough
> guide to intentions. Ie I'd expect that distributions would always tr=
y to
> base their stuff off a 2.6.<even> release.
If the users wouldn't even have to know, why do it? Who will benefit
from this, then?
I think a better approach, and one which is already working out well in
practice, is to put "more intrusive" features into -mm first, and only
migrate them into 2.6.x when they have 'stabilized'.
This could be improved: _All_ new features have to go through -mm first
for a period (of whatever length) / one cycle. 2.6.x only directly pick=
s
up "obvious" bugfixes, and a select set of features which have ripened
in -mm. 2.6.x-pre releases would then basically "only" clean up
integration bugs.
-mm would be the 'feature tree'. Of course, features which have matured
in other eligible trees might also work; the key point is the two-stage
approach and it doesn't matter whether the chaos stage has one or three
trees, as long as it's not more than that.
I think that would be natural extension of how things already work and
just tightens the process some.
(From a vendor perspective, this would mean we'd be safe picking up any
2.6.x tree + select choices from x+1-pre plus whatever we are force fed
by those who pay.)
If one wanted to get fancy, which I'm throwing in just to make everybod=
y
lose the core point of the argument: One could associate "points" with
each feature / patch in -mm, based on an estimation of how
intrusive/well-tested/dangerous/heavenly that patch is, and mandate tha=
t
only 42 points per 2.6.x release are allowed.
Of course, one could also apply common sense. But, that's not as silly.
Or way more so, but less amusing than voting wars.
> Comments?
The numbering scheme is more confusing and unclear, and complexity is
the enemy of reliability.
Sincerely,
Lars Marowsky-Brée <l...@suse.de>
--
High Availability & Clustering
SUSE Labs, Research and Development
SUSE LINUX Products GmbH - A Novell Business
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel"=
Greg KH
On Wed, Mar 02, 2005 at 02:21:38PM -0800, Linus Torvalds wrote:
>
> - 2.6.<even>: even at all levels, aim for having had minimally intrusive
> patches leading up to it (timeframe: a week or two)
>
> with the odd numbers going like:
>
> - 2.6.<odd>: still a stable kernel, but accept bigger changes leading up
> to it (timeframe: a month or two).
Ok, that's acceptable to me, but realize that this puts a bigger burden
on the maintainers to queue up patches for you. It's not that big of a
deal, just something to be aware of.
Speaking of which, does this mean I shouldn't hit you with all of my
pending stuff? I know some of the other subsystem maintainers have a
lot of stuff queued up too. Should we start this new numbering scheme
as of today? Or wait until 2.6.13?
thanks,
greg k-h
Russell King
> release number, depending on how long-term the instability is.
>
> - 2.6.<even>: even at all levels, aim for having had minimally intrusive
> patches leading up to it (timeframe: a week or two)
>
> with the odd numbers going like:
>
> - 2.6.<odd>: still a stable kernel, but accept bigger changes leading up
> to it (timeframe: a month or two).
> - 2.<odd>.x: aim for big changes that may destabilize the kernel for
> several releases (timeframe: a year or two)
> - <odd>.x.x: Linus went crazy, broke absolutely _everything_, and rewrote
> the kernel to be a microkernel using a special message-passing version
> of Visual Basic. (timeframe: "we expect that he will be released from
> the mental institution in a decade or two").
This sounds good, until you realise that some of us have been sitting
on about 30 patches for at least the last month, because we where
following your guidelines about the -rc's. Things like adding support
for new ARM machines and other devices, dynamic tick support for ARM,
etc.
If I'm going to have to sit on this stuff for another month, it'll bit
rot rather badly, and I might as well throw away all these patches now
and ask people not to send stuff other than pure bug fixes.
A slightly bigger problem is that I'm in the middle of merging a large
chunk of these for you right now, and at least Andrew has already pulled
some of these from my BK repo.
If we are going to do this, can we start it post 2.6.12 please?
--
Russell King
Linux kernel 2.6 ARM Linux - http://www.arm.linux.org.uk/
maintainer of: 2.6 Serial core
Josh Boyer
> It seems like a sensible approach, and it's not like the 2.4.x vs 2.5.x
> kind of even/odd thing didn't _work_, the problems really were an issue of
> too big granularity making it hard for user and developers alike. So I see
> this as a tweak of the "let's drop the notion althogether for now"
> decision, and just modify it to "even/odd is meaningful at all levels".
I like it, but what about important security fixes or mistakes like
2.6.8.1? In those situations, would you release that kernel under the
next even number and skip the odd number? Or would you release the fix
under an odd number and sorta throw off the "meaning"?
I'm not trying to be a pain, I'm really not.
josh
Randy.Dunlap
> Linus Torvalds <torv...@osdl.org> wrote:
>
>>The reason I put a shorter timeframe on the "all-even" kernel is because I
>>don't want developers to be too itchy and sitting on stuff for too long if
>>they did something slightly bigger.
>
>
> If they're feeling itchy they should dig in and help fix the bugs in 2.6.<odd>.
Yep, that's what I'll be doing. :)
--
~Randy
Willy Tarreau
For a long time, I've been hoping/asking for a more frequent stable/unstable
cycle, so clearly you can count my vote on this one (eventhough it might
count for close to zero). This is a very good step towards a better stability
IMHO.
However, I have a comment :
> - 2.6.<odd>: still a stable kernel, but accept bigger changes leading up
> to it (timeframe: a month or two).
> - 2.<odd>.x: aim for big changes that may destabilize the kernel for
> several releases (timeframe: a year or two)
> - <odd>.x.x: Linus went crazy, broke absolutely _everything_, and rewrote
> the kernel to be a microkernel using a special message-passing version
> of Visual Basic. (timeframe: "we expect that he will be released from
> the mental institution in a decade or two").
I don't agree with you on this last one (not the fact that you don't want
an mk+mp+vb combination :-)). The VERSION number (in the makefile meaning of
the term) only gets updated every 10 years or so. So it does not need to
jump. PATCHLEVEL increments are rare enough to justify lots of breaking.
I certainly can imagine people laughing at your OS when you jump from v2.6.X
to v4.0.X, it will have a smell of slowaris (remember 2.6 - 2.7 - 7 - 8 that
confused everyone ?). On the other side, the openbsd numbering scheme is far
simpler to understand, and I sincerely think that we should enter 3.0 after
2.8. As a side note, I've always wondered why we would not swap odds and
evens, so that we can keep the same major number between devel and release
(eg: devel 2.8, release 2.9 ; not devel 2.9, release 3.0). Anyway, people
have got used to stay away from the '.0' releases of any product on the
planet.
Regards
Willy
Mark Gross
Why be so wishy washy?
You can't say all that good stuff and then end it with " in practice this
release numbering is still nothing but a hint of the _intent_ of the
developer" and expect this to work out for more than a couple of builds
before things fall back to where they are today.
Keep the change sets for the even numbers bug fix and stability changes only.
While your at it schedule a cut off rc number by which any moderately risky
bug and stability changes are allowed into the even build cycle, say RC2,
after which only stability and bug fixes that are considered to be "safe"
changes are accepted.
You could also consider using the RC2 rule for odd releases so that there is
plenty of time to hash out the bugs from the bigger changes. Just reject big
changes after RC2. The developers will adjust and plan for that after a few
releases.
Thrashing your users with regressions is a very bad thing.
--mgross
Greg KH
>
> This could be improved: _All_ new features have to go through -mm first
> up "obvious" bugfixes, and a select set of features which have ripened
> in -mm. 2.6.x-pre releases would then basically "only" clean up
> integration bugs.
This is the way things work today already. The only exception being the
networking code, but hey, networking's always been special :)
thanks,
greg k-h
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
Dave Jones
> On Wed, Mar 02, 2005 at 02:21:38PM -0800, Linus Torvalds wrote:
> > In other words, we'd have an increasing level of instability with an odd
> > release number, depending on how long-term the instability is.
> >
> > - 2.6.<even>: even at all levels, aim for having had minimally intrusive
> > patches leading up to it (timeframe: a week or two)
> >
> > with the odd numbers going like:
> >
> > - 2.6.<odd>: still a stable kernel, but accept bigger changes leading up
> > to it (timeframe: a month or two).
> > - 2.<odd>.x: aim for big changes that may destabilize the kernel for
> > several releases (timeframe: a year or two)
> > - <odd>.x.x: Linus went crazy, broke absolutely _everything_, and rewrote
> > the kernel to be a microkernel using a special message-passing version
> > of Visual Basic. (timeframe: "we expect that he will be released from
> > the mental institution in a decade or two").
>
> This sounds good, until you realise that some of us have been sitting
> on about 30 patches for at least the last month, because we where
> following your guidelines about the -rc's. Things like adding support
> for new ARM machines and other devices, dynamic tick support for ARM,
> etc.
>
> If I'm going to have to sit on this stuff for another month, it'll bit
> rot rather badly, and I might as well throw away all these patches now
> and ask people not to send stuff other than pure bug fixes.
The fact that this new approach serialises the stable/devel lineation
whereas traditionally it was parallel (2.x.y/2.x+1.y) is going to be
a real pain for a lot of maintainers.
In short, instead of a single 'merge with linus tree', I'm now going to
need a 'merge with linus' and 'merge with linus next time' tree for every
tree I maintain.. It's not impossible to maintain, but its extra burden.
Burden which a lot of folks may consider not worth it.
Dave
Willy Tarreau
On Wed, Mar 02, 2005 at 03:04:01PM -0800, Greg KH wrote:
> /me kills my patchbomb script for now
>
> On Wed, Mar 02, 2005 at 02:21:38PM -0800, Linus Torvalds wrote:
> >
> > - 2.6.<even>: even at all levels, aim for having had minimally intrusive
> > patches leading up to it (timeframe: a week or two)
> >
> > with the odd numbers going like:
> >
> > - 2.6.<odd>: still a stable kernel, but accept bigger changes leading up
> > to it (timeframe: a month or two).
>
> Ok, that's acceptable to me, but realize that this puts a bigger burden
> on the maintainers to queue up patches for you. It's not that big of a
> deal, just something to be aware of.
Not necessaruly, because the rules could be more relaxed during -rc stage
in an odd release, and stable releases could be shorter, so at the end, the
"patch fridge" would be needed only between the very end of the odd version
and the start of the even one.
Another possibility is for developpers to start to submit/merge patches for
the next odd release while the even one is still in -rc.
> Speaking of which, does this mean I shouldn't hit you with all of my
> pending stuff? I know some of the other subsystem maintainers have a
> lot of stuff queued up too. Should we start this new numbering scheme
> as of today? Or wait until 2.6.13?
if there is so much pending stuff, why not sacrifice 2.6.12, use 2.6.13
to fix a bit and merge even more, then 2.6.14 would be the real first
stable release ?
Regards,
Willy
Zwane Mwaikambo
> __Stable__ would be a good thing. The entire 2.6 development has been a
> disaster from a stability viewpoint. I have to maintain a huge tree of
> patches in order to ship appliance builds due to the lack of stability
> for 2.6. I think that the even number releases will take longer but it's
> worth the wait.
What form of instability are you referring to?
Matt Mackall
>
> it a couple of times, I've talked to some people about it, and today Davem
> sent a suggestion along similar lines to me for 2.6.12.
>
> Namely that we could adopt the even/odd numbering scheme that we used to
> do on a minor number basis, and instead of dropping it entirely like we
> did, we could have just moved it to the release number, as an indication
> of what was the intent of the release.
One last plea for the 2.4 scheme:
a) all the crazy stuff goes in 2.6.x-preN, which ends up being
equivalent to 2.6.<odd> and friends in your scheme
b) bugfixes only in 2.6.x-rcN, which ends up being equivalent to
2.6.<even>-* in your scheme.
c) 2.6.x is always 2.6.x-rc<last> with just a version number change[1]
This has some nice features:
- alternates as rapidly as you want between stable and development
- no brown paper bag bugs sneaking in between -rc<last> and 2.6.x
- 2.6.* is suitable for all users, 2.6.*-rc* is suitable for almost
all users
- it's already in use for 2.4 and people are happy with it
I _really_ don't want to explain to people that they don't want to use
2.6.13 because it's an odd number but that 2.4.31 is just fine (and so
is 2.6.9). Nor do I want to teach my ketchup tool the difference
between 2.6-stable and 2.6-unstable.
> The problem with major development trees like 2.4.x vs 2.5.x was that the
> release cycles were too long, and that people hated the back- and
> forward-porting. That said, it did serve a purpose - people kind of knew
> where they stood, even though we always ended up having to have big
> changes in the stable tree too, just to keep up with a changing landscape.
I think naming the interim releases -pre/-rc has done this admirably
for 2.4.
--
Mathematics is the supreme nostalgia of our time.
Linus Torvalds
On Wed, 2 Mar 2005, Jeff Garzik wrote:
>
> 30? Try 310 changesets, in my netdev-2.6 pending queue.
Note that I don't think a 2.6.<even> would have problems with things like
driver updates.
This was somewhat brought on (at least for me, dunno about Davem) by
things like 4-level page tables etc stuff. I don't think most people even
realized how _smoothly_ that thing seems to have gone, even if the ppc64
people ended up having some really nasty debugging (and they came through
with flying colors, but they probably didn't much enjoy the thing).
I would not keep regular driver updates from a 2.6.<even> thing. But I
_would_ try to keep things like all the TSO pain, the 4-level page tables,
and in general big merges that have been in CVS trees etc, and can't claim
to be "lots of small stuff".
For example, there's some fundamental 16-bit PCMCIA cleanups pending in
the -mm tree, meaning that the kernel can work with PCMCIA cards without a
"cardmgr" deamon. That would be something that is _not_ appropriate in the
first stable version. That doesn't mean that individual PCMCIA device
drivers could not get updated.
But hey, you may be right. It might just not be obvious enough which class
any particular merge falls under.
Linus
Greg KH
> On Wed, Mar 02, 2005 at 03:04:01PM -0800, Greg KH wrote:
> > /me kills my patchbomb script for now
> >
> > On Wed, Mar 02, 2005 at 02:21:38PM -0800, Linus Torvalds wrote:
> > >
> > > - 2.6.<even>: even at all levels, aim for having had minimally intrusive
> > > patches leading up to it (timeframe: a week or two)
> > >
> > > with the odd numbers going like:
> > >
> > > - 2.6.<odd>: still a stable kernel, but accept bigger changes leading up
> > > to it (timeframe: a month or two).
> >
> > Ok, that's acceptable to me, but realize that this puts a bigger burden
> > on the maintainers to queue up patches for you. It's not that big of a
> > deal, just something to be aware of.
>
> Not necessaruly, because the rules could be more relaxed during -rc stage
> in an odd release, and stable releases could be shorter, so at the end, the
> "patch fridge" would be needed only between the very end of the odd version
> and the start of the even one.
It's still a lot of work to queue up pending patches. Over a certian
limit, it gets pretty unstable, trust me. I'm not complaining, just
letting you know that this will push more work on the maintainers, but
hey, we scale, so it's not that big of a deal :)
> Another possibility is for developpers to start to submit/merge patches for
> the next odd release while the even one is still in -rc.
No, that's not ok to do. We want people to focus on getting a -rc out.
thanks,
greg k-h
Jeff Garzik
> This sounds good, until you realise that some of us have been sitting
> on about 30 patches for at least the last month, because we where
> following your guidelines about the -rc's. Things like adding support
> for new ARM machines and other devices, dynamic tick support for ARM,
> etc.
30? Try 310 changesets, in my netdev-2.6 pending queue.
Jeff
Richard Purdie
> Namely that we could adopt the even/odd numbering scheme that we used
> to do on a minor number basis, and instead of dropping it entirely like
> we did, we could have just moved it to the release number, as an
> indication
> of what was the intent of the release.
How about taking the idea a bit further and have two active versions. Eg:
now 2.6.11 is out, new changes go into a 2.6.13 series. Any changes to make
2.6.11 (more :) stable go into a 2.6.12 series which is
stability/security/whatever improvements rather than devel work.
This way you can shorten the length of the time the odd series spends in -rc
and can spend more time accepting patches rather than having long periods
where developers queue them. Distributors are encouraged to use the even
numbers including the even -rc versions and to give feedback as to what
stability/whatever patches need adding to the even series to keep them
happy.
Just an idea...
Regards,
Richard
Linus Torvalds
On Wed, 2 Mar 2005, Lars Marowsky-Bree wrote:
>
> If the users wouldn't even have to know, why do it? Who will benefit
> from this, then?
They don't _have_ to know. But both users and developers can take
advantage of this to time their patches.
> I think a better approach, and one which is already working out well in
> practice, is to put "more intrusive" features into -mm first, and only
> migrate them into 2.6.x when they have 'stabilized'.
That wouldn't change. But we still have the issue of "they have to be
released sometime". This makes it clear to everybody when to merge, and
when to calm down.
Linus
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
Jeff V. Merkey
> Jeff V. Merkey wrote:
>
>>
>> __Stable__ would be a good thing. The entire 2.6 development has been
>> a disaster from
>> a stability viewpoint. I have to maintain a huge tree of patches in
>> order to ship appliance
>> builds due to the lack of stability for 2.6. I think that the even
>> number releases will take longer but it's worth the wait.
>>
>> Jeff
>
>
> Linus's release cycle estimate might be optimistic. :)
>
Yep, but at least he is addressing the issue and taking responsibility
and showing good prudent leadership and some
reasonable and sound ideas to correct it.
> I'm seeing lots more bug reports recently than I care to see. :(
Yep.
Jeff
Jeff Garzik
I maintain my netdev-2.6 queue by creating a ton of "subject-specific"
repositories locally,
> 8139cp/ bonding/ ieee80211/ mips/ sis900/ typhoon/
> 8139too/ e1000/ ixgb/ misc/ skge/ viro-iomap/
> 8139too-2/ ham/ janitor/ mv643xx/ sk_mca/ viro-iomap-orinoco/
> ALL/ ibmlana/ kill-iphase/ pcnet32/ smc91x/ viro-isa-ectomy/
> atmel/ ibmtr/ mii/ s2io/ sundance/ wifi/
and I pull all of these into a master repository netdev-2.6 (locally
'ALL'). netdev-2.6, in turn, gets pulled into -mm when Andrew makes a
new release. When I am ready to push something upstream, I pull one or
more repositories into my net-drivers-2.6 queue, which is essentially my
"push to upstream" queue.
This scheme allows me to cherry-pick fixes and features. If a critical
fix comes along, it doesn't have to wait for the other less-critical 309
changesets to go.
We could have a linus-pending-2.6 queue, managed by Linus, that
functions in a similar manner to my netdev-2.6:
a) Have a monthly or weekly official release. Could be automated, but
probably not.
b) snapshots of linus-pending-2.6 function as a development tree, taking
over some of the current -mm purpose... -mm is just too massive and
multi-purpose at this point
c) locally [or publicly?], Linus sorts patches and 'bk pulls' into
specific repositories, as I do with netdev-2.6. "sata", "sata-fixes",
"net drivers", "net drivers - janitorial", etc. could be potential pull
queues from me to Linus.
d) When features/fixes are ready to move from linus-pending-2.6 to
linux-2.6 repository, Linus just does a pull+push locally. Fixes would
likely go immediately into linux-2.6, unless it needs staging in
linus-pending-2.6 first. Potentially destabilizing stuff stays longer
in linus-pending-2.6.
Thus, Linus controls the patch flow (and stability/features) simply by
choosing when to pull stuff from the on-going dev tree that he manages.
e) Andrew continues doing what he's been doing: merging tons of
patches, staging big features, etc.
So what does this accomplish, besides increasing the complexity of
Linus's work? What's the point?
1) Creates a structure to enable linux-2.6 to be the on-going stable
tree, by creating an official on-going dev tree.
2) Release early, release often. The proposal that started this thread
does the opposite: it slows things down.
3) Makes it easier to manage the flow of changesets.
4) Takes pressure off -mm to be "everything under the sun." Users are
far more likely to test a tree that is blessed with Holy Penguin Pee,
and builds on all architectures. And this in turn gives -mm more freedom.
Jeff, trying to think outside the box
Neil Brown
If I understand you correctly, what you are effectively saying is that
people don't test the -rc releases enough, so you are going to start
giving these releases a more formal name: 2.6.ODD.
That will encourage more people to test them, so that when you do a
real release (now called 2.6.EVEN instead of 2.6.X), it will be more
stable.
I don't buy this. You won't fool anyone. People will just start
using the .EVEN releases and you won't get any more testing than you
currently do.
A different approach would be to not release a 'stable' kernel at all,
but rather release 'fixup' patches.
i.e. you release 2.6.X as a 'testing' kernel and then start work on
2.6.X+1.
When a patch comes along that fixes a *real*problem* in 2.6.X, you add
that to a list of addenda for 2.6.X. People who run 2.6.X can monitor
that list, applying patches that might be relevant to them.
You keep the addenda for 2.6.X going until 2.6.X+2 is released.
This way there is no clear distinction between "stable" and "testing"
kernels. There is, instead, gradual transition. People who want to
be cautious can decide to use
A testing kernel that has been out for at least 2 weeks - plus addenda
as their definition of 'stable'. Others might prefer
A testing kernel that has been out for at least 6 weeks - plus relevant addenda
as their definition.
People who don't want to think, can pay a distributor to choose a
suitable definition of 'stable'.
This way, you are giving your army of beta-testers clear information
on their level of risk, and allowing them to choose precisely what
level they want. Giving people information and choice is always good
when you want them to help you.
That is the 'testing' side of the equation. The other side is the
'developing' side.
I have been (naively?) assuming that Andrew Morton is the 2.6 kernel
maintainer, because that is what was announced, and I haven't seen any
announcement to alter that.
So I have been sending all my patches to Andrew to go in the -mm tree,
with the understanding that he would forward them on to Linus as
appropriate, and this has been working quite well.
I feel free to send any patch to Andrew at any time, but try to make
it clear which ones should be considered "work-in-progress", which
should be considered 'important-bug-fix' and which sit in between.
I had (even more naively?) been assuming that this is what other
developers were doing. i.e. that this was the approved approach.
But more recently I have discovered that quite a few key developers
develop against Linus' kernel and submit patches directly to him,
apparently bypassing Andrew. This leads to them holding back patches
when a release is approaching, rather than sending them straight to
Andrew for -mm and wider testing. This doesn't sound like a good
thing.
Now, I know our movement is all about freedom (and openness), and you
don't want to force developers into any behaviour patterns that aren't
essential, but I think it would be nice if there was some uniform
perspective on how patches should flow so that we all understood what
each other were doing.
My own preference would be:
- all patches go to Andrew and appear in -mm promptly
- Linus only gets patches from -mm
- most patches are only passed to Linus after they have
been in an -mm release for at least .... 1 week (?)
- some patches go straight to Linus even before a -mm
release if maintainer + Andrew + Linus review and agree
- some patches stay in -mm for extended periods getting refined
before making their way to Linus.
- some patches get ditched from -mm and never make it to Linus.
Is this too restrictive?
Is it too much work for Andrew?
Is it too little work for Linus :-?
Whatever the final answer, the key is to give all your volunteer
supporters (both testers and developers) good information and good
choices (and don't try to deceive them).
NeilBrown
Jeff Garzik
> Jeff Garzik <jga...@pobox.com> wrote:
>
>>IMO too confusing.
>>
>
>
> 2.6.even: bugfixes only
> 2.6.odd: bugfixes and features.
>
> That doesn't even confuse me!
>
>
>>Developers right now are sitting on big piles, and pushing that back
>>even further means every odd release means you are creating a
>>2.4.x/2.5.x backport situation every two releases.
>
>
> No, there is no backporting. If you have a bug, fix it in 2.6.12-pre.
> There is no need to maintain that bugfix in your 2.6.13-candidate tree.
You are missing where the backporting is.
If the time between big merges increases, as with this proposal, then
the distance between local dev trees and linux-2.6 increases.
With that distance, breakages like the 64-bit resource struct stuff
become more painful.
I like my own "ongoing dev tree, ongoing stable tree" proposal a lot
better. But then, I'm biased :)
Jeff
Andrew Morton
2.6.even: bugfixes only
2.6.odd: bugfixes and features.
That doesn't even confuse me!
> Developers right now are sitting on big piles, and pushing that back
> even further means every odd release means you are creating a
> 2.4.x/2.5.x backport situation every two releases.
No, there is no backporting. If you have a bug, fix it in 2.6.12-pre.
There is no need to maintain that bugfix in your 2.6.13-candidate tree.
It's still a linear progression of kernel trees. The only thing which
changes is the types of patches which we put into even releases.
Dave Jones
> > I think a better approach, and one which is already working out well in
> > practice, is to put "more intrusive" features into -mm first, and only
> > migrate them into 2.6.x when they have 'stabilized'.
>
> That wouldn't change. But we still have the issue of "they have to be
> released sometime". This makes it clear to everybody when to merge, and
> when to calm down.
So is the problem that people aren't listening when you say "lets slow down" ?
Why would this change things for people who obviously ignore what you say ? :)
I'll bet you'll still get flooded with "lets see if Linus takes this despite
what he said in his last announcement" patches if we moved to this model.
The only thing that would make a difference afaics, would be you putting
your foot down and just ignoring such submissions ?
Dave
Greg KH
>
> On Wed, 2 Mar 2005, Jeff Garzik wrote:
> >
> > 30? Try 310 changesets, in my netdev-2.6 pending queue.
>
> Note that I don't think a 2.6.<even> would have problems with things like
> driver updates.
>
> This was somewhat brought on (at least for me, dunno about Davem) by
> things like 4-level page tables etc stuff. I don't think most people even
> realized how _smoothly_ that thing seems to have gone, even if the ppc64
> people ended up having some really nasty debugging (and they came through
> with flying colors, but they probably didn't much enjoy the thing).
I think this statement proves that the current development situation is
working quite well. The nasty breakage and details got worked out in
the -mm tree, and then flowed into your tree when they seemed sane.
> I would not keep regular driver updates from a 2.6.<even> thing. But I
> _would_ try to keep things like all the TSO pain, the 4-level page tables,
> and in general big merges that have been in CVS trees etc, and can't claim
> to be "lots of small stuff".
So, any driver stuff is just fine? Great, I don't have an issue with
your proposal then, as it wouldn't affect me that much :)
I do understand what you are trying to achieve here, people don't really
test the -rc releases as much as a "real" 2.6.11 release. Getting a
week of testing and bugfix only type patches to then release a 2.6.12
makes a lot of sense. For example, see all of the bug reports that came
out of the woodwork today on lkml from the 2.6.11 release...
But accepting 310 netdev patches, 250 USB, 50 PCI, 50 I2c, and 100 ALSA
patches in one week and expect the tree to stay "stable" might be a
pretty unreasonable thing to wish for...
thanks,
greg k-h
Jeff Garzik
>
> On Wed, 2 Mar 2005, Jeff Garzik wrote:
>
>>30? Try 310 changesets, in my netdev-2.6 pending queue.
>
>
> Note that I don't think a 2.6.<even> would have problems with things like
> driver updates.
Nah, I agree with DaveJ -- there are definitely "dev" portions when it
comes to driver updates.
Judging from recent posting from Bart, it looks like he has an evil plot
to merge the IDE driver with libata. libata will also eventually
[perhaps with Bart's changes?] make the SCSI portion optional, as I have
long promised. And it's getting other new and destabilizing features.
There will be other changes in SCSI and block too, which want staging...
Some of the stuff I've been putting off until "2.7" will be re-thought
into something that appears in the on-going 2.6 series.
If you don't have driver stability, you don't have a useful kernel...
Jeff
Wakko Warner
know what part of the thread was directed towards me)
Jeff V. Merkey wrote:
> __Stable__ would be a good thing. The entire 2.6 development has been a
> disaster from
> a stability viewpoint. I have to maintain a huge tree of patches in
> order to ship appliance
> builds due to the lack of stability for 2.6. I think that the even
> number releases will take longer
> but it's worth the wait.
I agree about the stability of the 2.6 kernels. The system I'm using now
has always been 2.6 since I first installed it. I have noticed there were
stability issues with 2.6. I remember 2.6.7 was fairly good and was a bit
stable. 2.6.8.1 wasn't that stable (I'm sure I had the .1 patch on it,
however, attempting to do a lock over NFS caused the system to hard freeze).
2.6.9 was more unstable especially with USB. 2.6.10 (which I'm using now)
has to be IMO the most stable 2.6 kernel produced. I'm quite pleased with
it (and to all the kernel hackers, thanks for a great kernel).
P.S. System is a dual xeon 2.6ghz on a supermicro x5da8 1gb ram.
--
Lab tests show that use of micro$oft causes cancer in lab animals
David S. Miller
Jeff Garzik <jga...@pobox.com> wrote:
> If the time between big merges increases, as with this proposal, then
> the distance between local dev trees and linux-2.6 increases.
>
> With that distance, breakages like the 64-bit resource struct stuff
> become more painful.
>
> I like my own "ongoing dev tree, ongoing stable tree" proposal a lot
> better. But then, I'm biased :)
The problem is people don't test until 2.6.whatever-final goes out.
Nothing will change that.
And the day Linus releases we always get a pile of "missing MODULE_EXPORT()"
type bug reports that are one liner fixes. Those fixes will not be seen by
users until the next 2.6.x rev comes out and right now that takes months
which is rediculious for such simple fixes.
We're talking about a one week "calming" period to collect the brown paper
bag fixes for a 2.6.${even} release, that's all.
All this "I have to hold onto my backlog longer, WAHHH!" arguments are bogus
IMHO. We're using a week of quiescence to fix the tree for users so they
are happy whilst we work on the 2.6.${odd} interesting stuff :-)
Dave Jones
> I would not keep regular driver updates from a 2.6.<even> thing.
Then the notion of it being stable is bogus, given how many regressions
the last few kernels have brought in drivers. Moving from 2.6.9 -> 2.6.10
broke ALSA, USB, parport, firewire, and countless other little bits and
pieces that users tend to notice.
The reason that things like 4-level page tables worked out so well
was that it was tested in -mm for however long, so by the time it got
to your tree, the silly bugs had already been shaken out.
Compare this to random-driver-update. -mm testing is a valuable
proving ground, but its no panacea to stability. There's no guarantee
that someone with $affected_device even tried a -mm kernel.
For it to truly be a stable kernel, the only patches I'd expect to
drivers would be ones fixing blindingly obvious bugs. No cleanups.
No new functionality. I'd even question new hardware support if it
wasn't just a PCI ID addition.
Dave
Andrew Morton
>
> But more recently I have discovered that quite a few key developers
> develop against Linus' kernel and submit patches directly to him,
> apparently bypassing Andrew. This leads to them holding back patches
> when a release is approaching, rather than sending them straight to
> Andrew for -mm and wider testing. This doesn't sound like a good
> thing.
>
Only davem, AFAIK. All the other trees get auto-sucked into -mm for
testing. Generally the owners of those trees make the decision as to which
of their code has been sufficiently well-tested for a Linus merge, and when
that should happen.
> Now, I know our movement is all about freedom (and openness), and you
> don't want to force developers into any behaviour patterns that aren't
> essential, but I think it would be nice if there was some uniform
> perspective on how patches should flow so that we all understood what
> each other were doing.
>
> My own preference would be:
> - all patches go to Andrew and appear in -mm promptly
> - Linus only gets patches from -mm
> - most patches are only passed to Linus after they have
> been in an -mm release for at least .... 1 week (?)
> - some patches go straight to Linus even before a -mm
> release if maintainer + Andrew + Linus review and agree
> - some patches stay in -mm for extended periods getting refined
> before making their way to Linus.
> - some patches get ditched from -mm and never make it to Linus.
That's basically what happens now, except I don't physically send the
patches from those 32 bk trees to Linus.
Massimo Cetra
> Namely that we could adopt the even/odd numbering scheme that
> we used to do on a minor number basis, and instead of
> dropping it entirely like we did, we could have just moved it
> to the release number, as an indication of what was the
> intent of the release.
> Comments?
This is surely a good idea because end users (not developers) like me would
have greater possibility not to occur in a regression with an even release.
The real solution to the problem of having a really stable kernel is, IMHO,
to have a wide base of testers.
Usually, following a new stable release announce, lots of bugs get out
because people starts using the new kernel, just because they didn't try any
of the previous -RC releases.
So, why moving from 2.6.14 to 2.6.15 when, in 2/4 weeks, i'll have a more
stable 2.6.16 ?
Will users help testing an odd release to have a good even release ? Or will
they consider an even release as important as a -RC release ?
My thought is that the community should do some marketing on the actual
developing model to obtain a wider testing base, or, with the new proposed
model, let people know that their help is necessary to have a stable kernel
and they should download, compile and install odd releases.
Sincerely,
Massimo Cetra
Linus Torvalds
On Wed, 2 Mar 2005, Greg KH wrote:
>
> I think this statement proves that the current development situation is
> working quite well. The nasty breakage and details got worked out in
> the -mm tree, and then flowed into your tree when they seemed sane.
Actually, the breakage I was talking about got fixed in _my_ tree.
I'd love for the -mm tree to get more testing, but it doesn't.
> So, any driver stuff is just fine? Great, I don't have an issue with
> your proposal then, as it wouldn't affect me that much :)
I don't know about "any", but yeah.
> I do understand what you are trying to achieve here, people don't really
> test the -rc releases as much as a "real" 2.6.11 release. Getting a
> week of testing and bugfix only type patches to then release a 2.6.12
> makes a lot of sense. For example, see all of the bug reports that came
> out of the woodwork today on lkml from the 2.6.11 release...
A large part of it is psychological. On the other hand, it may be that
Neil is right and it would just mean that people wouldn't even test the
odd releases (..because they want to wait a couple of weeks for the even
one), so it may not actually end up helping much.
The thing is, I _do_ believe the current setup is working reasonably well.
But I also do know that some people (a fairly small group, but anyway)
seem to want an extra level of stability - although those people seem to
not talk so much about "it works" kind of stability, but literally a "we
can't keep up" kind of stability (ie at least a noticeable percentage of
that group is not complaining about crashes, they are complaining about
speed of development).
And I suspect that _anything_ I do won't make those people happy.
Linus
Andrew Morton
>
> So what was broken with the 2.6.8.1 type of 'hotfix kernel' release ?
That's an alternative, of course.
But that _is_ a branch, and does need active forward- and (mainly)
backward-porting work.
There's nothing wrong with it per-se, but it becomes a "stabilised version
of the kernel.org tree" or even a "production version of the kernel.org
tree". In other words it's somewhere on the line between the mainline
kernel.org tree and a distribution. How far along that line should it
be positioned?
Dave Jones
> The problem is people don't test until 2.6.whatever-final goes out.
> Nothing will change that.
>
> And the day Linus releases we always get a pile of "missing MODULE_EXPORT()"
> type bug reports that are one liner fixes. Those fixes will not be seen by
> users until the next 2.6.x rev comes out and right now that takes months
> which is rediculious for such simple fixes.
So what was broken with the 2.6.8.1 type of 'hotfix kernel' release ?
Apart from needing to do the 2.6.8.1 -> 2.6.8 -> 2.6.9 transition
which confused some people, it seemed to be easily understood for what it was,
and solved the same problem as this new proposal.
Dave
Ben Greear
> So, why moving from 2.6.14 to 2.6.15 when, in 2/4 weeks, i'll have a more
> stable 2.6.16 ?
> Will users help testing an odd release to have a good even release ? Or will
> they consider an even release as important as a -RC release ?
I think it would be useful for folks to test the 2.6.ODD release because
the understanding is that 2.6.ODD+1 will be out soon, and we are pretty
sure there won't be any large changes in that transition. Any out-side-the-tree
patches will probably apply to both of these w/out manual hacking,
which also makes testing easier.
I am less likely to test an ODD.pre-Z release because there is likely to be
a large pile coming after that, which means that even if pre-Z worked
fine, I still have to be very paranoid about the final release.
Ben
--
Ben Greear <gre...@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
Nick Piggin
> Jeff Garzik <jga...@pobox.com> wrote:
>
>> IMO too confusing.
>>
>
> 2.6.even: bugfixes only
> 2.6.odd: bugfixes and features.
>
> That doesn't even confuse me!
>
I actually second Matt's request; -RCs à la 2.4.
Then your above becomes:
2.6.x-rc: bugfixes only
2.6.x-pre: bugfixes and features
And then that doesn't confuse end users either.
Not to mention that if we adopt a strict release candidate release styl=
e
(ie. last -rc is renamed to release), then you get to move some of the
burden of testing to the end user. Hopefully that will encourage more
people to test -rcs rather than "tricking" them into doing the testing.
And if we really make an effort to only do the first -rc with the
expectation that it will be the *only* one, then that will be a pretty
good incentive to test.
Nick
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel"=
Neil Brown
>
> Only davem, AFAIK. All the other trees get auto-sucked into -mm for
> testing.
Ok, I got the feeling it was more wide spread than that, but I
apparently misread the signs (people mentioning that had 'patches
queued for Linus' and such).
> Generally the owners of those trees make the decision as to which
> of their code has been sufficiently well-tested for a Linus merge, and when
> that should happen.
I wonder if there is a problem here.
Who is in a position to judge when a patch is ready to be merged?
I suspect that it would be hard for a developer to be objective about
the readiness of their own patches (I know all my patches are perfectly
ready the moment I create them, despite what experience tells me :-)
Assuming we have a 'stable', a 'testing' and a 'devel' series
(whatever actual numbering gets used for them(*)), then maybe it is ok
for a developer to judge if it is ready for 'testing', but it should
really have either some minimum time in 'testing' or independent
review before being allowed into 'stable'.
Are you and Linus able to handle the independent review load?
Should every developer/maintainer find someone to review any patches
that they think should 'jump the queue'.
(Would anyone like to review my nfs/raid patches for me? I review
patches I get from others, but find it very hard to review my own
work. Andrew does a good job, but does miss things sometimes).
NeilBrown
(*) Options for naming:
Devel Testing Stable
2.ODD.X 2.EVEN.X 2.EVEN.X-ac
2.6.X-mm 2.6.X-rc 2.6.X
2.6.X-mm 2.6.ODD 2.6.EVEN
2.6.X-mm 2.6.X 2.6.X + patch addenda <--- my preferred
2.6.X-pre 2.6.X-rc 2.6.X
It doesn't matter much what you call them, but I think the three-way
distinction is needed, and there needs to be a well-understood set of
rules for patches moving from one to the next.
NeilBrown
Sven-Haegar Koch
> Linus Torvalds wrote:
>
>> Namely that we could adopt the even/odd numbering scheme that
>> we used to do on a minor number basis, and instead of
>> dropping it entirely like we did, we could have just moved it
>> to the release number, as an indication of what was the
>> intent of the release.
>
>> Comments?
>
> This is surely a good idea because end users (not developers) like me would
> have greater possibility not to occur in a regression with an even release.
What I would like to see as an enduser is (dreaming):
kernel 2.6.x - last released
often released (every 1-2 weeks) kernel 2.6.x.z
containing just the answers to the often repeating
lkml questions which are answered with "use $this simple patch"
kernel 2.6.y-pre/rc/bk - development, working towards 2.6.y
in practice your proposed 2.6.even changes, but these continued until the
next kernel is released, not stopped after 1-2 weeks with the worst fixes.
(a bit like the -as series, but with the "official blessing")
c'ya
sven
--
The Internet treats censorship as a routing problem, and routes around it.
(John Gilmore on http://www.cygnus.com/~gnu/)
Jeff Garzik
And it exacerbates an on-going issue: we are moving away from "release
early, release often", as this proposal just pushes the list of pending
stuff back even further.
Developers right now are sitting on big piles, and pushing that back
even further means every odd release means you are creating a
2.4.x/2.5.x backport situation every two releases.
To take a radical position on the other side, I would prefer a weekly
snapshot as the release, staging invasive things in -mm.
And I think -mm is not enough, even. We have to come up with new ways
to manage this ever-increasing flow of data into our tree.
Jeff
David Lang
> On Wed, 02 Mar 2005 19:29:35 -0500
> Jeff Garzik <jga...@pobox.com> wrote:
>
>> If the time between big merges increases, as with this proposal, then
>> the distance between local dev trees and linux-2.6 increases.
>>
>> With that distance, breakages like the 64-bit resource struct stuff
>> become more painful.
>>
>> I like my own "ongoing dev tree, ongoing stable tree" proposal a lot
>> better. But then, I'm biased :)
>
> The problem is people don't test until 2.6.whatever-final goes out.
> Nothing will change that.
>
> And the day Linus releases we always get a pile of "missing MODULE_EXPORT()"
> type bug reports that are one liner fixes. Those fixes will not be seen by
> users until the next 2.6.x rev comes out and right now that takes months
> which is rediculious for such simple fixes.
>
> We're talking about a one week "calming" period to collect the brown paper
> bag fixes for a 2.6.${even} release, that's all.
>
> All this "I have to hold onto my backlog longer, WAHHH!" arguments are bogus
> IMHO. We're using a week of quiescence to fix the tree for users so they
> are happy whilst we work on the 2.6.${odd} interesting stuff :-)
I understand the desire and benifit to do this sort of fixing, but I
really don't like extending the odd/even thing to other parts of the
kernel version numbers
with 2.6.8 a different path was attempted with 2.6.8.1 couldn't we just
use that numbering scheme (avoiding adding to the numbering confusion
further) and plan on releasing a 2.6.11.1 next tuesday (or so) with the
various paper-bag fixes?
If I understand bitkeeper properly Linus wouldn't even need to duplicate
the patches, he should be able to fork his tree to do the paper-bag fixes
in one fork while continueing development in the other one and then
recombining them when either generates a release.
David Lang
--
There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies.
-- C.A.R. Hoare
Zwane Mwaikambo
> On 2005-03-02T14:21:38, Linus Torvalds <torv...@osdl.org> wrote:
>
> > We'd still do the -rcX candidates as we go along in either case, so as a
> > user you wouldn't even _need_ to know, but the numbering would be a rough
> > guide to intentions. Ie I'd expect that distributions would always try to
> > base their stuff off a 2.6.<even> release.
>
> If the users wouldn't even have to know, why do it? Who will benefit
> from this, then?
>
> I think a better approach, and one which is already working out well in
> practice, is to put "more intrusive" features into -mm first, and only
> migrate them into 2.6.x when they have 'stabilized'.
>
> This could be improved: _All_ new features have to go through -mm first
> for a period (of whatever length) / one cycle. 2.6.x only directly picks
> up "obvious" bugfixes, and a select set of features which have ripened
> in -mm. 2.6.x-pre releases would then basically "only" clean up
> integration bugs.
>
> -mm would be the 'feature tree'. Of course, features which have matured
> in other eligible trees might also work; the key point is the two-stage
> approach and it doesn't matter whether the chaos stage has one or three
> trees, as long as it's not more than that.
Certainly -mm can be the feature tree, but i've noticed that not that many
people run -mm aside from developers. Meaning that a fair number of bugs
seep into Linus' tree before they get attended to. It would even be more
effective if we could get more -mm user coverage. A Linus based odd number
might be closer to that if we hope on people unwittingly running them.
Hua Zhong
>
> Then your above becomes:
> 2.6.x-rc: bugfixes only
> 2.6.x-pre: bugfixes and features
>
> And then that doesn't confuse end users either.
I'll jump in and third this. It looks the "honest" way. I know Linus is
always talking about "open source keeps people honest". Don't play game=
with
the naming. :-)
Hua
Dave Jones
> Dave Jones <da...@redhat.com> wrote:
> >
> > So what was broken with the 2.6.8.1 type of 'hotfix kernel' release ?
>
> That's an alternative, of course.
>
> But that _is_ a branch, and does need active forward- and (mainly)
> backward-porting work.
>
> There's nothing wrong with it per-se, but it becomes a "stabilised version
> of the kernel.org tree" or even a "production version of the kernel.org
> tree". In other words it's somewhere on the line between the mainline
> kernel.org tree and a distribution. How far along that line should it
> be positioned?
In an ideal world, we'd see a single 'y' release of 2.6.x.y, but if x+1 takes
too long to be released, bits of x+1 should also appear in x.y+1
The only question in my mind is 'how critical does a bug have to be to
justify a .y release. Once a new 'x' release comes out, the previous x.y
would likely no longer get any further fixes (unless someone decides the
new 'x' sucked so bad).
Linus using bitkeeper should actually remove most of the pain from this
multiple branch thing though. If fixes alternatively get checked into x.y
and new development goes into x+1, x+1 could do a daily pull of x.y
Thus saving the having to check in fixes to two seperate branches.
(which really really sucks under some SCMs).
Dave
Randy.Dunlap
>
> __Stable__ would be a good thing. The entire 2.6 development has been a
> disaster from
> a stability viewpoint. I have to maintain a huge tree of patches in
> order to ship appliance
> builds due to the lack of stability for 2.6. I think that the even
> number releases will take longer but it's worth the wait.
>
Linus's release cycle estimate might be optimistic. :)
I'm seeing lots more bug reports recently than I care to see. :(
> Linus Torvalds wrote:
>
>> This is an idea that has been brewing for some time: Andrew has mentioned
>> it a couple of times, I've talked to some people about it, and today
>> Davem
>> sent a suggestion along similar lines to me for 2.6.12.
>>
>> Namely that we could adopt the even/odd numbering scheme that we used
>> to do on a minor number basis, and instead of dropping it entirely
>> like we did, we could have just moved it to the release number, as an
>> indication of what was the intent of the release.
>>
>> The problem with major development trees like 2.4.x vs 2.5.x was that
>> the release cycles were too long, and that people hated the back- and
>> forward-porting. That said, it did serve a purpose - people kind of
>> knew where they stood, even though we always ended up having to have
>> big changes in the stable tree too, just to keep up with a changing
>> landscape.
>>
>> So the suggestion on the table would be to go back to even/odd, but do
>> it at the "micro-level" of single releases, rather than make it a two-
>> or three-year release cycle.
>>
>> In this setup, all kernels would still be _stable_, in the sense that we
>> don't anticipate any real breakage (if we end up having to rip up so much
>> basic stuff that we have to break anything, we'd go back to the 2.7.x
>> kind
>> of numbering scheme). So we should fear odd releases, but track them,
>> to make sure that they are good (if you don't track them, and problems
>> won't be fixed in the even version either)
>>
>> But we'd basically have stricter concerns for an even release, and in
>> particular the plan would be that the diff files would alternate between
>> bigger ones (the 2.6.10->11 full diff was almost 5MB) and smaller ones (a
>> 2.6.11->12 release would be a "stability only" thing, and hopefully the
>> diff file would be much smaller).
>>
>> We'd still do the -rcX candidates as we go along in either case, so as
>> a user you wouldn't even _need_ to know, but the numbering would be a
>> rough guide to intentions. Ie I'd expect that distributions would
>> always try to base their stuff off a 2.6.<even> release.
>>
>> It seems like a sensible approach, and it's not like the 2.4.x vs 2.5.x
>> kind of even/odd thing didn't _work_, the problems really were an
>> issue of
>> too big granularity making it hard for user and developers alike. So I
>> see
>> this as a tweak of the "let's drop the notion althogether for now"
>> decision, and just modify it to "even/odd is meaningful at all levels".
>>
>> In other words, we'd have an increasing level of instability with an
>> odd release number, depending on how long-term the instability is.
>>
>> - 2.6.<even>: even at all levels, aim for having had minimally
>> intrusive patches leading up to it (timeframe: a week or two)
>>
>> with the odd numbers going like:
>>
>> - 2.6.<odd>: still a stable kernel, but accept bigger changes leading
>> up to it (timeframe: a month or two).
>> - 2.<odd>.x: aim for big changes that may destabilize the kernel for
>> several releases (timeframe: a year or two)
>> - <odd>.x.x: Linus went crazy, broke absolutely _everything_, and rewrote
>> the kernel to be a microkernel using a special message-passing
>> version of Visual Basic. (timeframe: "we expect that he will be
>> released from the mental institution in a decade or two").
>>
>> The reason I put a shorter timeframe on the "all-even" kernel is
>> because I
>> don't want developers to be too itchy and sitting on stuff for too
>> long if
>> they did something slightly bigger. In theory, the longer the better
>> there, but in practice this release numbering is still nothing but a hint
>> of the _intent_ of the developers - it's still not a guarantee of "we
>> fixed all bugs", and anybody who expects that (and tries to avoid all
>> odd release entirely) is just setting himself up for not testing - and
>> thus bugs.
>>
>> Comments?
--
~Randy
Neil Brown
> >
> > So what was broken with the 2.6.8.1 type of 'hotfix kernel' release ?
>
> That's an alternative, of course.
>
> But that _is_ a branch, and does need active forward- and (mainly)
> backward-porting work.
>
> There's nothing wrong with it per-se, but it becomes a "stabilised version
> of the kernel.org tree" or even a "production version of the kernel.org
> tree". In other words it's somewhere on the line between the mainline
> kernel.org tree and a distribution. How far along that line should it
> be positioned?
I think there is a case for the "community" providing the most
"stable" kernel that it (reasonably) can without depending on
"distributions" to do that.
One reason is that (some) distributions are known to have released
kernels with quite broken and unreviewed patches, or with new
functionality that never ends up appearing in main-line for whatever
reason.
Further, it would surely be useful for all distributions to have one
central place that 'stablising' patches appear so they can
pick-and-choose from them rather than each keeping their own
independent set.
For the kernel, I am the "distribution" for my employer and I choose
which kernel to use, with which patches. I really don't want to hunt
around for all those stablisation patches, or sift through the patches
in 2.6.X+1-pre to find things to apply to 2.6.X. I would be really
happy there was a central place where maintainers can put suitably
reviewed "important bug fix"es for recent releases, and from where
kernel maintainers for any distribution (official or not) could pull
them.
Having said that, I am not in a position to offer my services to
maintain such a really-stable kernel branch, so I'll just cope with
whatever is provided.
NeilBrown
Russell Miller
> Then your above becomes:
> 2.6.x-rc: bugfixes only
> 2.6.x-pre: bugfixes and features
>
> And then that doesn't confuse end users either.
>
Speaking as an "ordinary" end user (there's nothing ordinary about me) I think
the idea of even/odd releases is silly. This accomplishes the same goals,
and is less confusing all told.
Linus's plan will work well for about... two releases, then people will wise
up and stop testing the odd releases. I know that's what I'll probably end
up doing.
--Russell
--
Russell Miller - rmi...@duskglow.com - Agoura, CA
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
Gene Heskett
>On Wed, 02 Mar 2005 19:29:35 -0500
>
>Jeff Garzik <jga...@pobox.com> wrote:
>
>The problem is people don't test until 2.6.whatever-final goes out.
>Nothing will change that.
Except more people who think like me. I usually enjoy playing the
canary, else I wouldn't do it.
More Canaries I say. I build, and run till the next rc comes out
almost every rc & I'm curently up on 2.6.11 for 5h:30m. I've only
got this one box thats fast enough to do that and still get anything
else done too, so this is the workhorse. I don't yelp about
deprecation notices during the build, I figure you people have eyes
too.
Basicly on my limited variety of hardware, I at least can play the
part of the canary here, and have a couple of times when problems
show up in less than a couple of days or during the build itself.
What you really need is a stronger plea for more people to go beat on
the rc releases, but I fear thats never going to happen as long as
there is nothing but the rc stuffs, and no -pre stuff for testing the
-mm stuff a little wider. Back in 2.4 days, even the -pre stuff was
often stable enough for extended uptimes, and I mean that I ran
May 1 2003 vmlinuz-2.4.21-pre7 on that 7.3 box until Jan 21 2005.
Its now got a 2.4.29 final on it and:
8:56pm up 23 days, 8:00, 6 users, load average: 2.12, 1.40, 1.14
sailing along just as smoothly as ever since Feb 7.
Not without a reboot over that nearly 2 years of course, but the
choice of when to do the reboot was mine in every case. Here on this
box, trying to play canary, uptimes are often less than a week,
sometimes less than 24 hours, and that isn't anywhere near long
enough to be able to say 'its stable'. Stuff is IMO, being shoved
out without an adequate amount of time to see what the 'canary' does.
If we could only get more people to 'bang on it, kick the tires etc',
AND fix those sorts of problems that do show up in the next
sequential rc, a lot of stuff like 2.6.8 could be headed off.
Note the caps on the word AND. That bugfix in the next rc release
seems to be broken when obviously needed fixes often have to go back
to the head of the queue and work their way back to Linus.
So how DO you go about getting the tires kicked by more users? I'm
not sure what would work.
One doesn't have to be a code monkey to do this 'canary' scene as long
as a bash script can be hacked up to do the majority of the work, I
have a couple of them that basicly make a new kernel install a no
brainer. Often under 30 minutes to being rebooted to a new rc from
going after the patch.
And I ramble, but this is my $0.02. And probably about what its worth
in the Grande Scheme of Things...
>And the day Linus releases we always get a pile of "missing
> MODULE_EXPORT()" type bug reports that are one liner fixes. Those
> fixes will not be seen by users until the next 2.6.x rev comes out
> and right now that takes months which is rediculious for such
> simple fixes.
I agree. Ditto for the 1394 fixes that have been upstream for at
least a month, maybe more. I haven't checked to see if kino can run
my movie camera yet, if it can, then maybe it made it into 2.6.11.
Otherwise I'll have to go nuke that dir in the drivers tree, and
again grab the svn from the 1394 site and rebuild/reinstall.
Provided the svn server is up, thats not a big deal here. The point
is that it is being tested, and it does work.
>We're talking about a one week "calming" period to collect the brown
> paper bag fixes for a 2.6.${even} release, that's all.
>
>All this "I have to hold onto my backlog longer, WAHHH!" arguments
> are bogus IMHO. We're using a week of quiescence to fix the tree
> for users so they are happy whilst we work on the 2.6.${odd}
> interesting stuff :-)
--
Cheers, Gene
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
99.34% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com attorneys please note, additions to this message
by Gene Heskett are:
Copyright 2005 by Maurice Eugene Heskett, all rights reserved.
Gene Heskett
>On Wed, 2 Mar 2005, Greg KH wrote:
>> I think this statement proves that the current development
>> situation is working quite well. The nasty breakage and details
>> got worked out in the -mm tree, and then flowed into your tree
>> when they seemed sane.
>
>Actually, the breakage I was talking about got fixed in _my_ tree.
>
>I'd love for the -mm tree to get more testing, but it doesn't.
>
something thats broken in an -mm release, I wasn't chased off to go
run a "more stable" release. Thats occured 2-3 times in the past
year.
I'll willingly play the canary as long as I don't wind up with a
totally hosed filesystem. So far, knock on wood, I've been fairly
lucky and have not had to do a bare metal recovery from amanda.
--
Cheers, Gene
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
99.34% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com attorneys please note, additions to this message
by Gene Heskett are:
Copyright 2005 by Maurice Eugene Heskett, all rights reserved.
Jeff Garzik
> On Wed, 02 Mar 2005 19:29:35 -0500
> Jeff Garzik <jga...@pobox.com> wrote:
>
>
>>If the time between big merges increases, as with this proposal, then
>>the distance between local dev trees and linux-2.6 increases.
>>
>>With that distance, breakages like the 64-bit resource struct stuff
>>become more painful.
>>
>>I like my own "ongoing dev tree, ongoing stable tree" proposal a lot
>>better. But then, I'm biased :)
>
>
> The problem is people don't test until 2.6.whatever-final goes out.
> Nothing will change that.
>
> And the day Linus releases we always get a pile of "missing MODULE_EXPORT()"
> type bug reports that are one liner fixes. Those fixes will not be seen by
> users until the next 2.6.x rev comes out and right now that takes months
> which is rediculious for such simple fixes.
>
> We're talking about a one week "calming" period to collect the brown paper
> bag fixes for a 2.6.${even} release, that's all.
If we want a calming period, we need to do development like 2.4.x is
done today. It's sane, understandable and it works.
2.6.x-pre: bugfixes and features
2.6.x-rc: bugfixes only
> All this "I have to hold onto my backlog longer, WAHHH!" arguments are bogus
> IMHO. We're using a week of quiescence to fix the tree for users so they
> are happy whilst we work on the 2.6.${odd} interesting stuff :-)
If you think it will be only a week, you're deluding yourself. It will
stretch out to a month or longer, and the backlog problems will be real.
A calming period is fine. But this even/odd mess is just silly.
Jeff
Chris Wright
> So what was broken with the 2.6.8.1 type of 'hotfix kernel' release ?
I agree, I think that's useful and needed. It's possible to get the
fixes committed to an effective branch in bk and pull that back into
mainline. So at each new release the last release hotfix branch would
stop. But does that solve a different problem? I don't think it'll
increase people testing intended-stable versions.
thanks,
-chris
Jeff V. Merkey
>On Wed, 2 Mar 2005, Jeff V. Merkey wrote:
>
>
>
>>__Stable__ would be a good thing. The entire 2.6 development has been a
>>disaster from a stability viewpoint. I have to maintain a huge tree of
>>patches in order to ship appliance builds due to the lack of stability
>>for 2.6. I think that the even number releases will take longer but it's
>>worth the wait.
>>
>>
>
>What form of instability are you referring to?
>
>
>
Where have you been? Read the bug reports and the types of bugs.
Jeff
Paul Jackson
in the cycle, to when that same point comes around again.
In 2.4 and before, the cycle time was a long time, I hear tell.
Perhaps, at some points, things were sufficiently chaotic that it
was difficult to discern any particular cycle time (FFT of releases
resembled pink noise ;?).
In 2.6.x since June 2004, the cycle time has been two months. Cool.
My employer (SGI) has to push new features through the main Linux
kernel, and then through the Linux distributors we work with, such as
SuSE and Red Hat, before they reach our customers. And we tend to live
on the leading edge, so depend on these features.
The longer the new feature cycle time in Linux, then the longer on the
average (depending on just how we line up) it takes us to get a feature
to our customers.
I really like the two month cycle we have now. It's fast.
If your 2.6.<odd>, 2.6.<even> proposal extended that cycle time to four
months (two months <odd>, two months <even>), that would hurt. Not the
end of Planet Earth, but still an owie.
The times you give, of a month or two for the <odd>, and a week or two
for the <even>, if taken literally, add up to roughly the same two
months, give or take a few weeks. This is good.
But I've long since learned to take initial time estimates from
engineers (and from my wife, when we're getting ready to go out) with
a few grains of salt.
How serious are you about these time estimates?
--
I won't rest till it's the best ...
Programmer, Linux Scalability
Paul Jackson <p...@sgi.com> 1.650.933.1373, 1.925.600.0401
Jeff Garzik
is a tacit acknowledgement that people only _really_ test the official
releases.
Which IMHO backs up my opinion that we simply need more frequent releases.
Part of this is a scalability problem. Linux probably has more changes
flowing into it than any other OS kernel on the planet. We must deal
with an ever-increasing number of changesets in a way that produces a
usable kernel for our users.
Jeff
Jeff Garzik
> On Wednesday March 2, ak...@osdl.org wrote:
>
>>Only davem, AFAIK. All the other trees get auto-sucked into -mm for
>>testing.
>
>
> Ok, I got the feeling it was more wide spread than that, but I
> apparently misread the signs (people mentioning that had 'patches
> queued for Linus' and such).
I don't think you misread, necessarily.
Andrew auto-sucks trees, which is different from sprinkling Holy Penguin
Pee on the trees.
Jeff
Andrew Morton
>
> On Wednesday March 2, ak...@osdl.org wrote:
> >
> > Only davem, AFAIK. All the other trees get auto-sucked into -mm for
> > testing.
>
> Ok, I got the feeling it was more wide spread than that, but I
> apparently misread the signs (people mentioning that had 'patches
> queued for Linus' and such).
I'd like to get my sticky paws onto Dave's tree actually, but haven't got
around to hassling. He doesn't buffer much, or for very long anwyay.
> > Generally the owners of those trees make the decision as to which
> > of their code has been sufficiently well-tested for a Linus merge, and when
> > that should happen.
>
> I wonder if there is a problem here.
> Who is in a position to judge when a patch is ready to be merged?
> I suspect that it would be hard for a developer to be objective about
> the readiness of their own patches (I know all my patches are perfectly
> ready the moment I create them, despite what experience tells me :-)
True. But if someone tries a big push when we're after -rc1 it will get
extra attention. Basically, nobody does that.
> Assuming we have a 'stable', a 'testing' and a 'devel' series
> (whatever actual numbering gets used for them(*)), then maybe it is ok
> for a developer to judge if it is ready for 'testing', but it should
> really have either some minimum time in 'testing' or independent
> review before being allowed into 'stable'.
>
> Are you and Linus able to handle the independent review load?
No. That's why I'm always spamming people and asking them to ack stuff.
I don't think Linus or I pay much attention to patches which come in via
-bk trees - if the owners of those trees break them, they get to fix them.
That does leave open the problem that people can merge crappy code which
happens to work. That's what hch is for ;)
Really, to a large extent the kernel doesn't use the "dictator" model any
more. It's more like the "various people have commit permissions" model,
only they don't actually autonomously do a physical commit.
> Should every developer/maintainer find someone to review any patches
> that they think should 'jump the queue'.
Queue-jumping is a pain, and we should only do it with good reason.
That being said, kernel development is remarkably decoupled, in that there
is rarely overlap between the various subsystems. Otherwise I couldn't sit
on up to 900 patches all the time. Sometimes there is overlap between the
32 bk trees, and I'll usually tell people so that it gets sorted out.
Reviewing is important, and it's a bit of a problem that patches can get
merged into the mainline tree via this particular path:
developer
-> subsystem maintainer (maybe cc'ed to obscure mailing list)
-> Couple of weeks in -mm for testing
-> Linus via bk pull
This bypasses the main review site: linux-kernel. Because the people on
<obscure mailing list> may not have the motivation/skills/experience. And
things do sneak through. Greg usually cc's linux-kernel on stuff as he
sends it to Linus, which is good. But it's a problem.
Good changelogging really helps the review process. If the submitter can
skilfully tell the reviewer what the patch does, why and how then the
review process becomes largely a matter of checking that the patch does
what he meant it to do. The _understanding_ process happens while you read
the changelog. In an ideal world.
> (Would anyone like to review my nfs/raid patches for me? I review
> patches I get from others, but find it very hard to review my own
> work. Andrew does a good job, but does miss things sometimes).
Yes, it's hard. If one is lucky, it's "hey, that locking looks funny".
And, usually, "your coding style is sucky". But apart from that, one needs
to be pretty heavily involved in the particular area to comment usefully.
So I think your question should be rephrased as "would anyone else like to
become nfsd/raid developers".
Christoph Hellwig
> I think there is a case for the "community" providing the most
> "stable" kernel that it (reasonably) can without depending on
> "distributions" to do that.
The point is that it's happening anyway. See Andres' -as tree which
is the basis for the Debian vendor kernel. Getting that up to an
official status as 2.6.x.y would be very nice (and having it on
linux.bkbits.net)
Ben Greear
> I also note that part of the problem that motivates the even/odd thing
> is a tacit acknowledgement that people only _really_ test the official
> releases.
>
> Which IMHO backs up my opinion that we simply need more frequent releases.
That doesn't really help in my opinion. We need the comfort of knowing
that if we do find a bug, then soon we will have a release that will fix
this bug, with a *very* high probability of not changing anything else
significantly, or adding new regressions. The current wait between
the official releases is too long to wait for a bug fix, but it is also likely
that other regressions have slipped in due to the large churn in the code.
If you simply release faster, without doing bug-fix only releases, then
I think we'll continue to see regressions.
I think the 2.6.X.y release scheme could work, as others have mentioned.
--
Ben Greear <gre...@candelatech.com>
Candela Technologies Inc http://www.candelatech.com
-
Andrew Morton
>
> Ditto for the 1394 fixes that have been upstream for at
> least a month, maybe more.
-mm always holds the latest 1394 tree. So you can run -mm, or just snarf
bk-ieee1394.patch from the broken-out directory.
Randy.Dunlap
> On Wed, Mar 02, 2005 at 04:00:46PM -0800, Linus Torvalds wrote:
>
> > I would not keep regular driver updates from a 2.6.<even> thing.
>
> Then the notion of it being stable is bogus, given how many regressions
> the last few kernels have brought in drivers. Moving from 2.6.9 -> 2.6.10
> broke ALSA, USB, parport, firewire, and countless other little bits and
> pieces that users tend to notice.
>
> The reason that things like 4-level page tables worked out so well
> was that it was tested in -mm for however long, so by the time it got
> to your tree, the silly bugs had already been shaken out.
>
> Compare this to random-driver-update. -mm testing is a valuable
> proving ground, but its no panacea to stability. There's no guarantee
> that someone with $affected_device even tried a -mm kernel.
>
> For it to truly be a stable kernel, the only patches I'd expect to
> drivers would be ones fixing blindingly obvious bugs. No cleanups.
> No new functionality. I'd even question new hardware support if it
> wasn't just a PCI ID addition.
Maybe I don't understand? Is someone expecting distro
quality/stability from kernel.org kernels?
I don't, but maybe I'm one of those minorities.
--
~Randy
Neil Brown
> A Linus based odd number
> might be closer to that if we hope on people unwittingly running them.
I think this is a very unhelpful attitude. Don't expect people to do
things unwittingly. It won't work.
You should expect people to be more intelligent and more confused than
you expect (yes, I know that is recursive).
Being more intelligent means you won't be able to trick them.
Being more confused means they will try out less things.
Constantly changing the naming will confuse people, and they will
experiment less.
Having some clear and often-stated gaols for different releases -
which are adhered to - will make people feel less confused and so more
willing to experiment. Asking nicely probably helps too.
--e.g.--
The latest 2.6.X is stable. Feel free to use it for any system.
The latest 2.6.X-pre is fairly stable. Please consider using it on
non-"mission critical" system.
The latest 2.6.X-mmY is under development. Please test it if you have
the opportunity.
Agree on that. Stick to it. Get everyone to plaster it on their
websites. Stick it at the bottom of all @vger.kernel.org mailing
lists. See what happens.
NeilBrown
Linus Torvalds
On Wed, 2 Mar 2005, Jeff Garzik wrote:
>
> If we want a calming period, we need to do development like 2.4.x is
> done today. It's sane, understandable and it works.
No. It's insane, and the only reason it works is that 2.4.x is a totally
different animal. Namely it doesn't have the kind of active development AT
ALL any more. It _only_ has the "even" number kind of things, and quite
frankly, even those are a lot less than 2.6.x has.
> 2.6.x-pre: bugfixes and features
> 2.6.x-rc: bugfixes only
And the reason it does _not_ work is that all the people we want testing
sure as _hell_ won't be testing -rc versions.
That's the whole point here, at least to me. I want to have people test
things out, but it doesn't matter how many -rc kernels I'd do, it just
won't happen. It's not a "real release".
In contrast, making it a real release, and making it clear that it's a
release in its own right, might actually get people to use it.
Might. Maybe.
Linus
Jeff Garzik
>
> On Wed, 2 Mar 2005, Jeff Garzik wrote:
>
>>If we want a calming period, we need to do development like 2.4.x is
>>done today. It's sane, understandable and it works.
>
>
> No. It's insane, and the only reason it works is that 2.4.x is a totally
> different animal. Namely it doesn't have the kind of active development AT
> ALL any more. It _only_ has the "even" number kind of things, and quite
> frankly, even those are a lot less than 2.6.x has.
>
>
>>2.6.x-pre: bugfixes and features
>>2.6.x-rc: bugfixes only
>
>
> And the reason it does _not_ work is that all the people we want testing
> sure as _hell_ won't be testing -rc versions.
>
> That's the whole point here, at least to me. I want to have people test
> things out, but it doesn't matter how many -rc kernels I'd do, it just
> won't happen. It's not a "real release".
People don't test 2.6-rc releases because they know they are not
"release candidate, with only bug fixes" releases, which is how the rest
of the world interprets the phrase.
Making them official releases in the even/odd manner is what neilb
implies. You'll just be diminishing the value of releases. A "real
release" won't be a real release anymore. You're just renaming the -rc
that isn't really an -rc.
Jeff
Nigel Cunningham
My first response is: this is a recipe for great confusion among users.
I'd far rather see things only make it into your tree when they've been
thoroughly tested (in -mm and prior to that). Following that strategy,
your tree could always be relied upon to be stable and -rcs would only
needed for dealing with the unforeseen interactions between otherwise
mature patches.
Regards,
Nigel
--
Nigel Cunningham
Software Engineer, Canberra, Australia
http://www.cyclades.com
Bus: +61 (2) 6291 9554; Hme: +61 (2) 6292 8028; Mob: +61 (417) 100 574
Maintainer of Suspend2 Kernel Patches http://softwaresuspend.berlios.de
Ryan Anderson
> The problem with major development trees like 2.4.x vs 2.5.x was that the
> release cycles were too long, and that people hated the back- and
> forward-porting. That said, it did serve a purpose - people kind of knew
> where they stood, even though we always ended up having to have big
> changes in the stable tree too, just to keep up with a changing landscape.
How about this idea, instead:
At 2.6.12, make two parallel BitKeeper trees:
2.6 and 2.7
Push bugfixes into 2.6.
Push *everything* that's not a bugfix into 2.7.
"bk pull" the 2.6 tree into the 2.7 tree each day when you wake up.
A week later, release 2.6.12.1 from the 2.6 tree, then immediately bk
pull the 2.7 tree into the 2.6 tree.
Release 2.6.13-rc1 at about the same time, and again only take bugfixes
into the 2.6 tree.
In 3-7 weeks, after a few more -rc iterations with just bugfixes,
release 2.6.13.
This should keep the differences between the trees down to something
somewhat... sane, and, hopefully, keep the stable tree stable.
People working on big changes can do them against 2.6.x if they need
stability, and know that if they stay current with each 2.6.x release as
they work, they should have a controllable amount of pain for merges.
My thinking is simply that if you're going to use BitKeeper, you might
as well abuse it for all that it's worth.
--
Ryan Anderson
sometimes Pug Majere
Dave Jones
>
> it a couple of times, I've talked to some people about it, and today Davem
> sent a suggestion along similar lines to me for 2.6.12.
>
> Namely that we could adopt the even/odd numbering scheme that we used to
> do on a minor number basis, and instead of dropping it entirely like we
> did, we could have just moved it to the release number, as an indication
Random ideas. Why not use the already established 2.6.x.y method ?
This allows development to continue on 2.6.x+1, and if needed, you
should be able to bk pull the 2.6.x.y[n] tree(s) into 2.6.x+1 no ?
My concern about this 'stable kernel every other release' method is
that if a particular development cycle drags on for some reason,
and certain bugs never got fixed in the previous release,
that's a long time users have to wait until they get things fixed.
It's somewhat akin to the problem with the infrequent out-of-tree merges
that some subsystem maintainers have. If the latest push they do
doesn't fix your bug, you typically have to wait until the next
release until they do another push.
Dave
Dave Jones
> >For it to truly be a stable kernel, the only patches I'd expect to
> >drivers would be ones fixing blindingly obvious bugs. No cleanups.
> >No new functionality. I'd even question new hardware support if it
> >wasn't just a PCI ID addition.
>
> Maybe I don't understand? Is someone expecting distro
> quality/stability from kernel.org kernels?
My complaint is the charade of calling it 'stable' when it clearly
wouldn't be anything of the sort, given that a majority of the bugs our
users experienced on rebasing were driver related.
The core itself may be rock-solid, but if we're continually pulling
in random driver updates of questionable quality with limited
testing, the result as a whole isn't stable.
> I don't, but maybe I'm one of those minorities.
Putting the onus on distributions to make things stable is no
excuse for the ever-increasing number of regressions each release.
This might sound over-dramatic, but it's the current state as far
as I'm concerned. The 2.6.8->2.6.9 update for Fedora users brought
a bunch of carnage that took time to shake out. 2.6.9->2.6.10 I'm
still picking up the pieces of. If the 2.6.10->2.6.11 update that
I'll do for Fedora in a week or two turns out to have less regressions
than the previous releases, I'll be stunned. Really.
Already I'm wondering how many userspace packages are going to randomly
stop working as they have done in previous releases. With the
clear delineation of stable/development, we were able to say things
like "we won't change a user visible interface in a stable series"
Now, we don't have that. So we find things ranging from slabtop to
alsa-lib completely break unless you update the userspace too.
regressions like this is what I'm bitching about. There's nothing
a vendor can do to make such things stable (other than dropping
the various patches that introduce the breakage, but at ~4000 csets
per release right now, there will be stuff that gets missed).
Whilst the slabinfo example was a non-driver related regression,
it's a good example of how little care we're taking these days
to make sure existing userspace continues to work correctly.
Some may suggest the close tracking of mainline is the problem.
Maybe they're right. Maybe we should have stuck with a 2.6.5 kernel
until Fedora Core 2 reached end of life, and gone with the old
'have hundreds and hundreds of patches piling up' approach.
But, as someone who has maintained vendor kernels that have tried
both methods, the sticking close to mainline approach wins hands down.
If something is broken, more often than not, I can bug the upstream
developer and ask "hey, this is a wierd problem our fedora users hit,
we don't have any patches against this code, can you take a look?"
and developers have been very responsive, and helpful on many occasions,
ultimatly leading bugs being fixed both in our kernel, and upstream.
If I asked most upstream developers about a problem we've been facing
with our 2.6.5 kernels, I'd get a much less helpful response.
And rightly so. In their position I'd do exactly the same thing.
Dave
Dmitry Torokhov
<torv...@osdl.org> wrote:
>
> Comments?
>
Just rename:
2.<even>.<odd>-rcX -> 2.<even>.y-preX
2.<even>.<odd> -> 2.<even>.y-rcX
2.<even>.<even> -> 2.<even>.y
--
Dmitry
David S. Miller
Jeff Garzik <jga...@pobox.com> wrote:
> People don't test 2.6-rc releases because they know they are not
> "release candidate, with only bug fixes" releases, which is how the rest
> of the world interprets the phrase.
That's not %100 true. No matter what -rc* really is, people perceive
it to be something based upon it's name and whether or not it is an
official release. As far as I can see it's %95 perception vs. reality.
And IT IS part of the reason the -rc's are not as good as they could be.
David S. Miller
Jeff Garzik <jga...@pobox.com> wrote:
> I also note that part of the problem that motivates the even/odd thing
> is a tacit acknowledgement that people only _really_ test the official
> releases.
>
> Which IMHO backs up my opinion that we simply need more frequent releases.
This more frequent release idea will basically mimmick what the
even/odd idea will do, except that even/odd will have specific
engineering goals. Development vs. pure bug fixing.
There are changes that simply have to sit for weeks if not months
in order for all the problems to be weeded out. I think something
like the 4-level pagetable stuff is the best example. And yes it
has to occur in Linus's tree so what precious few people do test
his live tree try out the new code.
Randy.Dunlap
> On Wed, Mar 02, 2005 at 07:10:47PM -0800, Randy.Dunlap wrote:
>
> > >For it to truly be a stable kernel, the only patches I'd expect to
> > >drivers would be ones fixing blindingly obvious bugs. No cleanups.
> > >No new functionality. I'd even question new hardware support if it
> > >wasn't just a PCI ID addition.
> >
> > Maybe I don't understand? Is someone expecting distro
> > quality/stability from kernel.org kernels?
>
> My complaint is the charade of calling it 'stable' when it clearly
> wouldn't be anything of the sort, given that a majority of the bugs our
> users experienced on rebasing were driver related.
> The core itself may be rock-solid, but if we're continually pulling
> in random driver updates of questionable quality with limited
> testing, the result as a whole isn't stable.
>
> > I don't, but maybe I'm one of those minorities.
>
> Putting the onus on distributions to make things stable is no
> excuse for the ever-increasing number of regressions each release.
Sure, I'm not trying to put the onus on distros, just saying
that they add some real value there, but that doesn't excuse
us from trying to make the mainline kernel as good as we can
make it.
--
~Randy
Jeff Garzik
> On Wed, 02 Mar 2005 22:40:57 -0500
> Jeff Garzik <jga...@pobox.com> wrote:
>
>
>>People don't test 2.6-rc releases because they know they are not
>>"release candidate, with only bug fixes" releases, which is how the rest
>>of the world interprets the phrase.
>
>
> That's not %100 true. No matter what -rc* really is, people perceive
> it to be something based upon it's name and whether or not it is an
> official release. As far as I can see it's %95 perception vs. reality.
> And IT IS part of the reason the -rc's are not as good as they could be.
The reasons -rcs are not as good as they could be is that they include
more than just bug fixes. Users are discouraged from testing because
they must scan LKML, or guess, which -rc that Linus/Andrew started
getting serious about "bugfixes only."
With the -pre/-rc scheme, it's clear to users.
With the even/odd scheme, you just devalue releases. Previously, all
releases were worthy of testing and use. Now, half of them aren't.
Jeff
Chris Wedgwood
> - 2.6.<even>: even at all levels, aim for having had minimally intrusive
> patches leading up to it (timeframe: a week or two)
>
> with the odd numbers going like:
>
> - 2.6.<odd>: still a stable kernel, but accept bigger changes leading up
> to it (timeframe: a month or two).
> - 2.<odd>.x: aim for big changes that may destabilize the kernel for
> several releases (timeframe: a year or two)
[...]
Why not change the "2.6 prefix" to 2.8, 3.0 (or whatever) if/when you
do go to a new naming scheme --- simply to make a clean break between
the new and the old. Plus it will give the suckdork crowd[1] bigger
numbers to drivel on about.
That said it would be a large numerical leap without and real feature
changes so perhaps that will further add to confusion?
Sigh.
[1] Well, and the CGL and similar people. "New CGL with improved
version numbers and fewer calories!"
Andrew Morton
>
> On Wed, Mar 02, 2005 at 04:00:46PM -0800, Linus Torvalds wrote:
>
> > I would not keep regular driver updates from a 2.6.<even> thing.
>
> Then the notion of it being stable is bogus, given how many regressions
> the last few kernels have brought in drivers. Moving from 2.6.9 -> 2.6.10
> broke ALSA, USB, parport, firewire, and countless other little bits and
> pieces that users tend to notice.
Grump. Have all these regressions received the appropriate level of
visibility on this mailing list?
I too am a little put off by the number of regressions which certain
(admittedly tricky) subsystems keep on introducing. One does wonder how
careful people are being at the development stage. And that's a thing
which we can surely fix.
For example, here's today's crop (so far):
include/linux/soundcard.h:195: warning: `_PATCHKEY' redefined
include/linux/awe_voice.h:33: warning: this is the location of the previous definition
drivers/i2c/chips/ds1337.c:60: `I2C_DRIVERID_DS1337' undeclared here (not in a function)
drivers/i2c/chips/ds1337.c:60: initializer element is not constant
drivers/i2c/chips/ds1337.c:60: (near initialization for `ds1337_driver.id')
drivers/i2c/chips/ds1337.c: In function `ds1337_get_datetime':
drivers/i2c/chips/ds1337.c:155: structure has no member named `id'
drivers/i2c/chips/ds1337.c: In function `ds1337_set_datetime':
drivers/i2c/chips/ds1337.c:206: structure has no member named `id'
drivers/i2c/chips/ds1337.c: In function `ds1337_detect':
drivers/i2c/chips/ds1337.c:333: structure has no member named `id'
drivers/i2c/chips/ds1337.c:343: structure has no member named `id'
make[3]: *** [drivers/i2c/chips/ds1337.o] Error 1
make[2]: *** [drivers/i2c/chips] Error 2
make[1]: *** [drivers/i2c] Error 2
make[1]: *** Waiting for unfinished jobs....
sound/pci/als4000.c: In function `snd_als4000_create_gameport':
sound/pci/als4000.c:572: warning: `r' might be used uninitialized in this function
drivers/char/watchdog/alim1535_wdt.c:319: warning: `ali_pci_tbl' defined but not used
drivers/char/sx.c:255: warning: `sx_pci_tbl' defined but not used
drivers/char/applicom.c:68: warning: `applicom_pci_tbl' defined but not used
make: *** [drivers] Error 2
make: *** Waiting for unfinished jobs....
sound/pci/trident/trident_main.c: In function `snd_trident_gameport_trigger':
sound/pci/trident/trident_main.c:3125: warning: `return' with a value, in function returning void
> For it to truly be a stable kernel, the only patches I'd expect to
> drivers would be ones fixing blindingly obvious bugs. No cleanups.
> No new functionality. I'd even question new hardware support if it
> wasn't just a PCI ID addition.
Agree.
Jeff Garzik
preferable to even/odd.
Just create a 2.6.X repo at each release. For bug fixes to 2.6.X,
commit to this repo, then pull into linux-2.6. For everything else,
pull straight into linux-2.6.
The linux-2.6 repo would be upstream, and linux-2.6.X repo would be
where bugfix-only releases come from.
Gene Heskett
>Gene Heskett <gene.h...@verizon.net> wrote:
>> Ditto for the 1394 fixes that have been upstream for at
>> least a month, maybe more.
>
>-mm always holds the latest 1394 tree. So you can run -mm, or just
> snarf bk-ieee1394.patch from the broken-out directory.
>
it does need to be pushed on down the line.
--
Cheers, Gene
"There are four boxes to be used in defense of liberty:
soap, ballot, jury, and ammo. Please use in that order."
-Ed Howdershelt (Author)
99.34% setiathome rank, not too shabby for a WV hillbilly
Yahoo.com attorneys please note, additions to this message
by Gene Heskett are:
Copyright 2005 by Maurice Eugene Heskett, all rights reserved.
David S. Miller
Jeff Garzik <jga...@pobox.com> wrote:
> If Linus/DaveM really don't like -pre/-rc naming, I think 2.6.x.y is
> preferable to even/odd.
All of these arguments are circular. If people think that even/odd
will devalue odd releases, guess what 2.6.x.y will do? By that line
of reasoning nobody will test 2.6.x just the same as they aren't
testing 2.6.x-rc* right now.
I think they will test the odd releases, because as a real release
they will get slashdot/lwn.net/etc. announcements.
That's one of the major things the -rc's don't get. Maybe it gets
a reference in lwn.net's weekly kernel article, but mostly kernel
geeks read those and that's not who we want testing -rc's (such
geeks already are doing so).
It has to be a "real" release. That does have an impact. However,
I am ambivalent about how to make them real. Even/odd, 2.6.x.y,
either is fine with me.
Valdis.K...@vt.edu
> I also note that part of the problem that motivates the even/odd thing
> is a tacit acknowledgement that people only _really_ test the official
> releases.
>
> Which IMHO backs up my opinion that we simply need more frequent releases.
Or more testers of things other than <whatever we call stable this week>
(which is where I see the *real* problem as being).
It doesn't matter *what* we call the "testing" releases. They aren't getting
tested enough. I've posted a fair number of "the latest -mm broke FOO" notes -
but assuming that "the first guy to hit it" is randomly distributed across all
the -mm users, there really can't be a whole lot of others doing it, as my
number seems to come up waaay too often if there's a large pool of testers...
Either that, or I really *am* the most bleeding-edge loon in my time zone.
Russell Miller
> That's the whole point here, at least to me. I want to have people test
> things out, but it doesn't matter how many -rc kernels I'd do, it just
> won't happen. It's not a "real release".
>
> In contrast, making it a real release, and making it clear that it's a
> release in its own right, might actually get people to use it.
>
> Might. Maybe.
>
Linus, I respect all of the work you have done on the Linux kernels over the
years, and I have been an avid user of Linux almost since its inception (when
I could get it to work with the hardware, at least in the early days ;-) And
the fact that my contributions to the kernel are almost nonexistent probbly
means you won't pay attention to a word I say anyway :-) that's alright, I'm
going to say it and you can listen if you want.
My respect for your accomplishments is why it pains me a great deal to have to
tell you that I think you're wrong.
I agree with the first part of your mail that I quoted above. Indeed, the -rc
releases are not a "real release", and therefore people aren't going to test
it.
What you are missing is that if you use the method you have proposed. odd
numberered kernels will stop being a "real release" as well to a great deal
of users.
I don't think you will actually gain anything here except for just changing
the kernel naming scheme yet *again*. I certainly don't think you're going
to solve the problem you are trying to solve.
The problem as stated is that people are not downloading and testing the test
releases.
Your solution to that problem is to make test releases look like real releases
and maybe people will test them anyway.
The solution should be to find a way to encourage people to download and test
the test releases. Perhaps a "bug bounty" of some kind (it doesn't have to
be money), or something similar, may prove to be a better motivator than
trying to trick the userbase. It's not going to work. If you take the
motivational approach, then it won't matter what you name the test releases,
people will test them anyway.
Several ideas right off the top of my head:
- a "bug bounty" as I mentioned above.
- a volunteer army of people, similar to the "kernel janitors", whose job is
to do QA for the kernel.
--Russell
--
Russell Miller - rmi...@duskglow.com - Agoura, CA
Russell Miller
> That's one of the major things the -rc's don't get. Maybe it gets
> a reference in lwn.net's weekly kernel article, but mostly kernel
> geeks read those and that's not who we want testing -rc's (such
> geeks already are doing so).
>
How do you know that they won't stop the announcements if this change is made?
--Russell
--
Russell Miller - rmi...@duskglow.com - Agoura, CA
Valdis.K...@vt.edu
> __Stable__ would be a good thing. The entire 2.6 development has been a
> disaster from
> a stability viewpoint. I have to maintain a huge tree of patches in
> order to ship appliance
> builds due to the lack of stability for 2.6. I think that the even
> number releases will take longer
> but it's worth the wait.
Remember - each patch you successfully push upstream is one less patch
you have to maintain. So re-read Documentation/SubmittingPatches, beat
the patches into shape, and send them on their way. ;)
Dave Jones
> Dave Jones <da...@redhat.com> wrote:
> >
> > On Wed, Mar 02, 2005 at 04:00:46PM -0800, Linus Torvalds wrote:
> >
> > > I would not keep regular driver updates from a 2.6.<even> thing.
> >
> > Then the notion of it being stable is bogus, given how many regressions
> > the last few kernels have brought in drivers. Moving from 2.6.9 -> 2.6.10
> > broke ALSA, USB, parport, firewire, and countless other little bits and
> > pieces that users tend to notice.
>
> Grump. Have all these regressions received the appropriate level of
> visibility on this mailing list?
For the most part these things are usually known about by their upstream
authors. To give an example: ALSA update in 2.6.10 broke sound for a
bunch of IBM thinkpads. As it turns out there are quite a lot of these
out there, so when I released a 2.6.10 update for Fedora, bugzilla lit
up like a christmas tree with "Hey, where'd my sound go?" reports.
(It was further confused by a load of other sound card problems, but
thats another issue). This got so out of control, Alan asked the ALSA
folks to take a look, and iirc Takashi figured out what had caused the
problem, and it got fixed in the ALSA folks tree, and subsequently, in 2.6.11rc.
Now, during all this time, there hadn't been any reports of this problem
at all on Linux-kernel. Not even from Linus' tree, let alone -mm.
Which amazes me given how widespread the problem was.
> I too am a little put off by the number of regressions which certain
> (admittedly tricky) subsystems keep on introducing. One does wonder how
> careful people are being at the development stage. And that's a thing
> which we can surely fix.
> For example, here's today's crop (so far):
>
> include/linux/soundcard.h:195: warning: `_PATCHKEY' redefined
> include/linux/awe_voice.h:33: warning: this is the location of the previous definition
compile time regressions we should be able to nail down fairly easily.
(someone from OSDL is already doing compile stats and such on each release
[too bad they're mostly incomprehensible to the casual viewer])
The bigger problem is runtime testing. If things aren't getting the
exposure they need, we're going to get screwed over by something or other
every time Linus bk pull's some random driver repo.
Dave
Randy.Dunlap
I'll ding the OSDL release builder about that (I agree with you)....
--
~Randy
David S. Miller
Russell Miller <rmi...@duskglow.com> wrote:
> How do you know that they won't stop the announcements if this change is made?
Nobody knows such things for sure, let's test it and find out :-)
Willy Tarreau
100% agree with you, Jeff. That's what I wrote in another mail.
A real -rc should have only a handful of patches. And even more
importantly, the final release MUST be EXACTLY the lastest -rc,
without any new surprize.
Willy
Willy Tarreau
> On Wed, 2 Mar 2005, Greg KH wrote:
> > I do understand what you are trying to achieve here, people don't really
> > test the -rc releases as much as a "real" 2.6.11 release. Getting a
> > week of testing and bugfix only type patches to then release a 2.6.12
> > makes a lot of sense. For example, see all of the bug reports that came
> > out of the woodwork today on lkml from the 2.6.11 release...
>
> A large part of it is psychological. On the other hand, it may be that
> Neil is right and it would just mean that people wouldn't even test the
> odd releases (..because they want to wait a couple of weeks for the even
> one), so it may not actually end up helping much.
It's not the same thing to test ONE release and test SEVERAL -rc. I take
my case as an example. I don't have enough time to keep up, so I try to
compile at least each release and test them on one of my systems, maybe
for curiosity. I would like to test the -rc, but the problem is that you
don't know how much you can expect this or that -rc to be close to usable.
You even expect it to fail or not build at all, so when you don't have much
time to spend on this, unfortunately, you don't test them.
On the contrary, Marcelo makes a strong difference between -pre and -rc.
And when I see a 2.4-rc, I expect it to be a potential candidate, so I
try to get some time to compile it just in case I would discover an awful
bug, and avoid the 2.6.8 -> 2.6.8.1 situation. Honnestly, I would have
reported the 2.6.8 NFS bug earlier if there had been a true -rc before
it (=one which does not change except for trivial bug fixes).
> The thing is, I _do_ believe the current setup is working reasonably well.
> But I also do know that some people (a fairly small group, but anyway)
> seem to want an extra level of stability - although those people seem to
> not talk so much about "it works" kind of stability, but literally a "we
> can't keep up" kind of stability (ie at least a noticeable percentage of
> that group is not complaining about crashes, they are complaining about
> speed of development).
>
> And I suspect that _anything_ I do won't make those people happy.
The only solution against this is to freeze for real and start the devel
tree in parallel. People are not complaining anymore about 2.4 change
speed, because every time a folk sends something, we tell him to push
that into 2.6 and not 2.4. The same thing must work with 2.6 and 2.7.
In the end, there would a general feeling that "2.6 was not as stable
as 2.8", etc... That's not a problem, there are always ups and downs
in software.
Regards,
Willy
Hua Zhong
> want testing sure as _hell_ won't be testing -rc versions.
At least they still test "real" releases..
So instead of making sure rc is really "release-candidate", we want to trick
people to test -pre as "real release", soon people will realize it and just
stop testing even real releases. The trick won't last. What other names will
we try then? Seriously, this is silly and the focus is wrong, and will
accomplish nothing but confusing people one more time.
Let's start by making rc really "release-candidate", not something with
last-minute unpredictable random changes. Why should I test rc when I know
the final release will be much different anyway? What is worse is that we
actually _complain_ about the fact that people do not take rc seriously,
while the release management doesn't take it seriously itself.
Hua
Jeff Garzik
> On Wed, 02 Mar 2005 23:46:22 -0500
> Jeff Garzik <jga...@pobox.com> wrote:
>
>
>>If Linus/DaveM really don't like -pre/-rc naming, I think 2.6.x.y is
>>preferable to even/odd.
>
>
> All of these arguments are circular. If people think that even/odd
> will devalue odd releases, guess what 2.6.x.y will do? By that line
> of reasoning nobody will test 2.6.x just the same as they aren't
> testing 2.6.x-rc* right now.
even/odd means that certain releases (even ones) are more magical than
others. That's weird, since users aren't used to that sort of thing in
any other project.
2.6.x.y and 2.6.x-rc [where rc == serious bugfixes only!] are
understandable to users, because they have seen that sort of thing before.
Users _aren't_ fooled by naming games. The current 2.6-rc proves that.
> I think they will test the odd releases, because as a real release
> they will get slashdot/lwn.net/etc. announcements.
>
> That's one of the major things the -rc's don't get. Maybe it gets
> a reference in lwn.net's weekly kernel article, but mostly kernel
> geeks read those and that's not who we want testing -rc's (such
> geeks already are doing so).
LWN, Slashdot and others will not be fooled though :) They will note
that release 2.6.<odd> is not a real release.
> It has to be a "real" release. That does have an impact. However,
> I am ambivalent about how to make them real. Even/odd, 2.6.x.y,
> either is fine with me.
2.6.x.y has a very real engineering benefit: it becomes a stable
release branch. That will encourage even more users to test it, over
and above a simple release naming change.
Users have been clamoring for a stable release branch in any case, as
you see from comments about Alan's -ac and an LKML user's -as kernels.
Jeff
Greg KH
> 2.6.x.y has a very real engineering benefit: it becomes a stable
> release branch. That will encourage even more users to test it, over
> and above a simple release naming change.
>
> Users have been clamoring for a stable release branch in any case, as
> you see from comments about Alan's -ac and an LKML user's -as kernels.
Sure they've been asking for it, but I think they really don't know what
it entails. Look at all of the "non-stable" type patches in the -ac and
as tree. There's a lot of stuff in there. It's a slippery slope down
when trying to say, "I'm only going to accept bug fixes."
Bug fixes for what? Kernel api changes that fix bugs? That's pretty
big. Some driver fixes, but not others? Driver fixes that are in the
middle of bigger, subsystem reworks as a series of patches? All of this
currently happens today in the main tree in a semi-cohesive manner. To
try to split it out is a very difficult task.
So, while I like the _idea_ of the 2.6.x.y type releases, having those
releases contain anything but a handful of patches will quickly get
quite messy.
Not to mention the issue of the need for me as a maintainer to mark
"bugfix only" specific patches and pull them out and submit them
separately. Due to api changes, and all sorts of other issues, that can
get to be a difficult job in itself.
Personally, I like the current, "test it all in -mm and then forward the
good bits to Linus" mode we are operating in. My only suggestion would
to possibly speed up the release cycle a bit faster than every two
months, like we currently are on. Once a month perhaps? That was how
we were working at the beginning of 2.6, and it seemed like the backlog
was much smaller then.
thanks,
greg k-h
Greg KH
> The thing is, I _do_ believe the current setup is working reasonably well.
> But I also do know that some people (a fairly small group, but anyway)
> seem to want an extra level of stability - although those people seem to
> not talk so much about "it works" kind of stability, but literally a "we
> can't keep up" kind of stability (ie at least a noticeable percentage of
> that group is not complaining about crashes, they are complaining about
> speed of development).
>
> And I suspect that _anything_ I do won't make those people happy.
That single sentence sums it up perfectly. When I have given talks
about how our current development cycle works, and what's happening with
it, people just feel odd seeing all of this change happen and get upset
at it. Perhaps it's because they never paid attention before, or that
they are new to Linux and like to believe that old-style development
models were somehow "better", and that they know we are doing something
wrong.
But when pressed about the issue of speed of development, rate of
change, feature increase, driver updates, and so on, no one else has any
clue of what to do. They respond with, "but only put bugfixes into a
stable release." My comeback is explaining how we handle lots of
different types of bugfixes, by api changes, real fixes, and driver
updates for new hardware. Sometimes these cause other bugs to happen,
or just get shaken out where they were previously hiding (acpi is a
great example of this issue.) In the end, they usually fall back on
muttering, "well, I'm just glad that I'm not a kernel developer..." and
back away.
Like I previously said, I think we're doing a great job. The current
-mm staging area could use some more testers to help weed out the real
issues, and we could do "real" releases a bit faster than every 2 months
or so. But other than that, we have adapted over the years to handle
this extremely high rate of change in a pretty sane manner.
So don't spend too much time worring that we need to make those types of
people happy. As Jeff said, no other OS is accepting changes at such a
high rate. But then again, no other OS supports more devices, on more
different processors. So we must be doing something right :)
Jeff Garzik
> Sure they've been asking for it, but I think they really don't know what
> it entails. Look at all of the "non-stable" type patches in the -ac and
> as tree. There's a lot of stuff in there. It's a slippery slope down
> when trying to say, "I'm only going to accept bug fixes."
We have all these problems precisely because _nobody_ is saying "I'm
only going to accept bug fixes". We _need_ some amount of release
engineering. Right now we basically have none.
> Bug fixes for what? Kernel api changes that fix bugs? That's pretty
> big. Some driver fixes, but not others? Driver fixes that are in the
> middle of bigger, subsystem reworks as a series of patches? All of this
> currently happens today in the main tree in a semi-cohesive manner. To
> try to split it out is a very difficult task.
Easiest to answer with a concrete example:
Linux 2.6.11 is released. Linus then does a
bk clone linux-2.6 linux-2.6.11
Bug fixes that
(a) 2.6.11 users really should have, or
(b) Linus/Andrew feels are important, or
(c) a subsystem maintainer feels are important [and does the work to
split out the fixes]
go into linux-2.6.11 repo, and then is pulled into linux-2.6 repo.
All other changes go into linux-2.6.
There's no need to over-think or over-work this. The goal is to provide
a stable 2.6.11 for users, until 2.6.12 is available.
My prediction is that several patches will flow into the linux-2.6.11
repo a week or so after a release, and then the flow will die off to a
trickle. Subsystem maintainers that care can submit patches/BK-pulls
for the stable release if they so desire.
Only important "oh shit, that should have been in 2.6.11" bug fixes need
apply. Bug fixes for reworks, API changes, etc. are -not- applicable to
linux-2.6.11 repo.
Since BitKeeper can handle nicely a
cd linux-2.6
bk pull ../linux-2.6.11
there is no duplication of bug fixes.
Jeff
Jes Sorensen
Greg> On Thu, Mar 03, 2005 at 02:52:21AM -0500, Jeff Garzik wrote:
>> Users have been clamoring for a stable release branch in any case,
>> as you see from comments about Alan's -ac and an LKML user's -as
>> kernels.
Greg> Sure they've been asking for it, but I think they really don't
Greg> know what it entails. Look at all of the "non-stable" type
Greg> patches in the -ac and as tree. There's a lot of stuff in
Greg> there. It's a slippery slope down when trying to say, "I'm only
Greg> going to accept bug fixes."
Greg> Bug fixes for what? Kernel api changes that fix bugs? That's
Greg> pretty big. Some driver fixes, but not others? Driver fixes
Greg> that are in the middle of bigger, subsystem reworks as a series
Greg> of patches? All of this currently happens today in the main
Greg> tree in a semi-cohesive manner. To try to split it out is a
Greg> very difficult task.
Greg> So, while I like the _idea_ of the 2.6.x.y type releases, having
Greg> those releases contain anything but a handful of patches will
Greg> quickly get quite messy.
Greg,
Wouldn't this actually happen automatically simply by Linus switching
to being a lot more picky about whats accepted into an even release. I
agree that if it becomes too formal it could probably turn into an
unmaintainable mess. However, by simply making it a golden rule, then
developers can just continue pushing their patches and the people
above can just shift things to -mm that aren't deemed suitable for the
even release.
I think this would work quite well.
Cheers,
Jes
Jeff Garzik
The pertinent question for a point release (2.6.X.Y) would simply be
"does a 2.6.11 user really need this fix?"
> Like I previously said, I think we're doing a great job. The current
> -mm staging area could use some more testers to help weed out the real
> issues, and we could do "real" releases a bit faster than every 2 months
> or so. But other than that, we have adapted over the years to handle
> this extremely high rate of change in a pretty sane manner.
I think Linus's "even/odd" proposal is an admission that 2.6.X releases
need some important fixes after it hits kernel.org.
Otherwise 2.6.X is simply a constantly indeterminent state.
We need to serve users, not just make life easier for kernel developers ;-)
Jeff
Greg KH
>
> Greg> So, while I like the _idea_ of the 2.6.x.y type releases, having
> Greg> those releases contain anything but a handful of patches will
> Greg> quickly get quite messy.
>
> to being a lot more picky about whats accepted into an even release. I
> agree that if it becomes too formal it could probably turn into an
> unmaintainable mess. However, by simply making it a golden rule, then
> developers can just continue pushing their patches and the people
> above can just shift things to -mm that aren't deemed suitable for the
> even release.
>
> I think this would work quite well.
Sure, but the point about people not testing the odd release would still
stand. As it is today, we've been pretty good about only allowing
bugfixes into the later -rc releases. But I know I start queuing
"bigger" fixes at that point in time, allowing them to get more testing
in -mm release, and generally trying to be conserative.
And sometimes, people really want those "big" fixes, and they switch to
using the bk-usb patchset, or the bk-scsi patchset. That happens a lot
for when distros work to stabilize their release kernels. But that is
quite different from trying to do that for a 2.6.x.y release.
thanks,
greg k-h
Matthew Frost
--- Willy Tarreau <wi...@w.ods.org> wrote:
> Hi Linus,
>
> For a long time, I've been hoping/asking for a more frequent
> stable/unstable cycle, so clearly you can count my vote on this one
> (eventhough it might count for close to zero). This is a very good step
> towards a better stability IMHO.
>
> However, I have a comment :
>
> > - 2.6.<odd>: still a stable kernel, but accept bigger changes
> > leading up to it (timeframe: a month or two).
> > - 2.<odd>.x: aim for big changes that may destabilize the kernel for
> > several releases (timeframe: a year or two)
> > - <odd>.x.x: Linus went crazy, broke absolutely _everything_, and
> > rewrote the kernel to be a microkernel using a special
> > message-passing version of Visual Basic. (timeframe: "we expect
> > that he will be released from the mental institution in a decade
or
> > two").
>
> I don't agree with you on this last one (not the fact that you don't
> want an mk+mp+vb combination :-)). The VERSION number (in the makefile
> meaning of the term) only gets updated every 10 years or so. So it does
> not need to jump. PATCHLEVEL increments are rare enough to justify lots
> of breaking. I certainly can imagine people laughing at your OS when
> you jump from v2.6.X to v4.0.X, it will have a smell of slowaris
> (remember 2.6 - 2.7 - 7 - 8 that confused everyone ?). On the other
> side, the openbsd numbering scheme is far simpler to understand, and I
> sincerely think that we should enter 3.0 after 2.8.
I hate to point out the obvious to someone of longer standing, but
according to the current system, since we increment integrally, 2.10.0
would be the successor release to the 2.9.X dev series, unless you know
something about a severe and profound internals shakeup scheduled for 2.9
that we don't. :)
OT, 3.0.0 is an even-numbered release, therefore stable. So what do you
call the odd-numbered unstable series that produces it? ;)
> As a side note, I've
> always wondered why we would not swap odds and evens, so that we can
> keep the same major number between devel and release (eg: devel 2.8,
> release 2.9 ; not devel 2.9, release 3.0). Anyway, people have got used
> to stay away from the '.0' releases of any product on the planet.
>
> Regards
> Willy
>
Matt
Yes, I have too much free time.
Greg KH
> Greg KH wrote:
> >Sure they've been asking for it, but I think they really don't know what
> >it entails. Look at all of the "non-stable" type patches in the -ac and
> >as tree. There's a lot of stuff in there. It's a slippery slope down
> >when trying to say, "I'm only going to accept bug fixes."
>
> We have all these problems precisely because _nobody_ is saying "I'm
> only going to accept bug fixes". We _need_ some amount of release
> engineering. Right now we basically have none.
Linus says that in -rc releases, are you not paying attention to that?
(yes, I know you are...)
> >Bug fixes for what? Kernel api changes that fix bugs? That's pretty
> >big. Some driver fixes, but not others? Driver fixes that are in the
> >middle of bigger, subsystem reworks as a series of patches? All of this
> >currently happens today in the main tree in a semi-cohesive manner. To
> >try to split it out is a very difficult task.
>
> Easiest to answer with a concrete example:
>
> Linux 2.6.11 is released. Linus then does a
> bk clone linux-2.6 linux-2.6.11
>
> Bug fixes that
> (a) 2.6.11 users really should have, or
> (b) Linus/Andrew feels are important, or
> (c) a subsystem maintainer feels are important [and does the work to
> split out the fixes]
>
> go into linux-2.6.11 repo, and then is pulled into linux-2.6 repo.
>
> All other changes go into linux-2.6.
>
> There's no need to over-think or over-work this. The goal is to provide
> a stable 2.6.11 for users, until 2.6.12 is available.
Will Linus start accepting 2.6.12 patches right after 2.6.10 is out?
> My prediction is that several patches will flow into the linux-2.6.11
> repo a week or so after a release, and then the flow will die off to a
> trickle. Subsystem maintainers that care can submit patches/BK-pulls
> for the stable release if they so desire.
>
> Only important "oh shit, that should have been in 2.6.11" bug fixes need
> apply. Bug fixes for reworks, API changes, etc. are -not- applicable to
> linux-2.6.11 repo.
>
> Since BitKeeper can handle nicely a
> cd linux-2.6
> bk pull ../linux-2.6.11
> there is no duplication of bug fixes.
That sounds very sane to me, I can live with it.
The main point is, will people really test the 2.6.odd releases to help
us out in making this new scheme work. If so, I'm all for it.
thanks,
greg k-h
Greg KH
> The pertinent question for a point release (2.6.X.Y) would simply be
> "does a 2.6.11 user really need this fix?"
"need this fix bad enough now, or can it wait until 2.6.12?"
> >Like I previously said, I think we're doing a great job. The current
> >-mm staging area could use some more testers to help weed out the real
> >issues, and we could do "real" releases a bit faster than every 2 months
> >or so. But other than that, we have adapted over the years to handle
> >this extremely high rate of change in a pretty sane manner.
>
> I think Linus's "even/odd" proposal is an admission that 2.6.X releases
> need some important fixes after it hits kernel.org.
I agree.
> Otherwise 2.6.X is simply a constantly indeterminent state.
Heh, true, but all software is in that state :)
> We need to serve users, not just make life easier for kernel developers ;-)
Damm those pesky users. Without them our life would be so much
easier...
greg k-h
Dave Jones
> And sometimes, people really want those "big" fixes, and they switch to
> using the bk-usb patchset, or the bk-scsi patchset. That happens a lot
> for when distros work to stabilize their release kernels.
For those that have no intention of staying close to mainline maybe.
The fact is that before a bitkeeper tree (or any other SCM tree) gets
merged with mainline, its subject to change. Sometimes completely dramatic
'throw it all away and start again' changes. I know this because I've done it,
I know you've done it with pci/usb trees, and various other folks have
done it with their trees.
Pulling whole random subsystem bk snapshots into a vendor tree without
significant review of all contained changesets is the path to madness
unless you're prepared to maintain the resulting mess forever.
Dave
Andrew Morton
>
> The reasons -rcs are not as good as they could be is that they include
> more than just bug fixes.
I thought we'd been fairly good about that, actually. The -rc1's always
come too early for me (I usually wait for all the bk merges to happen).
But once we hit -rc2 people are good? At least, I think I've only ever
seen one "ytf did you merge that now" email...
<hack, hack>
Here you go: http://www.zip.com.au/~akpm/linux/patches/stuff/khack.jpeg
That's number-of-lines-of-patch versus day, from the bk-commits list.
There's definitely a pattern there ;)
(Raw data at
http://www.zip.com.au/~akpm/linux/patches/stuff/commits.data.gz for anyone
who is inclined to spend more that seven seconds with gnuplot...)