Wood R4 Kernel 1.64

[Demetrius Dade]

Beforeinstalling R4i gold 3DS RTS card, make sure you ordered R4i gold 3DS card from its official resellers. Now R4i gold 3DS card is cheap on its reseller Dwtechz.com as it is on Black Friday sale! R4i gold 3DS RTS wood kernel or software updates regularly. Now the latest one is wood kernel v1.64b. It can run DS games on 3DS, DSi, DS Lite and DS. It is the same way to set up it. You can follow thte steps below.

There is a problem in multiple stable kernel releases that is causing data corruption in ext4 filesystems. It is caused by a problematic commit that is in multiple stable kernels:The commit got merged in 6.5-rc1 so all stable kernels that have91562895f803 ("ext4: properly sync file size update after O_SYNC directIO") before 6.5 are corrupting data - I've noticed at least 6.1 is stillcarrying the problematic commit.More information can be found in a Debian bug report. It has also delayed the release of Debian 12.3 images. "Please do not upgrade any systems at this time, we urge caution for userswith UnattendeUpgrades configured."(Thanks to Alex Ridevski for giving us a heads up on this.) to post comments Ext4 data corruption in stable kernels Posted Dec 9, 2023 23:37 UTC (Sat) by tux3 (subscriber, #101245) [Link] (4 responses)

This is a symptom of a larger problem: seems that patches/backports are sent to "stable" kernels in an almost willy-nilly fashion, and there is inadequate checking of what actually gets in. This is certainly not the first time that something like this has happened.

It would be useful to stop sending stuff to stable kernels unless it demonstrably fixes something that is reproducible for that particular kernel version, and is explicitly tested on that kernel version. Stuff like "it's good to have it just in case" certainly doesn't apply.

In this case it appears the commit was picked from 6.5-rc1 (ie, first merge set from preparing 6.5), and then the "stable" kernel was shipped without waiting for the commit that came along later in the 6.5 development cycle to fix the issue. It took two stable releases (6.1.64, 6.1.65) before enough time had passed for people to notice the issue was in backports too and undo/fix it (in 6.1.66), because the "stable" Linux kernel releases come out extremely frequently with non-trivial sets of patches in each (making it hard to review in detail).

I've got some sympathy for back porting "important" fixes from *released* Linux kernels to older "stable" kernel branches. But it should be carefully chosen, not "automatically cherry picked", and only the release kernels, not random release candidates (especially not at random from early release candidates).

And yes, you're right, this isn't the first time something like this has happened. It just happens to be one of the worst cases, because it could cause file system corruption on the most commonly used file system :-(

(In my case I narrowly avoided upgrading to a Debian release with the problematic kernel; I've got some systems I'd planned to upgrade tomorrow, but now won't until things are more stabilised again.)

Ewen

Ext4 data corruption in stable kernels Posted Dec 10, 2023 7:56 UTC (Sun) by rolexhamster (guest, #158445) [Link] (39 responses)

Indeed, why was it picked up from 6.5-rc1 (a test kernel!) and backported to stable kernels? Why would any commit be picked up from rc1 through to rc8 etc? It's basic risk management not to do that.

Because it is marked

CC: sta...@vger.kernel.org

That is the developer saying: It should be backported to stable, as soon as it hits mainline.

And that is exactly what happened.

>Why would any commit be picked up from rc1 through to rc8 etc?

To fix things?

>It's basic risk management not to do that.

No. It would just delay a lot of fixes.

> At the very least wait until 6.5.1 or 6.5.2. Even then, only if it fixes something actually demonstrable and testable.

The patch was marked for stable inclusion. Which means the author has demonstrated and tested the problem and has then thought it would be needed to backport it.

Mistakes happen.

Ext4 data corruption in stable kernels Posted Dec 10, 2023 12:08 UTC (Sun) by wtarreau (subscriber, #51152) [Link] (19 responses)

Definitely, there are still humans in the delivery chain. Everything went well this time and only two versions were affected in the end. I think we're just facing another grumpy user who wants 100% guarantee of zero bug. The same type of people who complain about unanticipated storms and who then complain about mistaken weather forecast when it announces rain that doesn't come. There's a solution to this: not using a computer nor anything made using a computer nor anything made by something made using a computer. Living in the woods making fire by hitting stones can have its fun but will not necessarily be safer.

For my stable kernel usages, I *tend* to pick one or two versions older than the last one if I see that the recent fixes are not important for me (i.e. I won't miss them). This helps to avoid such cases. But that's not rocket science, and for this one I would likely have updated to that version precisely because it included an ext4 fix!

Ext4 data corruption in stable kernels Posted Dec 10, 2023 23:27 UTC (Sun) by bgilbert (subscriber, #4738) [Link] (18 responses)

"Stable" is not a prediction of forces beyond developer control. It's an assertion of a quality bar, which needs to be backed by appropriate tools, testing, and developer time.

> For my stable kernel usages, I *tend* to pick one or two versions older than the last one if I see that the recent fixes are not important for me (i.e. I won't miss them).

As I understand Greg KH's position, anyone applying such a policy is irresponsible for not immediately installing the newest batch of patches.

Ext4 data corruption in stable kernels Posted Dec 11, 2023 13:47 UTC (Mon) by wtarreau (subscriber, #51152) [Link] (16 responses)

> "Stable" is not a prediction of forces beyond developer control. It's an assertion of a quality bar, which needs to be backed by appropriate tools, testing, and developer time.

Which is exactly the case. Look at the latest 6.6.5-rc1 thread for example:

@linu...

I've counted 17 people responding to that thread with test reports, some of which indicate boot failures, others successes, on a total of around 910 systems covering lots of architectures, configs and setup. I think this definitely qualifies for "appropriate tools", "testing" and "developer time", and I doubt many other projects devote that amount of efforts to weekly releases.

> > For my stable kernel usages, I *tend* to pick one or two versions older than the last one if I see that the recent fixes are not important for me (i.e. I won't miss them).

>

> As I understand Greg KH's position, anyone applying such a policy is irresponsible for not immediately installing the newest batch of patches.

No, for having already discussed this topic with him, I'm pretty sure he never said this. I even remember that once he explained that he doesn't want to advertise severity levels in his releases so that users upgrade when they feel confident and not necessarily immediately nor when it's written that now's a really important one. Use cases differ so much between users that some might absolutely need to upgrade to fix a driver that's going to ruin their data while others might prefer not to as a later fix could cause serious availability issues.

Periodically applying updates is a healthy approach, what matters is that severe bugs do not live long enough in the wild and that releases are frequent enough to help narrow down an occasional regression based on the various reports. I personally rebuild every time I reboot my laptop (it's quite rare thanks to suspend), and phone vendors tend to update only once every few months and that's already OK.

Ext4 data corruption in stable kernels Posted Dec 11, 2023 15:44 UTC (Mon) by bgilbert (subscriber, #4738) [Link] (9 responses)

Many other projects have CI tests that are required to pass before a new release can ship. If that had been the case for LTP, this regression would have been avoided. What's more, the problem was reported to affect 6.1.64 during its -rc period, but no action was taken to fix that release. 6.1.64 was released with the problem four days later.Mistakes happen! But this is an opportunity to improve processes to prevent a recurrence, rather than accepting the status quo.No, for having already discussed this topic with him, I'm pretty sure he never said this. I even remember that once he explained that he doesn't want to advertise severity levels in his releases so that users upgrade when they feel confident and not necessarily immediately nor when it's written that now's a really important one. Use cases differ so much between users that some might absolutely need to upgrade to fix a driver that's going to ruin their data while others might prefer not to as a later fix could cause serious availability issues.I have personally been complained at by Greg for fixing a stable kernel regression via cherry-pick, rather than shipping the latest release directly to distro users. I've seen similarly aggressive messaging in other venues. In fact, the standard release announcement says:All users of the x.y kernel series must upgrade.If downstream users are intended to take a more cautious approach, the messaging should be clarified to reflect that. Ext4 data corruption in stable kernels Posted Dec 12, 2023 4:52 UTC (Tue) by wtarreau (subscriber, #51152) [Link] (8 responses)

You should really see that as a pipeline. Even if the issue was reported you don't know if it was noticed before 6.1.64 was emitted. What matters is that the issue was quickly fixed. Sure we're still missing a way to tag certain versions as broken, like happened for 2.4.11 that was marked "dontuse" in the download repository. But it's important to understand that the constant flow of fixes doesn't easily prevent a release from being cancelled instantly.

I would not be shocked to see 3 consecutive kernels being emitted and tagged as "ext4 broken" there for the time it takes to get knowledge of the breakage and fix it.

> I have personally been complained at by Greg for fixing a stable kernel regression via cherry-pick, rather than shipping the latest release directly to distro users.

Here you're speaking about cherry-picking fixes. That's something extremely dangerous that nobody must ever do and that some distros have been doing for a while, sometimes shipping kernels remaining vulnerable for months or years due to this bad practice. The reason for recommending against cherry-picking is very simple (and was explained in lengths at multiple conferences): the ONLY combinations of kernel patches that are both tested and supported by the subsystem maintainers are the mainline and stable ones. If you perform any other assembly of patches, nobody knows if they work well together or if another important patch is missing (as happened above). Here the process worked fine because developers reported the missing patches. Imagine if you took that single patch yourself, nobody would have known and you could have corrupted a lot of your users' FSes.

So please, for your users, never ever cherry-pick random patches from stable. Take the whole stable, possibly a slightly older one if you don't feel easy with latest changes, add your distro-specific patches on top of it, but do not pick what seems relevant to you, that will eventually result in a disaster and nobody will support you for having done this.

Ext4 data corruption in stable kernels Posted Dec 12, 2023 9:58 UTC (Tue) by bgilbert (subscriber, #4738) [Link] (3 responses)

3a8082e126