Google Groups no longer supports new Usenet posts or subscriptions. Historical content remains viewable.
Dismiss
Bookworm system not responding on high memory usage
103 views
Skip to first unread message
Xiyue Deng
Mar 10, 2023, 8:00:06 PM3/10/23
to
Hi,
I have an AMD64 system[1] that has been running fine on Bullseye for a
few years, and recently following the soft freeze on Bookworm I upgraded
my system to try it out, and the system has been frequently losing
response. Initially I thought it was because of some issue of my
qemu-based Win11 virtual machine as it happens most frequently when it
was running and filed a bug report[2]. But then it happened again
without it running because some other program had slowly used up most of
the memory again, though not as frequently as the VM was running.
Now in retrospect, when I was using Bullseye the total memory was also
mostly used up most of the time, with a few hundreds of megabytes
reported as free and a few Gigs reported as cache, and it has been
running fine. I'm not sure what has changed in Bookworm and having to
manually restart the machine is a pretty annoying and unpleasant
experience.
Does anyone seeing a similar problem as well? What can I do to avoid
this? Any suggest is welcome.
Thanks in advance.
[1] System info from inxi:
CPU: 8-core AMD Ryzen 9 5900HX with Radeon Graphics (-MT MCP-)
speed/min/max: 1199/1200/4679 MHz Kernel: 6.1.0-5-amd64 x86_64 Up: 7m
Mem: 4844.4/31521.3 MiB (15.4%) Storage: 476.94 GiB (54.5% used) Procs: 535
Shell: Bash inxi: 3.3.25
[2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1032400
--
Manphiz
I have an AMD64 system[1] that has been running fine on Bullseye for a
few years, and recently following the soft freeze on Bookworm I upgraded
my system to try it out, and the system has been frequently losing
response. Initially I thought it was because of some issue of my
qemu-based Win11 virtual machine as it happens most frequently when it
was running and filed a bug report[2]. But then it happened again
without it running because some other program had slowly used up most of
the memory again, though not as frequently as the VM was running.
Now in retrospect, when I was using Bullseye the total memory was also
mostly used up most of the time, with a few hundreds of megabytes
reported as free and a few Gigs reported as cache, and it has been
running fine. I'm not sure what has changed in Bookworm and having to
manually restart the machine is a pretty annoying and unpleasant
experience.
Does anyone seeing a similar problem as well? What can I do to avoid
this? Any suggest is welcome.
Thanks in advance.
[1] System info from inxi:
CPU: 8-core AMD Ryzen 9 5900HX with Radeon Graphics (-MT MCP-)
speed/min/max: 1199/1200/4679 MHz Kernel: 6.1.0-5-amd64 x86_64 Up: 7m
Mem: 4844.4/31521.3 MiB (15.4%) Storage: 476.94 GiB (54.5% used) Procs: 535
Shell: Bash inxi: 3.3.25
[2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1032400
--
Manphiz
Timothy M Butterworth
Mar 10, 2023, 8:20:08 PM3/10/23
to
On Fri, Mar 10, 2023 at 7:57 PM Xiyue Deng <man...@gmail.com> wrote:
Hi,
I have an AMD64 system[1] that has been running fine on Bullseye for a
few years, and recently following the soft freeze on Bookworm I upgraded
my system to try it out, and the system has been frequently losing
response. Initially I thought it was because of some issue of my
qemu-based Win11 virtual machine as it happens most frequently when it
was running and filed a bug report[2]. But then it happened again
without it running because some other program had slowly used up most of
the memory again, though not as frequently as the VM was running.
Now in retrospect, when I was using Bullseye the total memory was also
mostly used up most of the time, with a few hundreds of megabytes
reported as free and a few Gigs reported as cache, and it has been
running fine. I'm not sure what has changed in Bookworm and having to
manually restart the machine is a pretty annoying and unpleasant
experience.
Does anyone seeing a similar problem as well? What can I do to avoid
this? Any suggest is welcome.
Thanks in advance.
Open the command prompt and run `su` to switch user to root. Then run `sync && echo 1 > /proc/sys/vm/drop_caches` as root. This will write RAM caches to the hard drive to free up memory. You have to run this as root as sudo, my preferred method, returns a permission disabled error.
[1] System info from inxi:
CPU: 8-core AMD Ryzen 9 5900HX with Radeon Graphics (-MT MCP-)
speed/min/max: 1199/1200/4679 MHz Kernel: 6.1.0-5-amd64 x86_64 Up: 7m
Mem: 4844.4/31521.3 MiB (15.4%) Storage: 476.94 GiB (54.5% used) Procs: 535
Shell: Bash inxi: 3.3.25
Your system has 32 GB of RAM, it should not be getting used up. Run `free -h` What desktop are you using: KDE, GNOME, LXQT etc? Are you using Wayland or X11? It looks like you have a memory leak in one of your applications. Try running `top` and press `m` to sort by memory utilization.
Tim
Xiyue Deng
Mar 11, 2023, 3:50:07 AM3/11/23
to
Timothy M Butterworth <timothy.m....@gmail.com> writes:
> On Fri, Mar 10, 2023 at 7:57 PM Xiyue Deng <man...@gmail.com> wrote:
>
> Hi,
>
> I have an AMD64 system[1] that has been running fine on Bullseye for a
> few years, and recently following the soft freeze on Bookworm I upgraded
> my system to try it out, and the system has been frequently losing
> response. Initially I thought it was because of some issue of my
> qemu-based Win11 virtual machine as it happens most frequently when it
> was running and filed a bug report[2]. But then it happened again
> without it running because some other program had slowly used up most of
> the memory again, though not as frequently as the VM was running.
>
> Now in retrospect, when I was using Bullseye the total memory was also
> mostly used up most of the time, with a few hundreds of megabytes
> reported as free and a few Gigs reported as cache, and it has been
> running fine. I'm not sure what has changed in Bookworm and having to
> manually restart the machine is a pretty annoying and unpleasant
> experience.
>
> Does anyone seeing a similar problem as well? What can I do to avoid
> this? Any suggest is welcome.
>
> Thanks in advance.
>
> Open the command prompt and run `su` to switch user to root. Then run `sync && echo 1 > /proc/sys/vm/drop_caches` as
> root. This will write RAM caches to the hard drive to free up memory. You have to run this as root as sudo, my preferred
> method, returns a permission disabled error.
>
>
> [1] System info from inxi:
> CPU: 8-core AMD Ryzen 9 5900HX with Radeon Graphics (-MT MCP-)
> speed/min/max: 1199/1200/4679 MHz Kernel: 6.1.0-5-amd64 x86_64 Up: 7m
> Mem: 4844.4/31521.3 MiB (15.4%) Storage: 476.94 GiB (54.5% used) Procs: 535
> Shell: Bash inxi: 3.3.25
>
> Your system has 32 GB of RAM, it should not be getting used up. Run `free -h` What desktop are you using: KDE, GNOME,
> LXQT etc? Are you using Wayland or X11? It looks like you have a memory leak in one of your applications. Try running
> `top` and press `m` to sort by memory utilization.
usage. As I mentioned, it usually happens when I use qemu (see [1] for
free and [2] for top). At another time it happened when deluge is
leaking memory (see [3] for free [4] for top).
Interestingly as you can see, in all such cases, even though the free
amount is low, the buff/cache is still pretty large so the system is not
really overloaded. Plus, on Bullseye such memory usage also happens all
the time and this never happened. I was suspecting that maybe the
kernel is panicking when memory hits certain limit, but I don't see it
in kern.log or syslog.
Any suggestion to restore to Bullseye status is appreciated. Thanks in
advance!
[1] `free -h` when using qemu:
total used free shared buff/cache available
Mem: 30Gi 14Gi 258Mi 216Mi 17Gi 16Gi
Swap: 979Mi 80Mi 899Mi
[2] `top` sorted by memory when using qemu:
top - 16:10:05 up 1:29, 11 users, load average: 1.83, 1.86, 2.06
Tasks: 494 total, 1 running, 493 sleeping, 0 stopped, 0 zombie
%Cpu(s): 8.3 us, 8.3 sy, 0.0 ni, 75.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 31522.7 total, 257.2 free, 14430.8 used, 17504.1 buff/cache
MiB Swap: 980.0 total, 899.5 free, 80.5 used. 17091.9 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
10131 libvirt+ 20 0 11.2g 8.1g 26140 S 213.3 26.2 75:08.67 qemu-sy+
6547 xiyueden 20 0 4432172 1.4g 207312 S 0.0 4.5 1:53.44 thunder+
...
[3] `free -h` when using deluge:
total used free shared buff/cache available
Mem: 30Gi 12Gi 1.9Gi 219Mi 17Gi 18Gi
Swap: 979Mi 2.2Mi 977Mi
[4] `top` sorted by memory when using deluge:
top - 10:40:05 up 3 days, 17:11, 11 users, load average: 1.25, 1.22, 1.20
Tasks: 492 total, 1 running, 490 sleeping, 0 stopped, 1 zombie
%Cpu(s): 25.0 us, 0.0 sy, 0.0 ni, 75.0 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
MiB Mem : 31521.3 total, 1909.2 free, 12762.9 used, 17529.7 buff/cache
MiB Swap: 980.0 total, 977.7 free, 2.2 used. 18758.4 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
7287 xiyueden 20 0 9030940 6.6g 503076 S 0.0 21.3 97:11.62 deluge-+
5271 xiyueden 20 0 4581328 1.6g 191000 S 6.7 5.2 108:23.57 thunder+
...
Manphiz
Timothy M Butterworth
Mar 11, 2023, 7:30:06 AM3/11/23
to
I have an AMD Ryzen 7 4700U with Radeon Graphics and the only time I see my RAM used up is when I am transcoding Video files.
System Idle running KDE 5.27.2, Google Chrome and Dolphin:
total used free shared buff/cache available
Mem: 14Gi 3.8Gi 9.4Gi 91Mi 2.2Gi 11Gi
System with VirtualBox running Kali Linux
total used free shared buff/cache available
Mem: 14Gi 8.9Gi 4.2Gi 110Mi 2.3Gi 6.1Gi
Swap: 14Gi 0B 14Gi
Swap: 14Gi 0B 14Gi
Xiyue Deng
Mar 12, 2023, 9:00:05 PM3/12/23
to
So after some more tries it looks like this issue is not directly memory
usage related. I've tried the following:
* Using older kernel version when I was on Bullseye.
* Have a cronjob to drop memory caches every minutes.
* Using Gnome on Wayland by default or Xorg.
And this can still happen when I was running a qemu-based Win11 VM using
virtual manager. So this rules out the possibility of a kernel issue
and OOM killer issue. All that is certain is that this issue can be
reproduced when running my qemu-based Win11 VM and in a few hours it
will trigger this lockup.
As this system has been running Bullseye for a few years with zero
problem, I'm hopeful this should work for Bookworm as well. If you have
anything in mind that may worth a try please feel free to share. The
more ideas the better.
Thanks in advance!
(Replies to Timothy below inline.)
So unfortunately this doesn't help either, as it happens again with very
low cache usage.
`free -h`:
total used free shared buff/cache available
Mem: 30Gi 13Gi 16Gi 206Mi 1.4Gi 17Gi
Swap: 979Mi 0B 979Mi
`top` excerpt:
top - 14:55:05 up 18 min, 11 users, load average: 1.77, 1.65, 1.09
Tasks: 504 total, 1 running, 503 sleeping, 0 stopped, 0 zombie
%Cpu(s): 12.5 us, 0.0 sy, 0.0 ni, 68.8 id, 0.0 wa, 0.0 hi, 6.2 si, 0.0 st
MiB Mem : 31519.9 total, 16972.6 free, 13759.0 used, 1447.6 buff/cache
MiB Swap: 980.0 total, 980.0 free, 0.0 used. 17760.8 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
8886 libvirt+ 20 0 11.1g 8.1g 26580 S 87.5 26.4 17:38.47 qemu-sy+
5434 xiyueden 20 0 4047004 1.2g 170036 S 0.0 4.0 0:41.00 thunder+
5143 xiyueden 20 0 7056664 526296 191152 S 0.0 1.6 2:19.65 gnome-s+
...
Thanks for sharing. I've allocated 8GB of memory for the Win11 VM so on
startup it will use around 15GB of memory (~50%) from the system, and I
should still have more than enough free memory. As I've mentioned in
the beginning of the letter, it now looks less likely a memory related
issue.
Manphiz
usage related. I've tried the following:
* Using older kernel version when I was on Bullseye.
* Have a cronjob to drop memory caches every minutes.
* Using Gnome on Wayland by default or Xorg.
And this can still happen when I was running a qemu-based Win11 VM using
virtual manager. So this rules out the possibility of a kernel issue
and OOM killer issue. All that is certain is that this issue can be
reproduced when running my qemu-based Win11 VM and in a few hours it
will trigger this lockup.
As this system has been running Bullseye for a few years with zero
problem, I'm hopeful this should work for Bookworm as well. If you have
anything in mind that may worth a try please feel free to share. The
more ideas the better.
Thanks in advance!
(Replies to Timothy below inline.)
low cache usage.
`free -h`:
total used free shared buff/cache available
Swap: 979Mi 0B 979Mi
`top` excerpt:
top - 14:55:05 up 18 min, 11 users, load average: 1.77, 1.65, 1.09
Tasks: 504 total, 1 running, 503 sleeping, 0 stopped, 0 zombie
%Cpu(s): 12.5 us, 0.0 sy, 0.0 ni, 68.8 id, 0.0 wa, 0.0 hi, 6.2 si, 0.0 st
MiB Mem : 31519.9 total, 16972.6 free, 13759.0 used, 1447.6 buff/cache
MiB Swap: 980.0 total, 980.0 free, 0.0 used. 17760.8 avail Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
5434 xiyueden 20 0 4047004 1.2g 170036 S 0.0 4.0 0:41.00 thunder+
5143 xiyueden 20 0 7056664 526296 191152 S 0.0 1.6 2:19.65 gnome-s+
...
startup it will use around 15GB of memory (~50%) from the system, and I
should still have more than enough free memory. As I've mentioned in
the beginning of the letter, it now looks less likely a memory related
issue.
Anssi Saari
Mar 14, 2023, 2:30:06 AM3/14/23
to
Xiyue Deng <man...@gmail.com> writes:
> As this system has been running Bullseye for a few years with zero
> problem, I'm hopeful this should work for Bookworm as well. If you have
> anything in mind that may worth a try please feel free to share. The
> more ideas the better.
To me the interesting question is, does the problem disappear if you go
> As this system has been running Bullseye for a few years with zero
> problem, I'm hopeful this should work for Bookworm as well. If you have
> anything in mind that may worth a try please feel free to share. The
> more ideas the better.
back to Bullseye? If not then it's likely a hardware problem.
Xiyue Deng
Mar 28, 2023, 3:10:06 AM3/28/23
to
Xiyue Deng <man...@gmail.com> writes:
> So after some more tries it looks like this issue is not directly memory
> usage related. I've tried the following:
>
> * Using older kernel version when I was on Bullseye.
> * Have a cronjob to drop memory caches every minutes.
> * Using Gnome on Wayland by default or Xorg.
>
> And this can still happen when I was running a qemu-based Win11 VM using
> virtual manager. So this rules out the possibility of a kernel issue
> and OOM killer issue. All that is certain is that this issue can be
> reproduced when running my qemu-based Win11 VM and in a few hours it
> will trigger this lockup.
>
> As this system has been running Bullseye for a few years with zero
> problem, I'm hopeful this should work for Bookworm as well. If you have
> anything in mind that may worth a try please feel free to share. The
> more ideas the better.
>
> Thanks in advance!
Bookworm and Bullseye, and this issue still happens. I guess this
largely lowers the possibility of a software cause. I've also done a
10-hour memtest session and it passed so I guess it was proven to be
clean as well.
For the next step, I'll go with the hardware aspect. I want to thank
for the helps, suggestions, and brainstorming from various people from
#debian{,-next} IRC channels! Will try to get to the bottom of this.
Xiyue Deng
Mar 30, 2023, 5:50:05 PM3/30/23
to
Xiyue Deng <xiyu...@debian-hx90.lan> writes:
> Xiyue Deng <man...@gmail.com> writes:
>
>> So after some more tries it looks like this issue is not directly memory
>> usage related. I've tried the following:
>>
>> * Using older kernel version when I was on Bullseye.
>> * Have a cronjob to drop memory caches every minutes.
>> * Using Gnome on Wayland by default or Xorg.
>>
>> And this can still happen when I was running a qemu-based Win11 VM using
>> virtual manager. So this rules out the possibility of a kernel issue
>> and OOM killer issue. All that is certain is that this issue can be
>> reproduced when running my qemu-based Win11 VM and in a few hours it
>> will trigger this lockup.
>>
>> As this system has been running Bullseye for a few years with zero
>> problem, I'm hopeful this should work for Bookworm as well. If you have
>> anything in mind that may worth a try please feel free to share. The
>> more ideas the better.
>>
>> Thanks in advance!
>
> So, to rule out possible software issues, I've done a clean install of
> Bookworm and Bullseye, and this issue still happens. I guess this
> largely lowers the possibility of a software cause. I've also done a
> 10-hour memtest session and it passed so I guess it was proven to be
> clean as well.
>
> For the next step, I'll go with the hardware aspect. I want to thank
> for the helps, suggestions, and brainstorming from various people from
> #debian{,-next} IRC channels! Will try to get to the bottom of this.
>
Actually after I decided to contact the customer service of my box[1],
> Bookworm and Bullseye, and this issue still happens. I guess this
> largely lowers the possibility of a software cause. I've also done a
> 10-hour memtest session and it passed so I guess it was proven to be
> clean as well.
>
> For the next step, I'll go with the hardware aspect. I want to thank
> for the helps, suggestions, and brainstorming from various people from
> #debian{,-next} IRC channels! Will try to get to the bottom of this.
>
after a few rounds of suggestions (reset CMOS, reinstall system, etc.),
they provided an update to the BIOS that supposed to Windows 10/11
freezing when accessing the fTPM module. After flashing the new BIOS,
I've been running the system on high load for 12+ hours without issue.
Though a much longer testing period is needed to make sure the fix is
sufficient, I think this is looking very promising! Will report back
after a week.
Hope this is useful for anyway having similar issues.
[1] https://store.minisforum.com/products/hx90
Xiyue Deng
Apr 10, 2023, 7:10:07 AM4/10/23
to
Elitemini HX90[1] and except a manual reboot my system has been running
totally fine! So I'd consider this issue as resolved. In case you are
using similar system from the same vendor and experiencing similar
system freezing issues, please contact the customer support for a
similar BIOS updates.
I'd like to thank the wonderful people at #debian{,-next} on IRC again
for helping me and the suggestions during the debugging!
0 new messages