Ram memory not freed after executing python script on ubuntu system

26 views
Skip to first unread message

Rahul Gupta

unread,
May 28, 2020, 8:24:27 AM5/28/20
to Linux Users Group
I am having a Ubuntu system which has 125 Gb of RAM. I executed few python scripts on that system. Those scripts uses numpy arrays and pandas. Now execution was over but still 50 gb of RAM and 2 Gb cache and 8.4 Gb of swap is occupied. At this moment nothing is running on the system. I have googled it. Most of th result shows that python garbage collector is poor in performance. I want this memory to be cleaned and re claim. One of the easiest way is to restart the system but i dont want to restart i want a way to do this when the system is up and running. Kindly tell me how to do this. Thanks

Daniel Eggleston

unread,
May 28, 2020, 10:01:37 AM5/28/20
to Linux Users Group
If the scripts are no longer running, then I assure you the garbage collector has nothing to do with it. My guess is the extra memory's taken by filesystem buffer. 

By what metric are you deciding that 50GB of RAM is still occupied? 
Has the swap usage come down since the scripts exited?
From: linuxus...@googlegroups.com <linuxus...@googlegroups.com> on behalf of Rahul Gupta <rahulgup...@gmail.com>
Sent: Thursday, May 28, 2020 12:51 AM
To: Linux Users Group <linuxus...@googlegroups.com>
Subject: [lug:18660] Ram memory not freed after executing python script on ubuntu system
 
I am having a Ubuntu system which has 125 Gb of RAM. I executed few python scripts on that system. Those scripts uses numpy arrays and pandas. Now execution was over but still 50 gb of RAM and 2 Gb cache and 8.4 Gb of swap is occupied. At this moment nothing is running on the system. I have googled it. Most of th result shows that python garbage collector is poor in performance. I want this memory to be cleaned and re claim. One of the easiest way is to restart the system but i dont want to restart i want a way to do this when the system is up and running. Kindly tell me how to do this. Thanks

--
--
You received this message because you are subscribed to the Linux Users Group.
To post a message, send email to linuxus...@googlegroups.com
To unsubscribe, send email to linuxusersgro...@googlegroups.com
For more options, visit our group at http://groups.google.com/group/linuxusersgroup
References can be found at: http://goo.gl/anqri
Please remember to abide by our list rules (http://tinyurl.com/LUG-Rules)
---
You received this message because you are subscribed to the Google Groups "Linux Users Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to linuxusersgro...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/linuxusersgroup/bb2dbfb9-64f5-43c5-825d-3fe92ffde586%40googlegroups.com.

Rahul Gupta

unread,
May 28, 2020, 10:46:32 AM5/28/20
to Linux Users Group
@Daniel Eggleston You may be right as i was wrtiting a very big numpy array file in python. But due to some issues i had to terminate the execution in between. The size if ram may be due to that processed numpy array that needs to be written at the end of the script. I was monitoring RAM  and other resources from System Monitor in ubuntu. Yes size of swap reduced from 14 gb to 8 gb. May i know if the file buffers are the caise of this huge usage of RAM. How to clean this unused wasted space of the RAM. Please help

Daniel Eggleston

unread,
May 28, 2020, 12:59:36 PM5/28/20
to Linux Users Group
Rahul, send us the output of "free -h". For us to help, we need to see what you're seeing. You keep asking us to help you "clean up" or "free" the RAM, but: we don't even know if it's actually in use.

For example:

$ free -h
              total        used        free      shared  buff/cache   available
Mem:           1.9G        419M        714M        132K        835M        1.3G
Swap:          975M         96M        879M

This shows I have 1.9GB of RAM on this VM. And "only" 714 M free -- but look at the far-right column, where I have 1.3 GB available. The implication here is that I've only got around 600MB of memory that's in use and not free for other applications.
Sent: Thursday, May 28, 2020 9:46 AM

To: Linux Users Group <linuxus...@googlegroups.com>
Subject: [lug:18665] Re: Ram memory not freed after executing python script on ubuntu system
 
@Daniel Eggleston You may be right as i was wrtiting a very big numpy array file in python. But due to some issues i had to terminate the execution in between. The size if ram may be due to that processed numpy array that needs to be written at the end of the script. I was monitoring RAM  and other resources from System Monitor in ubuntu. Yes size of swap reduced from 14 gb to 8 gb. May i know if the file buffers are the caise of this huge usage of RAM. How to clean this unused wasted space of the RAM. Please help

On Thursday, May 28, 2020 at 5:54:27 PM UTC+5:30, Rahul Gupta wrote:
I am having a Ubuntu system which has 125 Gb of RAM. I executed few python scripts on that system. Those scripts uses numpy arrays and pandas. Now execution was over but still 50 gb of RAM and 2 Gb cache and 8.4 Gb of swap is occupied. At this moment nothing is running on the system. I have googled it. Most of th result shows that python garbage collector is poor in performance. I want this memory to be cleaned and re claim. One of the easiest way is to restart the system but i dont want to restart i want a way to do this when the system is up and running. Kindly tell me how to do this. Thanks

--
--
You received this message because you are subscribed to the Linux Users Group.
To post a message, send email to linuxus...@googlegroups.com
To unsubscribe, send email to linuxusersgro...@googlegroups.com
For more options, visit our group at http://groups.google.com/group/linuxusersgroup
References can be found at: http://goo.gl/anqri
Please remember to abide by our list rules (http://tinyurl.com/LUG-Rules)
---
You received this message because you are subscribed to the Google Groups "Linux Users Group" group.
To unsubscribe from this group and stop receiving emails from it, send an email to linuxusersgro...@googlegroups.com.
Reply all
Reply to author
Forward
0 new messages