Memory leak when using remote control?

[alex.p...@gmail.com]

Hi there,

I'm using gqrx version 2.11.2 on a Pi 3 B/Stretch with a RTL-SDR.com radio, and a USB/Serial Port adapter with CI-V interface to interface with an IC-275H all mode 2m radio. This setup works really nice, except it stops working after a few hours. The waterfall in gqrx stops and it is unresponsive. Not just the program, but the Pi becomes unresponsive. Last night I was able to shut down gqrx gracefully and found that mmcqd was using quite a bit of CPU. I suspect the Pi is swapping memory to the SD card, possibly due to a memory leak.

This evening I've tried to narrow down the cause using "top".  Every two minutes and 16 seconds gqrx uses one percent more memory when remote control via TCP is turned on *and* gqrx-hamlib.py is running. The leak stops as soon as I stop remote control. When I restart remote control, there is still no leak, but as soon as I restart gqrx-hamlib, gqrx is starting to consume memory again at the same rate I mentioned before.

Has anyone seen this behavior before and perhaps have a solution?

73,

--Alex KR1ST

[Simon Kennedy]

Hi Alex,

That sounds odd as gqrx-hamlib only uses the standard gqrx UDP interface for sending remote controls. 

To narrow down further could you run gqrx with rigctl running and connected to gqrx (not running gqrx-hamlib). This will maintain a remote control link with gqrx so should identify further if the problem is with gqrx or somehow with gqrx-hamlib.

When you see increasing memory usage from gqrx are you changing frequency or any other settings or just letting it run?

73

Simon

G0FCU.

--
You received this message because you are subscribed to the Google Groups "Gqrx SDR" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gqrx+uns...@googlegroups.com.
To post to this group, send email to gq...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/gqrx/3c11c249-84f5-4747-af05-523b8b620b85%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Alexandru Csete]

Thanks for the info Alex. I will have to take a closer look and see
whether I can find some leaks in gqrx.

Simon: The leak could well be in gqrx itself, since there is code that
is only active when there is traffic on the remote connection. I will
try to test it using gpredict as source for frequency updates to see
if that also shows leak. But it might take a few days or weeks before
I have time.

Alex

[alex.p...@gmail.com]

Hi Simon,

When rigctl was running with gqrx it did not seem to cause the leak. Perhaps I'm misunderstanding things, but I thought gqrx-hamlib is what connects rigctl to gqrx.

73,

--Alex KR1ST

[alex.p...@gmail.com]

Hi Alex,

Thanks for looking into this. Take your time. I appreciate that you are making gqrx available for the Pi!

These are the commands I run on the Pi:

rigctld -m 304 -r /dev/ttyUSB0 -s 9600 --set-conf=serial_handshake=None,rts_state=ON,dtr_state=ON &
/home/pi/gqrx-sdr-2.11.2-linux-rpi3/gqrx &
python /home/pi/gqrx-hamlib-master/gqrx-hamlib.py &

When I turn the radio off, so that it will not respond to queries through the CI-V interface, the leak stops. It turn the radio back on, and the leaking starts again. If I can think of any more clues, then I'll post them here.

73,

--Alex KR1ST

[Simon Kennedy]

Hi Alex,

I've not looked at the code for some time I have actually forgotten how it works!

Best I think to leave it with Alex to test with gpredict. I can't help at the moment as my Linux machine is a mess. It's stuck on Ubuntu 16.10 so I cannot upgrade gqrx, my pybombs installation broke after I tried to upgrade to pybombs v2 and my Python implementation is broken. I am going to do a clean install of Ubuntu 18.04 LTS on a new SSD because trying to upgrade what I currently have will not solve anything.

Regards

Simon.

--

You received this message because you are subscribed to the Google Groups "Gqrx SDR" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gqrx+uns...@googlegroups.com.
To post to this group, send email to gq...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gqrx/ac7a1d6b-0efa-4ea5-9ee0-3bcdb62d8b80%40googlegroups.com.

[Robin Gape]

Simon,

not wishing to be a clever-clogs, but I have found that it is useful to
have several partitions dedicated to individual OS installations, with a
separate /data partition for stuff that persists between installs, such
as photos, music &c. That allows a totally whatsited installation to be
replaced with a fresh installation. The email, browser directories from
the abandoned installation may then be copied directly across to the new
installation, so that email &c. is not lost. [Alternative arrangements
are possible, of course.]

It also helps to keep an LTS installation available, and these days most
machines locally are only running LTS installs to avoid the perpetual
upgrade/re-install hassle.

It does require enough mass storage to be available, of course. These
days, even SSDs are almost cheap, and HDDs certainly are!

Good luck, and with some fellow feeling,

Robin, G8DQX

[alex.p...@gmail.com]

Hi Simon,

I did peek at the  gqrx-hamlib code and it looks fine to me. It's very straight forward. 

I'm not really a Linux/cpp/Qt developer but I wonder if the problem happens as soon as gqrx accepts a connection and the readyRead signal is emitted and startRead is called. It looks like all the processing, including writing the response to the socket, is done in the same function (or slot). I wonder what happens when a second readyRead signal is emitted, when the first one hasn't completed processing yet. It also seems that it can be possible that not all available data is read in startRead when the readyRead signal is emitted. I'm only looking at the code in Github and could very well be way off here and accomplish nothing but confuse matters by doing so. :-)

73,

--Alex KR1ST

[Alexandru Csete]

Hi Alex and Simon

I have done some tests today and found out that:

1. There appears to be no leak in gqrx when processing remote control command
2. There might be a leak when the remote client terminates the connection

The first part I tested by letting gpredict send frequency command to
gqrx at 10 Hz. It ran for hours without any increase in memory
consumption.

The second part I tested using a bash shell script that sent commands like

printf "F 442815000\n" | nc 192.168.1.225 7356

to gqrx 3-5 times per second. Here I could indeed observed rapidly
increasing memory consumption similar to what you reported.

Simon, now I am wondering whether gqrx-hamlib repeatedly opens and
closes the connection every time it sends a command?

Alex, if you want to test further, a tool like this one might be
better suited to monitor the memory consumption of a single program:
https://github.com/pixelb/ps_mem/blob/master/ps_mem.py

Alex

[Simon Kennedy]

Thanks Alex C, I'll check the code. 

Alex K, does the increased memory usage occur when no changes in frequency or other settings are made in gqrx or the connected rig?

Simon.

--
You received this message because you are subscribed to the Google Groups "Gqrx SDR" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gqrx+uns...@googlegroups.com.
To post to this group, send email to gq...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gqrx/CAHG%3DS_egOz55Dh9dy%3DLKTCVxDuBYnesn8qKRtqKJXOCyA-u-DA%40mail.gmail.com.

[Simon Kennedy]

I had a quick look and it does open a new connection every time it sends a command although no problems have been reported. Perhaps it is specific to the RPi. I should be able to change it easily although something at the back of my mind makes me think that I wrote it the way I did for a reason.

I'll try to look at it tomorrow.

Simon.

[alex.p...@gmail.com]

Hi Alex,

Thanks for checking into this. I appreciate it. I'm glad you were able to reproduce the problem.

I will check out the tool you pointed at. I was thinking of getting a Pi 3 B+ and set up a development environment on it. I found a previous post by you with your notes on how to build gqrx on the Pi. I do write software, but mostly ERP/Oracle/Linux based by day and Windows/.Net/C# by night. 

73,

--Alex KR1ST

[alex.p...@gmail.com]

Hi Simon,

Indeed, the memory leak does happen when no frequency changes take place on the radio or in gqrx. The gqrx-hamlib code does open a connection, and closes it, at least 5 times a second to rigctld and 5 times to gqrx even when no changes happen on either side. I guess that is what Alex C is referring to. I didn't realize this because I only checked the forever loop and not the getfreq and setfreq functions. Still, in itself I wouldn't think that this is necessarily bad. 

73,

--Alex KR1ST

[Simon Kennedy]

Hi Alex,

I have just changed the code to make the connection to gqrx and Hamlib once and not multiple times per second (I am a bad programmer but a good project manager!)

I am trying to publish this to pip now but having a problem because my Python install is messed up. Hopefully I can resolve the problem so you can get the new version to test.

73

Simon.

To unsubscribe from this group and stop receiving emails from it, send an email to gqrx+unsubscribe@googlegroups.com.

To post to this group, send email to gq...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gqrx/99f0707a-672c-44b3-bec7-e895c258572f%40googlegroups.com.

[Simon Kennedy]

Hi Alex,

I have created v2.7 in pip, 'sudo pip install -U gqrxHamlib' to upgrade.

I have also updated the github repo: https://github.com/g0fcu/gqrx-hamlib-gui

One minor point is that I get an error when running the version from pip but I don't when running my development version in Eclipse. I hope this is down to my broken Python install on this computer.

Please let us know if this fixes the problem.

73 Simon.

[alex.p...@gmail.com]

Hi Simon,

I installed the latest version, but get this error:

pi@raspberrypi:~ $ gqrxHamlib

Traceback (most recent call last):

  File "/usr/local/bin/gqrxHamlib", line 7, in <module>

    from gqrxHamlib import main

  File "/usr/local/lib/python2.7/dist-packages/gqrxHamlib/__init__.py", line 32, in <module>

    import xmlrpc.client

ImportError: No module named xmlrpc.client

I did make sure that python-qt4 was installed. Maybe I'm missing some library on my Pi? I usually just run the gqrx-hamlib.py script. I never used the GUI version before.

73,

--Alex KR1ST

[Simon Kennedy]

Hi Alex,

I've tested it now on my RPi :-) Initially I got the same error. The following commands need to be run.

Install PyQt4 for Python3: sudo apt-get install python3-pyqt4

Install gqrxHamlib using the Python3 pip installer: sudo pip3 install gqrxHamlib

Having run the above two commands it worked. I haven't updated the non-GUI version of the code so the GUI version is better :-)

I'm fairly confident this should work.

73 Simon.

To unsubscribe from this group and stop receiving emails from it, send an email to gqrx+unsubscribe@googlegroups.com.

To post to this group, send email to gq...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gqrx/6e0fde14-3044-4b84-aac9-552d089993e6%40googlegroups.com.

[alex.p...@gmail.com]

Hi Simon,

That made it work. Thanks! :)

I did notice that gqrxHamlib sets the mode in gqrx to that of the radio. I always use gqrx in the "demod off" mode as I only use it as a bandscope and tuning aide, and it saves a huge amount of CPU cycles. In itself that wouldn't be a problem for an initial state, but when I set the mode in gqrx back to "demod off", gqrxHamlib displays this error and stops synching:

pi@raspberrypi:~ $ gqrxHamlib

set gqrx   127.0.0.1   b'RPRT 0\n'

set hamlib   127.0.0.1   b'RPRT 0\n'

set hamlib   127.0.0.1   b'RPRT 0\n'

set hamlib   127.0.0.1   b'RPRT 0\n'

set hamlib   127.0.0.1   b'RPRT 0\n'

set hamlib   127.0.0.1   b'RPRT 0\n'

set hamlib   127.0.0.1   b'RPRT 0\n'

Traceback (most recent call last):

  File "/usr/local/lib/python3.5/dist-packages/gqrxHamlib/__init__.py", line 124, in run

    self.setfreq(self.gqrxIPv, GQRX_PORT, 'gqrx', int(rig_freq), sockGqrx)

ValueError: invalid literal for int() with base 10: ''

I guess it's because the radio has no such mode as "demod off" and can't get translated to Hamlib?

I also see the error above when I tune the VFO on the radio. It's like after a few kHz it loses synch.

I will test some more later. Tomorrow nigh there is a 2m Sprint in the US, so I will probably use it a lot. 

Thank you for your time and energy on this!

73,

--Alex KR1ST

[alex.p...@gmail.com]

Btw, I had it running for several hours and did not experience a memory leak anymore!

Like I mentioned before, I have to tune very slowly on the radio now or else gqrxHamlib will crash with the error I mentioned earlier. 

Many thanks!

73,

--Alex KR1ST

[Simon Kennedy]

Hi Alex,

did you go into the settings menu and uncheck the 'Sync mode' option? That should sort out one problem I hope.

As for fast tuning I don't experience that problem at all but I don't run it on a RPi. Generally if you get an error you can press the 'Disable sync' button and then press the sync button that you require and it will work again without having to exit the program.

I think I only need to install Hamlib on my Pi to enable me to try to replicate the fast tuning error. I'll try to do that today. Presumably you didn't experience the tuning problem with the non-GUI version.

Good news that the memory leak has gone!

Simon.

To view this discussion on the web visit https://groups.google.com/d/msgid/gqrx/c67a4cd2-50e3-4bb8-84ba-465ed0c5cb01%40googlegroups.com.

[Simon Kennedy]

Hi Alex,

I've just connected my FT-920 to my RPi and gqrx etc. I can tune the radio as fast as I can spin the dial and gqrx keeps up with no errors. I'll build a new version of gqrxHamlib with some more error logging later to try to help narrow down the problem.

Simon.

To unsubscribe from this group and stop receiving emails from it, send an email to gqrx+unsubscribe@googlegroups.com.

[Simon Kennedy]

Hi Alex, 

can you upgrade gqrxHamlib: sudo pip3 install -U gqrxHamlib

You will then see a lot more diagnostic messages, when it crashes can you send me the last 20 lines or so.

Simon.

On 9 April 2018 at 17:52, Simon Kennedy <si...@thekennedys.info> wrote:

Hi Alex,

I've just connected my FT-920 to my RPi and gqrx etc. I can tune the radio as fast as I can spin the dial and gqrx keeps up with no errors. I'll build a new version of gqrxHamlib with some more error logging later to try to help narrow down the problem.

Simon.

[Alexandru Csete]

On Sat, Apr 7, 2018 at 9:43 PM, Alexandru Csete <oz9...@gmail.com> wrote:

> 2. There might be a leak when the remote client terminates the connection

We now have a fix for this problem, thanks to Alexander Fasching
OE5TKM. I will make a new release over the next days.

Still, changing the client to keep the connection open would still be
a good idea IMHO.

Alex

[Alex Kr1st]

Hi Simon,

I did not try that setting. I will check it out tonight. Maybe it will fix the tuning problem as well.

73,

--Alex KR1ST

To view this discussion on the web visit https://groups.google.com/d/msgid/gqrx/CACopv_hXd4b3%3DTmnA3_U%3Dt_HngQ-faGZrERUqyaP%3DOMsemjv2w%40mail.gmail.com.

[Alex Kr1st]

Hi Simon,

I will do that as soon as I get home tonight.

Many thanks for your efforts!

73,

--Alex KR1ST

[Alex Kr1st]

That's great news, Alex! Thanks Alex, OE5TKM! Great collaboration.

73,

--Alex KR1ST

[Simon Kennedy]

Still, changing the client to keep the connection open would still be
a good idea IMHO.

This has been done from v2.7.0 of gqrxHamlib. Alex K confirmed it fixed the issue.

Simon. 

[alex.p...@gmail.com]

Hi Simon,

I upgraded to the latest version and I turned off the mode sync on the setup screen, and that took care of the "demod off" problem. I still have to turn the VFO knob very slowly. If I tune at a "normal" speed synching stops with the following error:

pi@raspberrypi:~ $ gqrxHamlib

call set gqrx freq 127.0.0.1 7356 gqrx @ 144242890 @ <socket.socket fd=11, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('127.0.0.1', 40158), raddr=('127.0.0.1', 7356)>

set gqrx   127.0.0.1   b'RPRT 0\n'

set hamlib   127.0.0.1   b'RPRT 0\n'

call set gqrx freq 127.0.0.1 7356 gqrx @ 144242900 @ <socket.socket fd=11, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('127.0.0.1', 40158), raddr=('127.0.0.1', 7356)>

set gqrx   127.0.0.1   b'RPRT 0\n'

set hamlib   127.0.0.1   b'RPRT -8\n'

call set gqrx freq 127.0.0.1 7356 gqrx @ RPRT -14 @ <socket.socket fd=11, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('127.0.0.1', 40158), raddr=('127.0.0.1', 7356)>

Traceback (most recent call last):

  File "/usr/local/lib/python3.5/dist-packages/gqrxHamlib/__init__.py", line 124, in run

    self.setfreq(self.gqrxIPv, GQRX_PORT, 'gqrx', int(rig_freq), sockGqrx)

ValueError: invalid literal for int() with base 10: 'RPRT -14'

I did not experience this problem with the older non-GUI version. Can you try 9600 baud for the CAT speed on your radio and see if that causes a problem on your radio as well? 9k6 is the highest this radio will go. These are the parameters I use for rigctld:

rigctld -m 304 -r /dev/ttyUSB0 -s 9600 --set-conf=serial_handshake=None,rts_state=ON,dtr_state=ON &

If I get some time I'll see if playing with the polling interval in grqxHamlib will help.

73,

--Alex KLR1ST

[Simon Kennedy]

Thanks Alex,

Hamlib or gqrx is returning a return code of -8 or -14, which I am not trapping. I will put some error checking in and look up those error codes. I suspect it might relate to the serial port speed.

Simon.

--
You received this message because you are subscribed to the Google Groups "Gqrx SDR" group.
To unsubscribe from this group and stop receiving emails from it, send an email to gqrx+uns...@googlegroups.com.
To post to this group, send email to gq...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/gqrx/83d3a6f0-ca3f-4cb6-96e2-c2f8760775e7%40googlegroups.com.

[Simon Kennedy]

Hi Alex,

v2.7.4 is available to install. It traps non-zero error codes and prevents trying to pass them to gqrx or Hamlib as frequencies. I can't replicate your exact issue as the CAT interface to my FT920 is fixed at 4800 baud. However, Hamlib error code 8 appears to indicate a protocol error so maybe you can lower the baud rate.

Simon.

To unsubscribe from this group and stop receiving emails from it, send an email to gqrx+unsubscribe@googlegroups.com.

[Alex Kr1st]

Hi Simon,

Thanks for the new version. I will try that when I get home. For some reason I thought your rig was set to a higher baud rate, probably because I run my Yaesus at 38k4.

73,

--Alex KR1ST

[alex.p...@gmail.com]

Hi Simon,

The latest version seems to run fine. Syncing does not get aborted anymore. If I tune at a "normal" speed it takes a bit for it to catch up, but it keeps working. No memory leak either.I didn't try to change the baud rate as I'd have to open up the radio to change some dip switches. Below are the messages that I'm receiving when tuning the radio.

Thanks for all your work!

73,

--Alex KR1ST

set gqrx   127.0.0.1   b'RPRT 0\n'

set hamlib   127.0.0.1   b'RPRT -9\n'

call set gqrx freq 127.0.0.1 7356 gqrx @ 144237310 @ <socket.socket fd=11, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('127.0.0.1', 56436), raddr=('127.0.0.1', 7356)>

set gqrx   127.0.0.1   b'RPRT 0\n'

set hamlib   127.0.0.1   b'RPRT -14\n'

call set gqrx freq 127.0.0.1 7356 gqrx @ 144236970 @ <socket.socket fd=11, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('127.0.0.1', 56436), raddr=('127.0.0.1', 7356)>

set gqrx   127.0.0.1   b'RPRT 0\n'

set hamlib   127.0.0.1   b'RPRT -14\n'

call set gqrx freq 127.0.0.1 7356 gqrx @ 144236050 @ <socket.socket fd=11, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('127.0.0.1', 56436), raddr=('127.0.0.1', 7356)>

set gqrx   127.0.0.1   b'RPRT 0\n'

set hamlib   127.0.0.1   b'RPRT -9\n'

call set gqrx freq 127.0.0.1 7356 gqrx @ 144234060 @ <socket.socket fd=11, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('127.0.0.1', 56436), raddr=('127.0.0.1', 7356)>

set gqrx   127.0.0.1   b'RPRT 0\n'

set hamlib   127.0.0.1   b'RPRT -9\n'

call set gqrx freq 127.0.0.1 7356 gqrx @ 144233240 @ <socket.socket fd=11, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=6, laddr=('127.0.0.1', 56436), raddr=('127.0.0.1', 7356)>

set gqrx   127.0.0.1   b'RPRT 0\n'

set hamlib   127.0.0.1   b'RPRT -9\n'

On Tuesday, April 10, 2018 at 12:26:49 PM UTC-5, Alex Kr1st wrote:

Hi Simon,

Thanks for the new version. I will try that when I get home. For some reason I thought your rig was set to a higher baud rate, probably because I run my Yaesus at 38k4.

73,

--Alex KR1ST

On Apr 10, 2018, at 11:31 AM, Simon Kennedy wrote:

Hi Alex,

v2.7.4 is available to install. It traps non-zero error codes and prevents trying to pass them to gqrx or Hamlib as frequencies. I can't replicate your exact issue as the CAT interface to my FT920 is fixed at 4800 baud. However, Hamlib error code 8 appears to indicate a protocol error so maybe you can lower the baud rate.

Simon.

[Simon Kennedy]

Hi Alex,

glad to hear it. The main error that Hamlib is giving from those you copied into your email is "Command rejected by the rig" so I guess there is a handshake issue on the serial connection.

Still, if it ain't broke then don't fix it!

73 Simon.

G0FCU.