radio-astronomy FFT recording with LimeSDR on Raspberry PI 3

[Mario C]

Hello.

I just received my LimeSDR and would like to use it with SoapySDR.

I recently worked with RTL-SDR dongles using a modified version of AD-Vega/rtl-power-fftw  driven by some python programs ( https://www.hackster.io/mariocannistra/radio-astronomy-with-rtl-sdr-raspberrypi-and-amazon-aws-iot-45b617 ).

I'm essentially writing the FFT values for certain portions of the radio spectrum to binary files and then rendering them with python as spectrograms with frequency on y and time on x.

For next version of this project I'd like to use R-PI 3 + LimeSDR with SoapySDR, possibly still driving all of this with python programs. 

I know of the bandwidth limitations i will have with the USB 2 available on the R-PI but I can live with that for the moment.

Possible further evolution could be on a pc, still on debian jessie but with USB 3 available. Unfortunately with higher power consumption due to the PC... will see.

The sw I'm using now is working at high speed in C++ to run FFTW on the IQ stream and save it to file, without losing IQ buffers or introducing delays and this allows me to work also with short integration time when needed.

I don't need a UI or interactive usage since this setup runs unattended all night long and potentially 24/7 for Sun monitoring projects as well.

I'm now seeking guidance on the following:

1) can I plan to build SoapySDR on the R-PI ? 

2) which of these 3 is feasible and/or simpler and/or best performing:

a) can I think of running FFTW on the IQ stream with SoapySDR in Python ?

b) or using specific Photos toolkits in a new C++ program ?

c) or is it possible to get the FFTW output directly from some LimeSuite component through SoapySDR avoiding the UI ?

3) do I need to build also LimeSuite or part of it to have the LimeSDR driver available to SoapySDR ?

I'm obviously open to any feedback and help.

Thank you,

Mario

[Josh Blum]

On 03/09/2017 08:51 AM, Mario C wrote:
> Hello.
> I just received my LimeSDR and would like to use it with SoapySDR.
>
> I recently worked with RTL-SDR dongles using a modified version of
> AD-Vega/rtl-power-fftw driven by some python programs (
> https://www.hackster.io/mariocannistra/radio-astronomy-with-rtl-sdr-raspberrypi-and-amazon-aws-iot-45b617
> ).
> I'm essentially writing the FFT values for certain portions of the radio
> spectrum to binary files and then rendering them with python as
> spectrograms with frequency on y and time on x.
>
> For next version of this project I'd like to use R-PI 3 + LimeSDR with
> SoapySDR, possibly still driving all of this with python programs.
> I know of the bandwidth limitations i will have with the USB 2 available on
> the R-PI but I can live with that for the moment.
>
> Possible further evolution could be on a pc, still on debian jessie but
> with USB 3 available. Unfortunately with higher power consumption due to
> the PC... will see.
>
> The sw I'm using now is working at high speed in C++ to run FFTW on the IQ
> stream and save it to file, without losing IQ buffers or introducing delays
> and this allows me to work also with short integration time when needed.
>
> I don't need a UI or interactive usage since this setup runs unattended all
> night long and potentially 24/7 for Sun monitoring projects as well.
>
> I'm now seeking guidance on the following:
>
> 1) can I plan to build SoapySDR on the R-PI ?

SoapySDR and LimeSuite should build on arm just fine FWIW

>
> 2) which of these 3 is feasible and/or simpler and/or best performing:
> a) can I think of running FFTW on the IQ stream with SoapySDR in Python ?
> b) or using specific Photos toolkits in a new C++ program ?
> c) or is it possible to get the FFTW output directly from some LimeSuite
> component through SoapySDR avoiding the UI ?

Theres a lot of options, so this is a pretty wide open question. In
general, python is going to be easier, but orders slower than a C/C++
program.

You can certainly use SoapySDR for python access to the LimeSDR, and
even for taking the FFTs.

For a C/C++ app its possible to build something against SoapySDR C/C++
API or even LimeSuite C/C++ API itself to get access to the samples. And
using something like FFTW or kissftt to do the Fourier transform.

Pothos has some FFT blocks based around kissftt:
https://github.com/pothosware/pothos-comms/tree/master/fft

>
> 3) do I need to build also LimeSuite or part of it to have the LimeSDR
> driver available to SoapySDR ?

When you configure limesuite with cmake, it prints a component summary.
So if SoapySDR development files are installed, you will see SoapyLMS7
as one of the enabled components in the cmake output.

-josh

[Tony B.]

Josh,

We have been trying to understand the differences between LimeSuite software (GUI and Lime device interface) and the SoapySDR interface.
These are not the same, but one might think that from the way some materials describe them at high levels.
Can you help us to understand what underlies the difference between the LimeSuite native API and the SoapyLMS7 interface?

For example, there are some simple things that one cannot do with the the SoapyLMS7 interface:   get board temperature, get RSSI, get / set decimation (lms7002m hardware).

Is it the case that the SoapySDR interface was late to the game and that's why the LimeSuite GUI uses the native interface instead?
Also fascinating is that Andrew Back recommended using the Python calibration tool you wrote *outside* of the LimeSuite GUI.  That is sure to be confusing, as it uses the SoapySDR (a good thing).
https://discourse.myriadrf.org/t/what-about-iq-imbalance-dc-offset-calibration/900/2

Is there some overview of the differences between the interfaces?
Thanks for helping us understand the ecosystem relationships.

Tony

[Josh Blum]

On 03/12/2017 04:50 PM, Tony B. wrote:
> Josh,
>
> We have been trying to understand the differences between LimeSuite
> software (GUI and Lime device interface) and the SoapySDR interface.
> These are not the same, but one might think that from the way some
> materials describe them at high levels.
> Can you help us to understand what underlies the difference between the
> LimeSuite native API and the SoapyLMS7 interface?

SoapySDR is intended to be a general purpose interface for SDR devices,
and it has a large number of hardware support modules basically hiding
under the same API.

This enables applications to be written against one API and support many
devices, and it enables devices to be wrapped in an interface that is
already supported amongst multiple applications.

And then LimeSuite is to the LimeSDR, just as libhackrf is to the
HackRF. So it has its own low level C and C++ driver library.

>
> For example, there are some simple things that one cannot do with the the
> SoapyLMS7 interface: get board temperature, get RSSI, get / set
> decimation (lms7002m hardware).

Well the decimation is definitely settable through the sample rate. The
RSSI and temperature may not be accessible at present, but there is a
place for them to be added (the sensors API in soapysdr).

In general soapysdr does a good job of making LimeSDR useful in SDR
applications, but its never going to really expose the entire
flexibility of the LMS7002M. That said, its possible to give direct
register access, so no door is truly closed.

>
> Is it the case that the SoapySDR interface was late to the game and that's
> why the LimeSuite GUI uses the native interface instead?

The gui started out as an evaluation GUI for the LMS7002M on the EVB7
board. Its come a long way since then, but its really about exposing
controls for calculations for every single register, stuff you would
expect in a board eval utility. But not necessary ideal for generic
tuning and streaming purposes.

> Also fascinating is that Andrew Back recommended using the Python
> calibration tool you wrote *outside* of the LimeSuite GUI. That is sure to
> be confusing, as it uses the SoapySDR (a good thing).
> https://discourse.myriadrf.org/t/what-about-iq-imbalance-dc-offset-calibration/900/2

The main reason behind this is that the SoapySDR wrappers for LimeSDR
focused on being able to re-tune rapidly for working with in in guis
like gqrx (for example). So we didnt want to re-calibrate every every
tune, which would otherwise be ideal for a fixed application.

The tradeoff was to look up corrections in a table, and to provide a
utility to generate calibration corrections in an offline fashion.

It makes quite near plots BTW. But I dont like that the utility brings
in extra dependencies, and it could probably be improved based on recent
calibration improvements that were just merged in LimeSuite. So while I
think that this is something that will be improved, we will probably
stick with offline calibration for the soapy sdr use case.

>
> Is there some overview of the differences between the interfaces?
> Thanks for helping us understand the ecosystem relationships.

I made some blogs on this FWIW:

* https://myriadrf.org/blog/limesdr-application-ecosystem/
* https://myriadrf.org/blog/limesuite-driver-architecture/

Hope that helps
-josh

[Tony B.]

On Sunday, March 12, 2017 at 10:10:20 PM UTC-6, Josh Blum wrote:

 

On 03/12/2017 04:50 PM, Tony B. wrote:
> Josh,
>
> We have been trying to understand the differences between LimeSuite

>

> For example, there are some simple things that one cannot do with the the
> SoapyLMS7 interface:   get board temperature, get RSSI, get / set
> decimation (lms7002m hardware).

Well the decimation is definitely settable through the sample rate. The
RSSI and temperature may not be accessible at present, but there is a
place for them to be added (the sensors API in soapysdr).

I had already seen where to add a temperature check them in the sensors API, but that led me to wonder whether there weren't enough SDR that could do it to warrant an API abstraction.
I can read the temperature from my usrp with two lines of code.  It looks like with the lms7002d, it's done this way:  https://github.com/myriadrf/LimeSuite/blob/b8372652aa26e9b5837d7fc236c3ff82488d18da/src/lms7002m/LMS7002M.cpp#L2547.

As for decimation, I'm talking about direct setting of the programmable rate decimation in the half band filters (p.10 of  http://www.limemicro.com/wp-content/uploads/2015/09/LMS7002M-Data-Sheet-v2.8.0.pdf)
LimeSuite reads these and sets these directly, here are a few hits for reading via lms->Get_SPI_Reg_bits(LMS7param(HBD_OVR_RXTSP):P

https://github.com/myriadrf/LimeSuite/blob/827cb0402b7b606505e58ac637876a1bc6f19958/src/lms7suiteAppFrame.cpp#L70
https://github.com/myriadrf/LimeSuite/blob/e70637205975820c739e1fd4943eca7e2878d354/src/API/lms7_device.cpp#L120
https://github.com/myriadrf/LimeSuite/blob/88c275b5ce8b42045b4ee3250d5109cea6324ac7/src/API/lms7_api.cpp#L525

The last ref is within LMS_SetClockFreq, which is called when there is a wxWidget event.
Within the RxTSP tab, one sets the HBD ratio directly.
I know you are familiar with these, but I don't see how setting the sample rate in the SoapySDR API, after tracing through Device.cpp and SoapyLMS7/*, pulls on the programmable hardware decimation or reads it.
Am I misunderstanding something?

These three examples were what I thought would be relatively simple examples of things I found in LimeSuite but not in the soapysdr lime api.

In general soapysdr does a good job of making LimeSDR useful in SDR
applications, but its never going to really expose the entire
flexibility of the LMS7002M. That said, its possible to give direct
register access, so no door is truly closed.

I understand there are many, many registers in LimeSuite / the lms7002m api.
This is the crux of what I'm asking:  does one have to look through all of both APIs and a fair amount of code to understand what's possible in each?

>
> Is it the case that the SoapySDR interface was late to the game and that's
> why the LimeSuite GUI uses the native interface instead?

The gui started out as an evaluation GUI for the LMS7002M on the EVB7
board. Its come a long way since then, but its really about exposing
controls for calculations for every single register, stuff you would
expect in a board eval utility. But not necessary ideal for generic
tuning and streaming purposes.

> Also fascinating is that Andrew Back recommended using the Python
> calibration tool you wrote *outside* of the LimeSuite GUI.  That is sure to
> be confusing, as it uses the SoapySDR (a good thing).
> https://discourse.myriadrf.org/t/what-about-iq-imbalance-dc-offset-calibration/900/2

The main reason behind this is that the SoapySDR wrappers for LimeSDR
focused on being able to re-tune rapidly for working with in in guis
like gqrx (for example). So we didnt want to re-calibrate every every
tune, which would otherwise be ideal for a fixed application.

The tradeoff was to look up corrections in a table, and to provide a
utility to generate calibration corrections in an offline fashion.

It makes quite near plots BTW.

Thanks for this explanation.  I was able to work through an issue (posted to github) in the Python code and generate the plots.  Your app shows the power of what one can do with SoapySDR and matplotlib and is a nice learning tool.  I found your matplotlib import statement inside a function definition, and since I had not seen that before, looked up why one might do that:  http://stackoverflow.com/questions/1024049/is-it-pythonic-to-import-inside-functions

But I dont like that the utility brings
in extra dependencies, and it could probably be improved based on recent
calibration improvements that were just merged in LimeSuite. So while I
think that this is something that will be improved, we will probably
stick with offline calibration for the soapy sdr use case.


>
> Is there some overview of the differences between the interfaces?
> Thanks for helping us understand the ecosystem relationships.


I made some blogs on this FWIW:

* https://myriadrf.org/blog/limesdr-application-ecosystem/
* https://myriadrf.org/blog/limesuite-driver-architecture/

I read these some time ago, but thank you for the reminder.

Tony

[Tony B.]

On Monday, March 13, 2017 at 9:13:17 PM UTC-6, Tony B. wrote:

I had already seen where to add a temperature check them in the sensors API, but that led me to wonder whether there weren't enough SDR that could do it to warrant an API abstraction.
I can read the temperature from my usrp with two lines of code.  It looks like with the lms7002d, it's done this way:  https://github.com/myriadrf/LimeSuite/blob/b8372652aa26e9b5837d7fc236c3ff82488d18da/src/lms7002m/LMS7002M.cpp#L2547.

I meant lms7002m.  Also, I don't mean to say that the usrp api is more elegant or useful.  I was just illustrating that both SDR give you ways to relatively simply obtain a temperature.

[Josh Blum]

On 03/13/2017 08:13 PM, Tony B. wrote:
>
>
> On Sunday, March 12, 2017 at 10:10:20 PM UTC-6, Josh Blum wrote:
>
>
>>
>> On 03/12/2017 04:50 PM, Tony B. wrote:
>>> Josh,
>>>
>>> We have been trying to understand the differences between LimeSuite
>>
>>>
>>> For example, there are some simple things that one cannot do with the
>> the
>>> SoapyLMS7 interface: get board temperature, get RSSI, get / set
>>> decimation (lms7002m hardware).
>>
>> Well the decimation is definitely settable through the sample rate. The
>> RSSI and temperature may not be accessible at present, but there is a
>> place for them to be added (the sensors API in soapysdr).
>>
>
> I had already seen where to add a temperature check them in the sensors
> API, but that led me to wonder whether there weren't enough SDR that could
> do it to warrant an API abstraction.
> I can read the temperature from my usrp with two lines of code. It looks
> like with the lms7002d, it's done this way:
> https://github.com/myriadrf/LimeSuite/blob/b8372652aa26e9b5837d7fc236c3ff82488d18da/src/lms7002m/LMS7002M.cpp#L2547.

I just added this for the SoapyLMS7, so device->readSensor("lms7_temp")
will return the temperature in degrees C

>
> As for decimation, I'm talking about direct setting of the programmable
> rate decimation in the half band filters (p.10 of
> http://www.limemicro.com/wp-content/uploads/2015/09/LMS7002M-Data-Sheet-v2.8.0.pdf
> )
> LimeSuite reads these and sets these directly, here are a few hits for
> reading via lms->Get_SPI_Reg_bits(LMS7param(HBD_OVR_RXTSP):P
>
> https://github.com/myriadrf/LimeSuite/blob/827cb0402b7b606505e58ac637876a1bc6f19958/src/lms7suiteAppFrame.cpp#L70
> https://github.com/myriadrf/LimeSuite/blob/e70637205975820c739e1fd4943eca7e2878d354/src/API/lms7_device.cpp#L120
> https://github.com/myriadrf/LimeSuite/blob/88c275b5ce8b42045b4ee3250d5109cea6324ac7/src/API/lms7_api.cpp#L525
>
> The last ref is within LMS_SetClockFreq, which is called when there is a
> wxWidget event.
> Within the RxTSP tab, one sets the HBD ratio directly.
> I know you are familiar with these, but I don't see how setting the sample
> rate in the SoapySDR API, after tracing through Device.cpp and SoapyLMS7/*,
> pulls on the programmable hardware decimation or reads it.
> Am I misunderstanding something?
>
> These three examples were what I thought would be relatively simple
> examples of things I found in LimeSuite but not in the soapysdr lime api.

Its a matter of making the LimeSDR usable in most common SDR
applications. So rather than caring about the decimation or the clock
rate, all the user really cares about is the sample rate.

So if you call device->setSampleRate(SOAPY_SDR_RX, 0, some_rate). The
decimation and clock rate are automatically set.

Supposing that you wanted a particular clock rate, you could set that too:

device->setMasterClockRate(clock_rate_in_hz)
device->setSampleRate(SOAPY_SDR_RX, 0, some_rate)

In this case, SoapyLMS7 respects your choice of clock rate, so setting
the sample rate selects one of the possible decimations or errors out if
its not a power of two relation.

>
>
>> In general soapysdr does a good job of making LimeSDR useful in SDR
>> applications, but its never going to really expose the entire
>> flexibility of the LMS7002M. That said, its possible to give direct
>> register access, so no door is truly closed.
>
>
> I understand there are many, many registers in LimeSuite / the lms7002m api.
> This is the crux of what I'm asking: does one have to look through all of
> both APIs and a fair amount of code to understand what's possible in each?
>

While theres always more bells and whistles, the general stuff is
accessible through any of these api options.

* setting a center frequency
* modifying gains
* selecting RF antenna path
* setting a sample rate
* streaming

You can basically ignore the other stuff as advanced use if you intend
to do something very specific.

BTW, did you see this header. Theres quite a lot of files within the
LimeSuite, but there is basically a single nice C wrapper for working
with the LimeSDR that has nice documentation:
https://github.com/myriadrf/LimeSuite/blob/master/src/lime/LimeSuite.h

-josh