Linrad single channel and dual channel raw file format

[rsku...@gmail.com]

Is there anything on the format of the Linrad single and dual channel raw file recording format that I could get my hands on?

Thanks in advance,

Rick Kunath, K9AO

[Franco Venturi]

Rick,

for most SDRs (including the SDRplay RSPs) it is just a 41 byte header followed by the I/Q stream (or streams if you are running a RSPduo in dual tuner mode) as pairs of I/Q values represented as 16 bit shorts.

The 41 byte header is made of the following fields (in C-like notation):

- int remember_proprietary_chunk; (always REMEMBER_UNKNOWN=-1)

- double timestamp; (seconds since 1/1/1970 00:00:00 UTC)

- double passband_center; (in MHz)

- int passband_direction;

- int rx_input_mode;

- int rx_rf_channels; (single tuner mode -> 1    dual tuner mode -> 2)

- int int rx_ad_channels; (single tuner mode -> 2 (I, Q)    dual tuner mode -> 4 (I1, Q1, I2, Q2))

- int rx_ad_speed; (sample rate in samples/s)

- unsigned char save_init_flag;

To give you a better idea, I am attaching a very simple C program I wrote to read a Linrad file in raw format for the dual tuner case and create a couple of I/Q files that can then be read in a GNU Radio source or any similar application that accepts raw I/Q streams of 16 bit shorts. Just be aware that the program does very little/no error checking, but it does work for me here.

Franco

[SP2BPD]

Franco,

I've compiled your program but have very little success in splitting my 2-channel recordings.

This may be related to the age of my files, more precisely to the version of Linrad in which they were saved.

But also, of the two freshly created files, the program only works successfully in one case.
After reading the file header, it doesn't even perform a single write loop, but throws out a message: “nread is not a multiple of 4 shorts - nread=140” (140 or other number).

Not complaining :-) Just sharing my experience.

Kind regards

Piotr, SP2BPD

[Rick Kunath]

What version of Linrad was used before Piotr? I know Franco found some
errors in how the dual-channel files were saved. Have you recorded some
dual-channel files with Franco's latest and tested those?

What SDR is this?

I know recording and playback in Linrad of Dual-Channel files works well
here for me. And I know Chris Smolinski of Black Cat systems has added
single and dual-channel Linrad file support to his Carrier Sleuth
application (you can select either channel on that for long term
waterfall processing) and also for his SDR Rewind (a playback) application.

Just a thought in case it is useful :)

Rick Kunath, K9AO

[Franco Venturi]

Piotr,

140 seems a very low number for a single read(). Is your I/Q file on a local disk or on share of some sort (NFS or Windows share)?

Also can you try to change the 'BUFSIZE' parameter at the beginning of the source code to say 128 (i.e. a multiple of 4 that is less than 140) to see if the progarm is able to read your file or at least move forward a little bit?

73,

Franco K4VZ

[Franco Venturi]

Piotr,

I modified that little C utility program to keep reading its input until it fills up the input buffer; this way there's no need to check for nread being a multiple of 4 shorts.

It should compile without errors or warnings.

I also ran a quick test test with a sample file generated by Linrad that I have here, and the output looks OK to me.

73,

Franco

[SP2BPD]

Rick,

The old files have been recorded years ago. But the new ones recently, with latest Linrad 5.03 rev. 1067. Some of the new ones recorded with my  Afedri, some in a virtual environment (two software  signal generators combined into  4-channel VAC on input of Linrad). So far only one file of several tested has been effectively split.

Franco,

It is all on my local disk.

With BUFSIZE = 64 they are 2 reads and 2 very small files (64 bytes each) are written. Then message is: "nread is not a multiple of 4 shorts - nread=12".

This is typical result of type 1.

Type 2 is following: With some input files I get two big files on output but they are often of slightly different size (for instance: one is 22005KB while the other 22338KB). 

Kind regards

Piotr, SP2BPD

P.S. I see that we wrote simultaneously. I will refer to your new message later.

[SP2BPD]

Franco, 

Quick test on 2 files. The programs behaves the same: writes two out 2 files od different size. 

I'll try with another files, but not today. It's late in Europe.

[Franco Venturi]

Piotr,

that's very strange.

I just downloaded Linrad from subversion (revision 1068), built it, and ran it for a minute or so streaming the I/Q data from the SDRplay RSPduo (that's the only SDR with two RX that I have here).

I then ran my C program (the latest version I sent out earlier), and the two split files are exactly the same in size:

$ ./split_linrad_raw_file fvtest001.raw fvtest001-A.raw fvtest001-B.raw
remember_proprietary_chunk: -1
timestamp: Sat Sep 14 17:32:32 2024
passband_center: 97.000000MHz
passband_direction: 1
rx_input_mode: 00000026
rx_rf_channels: 2
rx_ad_channels: 4
rx_ad_speed: 2000000
save_init_flag: 0

$ ls -l fvtest001*
-rw-r--r--. 1 franco franco 150863872 Sep 14 17:33 fvtest001-A.raw
-rw-r--r--. 1 franco franco 150863872 Sep 14 17:33 fvtest001-B.raw
-rw-r--r--. 1 franco franco 301727785 Sep 14 17:32 fvtest001.raw

I noticed you wrote that you have an Afedri (while I have a SDRplay RSPduo) and you mention VAC, which means you are running on Windows, while I am running Linux here.

I suspect the problem is that either Linrad saves the I/Q data in a different way for the Afedri (but I doubt it's that), or that my utility has a bug where it works differently on Windows vs Linux.

If you look at the source, it is not even 100 lines of code, so I think it shouldn't be too difficult to figure out why.

Franco

[Franco Venturi]

Piotr,

I think I figured it out.

Windows wants the 'O_BINARY' option when opening a binary file like these Linrad recordings, otherwise it opens them in text mode.

On Linux on the other hand there's no difference between binary files and text files when opening them, and therefore Linux doesn't even have this O_BINARY flag.

I modified the utility program to set the O_BINARY option (which I am defining as 0, if it isn't defined), so it should hopefully work the same way both on Linux and on Windows.

Attached is the latest version with that O_BINARY flag.

Franco

[Piotr Hewelt]

Franco,

Yes, now it works without a problem. Tested on several files of different origin.
The only warning I received when compiling under Windows was about line #24 and data types. I neutralized it as follows:

time_t timestamp = (int) timestamp_double;

Thank you and have a good day

Piotr, SP2BPD

[Leif Asbrink]

Hi All,

Today I uploaded Linrad-06.00-pre-r1069 on the repo. Revision 1069.

This version is a major rewrite of the heart of the receiver, the code
in narrowband_dsp(), a routine in wcw.c.

Linrad was originally written for Pentium 3 and older processors with only
a single cpu core. The logics to ensure that processing was done in the
optimum order was quite complicated.

I have now split the processing on three threads, that made things easier
since each thread just waits on a semaphore until the preceeding thread
has finished one data block.

My main reason for this is that I wanted better performance for the sideband
noise measurments, correlation mode 2. A greater ratio of smallest to largest
frequency offset for phase noise.

Linrad-06 is in an early state. CW and SSB modes are hopefully OK. AM is
hopfully also OK,but FM does not work and the transmit modes do definitely
not work.

The number of possible combinations of parameters is overwhelming. I have
only tested a couple of cases. Some failed - and as it turns out they failed
on linrad-05 also. I have corrected the mistakes causing the problems, but
there are likely other parameter combinations that cause problems.

In case you can find a reproducible problem in receive modes for CW or SSB,
please report on the mailing list.

Linrad-06 runs under FreeBSD and NetBSD. That is work by Ramiro Aceves, EA4NZ
who tested:
-NetBSD 10.0 amd64 in an Intel Nuc 8i7 PC
-NetBSD 10.0 amd64 in an Lenovo Thinkpad x260 laptop
-NetBSD 10.0 aarch64 in a Raspberrypi 4
That was before the major structural changes in the program flow, but I do not
think that would affect operation under BSD.

73

Leif

[Ramiro Aceves]

Hello Leif

Yesterday I discoverd that my home FM radio receiver  was a little out
of frequency and wanted to investigate it further.  I have just started
my GPSDO that have been switched off for years and appear not to work at
all, I does not adquire "fix", so I decided to use Linrad for something
more interesting than playing with Linux and the BSDs.  ;-)

I upgraded to the r1069 revision and notice some odd behaviours.
Sometimes it is difficult to abandon with X, sometimes the high
resolution graph freezes and you cannot pick a new frequency with the
mouse in the main graph.

I have to play more with it and give you more detailed and useful report.

This is under Debian GNU/Linux to avoid confussion.

Regards.



El 10/10/24 a las 22:42, Leif Asbrink escribió:

[Franco Venturi]

Thanks for the major rewrite Leif!

This morning I had some time so I compiled version 6.00pre (revision 1069) on my Linux Fedora.

The GCC compiler gave me some errors like this:

wide_graph.c: In function ‘wide_graph_add_signal’:
wide_graph.c:404:20: error: array subscript 1 is above array bounds of ‘double[1]’ [-Werror=array-bounds=]
  404 |     if(mix1_selfreq[i]<0)
      |        ~~~~~~~~~~~~^~~
In file included from wide_graph.c:34:
seldef.h:205:15: note: while referencing ‘mix1_selfreq’
  205 | extern double mix1_selfreq[MAX_MIX1];
      |               ^~~~~~~~~~~~

To take care of those errors, I surrounded those for loops (for(i=1; i<genparm[MIX1_NO_OF_CHANNELS]; i++)) with an 'if' statement on MAX_MIX1 like this:

    if(MAX_MIX1 > 1)

These are the places where I had to add that condition:

hires_graph.c:775
screensub.c:1129
screensub.c:1174
screensub.c:1216
wide_graph.c:402

I put the files with the changes in the GitHub repository: https://github.com/fventuri/linrad

I then ran the GitHib CI pipeline and it was able to compile it for Windows 64 bit without errors.

In case some wants to try it in Windows, you can find the binary for Windows 64 bit here: https://github.com/fventuri/linrad/releases/tag/6.00pre-1069-SM5BSZ

73,

Franco K4VZ

[Leif Asbrink]

Hello Ramiro,

I ran into a similar problem yesterday. If you enable DUMPFILE in
options.h you will get a list of threads where you can see which
one that does not exit. To know what the numbers mean, look in thrdef.h

Threads should exit when kill_all_flag is set or when thread_commandflag
is set to THRFLAG_KILL. A typical error is something like this:

while(condition)
{
lir_sleep(10000);
}

There has to be a check for a command to exit inside the loop.

There is a new version on the repo now.

73

Leif

[Leif Asbrink]

Hello Franco,

Interesting. It is an obvious programming error, but it was not
detected by my version of gcc that comes with the latest stable
release of Debian. Vers 12.7 bookworm. It is gcc version 12.2.0-14

I have added the test for MAX_MIX1 as you suggested, fixed a couple
of bugs and uploaded the new version: r1070.

73

Leif

[Franco Venturi]

Thanks Leif.

I just rebuilt the binary for Windows 64 bit based on subversion revision 1070: https://github.com/fventuri/linrad/releases/tag/6.00pre-1070-SM5BSZ

73

Franco

[Ramiro Aceves]

El 12/10/24 a las 18:31, Leif Asbrink escribió:

> Hello Ramiro,
>
> I ran into a similar problem yesterday. If you enable DUMPFILE in
> options.h you will get a list of threads where you can see which
> one that does not exit. To know what the numbers mean, look in thrdef.h
>
> Threads should exit when kill_all_flag is set or when thread_commandflag
> is set to THRFLAG_KILL. A typical error is something like this:
>
> while(condition)
> {
> lir_sleep(10000);
> }
>
> There has to be a check for a command to exit inside the loop.
>
> There is a new version on the repo now.
>
> 73
>
> Leif
>

Hi Leif, SVN updated the source coded and compiled again, I have been
playing with Linrad for a while and seems to be smooth now!. Thanks for
fixing it!

I'll be playing more during the day.

Regards.

Ramiro.