LMS6-1680 Decoder Updates

[Mark Jessop]

Hi all,

I've just pushed some updates to the LMS6-1680 decoder in the testing branch (1.5.4-beta2). These changes are highly experimental, and have the potential to increase decoding performance and handle frequency drift a bit better.

However, this comes at a cost of CPU usage - I've tweaked the sample rates such that it should run in real-time on a RPi 3 or better, but it needs real-world testing.

Could I please have some of the US stations with LMS6-1680 sondes in range try out these updates and report back? In particular I'm interested to know if you seem to get decodes at longer ranges than before, and can track for more of the flight. 

(Still no PTU decoding... sorry)

73

Mark VK5QI

[Chandler Heath]

Hi Mark,

How do I update to the test version?

On the PTU decode, I have some frames I believe are decoded and need someone to check them. Is this something I can work with you on?

--
You received this message because you are subscribed to the Google Groups "radiosonde_auto_rx" group.
To unsubscribe from this group and stop receiving emails from it, send an email to radiosonde_auto...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/radiosonde_auto_rx/CAJsuqdAGzzd%2BGqpc26jVS5RzgG7_y2OxyZjJ%2BxaKixJ80cEPnA%40mail.gmail.com.

[Mark Jessop]

If you are using docker, you need to replace 'latest' in the docker container URL with 'testing', e.g. ghcr.io/projecthorus/radiosonde_auto_rx:testing

If doing a native install, you need to run 'git checkout testing', then 'git pull' in your auto_rx dir, then re-build as normal.

Yes, any information on what fields in the frame correspond to PTU values would be useful.

Cheers,

Mark

[Tom Hanley]

I've been messing around with trying to decode the 400 MHz LMS-6 data ever since the update that captures the raw frames.  I'm sort of reaching my limit in terms of what I can deduce from the data and what I can find online.  I figured I'd post everything I've found thus far here and see if anyone has suggestions on where to go next:

The message payload appears to be 223 bytes long. Byte conventions are big-endian.  This first part is already known in the decoder, but I'm including it here for completeness:

(I'm using byte numbering starting at 1 rather than 0 here):

bytes 1-4, seems to be constant across all soundings (24 54 0 0 hex)

bytes 5-8, serial number, uint32

bytes 9-10, frame number, uint16

bytes 11-14, gps time of week in milliseconds, uint32

bytes 15-18, unknown/some other type of clock signal

bytes 19-22, latitude (deg N) int32 *360/2^32

bytes 23-26, longitude (deg E) int32 *360/2^32

bytes 27-30, altitude (millimeters above WGS84 ellipsoid) int32 (twos complement) 

bytes 31-33, east-west velocity (east +, mm/sec) int24 (twos complement) 

bytes 34-36, north-south velocity (north +, mm/sec) int24 (twos complement) 

bytes 37-39, up-down velocity (up +, mm/sec) int24 (twos complement) 

bytes 40-41, unknown/static

bytes 42-173 are 12 groups of 11 bytes for each of the 12 GPS channels, these include PRN, C/N, pseudorange, etc., but not entirely sure of the full 11 bytes

bytes 174-209, 12 "channels" of 3-byte data where the 7 most significant bits represent a denominator and the least significant 17 a numerator.  This was the hardest part to figure out as I've never seen data stored in this convention.  I don't know of the particular scaling/calibration here, but "channel 3" appears to contain temperature data and "channel 4" humidity.  All the rest of the channels are either empty or contain a different type of data that is fairly consistent across all the channels but not numerically equivalent.  I don't believe these sondes have a pressure sensor onboard from what I can tell.

bytes 210-212, similar data to the temperature mentioned above (same 24 bit numerator/denominator), but only populated every eighth frame, otherwise zero.  This could be data from the temperature sensor next to the humidity sensor as opposed to the one on its own boom or some type of processed temperature.

bytes 213-221, unknown, but minimal variance/values, so not likely raw data

bytes 222-223, very noisy/random, so I'm thinking these are a checksum.

I'm happy to pass along the .raw files to anyone who wants to explore further.  I was thinking one could compare the raw T/RH values to those reported out in the mandatory/significant data at sites such as this: https://ruc.noaa.gov/raobs/

If anyone knows where to find the raw high-resolution (1 Hz) LMS6 data from the US NWS, that would be the perfect source for trying to figure out how to convert raw sensor values.

Regards,
Tom Hanley

To view this discussion on the web visit https://groups.google.com/d/msgid/radiosonde_auto_rx/CAJsuqdDiJRewWadT7jjMn40AvdCGEFmNpPH3SrjhKANGjNt3qw%40mail.gmail.com.

[Mark Jessop]

The LMS6 sondes definitely have a pressure sensor. It's a large bellows-type sensor.

73

Mark

[Tom Hanley]

I've seen your pictures of it and there wasn't one in a recent 403 MHz LMS6 sonde I recovered from Sterling, Virginia and there's no mention of one here either:

https://wiki.recessim.com/view/LMS-6_Radiosonde

The report here mentions that they make versions with and without:

https://www.ecmwf.int/file/265705/download?token=B8Rcgpg6

My guess is they've gone away from using them and relying on GPS + hypsometric equation to save money.

[wtfbsyt]

I have recovered 3 of the LMS6 1680mhz radiosonde recently. Here is a look at the PCB.

Jim,N2NXZ

Sent from my Verizon, Samsung Galaxy smartphone

To view this discussion on the web visit https://groups.google.com/d/msgid/radiosonde_auto_rx/CAJsuqdD%3DB%3DfGiZq7o0p2nGVyCkJ_%2BST5QBi3NNV6JYNN%2B5KdfQ%40mail.gmail.com.

[Chandler Heath]

Hi Tom,

I made a few attempts to decode the frames from balloons, organized them in Excel and did some rudimentary graphing of the rightmost fields after doing some HEC2DEC conversions. I am not sure if the PTU values are 8 or 16 bit. Attached is the excel sheet I used to parse and graph from. I think we are close, hopefully it will help.

[David Marinelli]

Mark:

I am guessing from this note that you no longer need recordings of LMS6-1680, but just in case you do, I have a series of 17 x7 minute long .wav recordings of the Albany, New York sonde, obtained from about 20000 m to 33000m and back to ground.  They were sampled at 1.2Msps and each file is about 2 gigabytes.  If you would like them, let me know the best way to get them to you.

I have two identical rasp-pi3b setups, one running auto_rx 1.5.4 and the other running 1.5.5-beta4.  Next time a sonde comes in range, I will do a head to head comparison and let you know.

Cheers,

Dave

[Mark Jessop]

Yeah, we think we have the drift problems solved now, along with a good improvement in decode performance. 

I haven't really had any confirmation that 1.5.5-beta4 is working OK in the wild, so if you could test and let me know that would be great, so I can release 1.5.5 perhaps this weekend.

73

Mark

--
You received this message because you are subscribed to the Google Groups "radiosonde_auto_rx" group.
To unsubscribe from this group and stop receiving emails from it, send an email to radiosonde_auto...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/radiosonde_auto_rx/54a33126-e7d2-44d4-96e5-e7b47a61cf95n%40googlegroups.com.

[Chandler Heath]

Hi Mark,

I have noticed that the LMS6-1680  doesnt decode as long it normally does with the beta. I have set the decode width to 400k to achieve this, since rolling to beta I have not set that. 

To view this discussion on the web visit https://groups.google.com/d/msgid/radiosonde_auto_rx/CAJsuqdB4oBDQ5%2BNHMagDX91-F39m0d9ncPOg0USh2HX0s9mbnw%40mail.gmail.com.

[Mark Jessop]

Not sure what you mean by the decode width in this case?

In 1.5.5-beta4, the receive bandwidth has been increased to 240 kHz, but we now use IQ data in the demodulator instead of FM-demodulated data. The demod now detects offset frequencies and tracks frequency drift much better than with just a FM demodulator. I had to limit the bandwidth to 240 kHz due to CPU usage concerns (and it was even worse CPU usage in 1.5.5-beta3). 

I had a few stations in upstate New York report good success with 1.5.5-beta3, but perhaps beta4 has gone backwards...

If by setting the decode width to 400 kHz you mean you have increased the FM demod bandwidth to 400 kHz, then this has multiple effects. One is that you will be able track a little bit more drift, but the other is that you now have double the noise going into the demodulator, which means an instant 3dB hit in performance. As it is, the bandwidth setting in 1.5.4 is very wrong (200 kHz!) so *any* increase would have improved performance.

73

Mark 

[David Marinelli]

Mark:

I just finished a head to head comparison of 1.5.5-beta4 to 1.5.4. with the Albany LMS6-1680. I would say based on this one experience that the beta is at least as good and probably better than 1.5.4.  The beta got 12 decodes before 1.5.4 got one.  As the sonde got closer, 1.5.4 briefly passed the beta with 330 decodes to 115.  Then the reverse happened and the beta version got the most decodes at 1151 to 763 decodes, and got its last decode about a minute after 1.5.4. went silent.  I swapped SDRs, LNAs and antennas from time to time to try to exclude hardware effects and results were consistent.

Dave

[Mark Jessop]

Thanks for the feedback!

What I might do is include two options, a 'normal' (what's in 1.5.5-beta4 now) and an 'experimental' option.

'normal' will use the current settings, which uses a faster decimation routine, and doesn't blow out CPU as bad.

'experimental' will use a higher CPU decimation routine, which may have better performance, but pushes a RPi 3 to its limit. 

These settings will be in the 'alternate decode chains' section of the config file: https://github.com/projecthorus/radiosonde_auto_rx/blob/master/auto_rx/station.cfg.example#L466

I'll do a beta5 with these settings updates to let people test out the two different decode chains for a bit before I release.

73

Mark

To view this discussion on the web visit https://groups.google.com/d/msgid/radiosonde_auto_rx/6c3c9548-ecbb-40af-a339-3c7368605292n%40googlegroups.com.

[Mark Jessop]

OK, these changes are made. The testing branch is now at 1.5.5-beta5, with the docker image in the process of being built.

If you want to try out the alternate LMS6-1680 demod settings, you will need to add this line to your station.cfg file: 

https://github.com/projecthorus/radiosonde_auto_rx/blob/testing/auto_rx/station.cfg.example#L484

Please have a go with the different settings and let me know how you go. Note that with the 'experimental' mode, more CPU will be used, and I recommend at least a RPi 3B+.

73

Mark VK5QI

[Tom Hanley]

Getting back to my email about decoding temperature, pressure and humidity, I took a look at some LMS6-1680 .raw files that Chandler provided and am seeing very similar results, albeit with a few tweaks.  The LMS6-1680 sondes seem to send two separate packets of data in each burst.  The first is variable length, where the length in bytes = 44 + number of GPS satellites tracked x 13 and contains the same sort of GPS data as in the 400 MHz sonde, but each channel is stored in 13 bytes instead of 11.  The second packet is always 69 bytes and contains the T/P/RH data fields.  The last two bytes of each payload appear to be a checksum, as in the 400 MHz LMS6 variety.  In the shorter message, there looks to be the same 12 channels of 24-bit (7 bit denominator followed by 17 bit numerator) analog data as I noted earlier.  These start at the 12th byte in the message.  All the data in these channels, along with the 13th channel that appears thereafter look to be the same here as in the 400 MHz one.  The exception for the 1680 MHz case, is that this sonde contains a pressure sensor, with that data being stored in the second "channel" (bytes 15-17).  While this identifies the bytes where the raw T/RH/P data are stored, we still don't know the equations to convert these into real units.  The raw A/D values for pressure, for example, appear to be inversely proportional to the actual pressure (values increasing at higher altitudes).  With enough data logged and comparing to the values that are publicly reported out by the NWS, these could eventually be derived, assuming there aren't too many calibration coefficients that vary from sonde to sonde and don't get transmitted with each data packet.

Regards,

Tom

To view this discussion on the web visit https://groups.google.com/d/msgid/radiosonde_auto_rx/CAJsuqdAQSEZu2F2dgGN3PW6dpz5tRMrx2NJK4FAcXLWvuU4Jvw%40mail.gmail.com.

[Chandler Heath]

Hi Tom,

Thanks for the readout of the data, im glad that it was helpful.

For the A/D values, what are you seeing for the decoded values? Are they somewhere between 0-65535 like in process control I/O ranges for analog sensors?

I have a sonde that I can test with to control temp/humidity and local pressure I can sample locally if we need some correlation.

Thanks for your help!

To view this discussion on the web visit https://groups.google.com/d/msgid/radiosonde_auto_rx/CABf5MoVornt6QZvCTDoeLaHSP9_6pSOb1nyvORxqXzh6V7pYDg%40mail.gmail.com.

[Tom Hanley]

I've attached a csv file of the decoded raw values.  The columns are as follows:

1. Frame Number

2. GPS time of week in seconds

3. GPS altitude in meters

4. Channel 1 raw values (numerator/denominator)

5. Channel 2 raw values (numerator/denominator)

6. Channel 3 raw values (numerator/denominator)

7. Channel 4 raw values (numerator/denominator)

8. Channel 13 raw values (numerator/denominator)

Channels 5-12 all follow the same general trends as Channel 1 and all look to be A/D channels that aren't connected to anything or have floating voltages that all change together, which is why I've only included one of them here to give you a general idea of what's in there.  The data here, thanks again to Chandler, is from WMO station 72649 (Minneapolis), from the 2021-07-13 12:00 UTC launch.

Interestingly enough, what got reported out on the GTS network from the NWS ground station did not include any wind speed or direction information, even though the telemetry shows good GPS lat/lon info throughout the flight.

http://weather.uwyo.edu/cgi-bin/sounding?region=naconf&TYPE=TEXT%3ALIST&YEAR=2021&MONTH=07&FROM=1312&TO=1312&STNM=72649

If only there were a way to feed the sondehub info into GTS, the weather forecast for that region and downstream could've been just a little bit better...

Tom

[Tom Hanley]

One last thing I'll mention for anyone who has a LMS6-1680 sonde around, if you can see a manufacturer/part number for the pressure sensor (assuming it's not made by LMS), we might be able to find some documentation on the equations.  I believe you'll have to include temperature effects along with the raw pressure values to derive the actual pressure, which complicates things.

Tom

[Mark Jessop]

While bellows is probably an off-the-shelf part, it looks like everything around it is custom: 
https://photos.app.goo.gl/aJYZ5fMZC5dKfjXC6

73

Mark

[James Zelazny jr]

I also have recordings using HDSDR if they are of any value to anyone.

These are from the Buffalo,NY 1676 mhz Sonde. If anyone wants them let me know and I will link them to your email.

https://www.dropbox.com/sh/9p5h3ra0ntsbx8p/AADXq_9h5FtZ8EXT-mGf4lMEa?dl=0

Jim,N2NXZ

[Drew Riedle]

Mark

My name is Drew and I am located in Albuquerque New Mexico USA. Just wanted to give you some feedback on the beta5 build.  I have taken one of my PI 3's and have loaded up this experimental version.  Here in New Mexico we use the LMS6-1680 sondes.   I have never been able to decode the sonde immediately upon launch. Now I can with this build.  Also I am getting sequential frames of data about 75% of the time.  In the past this was not the case.  I am now able to track the sondes using my discone antenna to about 3000 meters during the final phase of the flight during descent.  The take off altitude here is 1800 meters.  I think you have greatly increased the performance needed to track the 1680 sondes.

Keep up the good work!

73's

Drew

N5GU

To view this discussion on the web visit https://groups.google.com/d/msgid/radiosonde_auto_rx/CAJsuqdAQSEZu2F2dgGN3PW6dpz5tRMrx2NJK4FAcXLWvuU4Jvw%40mail.gmail.com.