Connect to HL2 over VPN

[Nigel Head]

I have tried to get my HL2b8 working over a VPN (remote LAN) with the idea of remotely operating the HL2 located at a nearby excellent hilltop site (my local radio club site) where we have a full time internet connection.

I failed to get Quisk to 'see' the HL2 over the VPN.

I tried with a router to router VPN and a VPN software client, neither work.

I don't want to run a remote desktop or VNC connection at the hilltop site.

After a bit of digging, I think the problem is that Broadcast packets are not sent over a VPN link and Quisk and the HL2 need that to instigate communications?

Anyone got an HL2 running remotely over a VPN - is it actually possible?

Can the HL2 have a 'fixed' IP address and access it remotely that way?

Nigel - G4ZAL

[Steve Haynal]

Hi Nigel,

I am actually working on something very similar where I want to place a HL2 at my sister's house in the country. I'll be going the VNC route as my sister's bandwidth is not sufficient for HL2 traffic.

I think you are correct that VPN or ssh tunnel will not pass UDP broadcast packets, and the discovery process requires this.

You can build your own gateware with a fixed IP. Search for "localparam       IP = {8'd0,8'd0,8'd0,8'd0};" and enter your desired fixed IP. You must then rebuild the hl2b5up gateware. This is easiest to do by using the Quartus (free tool) GUI and opening hermeslite.qpf. Make sure the hl2b5up subproject is selected. If this is difficult for you, let me know the IP you'd like and I will send you a custom gateware.

I'm not sure if/how you will get software to recognize the HL2 even with fixed IP without broadcast.

This is something we should definitely work out. I'd like to help with more remote use of the HL2.

73,

Steve

kf7o

[Alan Hopper]

Hi Steve,

I think we still really need a discovery response so software can detect things like firmware version. One option might be to allow discovery response from a direct(non broadcast) request.  Software can then be given a list of IPs to send discovery packets to, from then on the software can work as normal.

73 Alan M0NNB 

[Nigel Head]

Hmmm, downloading Quartus and support files.... ~3Gb

I don't know the first thing about this, but willing to have a bash at it - hopefully I won't brick my HL2 !

I'm recently retired so have a bit of time and willing to learn, but coming from a predominantly mechanical engineering background, if something doesn't quite fit, you hit it with a hammer and if that didn't work, you get a bigger hammer !!

I guess if we are to get HL2 running over a VPN, then the only way is to get away from Broadcast requests (as these are not passed over a VPN).  Alan's suggestion implies a change to all HL2 supported software as well?

Looking at it, the simplest way is to run a remote desktop/VNC with audio passed over the connection.  I wonder how good that will be?

Problem for me is that at our club site, we only have solar power most of the time (except club nights/activity when we run a diesel generator with 15Kw ability).  Maybe a R-Pi desktop might be do-able at the remote site (as it consumes relatively little DC power)?

Nigel.

[Alan Hopper]

Nigel,

you might find that powerSDR will work without discovery, I think it has an option to add a fixed ip address, I have not tried this. The radio does not need the discovery to work, you can see this by discovering it in Spark and then powering the radio down and up, the radio will then happily run.  If you know the ip address of the radio you could test with powersdr without rebuilding the firmware.

If this does work with no mods I would still not want to do this in Spark as without discovery the user would have have to enter things like number of receivers, firmware version etc which is just asking for trouble.

73 Alan M0NNB

[Christopher KB3CS]

if you have bandwidth to burn and an unfiltered machine at your disposal, yes, i think it's possible with a L2 VPN instead of the usual L3 VPN.

  https://www.softether.org/

  - 11 (base 72) -

On Wednesday, September 11, 2019 at 4:09:49 PM UTC-4, Nigel Head wrote:

[...]

[Jaroslav Škarvada]

I used DHCP server which set fixed IP address to my Hermes Lite 2.0
according to its MAC address. I ran patched Quisk where it used the IP
address from the configuration dialog (Hardware/IP setting and netmask)
to communicate directly with the radio without discovery (or more
precisely, when the discovery via broadcast failed it tried the direct
configured IP). This worked reliable and it should also work through
tunnels. Recently, there were some changes in the Quisk, so maybe the
current Quisk can do this now out of the box without patching (I haven't
tried)

73! Jaroslav, OK2JRQ


'Alan Hopper' via Hermes-Lite napsal(a):

> --
> You received this message because you are subscribed to the Google
> Groups "Hermes-Lite" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to hermes-lite...@googlegroups.com
> <mailto:hermes-lite...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/hermes-lite/fa2d5977-d08b-4884-82cc-c81e119f9020%40googlegroups.com
> <https://groups.google.com/d/msgid/hermes-lite/fa2d5977-d08b-4884-82cc-c81e119f9020%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Don Solberg]

My ham shack is located 200 miles from my home so I operate remotely most of the time.  I use a VNC, and HL2 with Powersdr works very well.  I had been using TeamViewer but they recently disconnected me for "commercial" use.  I have filled out and submitted their appeal form but one week later still no TeamViewer.  I switched to Any Desk and it is working very well.  I have been using the HL2 has a panadapter and I can also listen to the audio with Any Desk.

73,

Don
K9AQ

[James Ahlstrom]

Hello Group,

I believe Jaroslav is correct. I patched my Quisk with the known IP address of the HL2, and Quisk started up and worked fine. No changes in gateware (FPGA code on the HL2) were required. But there are some complications.

As Alan pointed out, the broadcast discovery method returns the firmware version (code version). This may be needed to determine what commands the HL2 accepts. But the firmware version is also returned as C4 when C0 is zero. So we can get the firmware version after the HL2 starts sending samples. We would really like to know it before then, but maybe it is not really important.

The HL2 IP address on the Quisk config screen is used to set an IP after the HL2 is discovered by the broadcast method. That is part of the Hermes protocol, although I believe the HL2 does not implement it. So to support Hermes hardware other than the HL2, an additional IP field (or an option) is required.

You still have the problem of knowing the IP address of the HL2. If it was assigned by DHCP, you have to get it from the DHCP server. Possible but tedious. Most (maybe all) DHCP servers can be programmed to assign a fixed known IP address based on the MAC, so that solves the problem.

So I believe no changes in gateway code are required, and I can just release a Quisk that has a field for a known IP. Since Quisk is open source, other software can just copy the code. But a few gateware changes might be helpful. I tried sending the discover packet to the known IP address instead of a broadcast address and the HL2 did not answer. If it did answer, we could get the firmware version. And if the gateware could accept and store a known IP address in the digital bias pots, Quisk could program a known IP without changing the gateway code.

Jim

N2ADR

[Nigel Head]

Jim,

Where can I find/how do I apply your patch?

I tried as per Jaraslov and result is the same (my Quisk is currently 'standard' 4.1.44) - I used the field in Quisk config screen, but now understand it's different from the patch.

I have set my HL2 in my router and can Bind the MAC to an IP (can also do the same at the remote site), so the HL2 LAN IP should stay 'static' even after router reboot(s).

I would much prefer to stay on a standard gateware rather than attempting to roll-my-own and a field in Quisk for known/fixed IP addressing would be wonderful !

Thanks in advance,

Nigel - G4ZAL

PS - Haven't tried powerSDR etc

[Alan Hopper]

Hi List,

 just tested a modified (very tiny mod) firmware that allows discovery to a known ip address and it appears to work here.  There is a pr here https://github.com/softerhardware/Hermes-Lite2/pulls if anyone wants to play with it. It won't magically work with the current Spark release.

73 Alan M0NNB

[Alan Hopper]

Hi List,

just had a thought, if the subnet mask is small it is quite possible to send a discovery packet to every ip address, so with the mod above everything could work with no extra user configuration.  What is the typical setup for subnet masks with vpns people might use?

73 Alan M0NNB

[Nigel Head]

Hi Alan,

When using my software VPN client and connecting to the remote site, my IP for the VPN connection is 192.168.x.x/255.255.255.255 according to Windoze 7.

I know the remote site router issues a /24 (255.255.255.0) subnet to LAN clients.

HTH

Nigel.

[Alan Hopper]

Hi Nigel,

maybe specifying a range or mask of ips is the simplest way, eg  192.168.1.x . Sending out a few hundred discovery packets is nothing in the grand scheme of things and would allow the remote radio to use dhcp.  I'm just trying to reduce the amount of configuration needed.

73 Alan M0NNB

[James Ahlstrom]

Hello Nigel,

The test version is in Quisk and will be in the next release 4.1 45. I just wanted to wait a bit in case someone has a different idea of what should be done.

Jim

N2ADR

[James Ahlstrom]

Hello Alan,

When there is more than one network interface in the PC, each one has its own broadcast address. Quisk has code to get a list of interfaces and send a broadcast to each one. I forget who needed that, but someone did or it wouldn't be in Quisk!

Jim

N2ADR

[Alan Hopper]

Hi Jim,

yep Spark does the same, broadcast seems a bit odd, sending to a defined ip is much simpler as the os picks the correct interface.  I'm not too hot on what goes on in vpns so am rather guessing as to what is best.

73 Alan M0NNB

[Steve Haynal]

Hi Alan,

Thanks for the patch. I was thinking along similar lines last night but you beat me to it. I will try to release gateware with this patch this weekend. I still want to check into the problem Taka reported and include any required fix in the next gateware. Thanks also for the programming C# code.

73,

Steve

kf7o

[gerry kavanagh]

One possible solution might be to use an MPLS tunnel. Two cheap OpenWRT or Mikrotik routers could do this.

There woudl be some additional traffic overhead, but I guess this will apply to any tunnel.

/ Gerry

[James Ahlstrom]

Hello Group,

The new Quisk 4.1.45 has a new config screen hardware option "Hermes known IP". If you know the IP address of the Hermes hardware, enter it there. Otherwise Quisk will search for the hardware using the usual broadcast method.

More specifically, if an IP is entered, that IP (and not a broadcast address) will be used for a discover message. Steve's new firmware will answer with the firmware version. Older firmware will not answer, but the address will be used anyway, and the HL2 will operate with an assumed firmware version of 62. So this feature should work with old or new firmware.

Jim

N2ADR

[Roger Critchlow]

This directed discovery procedure works in the crostini linux container on my chromebook.  I wasn't able to get broadcast discovery to work from inside the container, because the container networking is constrained, but the direct discovery just worked fine for quisk.  The audio is a little crackly and the quisk config>status page is reporting errors in audio handling and unstable audio latencies, but at 48ksps the UDP latency is stable and error free.  At 92kbps some UDP errors start to show up.  This is over WiFi with the HL2 plugged into the access point.

-- rec -- ad5dz --

--

You received this message because you are subscribed to the Google Groups "Hermes-Lite" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hermes-lite...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hermes-lite/f27dc224-dedf-4566-9623-4db9b288b9b9%40googlegroups.com.

[Nigel Head]

I'm away at the moment, but will be back for the weekend and will load the new gateware and install Jim's new Quisk version and test just as soon as I can...

Nigel - G4ZAL

[Jaroslav Škarvada]

James Ahlstrom napsal(a):

> --
> You received this message because you are subscribed to the Google
> Groups "Hermes-Lite" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to hermes-lite...@googlegroups.com
> <mailto:hermes-lite...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/hermes-lite/f27dc224-dedf-4566-9623-4db9b288b9b9%40googlegroups.com
> <https://groups.google.com/d/msgid/hermes-lite/f27dc224-dedf-4566-9623-4db9b288b9b9%40googlegroups.com?utm_medium=email&utm_source=footer>.

Hi Jim,

thank you very much for this

73! Jaroslav, OK2JRQ

[Steve Haynal]

Hi Jim,

Thanks for adding the known IP feature. Also, thanks for fixing the bandscope bug. The lines that Claudio pointed out a few years ago are gone. I have a couple of questions about the bandscope as it is behaving differently now. Maybe there is some new configuration that I am missing?

** At LNA of +19dB, the noise floor for most of the spectrum is below the default -120dB. I can't figure out how to extend the display below -120dB. The slider controls on the bottom right which usually do this for other displays are not having any effect.

** If I increase the LNA gain, I can eventually see the floor for the whole spectrum. But at the point when the ADC starts to clip continuously, the ADC N% in the signal DB widget is still only at 1 or 2%. Is this correct? How should I interpret the ADC N% value?

73,

Steve

kf7o

[James Ahlstrom]

Hello Steve,

On Saturday, September 21, 2019 at 12:10:41 PM UTC-4, Steve Haynal wrote:

** At LNA of +19dB, the noise floor for most of the spectrum is below the default -120dB. I can't figure out how to extend the display below -120dB. The slider controls on the bottom right which usually do this for other displays are not having any effect.

On the Graph screen,  the Yz control will change the baseline. On the Waterfall, the Yz controls the waterfall colors. Hold down Shift and use Yz to control the baseline on the graph above the waterfall.

** If I increase the LNA gain, I can eventually see the floor for the whole spectrum. But at the point when the ADC starts to clip continuously, the ADC N% in the signal DB widget is still only at 1 or 2%. Is this correct? How should I interpret the ADC N% value?

I see the same thing. The N% is the maximum value received from the ADC. But only a fraction of the ADC samples are returned, and there must have been a 100% value that was not returned. Even a single clip will turn on the clip indicator for a couple seconds, so that a clip is not missed. It is on my list to improve the clip indicator so it is more intuitive, but I didn't get to it yet.

Jim

N2ADR

[Nigel Head]

I've now tested my HL2b8 with latest firmware and Jim's updated Quisk and can now connect over a VPN (Win10 and both Microsoft's built in VPN or Drayteks' software client).

The connection needs to be pretty decent (wifi strength or hardwired) or the received audio is choppy.

I will make more substantial tests over the coming week, as time permits...

My thanks to all involved in getting us this far.

Nigel - G4ZAL

[Steve Haynal]

Hi Nigel,

Glad to hear that it is at least connecting. Are you running at 48kHz bandwidth? That may help. It is always going to be a challenge to transport the raw udp to/from the HL2 over any high latency and/or low bandwidth network. I think the right solution would be to have a server/client model with 2.7 kHz audio transported using some compressed format like codec2. There was the ghpsdr3 SDR project working in that direction, but it has been inactive for many years now and may be hard to build. For what I do, FT8,FT4 with no local audio, VNC is fine.

73,

Steve

kf7o

[James Ahlstrom]

Hello Group,

I did a little work on the clip indicator to see if I could improve it, but is is more complicated than I thought. I programmed "overrange2", an average of the fraction of packets returned from the hardware that had the overrange bit set, 0.0 to 1.0.

I connected a 7.2 MHz sin wave to HL2, and varied the level. As expected, overrange2 started at zero, and at the clip level rapidly moved to 1.0. So every packet had a clip. But there was no distortion visible on the graph until I increased the level further. The ADC level reported by the bandscope was 2% for OK, 4% for OK, and 6% for barely clipping. The ADC level stayed at 6% for the higher levels that distorted. It looks like the ADC is clipping only on the positive and negative peaks, and all the intermediate levels are OK. So a bit of clipping would just flatten the sin tops, and not do much damage. For serious clipping, the lower parts of the sin wave would still be OK. But it is suspicious that the bandscope ADC level maxes out at 6%.

I then connected an antenna and tuned to 7.2 MHz. I varied the preamp gain and was able to see clipping at maximum gain. The overrange2 changed rapidly from 0.0 to 0.8 as clipping was reached. I thought maybe I could display intermediate values, but they seem irrelevant. At max preamp gain, overrange2 was 0.9, so most packets had clipping, but no distortion was visible in the graph, and the audio was unaffected. It looks like when the HL2 barely clips, the tops of the waveform are just flattened a bit. It is only when the waveform is badly chopped off that communication is affected.

So firmware changes are need to improve the clip indication; namely, a fraction of samples that are clipped, and a measurement of how much they are clipped (not much, chopped off by half, etc.). The former is inconvenient, and the second is impossible, as it requires measuring the ADC level above its maximum level. So I don't think I can improve the clip indication. It is not very useful, as a large amount of clipping can be tolerated before communication suffers.

Next I will work on the bandscope ADC level. It should act more like the overrange2 I programmed.

Jim

N2ADR

[James Ahlstrom]

Hello Steve,

The maximum level of my bandscope samples is 0.0625 at strong clip. Inverted, that is 16. I presumed that the 12-bit ADC samples are left shifted four bits to make a 16-bit bandscope sample. But maybe the ADC samples are sign extended to fit in 16 bits. Can you verify that the maximum bandscope sample is 2**11 - 1?

Jim

N2ADR

[Steve Haynal]

Hi Jim,

Currently the bandscope data is sign extended 12-bit ADC samples. It would be more compatible with openhpsdr radios to shift these and make them 16-bit data. I can do that in the next gateware release.

The clip condition was reworked a while ago as it was too eager. Currently clip will be set in a HL2-to-PC packet if the absolute max value (12'b100000000000  or 12'b011111111111) is seen at least 3 times since the last HL2-to-PC packet was sent. If you think this is still too eager, I can make it less likely in the next gateware release. I think that Alan likes to look at the bandscope data directly and make a more sophisticated decision on clipping.

73,

Steve

kf7o

[Alan Hopper]

Hi Steve and Jim,

I have always wanted a bit more clip information from the radio, maybe the number of clipped samples since the last packet and the peak adc reached, useful for rf agc and peak display.  I did play with statistics on the histogram of adc samples to predict how close or how far into clipping we are but having real data would be better.  

73 Alan M0NNB

[James Ahlstrom]

Hello Steve and Group,

On Monday, September 23, 2019 at 11:31:52 PM UTC-4, Steve Haynal wrote:

Currently the bandscope data is sign extended 12-bit ADC samples. It would be more compatible with openhpsdr radios to shift these and make them 16-bit data. I can do that in the next gateware release.

It would be best to change the gateware to return 16-bit integers; that is, left shift the 12-bit data by four bits. If all Hermes software does this, the maximum ADC sample is independent of the original number of bits, which could be 12, 14, 16. So please change it if you don't mind.

 

The clip condition was reworked a while ago as it was too eager. Currently clip will be set in a HL2-to-PC packet if the absolute max value (12'b100000000000  or 12'b011111111111) is seen at least 3 times since the last HL2-to-PC packet was sent. If you think this is still too eager, I can make it less likely in the next gateware release. I think that Alan likes to look at the bandscope data directly and make a more sophisticated decision on clipping.

In my testing with a sin wave, it seemed that the clip indicator turned on at 75% of the maximum ADC value in the bandscope samples. Do you see any reason this might be happening, or am I looking at noise?

Anyway, I don't think any other gateway changes are useful, since the issue is not whether the ADC clips, but whether it clips enough for communication to be affected. Only measuring the signal amplitude above the max can answer this. I ran an amusing experiment yesterday. I tuned in to a strong 40 meter broadcast signal, and advanced the preamp gain into strong clipping. At moderate clipping, the audio was unaffected. But at max clipping, I could hear another broadcast signal mixed in with the first. But even then, I could still hear the first signal clearly. When testing with sin waves, the clips and distortion happen at a regular rate, and the spectrum degrades rapidly above clip. But with real radio signals, clipping is less damaging.

Jim

N2ADR

[Nigel Head]

Steve, Jim,

I made a bit more of a constructive test this evening and it didn't go so well.

Previously I tested on a neighbours weak wifi and only on receive.

This evening I was hard wired at our club hilltop site and receive was good at 48k and slight audio choppiness at 96k.

I was constantly pinging my HL2 over the VPN and response times were around 25ms.

However, the moment I tried to transmit, Quisk would 'freeze' and not recover (ping response was still fine).  It was the same every time I tried to Tx.

Any idea why Quisk should 'freeze/hang' on Tx? (I was on Windows10 and it was the same for the Microsoft VPN and the Draytek software client VPN).

I will try from a relatives home in the next couple of days to see if I can replicate it...

Nigel - G4ZAL

[Alan Hopper]

Hi List,

I've just released a version of SparkSDR that should work over vpn if the latest firmware is used http://www.ihopper.org/radio/previews.htm . There is a section in the help on setting it up, you can select a range of ip addresses that it will try so the radio can still use dhcp.

73 Alan M0NNB

[Steve Haynal]

Hi Jim,

I will release updated testing gateware with the 16-bit integers this weekend.

Are you looking at the clip indicator sent to the PC via UDP, or the LEDs? There is one LED that lights when the ADC sees data at 75%. This is the good ADC level and indicates that you are making use of a reasonable number of ADC codes. There is a second LED that lights when the ADC sees 3 absolute maximum or minimum values. This second LED is the same clip indicator that is sent back to the the PC. Quisk should not see the 75% indicator, and it is a bug if it does, but the RTL appears to be sending only the clip indicator to software. Any gateware from about the last year should behave this way. See the LED wiki page for more details.

Your experiments are interesting.

73,

Steve

kf7o

[Steve Haynal]

Hi Nigel,

Are you able to transmit with Quisk with no freeze when not going over VPN, with the HL2 local? There is the watchdog timer that can kick in after 2.7 seconds. Do you still see wireshark traffic when the freeze occurs? The VPN response times may be varying. You may see packets out of order. There is a status screen on Quisk which reports latency and errors. What does this say before and after the freeze?

Which VPN are you using? You must have a router/gateway or PC at the remote end to handle the VPN. Is it possible to just open an ssh tunnel?

73,

Steve

kf7o

[Steve Haynal]

Hi Alan,

Sounds great! Thanks.

73,

Steve

kf7o

[Alan Hopper]

Hi Steve, Jim,

this change will break the rf agc and adc level features of Spark, easy enough to fix but I would like to add the bandscope bit depth and number of contiguous packets to the discovery packet, this will allow software to cope with future changes without modification.  This is a much more flexible approach than relying on the version number and allows experimentation with firmware without having to modify software (just like the number of receivers).

I still believe an overflow count/percentage and max adc value sent back will give better results for both agc and level meter. The bandscope is such a small sample of the real signal that it generally under reads the real situation ( as seen in spark by the red overflow flashing when the meter is still low). 

73 Alan M0NNB

[Christopher KB3CS]

am i right that an alternative way to view the proposed change (signed 16 bit bandscope values returned instead of 12 bit sign-extended values) will result in HL2 and Spark both getting 'fixed' (in the sense of now conforming to the Hermes design)?

  - 2b (base 31) -

On Wednesday, September 25, 2019 at 2:46:04 AM UTC-4, Alan Hopper wrote:

Hi Steve, Jim,

this change will break the rf agc and adc level features of Spark, easy enough to fix

[...]

[Alan Hopper]

Hi Christopher,

'fixed' is not really the case, HL2 is not the same as Hermes and we have tweaked the protocol a number of times to make the most of HL2's features . I have no problem with switching to 16 bit values for bandscope but lets do it in a forward thinking manner, after this change  HL2 will not magically be the same as Hermes as the number of contiguous bandscope packages will not be the same so current specific Hermes software is unlikely to correctly interpret it.  The original Hermes Protocol 1 is fairly dead in the water as the HPSDR group's direction is Protocol 2 which has never really made sense for HL2, so we can play fairly fast and loose with it as long as the basic features work in old software.  HPSDR was brilliantly forward thinking and HL2 made it affordable, it would be a shame to loose the forward thinking part.

73 Alan M0NNB

[Nigel Head]

Hi Steve,

I can (and did) Tx (and Rx) just fine with Quisk when back on my own LAN.  My HL2 was still working fine when I returned home from our club site. 

I don't have wireshark on my laptop so couldn't check.
I didn't take a note of latency or errors, but on Rx it seemed fine at 48k.

On Tx, it appeared to freeze Quisk almost instantly (didn't seem like 2.7 secs for the watchdog).

What alerted me, was a 'snick' in my headset earpiece just a fraction after starting to Tx (same every time I tried) both on VOX or clicking PTT on screen.

I'll try to be more observant on future tests.

Nigel

[James Ahlstrom]

Hello Christopher,

The Hermes protocol states:

HPSDR sends to EP4 a block of 4096 x 16 bit raw ADC samples.

So it is silent about whether 12-bit samples are left shifted or sign extended. So someone with the proper HPSDR hardware will have to determine this experimentally. Alas, this is often necessary for protocols.

Jim

N2ADR

[Christopher KB3CS]

yes, thank you! i did locate the file "Metis- How it works_V1.33.pdf" on August 16 and read it through a couple times.

i would interpret "16 bit raw" to mean 16 bit unsigned samples. would that make the controlling software's job simpler or more complicated?

i do know someone with a complete HPSDR and at least one ANAN. would it be useful to have him run some experiments and collect some data?

considering "forward thinking", FWIW, my thought is HL2 should emit 16 bit samples even if the hardware produces fewer bits. 10 bits yesterday sent as 16 bits. 12 bits today also sent as 16. 14 or 16 bits tomorrow ... 

is it time to take the "How it works" document and add some more details relating to HL2? then, of course, the edited file's name would be "HL2 - How it works", yes?

  - 12 (base 71) -

[Alan Hopper]

Hi, Jim, Christopher,

I have an Orion board as well as an HL2 and Spark works with both (although it is a while since I tried P1 on the Orion). The spec is not clear about the format of these samples. I interpret them as signed little endian int16 and this seems to work, the endianness is opposite to the iq data which is a bit odd but actually saves a bit of cpu.  There is description of the HL mods to the protocol in the wiki.  As I said before I'm happy to change to 16 bit but this will break any software that is designed for hl2 and won't make hermes software magically work due to there being fewer contiguous packets so the immediate effect is purely negative.  To correctly interpret  the bandscope data you need to know the number of contiguous packets, this is lower on the HL2 and may well change in future releases, eg. if you are not interested in lots of receivers it might be worth trading a few for a higher res bandscope. Adding the number of contiguous packets to the discovery packet allows software to cope with this without needing updating for each firmware version.

Another option is to accept that software already needs to treat hl2 specially and make the most of the 12 bits and reduce network traffic by packing the data as 12 bit.

73 Alan M0NNB

[Christopher KB3CS]

iirc, the fellow i mentioned with a HPSDR has a Hermes board.

it's good there is at least some writing about the modifications to the Hermes1 protocol. i'm saying it would soon be time to make a more complete document out of the existing "How it works" document and then add in the modifications and then call the new document by a name suitable for a protocol document for Hermes Lite 2, and then i added the query to the suggesting in case others might agree.

locating important information in one location is similar to locating your raw materials and tools in a single location, yes?

additionally, the existing frames sent by HL2 are themselves 1075 bytes long. 16 bits * 4096 - 12 bits * 4096 = 4 bits * 4096 = 2048 more octets. hmm.

we live in a jumbo frame world where 8192 byte frames are acceptable. using 3123 bytes leaves plenty of room before reaching 8192.

thoughts?

  - 1e (base 59) -

On Sunday, September 29, 2019 at 2:05:49 AM UTC-4, Alan Hopper wrote:

Hi, Jim, Christopher,

I have an Orion board as well as an HL2 and Spark works with both (although it is a while since I tried P1 on the Orion). The spec is not clear about the format of these samples. I interpret them as signed little endian int16 and this seems to work, the endianness is opposite to the iq data which is a bit odd but actually saves a bit of cpu.  There is description of the HL mods to the protocol in the wiki.  As I said before I'm happy to change to 16 bit but this will break any software that is designed for hl2 and won't make hermes software magically work due to there being fewer contiguous packets so the immediate effect is purely negative.  To correctly interpret  the bandscope data you need to know the number of contiguous packets, this is lower on the HL2 and may well change in future releases, eg. if you are not interested in lots of receivers it might be worth trading a few for a higher res bandscope. Adding the number of contiguous packets to the discovery packet allows software to cope with this without needing updating for each firmware version.

Another option is to accept that software already needs to treat hl2 specially and make the most of the 12 bits and reduce network traffic by packing the data as 12 bit.

73 Alan M0NNB

On Saturday, September 28, 2019 at 10:28:35 PM UTC+1, Christopher KB3CS wrote: 

yes, thank you! i did locate the file "Metis- How it works_V1.33.pdf" on August 16 and read it through a couple times.

i would interpret "16 bit raw" to mean 16 bit unsigned samples. would that make the controlling software's job simpler or more complicated?

i do know someone with a complete HPSDR and at least one ANAN. would it be useful to have him run some experiments and collect some data?

considering "forward thinking", FWIW, my thought is HL2 should emit 16 bit samples even if the hardware produces fewer bits. 10 bits yesterday sent as 16 bits. 12 bits today also sent as 16. 14 or 16 bits tomorrow ... 

is it time to take the "How it works" document and add some more details relating to HL2? then, of course, the edited file's name would be "HL2 - How it works", yes?

  - 12 (base 71) -

[...]

[Alan Hopper]

Hi Christopher,

there are certainly gains to be made through using jumbo packets or even the maximum non jumbo size, In profiling my app a surprising amount of time is spent in the basics of receiving and sending udp packets (using RIO on win64 and boost asio elsewhere) and I believe bigger packets would help. With low receiver numbers and low sample rates you have to be careful not to increase latency so big is not always better. We have talked about a whole new protocol in the past but so far the gains have not justified the effort and resulting incompatibility, an option with smoothly zoomable display data would tempt me.  The current way the bandscope data works on hermes and HL2 is that a number of contiguous samples are grabbed and stored in a buffer in the fpga, the buffer is then sent out over a number of udp packets and the sequence number is used to detect which packets belong to the same set. It is the buffer size that restricts the resolution of the bandscope on the HL2 as it uses up fpga reources.

73 Alan M0NNB

[Steve Haynal]

Hi Alan and Jim,

I did not find the time this weekend to try this change. Alan has a good point that other accommodations for the HL2 are necessary (length of the bandscope data) so there may be little to gain in terms of compatibility by going to 16-bit integers. I want to try this change with PowerSDR, linhpsdr and pihpsdr so see if there is any advantage. I'll only release the change if there are advantages. 

This is not a critical issue as the Quisk bandscope works well enough as is. Only some amplitudes and ADC percent usage is off a bit.

73,

Steve

kf7o

[James Ahlstrom]

Hello Steve,

With the latest gateware the bandscope samples are still 12 bits, not 16 bits. Are we not going to change to 16 bits? If not, I have to modify Quisk by adding the number of bits that equals full scale which will be different for different hardware.

Jim

N2ADR

[Steve Haynal]

Hi Jim,

Sorry, I have procrastinated about this. Alan brought up the point that even with 16 bits not all openhpsdr software will work as the sample set lengths are different. I wanted to check other software. Linhpsdr, Quisk and PowerSDR assume 16 bit. PowerSDR wide bandscope may not work properly due to the difference in sample set lengths. Any reports? At one point with the Hermes-Lite 1, John Melton said the length of the sample set didn't matter for him. Maybe he and also PowerSDR use several short FFTs. SparkSDR assumes 12-bit. I'm not sure if Simon has add support for this in SDR Console.

Maybe you, Alan and Simon could decide what to do here? I will do whatever you decide. My vote would be slightly in favor of switching to 16 bits. I can make any change in the next gateware release.

73,

Steve

kf7o

[James Ahlstrom]

Hello Steve,

The issues of bandscope block size versus the format of the integer data are different. If 12-bit integers are sign extended into 16-bit integers, the range is +/- 2**11. If 12-bit integers are left shifted into 16-bit integers the range is +/- 2**15. I need to know the range of the data. I need to know the block size too, but that is a separate issue.

I would prefer bandscope data to be left shifted into whatever size integers are used. I think that makes the most sense. Remember that I have to support multiple hardware, not just HL2. But I can live with the HL2 returning 12-bit data because I can look for the board ID of 6.

Jim

N2ADR

[James Ahlstrom]

Hello Steve,

I still think left shifting makes the most sense. But I can use board id 6 and the code version to change the bandscope scale, so I can easily accommodate whatever you want to do.

Jim

N2ADR

[Steve Haynal]

Hi Jim,

Unless Alan or someone else comes up with some really convincing arguments, I will convert to 16-bits this weekend and release a testing gateware.

73,

Steve

kf7o

[Alan Hopper]

Hi Jim and Steve,

I'm not going to argue against this, it is slightly annoying that it will break my software but I can live with that if it is for the better good. I see it as more reason to describe features and capabilities in the discovery packet rather than relying on a single version number, had this been done already the change would not break Spark. 

73 Alan M0NNB

[Steve Haynal]

Hi Alan,

Maybe we can start adding information to the discovery packet now. I don't really like the string of HERMESLITE that is sent and it is not needed.

73,

Steve

kf7o

[Alan Hopper]

Hi Steve,

I suspect we can remove that string, the only place I can find it used is here https://github.com/k3it/HermesIntf/blob/8335e03fee9a56f768e2d5ce96acd04b17c313b8/HermesIntf/Hermes.cpp and it is only used if id 6 is not used. I think I'm right that we no longer have an option to pretend to be a Hermes.

73 Alan M0NNB