HL2 Wifi Buffer

[James Ahlstrom]

This project is software to add buffering to the Tx path of the Hermes-Lite2. This should enable the HL2 to be used on a WiFi network, or remotely on a jittery Internet connection. The software is being rushed out because I am leaving on a ski trip. Hopefully you will find lots of bugs for me to work on when I get back.

The github page for this project is https://github.com/jimahlstrom/HL2WifiBuffer.

Jim

N2ADR

[G4ZAL]

Hi Jim,

I had a little play with your software and early results are encouraging.

I made a little write up here...

https://g4zal.blogspot.com/2024/02/n2adr-wifi-buffer.html

I do think the quality (throughput) of the R-Pi wifi adapter will be the deciding factor.

My 2 wifidongles were not very good, but a USB cabled ethernet adapter seems fine (only 10/100).

Rx is OK, but I wouldn't trust Tx in Thetis as yet.

I'll have a play with some of the buffer settings.

Nigel, G4ZAL

[ron.ni...@gmail.com]

My experience is that WiFi adapters (and whatever is running the SDR app) that support 802.11ac or ax work a lot better for streaming HL2 data than do WiFi radios that only support 802.11a/b/g/n. 73, Ron, n6ywu

[Will VA3JWO]

Hi guys, 

I also tried Jim's Wifi buffer on a debian 12 VM instance in proxmox, but with my Wireguard VPN. For sure, it works great in passing the data properly for me with decent results on Thetis and SparkSDR and removed the relay chattering from the remote TX. Instead of selecting wlan interface in the txt file, I used wg0 for Wireguard tunnel. I also modified the code to use the broadcast address in my local subnet the HL2 is on (172.16.10.255) instead of 169.254.255.255 to find locally the HL2 with success before making the software with immediate good results.

Unfortunately I am still battling the badly distorted TX audio from remote software to HL2 with a well working connection with other platforms.

[Ron Lewkowicz]

My WIFI is b/g/n only. The router is at the opposite end of the house.  I run with PiHPSDR on a laptop over WIFI and notice only very rare Seq Err.  Biggest issue was people complained my laptop mic sounded like I was talking from across the room so I got a USB headset to plug into the laptop.

[G4ZAL]

Hi Will,

As an aside, have you tried Quisk 'remote' ?
I made a short how-to here

https://g4zal.blogspot.com/2023/01/quisk-remote-operation.html

I've just had a couple short SSB QRP QSO's and no reported issues (I deliberately didn't ask either!) on Tx in Quisk and Thetis.

Didn't want to run the amp in case there really were any problems.

Oddly, the Tx errors in Thetis aren't happening right now...?!

Also, my logging program works OK with the virtual COM ports for CAT control over Jims buffer s/w link.

More as it happens...

Nigel, G4ZAL

[G4ZAL]

Had a few more QSO's with linear on and no audio issues reported.

One 'niggle' I seem to have is with VOX - when activated and I have finished an over, the VOX de-activates and then there is a 'pop' of received audio which trips the VOX again, which then de-activates and 'pop' and so on.... until I disable VOX.

I've tried tweaking VOX levels In Quisk (limited) and in Thetis (lots of options), but it still continues to trip.

Doesn't happen when not using the wifi buffer link (no 'pop' of rx audio on VOX de-activating).

Nigel, G4ZAL

[Steve Haynal]

Hi Jim,

Thanks for doing this! Does this software try to "fix" reordered UDP packets? I guess it should be possible. No way to fix dropped packets, but should also be possible to report any dropped or reordered UDP packets which could be helpful in debugging some setups.

73,

Steve

kf7o

[James Ahlstrom]

Hi Steve,

No, there is no code to re-order packets. This is a Tx buffer only. And yes, I could add code to report packets that are out of order or dropped. But I am leaving tomorrow on a ski trip so it will have to wait until I get back.

Let me add a little more detail. Mostly the software is a FIFO Tx buffer, but there are complications. It must support the broadcast UDP discover packet, so there are some packet rewrites. And it must work on port 1024 and 1025 simultaneously. If the RQST bit is set, the request is time critical, and the packet is sent immediately. This requires rewriting all the sequence numbers in the buffer. The software runs in four threads with blocking reads because I think this will result in the best speed on a slow processor. So it is nice to have four cores, but probably not required. Even the humble Raspberry Pi 3B+ has four cores.

It is fun to run the software with a delay of zero to copy only with no buffering. The web page displays the jitter. I find my WiFi runs nicely for a minute or two, and then there are a number of drops all at once. I am not sure what is happening. A good plan is to have all local WiFi traffic on the 2.4 GHz band and use the HL2 with adapter on the 5 GHz band. But that requires a strong 5 GHz signal and 5 GHz doesn't like walls.

Jim

N2ADR

[James Ahlstrom]

Hello Nigel,

>>One 'niggle' I seem to have is with VOX - when activated and I have finished an over, the VOX de-activates and then there is a 'pop' of received audio

I see the problem. The transmit audio is delayed but the received audio is not delayed. So you are receiving the ending portion of your transmission. If you run Quisk with full duplex (FDX) you will hear the delayed audio of your transmission. 

I will add an option in the next release of Quisk to suppress the Rx audio for an adjustable time when switching from Tx to Rx.

Jim

N2ADR

[James Ahlstrom]

Hello Group,

I updated my WiFi buffer to version 1.2. I think that this version should be ready for prime time. Please test.

Jim

N2ADR

[G4ZAL]

Hi Jim,

I have previously tested v1.0 and v1.1 successfully.

Tried with v1.2 on same R-Pi setup and no connection to any software (Quisk/SparkSDR/Thetis)?

Only difference is that as I have two HL2's and have set individual MAC addresses and individual fixed IP on both (my original HL2 is still on default MAC address and is what I am using for the tests).

It seems that with fixed IP address the HL2 is not dropping back to APIPA ??

Using v1.1 the output is as below (as v1.2 gives no output) - my PC is on ...100

pi@r-pi-1:~/HL2WifiBuffer-v1-1 $ ./hl2_wifi_buffer
delay 300 TX_BUF_COUNT 8192 txbuf_used 114
WiFi interface eth1 address 192.168.1.228
HL2 interface eth0 address 169.254.1.1
WiFi1024 got   63 from 192.168.1.100 port 57616:  EF FE  2  0  0  0  0  0  0  0
WiFi1024 got   63 from 192.168.1.100 port 57616:  EF FE  2  0  0  0  0  0  0  0
WiFi1024 got   63 from 192.168.1.100 port 57616:  EF FE  2  0  0  0  0  0  0  0
WiFi1024 got   63 from 192.168.1.100 port 57616:  EF FE  2  0  0  0  0  0  0  0
WiFi1024 got   63 from 192.168.1.100 port 57616:  EF FE  2  0  0  0  0  0  0  0
WiFi1024 got   63 from 192.168.1.100 port 57616:  EF FE  2  0  0  0  0  0  0  0
WiFi1024 got   63 from 192.168.1.100 port 57616:  EF FE  2  0  0  0  0  0  0  0
WiFi1024 got   63 from 192.168.1.100 port 57616:  EF FE  2  0  0  0  0  0  0  0
WiFi1024 got   63 from 192.168.1.100 port 57616:  EF FE  2  0  0  0  0  0  0  0
WiFi1024 got   63 from 192.168.1.100 port 57616:  EF FE  2  0  0  0  0  0  0  0

Any further testing I can do, just let me know.

Nigel, G4ZAL

[G4ZAL]

Hi Jim,

Small update...

Removed the fixed IP from my original HL2 and tried the wifi buffer v1.2 again and it will now connect.

Also tried to swap in my 2nd HL2 and wifi buffer v1.2 appears to 'see' it on 169.254.84.212 but does not connect (with stop/restart of wifi buffer)

Attached image is HL2#1 connected and with HL2#2 it 'sees' it in Hermes Lite section, but no data in WiFi section regarding up/down rate.

It definitely 'sees' the HL2#2 because if I don't power it, the Hermes Lite section is blank regarding data.

Nigel

G4ZAL

[James Ahlstrom]

These are basic networking problems. The HL2 RPi interface eth0 should have an address of 169.254.1.1 so the network is APIPA. Therefore it can not connect to a fixed IP of the HL2 unless that is APIPA too. If you have a fixed address on the HL2, use the same network on eth0.

In general, note that you must have two valid networks, one for eth0 and one for WiFi. The network addresses must be different, and all device must belong to the correct network.

Jim

N2ADR

[G4ZAL]

Agreed Jim, I was curious to see if the HL2 would also 'fall back' to APIPA even though it had a fixed IP address because your buffer software does 'see' it as APIPA on eth0.

I guess the HL2 does not respond as it assumes to be addressesed over the fixed address.

I have now reset both HL2's to DHCP and set individual reservations for both in my router as a way of having 'fixed IP'.

They both have individual MAC addresses set.

Nigel, G4ZAL

[James Ahlstrom]

Hello Nigel,

I think there is still some confusion here. The IP addresses on the Wifi buffer status screen are the addresses of the interfaces on the single board computer (SBC), not the address of the HL2. The WiFi address of the SBC is commonly 192.168.1.x and is set by DHCP. It connects the SBC to your PC by WiFi. The Ethernet port on the SBC is commonly eth0 and it has whatever address you set. I set eth0 to 169.254.1.1 so it can talk to an HL2 using the APIPA address of the HL2.

You can set a fixed IP on your HL2, but then you must set eth0 to a different address on the same network, and the network must not be the same as the WiFi interface.

Jim

N2ADR

[ron.ni...@gmail.com]

The way I set up my buffering relay is by using 2 different port numbers.  I use the default UDP port 1024 to talk between the Pi and the HL2.  Then I configure a different port number on the buffering relay and in my SDR app to use to talk between the SDR software and the Pi.  That way I can wire both the Pi and the HL2 to the same router or WiFi access point, and only use WiFI with the PC (iPad or iPhone) running the SDR app.  The Pi then needs only one IP address.  Does any other SDR software allow configuring the (potentially virtual) HL2's UDP port number, instead of just using the hardwired default of port 1024?  73, Ron, n6ywu

[John Williams]

Thetis appears to allow multiple ports.

---

From: herme...@googlegroups.com <herme...@googlegroups.com> on behalf of ron.ni...@gmail.com <ron.ni...@gmail.com>
Sent: Saturday, April 6, 2024 12:17 PM
To: Hermes-Lite <herme...@googlegroups.com>
Subject: Re: HL2 Wifi Buffer

 

--
You received this message because you are subscribed to the Google Groups "Hermes-Lite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hermes-lite...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hermes-lite/ebab69e0-629f-46e9-b4c2-72300dcacd20n%40googlegroups.com.

[James Ahlstrom]

Quisk allows setting a port other than 1024.

Jim

N2ADR

[Ron Lewkowicz]

If it's possible to use a different port to run this buffer software on a single network adapter would it also be possible to use a virtual interface?  If the HL2 could access the network on eth0 as normal and the SDR software access via the buffer on the same network adapter using the IP of eth0:0 for example. All could be on the same LAN netmask an no APIPA address involved. 

[James Ahlstrom]

I don't understand how it is possible to use two port numbers instead of two networks. The RaspPi has two network interfaces and each requires an IP address. If the RaspPi has only one IP address even though it has two interfaces, how can it perform routing?

I also don't understand the motivation. If the HL2 and RaspPi are both connected to the same Ethernet, the RaspPi must contend with all the other traffic on the network, not just traffic from the HL2. And what is the problem with APIPA? The HL2 defaults to APIPA if it can't find a DHCP server.

Jim

N2ADR

[ron.ni...@gmail.com]

I don't run WIFi on my Pi's, as the Pi's WiFi network interface is slower and more jittery-er than my bigger router/access points.  Instead, I run my hl2_tcp UDP to TCP relay/transcoder, which buffers, recodes, and reroutes from TCP port 1234 rtl_tcp protocol to UDP port 1024 Metis protocol.  Then I can run my SDR app using the RTL-SDR rtl_tcp protocol to access my HL2 through my Pi via port 1234, but with hl2_tcp buffering that removes network latency hiccups, while running the HL2 on the Pi's port 1024.  I am working on replacing the rtl_tcp TCP port code on the Pi with Metis UDP port code visible to my SDR app, but this will still require 2 port number pairs, both UDP, instead of one TCP and one UDP.

I don't connect the Pi directly to the HL2 using the PI's ethernet port and the self-assigned IP, because then the HL2 won't be visible over UDP port 1024 to other various other computers and servers on my wired network.  And also because I have more than one HL2 on my network, and want to be able to have the relay Pi access any or all of them simulaneously.

73, Ron, n6ywu

[James Ahlstrom]

On Sunday, February 25, 2024 at 3:46:37 PM UTC-5 Steve wrote:

Hi Jim,

Thanks for doing this! Does this software try to "fix" reordered UDP packets? 

This is an update. Version 1.2 of my WiFi buffer will now re-order UDP packets based on their sequence number. And it counts and displays out-of-order packets. A packet is out of order if the sequence number goes backwards. If there is a skip in sequence (100, 102, 103, 105, 106,   ...) this is not reported when it occurs. It may happen that packet 104 will be received later. If the software is sending the buffer to the HL2, and packet 104 is then found to be missing, a missing packet is reported and a packet with Tx samples of zero is sent.

In operation, the newest Tx samples are written to the top of the buffer with the possibility that some are missing,  and that out-of-order packets may be inserted lower than the top. Meanwhile packets are sent to the HL2 from the bottom of the buffer. Eventually, missing packets will be discovered and replaced with silence.

My experience with my WiFi in my loft is that there are no out-of-order packets. But there are occasional missing packets, and occasional "storms" of many missing packets which are associated with longer WiFi delays. A typical maximum delay over five seconds is 25 msec, but I have seem delays of 400 msec, well in excess of my 300 msec buffer. The software tries to recover from buffer underflows and overflows as best it can. But other WiFi networks can be different. And using the software over the Internet without WiFi can also be different.

Jim

N2ADR

[James Ahlstrom]

Hello Ron,

I think I understand your network now. I always envisioned the buffer as a "client Wifi adapter" to connect a remote HL2 to WiFi. 

On Tuesday, April 9, 2024 at 11:46:48 AM UTC-4 Ron wrote:

 I am working on replacing the rtl_tcp TCP port code on the Pi with Metis UDP port code visible to my SDR app, but this will still require 2 port number pairs, both UDP, instead of one TCP and one UDP.

What is the reason to replace the rtl_tcp code? It may be easy to re-write my buffer code to do what you want.

Jim

N2ADR

[Clifford Heath]

If you count the missing packets and subtract the number of out of order packets you can get the dropped packet count. In case that's useful to someone.

Clifford Heath 

--

You received this message because you are subscribed to the Google Groups "Hermes-Lite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to hermes-lite...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/hermes-lite/9f5ecea5-29af-4f07-8f3d-ce59496e24fan%40googlegroups.com.

[Scott Strehle]

I started playing around with the wifi buffer yesterday and have ran into a few problems.  I am running a raspberry pi 3 b+.  I put a clean install of the latest raspbarian 64 bit os on the pi.  It seems very slow at exicuting  things.  not sure if there is a better option.  I got the make file to compile the software and I have been able to run it and connect to the hl2 via eth0 and thetis vial onboard wifi.  It seems to work well with tune and two tone signals for a minute or two and then it crashes and looses network connection.  Latency is low on this setup and jitter was reading 15 to 60 ms .  After a crash it is very difficult to get things to re connect. 

 

So I then tried to connect  the pi to eth1 with a usb ethernet dongle configured the same as the wifi setup and I cannot get things to connect to the buffer from the pc with Thetis on it.   I tried changing the text file by removing the wlan0 and inserting eth1 in its place and no change I then tried just removing wlan0 and letting the software populate the connection itself like the text file says and no change either.  I can ping the pi from the pc after each change and that appears to be fine but it will not connect to the buffer via the 192.168.88.208:8080 web browser link.  after a crash on the wifi link I have the same problem and cannot reconnect.

Also I have a question about running the buffer on the pi.  I initially run it from the cmd line with the command ./hl2_wifi_buffer.  my question is how do I stop the buffer I have used a ctrl c command and assume it stops the buffer with that.  Is that correct?  I then try and make changes to the text file and then execute the  ./hl2_wifi_buffer command does that then start the buffer again?  Am I missing something here?  do i need to stop the buffer when making a change or does the buffer constantly check the text file for info and there is no need to stop and restart things?  

Ultimately the system will run remotely hooked up to ethernet  with the buffer there to remove the jitter from the ISP wireless network between my home location and a remote location. so I need to configure it with eth0 and eth1.

Thanks

Scott

KJ7RHP

[James Ahlstrom]

When you start the buffer with ./hl2_wifi_buffer, it reads the configuration from hl2_wifi_buffer.txt. When you enter control-C, the buffer stops. You need to stop and restart the buffer to cause it to read the configuration.

You need to figure out how to get the web page from port 8080. It sounds like you have a network configuration problem, and information to solve it is on the web page.

Jim

N2ADR