Fwd: HL2 RELAY CLICKING DURING TX

[Prabir Debnath]

---------- Forwarded message ---------
From: Steve Haynal <herme...@gmail.com>
Date: Mon, 13 Jul, 2020, 6:56 AM
Subject: Re: HL2 RELAY CLICKING DURING TX
To: Prabir Debnath <probir....@gmail.com>

Hi Probir,

I don't think you need to shield the relays. Relay clicking is very rare. Most people, including myself, never see this during TX.

Is your power supply above 11V during TX?

Have you confirmed that your network is not dropping TX packets so that the radio is leaving TX? (100 or 1000Mbs wire network, no wifi).

Can we move this discussion to the Google Groups so that others may benefit?

73,

Steve

kf7o

On Sun, Jul 12, 2020 at 6:16 PM Prabir Debnath <probir....@gmail.com> wrote:

Hi Steve,

R117 is Installed. Now if we shield the Relay do I need to shield the TxRx relay on Main board or need to shield all the Relays. My power supply is having less than 25mv of ripple which is quite below the standard I believe. 

73 

Probir VU2BQF 

On Fri, 26 Jun, 2020, 10:55 AM Steve Haynal, <herme...@gmail.com> wrote:

Hi Probir,

Relay clicks can occur if you have a power network connection (the radio thinks you have stopped transmitting and goes back to receive) and/or some of your buffer latencies and hang times are not set optimally. See for "latency" and "hang" on the protocol page:

https://github.com/softerhardware/Hermes-Lite2/wiki/Protocol

Also, there were problems with relay clicks when the TX inhibit input picked up strong RF from a PA. This was solved with R117 which is included in recent builds. You can check if R117 is installed, or provide better shielding if you are using a higher power PA.

73,

Steve

kf7o

On Thu, Jun 25, 2020 at 10:03 PM Probir <probir....@gmail.com> wrote:

Hi Steve,

 

Couple of times I have noticed during Transmit – IN THE MID WAY – I can hear a RELAY is being switched ON – the sound is clear – it’s a relay operation.

 

I was just wandering what it could be. 

 

73

PROBIR VU2BQF

 

[Prabir Debnath]

[Prabir Debnath]

Hi Steve.,

Could you please throw some light whether the network packet is dropping during Tx. I have a direct connection with my ethernet port of my PC with no WiFi. 

73

Probir VU2BQF 

On Mon, 13 Jul, 2020, 6:56 AM Steve Haynal, <herme...@gmail.com> wrote:

Hi Probir,

I don't think you need to shield the relays. Relay clicking is very rare. Most people, including myself, never see this during TX.

Is your power supply above 11V during TX?

Have you confirmed that your network is not dropping TX packets so that the radio is leaving TX? (100 or 1000Mbs wire network, no wifi).

Can we move this discussion to the Google Groups so that others may benefit?

73,

Steve

kf7o

On Sun, Jul 12, 2020 at 6:16 PM Prabir Debnath <probir....@gmail.com> wrote:

[Ronald Nicholson]

Do you have an ethernet cable connected directly from your PC to the HL2?  

Or from your PC to a router to which your HL2 is connected?

I've seen some really slow routers drop UDP packets (or whatever else is causing UDP sequence number errors on Rx, and continuous rapid relay clicking on Tx).  Had to switch them out for a newer faster router.

You can run a cable directly from a PC's (or Mac, Raspberry Pi) ethernet port (or using a dongle with aniPad, or iPhone).  Most modern ethernet ports do not require a cross-over ethernet cable, just a regular jumper.  If you do this, the HL2 will not get a DHCP address.  But wait about 30 seconds, and you can connect to it using the HL2's self-assigned IP address (usually something in the form 169.254.x.x).

73, Ron,  n6ywu

------

[Jaroslav Škarvada]

Hi,

I am observing the same - during TX sometimes the relays clicks.
Currently, I wasn't successful to find out which relay(s) it is. I have
direct wired connection between the HL2 and PC, no router is between.
For me it mostly happens when I TX FT8 and simultaneously do something
on the PC - e.g. surf web. I am using Linux Fedora 31 and Quisk. I
reported this problem to Quisk upstream.

I guess it's due to the latency of my kernel scheduler and that Quisk is
not fast enough in sending the packets to the HL2 in such case. The
machine is loaded to cca. 1.5 (it has two physical cores with
hyperthreading, i.e. 4 logical cores in total, so there should be still
some performance left). Regarding the top output it shows:
quisk 70 %
pulse 13 %
xorg 17 %
jt9 30 % (just for the decode phase)

I am going to try re-nice/chrt Quisk for lower latency

73! Jaroslav, OK2JRQ

Prabir Debnath napsal(a):

> <mailto:herme...@gmail.com>> wrote:
>
> Hi Probir,
>
> Relay clicks can occur if you have a power network connection
> (the radio thinks you have stopped transmitting and goes back to
> receive) and/or some of your buffer latencies and hang times are
> not set optimally. See for "latency" and "hang" on the protocol
> page:
> https://github.com/softerhardware/Hermes-Lite2/wiki/Protocol
>
> Also, there were problems with relay clicks when the TX inhibit
> input picked up strong RF from a PA. This was solved with R117
> which is included in recent builds. You can check if R117 is
> installed, or provide better shielding if you are using a higher
> power PA.
>
> 73,
>
> Steve
> kf7o
>
>
> On Thu, Jun 25, 2020 at 10:03 PM Probir
> <probir....@gmail.com <mailto:probir....@gmail.com>> wrote:
>
> Hi Steve,

> Couple of times I have noticed during Transmit – *IN THE MID
> WAY* – I can hear a RELAY is being switched ON – the sound

> is clear – it’s a relay operation.
> I was just wandering what it could be.
> 73
> PROBIR VU2BQF
>

> --
> You received this message because you are subscribed to the Google
> Groups "Hermes-Lite" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to hermes-lite...@googlegroups.com
> <mailto:hermes-lite...@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/hermes-lite/CAMh-vQ9pE9BBnN_siW7GUQrD_UMNzs8yP09fMxeLL7Fb3z1f6Q%40mail.gmail.com
> <https://groups.google.com/d/msgid/hermes-lite/CAMh-vQ9pE9BBnN_siW7GUQrD_UMNzs8yP09fMxeLL7Fb3z1f6Q%40mail.gmail.com?utm_medium=email&utm_source=footer>.

[Evan Hand]

Could it be a slow wifi connection, or maybe a nic load?

[Ronald Nicholson]

Yes. 

I can create HL2 relay chatter by (1) walking a distance away from a working WiFi access point connection, (2) trying to Tx using a old slow WiFi access point, (3) saturating a wired routers bandwidth with other traffic when using 100% wired connections, or (4) running the UDP code on too low a priority thread on a busy computer even when using a single ethernet cable directly between the computer and the HL2 (no router).

That says that some tool or utility might be helpful to test a UDP or ethernet connection to the HL2 before trying to use it for Tx.  Maybe unplug the HL2 and put a Raspberry Pi on its ethernet cable running a Metis emulator with full UDP packet logging (inter-packet timing variations and loss).

73,

Ron

n6ywu

------

[Matthew]

Ron,

How many receivers was this with and what sample rate? Are you sending packets to the HL2 synchronised to the rx packets? Or dumping a number of TX packets to the HL2 at once when software buffers get full (PowerSDR does this)?

To anyone having issues, to troubleshoot I suggest using SparkSDR, Quisk or linHPSDR. these all schedule TX packets send to the HL2 (feel free to comment if other software also does this). Next, keep the number of receivers low, keep the sample rate at 48000. Then increase either/both sample rate/receivers, this will give some useful clues.

73 Matthew M5EVT.

[Ronald Nicholson]

Hi Matthew ,

My SDR code is currently only using a single HL2 receiver, but at configurable sample rates from 48k to 384k (and at much higher sample rates with other SDRs, such as the LimeSDR Mini).  The HL2 Tx issues I see do not seem to vary with Rx sample rate. But higher Rx rates do have more issues under poor WiFi connection conditions.

I've tried both (1) syncing Tx packets to Rx packets (1:1 at 48k, 1:8 at 384k), and (2) scheduling them at 1 per 2.625 milliseconds using the OS mach kernel timers. The latter (2) seems to work slightly better under degrading network conditions (but then won't 48k sample rate sync between computer time and HL2 clocks eventually become an issue?)  My code is not doing any kind of batching or dumping of multiple UDP packet.  Each UDP sendto is done separately.  Would batching help?  Or doing an initial batch dump to pre-fill some fifo then scheduling packets individually?

My current suspicion is that the UDP latency jitter issues have more to do with buffering and batching behavior in routers, the WiFi radio stacks (computer and router), and/or somewhere within the OS scheduler and network stack below the socket API.

73,

Ron

n6ywu

-------

[Steve Haynal]

Hi All,

Just a few notes on relay chatter as several people have mentioned it. I've seen it occur in 3 situations:

** Defective N2ADR board or HL2 TR. This is very fast chatter or buzzing. Requires replacing the MCP23008 or a relay. Very rare. I've seen it once or twice.

** RF pickup on the TX inhibit line if there is no R117 installed. Again, a very fast chatter or buzzing. Very rare. I've heard of it once or twice, and never after R117 was installed.
** TX buffer in FPGA empties and enough time passes so that HL2 leaves transmit. Here the chatter can range from just one or two a minute to several a second. This is common.

For the last reason, TX buffer empties, I think we need to establish a minimum QOS that the network must provide for the HL2 to operate well. In most typical home networks, this level of QOS is met. In some networks where wifi is in use, or direct connection between a PC and H2 is made, this level of QOS may be harder to meet.

All TX data sent to the HL2 is always at a rate of 48kHz. This means one IQ sample is required by the HL2 (or consumed by the HL2) every 20.8us. 63*2=126 IQ samples can be fit into every UDP packet sent to the HL2 given the openhpsdr protocol 1. This means that every UDP packet represents 126*20.8us = 2.62ms. In the stock gateware, the TX buffer is 2048 entries deep, or 2048*20.8us = 42.6ms deep.  Two problems can occur with the buffer: overflow and underflow.

Overflow occurs if the data sent to the HL2 exceeds 42.6 ms (surplus of 42.6ms) at any time. We have seen this in the past when some software or network devices group multiple UDP packets into a burst. For example, a burst of 16 UDP packets is just under 42ms and will probably overflow the buffer since usually there is something in the buffer. During overflow, the extra data is just thrown away and never inserted into the buffer. This results in some clicks in your TX audio, but no relay chattering. Note that if overflow occurs, software assumes the HL2 received all the data, but not all the data was inserted into the buffer. This means that the HL2 is likely to run out of data and enter underflow after a large overflow event. 

Underflow occurs when the HL2 exhausts all the data in the TX buffer. This is expected at the end of transmit but may occur in the middle of transmit if the network QOS is not acceptable. To mitigate this problem, there are two variables which software can set in the HL2: TX buffer latency (0-31ms) and PTT hang (0-31ms). When data arrives in the TX buffer and it is no longer empty, the FPGA waits for TX buffer latency before beginning to withdraw samples and send them to the DSP and DAC. This servers two purposes, to provide some buffering for jitter in UDP packet reception times and to allow the relays to engage. The relays engage immediately when the TX buffer is no longer empty. By default, the TX buffer latency is set to 10ms, It can be increased to provide more buffering for UDP packet jitter, but at the expense of total latency. If the TX buffer empties, the HL2 does not immediately leave transmit. Instead, it waits for the PTT hang time in hope that more data will arrive. If more data does not arrive within the PTT hang time, the relays are disengaged and the whole TX sequence must start again. So if the TX buffer empties but more data comes before the PTT hang time expires, you will hear clicks in your TX signal but no relay chatter. You will only hear relay chatter if the TX buffer empties and the PTT hang expires, but then more TX data arrives very soon. Each event like this will engage/disengage the relay and cause a click. The frequencies of these events determines if you hear just a few clicks or a buzz.

For people who would like help with relay chatter issues, I'd like to request more information. Please provide answers to the following questions:

** What software are you using?

** How much compute power does your host have: low, medium, high?

** Describe your network HL2 to PC connection. Is it 100Mbs or 1000Mbs? Are there router/switch hops between your PC and the HL2? If so, how many? Are there any wifi hops?

** What is the idle status of the 4 LEDs on the front of the HL2?

** Is your relay chatter a consistent buzz, or a low rate of clicks? Does it vary?

** What is your power supply voltage during TX?

** Do you hear the chatter with the N2ADR board removed?

** Does using the new testing gateware (72p1) have any effect? Does adjusting TX buffer latency and PTT latency have any effect.

There may be more questions to add, but at least this is a start.

73,

Steve

kf7o

[Jaroslav Škarvada]

Hi,

I am replying to some questions (I will reply more once I will know the
answers - it may take some time, I am now out of shack :).

** What software are you using?

Linux, Fedora 31, 64 bit, quisk-4.1.65 and wsjtx-2.2.2. I think so far
it happened to me only when I was operating FT-8. Mostly it happened
when I was simultaneously touching the machine during the TX (e.g.
replying IRC messages, e-mails, surfing web, ..., but nothing CPU
intensive).

** How much compute power does your host have: low, medium, high?

I think low to medium - the machine is Lenovo ThinkPad x240 with 2
physical cores (i7-4600U CPU @ 2.10GHz) with hyperthreading (i.e. 4
logical cores in total) and 8 GB RAM, it's loaded to cca. 1.5 during the
FT-8 TX.

** Describe your network HL2 to PC connection. Is it 100Mbs or 1000Mbs?
Are there router/switch hops between your PC and the HL2? If so, how
many? Are there any wifi hops?

It's direct cable connection (no cross cable) between the laptop and the
HL2. The laptop is running DHCP server and is also simultaneously
connected to the internet through slow WiFi (802.11/g). The cable
connection should be 1 GBit but I will re-check and do some throughput
tests between two laptops.

** What is the idle status of the 4 LEDs on the front of the HL2?

IIRC the first two LEDs are on, the second two are off, but I will recheck.

** Is your relay chatter a consistent buzz, or a low rate of clicks?

It low rate clicks. IIRC usually it seems like one time/occasional click
cca. in 5 - 10 seconds interval after the PTT changes to on, sometimes
it's occasional click randomly during the transmission. It happens
mostly when I start interacting with the machine through the keyboard or
touchpad (or at least it seems so).

Does it vary?
It vary only in the way that mostly it clicks in 5 - 10 seconds interval
after the PTT changes to on (it looks like big relay delay in the path)
and sometimes (less often) it clicks randomly during the transmission.
It happens mostly when interacting with the machine, but very rarely it
also happened when not touching the machine.

** What is your power supply voltage during TX?

Mostly 13.8 V. I tried 12 V / 3 A wall adapter and also 13.8 V 30 A and
40 A ham shack power supplies.

** Do you hear the chatter with the N2ADR board removed?

TBD, I haven't tried yet.

** Does using the new testing gateware (72p1) have any effect? Does
adjusting TX buffer latency and PTT latency have any effect.

TBD, I haven't tried yet.


I was able to reproduce this issue on two different places with three
different power supplies and with three different antennas and coaxial
cables.

I am going to try:
- re-nicing/chrt the quisk process
- temporally changing to different front-end SW
- changing the buffer setting
- changing the HL2 gateware to the recommended testing one
- checking the R117
- removing the N2ADR board
- temporally swapping the computer for more powerful model

thanks & regards

Jaroslav, OK2JRQ

Steve Haynal napsal(a):

> --
> You received this message because you are subscribed to the Google
> Groups "Hermes-Lite" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to hermes-lite...@googlegroups.com
> <mailto:hermes-lite...@googlegroups.com>.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/hermes-lite/31fc2955-909d-4990-b7c9-bb9189346812o%40googlegroups.com
> <https://groups.google.com/d/msgid/hermes-lite/31fc2955-909d-4990-b7c9-bb9189346812o%40googlegroups.com?utm_medium=email&utm_source=footer>.

[Chris Gerber]

Hi All

I am mostly a Digimode user of Olivia or RTTY, FT8 etc. I also have relay clicking during TX, with power out of only about 2 W.

But that happens only when using Powersdr mRX PS (newest version) for my Hermes Lite 2. When using Simons Console program

which I also use on a Aanan 10, no probles at all. Also found that Output Power adjustment with Powersdr program is rather strange,

cant reduce Tune power to zero. Thats never occuring withe Simons Console program.

To mention that I have reset the Powersdr Data set fully to new.

I am rather new with the  HL2 from Makerfabs, but longtime user of Anan 10 with all programms available.

73 Chris HB9BDM

--
You received this message because you are subscribed to the Google Groups "Hermes-Lite" group.

To unsubscribe from this group and stop receiving emails from it, send an email to hermes-lite...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/hermes-lite/31fc2955-909d-4990-b7c9-bb9189346812o%40googlegroups.com.

[Steve Haynal]

Hi Chris and Jaroslav,

Thanks for the feedback. To me it sounds like a network QOS or software issue in both cases. PowerSDR is known to send batches of UDPs. You might try Reid's fork as I think he made some changes there. Simon put in effort to make SDR console work well with the HL2. I have no issues with Quisk and WSJT-X running on Linux mint on a medium class machine. I do have a gigabit switch between my PC and HL2. I wonder if direct connect between PC and HL2 causes the network to behave differently, such as batching UDPs or having lower default QOS. It would be interesting to hear if the experimental 72p1 gateware with double the size of TX buffer has any effect for you.

73,

Steve

kf7o

To unsubscribe from this group and stop receiving emails from it, send an email to hermes-lite+unsubscribe@googlegroups.com.

[Jaroslav Škarvada]

I have now trouble to reliable reproduce the problem (with firmware
v70). I have new Linux kernel and new quisk (4.1.66) and some more
Fedora distro updates. Maybe it's just coincidence and I was just lucky
not to hit it again, maybe something improved in the SW, I don't know. I
will definitely try to isolate the problem, but in the current state
it's not blocker for me

73! Jaroslav, OK2JRQ

Jaroslav Škarvada napsal(a):

[Anonimous Barby]

Steve,

I gave an update in another thread also, my symptoms seem to went away with a direct Ethernet wired connection.
To answer your questions:

Software: Quisk 4.1.64
HL2 gateware : 70
Host: MSI GL62M intel i5 gen 7 4-core 2.5 GHz 128Gbytes SSD, Debian 10 Linux so medium?
Network: single wifi router 1000Mbs, was clicking through wifi, but not when directly wired.
Led status: idle Run on, others off
Symptom: occasional clicking, but sometimes many in a row, but no buzz
Supply V: 12 V now use E3631A supply.
N2ADR removed: no click, but also only run 5 sec at a time.
Gateware: 72p1 not tried; not sure yet where in Quisk to adjust TX and PPT latency.

Hope this helps, KK6BC.

[Steve Haynal]

Hi KK6BC and Jaroslav,

Thanks for the updates. Wifi has always been a challenge to keep the TX buffer at good levels. I hope the Python utility I am working on will make it east to adjust the TX and PTT latency, and then we can learn some more. Everyone's setup is a bit different!

73,

Steve

kf7o

[Ronald Nicholson]

Has anyone successfully transmitted from an  HL2 using a WiFi connection?

If so, what model WiFi access point?

So far, I've only found one example of one out-of-production router that works for HL2 Tx.  And only at very short range.

If this is a common issue, probably a warning should be added to the HL2 setup notes (saying to only use a wired ethernet connection until one can test any WiFi connection for ep6 errors, Tx buffer underflow, and relay clicking issues.)

73,

Ron

n6ywu

------

On Monday, July 20, 2020 at 7:11:38 PM UTC-7, Anonimous Barby wrote:

[Matthew]

See comments here regarding wifi.

73 Matthew M5EVT.

[Steve, G4GXL]

<<Has anyone successfully transmitted from an  HL2 using a WiFi connection?

If so, what model WiFi access point?>>

I have only had my HL2 for 2 weeks but have used a WiFi connection from the start.

I did have a few problems using 2.4GHz but everything is working perfectly on the 5 GHz network.
My router was provided by my broadband supplier (Vodafone) - I don't know what actual model is in the rebranded Vodafone casing.

I have had around 50 SSB contacts and a few more than that on FT8 - all using SDR Console so far.
For most I was about 5m away from the router with two walls between.

Could someone else with problems see if they get resolved by switching to 5GHz ?

73
Steve, G4GXL

[Ronald Nicholson]

Yes. After a bit more experimentation, I found that on a 802.11ac router that allowed separate SSIDs for 5G and 2.4G, if I connected to only the 5G network, and dedicated the 5G network for just HL2 traffic, that WiFi connection seemed to allow reliable HL2 Tx from across the room, maybe a pinch farther.

73,

Ron

n6ywu

------

[Alan Hopper]

Hi Ron,

my GL-AR150 pocket router works with no problem, maybe because I only use it in a very simple mode with just a single pc connected to it and the hl2 wired into it when away from home. My old BT home hub5 home router also works fine the very large majority of the time although it does make an audible noise when you stress it with lots of receivers.

73 Alan M0NNB

[Steve Haynal]

Hi All,

It is interesting that 5GHz is better for HL2 use. This agrees with this paper on measuring wifi UDP performance. The gateware with larger TX buffer should help, provided the TX buffer latency is changed too. We can also increase the buffer by reducing the TX sampling rate. Also, Matthew noted that gear locking the TX UDP to the RX UDPs can be detrimental if there is high RX UDP jitter. Perhaps software can have a timeout to send a TX UDP so as not to be delayed by a late RX UDP.

73,

Steve

kf7o

[Ronald Nicholson]

Hi Alan,

Are there many other WiFi routers in your building or nearby neighborhood?

One problem I may be having in my dense urban/surburban neighborhoods is 2.4 GHz band pollution.  My laptop reports from between one to around three dozen different visible WiFi SSIDs, mostly on the 2.4 GHz band.  It might be that 5 GHz WiFI band is less saturated around my desk, resulting in less busy channel backoff, packet aggregation, and etc. for me than when using 2.4 GHz WiFi.

Or perhaps your router's firmware is less aggressive about trying to optimize throughput by packet aggregation than the routers I've tried.

73,

Ron

n6ywu

On Friday, July 24, 2020 at 12:30:03 PM UTC-7, Alan Hopper wrote:

[Alan Hopper]

Hi Ron,

that makes sense, I only have a handfull of neighbours and thinking about where I have used the pocket router they are even quieter locations.  I'm looking at sending tx packets on rx timeouts, my original code did this and for reasons I have forgotten it did not get into the xplat version.

73 Alan M0NNB

[Ronald Nicholson]

My software is scheduling outgoing Tx UDP packets on an OS timer, but then checking the long term (per minute) average Rx and Tx packet rates to potentially tweak the timer, and prevent long term drift from the two rates being in an integer gear ratio relationship.  (I really should do a some sort of closed loop servo analysis for stability.)

73,

Ron

n6ywu

------

[Ronald Nicholson]

For break-in CW keying, do most operators just let the relays (Tx and N2ADR filters) click twice for every dot or dash?

If so, whet's the life expectancy of the relays under frequent duty cycling?

Has anyone tried putting putty or other dampening material on or around the relays to reduce the clicking sound level?

Thanks and 73,

Ron

n6ywu

------

[Matthew]

Ron,

Perhaps I am not a "proper" cw operator, but I usually operate at around 100-150 ms break in (and somewhere between 25-30 wpm). At 30 wpm, a dot will be 40 ms long. At these sort of timings, I think signal path latency will come into play and hinders true QSK benefits. I guess it depends what someone wants to use full break in for, if they want to hear their QSO partner with a "BK", I think this is challenging with SDR. For general contesting and DXing I find 100 ms to be sufficient. Lets not start a debate about true QSK for contesting/DXing.

For someone wanting full break in they probably want to design a different N2ADR board with PIN diodes for fast QSK, the HL2 TX/RX relay can be left in TX and rx can be via RF3. I'm not sure we have encountered anyone wanting > 30 wpm and fast/true QSK with the HL2 yet.

You could look up the MTBF for the relay, but I suspect this probably isn't representative of it being used for a QRP relay switch. The better way is to get one to fail, then get 4 more to fail and see if your variance is sufficiently low...

73 Matthew M5EVT.