There are ham operations that a remote operator needs to carry out that do not require real-time speed. Examples are switching antennas, adjusting radio parameters, fiddling with amplifier settings, and others. The one action that is most time-critical is when the operator responds to an on-the-air signal. If that does not happen quickly, time is lost by repeats, others getting to make that QSO first, or the QSO is not happening at all. That is theoretically true for both local and remote operations, but most critical for success while operating contests remotely.
I have participated in contests for a long time. I am just getting started with remote contesting and noticed a subjective increase in doubling. It happens more often that I am trying to respond to a caller and we both transmit at the same time. It seems I waited too long. The reason is obvious: there is some delay in my setup that puts my signal on the air later and the other station has already started transmitting again.
When breaking down this delay further, it seems the 2 most crucial components that affect this overall are:
- The time of the RX audio to reach the control operator’s ears, and
- The time of the control operator’s action (keyboard stroke) to key the transmitter.
I will call the components “RX audio latency” and “keyboard latency”.
My goal is to the measure the delay of each properly and compare alternative solutions.
Receive audio latency test setup
In these tests, I am comparing two receivers. To establish a baseline, I first use 2 local receivers and later I use one local and one remote receiver.
Both receivers are tuned into the same strong station. The ear will already confirm that there is a time difference of the received signals. It is easier to detect with strong stations that transmit at a certain interval. For this purpose, I used WWV.
Figure 1 RX audio latency test setup
Both receivers’ audio (one direct and one coming from the remote) will be fed into its own sound card microphone jack. Both sound cards are the same. As we are only interested in the difference of the latency, using similar sound cards and measuring the delay between both audio channels will be sufficient. The right picture shows how to connect if you have the 2 audio signals coming from local hardware. I used this setup when both receivers are sitting on the same work bench. There are other scenarios to measure, where the audio arrives from the remote location. In that case I used a 3.5mm jumper cable to feed the remote audio into the microphone input (I did not find a better solution. Since the audio in this case comes from the remote, I must convert it from a playback to a recording device)
Figure 2 Sound card connections
It’s easy to record a stereo signal and measure the delay between left and right channels with the open-source tool Audacity. In order to get the signals mixed this way I choose Voicemeeter Banana.
In Audacity, chose the output of the B1 channel of VoiceMeeter as the input, and chose stereo. Then record the input and when you see a good signal that can be measured, stop the recording (1, 2). Use the mouse to drag a selection from the signal’s start in the left channel to the start in the right channel (3). The program then shows the time of the interval, which is the latency we want to measure (4). In the example screen shot it shows 96ms, which is the delay of the audio the Flex 6600M adds with medium-sharp filter settings.
Figure 3 Audacity
RX audio latency measurements
The total latency is made up by the parts illustrated in this next figure.
Figure 4 Latency components
For any remote operation we will have audio processing on either side and network latency to deal with. Network latency can be measured and estimated easily.
Even though my initial measurements included network latency, I have taken it out of the results, as it will be different for everyone and can easily be measured and added.
I have also removed the latency because of SDR filtering. Its easily to measure, without needing a remote setup.
The numbers below include the audio processing on both sides of the link.
|Audio 1 source||Audio 2 source||Audio delay|
|Local RX, speaker output jack||Remote RX, speaker output jack, audio via murmur/mumble (see Appendix)||165ms|
|Same||Remote RX, speaker output jack, audio via RemAud server/client (see Appendix)||98ms|
|Same||Remote RX, speaker output jack, RemoteRig (untested, numbers provided by vendor)||80ms|
For my internet link with 45ms of latency the total latency is between 210 and 125ms for any software or hardware solution. That’s without any SDR filters enabled.
In order to estimate the latency you would experience, add your internet latency and SDR filter latency (if you have any) to the numbers above.
Since I have a Flexradio 6600M, here are a few specific measurements for the Flex using SmartSDR.
|Audio 1 source||Audio 2 source||Audio delay|
|Local RX, speaker output jack||Local Flex 6600M, speaker output jack, lowest filter setting||48ms|
|Same||Local Flex 6600M, speaker output jack, 2nd filter setting||64ms|
|Same||Local Flex 6600M, speaker output jack, 3rd filter setting||96ms|
|Same||Local Flex 6600M, speaker output jack, 4th filter setting||160ms|
|Same||Local Flex 6600M, SmartSDR with PC Audio, 3rd filter setting, LAN (< 1ms latency)||572ms|
SmartSDR with PCAudio seems to add substantial latency into the audio chain. However, it may depend on the use case if it is an issue or not. Personally, I like the simplicity of using SmartSDR/PC Audio for casual operations. I avoid using it when I need fast-paced exchanges, as in contests.
- Any software tools used here add latency on their own. I have not tested the audio latency of VoiceMeeter Banana, the sound cards or Audacity. However, since I route both channels through the same components, the added latency will be in both channel’s latency. Measuring the difference of both channels spectrum should cancel them all out.
- Window’s audio chain is known to have latency. The software solutions tested incur this on both sides.
- RemAud supports stereo, murmur/mumble does not. That makes mumble unusable for remote SO2R operation.
- A RemoteRig is not physically available to test. RemoteRig is advertised to be designed for low latency and may not have the overhead of the Windows operating system. The information gathered from RemoteRig support is this:
If you use the default setting packet size = 20ms and jitterbuffer delay = 3, the latency will be
20ms to sample the first packet
+45ms your internet latency
+3 x 20= 60 ms buffering
The question I tried to answer was: How much is added to the delay because of the operator is not close to the radio and therefore any keyboard action will take some time to reach the radio.
Figure 4 keyboard latency test setup
I put Meinberg NTP time sync software onto both computers. I also used https://time.is/ to verify that the time was close to exact. Then I added a small AutoHotKey script on the users PC that logs the timestamp of when the Enter key is hit.
OutputDebug, key was hit at %A_Min%.%A_Sec%.%A_MSec%
Next, I wrote a small FlexAPI-based Windows Forms app that shows the exact time when the FlexRadio is being put into transmit (Radio.MOX property).
Keyboard latency measurement (Anydesk, remote N1MM+)
When I compared these timings, the difference was around 50 – 60ms. Considering a 45ms network latency (and a 0ms setting for PTT lead time in N1MM+), that is a very good result. It turns out that AnyDesk has implemented this very well.
This result is so good, that I did not measure how long it would take with N1MM+ close to the control operator position, but I assume it would be similar.
VoiceMeter Banana setup
Figure 5 VoiceMeeter Banana
In the screen shot you can see that my inputs 1 and 2 are coming from the 2 sound cards. In order to route the inputs to the proper output chose B1 and B2 for both inputs (1, 2). To get radio 1’s left channel and radio 2’s right channels mixed into the output, I adjusted the balance of the inputs appropriately (3, 4).
Optimal murmur/mumble setup
For setup instructions, follow: https://www.youtube.com/watch?v=DBfyQRvOmag
Written by Bernd, DF3CB, can be found here https://www.df3cb.com/remaud/
My settings during the measurements were 20ms buffer, stereo.
Also, I opted for not using a VPN, but to use port forwarding for the one port that murmur exposes. VPNs will add latency and can also complicate LAN IP ranges.
Special thanks for Michael, VA3MW and Steve, N2IC for valuable input.
4 Replies to “Ham radio remote operations latency measurements”
Very interesting, indeed. What is missing from your research is the impact that latency has on OPERATING. I’ve been operating remotely for more than 6 years, on a large number of stations. In most cases, I do use the Microbit RemoteRig products.
I’m a contester, and operate at high speeds on CW, as will as running on SSB, and in DX pileups. In no case does the latency bother me in any way. My measured IP latency (exclusive of any of the measurements of audio) has varied from 26mS (a station 140km away) , 160ms (a station 4500 km away), 250ms (stations in the 5000-6500km range). It is the number of Internet routers that really plays havoc with Internet latency.
In general, for the remote op with fairly consistent overall latency, the only difference is you are percieving the world delayed by your overall latency. If you are calling in a pileup, your TX delay will be perhaps a fraction of a callsign behind someone else, which I’ve never found to be an issue, ever.
What really plays havoc with remote operation is jitter, the instantaneous change in latency over time. This causes havoc with buffering in CODECs. Usually, problems with excessive jitter are caused by last-mile issues for packet delivery: TDMA-based point-to-point or point-to-Multipoint WISPs; cheap wireless routers at the station location, etc. Wired all the way is the best solution. I have friends with endless remote problems, even with fiber-to-the-home to the operating QTH, yet continue to have trouble where the station is served by a less-than supreme ISP. Of course, often people do not have a choice. However, if you are plagued by drop-outs and “pops”, look at your IP latency, especially in the last-mile connections on each end.
Beware that SO2R or M/S reply time is bigger than high numbers you measured. Main problem are CW CQ maniacs with <300 msec loop repeat time. FT8 tolerates up to 2 secs.
Great and interesting work! It helps to understand why things do waht they do. 73 Kari SM0HRP