Missed OSC triggers

[Brendan Aanes]

Hi all,

I have a show running in which all Qlab cues are triggered over OSC
from ETC Eos software running on a windows laptop. Since we began
performances a few days ago, 1-3 sound cues in each performance have
failed to trigger (out of around 70 cues). It's usually been different
cues each time, but sometimes the same cue will fail in two
performances but then work fine in the following. It doesn't seem
related to time between cues, as it's not usually the first cue in a
sequence that fails.

Qlab system is a 2020 Macbook Pro (intel) running Qlab 4.7 in MacOS Ventura.

The network is very simple - it's now just the two computers connected
directly without any switch in between, set to static IPs in the same
subnet. The qlab computer has no other network adapters attached and
Wifi is off. The static IPs are in a different subnet than the IPs
used for the lighting network. The LX team's response when this
started was "oh OSC just sometimes fails" which I don't think is a
satisfactory solution.

Any ideas what to look for to sort this out? It's been ages since I
did a system where an ETC console triggered Qlab over anything but
MSC, but I have done a number of shows with Qlab triggering other Qlab
systems or ETC hardware via OSC and those have all been 100% reliable
or very close.

[Richard Williamson]

I’ve been through the same issues - it seems to be an issue with the amount of superfluous osc messages eos sends out by default, and the fact that UDP messages are not guaranteed to be delivered.

I fixed it by using oscRouter from ETC to create a tcp connection between the mac and the console, and then forward the messages to qlab. There is also a possibility that using eos’s osc filter methods may help but I haven’t tried this yet

Richard

Sent from my iPhone

> On 27 Feb 2023, at 08:08, Brendan Aanes <baa...@gmail.com> wrote:
>
> Hi all,

> --
> Contact support anytime: sup...@figure53.com
> Follow QLab on Twitter: https://twitter.com/QLabApp
> User Group Code of Conduct: https://qlab.app/code-of-conduct/
> ---
> You received this message because you are subscribed to the Google Groups "QLab" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to qlab+uns...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/msgid/qlab/CAE3XBDT-%3D%2BkCVXviGuaFaHU3b9WwZXyUwMxZwtmi7xEn2c5Urg%40mail.gmail.com.

[Kyle Jensen]

Are you running TCP or UDP cues to QLab? If possible, using TCP has solved this issue in the past for me.

Kyle Jensen (He/Him/His)

Sound Designer | Audio Engineer

Tel: +1 (585) 738-0508

E-Mail: ky...@kylejensenmedia.com

Web: kylejensenmedia.com

This e-mail, and any attachments contained herein, are intended only for the person or entity to which it is addressed and may  contain confidential and/or privileged material. Any review, re-transmission, copying, dissemination or other use of this information by persons or entities other than the intended recipient is prohibited. If you received this in error, please contact the sender and delete the material from any computer.

[Brendan Aanes]

It's over UDP. The EOS software does not have an option to transmit OSC via TCP, though from Richard's response above it sounds like involving the OSCrouter will allow that. In the current EOS software implementation there is now an option to send custom UDP strings, which allows it to send a syntax Qlab understands without using the OSC router.

To view this discussion on the web visit https://groups.google.com/d/msgid/qlab/CAD1j_adSP_hTpfxG5WYDG_RNP37a6-QczA8XXi1nR8L8qM85%2Bw%40mail.gmail.com.

[Sam Kusnetz]

I’ve found that cutting down on the lighting console’s chattiness can help.

This is achieved via Eos’ “filter” OSC command, which needs to be sent by QLab once per reboot.

Send this message:

/eos/filter/add=/cue/*

Now the Eos will ONLY send OSC messages to QLab that start with “/cue/“

All other OSC messages generated by Eos, of which there are many, will be hushed.

This won't necessarily solve it, but might! On at least one show, it solved it for me.

Sam

—

Sam Kusnetz (he/him) | Figure 53

s...@figure53.com

To view this discussion on the web visit https://groups.google.com/d/msgid/qlab/CAE3XBDSq_JxmwPtJdhwC__2iZeEnYkiyPstvdd9sF3qrYSEdAQ%40mail.gmail.com.

[Andrew Nelson]

I've been experiencing this same problem, and am eager to try the filter option. Sniffing with Wireshark it appears that when a cue is fired, EOS can send 15-30 OSC messages. The first one is the "fire" message and the rest are "pending" or various status messages. We found that when a cue was missed from EOS, the first 8-10 of these OSC messages were missing, most importantly the "fire" message. 

What I have done that has helped for some reason, it to set up a "heartbeat" on the receiving machine to ping the lighting console every 5 or 10 seconds. While that would seem to add more network chatter, it has so far prevented missed cues. It was a bit a desperate attempt, but not using it results in missed cues on over 60% of shows, and using it has resulted in no missed cues. We are using it to trigger a caption system and occasionally QLab. We have observed the receiving problem from both. The heartbeat helped the Windows recipient, and we'll be testing it for the QLab computer (simultaneously with the Windows machine).

I look forward to seeing if the filter option helps.

Here's the "heartbeat" batch file I use from a Windows machine to the ION:

@ECHO OFF
set IPADDRESS=192.168.100.101
set INTERVAL=10
:PINGINTERVAL
ping %IPADDRESS% -n 1 > nul
timeout %INTERVAL%
GOTO PINGINTERVAL

[sstaub]

The main problem is that EOS and QLab act as a TCP Server. I made a feature request for EOS to implement a TCP Client.

This will also helpful on QLab to implement additional a TCP Client.

Sometimes the Ethernet cables can have defects. Personal, I never had a problem with UDP.

It is meanwhile possible to run the EOS and QLab application on the same laptop using the loopback address 127.0.0.1 

[Chris Ashworth]

Hi Stefan,

QLab does support outgoing TCP 

Of course the receiving server must also accept TCP connections for this to work

-C

[Andy Kanturek]

For our EOS Element2 and QLab, we found that EOS server and QLab server cannot use TCP at the same time.  Instead, we set the EOS as a TCP server port 8000 to send to QLab machine running ETC’s OSCRouter acting as a TCP client listening on port 8000 which filters and transforms “fire" OSC to QLab “start” message.  Then, it sends OSC message to QLab via UDP on the same machine, 127.0.0.1 port 53000.  

It works well.  Hope this helps.  

PS Attached is our configuration:

 

On Jan 22, 2024, at 9:48 AM, Chris Ashworth <ch...@figure53.com> wrote:

Hi Stefan,

QLab does support outgoing TCP 

Of course the receiving server must also accept TCP connections for this to work

-C

<Screen Shot 2024-01-22 at 10.47.19 AM.png>

On January 22, 2024 at 9:56:55 AM, sstaub (stefan...@lichtkunst-kunstlicht.de) wrote:

The main problem is that EOS and QLab act as a TCP Server. I made a feature request for EOS to implement a TCP Client.

This will also helpful on QLab to implement additional a TCP Client.

Sometimes the Ethernet cables can have defects. Personal, I never had a problem with UDP.

It is meanwhile possible to run the EOS and QLab application on the same laptop using the loopback address 127.0.0.1 

-- 
Contact support anytime: sup...@figure53.com

Follow QLab on Threads: https://threads.net/@QLabApp

User Group Code of Conduct: https://qlab.app/code-of-conduct/
--- 
You received this message because you are subscribed to the Google Groups "QLab" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qlab+uns...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/qlab/CADv1a49yQVebxfL0JZA6t%2BoWxRZ9V%3DFcHp5yMAfO4-0n45JHrQ%40mail.gmail.com.

[Andrew Nelson]

Short version: Filters plus custom string OSC messages are great.

Gory details if interested:

I recently had great success with the Eos filters. We send these OSC messages to the LX console:

• Filter only QLab cues:
  • /eos/filter/add /cue/*
• Filter only EOS ‘fire’ cues:
  • /eos/filter/add /eos/out/event/*

They both worked and brought down the number of outbound OSC messages from LX to 2 - the standard Eos message and the custom string (QLab OSC).

However, we observed 100% success with the QLab OSC and only about 75% with the Eos messages. In the past with no custom string and no filter we’ve dropped 1-3 messages in 150. 

Looking at network traffic with only 1 outgoing message and no filter:

Each cue sends around 10 or more OSC messages (fire is first).

On dropped messages, the first 4-6 messages are missing.

Looking at network traffic with 2 outgoing messages and no filter:

Eos cue sends around 10 or more OSC messages (fire is first).

One message is sent next in the custom string format.

Missed 1% of Eos messages, and 1 of 300 custom cues

Looking at network traffic with 2 outgoing messages and filters applied:

Each cue sends 2 OSC messages - 1st Eos standard, 2nd custom string

On dropped messages, the Eos message is missing, but the custom string is seen.

Missed ~75% of Eos messages, 0% of custom string

We had a case recently where we need to trigger QLab and a caption system (UDP only) from LX. Once I changed the caption system to listen for the QLab-style OSC messages, we started receiving 100% of our cues. Makes me think this is why using the custom string for QLab often works great, and if the filter is applied, it works perfectly.

Not knowing for sure what effect reapplying a filter has (2-show days), we start by using a clear filter message followed by the filter messages for every show.

/eos/filter/clear

1 sec delay

/eos/filter/add /cue/*

/eos/filter/add /eos/out/event/*