Intermittent OSC triggers from ION

[John Gromada]

Hello all - anyone had this problem? Sadly we have our Qlab rig triggered by an ION via OSC. In nearly every performance one of more cues don't get triggered. After the show the cues fire fine- they're programmed properly in the Ion and in Qlab. It's not always, but often the same one or two cues that don't fire. Anyone had this experience? 

John

[Alec Sparks]

Yes! Had this issue on a 2013 Mac Pro (trashcan)

My diagnosis was that the ethernet interface software/driver had an issue where it would occasionally just ignore incoming packets sometimes. It occurred randomly, and for short bursts of time. Something like 10-30 seconds of issue, then it would go back to working for hours.

Running Wireshark, the network troubleshooting tool, in the background on the QLab computer seems to put the port into a different mode that doesn't ignore packets. That fix has worked for me through multiple shows, though it's less than ideal.

[Jason Dino]

What sort of interface are you using? Do you have an audio interface and using midi?

Thanks 

Jason

--
Contact support anytime: sup...@figure53.com
Follow Figure 53 on Twitter: https://twitter.com/Figure53
User Group Code of Conduct: https://figure53.com/help/code-of-conduct/
---
You received this message because you are subscribed to the Google Groups "QLab" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qlab+uns...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/qlab/5931f97d-24df-45f1-ac30-09354f71bdc9%40googlegroups.com.

--

Sent from Gmail Mobile

[sam kusnetz]

I’ve also solved this by configuring the Ion to not send all the OSC it usually does. By default it’s very chatty; turning that off made my problems vanish.

Now, I have to tell you, I definitely cannot remember how I did that.

Best

Sam

--

Sam Kusnetz | Figure 53

http://qlab.tips | http://qlab.tv

(mobile)

To view this discussion on the web visit https://groups.google.com/d/msgid/qlab/CAJ0FTEJU7hx%2BVU_4Lcf8mfMrxUEZ7SQQ55GVFMbuPLnZmxJPzg%40mail.gmail.com.

[John Gromada]

Thanks Sam and all - I'll try to figure out how to modify the OSC strings on the ION with some of my Eos lighting geek friends... We have wireshark running...

john

[Stephen Harrison]

This has been an ongoing problem for lots of people for a long time…

I have had similar problems across many macs and many EOS consoles. As best as I can tell, it appears that the Ion ethernet port periodically drops off the network. I don't know the cause (drivers, maybe), but I have observed 30-40s periods when no OSC commands come out of the desk, during which time the desk doesn't even respond to ping requests.

The 'solution' I have found, which hasn't failed me yet, is to run a terminal session on the QLab computer, that constantly pings the EOS desk. This seems to keep the desk's network port alive. YMMV but I now trust it to run a show, which I never used to.

Best,

Stephen

[Jeromy Hopgood]

John, I'm curious how you got your issue resolved. I have a projection design that is starting to show the same problems. Things ran perfectly during tech, but in previews we randomly started losing some cues. Unfortunately, this created problems where a certain cue doesn't fade out, but we don't notice until later because it is hidden behind another cue. Stephen, has your pinging solution continued to pan out? Thanks all!

[Stephen Harrison]

I’ve just completed a tour where all lighting cues were triggered from QLab via OSC. The only time we had a problem was when we tried connecting the Mac and Ion network ports directly (another known/suspected cause of problems). With a switch in place, and with a terminal running ping in the background, we didn’t have any other issues for the 7 week run.

I run a ping as a standard part of my workflow now and so far so good. Now that could be coincidence, but so far it works for me.

Stephen

--
Contact support anytime: sup...@figure53.com

Follow QLab on Twitter: https://twitter.com/QLabApp
User Group Code of Conduct: https://qlab.app/code-of-conduct/
---
You received this message because you are subscribed to a topic in the Google Groups "QLab" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/qlab/jNqQnOYD3ZY/unsubscribe.
To unsubscribe from this group and all its topics, send an email to qlab+uns...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/qlab/cafa2c3b-db82-49e2-915e-480e432bf839%40googlegroups.com.

[Corbin White]

There are lots of things to consider here. Are you using TCP or UDP? TCP is more reliable in theory because it is connection-oriented, so as long as you're sending a heartbeat the connection should stay alive and no network interfaces should disconnect from the network. If things tend to break down during the show proper and not before, after, or during rehearsal time, I'm wondering what the network is that you're using to send OSC. Is this wired or wireless? What other traffic might cause the network to become congested during show time? If there is congestion, this is another reason to use TCP. Although flow control can sometimes slow things down, UDP has no obligation to retransmit lost or delayed packets due to high throughput; TCP on the other hand will ensure your OSC messages get delivered. 

After all that the EOS NICs are a valid concern. Problems with a direct connection might suggest an issue with Auto MDI-X. But dropping off the network completely sounds like bad drivers, poor engineering, mismanaged DHCP (I hope these are static IPs!), or, easiest thing to fix, a bad cable. 

Hope I'm not preaching to the choir here. OSC is great, but networks have fun ways of being pains in the butt.

Best,

Corbin

You received this message because you are subscribed to the Google Groups "QLab" group.

To unsubscribe from this group and stop receiving emails from it, send an email to qlab+uns...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/qlab/0E227BC0-9D0D-4850-BE7E-8627B4CF35E9%40redleopard.org.

[Cal W]

Hi All,

I am looking at an upcoming project where we plan to trigger QLAB V5 from an ETC EOS via OSC.

I recall previously experiencing a bit of flaky'ness with UDP packets not always making it to the QLAB machine but neither application can initiate a TCP connection per this article interestingly QLAB v5 lists OSC are receivable via TCP but it doesn't seem like I can initiate this communication from the EOS side.

Does anyone have any luck with this.?

Cheers

[Andy Kanturek]

On our rig, EOS sends cue numbers to QLab to trigger video and sound cues over TCP.  In our simple wired network, only 1 switch, UDP packets from EOS are unpredictable for some odd reason. TCP only works reliably.

We use OSCRouter on Mac to overcome EOS OSC server to QLab OSC server issue. EOS acts as TCP server sending OSC to OSCRouter on Mac configured to listen for TCP message.  Then, it routes OSC to local QLab UDP patch. 

It works well with one caveat, EOS must boot up first before launching OSCRouter. Otherwise, OSCRouter configuration must be refreshed with apply button. This re-establishes TCP session between QLab and EOS. 

On Jul 25, 2024, at 9:07 PM, Cal W <cal.w...@gmail.com> wrote:

Hi All,

Follow QLab on Threads: https://threads.net/@QLabApp

User Group Code of Conduct: https://qlab.app/code-of-conduct/
---
You received this message because you are subscribed to the Google Groups "QLab" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qlab+uns...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/qlab/8608fe64-a137-4db9-a193-c6e8bb5d83e5n%40googlegroups.com.

[Paul]

Clearly there is an issue with OSC between EOS and Qlab as lots of people have mention it, although details remain elusive and I have not managed to reproduce it.  It maybe that packets get dropped on EOS network interface, particularly if it is busy sending sACN or ArtNet as well. EOS also sends a LOT of spurious info via OSC without being asked, which may not help! But really need more info about software and OS versions and protocols in use to try to reproduce.

 Just to clarify Qlab and EOS can both LISTEN to OSC on UDP and TCP; EOS can only send over UDP; Qlab can send OSC over UDP or TCP.  (the statement in last post about EOS sending to OSC Router via TCP does not make sense; the "server" is the process which listens to connections).  In general I would suggest sending from Qlab to EOS (using TCP if required) as it's easier to setup. 

Here is how to configure OSC over TCP

[Andy Kanturek]





To clarify,  EOS is acting as the client sending OSC via TCPIP to OSCRouter which is the TCPIP server.  Then, OSCRouter becomes the client routing OSC message to QLab server via local UDP. 

We use this to have EOS drive video projection and audio to Yamaha soundboard.  It works well. 

Again, take heed of the starting order with EOS before OSCRouter.  Otherwise, OSCRouter “apply” button must be pressed to refresh configuration.  

On Jul 27, 2024, at 9:53 AM, Paul <paulthom...@gmail.com> wrote:

Clearly there is an issue with OSC between EOS and Qlab as lots of people have mention it, although details remain elusive and I have not managed to reproduce it.  It maybe that packets get dropped on EOS network interface, particularly if it is busy sending sACN or ArtNet as well. EOS also sends a LOT of spurious info via OSC without being asked, which may not help! But really need more info about software and OS versions and protocols in use to try to reproduce.

 Just to clarify Qlab and EOS can both LISTEN to OSC on UDP and TCP; EOS can only send over UDP; Qlab can send OSC over UDP or TCP.  (the statement in last post about EOS sending to OSC Router via TCP does not make sense; the "server" is the process which listens to connections).  In general I would suggest sending from Qlab to EOS (using TCP if required) as it's easier to setup. 

Here is how to configure OSC over TCP

To view this discussion on the web visit https://groups.google.com/d/msgid/qlab/2eb17f29-24d6-4e62-a94d-0b49ca8e906en%40googlegroups.com.

[Paul]

Sorry if I'm being dumb here but that makes no sense to me!  

EOS doesn't not have the ability to initiate a TCP connection, so can't send OSC over TCP. There is nowhere in EOS that allows you to configure a host to send via TCP (as their is for UDP). Indeed this whole discussion would be moot if EOS could send over TCP as then it send that directly to QLab (which listens on UDP and TCP) and there would be no need for any intermediate software.  And if OSCRouter forwards OSC message via UDP then that would not solve the problem of UDP packets getting lost on the network (if indeed that was the problem).  

[Richard Williamson]

Eos sends all its messages over an active TCP connection whether it is acting as a client or a server. So when an application such as OSC router connects to eos via TCP it will then receive the messages. This is why the solution using OSC router as a bridge between eos and qlab is one of the most reliable (as it allows both applications to communicate via TCP)

Sent from my iPhone

On 30 Jul 2024, at 11:49, Paul <paulthom...@gmail.com> wrote:

Sorry if I'm being dumb here but that makes no sense to me!  

To view this discussion on the web visit https://groups.google.com/d/msgid/qlab/178f34bb-c2b0-4d9a-afe9-ea87c2462a4en%40googlegroups.com.

[blindeye 1995]

EOS by Default only sends UDP Not TCP. Eos Has the abilaty to send over an tcp connection if the Other Side is in OSC TCP Client Mode. so its saying basicly as long as QLAB and EOS only Supporting Server Mode On the OSC side there is now workaround. OSC Router ( if configured Correctly) does both its fowarding TCP and UDP OSC packets between its targets in client mode.

Greetings Max

[Paul]

Ah thanks Richard that makes sense!

Although I see a few problems with that approach: versions of OSC messaging would need to match; a certain amount of latency would be added and there is still the possibility of UDP packet loss from onward transmission). And any TCP packet loss that did occur would, due to the way TCP handles resending packets, would add further latency.  It sounds rather clunky! 

In that case then QLab itself could initiate the TCP connection  and receive the whole OSC stream via TCP from EOS. (Qlab would silently discard messages which it doesn't understand)

Yes I too general I would trigger EOS from Qlab via OSC as it's easier to setup, adjust timings and run.

Hopefully ETC will add TCP initialising sending and configuration options in future. 

[Richard Williamson]

It’s been the most reliable solution that I’ve seen - however you do OSC the versions need to match and since this is all TCP (no UDP is involved) it is solid. 

--

Richard Williamson
www.richard-williamson.com
ric...@richard-williamson.com
07957 457 362

To view this discussion on the web visit https://groups.google.com/d/msgid/qlab/7b0d736b-ec4a-4715-99bc-925f8b8f7604n%40googlegroups.com.

[Andy Kanturek]

When I get back from vaca, I send our config in EOS, OSCRouter and QLab.  Sounds difficult but it is easy once you know the trick.  Plus, there is minimal delay. 

On Jul 30, 2024, at 9:29 AM, Richard Williamson <ric...@theatre.support> wrote:



To view this discussion on the web visit https://groups.google.com/d/msgid/qlab/b91fbb8b-c7e2-4ba1-acf8-d82a8b5244c6%40Spark.

[Andy Kanturek]

For our EOS Element2 and QLab, we found that EOS server and QLab server cannot use TCP at the same time.  Instead, we set the EOS as a TCP client port 8000 to send to QLab machine running ETC’s OSCRouter acting as a TCP server listening on port 8000 which filters and transforms “fire" OSC to QLab “start” message.  Then, it sends OSC message to QLab via UDP on the same machine, 127.0.0.1 port 53000.  

It works well.  Hope this helps.  

PS Attached is our configuration:

On Jul 30, 2024, at 11:58 AM, Andy Kanturek <barisa...@gmail.com> wrote:



[Cal W]

Yes I ended up contacting both ETC and Figure 53 about this a few days ago just asking if either party has chatted to try and improve this situation. They just kind of both pointed at each other and said 'talk to them' I would say QLAB and ETC are both 2 of the most prevalent systems in theatre from small community projects up to major multi-national commercial theatre shows. So it'd be great if they did some sort of integration properly between them. But this is a business issue and not a technical issue.

As some have recommended I have done some testing and OSC router with TCP does seem to be the way to go. I will use that for this project and keep fingers crossed for some progress int the future.

Cheers

[Sam Kusnetz]

On Jul 31, 2024 at 9:46 PM -0400, Cal W <cal.w...@gmail.com>, wrote:

Yes I ended up contacting both ETC and Figure 53 about this a few days ago just asking if either party has chatted to try and improve this situation. They just kind of both pointed at each other and said 'talk to them' I would say QLAB and ETC are both 2 of the most prevalent systems in theatre from small community projects up to major multi-national commercial theatre shows. So it'd be great if they did some sort of integration properly between them. But this is a business issue and not a technical issue.

That’s not the whole story, actually.

Folks from ETC and from Figure 53 have spoken at length about this and other similar issues, and we’re working on them. The thing is that ETC and Figure 53 each work in our own ways, and our development cycles were never expected to have to line up and work together. So it’s actually harder than it might seem to make something like this work out.

On top of that, there actually are technical issues which need to be resolved. Nothing dramatic, but if we’re going to do something like this we should do it properly. So it takes some time and care.

Yours

Sam

––
Sam Kusnetz [he/him/his] (what is this?)
Figure 53
https://qlab.app | https://figure53.com

[Chris Ashworth]

I’m a big big fan of ETC, but I think they’re a bit behind on their details on this one — their documentation about QLab not supporting TCP is no longer correct, and I philosophically believe that establishing a TCP connection to a device they want to send messages to is a feature that should be implemented on their end, without needing any assistance from us on this one.

But also, as Sam said, we are in contact with them periodically, and very much enjoy working with each other, even if our development schedules often have different things pulling us in different directions.

-C

[Richard Williamson]

Hello

Just to add my final thoughts on this - my only disagreement with Chris’s point "I philosophically believe that establishing a TCP connection to a device they want to send messages” Is that TCP is a bi-directional protocol and I would prefer to only need to make one connection between Eos and QLab rather than the current two - I often have cues firing in both directions and it would cleaner to just have the one connection

Richard

--

Richard Williamson
www.richard-williamson.com
ric...@richard-williamson.com
07957 457 362

--
Contact support anytime: sup...@figure53.com
Follow QLab on Threads: https://threads.net/@QLabApp
User Group Code of Conduct: https://qlab.app/code-of-conduct/
---
You received this message because you are subscribed to the Google Groups "QLab" group.
To unsubscribe from this group and stop receiving emails from it, send an email to qlab+uns...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/qlab/CADv1a4-RbpenDuHct8YPUV6dFajAcC%2BAtDg%3DgXq4Hp7fn8j7Vw%40mail.gmail.com.

[Chris Ashworth]

Thanks Richard — I’ve added that point to our issue tracker on the topic.

Best,

C

[sstaub]

I have made many features request and bug reports to ETC because OSC is meanwhile a big black hole on EOS Family.

The main problem is on EOS side, they should introduce a TCP client mode for sending data, like QLab does meanwhile.

The R&D for the EOS Family have many, many problems with their new consoles so development on features stopped ages ago.

For UDP I see many Layer 8 problems, defect cables and ugly switches cause many problems.

Some people use a $ 20.- Switch for their 100k consoles and wondering what happens - or not.

In small system I use Cisco CB350 or Aruba 1930 switches without a problem. But used also UDP on larger german state theaters without a problem.

[Chet Miller]

So, wrt UDP, I think the danger in this day and age is a touch exaggerated. A well designed system should be rock solid. 

For this case in particular I definitely blame ETC. Broadway video circa 2020 a bunch of us tried to transition away from MSC control to UDP OSC control from Eos and on the simple show I programmed, I would drop a cue a show. There is maybe 50 cues in the show total? And that was d3, so qlab was uninvolved.

For comparison, I have a show running the typical 8 show week (that actually just hit 100 performances) where I designed a show control system to do lighting (eos), sound (qlab), and video (d3) over UDP OSC with a couple hundred cues. I am unsure if we have had a dropped trigger yet and we just hit 100 performances. Now, this is designed to avoid issues as much as possible with the biggest thing being I don't share the eos show control network with their aACN output, just cause that's a tonne of network traffic. But my sound nic shares the control network and once we hit show conditions that's been totally fine.

[Mike Post]

> For comparison, I have a show running the typical 8 show week (that actually just hit 100 performances) where I designed a show control system to do lighting (eos), sound (qlab), and video (d3) over UDP OSC with a couple hundred cues. I am unsure if we have had a dropped trigger yet and we just hit 100 performances. Now, this is designed to avoid issues as much as possible with the biggest thing being I don't share the eos show control network with their aACN output, just cause that's a tonne of network traffic. But my sound nic shares the control network and once we hit show conditions that's been totally fine.

That’s very interesting and makes sense. Unfortunately, many of us don’t have EOS consoles with multiple NICs installed, but suggesting the congestion of DMX/sACN/ArtNet traffic is impeding UDP messaging is a bit of an eye opener.

Reminds me of someone who told me they just couldn’t get Dante to work over the ETC network… (sigh)

Mike Post
mdp...@mac.com
http://mdpostdesign.com

[Dominic Bilkey]

Yes,
Unfortunately using OSC on the same network as sACN can lead to troubles.
Obviously, this is dependent upon several factors such as the number of universes being output, the scale of the traffic etc.

As per Chet's email best practise would always be to try and use a separate NIC on the EOS network to deal with OSC traffic. However, this isn't always possible....
One thing that is always worth checking is that the sACN network has IGMP implemented correctly on that network.
sACN is inherently multicast-tastic and can saturate devices quite quickly.

I've been in several situations with unreliable OSC in the EOS network and nearly always it has turned out to be an unmanaged (or incorrectly managed) IGMP setup where those pesky multicast sACN messages are running amuck. (or the show had been programmed in cue only throughout and every single cue was pushing out every parameter!)

D

mdp...@mac.com <mailto:mdp...@mac.com>
http://mdpostdesign.com <http://mdpostdesign.com>


--
Contact support anytime: sup...@figure53.com <mailto:sup...@figure53.com>
Follow QLab on Threads: https://threads.net/@QLabApp <https://threads.net/@QLabApp>
User Group Code of Conduct: https://qlab.app/code-of-conduct/ <https://qlab.app/code-of-conduct/>

---
You received this message because you are subscribed to the Google Groups "QLab" group.

To unsubscribe from this group and stop receiving emails from it, send an email to qlab+uns...@googlegroups.com <mailto:qlab+uns...@googlegroups.com>.
To view this discussion on the web visit https://groups.google.com/d/msgid/qlab/28D7B485-6FC2-4003-8851-3225FC48CD84%40gmail.com <https://groups.google.com/d/msgid/qlab/28D7B485-6FC2-4003-8851-3225FC48CD84%40gmail.com>.

[Paul]

I wanted to point out I have never had an issue with UDP between EOS and QLab. As others have said, on a modern, properly configured switched network you should never loose packets (ethernet frames) on a LAN.  I have also not be able to reproduce the problem of dropping UDP packers from EOS to QLab - despite trying various levels of (sACN, NDI) traffic on the LAN.  Every recent EOS console has 2 network ports so you should never need to send sACN and OSC over the same interface/segment. You can use TCP when sending from QLab to EOS. It would be good if ETC EOS supported a TCP client for OSC, although I'm not sure there is a problem here to be solved. (based on my experience of 100s of shows running both).  I suspect it maybe Windows (EOS consoles) dropping packets after a timeout but really need more info from those who have experienced packet loss (missing triggers) so it can be reproduced and a solution found.

[blindeye 1995]

I have These issues with an Element 2 only 1 Network Port used for ArtNet and OSC. I have midi over Ethernet on the Same Network A&H Qu 16 Sound Desk As the Router i use a Fritz box Made by AVM and a Netgear GS108 as a Switch Cathleen are all CAT5E 

Greetings 

Max

Am 04.08.2024 um 21:53 schrieb Paul <paulthom...@gmail.com>:

I wanted to point out I have never had an issue with UDP between EOS and QLab. As others have said, on a modern, properly configured switched network you should never loose packets (ethernet frames) on a LAN.  I have also not be able to reproduce the problem of dropping UDP packers from EOS to QLab - despite trying various levels of (sACN, NDI) traffic on the LAN.  Every recent EOS console has 2 network ports so you should never need to send sACN and OSC over the same interface/segment. You can use TCP when sending from QLab to EOS. It would be good if ETC EOS supported a TCP client for OSC, although I'm not sure there is a problem here to be solved. (based on my experience of 100s of shows running both).  I suspect it maybe Windows (EOS consoles) dropping packets after a timeout but really need more info from those who have experienced packet loss (missing triggers) so it can be reproduced and a solution found.

To unsubscribe from this group and stop receiving emails from it, send an email to qlab+uns...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/qlab/e8884ba2-e313-467c-8037-a18eb720601an%40googlegroups.com.

[Paul]

Ah! yes, I can well see that an Element might have problems using ArtNet and OSC. Not sure of the hardware specs of Element but I suspect limited memory. Probably time to upgrade to an Ion if you need both ArtNet/sACN and OSC. Is there any reason you can't use DMX out of the Element? ArtNet using broadcast and MIDI - an awful combination over a network.

[blindeye 1995]

Yes there is a reason its because ETC dosnt Follow the standart 30hz its 29,97hz and some devices cant Follow that. 

It cant be Out of Memory its an Element 2 its build for 12 universes we use at Max 3 Universes. 

Greetings 

Max

Am 07.08.2024 um 17:17 schrieb Paul <paulthom...@gmail.com>:

Ah! yes, I can well see that an Element might have problems using ArtNet and OSC. Not sure of the hardware specs of Element but I suspect limited memory. Probably time to upgrade to an Ion if you need both ArtNet/sACN and OSC. Is there any reason you can't use DMX out of the Element? ArtNet using broadcast and MIDI - an awful combination over a network.

To view this discussion on the web visit https://groups.google.com/d/msgid/qlab/41648fef-e5d0-40aa-b2f9-b34f74c99ca2n%40googlegroups.com.