Re: [QLab] OSC API /updates method?

[Andy Leviss]

On Tuesday, May 14, 2013, Mike Wyer wrote:

I've tried setting "/updates 1" both with and without a /workspace/{UID} prefix, with both TCP and UDP, and never once received an update from QLab. I've received replies to other messages ok, but not updates. How is it supposed to work?

How long are you waiting until you change something after subscribing for updates? It's not been updated in the API, it seems, but it times out after a short duration (I think it ended up at 20s) if you don't send a periodic thump to let QLab know you're still there. Try thumping every 15-20 and see if that helps. 

(I still don't really like this, and think it somewhat defeats the purpose of subscribing for updates, FWIW. Worst case, burden should be on QLab to ask if I'm still listening, and way less frequently than 20s.)

-Andy

[Mike Wyer]

On Tuesday, May 14, 2013 12:31:16 PM UTC+1, Andy Leviss wrote:

On Tuesday, May 14, 2013, Mike Wyer wrote:

I've tried setting "/updates 1" both with and without a /workspace/{UID} prefix, with both TCP and UDP, and never once received an update from QLab. I've received replies to other messages ok, but not updates. How is it supposed to work?

How long are you waiting until you change something after subscribing for updates? It's not been updated in the API, it seems, but it times out after a short duration (I think it ended up at 20s) if you don't send a periodic thump to let QLab know you're still there. Try thumping every 15-20 and see if that helps. 

I've tried updating things immediately, eg by hitting Go within 2 seconds of requesting updates, and I've been sending heartbeat thumps at 10-second intervals (which do get replies from QLab, so the connection is still live). I think part of the problem is that the docs don't say what to expect- which events generate an update, what an update message looks like, etc.

I've also tried using the TCP transport which isn't supposed to timeout the connection, but it's not clear how the push updates work in the context of a bidirectional SLIP-controlled link. Speaking of which, the docs say to use the "double-end" dialect of SLIP (sending the END char before and after each message to reduce the impact of line noise), but as far as I can tell the QLab implementation doesn't actually expect this.

 

(I still don't really like this, and think it somewhat defeats the purpose of subscribing for updates, FWIW. Worst case, burden should be on QLab to ask if I'm still listening, and way less frequently than 20s.)

It's UDP after all- no-one should care too much if the packets don't make it to the intended destination. It makes sense to tidy up unnecessary "connections" if the state in QLab starts getting too big, but QLab could simply require an ACK from the subscriber after an update. No ACK within 10s of the last update? Stop sending the updates.

 

-Andy

[Christopher Ashworth]

Hi Mike, (and Andy),

On May 14, 2013, at 3:01 AM, Mike Wyer <mi...@wyer.org> wrote:

> I've tried setting "/updates 1" both with and without a /workspace/{UID} prefix, with both TCP and UDP, and never once received an update from QLab. I've received replies to other messages ok, but not updates. How is it supposed to work?
>

> Relevant chunk of log:
> 2013-05-14 07:46:18.700 QLab[8570:303] [UDP socket 192.168.0.121:53001] sent OSC message: /workspaces
> 2013-05-14 07:46:18.701 QLab[8570:303] [UDP socket 192.168.0.121:53001] getting OSC reply: /reply/workspaces {"address":"/workspaces","status":"ok","data":[{"hasPasscode":false,"uniqueID":"B864B1FB-0070-46FE-AD80-29FFF4F94755","displayName":"launchpad.cues"}]}
> 2013-05-14 07:46:18.701 QLab[8570:303] [UDP socket 192.168.0.121:53001] sent OSC message: /workspace/B864B1FB-0070-46FE-AD80-29FFF4F94755/updates 1
> 2013-05-14 07:46:18.702 QLab[8570:303] [UDP socket 192.168.0.121:53001] sent OSC message: /thump
> 2013-05-14 07:46:18.703 QLab[8570:303] [UDP socket 192.168.0.121:53001] getting OSC reply: /reply/workspace/B864B1FB-0070-46FE-AD80-29FFF4F94755/thump {"status":"ok","data":"thump","workspace_id":"B864B1FB-0070-46FE-AD80-29FFF4F94755","address":"/workspace/B864B1FB-0070-46FE-AD80-29FFF4F94755/thump"}
>
> And then I can do anything in QLab, like launch a cue, tweak a setting, delete a cue, and never a peep from the OSC API about anything being updated.

The message are being sent, but they weren't being logged. I've just added logging so the update messages will also appear in the console.

> I'm prepared to believe I'm doing it wrong, or expecting something it doesn't support, but the OSC API docs are pretty vague: set /updates to true and you will receive push notifications of cue updates. And that's pretty much it.
>
> Incidentally, the log messages are a bit confusing as QLab *received" (rather than "sent") the /workspaces message, and *sent* the reply (rather than "getting").

This is accurate. The text [UDP socket 192.168.0.121:53001] indicates the "subject" of the sentence, so that socket did indeed send and get the listed message.

On May 14, 2013, at 7:31 AM, Andy Leviss <An...@DucksEcho.com> wrote:
>
> (I still don't really like this, and think it somewhat defeats the purpose of subscribing for updates, FWIW. Worst case, burden should be on QLab to ask if I'm still listening, and way less frequently than 20s.)

Makes sense, I'll look in to changing this.

On May 14, 2013, at 8:12 AM, Mike Wyer <mi...@wyer.org> wrote:

> I think part of the problem is that the docs don't say what to expect- which events generate an update, what an update message looks like, etc.

Events are generated when state changes; usually something like editing a cue. So, if you change the name of a cue, you'll get an update notification for that cue. If the playback position of the cue list changes, you'd get a specific update just for that.

The docs describe exactly what an update message looks like:

http://figure53.com/qlab/documentation/osc-api/#update-format

In time we may make these more granular, but they're enough for a lot as-is. (The QLab Remote iPad app uses this API.)

Cheers,
Chris

[Mike Wyer]

Hi Chris,

Thanks for getting back to me.

I've tried again with 3.0.4 and it's still not working for me. Can anyone spot where I'm going wrong?

Sticking with UDP for now, as that seems the best transport for handling the async updates.

I have a trivial script which listens on 53001, printing everything it sees, and sends the following messages to 53000:

/version

/alwaysReply 1

/connect

/updates 1

then a /thump every 20 seconds.

After starting the script and waiting for the thumps to start, I created a new group, renamed it, hit GO manually, then deleted the group again.

Neither /alwaysReply nor /updates get any response from QLab, though it registers that they arrived, and I didn't see any update messages (full log from QLab below, followed by the output from my script).

Is there something special about the way the iPad client is connecting or a particular value which happens to work for the /updates command?

2013-05-15 20:53:07.304 QLab[12073:303] [UDP socket 192.168.0.121:53001] sent OSC message:   /version

2013-05-15 20:53:07.305 QLab[12073:303] [UDP socket 192.168.0.121:53001] getting OSC reply:  /reply/version {"address":"/version","status":"ok","data":"3.0.4"}

2013-05-15 20:53:12.305 QLab[12073:303] [UDP socket 192.168.0.121:53001] sent OSC message:   /alwaysReply 1

2013-05-15 20:53:17.306 QLab[12073:303] [UDP socket 192.168.0.121:53001] sent OSC message:   /connect

2013-05-15 20:53:17.307 QLab[12073:303] [UDP socket 192.168.0.121:53001] getting OSC reply:  /reply/workspace/B864B1FB-0070-46FE-AD80-29FFF4F94755/connect {"status":"ok","data":"ok","workspace_id":"B864B1FB-0070-46FE-AD80-29FFF4F94755","address":"/workspace/B864B1FB-0070-46FE-AD80-29FFF4F94755/connect"}

2013-05-15 20:53:22.307 QLab[12073:303] [UDP socket 192.168.0.121:53001] sent OSC message:   /updates 1

2013-05-15 20:53:32.308 QLab[12073:303] [UDP socket 192.168.0.121:53001] sent OSC message:   /thump

2013-05-15 20:53:32.309 QLab[12073:303] [UDP socket 192.168.0.121:53001] getting OSC reply:  /reply/workspace/B864B1FB-0070-46FE-AD80-29FFF4F94755/thump {"status":"ok","data":"thump","workspace_id":"B864B1FB-0070-46FE-AD80-29FFF4F94755","address":"/workspace/B864B1FB-0070-46FE-AD80-29FFF4F94755/thump"}

2013-05-15 20:53:48.624 QLab[12073:303] GO [seconds=879.432399 hostTime=342565277907694 userData=(null)] : Group [ID:0 NUMBER: NAME:Main Cue List]

2013-05-15 20:53:48.625 QLab[12073:303]    start Group [ID:0 NUMBER: NAME:Main Cue List] at time [seconds=879.432399 hostTime=342565277907694 userData=(null)]

2013-05-15 20:53:48.625 QLab[12073:303]    load Group [ID:0 NUMBER: NAME:Main Cue List] to time 0

2013-05-15 20:53:48.626 QLab[12073:303]    load Group [ID:20 NUMBER:18 NAME:New group] to time 0

2013-05-15 20:53:48.626 QLab[12073:303]    callBackToAct Group [ID:0 NUMBER: NAME:Main Cue List] at time [seconds=879.432399 hostTime=342565277907694 userData=F5BDBE4A-F1B8-4B50-8787-8803AE666060]

2013-05-15 20:53:48.627 QLab[12073:303]    start action for Group [ID:0 NUMBER: NAME:Main Cue List]

2013-05-15 20:53:48.627 QLab[12073:303]    start Group [ID:20 NUMBER:18 NAME:New group] at time [seconds=879.432399 hostTime=342565277907694 userData=F5BDBE4A-F1B8-4B50-8787-8803AE666060]

2013-05-15 20:53:48.628 QLab[12073:303]    callBackToAct Group [ID:20 NUMBER:18 NAME:New group] at time [seconds=879.432399 hostTime=342565277907694 userData=65288FD4-2BC6-4C35-8AA1-32E7E51286FA]

2013-05-15 20:53:48.629 QLab[12073:303]    start action for Group [ID:20 NUMBER:18 NAME:New group]

2013-05-15 20:53:48.629 QLab[12073:303]    callBackToContinue Group [ID:20 NUMBER:18 NAME:New group] at time [seconds=879.432399 hostTime=342565277907694 userData=65288FD4-2BC6-4C35-8AA1-32E7E51286FA]

2013-05-15 20:53:48.629 QLab[12073:303]    completed Group [ID:20 NUMBER:18 NAME:New group]

2013-05-15 20:53:48.630 QLab[12073:303]    callBackToContinue Group [ID:0 NUMBER: NAME:Main Cue List] at time [seconds=879.432399 hostTime=342565277907694 userData=F5BDBE4A-F1B8-4B50-8787-8803AE666060]

2013-05-15 20:53:48.630 QLab[12073:303]    completed Group [ID:0 NUMBER: NAME:Main Cue List]

2013-05-15 20:53:51.504 QLab[12073:303]    stop Group [ID:20 NUMBER:18 NAME:New group]

2013-05-15 20:53:52.308 QLab[12073:303] [UDP socket 192.168.0.121:53001] sent OSC message:   /thump

2013-05-15 20:53:52.309 QLab[12073:303] [UDP socket 192.168.0.121:53001] getting OSC reply:  /reply/workspace/B864B1FB-0070-46FE-AD80-29FFF4F94755/thump {"status":"ok","data":"thump","workspace_id":"B864B1FB-0070-46FE-AD80-29FFF4F94755","address":"/workspace/B864B1FB-0070-46FE-AD80-29FFF4F94755/thump"}

####################################################################

The listener on 35001 saw:

["/reply/version\000\000,s\000\000{\"address\":\"/version\",\"status\":\"ok\",\"data\":\"3.0.4\"}\000", ["AF_INET", 49611, "192.168.0.121", "192.168.0.121"]]

["/reply/workspace/B864B1FB-0070-46FE-AD80-29FFF4F94755/connect\000\000\000,s\000\000{\"status\":\"ok\",\"data\":\"ok\",\"workspace_id\":\"B864B1FB-0070-46FE-AD80-29FFF4F94755\",\"address\":\"/workspace/B864B1FB-0070-46FE-AD80-29FFF4F94755/connect\"}\000\000\000", ["AF_INET", 57227, "192.168.0.121", "192.168.0.121"]]

["/reply/workspace/B864B1FB-0070-46FE-AD80-29FFF4F94755/thump\000,s\000\000{\"status\":\"ok\",\"data\":\"thump\",\"workspace_id\":\"B864B1FB-0070-46FE-AD80-29FFF4F94755\",\"address\":\"/workspace/B864B1FB-0070-46FE-AD80-29FFF4F94755/thump\"}\000\000", ["AF_INET", 58200, "192.168.0.121", "192.168.0.121"]]

["/reply/workspace/B864B1FB-0070-46FE-AD80-29FFF4F94755/thump\000,s\000\000{\"status\":\"ok\",\"data\":\"thump\",\"workspace_id\":\"B864B1FB-0070-46FE-AD80-29FFF4F94755\",\"address\":\"/workspace/B864B1FB-0070-46FE-AD80-29FFF4F94755/thump\"}\000\000", ["AF_INET", 54727, "192.168.0.121", "192.168.0.121"]]

On Tuesday, May 14, 2013 8:01:10 AM UTC+1, Mike Wyer wrote:

I've tried setting "/updates 1" both with and without a /workspace/{UID} prefix, with both TCP and UDP, and never once received an update from QLab. I've received replies to other messages ok, but not updates. How is it supposed to work?

Relevant chunk of log:

2013-05-14 07:46:18.700 QLab[8570:303] [UDP socket 192.168.0.121:53001] sent OSC message:  /workspaces

2013-05-14 07:46:18.701 QLab[8570:303] [UDP socket 192.168.0.121:53001] getting OSC reply: /reply/workspaces {"address":"/workspaces","status":"ok","data":[{"hasPasscode":false,"uniqueID":"B864B1FB-0070-46FE-AD80-29FFF4F94755","displayName":"launchpad.cues"}]}

2013-05-14 07:46:18.701 QLab[8570:303] [UDP socket 192.168.0.121:53001] sent OSC message:  /workspace/B864B1FB-0070-46FE-AD80-29FFF4F94755/updates 1

2013-05-14 07:46:18.702 QLab[8570:303] [UDP socket 192.168.0.121:53001] sent OSC message:  /thump

2013-05-14 07:46:18.703 QLab[8570:303] [UDP socket 192.168.0.121:53001] getting OSC reply: /reply/workspace/B864B1FB-0070-46FE-AD80-29FFF4F94755/thump {"status":"ok","data":"thump","workspace_id":"B864B1FB-0070-46FE-AD80-29FFF4F94755","address":"/workspace/B864B1FB-0070-46FE-AD80-29FFF4F94755/thump"}

And then I can do anything in QLab, like launch a cue, tweak a setting, delete a cue, and never a peep from the OSC API about anything being updated.

I'm prepared to believe I'm doing it wrong, or expecting something it doesn't support, but the OSC API docs are pretty vague: set /updates to true and you will receive push notifications of cue updates. And that's pretty much it.

Incidentally, the log messages are a bit confusing as QLab *received" (rather than "sent") the /workspaces message, and *sent* the reply (rather than "getting").

Background:

I'm trying to get QLab to play nicely with the Novation Launchpad control surface. While I can use MIDI note messages to control the button lights on the launchpad, it's a real pain to create a Group around every cue and program them in.

Initially, I thought I could write a script to update the .cues plist to do that, but the data is pretty opaque at first glance, and it would be much cooler to have the launchpad work with any workspace, so I moved to the OSC api.

But unless I can tell when a cue has been armed or triggered, I can't update the state of the launchpad.

Thanks,

Mike

[Christopher Ashworth]

Hi Mike,

I did a quick test based on your examples, built in QLab. I've attached the workspace.

This one is slightly simplified from yours, with just

/connect
/updates 1
/thump

And a start cue that then re-triggers the thump every 20 seconds.

Setting the log level to 2, and then running these cues, I see updates sent out in the console log, e.g. when changing the playback position:

2013-05-15 17:02:29.231 QLab[13164:303] [UDP socket 127.0.0.1:53001] getting OSC update: /update/workspace/7386E044-244A-402B-A2A1-C26ECDE39807/cueList/0/playbackPosition 3
2013-05-15 17:02:29.705 QLab[13164:303] [UDP socket 127.0.0.1:53001] getting OSC update: /update/workspace/7386E044-244A-402B-A2A1-C26ECDE39807/cueList/0/playbackPosition 4
2013-05-15 17:02:29.881 QLab[13164:303] [UDP socket 127.0.0.1:53001] getting OSC update: /update/workspace/7386E044-244A-402B-A2A1-C26ECDE39807/cueList/0/playbackPosition 5
2013-05-15 17:02:30.091 QLab[13164:303] [UDP socket 127.0.0.1:53001] getting OSC update: /update/workspace/7386E044-244A-402B-A2A1-C26ECDE39807/cueList/0/playbackPosition 8

If you try running this do you see the same?

Best,
Chris

[Mike Wyer]

Aha!

I was sending an external OSC command "/updates 1" from my client program, which wasn't working.

But if I put an OSC cue into my QLab workspace, getting QLab to send "/updates 1" to itself, it works perfectly.

Since I already have a cue set to run when the workspace opens, this works fine in practice.

I'm not sure if it's the intended behaviour, but it's fantastic for me.

Thanks for taking the time to help on this issue.

I'll share anything useful I come up with for using the Launchpad as a control surface.

Cheers,

Mike

[Chris Ashworth]

Interesting, there must be a bug here-- the updates should certainly go out to your client if it requests them, and in fact they are supposed to be tracked in a client by client basis, so that only clients that have sent the /updates 1 command would actually get the updates.

Thanks, I'll have to dig deeper here. I don't suppose your script is something I could try for testing?

(mobile)

[Mike Wyer]

Test script attached. It is just a scratchpad for playing with raw OSC in ruby right now, so please don't expect much sophistication or elegance :)

--
--
Change your preferences or unsubscribe here:
http://groups.google.com/group/qlab

Follow Figure 53 on Twitter: http://twitter.com/Figure53

---
You received this message because you are subscribed to a topic in the Google Groups "QLab" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/qlab/NJYWDD8hrzY/unsubscribe?hl=en.
To unsubscribe from this group and all its topics, send an email to qlab+uns...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[Mike Wyer]

Not sure the attachment arrived intact. Here's the content:

#!/usr/bin/env ruby

require 'socket'

SLIP_END = 192.chr

SLIP_ESC = 219.chr

NULL = 0.chr

class String

  def osc_pad

    out = self + "\0" + ("\0" * (3 - (self.length) % 4))

    if out.length % 4 != 0

      puts "waaah"

      p out, out.length

    end

    return out

  end

end

osc_client = UDPSocket.open

osc_client.connect('127.0.0.1', 53000)

osc_server = UDPSocket.open

osc_server.bind('127.0.0.1', 53001)

Thread.new do

  while true

    p osc_server.recvfrom(1500)

  end

end

def test_osc_pad

  p '/updates 1'.osc_pad

  p '/updates 1'.osc_pad.length

  p '/alwaysReply 1'.osc_pad

  exit 0

end

#osc_client.send('/workspaces'.osc_pad, 0)

osc_client.send('/version'.osc_pad, 0)

sleep 1

osc_client.send('/alwaysReply 1'.osc_pad, 0)

sleep 1

osc_client.send('/connect'.osc_pad, 0)

sleep 1

osc_client.send('/updates 1'.osc_pad, 0)

osc_client.send('/go'.osc_pad, 0)

sleep 10

while true

  osc_client.send('/thump'.osc_pad, 0)

  sleep 20

end

To unsubscribe from this group and all its topics, send an email to qlab+unsubscribe@googlegroups.com.

[Christopher Ashworth]

Hi again Josh and Mike,

Believe it or not, I never actually forgot about this. And I've finally and a chance to sit down and work on it this morning.

Josh, first, thanks for calling my attention to the fact that we weren't using double SLIP on outgoing messages. That is now fixed.

As regards the rest:

For the Ruby script Mike attached, I finally figured out what is happening here: it's not actually sending commands that are in QLab's API.  They LOOK like they are, but they're not.

Specifically:

osc_client.send('/alwaysReply 1'.osc_pad, 0)

This is sending a message where the address is "/alwaysReply 1".   NOT the address "alwaysReply" with the argument "1", but the address "/alwaysReply 1" with no argument at all.

This is why this particular script is not behaving as you expect, because the messages it is sending are a mix of legit API commands and invalid commands.

Now, this doesn't necessarily mean we don't have a bug here.  Your report about sending an "/updates 1" command not working the second time sounds like there still could be a problem there, but I'm unsure how to exactly try to reproduce what you were doing, as I don't have the script you were running.

As such, I'm open to new reports of bugs, although in this case the errors in the original script muddy the waters and may have produced behavior that looked like bugs when they were not.

Best,

Chris

On May 19, 2013, at 5:00 PM, Josh Senick <jse...@gmail.com> wrote:

Hi Mike and Chris,

Hope this will help and doesn't muddy the waters.  Something odd is definitely going on.

First up, when QLab is sending packets over TCP, it doesn't use the double-end SLIP dialect, it uses single-end.  It seems to accept double-end packets just fine.  Whether this is something that should be changed is up to Chris; once I coded my SLIP parser to use the single-end dialect while receiving, my scripts began to work perfectly.

When connecting via TCP, the '/updates 1' works fine, at least at first.  However, if I quit my script and run it again, the second time QLab receives the '/updates 1' command, it doesn't send any updates.  I have to restart QLab to get '/updates 1' to work again.

Over TCP, sending '/workspace/[id]/updates 1' works every time, even if QLab had stopped responding to '/updates 1'.  However, if I run a second copy of a script and have IT send the '/workspace/[id]/updates 1' command as well, only the FIRST script receives updates.

Here's where it gets really weird:

If I run your ruby script, which sends '/updates 1' over UDP, and then send '/updates 1' over TCP with my script, YOUR script receives the updates and mine doesn't.

Chris, hope this helps you track down the problems!  Please let me know if you need code samples or any other info.

Josh

[Mike Wyer]

Thanks for putting me straight- I had a feeling I was missing something, and for some reason I hadn't understood the detailed message format description (address, type_tag, arguments).

Now that the script is following the actual OSC protocol, I'm seeing updates as expected.

Just needed this change:

osc_client.send('/updates'.osc_pad + ',i'.osc_pad + [1].pack('N'), 0)

And now I get useful output like:

["/update/workspace/EACE925F-D241-4331-845A-F46323E448C0/cueList/0/playbackPosition\x00\x00\x00,s\x00\x00180\x00", ["AF_INET", 49407, "127.0.0.1", "127.0.0.1"]]

["/update/workspace/EACE925F-D241-4331-845A-F46323E448C0/cue_id/180\x00\x00\x00,\x00\x00\x00", ["AF_INET", 49407, "127.0.0.1", "127.0.0.1"]]

Clearly I need to wrap the calls a lot more neatly than that, but it does at least generate the right sequence of bytes.

Thanks,

Mike

--

--
Change your preferences or unsubscribe here:
http://groups.google.com/group/qlab
 
Follow Figure 53 on Twitter: http://twitter.com/Figure53
 
---
You received this message because you are subscribed to a topic in the Google Groups "QLab" group.

To unsubscribe from this topic, visit https://groups.google.com/d/topic/qlab/NJYWDD8hrzY/unsubscribe.

To unsubscribe from this group and all its topics, send an email to qlab+uns...@googlegroups.com.