Racket bindings
14 views
Skip to first unread message
Ken Harris
Apr 12, 2022, 5:42:52 PM4/12/22
to open-lighting
Hello, OLA!
I’ve found <https://docs.openlighting.org/ola/doc/latest/rpc_system.html> and it all seems feasible, though a bit outside my area of expertise. I’m not clear on why I need a custom “service generator” (isn’t that what “protoc” is for?), or exactly what RpcChannel / RpcController need to do (the unofficial Clojure/Scheme bindings don’t seem to have anything with a name like that — ?). I’m certainly misunderstanding how this all is supposed to fit together.
I installed a protobuf library for Racket, and running "protoc --racket_out=dict_stubs/ Dictionary.proto” (with the sample .proto file in the docs) generates a small Dictionary.rkt file with a couple definitions, so that seems like a good start, but I'm not sure where to go from here.
Can anyone help dumb down the process even more for someone who’s never done protobuf RPC? Thanks!
Thomas White
Apr 13, 2022, 3:58:13 AM4/13/22
to open-l...@googlegroups.com
Hi Ken,
For my project Starlet (high-level Guile Scheme DSL for theatrical
lighting control: https://github.com/taw10/starlet), I briefly looked
at writing a Protobuf interface, but found it much easier just to write
a thin shim between Guile and the OLA C++ StreamingClient API:
https://github.com/taw10/starlet/blob/main/src/guile-ola.cpp
It could probably have been achieved in pure Scheme using the foreign
function API, but I found it easier just to write those 100-or-so lines
of C++. You have both these options in Racket, I think, and more. In
an earlier version, I used the JSON API (HTTP), but I found the
overhead too high.
Good luck with the project! I love that there are multiple projects
for "Lispy" lighting control (at least three, including yours).
Tom
--
Thomas White <t...@bitwiz.me.uk> <t...@physics.org>
For my project Starlet (high-level Guile Scheme DSL for theatrical
lighting control: https://github.com/taw10/starlet), I briefly looked
at writing a Protobuf interface, but found it much easier just to write
a thin shim between Guile and the OLA C++ StreamingClient API:
https://github.com/taw10/starlet/blob/main/src/guile-ola.cpp
It could probably have been achieved in pure Scheme using the foreign
function API, but I found it easier just to write those 100-or-so lines
of C++. You have both these options in Racket, I think, and more. In
an earlier version, I used the JSON API (HTTP), but I found the
overhead too high.
Good luck with the project! I love that there are multiple projects
for "Lispy" lighting control (at least three, including yours).
Tom
Thomas White <t...@bitwiz.me.uk> <t...@physics.org>
Peter Newman
Apr 14, 2022, 8:36:51 AM4/14/22
to open-lighting
Hi Ken,
I'll confess the Protobuf stuff is one of the bits I'm less familiar with as it generally just works!
I think the reason for writing your own service generator is/was:
"Early versions of protobufs came with a service generator.
The generic service generator has been deprecated since the 2.4.0
release since code generated was rather inflexible (in trying to be all
things to all people it left many needs un-addressed)."
The service generator is a plugin to protoc which essentially just specifies what the code it writes should look like, protoc still deals with parsing the protobuf definitions etc.
The CHICKEN Scheme library just seems to wrap libola, so doesn't implement the RPC stuff. The clojure library has hardcoded them here:
You can see the Python RPC Channel stuff here:
The actual OLA protobuf is here:
And the RPC which it's wrapped within is here:
Finally some other successes/attempts here if they're useful:
Let us know if you get stuck/need more help.
Tom, I've linked to your implementation from that issue. If you wanted to split it out to a standalone library I could add it to the list on our website too...
Ken Harris
Apr 14, 2022, 1:00:31 PM4/14/22
to open-l...@googlegroups.com
Hi, Tom,
Thanks for replying!
That’s an interesting approach. I’m not a C++ programmer, and my impression is that C++ ABIs aren't exactly stable. Have you had any issues there? That approach seems like more work, to me, too.
In general, I’d rather stick to a stable, documented interface (i.e., RPC) when possible. Also, avoiding C++. :-)
I noticed a couple references to a JSON API but didn’t look too closely. It seemed like a less-supported or less-recommended (?) interface. JSON does seem like significantly higher overhead than protobuf, especially for sending DMX.
- Ken
Thanks for replying!
That’s an interesting approach. I’m not a C++ programmer, and my impression is that C++ ABIs aren't exactly stable. Have you had any issues there? That approach seems like more work, to me, too.
In general, I’d rather stick to a stable, documented interface (i.e., RPC) when possible. Also, avoiding C++. :-)
I noticed a couple references to a JSON API but didn’t look too closely. It seemed like a less-supported or less-recommended (?) interface. JSON does seem like significantly higher overhead than protobuf, especially for sending DMX.
- Ken
Ken Harris
Apr 14, 2022, 1:10:12 PM4/14/22
to open-l...@googlegroups.com
Hi, Peter,
Hmm, the protobuf service generator route doesn’t seem very popular!
At this point, I’m inclined to start with the “Very Simple Example” (in Python), port it to Racket, and then gradually refactor it into the interface I want. For the foreseeable future, I’ll only need a tiny bit of OLA’s protocol ("rpc UpdateDmxData”, plus the 2 messages it uses).
Except for a couple changes to plug-in IDs, I don’t see any changes to Ola.proto in the past 7 years, so I think it’s prudent to hand-code my own protobuf structures, and not waste any time trying to get an automatic Ola.proto-->Racket service compiler to work perfectly. I can always revisit this decision if Ola.proto starts changing again, or if I need a lot more of it.
My goal at this point is to build a prototype to test some lighting ideas, so architecting a complete and independently-usable library (for OLA -- or for any other feature of my system) is not really on my short-term to-do list. Maybe someday.
Thanks for the pointers!
P.S., there’s mention of an IRC channel on the Open Lighting webpage, but it’s virtually empty. Is there a new channel where people hang out?
- Ken
Hmm, the protobuf service generator route doesn’t seem very popular!
At this point, I’m inclined to start with the “Very Simple Example” (in Python), port it to Racket, and then gradually refactor it into the interface I want. For the foreseeable future, I’ll only need a tiny bit of OLA’s protocol ("rpc UpdateDmxData”, plus the 2 messages it uses).
Except for a couple changes to plug-in IDs, I don’t see any changes to Ola.proto in the past 7 years, so I think it’s prudent to hand-code my own protobuf structures, and not waste any time trying to get an automatic Ola.proto-->Racket service compiler to work perfectly. I can always revisit this decision if Ola.proto starts changing again, or if I need a lot more of it.
My goal at this point is to build a prototype to test some lighting ideas, so architecting a complete and independently-usable library (for OLA -- or for any other feature of my system) is not really on my short-term to-do list. Maybe someday.
Thanks for the pointers!
P.S., there’s mention of an IRC channel on the Open Lighting webpage, but it’s virtually empty. Is there a new channel where people hang out?
- Ken
Peter Newman
Apr 18, 2022, 5:58:53 PM4/18/22
to open-lighting
We try hard to keep our C++ API stable too, primarily adding stuff in a backwards compatible way and occasionally deprecating but rarely actually removing stuff. We follow https://semver.org/ too.
TBH most of the general improvements to OLA are new plugins, RDM test stuff and improvements to some of the example programmes using the API. Plus keeping up with other people's changing libraries...
Talking RPC would avoid another step in the process, although may be an unnecessary complication if you just want to send DMX. You're right though, I wouldn't recommend the JSON API for sending DMX at any serious frame rate.
If there's a working Protobuf service compiler, I'd strongly suggest giving that a go, it's likely to be less hassle than hand coding, especially if you want to ever do more than send DMX. You can see some bit-banged RPC here if you really want (although note the endian issue, as it's normally only ever local to one machine and architecture): https://github.com/OpenLightingProject/ola/pull/1687/files
Yeah the library query was mostly aimed at Tom. If/when you get some code that works though, stick a comment in that issue, at least it's a potential starting point for others.
If you just want to send DMX in the easiest way possible, outputting it via STDOUT into http://docs.openlighting.org/ola/man/man1/ola_streaming_client.1.html is probably easiest!
I've personally not had much luck rejoining the IRC channel since all the Freenode chaos happened, but it was never very busy and somewhat rare for realtime discussions to actually overlap. The history/log of the mailing list at least means people can find stuff again...
Thomas White
May 1, 2022, 8:28:16 AM5/1/22
to open-l...@googlegroups.com
Hi,
Sorry for the delayed response - it's been a packed couple of weeks.
About the ABI in particular, I don't see this being a problem since I
compile the whole project from source far more often than the ABI could
change. The OLA *API* is also much more stable than my own project will
ever be. I'm also not much of a C++ programmer, but writing this small
interface layer was no problem at all.
A native Protobuf implementation would probably be better, because it
means less foreign code running in the Scheme process (making life
easier for Guile's memory management). But without a Protobuf
implementation for Guile, it was more work than I have time for.
About IRC, you can find me on #openlighting on Libera since a couple of
weeks ago, but an official channel registration would need some action
from Peter. It's very quiet there, and the mailing list also has its
advantages.
Best wishes,
Tom
Sorry for the delayed response - it's been a packed couple of weeks.
About the ABI in particular, I don't see this being a problem since I
compile the whole project from source far more often than the ABI could
change. The OLA *API* is also much more stable than my own project will
ever be. I'm also not much of a C++ programmer, but writing this small
interface layer was no problem at all.
A native Protobuf implementation would probably be better, because it
means less foreign code running in the Scheme process (making life
easier for Guile's memory management). But without a Protobuf
implementation for Guile, it was more work than I have time for.
About IRC, you can find me on #openlighting on Libera since a couple of
weeks ago, but an official channel registration would need some action
from Peter. It's very quiet there, and the mailing list also has its
advantages.
Best wishes,
Tom
Reply all
Reply to author
Forward
0 new messages