Simple Perl Interface for generating ArtNet packets

[dcor...@googlemail.com]

Hi @ll,

after trying around with python I must say that I am not a python expert. So I began to write my own Perl-package for sending DMX data to my (Raspi-)ArtNet node. I put my code to Sourceforge so anyone can check it out. Currently it's very alpha - but sending DMX data works fine. You can check it out via SVN from:

https://sourceforge.net/projects/simpleartnet/

I think I will make a downloadable ZIP after getting all my ideas working well (e.g. multithreading, fading-function, ...). If you have questions or ideas or you will like to work on the project: please let me know!

Regards, Holger

[Simon Newton]

It would be more useful if you could write a Perl client for OLA.
There are protobuf modules listed at
https://code.google.com/p/protobuf/wiki/ThirdPartyAddOns


Simon

> --
> The Open Lighting Group: open-l...@googlegroups.com, #openlighting
> (irc.freenode.org)
> To unsubscribe from this group, send email to
> open-lightin...@googlegroups.com
> For more options, visit https://groups.google.com/groups/opt_out?hl=en

[dcor...@googlemail.com]

Simon,

I didn't know something about ProtoBuf but I just read the documents from Google. I don't think that ProtoBuf can help with generating ArtNet packets because the format of the binary serialized structure is not flexible enough. Just correct me, but there are some problems with them:
- data format is not binary representation, e.g. there is also a number stored to identify the used variable (I think it's needed for flexible extending the protocol)
- no uint8 and other data types - so no possibility to encode needed data
- no information about (integer-)data representation in big- or little-endian mode.

I think it will not work... so coding the protocol structure for each language will be the only way, or not?

Regards, Holger

[Peter Newman]

Simon means talk Protobuf to OLA and make that do the hard work of generating ArtNet, so it's compliant etc.

[dcor...@googlemail.com]

Sorry for my lack of understanding, but does olad understand a ProtoBuf protocol? How do I configure this (I don't see any usable protocol when generating a universe)? I have:

ola-artnet.conf
ola-dummy.conf
ola-e131.conf
ola-espnet.conf
ola-ftdidmx.conf
ola-karate.conf
ola-kinet.conf
ola-opendmx.conf
ola-osc.conf
ola-pathport.conf
ola-sandnet.conf
ola-shownet.conf
ola-spi.conf
ola-stageprofi.conf
ola-usbdmx.conf
ola-usbserial.conf

Is the right proto-description inside the file open-lighting/common/protocol/Ola.proto?

Thanks, Holger

[Simon Newton]

On Tue, Sep 3, 2013 at 6:42 AM, <dcor...@googlemail.com> wrote:
> Sorry for my lack of understanding, but does olad understand a ProtoBuf
> protocol?

Yes, take a look at
http://www.opendmx.net/index.php/OLA_developer_info#RPC_System

>How do I configure this (I don't see any usable protocol when
> generating a universe)? I have:
>
> ola-artnet.conf
> ola-dummy.conf
> ola-e131.conf
> ola-espnet.conf
> ola-ftdidmx.conf
> ola-karate.conf
> ola-kinet.conf
> ola-opendmx.conf
> ola-osc.conf
> ola-pathport.conf
> ola-sandnet.conf
> ola-shownet.conf
> ola-spi.conf
> ola-stageprofi.conf
> ola-usbdmx.conf
> ola-usbserial.conf

Those are just config files. Protobufs are used to communicate between
the clients and olad.

>
> Is the right proto-description inside the file
> open-lighting/common/protocol/Ola.proto?

That's the one.


Once your (Perl) client can talk protobuf rpc to olad you can use any
of the devices / protocols that OLA supports.

Simon

>
> Thanks, Holger
>
> Am Dienstag, 3. September 2013 14:59:52 UTC+2 schrieb Peter Newman:
>>
>> Simon means talk Protobuf to OLA and make that do the hard work of
>> generating ArtNet, so it's compliant etc.
>

[dcor...@googlemail.com]

Hi all,

I started to work with Perl Google::ProtocolBuffers. But my current problem is not the Perl implementation but the lack of information about the interface documentation of Ola.proto.

What I want to do is setting DMX values in a specified universe on a specified ola-server. I know how to build server connections on port 9010 tcp. I also know how to parse the Ola.proto and how to encode data for the server. What I miss is a idea of what I have to do to pack the data or call which function. Do I have to connect to a universe first? Must I use  Ola::Proto::RegisterDmxRequest() or can I send direct DmxData objects?

I there a description of how to speak to the server with protobuf?

Thanks, Holger

[Simon Newton]

On Sat, Sep 7, 2013 at 2:35 AM, <dcor...@googlemail.com> wrote:
> Hi all,
>
> I started to work with Perl Google::ProtocolBuffers. But my current problem
> is not the Perl implementation but the lack of information about the
> interface documentation of Ola.proto.
>
> What I want to do is setting DMX values in a specified universe on a
> specified ola-server. I know how to build server connections on port 9010
> tcp. I also know how to parse the Ola.proto and how to encode data for the
> server. What I miss is a idea of what I have to do to pack the data or call
> which function. Do I have to connect to a universe first? Must I use
> Ola::Proto::RegisterDmxRequest() or can I send direct DmxData objects?
>
> I there a description of how to speak to the server with protobuf?

The RPC format is defined in common/rpc/Rpc.proto . It's a matter of
serializing the proto messages from Ola.proto and wrapping them in a
RpcMessage proto which also contains a transaction #, method name and
request type.

There is a python implementation in python/ola/rpc/StreamRpcChannel.py
and a C++ one in common/rpc/StreamRpcChannel.cpp


I'll be in the IRC room most of today if you want to chat about it.

[dcor...@googlemail.com]

Thanks for the fast response,

I tried to use the Rpc.proto along the Ola.proto but the real problem is the Perl module for parsing the protobuf descriptions. The manual says:

...

Declarations of RPC services are currently ignored, but their support is planned (btw, which Perl RPC implementation would you recommend?)

...

S**t... that's really bad because I thought I have the right solution for speaking to the server via protobuf :-(
I have to check other implementations or I have to change my strategy to use Perl.

Regards, Holger

[Simon Newton]

On Sat, Sep 7, 2013 at 10:12 AM, <dcor...@googlemail.com> wrote:
>
> Thanks for the fast response,
>
> I tried to use the Rpc.proto along the Ola.proto but the real problem is the
> Perl module for parsing the protobuf descriptions. The manual says:
>
> ...
>
> Declarations of RPC services are currently ignored, but their support is
> planned (btw, which Perl RPC implementation would you recommend?)

There are two separate parts:
i) a generator which produces the stubs from the services declarations
ii) an RPC mechanism

The protobuf package has generators (i) for Python, Java & C++ but
they are marked as deprecated [1] and implementations are encouraged
to use the plugin system [2] to write their own. Unfortunately this
means you need to write C++ to generate Perl. OLA still uses the old
format but I've been meaning to switch over since at some point
support for the legacy generators will go away. I suspect our code
will also become simpler if we don't have to use the generic services.


There is no open source standard for ii). OLA has it's own
(StreamingRPCChannel) in C++, Python & Java (partial support).

If you're not scared away by all of this jump on IRC and let's talk.
I'm happy to put time in if it makes it easier for other people to
write clients in the favorite language.

[1] Search for cc_generic_services in
https://developers.google.com/protocol-buffers/docs/proto
[2] https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.compiler.plugin.pb


Simon

[dcor...@googlemail.com]

Hi Simon,

Am Samstag, 7. September 2013 19:23:20 UTC+2 schrieb Simon Newton:

On Sat, Sep 7, 2013 at 10:12 AM,  <dcor...@googlemail.com> wrote:
>
> Thanks for the fast response,
>
> I tried to use the Rpc.proto along the Ola.proto but the real problem is the
> Perl module for parsing the protobuf descriptions. The manual says:
>
> ...
>
> Declarations of RPC services are currently ignored, but their support is
> planned (btw, which Perl RPC implementation would you recommend?)

There are two separate parts:
 i) a generator which produces the stubs from the services declarations
 ii) an RPC mechanism

The protobuf package has generators  (i) for Python, Java & C++ but
they are marked as deprecated [1] and implementations are encouraged
to use the plugin system [2] to write their own. Unfortunately this
means you need to write C++ to generate Perl.  OLA still uses the old

This seems to be more time consuming than I thaught... :-(
 

format but I've been meaning to switch over since at some point
support for the legacy generators will go away. I suspect our code
will also become simpler if we don't have to use the generic services.


There is no open source standard for ii). OLA has it's own
(StreamingRPCChannel) in C++, Python & Java (partial support).

If you're not scared away by all of this jump on IRC and let's talk.
I'm happy to put time in if it makes it easier for other people to
write clients in the favorite language.

Sorry - yesterday there was no more time for going on IRC.
This is a really interesting thing, but I just have not enough time for this at the moment. I think I will use my Perl-ArtNet-Hack to send DMX values until I have more time to get more in touch with Google-Protobuf. When I finished my current light project I can try to get back on this.
 

[1] Search for cc_generic_services in
https://developers.google.com/protocol-buffers/docs/proto
[2] https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.compiler.plugin.pb

[2] seems to be not easy to understand if someone does not dive in deeper in the material of Protobuf. C++ and other languages are not the problem for me. I will try to read something more about this.

Regards, Holger

[Simon Newton]

No problem. I've looked over the proto API code and it doesn't look
too difficult to write a service generator. I'll have a go at that
this week - once the C++ one is done it should be pretty easy to
modify it for Python / Java / Perl etc.


Simon

[Peter Newman]

If you want a quick and dirty interface in Perl, with the benefits of OLA doing all the lighting comms properly, have a look at the JSON API Holger. See here for more info including a basic Perl example of sending a universe:
http://www.opendmx.net/index.php/OLA_JSON_API

[dcor...@googlemail.com]

Peter,

hehe - that's a nice one :-) Nearly like my ArtNet hack I am currently using.

I think the JSON interface is a little bit slower than raw ArtNet or Protobuf. I will make some timing tests.

Thanks, Holger