maps and devices, libmapper script, and every other language

[Travis West]

Part 1: Maps and devices are conceptually the same thing

A device does three things. A device groups together some conceptually related endpoints aka signals. When the input signals to the device receive data, the device does something, or changes something about what it is doing. As a result of the things the device does, it sometimes updates its output signals.

A map does three things. A map connects together some conceptually related endpoints. When input signals to the map are updated, the map does something, or changes something about what it is doing. As a result of the things the map does, it sometimes updates its output signals.

Maps and devices are basically the same thing. They have inputs and outputs related to which they do some computation or transformation.

Part 2: Maps and devices in libmapper are different things.

Except for some important differences. Devices can be implemented in any language, and as such may access special resources such as sensors and transducers. Maps are always implemented in mapper script. Because all libmapper network participants can run a mapper script VM, this means maps can run anywhere on the network. However, maps cannot directly access special resources.

Part 3: Maps and devices in libmapper should be more like each other

If we added a way for maps to describe their endpoints (calls equivalent to mpr_sig_new in mapper script so that we can give map endpoints a name, range, unit, etc.), then it would be possible to implement devices in mapper script. This would be very very useful, and also super cool. It would be easy to extend existing devices with new signals in this way, where the new signals are derived from the existing ones. It would also be possible to define common devices like envelope generators, LFOs, sequencers, loopers, etc. purely in mapper script, and instantiate multiple copies of them, and so one. These mapper-script devices could run anywhere on the network (wherever it's most efficient probably?). It would also be easy to restore these devices between sessions; you would never need to worry about a script device being offline.

If we added a way for libmapper network participants to advertise capability to run scripts in languages other than mapper script, then it would be possible to implement things in these languages. For a concrete example, suppose we added a compile-time optional binding for faust. Any libmapper client running a version of the library compiled with this binding enabled would be able to run faust code. We could then write our maps in faust code, and indeed, we could also define devices in faust code. The faust code could run wherever it's most efficient to run it, and it would be trivial to restore these devices between sessions.

Wouldn't it be nice if your synth was as easy to restore as your maps?

Wouldn't it be nice if you could write maps in whatever language you already know and love instead of having to learn yet another language?

Part 4: Conclusion

So there are two proposals embedded here. One: allow devices to be defined and instantiated based on scripting languages such as mapper script. Two: allow compile-time-optional bindings to embed arbitrary 3rd party scripting languages into the libmapper ecosystem.

We have traditionally focused our binding efforts on binding libmapper to 3rd party ecosystems. The crux of my suggestion here is that we consider doing it the other way around. I think you'll agree that doing so offers some interesting possibilities. Faust, libpd, lua, javascript, glsl... in libmapper?! I think that would be rad.

TW

[Joseph Malloch]

Hi Travis – sorry for taking so long to reply. Hopefully my thoughts below are at least semi-coherent :)

1) most devices either have only inputs (e.g. “synthesizers”) or only outputs (e.g. “controllers”), with the “intermediate device” which performs some processing and then passes data downstream used much less often. As such, in my mind devices really only do one thing: group together some related signals. One could also think of libmapper devices as analogous to device drivers on a PC – communication with hardware at a low level, and not usefully reprogrammable/reconfigurable by users.

2) maps on the other hand are “soft”: dynamic media, user-programmable, ephemeral and interactive. When they gather together “conceptually related endpoints” the signals are “appropriated" rather than designed to be related. The source signals for a convergent map may be wildly different and belong to different devices—any design space relating these signals to each other may be specific to the user/interaction/piece and not make sense outside of this context.

3) I am open to using a different term for ”devices” if it helps explain or understand the concepts, but in my view more power should be moved from devices to maps. For example, I believe that any signals other than raw sensor data should in fact be “named maps” – essentially reprocessed and labeled datastreams but with the processing user-accessible and treatable at runtime. I also think this is the preferred solution to your concern about devices being offline: LFOs, sequencers, loopers, etc should be part of the maps (see MapLooper & timed maps proposal).

4) The idea of embedding more language support in libmapper is intriguing but comes with some complications. How do we maintain compatibility? In your concept not all devices support all languages so don’t we run the risk of having some software that won’t run on the currently available resources?

5) As a counter to your proposal: our philosophy so far has been to avoid asking users to think of libmapper as a platform, but instead to keep developing using their preferred languages and environments and use libmapper as lightweight glue to enable discovery and connection with others. This is why the language bindings always try to match the target language rather than impose libmapper conceptual models. We always intended that users of libmapper continue to write code in other languages to create interesting systems that could be glued together by libmapper.

6) Jean-Michaël suggested another approach that might be interesting as a compromise – that libmapper (and Ossia and perhaps others) would use a common WASM mapping description. We discussed the possibility of having both libmapper expression-script and WASM representations, which might imply the ability to use compiled WASM from other languages also.

Cheers,

Joe

--
You received this message because you are subscribed to the Google Groups "dot_mapper" group.
To unsubscribe from this group and stop receiving emails from it, send an email to dot_mapper+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/dot_mapper/CAGAPLrmX8iArw3j2yBKgJ4Dj1s1Rf-QhqiCrL_vegs2eC05RpQ%40mail.gmail.com.

[Travis West]

Hi Joe,

I pretty much agree with everything you're saying, and really appreciate this discussion.

I want to point out some interesting friction inherent in these ideas. In your point 3, you suggest moving more power from devices towards maps. I think this is a great idea, for many reasons. It makes session management simpler (maps never go offline), it potentially makes our maps more portable or platform independent (maps run on libmapper's abstract machine rather than any particular physical operating environment), it makes your code inherently compatible with everything that libmapper has bindings for, and it opens up the possibility to run processing wherever there is the most processing power and optimize network usage and other things like that to improve the quality of our signals. I like it.

But that directly runs against our philosophy expressed in your point 5. We want people to be able to keep writing code and making things work in whatever environment is most familiar, appropriate, whatever, and not have to think much about libmapper. We want to keep adding language and environment bindings so that libmapper just holds together a heterogeneous collection of things without strongly imposing libmapper's conceptual model.

Currently, we can't have it both ways. The more you use maps for everything, the more libmapper imposes itself and becomes a platform rather than a glue. The more you use devices for everything, the more you lose the potential management, portability, compatibility, and optimization benefits of maps.

Having WASM-defined maps (and why not also devices?) would definitely be one way of harmonizing these points. You could in principle define your maps in anything that can run on the WASM virtual machine, which includes a lot of languages. So you get to use libmapper as glue for whatever WASM-enabled language you prefer, as in point 5. But you would also get the benefits of relying more on code-defined maps, as in point 3. This is exactly what I am trying to suggest, only I would argue may not be any strong conceptual reason to stop at just WASM. We could also have language-embeddings so that you can define maps (and why not also devices?) in pd, faust, lua, javascript, etc. etc.

Your point 4 is very valid. Having lots of language embeddings would mean that we may not always benefit from some of the advantages of point 3, in case there is no network participant that can run your XYZ-lang code. I think in practice this would probably not be a big issue. Whatever our best supported platform is (probably MacOS or Linux), you would just have to run one computer on the network with a single device or monitor (looking at you Webmapper...), and it would be able to pick up all of the unusual embedded languages. But you're absolutely right that this might lead to unpleasant complications.

WASM definitely seems like an appropriate place to start looking for whoever eventually takes up these ideas and tries to make a prototype.

Cheers!

Travis

To view this discussion on the web visit https://groups.google.com/d/msgid/dot_mapper/8E05D9B5-47F6-4468-BD92-891EE79DD3A6%40gmail.com.