Can Julia handle a large number of methods for a function (and a lot of types)?

[Oliver Schulz]

Hello,

I'm designing a scheme to control and read out hardware devices (mostly lab instruments) via Julia. The basic idea is that each device type is characterized by a set of features/properties that have a value type and dimensionality and that can be read-only or read/write (a bit like SCPI or VISA, conceptually).

Features/properties would be things like "InputVoltage" for a multimeter, "Waveform", "TriggerLevel", etc. for an oscilloscope, and so on. They would be represented by (abstract or singleton) types. Obviously, certain features would be supported by many different device classes.

For the API, I was thinking of an interface like this:

current_feature_value = mydevice[SomeDeviceFeature]
mydev[SomeDeviceFeature] = new_feature_value

resp. things like

mydev[SomeMultiChannelFeature, channel]

for feature with multiple instances (e.g. in muli-channel devices). The implementation would, of course, be

getindex(dev::SomeDeviceType, ::Type{SomeDeviceFeature}) = ...
setindex!(dev::SomeDeviceType, value, ::Type{SomeDeviceFeature}) = ...

etc. This would mean having a large number of classes for all features (let's say around 200), and an even larger number of methods ( O(number_of_device_classes * typical_number_of_features_per_device), let's say around 5000) for getindex and setindex!. Can Julia handle this, or would this result in long compile times and/or bad performance?

An alternative would be, of course, to represent each feature by a specific set of methods like "trigger_level_set, "trigger_level_get", etc. However, I like the idea of representing a device feature as something that can be passed around (possibly between hosts). Also, there may be other feature operations beyond a simple set and get, which would many several functions for each feature.

Any insights?

Cheers,

Oliver

[Jeffrey Sarnoff]

Assuming your handshaking and signal management is keeping information available in some easy to discern and decode manner and you are not pushing latency issues, Julia should be a good platform for that.  Compilation time usually is not a concern -- and you can make your package autoprecompile (which it will do when loaded the first time and after alteration). If you pay attention to the general guidelines here  performance tips  and this note (still quite useful), see how Ian Dunning improved this code and use the Devectorize package  if you find yourself writing that sort of code.

[Jeffrey Sarnoff]

There are alternative ways to orchestrate your API. Given the always-on-duty nature of your microagents and the fact that (once configured and mapped into the software as a useable description) it is helpful that your information propagators are persistent (oscilloscope A measures properties prop1..N and analyzes behaviours analysis1..N).

You may find it useful to think about minimizing the real time work of the app, trading some real time activity for work done both before and after scanning, scooping and styling the datastream.  Organizing the app to keep even small self-contained stages of action or reflection very loosely coupled and choosing types/structures to ease specific subtasks also may help.

[Oliver Schulz]

Hi Jeffrey,

On Sunday, March 27, 2016 at 3:51:18 PM UTC+2, Jeffrey Sarnoff wrote:

Assuming your handshaking and signal management is keeping information available in some easy to discern and decode manner and you are not pushing latency issues, Julia should be a good platform for that.

I have no doubt of that. :-) Both in terms of execution speed, and other features (native coroutines, etc.) Julia is an excellent platform for this. I used to do this in Scala (using Akka actors), which was fine, but I believe it can be done even better in Julia (at least given certain requirements and boundary conditions).

 

 Compilation time usually is not a concern -- and you can make your package autoprecompile [...] pay attention to the general guidelines here  performance tips  [...] and use the Devectorize package

Yes, I will certainly use precompilation as far as possible (well, not for actual user measurements scripts, etc. but for the device implementations), and I am familiar with the Julia performance tips and memory management recommendations. My question rather revolved around whether Julia's dispatch will be happy with O(1000) (or more) methods for a functions.

[Jeffrey Sarnoff]

Glad to know you are up to speed ... 

Try this in v0.4 and in v0.5-dev: length( methods(Base.call) )   v0.4 gives 1000, v0.5 gives 1 (deprecated)

So, Julia is known to work with 1000-fold signatured methods.  The fact that v0.5 is better groomed suggests its worthwhile to consider approaching your same outcome more parsimoniously or with greater subspecificity .. lifting some largely clustered highly overlapping signature-driven doings and letting ... all the oscilloscopes be so scoped, all the multimeters be multiply metered ... so what you think about as a single method with signatures everywhichway, is realized as subspecializations of a [partially] shared abstractions.

abstract Device

abstract Scope <: Device

abstract Meter <: Device

@enum vendor Tektronix, Fluke

type LogicAnalyzer{vendor} <: Scope

     ...

end

type Multimeter{vendor} <: Meter

     ...

end

TLA7000    = LogicAnalyzer{Tektronix}( ... )

Hydra2638A = Mutimeter{Fluke}( ... )

handshake{T<:Device}(anyDevice::T) = ...

read{T<:Scope}(scope::T) = ..

read{T<:Scope}(scope::T{Tektronix}) = ..

read{T<:Meter}(meter::T) = ..

[Jeffrey Sarnoff]

    I use this to think about inter&intra type relationships -- and wrote it above,  ymmv   ymwv!

          read{T<:Scope}(scope::T{Tektronix}) = ...

[Andrew Keller]

Hi Oliver,

Just a heads up that I've been working on something similar for the past six months or so. I'm fortunate enough to be starting a new project where I have no legacy code and the freedom to play around a bit. I think we have similar ideas: properties are abstract / singleton types in my code as well. I really like your idea of using `getindex` and `setindex!` but I didn't think of that initially (sigh...) and at present have defined methods like:

configure(ins::Instrument,::Type{DeviceFeature}, value, channel::Integer=1)
inspect(ins::Instrument, ::Type{DeviceFeature})

My approach would have the same scaling issues you worry about. I hope it won't be such an issue and it would be kind of a shame to resort to separate functions for each instrument property in Julia, so I'm just going full speed ahead.

My code is not yet ready for public use and is not stable, but the source is in my repo if you're curious. I should caution that its a bit of a mess right now and the documentation is not up-to-date, so some of it reflects things that are not be true anymore, or ideas I have abandoned / improved upon. Seeing as how I need to do some real science soon I hope the source code will stabilize within a few weeks. I would welcome discussion if you want to combine efforts on this. I'm not sure if I want to invite PRs yet until I have the project in a more stable state but discussion is always good.

As I've been working on this since I started learning Julia, there are a number of things I did early on that I regret and am now fixing. One thing you'll find becomes annoying quickly is namespace issues: if you want to put different instruments in different modules, shared properties or functions may require a lot of careful imports/exports that tend to break easily. I'd suggest that rather than hard coding properties in source files, you store what you need in some standard external format. I'm now storing my SCPI codes, Julia property names, argument types, etc. in a JSON file that Julia parses to generate code with metaprogramming. With this approach I believe I'll be able to write code to scrape instrument manuals, and thereby generate code for new instruments semiautomatically. You'd still need to decide on property names and such, and write regular expressions to scrape the manuals, but that's not so bad.

Finally, I think some really great things could be done with this sort of code if Julia had better native support for physical units. For instance, on a vector network analyzer you could write something like:

vna[Frequency] = 4GHz:1MHz:10GHz

and thereby set frequency start, stop, and step at once. I have a units package for Julia v0.5 that I'm also developing when I have the time (Unitful.jl), but there are some tweaks to Base that would help the package play nicely. If that sounds appealing, it'd be great if you could voice your support for that, e.g. here: https://github.com/JuliaLang/julia/pull/15394

Best wishes,

Andrew Keller

[Oliver Schulz]

Hi Andrew,

sorry, I overlooked your post, somehow ...

On Monday, March 28, 2016 at 3:47:42 AM UTC+2, Andrew Keller wrote:

Just a heads up that I've been working on something similar for the past six months or so.

Very nice - I'm open collaborate on this topic, of course. At the moment, we're still actively using my Scala/Akka based system  https://github.com/daqcore/daqcore-scala) in our group, but I'm starting to map out it's Julia replacement. daqcore-scala currently implements some VME flash-ADCs and some slow-control devices like a Labjack, some HV Supplies, vaccuum pressure gauges, etc., but the architecture is not device specific. The Julia version is intended to be equally generic, but will initially target the same/similar devices. I'd like to make it more modular, though, it doesn't all have to end up in one package.

One requirement will be coordinated parallel access to the instruments from different places of the code (can't have one Task sending a command to an instrument while another task isn't done with his query - which may span multiple I/O operations - yet). Also multi-host operation for high-throughput applications might become necessary at some point. The basis for both came for free with Scala/Akka actors, and I started Actors.jl (https://github.com/daqcore/Actors.jl) with this in mind. Another things that was easy in Scala was representing (potentially overlapping) devices classes using traits. I'm thinking how do best do this in Julia - but due to Julias dynamic nature, explicit device classes may not be absolutely necessary. Still, I'd like to used as much explicit typing as possible: When dealing with hardware, compile-time checks are very much preferable to run-time checks with live devices (can't afford to implement a full simulator for each device). :-)

 

As I've been working on this since I started learning Julia, there are a number of things I did early on that I regret and am now fixing. One thing you'll find becomes annoying quickly is namespace issues: if you want to put different instruments in different modules, shared properties or functions

Yes, that was one of the reasons I wanted to explore getindex/setindex, it avoids namespace conflicts on the functions. Of course this shifts the issue to the device feature classes, but I hope that a central package can host a set that will cover most devices (and be open to people contributing additional common features). Highly device-specific features can live within the namespace of the package implementing the device in quesiton,

and shouldn't be exported.

 

For instance, on a vector network analyzer you could write something like:

vna[Frequency] = 4GHz:1MHz:10GHz

Yep, this is exactly what I had in mind. Of course there will also have to be a producer/consumer mechanism for devices with high-rate/continuous output (e.g. fast waveform digitizers). Channels and/or actors are an attractive approach.

 

and thereby set frequency start, stop, and step at once. I have a units package for Julia v0.5 that I'm also developing when I have the time (Unitful.jl)

 

I've seen Unitful.jl - how does it compare to SIUnits.jl, especially performance-wise? Are they orthogonal or two approaches to the same problem?

Cheers,

Oliver

[Andrew Keller]

Hi Oliver,

Regarding your first question posed in this thread, I think you might be interested in this documentation of how functions will work in Julia 0.5 if you haven't read it already. There is some discussion of how methods are dispatched, as well as compiler efficiency issues. I hope you don't mind that I've tried out your setindex and getindex approach. It is very pleasant to use but I have not benchmarked it in any serious way, mainly because I'm not sure what a sensible benchmark would be. If you'd like me to try out something I'll see what I can do.

It sounds like you have probably been thinking deeply about instrument control for a much longer period of time than I have. I'll write you again once I've gotten our codebase more stable and documented, and I welcome criticism. I haven't given much thought to coordinated parallel access yet but I agree that it will be important. A short summary is that right now we have one package containing a module with submodules for each instrument. Each instrument has an explicit type and most methods are generated using metaprogramming based off some template for each instrument type. Most instrument types are subtypes of `InstrumentVISA`, and there are a few methods that are assumed to work on all instruments supporting VISA. I must say, it is not obvious what the best type hierarchy is, and I could easily believe that traits are a better way to go when describing the functionality of instruments. You can find any number of discussion threads, GitHub issues, etc. on traits in Julia but I don't know what current consensus is.

Unitful.jl and SIUnits.jl globally have the same approach: encode the units in the type signature of a value. Accordingly, Unitful.jl should have great performance at "run-time," and is a design goal. Anytime some operation with units is happening in a tight loop, it should be fast. I have only had time to look at some of the generated machine code but what I've looked at is the same or very close to what is generated for operations without units. I have not optimized the "compile-time" performance at all. Probably the first time a new unit is encountered, some modest performance penalty results. I'd like to relook at how I'm handling dimensions (length, time, etc.) because in retrospect I think I'm doing some odd things. An open question is how one could dispatch on the dimensions (e.g. x::Length). I tried this once and it sort of worked but the type-gymnastics became very ugly, so maybe something like traits would be better.

Last I checked, the two packages are different in that Unitful.jl supports: rational powers of the units (you can do Hz^(1/2), useful for noise spectra); non-SI units like imperial units; the ability to add new units on the fly with macros; preservation of exact conversions. My package only supports Julia 0.5 though. I think the differences between the packages arise mainly because SIUnits.jl was written when Julia was less mature. SIUnits.jl is still sufficient for many needs and is remarkably clever.

Thanks for linking to your code. I have no experience with Scala but I will take a look at it.

Best,

Andrew

[Oliver Schulz]

Hi Andrew,

On Saturday, April 2, 2016 at 7:10:00 PM UTC+2, Andrew Keller wrote:

Regarding your first question posed in this thread, I think you might be interested in this documentation of how functions will work in Julia 0.5 if you haven't read it already.

Yes, I'm aware that some big changes are coming with 0.5, and many of them (e.g. threads and fast anonymous functions) are of course highly relevant to this kind of application. For now I'm kinda stuck with 0.4 for actual use cases, because so many packages (e.g. PyPlot, Gadfly, ...) have trouble with 0.5 at the moment. But once 0.5 is ready, I will probably not even try to keep things compatible with 0.4, as this is a new project anyway.

 

I hope you don't mind that I've tried out your setindex and getindex approach. [...] It is very pleasant to use but I have not benchmarked it in any serious way [...] If you'd like me to try out something I'll see what I can do.

Thanks, you're more than welcome! The more the merrier, and this can only profit from wide testing. It would be good to try how this performs with, say, about 500 feature types and 500 device types, each implementing like 100 features. I hope this will still perform well in dynamic dispatch situations - maybe one of the Julia experts can weigh in here?

 

It sounds like you have probably been thinking deeply about instrument control for a much longer period of time than I have.

Well, thinking and learning a lot from my earlier mistakes. :-)

 

You can find any number of discussion threads, GitHub issues, etc. on traits in Julia but I don't know what current consensus is.

I did play with some of the current approaches to traits in Julia a while ago (Mauro's and Tim's work), and it's definitely something to watch (I'd love to see something like that in Base some day). For now, I hope we may be able to get by without explicit "device classes".

 

Thanks for linking to your code. I have no experience with Scala but I will take a look at it.

Don't judge to harshly. :-) This has been a work in progress for many years, and it is in active use for two long-term physics experiments and multiple lab applications - but since it was always driven by our current needs, I often didn't find time to port new ideas and concepts to older portions of the code. One of the goals was always to use it for both high-rate physics DAQ, and low-rate "slow-control"/SCADA applications. I was planning a major overhaul, but recently decided that Julia will be a better platform in the long run for various reasons (one of them that most students don't have time to learn several programming languages, and Scala isn't really an option for our kind of data analysis).

Implementing specific devices has actually only been a fraction of the work - a large part has always been implementing communication protocols. For various devices, I needed (and implemented) VXI11, Modbus, SNMP, VME (over ethernet bridge), CANOpen (for one speficic gateway), and various vendor specific ASCII and binary protocols (e.g. Pfeiffer vaccum, old Keithley ASCII, etc.). Plus things like an SCPI parser, etc.. I look forward to port all of that to Julia - well, eventually ... ;-)

I'd like the core to stay pure-Julia, though this will be challenging with VXI11 and SNMP, as there's no native-Julia ONC-RPC or SNMP library. And for devices that really need a vendor-specific VISA driver (because SCPI over VXI11 is not enough) Instruments.jl (https://github.com/BBN-Q/Instruments.jl) may come in play (haven't tried it yet).

I'd love to work with other people interested in this - Julia is great at making data analysis fast and easy, not reason it shouldn't be as great at taking the data in the first place!

 

  

Unitful.jl and SIUnits.jl globally have the same approach [...] My package only supports Julia 0.5 though. [...]An open question is how one could dispatch on the dimensions (e.g. x::Length).

Ah, right, now I remember - i kinda mixed up SIUnits and Unitful, sorry. I did give Unitful a quick try when you announced it on the list, and I was very impressed. But then I kinda had to force myself to put it aside for a while, since I can't really switch to 0.5 yet. ;-) I do recall the discussion about dispatching on units, though - that would be way cool, but even without, Unitful will be a great ingredient to any Julia data acquisition solution.

Cheers,

Oliver

[Stefan Karpinski]

I'm afraid that changing the subject here removes all context for this message in most email systems. Groups seems to keep the message in context:

https://groups.google.com/d/msg/julia-users/Dt6nbfhtaNQ/TCy6zT9HAAAJ

[Oliver Schulz]

Oh, darn - sorry, Stefan. I wanted to change the topic since the discussion had moved quite a bit from the original question, but I didn't consider email clients.

[Stefan Karpinski]

No worries. I just wanted to provide a link for context.

[Oliver Schulz]

So, since we got you on this thread, Stefan - what's your take on the original question? :-)

I mean, is it Ok to have (potentially) thousands of methods for getindex and setindex, or will this have serious consequences for compile time and/or dynamic dispatch situations?