Devices with Multiple Pseudoclocks

[Sean Donnellan]

Hi,

I would like to understand better whether the architecture of the hardware I am building would be suitable for use with Labscript.

I am developing a range of experimental control boards. Each board is connected to the other boards through a header. A single board is chosen to communicate with the PC and becomes the master board. The data sent from the PC is tagged with a board/channel number and the master board distributes it amongst the slave boards.

Each board has an FPGA on it. This means each channel of the board can have its own pseudoclocks. All boards are clocked from the same source (oscillator or synth input on one of the boards).

The biggest problem I can see is the ability to have many pseudoclocks per device. I would like to have each board appear as a separate device in Labscript. A DAC board for instance has 8 analog + 26 digital channels and so would have 34 pseudoclocks in a single device.

Another possible problem is having many devices (boards) but only having a single protocol to send and receive the data and clocks over a single USB cable. The protocol should work fine, I am just unsure if I can simply write code in the BLACS driver(s) to group together the pseudoclocks and data for multiple channels as I wish. Or does each device driver have to remain separate.

Thank you for the continued help. I thought it would be best to check how insurmountable these issues are before going further.

Sean

[Philip Starkey]

Hi Sean,

Just so I'm sure I've got my head around your setup, you have:

• Multiple boards, where each board has a single FPGA on it.
  
• All of the boards are connected together, and all communication will be funneled through a single "master" board, which distributes to relevant communications to slave boards
• Each board has multiple output channels controlled by a single FPGA, but the programmed FPGA logic is such that each channel is effectively it's own pseudoclock (that is that the FPGA accepts instructions to update a single channel and does not update the other output states when it processes the instruction)

Does that summarise the setup?

If so, it is a little unusual (we usually deal with devices where each instruction has to specify the state of all the outputs, even if only one has changed) for us, but I don't think that will be a problem.

I think I can see a reasonably simple way to implement your scenario. It will just require overriding some of the default behaviour of the pseudoclock class so that it doesn't combine the change times of all the channels before calling expand_timeseries. This would mean that each board is classed as a pseudoclock in labscript, with normal channels (Analog/digital) attached. However the behaviour (and instruction sets saved) would be equivalent to each channel effectively being individually pseudoclocked. I'll elaborate on this a bit more once you confirm I understand your setup correctly!

On a somewhat related note, how will your boards be triggered to begin the experiment? Currently we expect that the master pseudoclock (presumably your master board) receives a software command to start. And this pseudoclock sends digital triggers to the other boards to begin. Does that fit in with your setup?

Cheers,

[Sean Donnellan]

Hi Philip,

Thanks for the quick reply!

Yes your summary of the hardware is correct. Each channel of a single board has its own separate memory (for storing a pseudoclock and the data) with its own address ports (together with a block of FPGA logic). The clocking out of data for each channel is totally separate.

That sounds great that you think it should be possible to incorporate the hardware. If you could expand a little and I will have a look at the labscript code. So I would like to end up with a separate clocking signal array and data array for each channel of the device. Can you see the changes being made in such a way that the channels governed by a particular pseudoclock can be configured? Such that devices that have multiple channels per pseudoclock can still be used.

Yes the trigger is sent from the software to the master board that will in turn trigger the daughter boards.

Thanks again,

Sean

[Philip Starkey]

Hi Sean,

So I think what you want to do is this:

You will want to write one or more classes that subclasses PseudoClock. I think you'll probably want one class per type of board, but you could in theory create a generic class that is configured by parameters at instantiation (eg, the number of analog outputs could be a parameter of __init__ rather than hard-coded to a class, but that is up to you!)

Your PseudoClock subclass(es) will need to override the generate_code() method. You can see an example in the PineBlaster and PulseBlaster subclasses (the PineBlaster and PulseBlaster are good references for pseudoclock implementations by the way). You'll notice that the first line of the PineBlaster.generate_code() and PulseBlaster.generate_code() methods call Pseudoclock.generate_code(). You'll want to do this too. Now the PseudoClock.generate_code() method calls self.generate_clock(). You'll want to override the method called generate_clock() in your subclass(es) to change the default behaviour (do not call PseudoClock.generate_clock() from your generate_code method, calling PseudoClock.generate_code()results in your generate_clock method being called). You will want to copy and paste the code from the PseudoClock.generate_clock() method, into your generate_clock() method, and them modify it slightly. First I'll step you through what the important parts of that code fragment:

def generate_clock(self)

          # These lines are pretty self-explanatory. You shouldn't need to modify them

    outputs = self.get_all_outputs()

    self.do_checks(outputs)

    self.offset_instructions_from_trigger(outputs)

    

    #This returns an array of time points at which at least one of the channels/devices connected to the pseudoclock

    # has requested a state change

    change_times = self.collect_change_times(outputs)

    # This now loops over all outputs (including all channels on devices, like NI cards, that are clocked by this pseudoclock)

    # and gets them to effectively generate dummy instructions for all of the change_times that haven't already explicitly had an

    # instruction set

    for output in outputs:

        output.make_timeseries(change_times)

    # This method creates two things:

    #    all_times: a list of times....sortof. Each element is either a single time point, 

    #               or a numpy array of time points. This is usually used by each Output instance

    #               to generate output values at the times the clock will tick.

    #    clock: a high level description of the clock, stored as a list of dictionaries. 

    #           The dictionaries have information like the start time of this clock instruction

    #           the temporal step size, and the number of ticks at this step size

    #           So each dictionary can represent multiple clock ticks at a given rate.

    all_times, clock = self.expand_change_times(change_times, outputs)

    

    # This tells the Output instances to generate the output values for the times the clock will tick at

    for output in outputs:

        output.expand_timeseries(all_times)

    # This saves some useful information!

    self.clock = clock

    self.change_times = fastflatten(change_times, float)

    # This one might be of particular interest to you as it contains the time of every clock tick.

    self.times = fastflatten(all_times,float)

So, what you need to do is put a bunch of this code inside a loop, where the loop is looping over each channel (for devices where each channel is completely separate from other) or groups of channels (for those devices where channels share a "clock")

It should look something like this (note this is pseudocode, not guaranteed to work!)

def generate_clock(self)

          # These lines are pretty self-explanatory. You shouldn't need to modify them

    outputs = self.get_all_outputs()

    self.do_checks(outputs)

    self.offset_instructions_from_trigger(outputs)

    

    # get groups of channels that share a clock somehow

    groups = {'my group 1':[list of instances of Outputs, like instances of the AnalogOut class, that share a clock]}

    change_times = {}

    self.clock = {}

    self.change_times = {}

    self.times = {}

    # You need to put the original code in a loop something like:

    for group_name, list_of_channels_in_group in groups.items():

        change_times[group_name] = self.collect_change_times(list_of_channels_in_group)

        for output in list_of_channels_in_group:

            output.make_timeseries(change_times[group_name])

        all_times, clock = self.expand_change_times(change_times, outputs)

        for output in list_of_channels_in_group:

            output.expand_timeseries(all_times)

        self.clock[group_name] = clock

        self.change_times[group_name] = fastflatten(change_times, float)

        self.times[group_name] = fastflatten(all_times,float)

def generate_code(self)

    # calling this will result in your overridden function generate_clock being called

    PseudoClock.generate_code(self, hdf5_file)

    # you now have a list of time points in self.times and self.clock, for each grouping of channels

    # you can access the values for each output in output.raw_output

    # so something like:

    

    groups = .... <fill in the blanks>

    for group_name, list_of_channels_in_group in groups:

        for output in list_of_channels_in_group:

            output.raw_output # this is your list of output values for the channel

            # write this information to the HDF5 file in the appropriate format.

            # store it in '/devices/<device_name>/<whatever you want>'

Other things you should be aware of:

• You should specify the type of children (eg devices, or channels) that can be connected to your pseudoclock by listing the allowed classes in self.allowed_children.
• The slave boards will need to be connected to the master board in labscript via the trigger_device and trigger_connection parameters passed to the __init__ method of the slave pseudoclocks. you may need to "invent" physical connections depending on how the master board triggers the slave boards.

Hopefully that has given you enough to get started! Apologies if it isn't completely coherent, I didn't have much time to read over it!

As always, happy to answer any questions you have and to point you in the right direction (especially if something I wrote isn't clear). I know it can be time consuming trying to understand someone elses code! :)

Cheers,

--

Philip Starkey

School of Physics

Monash University

[Chris Billington]

Hi Sean,

Just to chime in, this sounds somewhat similar to another device that I had to implement a little hackily, the AdWin, the code for which can be found in labscript/devices/adwin.py

It similarly is a device with a bunch of somewhat independent boards, each of which has some digital or analogue outputs. So what I did with it was to define a toplevel device class, which was a subclass of Pseudoclock, and define a class for each of the the individual boards, also subclasses of Pseudoclock.

So it might also be worth looking there for examples of how the compilation process can be modified when devices don't quite match the 'global pseudoclock' model, though I don't think it matches exactly what you're doing.

As for programming the thing at run time in BLACS, I made a BLACS device tab that actually communicated with the device, and a bunch of other device tabs for the individual boards. At run time, the main tab doing communication would program all the boards. In 'manual mode', the individual tabs would emit events requesting the main tab to update individual outputs on a specific board. These inter-process events were part of the subproc_utils module, (now called zprocess). If you are interested in seeing this example too, it's in the older gtk branch of BLACS in the mercurial repository, in BLACS/hardware_interfaces/adwin.py is the main one, and adwin_ao_card.py and adwin_do_card.py are the tabs for the individual boards. Things were a little different in BLACS back then, but if you can see how the event passing bits work, you might want to do something similar for your device. I'm happy to provide more information on how to use these events if it's not obvious from the code and comments, as I haven't documented the zprocess module at all.

Cheers,

Chris

--
You received this message because you are subscribed to the Google Groups "The labscript suite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to labscriptsuit...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--
Chris Billington

School of Physics
Monash University

Victoria 3800, Australia
Ph: +61 423952456