Possible variations in labscript architecture

[www.rohitp...@gmail.com]

Hi all,
We want to implement the labscript suite for the computer control of our new ultracold quantum gas experiment. After going through labscript documentation, I would like to have suggestions on the correct architecture for implementation. The default labscript theme uses external pseudo clock devices (like pulseblaster) for time synchronization of various I/O cards (like those from NI).

I was wondering if one could use an I/O card itself as a master clocking card too for triggering other cards. In this connection I could think of four alternative ways of implementing labscript. I would like to have suggestions on which would be the correct and the best way to go.

Plan A :
Buy only NI PXI I/O cards (we would like PXI cards so that they sit nicely in a PXI chasis and can use the PXI backplane trigger lines for cross-card communication). Use the internal clock of one of the cards (which will be also used as an I/O card) to trigger other cards. This avoids need of buying a separate base clock card.

Plan B:
Buy NI PXI I/O cards with PXI chasis and use the internal clock of PXI chasis to trigger all cards. This resembles original labscript architecture but we are not sure about technical caveats.

Plan C:
Buy NI PXI I/O cards, a NI PXI timing/synchronization module and PXI chasis. Use the PXI timing module as a master clock for all cards. This again resembles original labscript architecture.

Plan D:
Buy the exact hardware what the labscript documentation mentions.

Would be helpful to have your valuable reviews on these plans.

[Philip Starkey]

Hi,

labscript is reasonably strict on having some sort of pseudoclock implemented; that is a device which can produce a variable frequency clock on a digital output (or equivalently, generate an arbitrary digital pulse sequence).

Because of this, I'm unsure if plan A/B/C will work because I'm not certain that they will be able to produce a variable frequency clock. As far as I'm aware, both the internal clock and the PXI timing module produce fixed frequency pulses (although I'll admit I may be wrong here as I'm not up to date with the latest NI capabilities, so please correct me if you know something I do not!).

The options I think are most likely to work are:

Plan D: Buy either a PulseBlaster (expensive) or PineBlaster (cheap - but may have some trouble if you want to use more than 1 at a time due to some bugs in the external trigger code. They are also difficult to externally reference to a 10MHz signal)

Plan E: Dedicate one of the PXI cards to producing a variable frequency clock from the fixed frequency internal clock. This card would then be able to clock the other cards, and satisfy the labscript suite requirements. This hasn't been done before, but wouldn't be too hard for you to implement. Basically you need to subclass PseudoclockDevice in labscript and unpack the variable frequency clock generated internally by the labscript code into a set of instruction that can be stepped through with a fixed frequency. This is a bit of a waste of an NI card (because it won't be able to do anything else but generate this clock), but might be cheaper than buying a PulseBlaster (depending on PulseBlaster/NI card purchased) and would not suffer from the PineBlaster caveats.

Plan F: Develop your own pseudoclock device (for example using an FPGA) that doesn't have any of the caveats of the PineBlaster. I've done some work towards this already, and have a functioning prototype using some open source code from another control system, and a $200USD FPGA development board. However I have yet to release it because I haven't had time to test it on an actual ultracold atom experiment. I would be happy to share the code with you if you wanted to help finish the testing.

I hope this helps your planning! Please let me know if you have further questions or want me to clarify, or explain in more detail, something I wrote.

Cheers,

Phil

[Lincoln Turner]

Hi Rohit & Phil,

Let me add two more options to this already rather long list:

Plan D1: I saw only last week that spincore are now selling a Pulseblaster card that fits in a PXI slot: the PulseBlasterESR-CompactPCI. While it is nominally a "compact-PCI" card, it will fit in a PXI (or PXIe) slot, and I've confirmed this with Spincore. They are sending me one to evaluate, as we have some colleagues building a new system with NI PXIe cards in an NI crate. I'll post here when we've confirmed that it works, but it is very likely to do so. Labscript already contains some code for the PB ESR series of devices, which will require at most minor modifications. Best of all, this cards is "only" USD995, which is small compared to the cost of NI PXIe cards and crates. Of course, it is still al lot compared to FPGA dev boards that ought to be able to do the same thing.

Plan G: Use (two) counter/timers with some complicated (but entirely software-configured) routing of their terminals, on your existing NI X-series cards to generate the pseudoclock. I recently did a block diagram that I am pretty confident shows that the hardware can serve as a pseudoclock. This would be a major step forward for labscript as it would free us from external pseudoclock hardware. But my idea needs to be tested in hardware, and it would need some (fairly minor I expect) additions to core labscript compiler to drive it. Realistically this won't be developed by us in the immediate future.

If you are going with a PXIe crate already, then Plan D1 is pretty easy and very likely to work (and we will have tested it in the next few weeks). 

The NI cards that are specifically for synchronisation and timing probably do have the flexibility to generate pulse sequences for a pseudoclock but one of these I looked at the other day cost >$7000! For this price you could have at PB with a couple of channels of DDS, and quite a bit of change.

Cheers,

Lincoln

--
You received this message because you are subscribed to the Google Groups "The labscript suite" group.
To unsubscribe from this group and stop receiving emails from it, send an email to labscriptsuit...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

--

Lincoln Turner

Senior Lecturer

Fellow - Monash Education Academy

School of Physics & Astronomy

Monash University

G.06, New Horizons Centre

20 Research Way, Clayton Campus

Victoria 3800, Australia

T: +61 3 9905 1726

bec.physics.monash.edu.au

[Rohit Prasad Bhatt]

Hi Lincoln, Hi Philip,

Thanks for setting the conversation rolling and your insightful suggestions. I will think on all possibilities you have suggested and then get back to you with what we would prefer (possibly with more questions).

Regards,

Rohit

[Rohit Prasad Bhatt]

Hi all,

So after discussion among ourselves we think that the obvious way to go is to test with the PulseBlasterESR-CompactPCI (Plan D1). This is the quickest and the easiest way to go about. Also it seems easy to set up its compatibility with labscript. But I am not sure if one could route the clocks generated to different cards via the PXI backplane lines or one needs to make a wired connection between pulse blaster and the PFI port of NI I/O card.

As backup plans, I personally think that the order of approach could be : FPGA clocking device (Plan F, since we will have support from Philip), using NI card itself as a I/O and clocking device (Plan G, testing Lincoln's ideas), dedicating an NI card to variable clock generation (Plan E).

Or get rid of this backup list and just invest in the conventional way of Plan D which must work. Not sure about this though. We would be really happy to see Plan D1 working so that we do not have to fiddle too much with technicalities of clocking device.

Now although NI provides something along the lines of a variable frequency clock (e.g. Astronics PXIe-1209), the number of output channels is only 2.

Thanks for providing all these suggestions.

Regards,

Rohit

[Lincoln Turner]

A quick update...

Our loaner PulseBlasterESR-CompactPCI arrived today at Swinburne, but it does not fit in the PXIe-1082DC rack that they have there! It appears to need a plain PXI slot, not a PXIe or PXIe-hybrid slot, despite what Spincore said on this. 

To proceed with this device we have two options:

1. Modify the spincore card - should be a relatively straightforward connector mod - I've asked spincore if they can do this.

2. Buy a different crate with at least some conventional PXI slots in it. 

I'll update the list with what we find out.

Lincoln

[Philip Starkey]

Hi Rohit,

If the PXI/PXIe issue Lincoln describes above is a problem for you (and he can't convince spincore to change it) then I think the FPGA solution (Plan F) will work fine. I'm currently in the process of uploading the work I've done on this, but I need to make sure that the labscript device implementation still works (I wrote the labscript_device support a couple of years ago, but as it hasn't been part of the main repository, I may need to update it - particularly to make sure it works with Python 3 which the rest of the labscript suite now supports!). Anyway, I'll let you know when I've finished that.

Another colleague has also been working on a separate FPGA design, which he might add labscript support for as well (and it's cheaper, and faster than the one I've worked on). So hopefully there will be a few FPGA options to choose from in the near future if you have to go to the backup plan!

I also had a look at the Astronics card you linked to. Unfortunately I don't think it can change the clock frequency under precision timing (looked like you had to issue updates to the clock frequency in software) so I don't think it's suitable for labscript either.

Hopefully the PulseBlaster PXI card is suitable (although I think I agree with you that it's likely you'll have to route the BNC output to a PFI in on the NI card, and possibly replace the crystal oscillator with something derived from one of the clocks in the PXI chassis...the manual doesn't seem to mention any sort of routing through the PXI connector which is unfortunate). 

Let us know if you have any more thoughts or questions :)

Cheers,

Phil

Philip Starkey

PhD Candidate

Teaching Associate (Sessional)

School of Physics & Astronomy

Monash University

G07A, New Horizons Centre

[Rohit Prasad Bhatt]

Hi all,
In continuation with the conversation so far, we have some plans to move on :

1. We plan to buy NI PXIe-1078 chassis, with PXIe-6739 analog output card and PXIe-6537 digital I/O card. Now this chassis does not have any conventional PXI slot in it and as Lincoln mentioned, the PulseBlasterESR-CompactPCI won't hence fit into the chassis. But we could buy, a PulseBlaster PB24-100-4k-PCIe (LINK) which sits on the mother-board of our computer and then route the clock to NI cards with cables. This gives an advantage that in future if we want to expand our system by buying more of PXIe-6739 or PXIe-6537 cards, we will have compatible slots available for them.

We had also thought of buying NI PXIe-1062Q chassis, which has four conventional PXI slots and hence might fit the PulseBlasterESR-CompactPCI, but then the other 3 PXI slots are kind of useless as they won't support the PXIe cards bought in future.

2. Try variable time base clock generation in software. Now we don't have much idea on this path. I personally can only vaguely recall the name Cicero from MIT which generates a variable time base. Otherwise we could also try the FPGA based solutions from Phillips suggestions, if those cards will fit in our NI PXIe-1078 chassis.

Are we correct to think that 1. is a convenient and safer way of moving on as compared to 2? If yes then we could place our order already and it would be nice.

As an aside I also noticed that Spincore provides a PulseBlasterUSB (LINK) which is somewhat expensive but is more compact, comes in an enclosure and provides breakout to all clock signals.

[Philip Starkey]

Hi Rohit,

Path 1 is definitely the more well travelled path. We have PCI/PCIe NI cards in a PC that are fed a clocking signal externally via a PulseBlaster (which the NI cards pass amongst themselves via a RTSI cable between the cards as we don't use PXI/PXIe cards).

We have support for both the PulseBlaster USB and PulseBlaster ESR Pro-500 already in our labscript_devices repository, and I believe they are in use at other institutions running the labscript suite (we use the PulseBlaster DDS-II-300-AWG at Monash, which they don't seem to sell anymore). Adding support for the other variants should be very easy (it's just a different clock resolution as far as I'm aware).

Regarding path 2, Cicero doesn't actually generate the variable timebase in software, it uses a USB FPGA device running custom firmware. This is what my FPGA device is based on (it's not quite the same because I had to fix a bug in their firmware). It's not a PXIe device though (apologies if that wasn't clear in my previous emails) so doesn't really have any advantage over the PulseBlaster PB4-100-4k other than price ($200 vs $500). You can follow the progress of integrating this device with labscript here, but I'm mainly doing it as a pathway for people to move easily from Cicero to labscript. It's certainly not production ready yet.

So yes, path 1 with a PulseBlaster is the safest option, and is the best option that is ready for you to use right now. A significant fraction of labs running the labscript suite are using a PulseBlaster device of some kind. Even if you buy a PulseBlaster that we don't already have direct support for, it should either just work with an existing Python class, or need a few lines of code in a subclass in order to accommodate the slightly different specifications. 

Cheers,

Phil

[Rohit Prasad Bhatt]

Hi all,

I have a question regarding the difference between PulseBlaster PB24-100-4k-PCIe (LINK) and PulseBlasterUSB (LINK). Would the USB connection introduce any latency?

I would say NO because the card has an onboard buffer memory. So while this PulseBlasterUSB is triggering the experiment it should fetch data from its own buffer memory and not depend on USB connection to the host computer.

I look forward to your opinions.

@ Phil : Since you are making your own pseudo clock, may I suggest you to also consider an enclosed version of it something like PulseBlasterUSB. Actually people have made PulsePal (LINK) but it is not fast enough for cold atom experiments.

[Chris Billington]

Hi Rohit,

It's possible the PulseBlasterUSB might be slower to program than the PCI card, but yes, once it's programmed the minimum instruction length is set by on-board limitations. BLACS does make status checks to ask whether the PulseBlaster is done yet, and so these may be slower over USB, this would imply a slight delay before BLACS is aware that the shot is complete. However the delay due to USB is probably small compared to other software delays in BLACS. And the slower programming, if it's a thing, is well-mitigated by BLACS' 'smart programming' feature, where only the instructions that have actually changed are reprogrammed and not the whole memory of the device.

So in practice I think there's little if any difference between using  a USB or PCI PulseBlaster.

-Chris