"Heartbeat failure" error

[Jasmine Kalia]

Hello all,

We are using an Andor Neo camera to capture 8 images in succession (a follow-up to my previous post here), using a workaround to access the camera's PIV mode. We somewhat successfully wrote the blacs worker to use this camera. 

However, after a few shots, all the devices in blacs (our NI DAQ digital and analog cards, the prawnblaster, and the cameras) all crash and have to be manually restarted/reinitialized. We are given the error "Heatbeat failure" in the worker process terminal. The BLACS terminal shows that the Neo camera fails to transition to manual mode. I believe that when the next shot runs and tries to transition the devices to buffered, they all crash. We suspect that the time needed to save the images is too long for the timeout to transition to manual, and this is causing the crash. However, we tried to increase the timeout in blacs\experiment_queue.py but this did not seem to help. 

Strangely, we don't really encounter any "failed to transition to manual" type errors. The individual devices just crash. In the BLACS terminal, you can even see that it claims "All devices are back in static mode." Any thoughts or advice would be appreciated. I have attached images of the BLACS GUI with the device tabs in the error state, and the BLACS terminal. 

[Jasmine Kalia]

Hi all, just posting this here in case anyone ever encounters the same error. Our issue ended up being caused by sending 8 large images to the BLACS GUI. We commented out the line which sends the image to the GUI in the camera blacs worker and labscript works fine again.