Caffe feature extraction blocking other threads

[Josh]

I am developing a multi-threaded program in C++ / Qt that uses Caffe to extract image features.

There are three threads:

• main: grabs frames from a camera, feeds the frames to thread1 and thread2, and displays the results of the threads.

• thread1: extracts features from image patches and performs classification once every n frames.

• thread2: propagates the classification results of thread1 to intermediary frames.

However, thread1 blocks thread2 during Caffe feature extraction but it does not block during other processing steps.

While this behaviour would be expected and necessary if a dependency is not met (i.e. thread1 is processing frame n and thread2 attempts to propagate the result to frames n+1 to 2n-1), thread2 pauses midway through its propagation cycle when features are being extracted from a "future" frame.

How can I use Caffe in a non-blocking way within Qt threads?

[Neil Nelson]

Josh, thread1 apparently takes a set of images (n frames) and outputs a classification. While it is doing that it blocks thread2. You mention blocking during Caffe feature extraction. Is your meaning that the blocking only occurs during a part of the Caffe sequence or blocks through the whole sequence? I suspect that it blocks through the whole sequence.

I am not familiar with QT Threads but suspect that the classification variable is on the call to the Caffe sequence (thread1) and that QT Threads blocks access to the classification variable for the reason that it is an update variable. Since thread2 depends on the classification variable it waits until thread 1 is complete.

You say thread2 pauses while thread1 is processing a future frame. Now if we copy the value of the classification variable of thread1 to a different variable after each run of thread1, the new variable will not be blocked while thread1 runs again. The new variable will be blocked for that very short period of copying the value from the output of thread1.

Those interested in threading should look at Intel's Threading Building Blocks.