Multiprocessing with Caffe in Python

[Rohan]

I have a pretrained caffe model and I'm trying to make a prediction with it. However, I am trying to perform this prediction in a separate process using python's multiprocessing module so as to not slow down the main program. However, the process seems to freeze when trying to make the prediction. Is this possible with caffe? If not, are there other methods to do efficient parallel processing with caffe in python?

Thanks! 

[Alex Orloff]

do you have at least 2 gpu?

четверг, 28 января 2016 г., 3:32:46 UTC+3 пользователь Rohan написал:

[Rohan Bopardikar]

No I only have one, will I have to recompile in CPU only mode to use multiprocessing.

Note that I'm not trying to train a model, just make a prediction of a single data point with a model I've already trained 

Regards,

Rohan 

--
You received this message because you are subscribed to a topic in the Google Groups "Caffe Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/caffe-users/zVH98Gl9REo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to caffe-users...@googlegroups.com.
To post to this group, send email to caffe...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/caffe-users/eb8cbcc7-05a7-4a24-b3a3-f917e25bd5a6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[Jan C Peters]

It may well be possible that the pycaffe interface is not thread-safe, but with multiple processes it sould not really be a problem. I have had several caffe processes (with GPU) running at the same time, and noticed no interferences (other than maybe every individual process taking a bit longer than when run without another process competing for the GPU). So I suggest you try to debug that somehow.

Jan

[Rohan]

Did you use the python multiprocessing module to run the processes? 

[Rohan]

Also I've switched up my model to use a producer-consumer framework, so the network is loaded in the worker (consumer) itself and images are passed to each other via python's multiprocessing queue. However, it is still unable to make the prediction. I've included sample code below

class Consumer(multiprocessing.Process):

    def __init__(self, task_queue, result_queue):
        multiprocessing.Process.__init__(self)
        self.task_queue = task_queue
        self.result_queue = result_queue
        #LOADING CAFFE NETWORK CONFIGS
        #omitted for brevity

    def run(self):
        while True:
            image = self.task_queue.get()
            prediction = net.predict([img])
        return

tasks = multiprocessing.Queue()
results = multiprocessing.Queue()
consumer = Consumer(tasks,results)
consumers.start

while True:
    #Grab image from stream
    tasks.put(image)
    #Do other stuff

[Jan C Peters]

I did use the multiprocessing module, but from the process I actually executed the caffe utility with a commandline like "caffe train -solver ..." instead of directly using the pycaffe API (I had my reasons to do it this way). All I can say for sure is that this way it worked.

Jan

[Rohan Bopardikar]

Did you use the subprocess module to do so?

--

You received this message because you are subscribed to a topic in the Google Groups "Caffe Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/caffe-users/zVH98Gl9REo/unsubscribe.
To unsubscribe from this group and all its topics, send an email to caffe-users...@googlegroups.com.
To post to this group, send email to caffe...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/caffe-users/a6f0a10f-120b-4d36-8982-7f87405d58ba%40googlegroups.com.

[Jan C Peters]

Actually you are right, I checked it and I did indeed use the subprocess module, but the whole story is more complicated: I used a flask-based webfrontend to control the definition and training of several networks, so the actual hierarchy of threads and processes being spawned there might be quite complicated, but I am sure I had more than one process running at a time that used caffe on the GPU for either training or testing. And I had an additional process using caffe to do supervision of the training and forward feeding of individual images (during training the network in another process). All in all similar to DIGITS I guess. Although not quite as flashy and userfriendly :-).

To summarize, I am pretty confident that caffe can be used concurrently or even in parallel by different processes.

To get to the root of your problem you need to find out where exactly your program crashes. And if its only by "print"-style debugging.

Jan

[Rohan]

I've done some print style debugging and my program (below) freezes on the line where the prediction is supposed to be made. There is no error, the process just does not get past that line. 

class Consumer(multiprocessing.Process):

    def __init__(self, task_queue, result_queue):
        multiprocessing.Process.__init__(self)
        self.task_queue = task_queue
        self.result_queue = result_queue
        #LOADING CAFFE NETWORK CONFIGS
        #omitted for brevity

    def run(self):
        while True:
            image = self.task_queue.get()
            prediction = net.predict([img])
        return

tasks = multiprocessing.Queue()
results = multiprocessing.Queue()
consumer = Consumer(tasks,results)
consumers.start

while True:
    #Grab image from stream
    tasks.put(image)
    #Do other stuff

The program hangs on the line:

prediction = net.predict([img]) 

It is unable to get past that.

To view this discussion on the web visit https://groups.google.com/d/msgid/caffe-users/c6108bf2-be92-4f6f-855f-41fe3cef4d7e%40googlegroups.com.

[Jan C Peters]

So the "image = ..." is not the one hanging? Since this one really waits for input from a different process. But if that is not the problem: try to break down the call net.predict. What is net anyway? Is it an instance of caffe.Net or this Classifier wrapper (or something else entirely)? And: Does everything work if you use CPU mode?

Jan

[Azamat Tokhtaev]

I had the same issue. So just before running detection you need to set that it is GPU mode again. 

caffe.set_mode_gpu()
caffe.set_device(0)  # set gpu_id 0

[Azamat Tokhtaev]

So a better example would be:

class Detection(object):
    def __init__():
        ... # get prototxt and model_path

        caffe.set_mode_gpu()

        caffe.set_device(0)  # set gpu_id 0

        self.net = caffe.Net(prototxt, model_path, caffe.TEST)
    def detect(self, data):
        

        caffe.set_mode_gpu()

        caffe.set_device(0)  # set gpu_id 0

        self.net.predict(...) # do the prediction  code