what is the caffe.Net's definition and the difference between deploy.prototxt and train_val.prototxt
2,549 views
Skip to first unread message
Yin Zheng
Sep 10, 2015, 12:17:58 PM9/10/15
to Caffe Users
Hi Guys,
I am new to Caffe. When I am learning the python interface of caffe (http://nbviewer.ipython.org/github/BVLC/caffe/blob/master/examples/00-classification.ipynb ) I fount caffe.Net() is a very important class, which, I guess, is used to build/load the model.
But I did not find the reference of the class initialization. In the example of http://nbviewer.ipython.org/github/BVLC/caffe/blob/master/examples/00-classification.ipynb , this class is initialized as
net = caffe.Net(caffe_root + 'models/bvlc_reference_caffenet/deploy.prototxt',
caffe_root + 'models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel',
caffe.TEST)
It seems that the first parameter is the model definition, the second parameter is the pretrained model, and the 3rd third parameter is used to specified the test phrase. I am not sure whether I am right and I wonder whether there is some reference about this class? What if the definition of the model does not match the pretrained model?
And another question is in Caffe zoo, it usually provide a train_val.prototxt and a deploy.prototxt. For example, in https://github.com/BVLC/caffe/tree/master/models/bvlc_reference_caffenet there are both train_val.prototxt and deploy.prototxt. But the example use the deploy.prototxt in the above example. Same thing happens in https://github.com/BVLC/caffe/tree/master/models/bvlc_googlenet
My 2nd question is what is the difference between deploy.prototxt and train_val.prototxt?
Any help is appreciated! Thanks in advance!
Best wishes,
Yin
Dhawal Arora
Feb 5, 2016, 11:05:46 AM2/5/16
to Caffe Users
Hi,
Did you get the information to this? I am also relatively new to caffe. So having a hard time figuring the need of train_val. For data http://vision.stanford.edu/teaching/cs231n/slides/caffe_tutorial.pdf
this tutorial shows that there's Datalayer.prototxt and deploy.prototxt and solver.prototxt solve the rest of the purpose. So i'm not sure for train_val's purpose. This tells https://groups.google.com/forum/#!topic/caffe-users/v7XlpZO0SW4 its used to add the input data and the output loss to deploy.prototxt. Still not sure though. Please do inform if you get the confirmed solution.
Thanks.
Cheers.
Dhawal Arora
Did you get the information to this? I am also relatively new to caffe. So having a hard time figuring the need of train_val. For data http://vision.stanford.edu/teaching/cs231n/slides/caffe_tutorial.pdf
this tutorial shows that there's Datalayer.prototxt and deploy.prototxt and solver.prototxt solve the rest of the purpose. So i'm not sure for train_val's purpose. This tells https://groups.google.com/forum/#!topic/caffe-users/v7XlpZO0SW4 its used to add the input data and the output loss to deploy.prototxt. Still not sure though. Please do inform if you get the confirmed solution.
Thanks.
Cheers.
Dhawal Arora
Jan C Peters
Feb 8, 2016, 3:29:59 AM2/8/16
to Caffe Users
Hi Yin,
pycaffe docs:
It seems there is no real documentation other than example code. But you already figured out the most important bits: If you know how to use the net class (usually loading, forwarding and inspecting the blobs is enough) and the blob class, that should be enough to do the usual stuff (looking at filters, computed features, predictions, etc.). If you also want to train via pycaffe you should take a look at the Solver class, too. To get an idea what pycaffe can do and what classes it defines, take a look at the code: https://github.com/BVLC/caffe/blob/master/python/caffe/_caffe.cpp (the lower half is most interesting). It basically just provides a python wrapper to the C++ API. If details are unclear you can look at the code to figure it out.
deploy vs normal network definition:
I know there is sparse information on this in the official docs, but here on this list you should find several answers to this very question, I have written several myself. Basically a "deploy" config is almost the same as the config used for training, but without any input layers. Instead it contains a definition of the input blobs (names and sizes, see protobuf properties "input" and "input_shape"). The intention is to fill these input blobs through the API (C++ or python) before feeding through the network. This is a good method to classify/predict single images or data in memory without needing to put it into a database first. If you use the normal training config of the network with this interface, you can still write to the input blobs through the API, but the net.forward() call will overwrite your data with the actual input loaded by your data layers. This is the (only) reason there is this deploy config.
Jan
P.S. Your understanding of how the caffe.Net constructor is to be used is correct: first arg is deploy config of the network, second arg is snapshot of trained network, third is caffe.TRAIN or caffe.TEST, to indicate which mode you intend to use the network in.
pycaffe docs:
It seems there is no real documentation other than example code. But you already figured out the most important bits: If you know how to use the net class (usually loading, forwarding and inspecting the blobs is enough) and the blob class, that should be enough to do the usual stuff (looking at filters, computed features, predictions, etc.). If you also want to train via pycaffe you should take a look at the Solver class, too. To get an idea what pycaffe can do and what classes it defines, take a look at the code: https://github.com/BVLC/caffe/blob/master/python/caffe/_caffe.cpp (the lower half is most interesting). It basically just provides a python wrapper to the C++ API. If details are unclear you can look at the code to figure it out.
deploy vs normal network definition:
I know there is sparse information on this in the official docs, but here on this list you should find several answers to this very question, I have written several myself. Basically a "deploy" config is almost the same as the config used for training, but without any input layers. Instead it contains a definition of the input blobs (names and sizes, see protobuf properties "input" and "input_shape"). The intention is to fill these input blobs through the API (C++ or python) before feeding through the network. This is a good method to classify/predict single images or data in memory without needing to put it into a database first. If you use the normal training config of the network with this interface, you can still write to the input blobs through the API, but the net.forward() call will overwrite your data with the actual input loaded by your data layers. This is the (only) reason there is this deploy config.
Jan
P.S. Your understanding of how the caffe.Net constructor is to be used is correct: first arg is deploy config of the network, second arg is snapshot of trained network, third is caffe.TRAIN or caffe.TEST, to indicate which mode you intend to use the network in.
Jan C Peters
Feb 8, 2016, 3:40:30 AM2/8/16
to Caffe Users
(I forgot to add)
Regarding your question "what if the model does not match the network config":
When you call the Net constructor in python, the network is in any built according to the prototxt config. When the *.caffemodel file is loaded, for every layer specified in that file it tries to match it to one of the layers of the prototxt-defined network (layers are matched by name only). If the shapes of the parameters also match, the weights/biases/etc. is just copied over; if it does not, you should get an error message about a shape mismatch. If a layer in the prototxt is not present in the caffemodel, it gets initialized as specified in the prototxt (usually with some kind of random values). Every layer in the caffemodel that is not found in the prototxt is just ignored.
You can use this knowledge to only fill speciic layers with pretrained weights for fine-tuning for example: just rename the layers you want initialized randomly and keep the names of the layers you want to be filled with pretrained weights.
Jan
Regarding your question "what if the model does not match the network config":
When you call the Net constructor in python, the network is in any built according to the prototxt config. When the *.caffemodel file is loaded, for every layer specified in that file it tries to match it to one of the layers of the prototxt-defined network (layers are matched by name only). If the shapes of the parameters also match, the weights/biases/etc. is just copied over; if it does not, you should get an error message about a shape mismatch. If a layer in the prototxt is not present in the caffemodel, it gets initialized as specified in the prototxt (usually with some kind of random values). Every layer in the caffemodel that is not found in the prototxt is just ignored.
You can use this knowledge to only fill speciic layers with pretrained weights for fine-tuning for example: just rename the layers you want initialized randomly and keep the names of the layers you want to be filled with pretrained weights.
Jan
Am Donnerstag, 10. September 2015 18:17:58 UTC+2 schrieb Yin Zheng:
Yin Zheng
Feb 8, 2016, 5:55:26 AM2/8/16
to Jan C Peters, Caffe Users
Hi Jan,
Thanks for the information! It is really appreciated!
Best wishes,
Yin
--
You received this message because you are subscribed to a topic in the Google Groups "Caffe Users" group.
To unsubscribe from this topic, visit https://groups.google.com/d/topic/caffe-users/eCygJO2LG30/unsubscribe.
To unsubscribe from this group and all its topics, send an email to caffe-users...@googlegroups.com.
To post to this group, send email to caffe...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/caffe-users/982c6005-6049-4b3e-83fe-ed013176f8e3%40googlegroups.com.
------------------------------------------------------------------------------------------------------
Wir mussen wissen, Wir werden wissen.
——David Hilbert(1862～1943)
Shawn Liu
Jun 9, 2016, 5:57:44 AM6/9/16
to Caffe Users
Thanks for the info! I was using a train_val.prototxt for predictions and got really bad results. I guess I should be using the deploy.prototxt!
Regards,
Siyuan
chandan v
Feb 7, 2018, 5:32:08 AM2/7/18
to Caffe Users
My doubt is during the inference when this deploy.prototxt is used, How will it know the format in which the output has to be given?
For example if we take KITTI dataset trained model for inference, how will this model get to know the format (eg : bounding box order) in which the inference has to be made?
Does the .caffemodel file has the knowledge about it? Or is there something else that determines the format of the output.
FYI
KITTI dataset format :
Reply all
Reply to author
Forward
0 new messages