How can I train SVM in Matlab, with svmtrain command, but for more then 2 groups??
Eizo
I was playing with the option to implement SVM in my OCR algorithm,
but when checking the SVM that Matlab has to offer I saw that it can
support only 2 groups training and classification ?
Am I am missing something ?
I was using the svmtrain command, and tried to train it for 3 groups
(e.g. three characters.) and I got the error saying that svm train
support only two groups...
Am I am missing something in the SVM implementation ? or it is really
that Matlab support only two groups, and when you want to implement
something bigger then that, like ocr for 10 characters (e.g. 10
groups) , matlab doesn't support that (at least in it's ready made
tool box).
Thanks
O
Bruno Luong
> Am I am missing something ?
SVM is a two-class supervised learning technique, and Matlab does just that. For multi-classes you might want to apply successively 2-classes learning processes. See relevant papers for how to use correct strategy ("winner takes all" is the most popular).
There are several generalization of SVM allowing multiclasse classification, but I don't think Matlab has any of those technique.
Bruno
Pekka Kumpulainen
At least the SVM in the Bioinformatics toolbox does only two-class classification.
There are other tools available. LIBSVM for example supports multi-class and has MATLAB interface
http://www.csie.ntu.edu.tw/~cjlin/libsvm/
Eizo
wrote:
> Eizo <odpe...@gmail.com> wrote in message <4a19b764-d8e6-40a2-b399-c2d6167f2...@d10g2000yqh.googlegroups.com>...
> > Am I am missing something ?
>
> SVM is a two-class supervised learning technique, and Matlab does just that. For multi-classes you might want to apply successively 2-classes learning processes. See relevant papers for how to use correct strategy ("winner takes all" is the most popular).
>
> There are several generalization of SVM allowing multiclasse classification, but I don't think Matlab has any of those technique.
>
> Bruno
Then how people use SVM for face recognition or OCR, if it is just 2
class learning technique?
(They use the multi class trick ? )
op
Eizo
wrote:
> > Am I am missing something ?
>
> SVM is a two-class supervised learning technique, and Matlab does just that. For multi-classes you might want to apply successively 2-classes learning processes. See relevant papers for how to use correct strategy ("winner takes all" is the most popular).
>
> There are several generalization of SVM allowing multiclasse classification, but I don't think Matlab has any of those technique.
>
> Bruno
Where can I read on the Multi-class methods? .. maybe you know on
some links to some tutorials on that matter ?
All the papers and presentation I read deal only with 2 classes..
Ilya Narsky
Allwein, Schapire and Singer "Reducing multiclass to binary: a unifying
approach for margin classifiers", J. of Machine Learning Research 1 (2000)
113-141
Kuncheva "Using diversity measures for generating error-correcting output
codes in classifier ensembles", Pattern Recognition Letters 26 (2005) 83-90;
also references therein
Tibshirani and Hastie "Margin trees for high-dimensional classification",
available from http://www-stat.stanford.edu/~hastie/
For SVM:
Lee, Lin and Wahba "Multicategory support vector machines: theory and
application to the classification of microarray data and satellite radiance
data", J. of ASA 99 (2004)
Bruno Luong
- Winner takes all -> B. Schölkopf, C. Burges, and V. Vapnik. Extracting support data for a given task. In KDD'95, 1995.
- One againsts one -> S. Knerr, L. Personnaz, and G. Dreyfus. Single-layer learning revisited : A stepwise procedure for building and training a neural network. In F. Fogelman-Soulié and J. Hérault, editors, Neurocomputing: Algorithms, Architectures and Applications, volume F68 of NATO ASI Series, pages 41-50. Springer-Verlag, 1990.
- Decision graph: J.C. Platt, N. Cristianini, and J. Shawe-Taylor. Large margin DAGs for multiclass classi cation. In NIPS 12, pages 547-553, 2000.
Bruno
Eizo
wrote:
While exploring the net for some Matlab code that can do multiclass
SVM. I was wondering, is it possible to train the SVM with a character
(e.g one vector that contain binary letter information, with 1's and
0's) like in the neural net training ?
Because all the demos, and all the articles I read deal with 2 dim
info.
While it is possible to find some features that describe letters in
the 2 dim space, I was looking to speed things up, by entering the
letter as is in its raw binary info (like neural net).
Thanks.
rosha...@raisoni.net
Alan_Weiss
there are other Statistics Toolbox functions that support that support
multiple classes, such as discriminant analysis
http://www.mathworks.com/help/stats/discriminant-analysis.html
classification trees
http://www.mathworks.com/help/stats/classification-trees-and-regression-trees-1.html
nearest neighbor
http://www.mathworks.com/help/stats/classification-using-nearest-neighbors-1.html
and ensemble methods
http://www.mathworks.com/help/stats/ensemble-methods.html#btgalua
Alan Weiss
MATLAB mathematical toolbox documentation