Hi I was playing with the option to implement SVM in my OCR algorithm, but when checking the SVM that Matlab has to offer I saw that it can support only 2 groups training and classification ? Am I am missing something ?
I was using the svmtrain command, and tried to train it for 3 groups (e.g. three characters.) and I got the error saying that svm train support only two groups...
Am I am missing something in the SVM implementation ? or it is really that Matlab support only two groups, and when you want to implement something bigger then that, like ocr for 10 characters (e.g. 10 groups) , matlab doesn't support that (at least in it's ready made tool box).
Eizo <odpe...@gmail.com> wrote in message <4a19b764-d8e6-40a2-b399-c2d6167f2...@d10g2000yqh.googlegroups.com>... > Am I am missing something ?
SVM is a two-class supervised learning technique, and Matlab does just that. For multi-classes you might want to apply successively 2-classes learning processes. See relevant papers for how to use correct strategy ("winner takes all" is the most popular).
There are several generalization of SVM allowing multiclasse classification, but I don't think Matlab has any of those technique.
"Bruno Luong" <b.lu...@fogale.findmycountry> wrote in message <hgdv9c$o9...@fred.mathworks.com>... > Eizo <odpe...@gmail.com> wrote in message <4a19b764-d8e6-40a2-b399-c2d6167f2...@d10g2000yqh.googlegroups.com>...
> > Am I am missing something ?
> SVM is a two-class supervised learning technique, and Matlab does just that. For multi-classes you might want to apply successively 2-classes learning processes. See relevant papers for how to use correct strategy ("winner takes all" is the most popular).
> There are several generalization of SVM allowing multiclasse classification, but I don't think Matlab has any of those technique.
> Bruno
At least the SVM in the Bioinformatics toolbox does only two-class classification. There are other tools available. LIBSVM for example supports multi-class and has MATLAB interface http://www.csie.ntu.edu.tw/~cjlin/libsvm/
On Dec 17, 9:03 pm, "Bruno Luong" <b.lu...@fogale.findmycountry> wrote:
> Eizo <odpe...@gmail.com> wrote in message <4a19b764-d8e6-40a2-b399-c2d6167f2...@d10g2000yqh.googlegroups.com>... > > Am I am missing something ?
> SVM is a two-class supervised learning technique, and Matlab does just that. For multi-classes you might want to apply successively 2-classes learning processes. See relevant papers for how to use correct strategy ("winner takes all" is the most popular).
> There are several generalization of SVM allowing multiclasse classification, but I don't think Matlab has any of those technique.
> Bruno
Then how people use SVM for face recognition or OCR, if it is just 2 class learning technique? (They use the multi class trick ? )
On Dec 17, 9:03 pm, "Bruno Luong" <b.lu...@fogale.findmycountry> wrote:
> Eizo <odpe...@gmail.com> wrote in message <4a19b764-d8e6-40a2-b399-c2d6167f2...@d10g2000yqh.googlegroups.com>... > > Am I am missing something ?
> SVM is a two-class supervised learning technique, and Matlab does just that. For multi-classes you might want to apply successively 2-classes learning processes. See relevant papers for how to use correct strategy ("winner takes all" is the most popular).
> There are several generalization of SVM allowing multiclasse classification, but I don't think Matlab has any of those technique.
> Bruno
Where can I read on the Multi-class methods? .. maybe you know on some links to some tutorials on that matter ? All the papers and presentation I read deal only with 2 classes..
> Eizo <odpe...@gmail.com> wrote in message > <4a19b764-d8e6-40a2-b399-c2d6167f2...@d10g2000yqh.googlegroups.com>... > > Am I am missing something ?
> SVM is a two-class supervised learning technique, and Matlab does just > that. For multi-classes you might want to apply successively 2-classes > learning processes. See relevant papers for how to use correct strategy > ("winner takes all" is the most popular).
> There are several generalization of SVM allowing multiclasse > classification, but I don't think Matlab has any of those technique.
> Bruno
Where can I read on the Multi-class methods? .. maybe you know on some links to some tutorials on that matter ? All the papers and presentation I read deal only with 2 classes..
Allwein, Schapire and Singer "Reducing multiclass to binary: a unifying approach for margin classifiers", J. of Machine Learning Research 1 (2000) 113-141
Kuncheva "Using diversity measures for generating error-correcting output codes in classifier ensembles", Pattern Recognition Letters 26 (2005) 83-90; also references therein
Lee, Lin and Wahba "Multicategory support vector machines: theory and application to the classification of microarray data and satellite radiance data", J. of ASA 99 (2004)
- Analysis of Multiclass Support Vector Machines, Shigeo Abe
- Winner takes all -> B. Schölkopf, C. Burges, and V. Vapnik. Extracting support data for a given task. In KDD'95, 1995.
- One againsts one -> S. Knerr, L. Personnaz, and G. Dreyfus. Single-layer learning revisited : A stepwise procedure for building and training a neural network. In F. Fogelman-Soulié and J. Hérault, editors, Neurocomputing: Algorithms, Architectures and Applications, volume F68 of NATO ASI Series, pages 41-50. Springer-Verlag, 1990.
- Decision graph: J.C. Platt, N. Cristianini, and J. Shawe-Taylor. Large margin DAGs for multiclass classi cation. In NIPS 12, pages 547-553, 2000.
On Dec 18, 9:01 pm, "Bruno Luong" <b.lu...@fogale.findmycountry> wrote:
> - Analysis of Multiclass Support Vector Machines, Shigeo Abe
> - Winner takes all -> B. Schölkopf, C. Burges, and V. Vapnik. Extracting support data for a given task. In KDD'95, 1995.
> - One againsts one -> S. Knerr, L. Personnaz, and G. Dreyfus. Single-layer learning revisited : A stepwise procedure for building and training a neural network. In F. Fogelman-Soulié and J. Hérault, editors, Neurocomputing: Algorithms, Architectures and Applications, volume F68 of NATO ASI Series, pages 41-50. Springer-Verlag, 1990.
> - Decision graph: J.C. Platt, N. Cristianini, and J. Shawe-Taylor. Large margin DAGs for multiclass classi cation. In NIPS 12, pages 547-553, 2000.
> Bruno
While exploring the net for some Matlab code that can do multiclass SVM. I was wondering, is it possible to train the SVM with a character (e.g one vector that contain binary letter information, with 1's and 0's) like in the neural net training ?
Because all the demos, and all the articles I read deal with 2 dim info. While it is possible to find some features that describe letters in the 2 dim space, I was looking to speed things up, by entering the letter as is in its raw binary info (like neural net).
