[tegaki] SVM, zinnia and ideas

[Ian Johnson]

Hello all,

So I've just done a little presentation in my class about using LIBSVM
to attempt Chinese handwriting recognition, of course using many of
the tools available in the tegaki project! (tegaki-train and the
CharacterCollection class were incredibly useful)
Now that I'm a little bit more familiar with the SVM technique and
using LIBSVM I'm left with more questions and ideas than answers. I'm
wondering first off if anyone could give a high level overview of what
zinnia uses for feature vectors in its use of SVM? I'm still a bit of
a C++ newb, and it looks like a custom implementation of SVM as well.

As for what I tried, its a naive almost brute force approach where I
rendered samples as images (I started with 100x100) and took the data
array as my feature vector (with the color values scaled to [0,1] ).
At first I was playing with just the 10 Chinese numerals and
recognition seemed good, so I tried with the tomoe "light" data set of
6000+ characters. LIBSVM generated a 1GB training set and a 400MB
(text) model file which took about 1.5 hours to train. Of course these
are 10,000 element feature vectors. After that I realized that
recognition was not good for anything more complicated than "一". I was
only using one sample per character class.

Now, the thing is I would like to continue trying this "offline"
approach because I feel it could be useful in some applications. My
first thought is to try again with the 10 characters and use many
samples. I would also do some cross-validation to look for optimal
paramaters (where are these set in zinnia, and are they modified with
different trained sets?). Assuming this gives results I then have the
fun task of trying to deal with the file sizes. I recognize that
zinnia uses binary storage for the model file, and I haven't even
looked if LIBSVM supports this.

[Mathieu Blondel]

On Tue, Dec 1, 2009 at 5:25 AM, Ian Johnson <enj...@gmail.com> wrote:
> So I've just done a little presentation in my class about using LIBSVM
> to attempt Chinese handwriting recognition, of course using many of
> the tools available in the tegaki project! (tegaki-train and the
> CharacterCollection class were incredibly useful)

Great. Hopefully more and more students and researchers will use
Tegaki as the framework for their research.

> Now that I'm a little bit more familiar with the SVM technique and
> using LIBSVM I'm left with more questions and ideas than answers. I'm
> wondering first off if anyone could give a high level overview of what
> zinnia uses for feature vectors in its use of SVM?

Have you seen this thread?
http://groups.google.com/group/tegaki-hwr/browse_thread/thread/2dc2da1f7c38db47

One thing I don't fully understand in zinnia yet is the feature vector
size. There exists SVM kernels to deal with more complex objects like
sequences, trees and graphs but zinnia uses a simple linear kernel
which means that the vector size should be fixed. But a character has
a variable number of strokes so I wonder how a character is turned
into a fixed size vector.

> a C++ newb, and it looks like a custom implementation of SVM as well.

scipy has a SVM package that uses libsvm. You can store your 100x100
matrices as numpy arrays.

> As for what I tried, its a naive almost brute force approach where I
> rendered samples as images (I started with 100x100) and took the data
> array as my feature vector (with the color values scaled to [0,1] ).
> At first I was playing with just the 10 Chinese numerals and
> recognition seemed good, so I tried with the tomoe "light" data set of
> 6000+ characters. LIBSVM generated a 1GB training set and a 400MB
> (text) model file which took about 1.5 hours to train. Of course these
> are 10,000 element feature vectors. After that I realized that
> recognition was not good for anything more complicated than "一". I was
> only using one sample per character class.

Yeah just using the pixels won't get you anywhere... I've been doing
online handwriting recognition only so far so I'm not very familiar
with the features they use in offline recognition. Have you done a
literature review? I saw a few papers about offline handwritten
Chinese character with SVM.

I had a discussion with a friend doing offline handwriting recognition
once. If I remember correctly, one simple feature extraction he
suggested was to use a sliding window, where the height of the window
is 100 (your image height) and the width is a parameter (let's say
10). Then you can scan your image from left to right with a step size.
If the step size is 10, it means that there's no overlap. If the step
is say 5, then there is overlap between a window and the next window.
If the step size is 5, you end up with 20 windows. Your feature vector
is then a 20-element vector where each element is the black pixel
count in the window. I have no idea if that works well though.
Likewise, you could try to divide your image into blocks (for example
5x5 blocks). Both approaches will greatly reduce the dimension of your
vectors.

Since we're dealing with a great number of character classes, mixing
online and offline can be a good way to provide more information to
the decoder.

> Now, the thing is I would like to continue trying this "offline"
> approach because I feel it could be useful in some applications.

I think so too.

Mathieu