Gestures Recognition Engine for Capital Letters

[Scott Ferguson]

This gestures recognition engine interprets one or more drags on the canvas as a letter of the alphabet.

This requires one or two stages of analysis.

The first stage interprets the strokes that make up a letter.

The Strokes pattern for building each letter is indicated with the following image:

https://sporadichappiness.files.wordpress.com/2012/03/capital_1.gif?w=630

The starting and ending points are examined for each stroke by the 'Stroke' procedure which returns a number representing whether the the x or y value increased (1), decreased (-1) or there was no significant change (0).

There are 9 possible outcomes from a stroke event as seen in the PATTERNS diagram below:

(-1,-1) (0,-1) (1,-1)

(-1, 0) (0, 0) (1, 0)

(-1, 1) (0, 1) (1, 1)

Characters that can be interpreted this way are: A B E F G H K L M N O Q R T U W Y Z

Each of these strokes may draw either a straight or curved line from the starting and ending points. 

Only the stroke ordered pairs are required to interpret the above characters.

Any curves are ignored.

The gesture for A is to drag down and to the left (-1 1), down and to the right (1 1) then from left to right (1 0).

The gesture for O is (0 0) as the starting and ending values are approximately the same.

Gestures for the characters are as follows:

A: ((-1 1) (1 1) (1 0)) -- down+left, down+right, right

B: ((0 1) (0 1) (0 1))  -- down, down, down (only start and end points are recorded)

C: ((0 1))*  -- down

D: ((0 1) (0 1))* -- down, down

E: ((0 1) (1 0) (1 0) (1 0)) -- down, right, right, right

F: ((0 1) (1 0) (1 0)) -- down, right, right

G: ((0 1) (1 1)) -- down, down+right

H: ((0 1) (0 1) (1 0)) -- down, down, right

I: ((0 1) (1 0) (1 0))* -- down, right, right

J: ((-1 1))* -- down+left

K: ((0 1) (-1 1) (1 1)) -- down, down+left, down+right

L: ((0 1) (1 0)) -- down, right

M: ((0 1) (1 1) (-1 1) (0 1)) -- down, down_right, down+left, down

N: ((0 1) (1 1) (0 1)) -- down, down+right, down

O: ((0 0)) -- touched

P: ((0 1) (0 1))* -- down, down

Q: ((0 0) (1 1)) -- touched, down+right

R: ((0 1) (0 1) (1 1)) -- down, down, down+right

S: ((-1 1))* -- down+left

T: ((1 0) (0 1)) -- down, right 

U: ((1 0)) -- right

V: ((1 1) (-1 1))* -- down+right, down+left

W: ((1 1) (-1 1) (1 1) (-1 1)) -- down+right, down+left, down+right, down+left

X: ((1 1) (-1 1))* -- down+right, down+left

Y: ((1 1) (-1 1) (0 1)) -- down+right, down+left, down

Z: ((1 0) (-1 1) (1 0)) -- right, down+left, right

Where there is an '*' above, there is more than one letter that matches the pattern.

The following letters require additional tests:

D (EndY for stroke 1 - EndY for stroke 2 < MaximumStartEndPointsDifference) vs. P

V absolute value of (EndX for stroke 1 - EndX for stroke 2) < MaximumStartEndPointsDifference vs. X

C (MinY < BeginY) vs. I

S (MinY < BeginY) vs. J

In order to determine if a gesture is completed, a clock timer looks at the time interval between strokes.

If less than DelayBetweenGestures milliseconds has elapsed between strokes, then the gesture is being built.

If more than DelayBetweenGestures milliseconds has elapsed between strokes, then the gesture is complete and can be analysed.

---

recogizing patterns by timed sampling:

http://stackoverflow.com/questions/20821358/gesture-detection-algorithm-based-on-discrete-points

The following gestures were drawn with a finger and the interpreted letter displayed above the canvas.

Most drawings are interpreted correctly.

W should be drawn stretched out a bit to be interpreted correctly.

M should be drawn vith vertical sides, not like an inverted W.

[Taifun]

very nice! keep up the good work!

Taifun

[Bill Cooke]

Stimulating!

 As an engine which allows us to write, it is interesting.

 As an engine which constructs / selects responses to any incoming stimulation, it proposes an efficient technique for any gesturing a quick selection of response.  A large set of applications share this characteristic.  (Thumbs up / Thumbs down could be done by buttons, but ...  )

[Scott Ferguson]

In reality, it is a rather crude hack in that it requires that the strokes be in a specific order and it ignores arcs so can easily be tripped up.

And it is not analysing the drawn image at all.

But it does seem to work OK if you follow the rules :-)

Thanks for the + feedback, guys!

---