Some Questions to Rev 619

[augomat]

hi,

as suggested by Stephen Gilbert I'll post my questions here altough,
if I will really choose sparseUI some kind of "more online"-
communication medium might be an advantage.

Ok, so that's the thing. We are building a huge multi-touch screen
here at TU Graz, Austria and I'm looking around for some gesture-
recognition frameworks. So I checked out the head revision and startet
playing around with the stuff. Following the questions that arose (in
no significant order):

*) Demo Applikations:
Are there any feature descriptions for the demos? Some have kind of
obvious functionality (finger paint, photo organizer), some do kind of
strange things (tangrams, ..) and for now i had not had the time to
look through the code more intensively (altough guessing the idea of
code always is difficult).

*) CCServer.java: import
com.sun.xml.internal.ws.api.streaming.XMLStreamReaderFactory.Default;
is this import really needed?

*) Flick Gesture:
To my point of view, it doesn't really work until now. I must admit i
don't have a multi-touch table to try but with the TUIO Simulator
Pointer-Hook Applikation i was able to get those Dialogboxes whether
the flick was fast or slow, but i couldn't see the flick-effect taking
place in the application (the foto just stood still).
Am I right with this observation, and if yes, by when can this gesture
be expected?

*)Flick.java::270
There is some hardcoded stuff in there which crashes the server. Not
difficult to fix though, but i would suggest to avoid this form of
hardcoding, because its difficult to find and a bit annoying. The
approach taken in in picture-organizer (central definition of
hardcoded values) seems a lot friendlier to me for people trying to
get the source of the svn-repo working.

*) When trying some quick gestures, eventually multi-touch, i was able
to produce this exception:
[ClientConnection] Has GroupID: 0
[ClientConnection] Client did not claim point
[InputDeviceConnection] ID: 574522436 STATE: MOVE X: 520.26666 Y:
0.235
Exception in thread "Thread-0"
java.lang.ArrayIndexOutOfBoundsException: 62
at sparshui.server.InputDeviceConnection.readTouchPoint
(InputDeviceConnection.java:133)
at sparshui.server.InputDeviceConnection.readTouchPoints
(InputDeviceConnection.java:160)
at sparshui.server.InputDeviceConnection.receiveData
(InputDeviceConnection.java:114)
at sparshui.server.InputDeviceConnection.run
(InputDeviceConnection.java:186)
at java.lang.Thread.run(Unknown Source)

*) In Fingerpaint-App: When leaving the clear button on the top,
without unpressing (so without processDead-Event), this happens:
Pens cleared.
Exception in thread "AWT-EventQueue-0"
java.util.ConcurrentModificationException
at java.util.LinkedList$ListItr.checkForComodification(Unknown
Source)
at java.util.LinkedList$ListItr.next(Unknown Source)
at sparshui.demos.PaintArea.pensCleared(PaintArea.java:80)
at sparshui.demos.ClearButton.processMouseEvent
(ClearButton.java:38)
at java.awt.Component.processEvent(Unknown Source)
at java.awt.Container.processEvent(Unknown Source)
at java.awt.Component.dispatchEventImpl(Unknown Source)
at java.awt.Container.dispatchEventImpl(Unknown Source)
at java.awt.Component.dispatchEvent(Unknown Source)
at java.awt.LightweightDispatcher.retargetMouseEvent(Unknown
Source)
at java.awt.LightweightDispatcher.trackMouseEnterExit(Unknown
Source)
at java.awt.LightweightDispatcher.processMouseEvent(Unknown
Source)
at java.awt.LightweightDispatcher.dispatchEvent(Unknown
Source)
at java.awt.Container.dispatchEventImpl(Unknown Source)
at java.awt.Window.dispatchEventImpl(Unknown Source)
at java.awt.Component.dispatchEvent(Unknown Source)
at java.awt.EventQueue.dispatchEvent(Unknown Source)
at java.awt.EventDispatchThread.pumpOneEventForFilters(Unknown
Source)
at java.awt.EventDispatchThread.pumpEventsForFilter(Unknown
Source)
at java.awt.EventDispatchThread.pumpEventsForHierarchy(Unknown
Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.pumpEvents(Unknown Source)
at java.awt.EventDispatchThread.run(Unknown Source)
Pens cleared.
Strokes cleared.

*) Group.cpp
When the programm supports a gesture, but the server does not, the
server crashes because GestureFactory returns null. Crash happens on
group.cpp::106 when iterating over the gestures. This bug might be
covered by the first issue by jroltgen.

*) Screen Resolution Problem
In the release version of fingerpaint this works just fine but i got
very confused with the tangrams application because as i didn't like
the full-screen-mode (and for testing purposes) i set
Constants.ON_TOUCHTABLE to true. That shrank the application to
1024x768 but the TUIO messages are interpreted (i would say) in a
wrong way, as the application interprets them as if the application
would be fullscreen resulting in an increasing offset when moving more
to the right of the app/screen.
the same problem happens in the fingerpaint-app in the head-rev,
while, as stated above, the realease version of fingerpaint does not
suffer this bug.

*) Picture Organizer
Works quite well, but sometime (couldn't figure out when exaktely this
happens) i suppose that points somehow doesn't get deleted so that
instead of dragging around, a zoom/rotate gesture is recognized. Any
ideas how this can happen? Setup was SparseUI-Java-Server,
PicOrganizer in full-screen and the mouse-hook tuio-simulator. Try
just fiddeling around with the app, it always happend to me when i was
playing with the app for a while.

*) Gesture Ordering
I read the paper on the SVN. I definitely agree that implementing
order through inheritance is a very bad idea. But as i didn't have a
multi-touch table to try, i can't really classify the extend of the
missing gesture ordering.
In my point of view for a multi-touch gesture-recognition framework it
should be on of its *essential* parts not only recognizing parameters
supposing this or another gesture, but even more import seems the
dedection of which gesture the user is actually performing. this
obviously can't be a 0 or 1 decision but must take place on a
continuous scale.
but what i don't understand is: how can a static gesture ordering help
you with this? or am i missing the point?
and: you definitely have more experience with using your framework :)
to what extend is the gesture ordering/right gesture dedection-issue
really a problem? what must be done to reproduce erratic behavior in
the GUI-Application?

*) Projects State
Do you have a TODO-List, or a list of missing features or something
like that?

ok, that were the points for now, more to come maybe :)

best regards,
georg

[Jay]

Georg,

Thanks for all of the comments/suggestions! We've been a bit swamped
here recently, so sorry it took so long on this reply. Please keep us
informed of things that either encourage or discourage you from using
Sparsh-UI as a gesture recognition framework. Below are brief
responses to the things you brought up:

*) Demo applications
We don't really have a feature description for the demos right now -
something we're continually working on is better documentation.

*) Import
Probably left over from when we were working on an xml-based messaging
protocol, no longer needed.

*) Flick
Should be a commit coming soon from one of our developers that fixes
issues with the flick gesture

*) When trying some quick gestures, eventually multi-touch, i was able
to produce this exception:

This seems to be an issue with the input device. Which device driver
(or custom) are you using?

*) Group.cpp
Bug will be fixed in commit coming in with flick gesture.

*) Screen Resolution Problem
The ON_TOUCHTABLE flag is reserved specifically for our use with one
of our multitouch devices, our homebrew multitouch table. The flag
allows it to work with our particular situation, not really for use
outside that.

*) Picture Organizer
We've experienced the same bug you've detected. We haven't had time
to figure out where it's coming from - suggestions or changes would be
welcome at this point.

*) Project State
As a research group, we've been sidetracked a bit from active Sparsh-
UI development, though development continues as required by our other
projects that use Sparsh-UI. At this point, we don't have a TODO list
for Sparsh-UI, but are always working on adding more gestures. We are
also currently cooking up some extensions to Sparsh-UI that should
make it rather more awesome than it already is :)

Jay

[augomat]

hi!

Thanks for the reply, as you see it also took me a while.

*) Demo apps
Better documentation...well...who does not work on this? :)
But actually it would be *really* nice to find a short readme in
every /apps-subdirectory. this shouldn't take more than 15 minutes and
it would clarify many things. As I said, trying to guess the intention
of an application that you are not really able to use by just looking
at the code is somewhat difficult and time-consuming.

*) Quick-Gesture (Device?)-Bug
I was using the provided touchlib-adapter.

*) Screen Resolution
I understand that i may not modify this variable, but i hope i could
make clear that this seems to be a bug as it works with the
fingerpaint release-version (the one that is not on svn).

*) Compiler-Error C++-Version
Trying to compile SparseUI C++ Verision with the current SVNRepo
(rev627) gives a compiler-error about a missing debug_msg(). This is
quite easy to fix but you might update the debug.h as it seems that
there is the problem.

*) PhotoApp: Flick Gesture
Flick Gestures now maybe work but I wasn't able to test it with the
photoapp as the FlickEvent do not get processed by now. Where or how
can a test the gesture?

*) PhotoApp: Only Drag-Gesture
When only enabling the MultiPointDragGesture but doing a zoom gesture
the picture does not increase in size (as excepted) but the mouse-
movment is then applied as if the picture was bigger. just try to see
what i mean.

*) Forgetting to delete a Point-Bug
Also experienced in Tangrams-App. There it is clearly visible as you
see the touch-points on the screen. it should be so hard to find out
why the point doesn't get deleted then. (but i was kind of suprised
when playing around one of the triangles turned suddenly red :>, see
*) Demo Apps =P)

*) Group.cpp
Can't see any changes to the C++-SourceTree as of rev592 initial
import so i suppose that didn't happen by now.

*) Extensions
I'd be very interested in this as I'm currently writing an extension
too. What type of extensions will you implement? And will their
implementation affect the architecture of the framework?

*) Gesture Ordering
Unfortunately no answer to this topic was provided.


best regards,
georg

[augomat]

Another thing:

*) Eventtype.h
There are two, in /common and in /common/events. i obviousely updated
the wrong one so one should be deleted.

- g

> ...
>
> Erfahren Sie mehr »

[Jay Roltgen]

Georg,

Since you're so engaged in the development of the Sparsh-UI project, we would like to make you a contributor.  We would really appreciate your help improving Sparsh-UI.  We have created a branch for you in the repository and given you a user account on the googlecode.  Your branch is in branches/Georg.  Please make changes as you feel necessary inside of that branch, and we'll discuss merging your changes back into the main trunk as we review them.

Thanks for your help!

Jay
Virtual Reality Applications Center
Iowa State University