Fwd: Jmol and SparshUI
3 views
Skip to first unread message
Robert Hanson
Nov 29, 2009, 6:09:28 PM11/29/09
to spar...@googlegroups.com
---------- Forwarded message ----------
From: Robert Hanson <han...@stolaf.edu>
Date: Sun, Nov 29, 2009 at 4:53 PM
Subject: Jmol and SparshUI
To: spar...@googlegroups.com
Cc: Prasad RS <pras...@iastate.edu>, Jay Roltgen <jrol...@gmail.com>, Stephen Gilbert <gil...@iastate.edu>, Jmol Developers <jmol-de...@lists.sourceforge.net>
Stephen, Jay, Prasad, and friends,
Jmol (http://jmol.sourceforge.net, http://chemapps.stolaf.edu/jmol/) is an open-source (LGPL) Java-based molecular visualization tool that can be utilized as an applet, a stand-alone application, or as a component of a larger program.
This weekend I successfully integrated SparshUI into Jmol. (Still waiting for a machine to test this on, but I included an adapted TouchSimulator, so at least it is working in that mode.) I'm sure there are plenty of fixes to do, but I am very pleased with how this looks, and in case anyone out there is interested in multi-touch molecules, this should do the trick.
I'm sure in this message you will detect my naivete in this multitouch area. I've just gotten going on it; bear with me!
The code for this integration can be found at the sourceforge site http://jmol.svn.sourceforge.net/viewvc/jmol/trunk/Jmol/ and, of course, is available for anonymous checkout by interested parties using Eclipse/SVN. The main classes and packages involved include:
http://jmol.svn.sourceforge.net/viewvc/jmol/trunk/Jmol/src/org/jmol/viewer/ActionManagerMT.java
This class controls all the mouse and keyboard action of Jmol. This particular subclass adds the multitouch capability.
http://jmol.svn.sourceforge.net/viewvc/jmol/trunk/Jmol/src/com/sparshui/
The package with the adapted SparshUI code.
http://jmol.svn.sourceforge.net/viewvc/jmol/trunk/Jmol/src/org/jmol/multitouch/sparshui/
The package with my client adapter and custom Gesture class.
Don't expect this code to work with actual multitouch hardware yet -- this is just using a mouse and touch simulation. I'm still awaiting my HP IQ500 machine for that. (And I'm rather pessimistic now that we can actually get this going properly with that -- we'll see. I'm not happy with what I read in the HP TouchSmart SDK about the messages it sends.)
Stephen, do you think it will be possible to use SparshUI effectively on the HP? Have you tried that? (I'm worried that we can't trap events, and we'll see scrolling of the browser window, for example, when a user tries to scroll within the applet. (Currently the mouseWheel event must be consumed to avoid this.) I'm not sure how the SparshUI business allows for consuming window messages so they don't go through to a lower layer. I've asked for Windows 7 on the machine I get, but I don't know how well that will work or even if that is the right direction to go.
There were several nuances that I needed to solve to effect this integration. I thought you might be interested:
-- client-side definition of gesture classes. I saw this was in the plans, but I didn't see a solution there already. I accomplished this by adapting the GestureFactory to allow for class names and use of Class.forName(name).newInstance(). Then, instead of just delivering SparshUI gesture ordinals, I adapted the interface to allow for delivery of class names (Strings). I simply use negative numbers instead of ordinals for the "gestureID" and then use that number to indicate the length of the byte-encoded string that follows. So I now have a "TwoPointGesture" class that I can use. Works like a charm! (More on this below.)
-- multiple window targets. Since applets and Java applications could have any number of instances, I simply gave each client its own "groupID". Actually, in Jmol I allow for 256 "sub-groupIDs" that an applet or application could in principal use like normal SparshUI groupIDs. This is in case I can't figure out how to get true Windows multitouch messages to my application. I don't think this is working properly quite yet.
--special handling flags. I was probably just doing something wrong, but I was having trouble with the gesture classes not resetting properly, so I added a mask to the groupID sent back to the server. It basically looks like this in clientConnection.ProcessBirth:
int groupID = getGroupID(touchPoint.getLocation());
int jmolFlags = (groupID & 0xF0000000);
if (jmolFlags != 0) {
switch (jmolFlags) {
case 0x10000000:
// reset flag
_groups = new HashMap();
break;
}
groupID &= ~jmolFlags;
}
Maybe this wasn't necessary; I may have just not properly concluded the gestures. It's basically a hack, but it does provide a simple mechanism to transmit more than just group IDs back to the server. I don't presume to have the real solution to this - just showing what I came up with today. I send it anytime a gesture has completed or a new model has been loaded. We'll see if that works....
-- more than anything, I wanted my gesture server to determine the type of gesture on the fly. (Like I presume Windows 7 does.) I wasn't sure what the idea was behind asking a client what gestures it wanted based on birth location. Initially I asked for all of them, then quickly realized that then I would get all these events back regardless of the actual motion involved. So that didn't work for me. My idea was to create a generic "two-point" gesture that could deliver zoom, rotate-Z, or double-finger drag events back to the client. This seems to work reasonably well, at least in simulation.
Is there some other way that SparshUI somehow takes care of that? I may have missed it. The code for that is in http://jmol.svn.sourceforge.net/viewvc/jmol/trunk/Jmol/src/org/jmol/multitouch/sparshui/TwoPointGesture.java (If you look closely there, you will realize that I'm not using TouchPoint ids. That's because I wasn't convinced the HP could deliver those. Anyway, it basically works.
Basic questions remain:
-- Will I really be able to trap (and consume) Windows window events on the HP?
-- Will Windows 7 just automatically turn some simple gestures into, for example, mouse wheel events? (Making some of my work here unnecessary?)
-- In general, what is the relation between SparshUI and Windows 7 WM_GESTURE and WM_TOUCH events? (What is known about Windows 7 now?)
-- I'm very interested in seeing how people have used SparshUI in real applications (not just test suites). Has much been done in the molecular visualization area?
That's plenty for now!
Thanks very much for all the great documentation. Very helpful!
Bob Hanson
Principal Developer, Jmol
--
Robert M. Hanson
Professor of Chemistry
St. Olaf College
1520 St. Olaf Ave.
Northfield, MN 55057
http://www.stolaf.edu/people/hansonr
phone: 507-786-3107
If nature does not answer first what we want,
it is better to take what answer we get.
-- Josiah Willard Gibbs, Lecture XXX, Monday, February 5, 1900
From: Robert Hanson <han...@stolaf.edu>
Date: Sun, Nov 29, 2009 at 4:53 PM
Subject: Jmol and SparshUI
To: spar...@googlegroups.com
Cc: Prasad RS <pras...@iastate.edu>, Jay Roltgen <jrol...@gmail.com>, Stephen Gilbert <gil...@iastate.edu>, Jmol Developers <jmol-de...@lists.sourceforge.net>
Stephen, Jay, Prasad, and friends,
Jmol (http://jmol.sourceforge.net, http://chemapps.stolaf.edu/jmol/) is an open-source (LGPL) Java-based molecular visualization tool that can be utilized as an applet, a stand-alone application, or as a component of a larger program.
This weekend I successfully integrated SparshUI into Jmol. (Still waiting for a machine to test this on, but I included an adapted TouchSimulator, so at least it is working in that mode.) I'm sure there are plenty of fixes to do, but I am very pleased with how this looks, and in case anyone out there is interested in multi-touch molecules, this should do the trick.
I'm sure in this message you will detect my naivete in this multitouch area. I've just gotten going on it; bear with me!
The code for this integration can be found at the sourceforge site http://jmol.svn.sourceforge.net/viewvc/jmol/trunk/Jmol/ and, of course, is available for anonymous checkout by interested parties using Eclipse/SVN. The main classes and packages involved include:
http://jmol.svn.sourceforge.net/viewvc/jmol/trunk/Jmol/src/org/jmol/viewer/ActionManagerMT.java
This class controls all the mouse and keyboard action of Jmol. This particular subclass adds the multitouch capability.
http://jmol.svn.sourceforge.net/viewvc/jmol/trunk/Jmol/src/com/sparshui/
The package with the adapted SparshUI code.
http://jmol.svn.sourceforge.net/viewvc/jmol/trunk/Jmol/src/org/jmol/multitouch/sparshui/
The package with my client adapter and custom Gesture class.
Don't expect this code to work with actual multitouch hardware yet -- this is just using a mouse and touch simulation. I'm still awaiting my HP IQ500 machine for that. (And I'm rather pessimistic now that we can actually get this going properly with that -- we'll see. I'm not happy with what I read in the HP TouchSmart SDK about the messages it sends.)
Stephen, do you think it will be possible to use SparshUI effectively on the HP? Have you tried that? (I'm worried that we can't trap events, and we'll see scrolling of the browser window, for example, when a user tries to scroll within the applet. (Currently the mouseWheel event must be consumed to avoid this.) I'm not sure how the SparshUI business allows for consuming window messages so they don't go through to a lower layer. I've asked for Windows 7 on the machine I get, but I don't know how well that will work or even if that is the right direction to go.
There were several nuances that I needed to solve to effect this integration. I thought you might be interested:
-- client-side definition of gesture classes. I saw this was in the plans, but I didn't see a solution there already. I accomplished this by adapting the GestureFactory to allow for class names and use of Class.forName(name).newInstance(). Then, instead of just delivering SparshUI gesture ordinals, I adapted the interface to allow for delivery of class names (Strings). I simply use negative numbers instead of ordinals for the "gestureID" and then use that number to indicate the length of the byte-encoded string that follows. So I now have a "TwoPointGesture" class that I can use. Works like a charm! (More on this below.)
-- multiple window targets. Since applets and Java applications could have any number of instances, I simply gave each client its own "groupID". Actually, in Jmol I allow for 256 "sub-groupIDs" that an applet or application could in principal use like normal SparshUI groupIDs. This is in case I can't figure out how to get true Windows multitouch messages to my application. I don't think this is working properly quite yet.
--special handling flags. I was probably just doing something wrong, but I was having trouble with the gesture classes not resetting properly, so I added a mask to the groupID sent back to the server. It basically looks like this in clientConnection.ProcessBirth:
int groupID = getGroupID(touchPoint.getLocation());
int jmolFlags = (groupID & 0xF0000000);
if (jmolFlags != 0) {
switch (jmolFlags) {
case 0x10000000:
// reset flag
_groups = new HashMap();
break;
}
groupID &= ~jmolFlags;
}
Maybe this wasn't necessary; I may have just not properly concluded the gestures. It's basically a hack, but it does provide a simple mechanism to transmit more than just group IDs back to the server. I don't presume to have the real solution to this - just showing what I came up with today. I send it anytime a gesture has completed or a new model has been loaded. We'll see if that works....
-- more than anything, I wanted my gesture server to determine the type of gesture on the fly. (Like I presume Windows 7 does.) I wasn't sure what the idea was behind asking a client what gestures it wanted based on birth location. Initially I asked for all of them, then quickly realized that then I would get all these events back regardless of the actual motion involved. So that didn't work for me. My idea was to create a generic "two-point" gesture that could deliver zoom, rotate-Z, or double-finger drag events back to the client. This seems to work reasonably well, at least in simulation.
Is there some other way that SparshUI somehow takes care of that? I may have missed it. The code for that is in http://jmol.svn.sourceforge.net/viewvc/jmol/trunk/Jmol/src/org/jmol/multitouch/sparshui/TwoPointGesture.java (If you look closely there, you will realize that I'm not using TouchPoint ids. That's because I wasn't convinced the HP could deliver those. Anyway, it basically works.
Basic questions remain:
-- Will I really be able to trap (and consume) Windows window events on the HP?
-- Will Windows 7 just automatically turn some simple gestures into, for example, mouse wheel events? (Making some of my work here unnecessary?)
-- In general, what is the relation between SparshUI and Windows 7 WM_GESTURE and WM_TOUCH events? (What is known about Windows 7 now?)
-- I'm very interested in seeing how people have used SparshUI in real applications (not just test suites). Has much been done in the molecular visualization area?
That's plenty for now!
Thanks very much for all the great documentation. Very helpful!
Bob Hanson
Principal Developer, Jmol
--
Robert M. Hanson
Professor of Chemistry
St. Olaf College
1520 St. Olaf Ave.
Northfield, MN 55057
http://www.stolaf.edu/people/hansonr
phone: 507-786-3107
If nature does not answer first what we want,
it is better to take what answer we get.
-- Josiah Willard Gibbs, Lecture XXX, Monday, February 5, 1900
--
Robert M. Hanson
Professor of Chemistry
St. Olaf College
1520 St. Olaf Ave.
Northfield, MN 55057
http://www.stolaf.edu/people/hansonr
phone: 507-786-3107
If nature does not answer first what we want,
it is better to take what answer we get.
-- Josiah Willard Gibbs, Lecture XXX, Monday, February 5, 1900
Reply all
Reply to author
Forward
0 new messages