Jmol and SparshUI -- multitouch capability for the HP TouchSmart computer

[Robert Hanson]

SparshUI and Jmol developers,

OK, Jmol is set for multi-touch.

I have pretty much completed my adaptation of SparshUI to work with the Jmol application (either stand-alone or embedded) and the Jmol applet. Still testing, but it seems to be quite robust.

It's a rather loose adaptation -- I'm not using any of the standard SparshUI gestures but instead two of my own -- SinglePointGesture and TwoPointGesture -- and I'm using two ports (5946 for clients and 5947 for a single driver).
Code can be found at http://jmol.svn.sourceforge.net/viewvc/jmol/trunk/Jmol/src/com/sparshui and in http://jmol.svn.sourceforge.net/viewvc/jmol/trunk/Jmol/src/org/jmol/multitouch. Preliminary demo is on YouTube at http://www.youtube.com/watch?v=7x8uDRQKIg0


Implemented gestures include:

- single touch up/down/move (standard mouse down,drag,up -- model rotation)
- single-touch click (standard mouse click)
- single-touch long click (model translation)
- two-finger slide (model translation)
- two-finger stretch (zoom in)
- two-finger pinch (zoom out)

On the HP TouchSmart one first starts the device connection -- JmolMultiTouchDriver.exe.
Then you start the Jmol application using the -Msparshui flag or, for the embedded application, initialize it with the command option -multitouch-sparshui. For the applet, include the JavaScript command:

  JmolSetCallback("multiTouchSparshUI", "true")

prior to

  JmolApplet(...)

Or, with my latest version of Jmol.js, you can add parameters to the URL line, so with that you can adapt any page to work with multi-touch. For example, if you have JmolMultiTouchDriver.exe running on your HP TouchSmart, the following should work:

http://chemapps.stolaf.edu/jmol/docs/examples-11/molecules/molecules.htm?PARAMS=multiTouchSparshUI:true

The driver will cycle while waiting for a connection, and the first Jmol application or applet that loads will start a gesture server that connects with it. If additional applets or applications load, then their gesture server will not connect, but they will find the other gesture server.

The problem of multiple overlapping windows was solved easily by only allowing SparshUI action within a window if that window has the focus. So one needs to first click on a model to activate SparshUI action within that window.

There might be a problem if one applet starts a server and then closes while another is using it. In addition, multiple applets on the same page are no problem, but applets on different pages within the same tabbed browser system probably cause problems. Just closing a page doesn't necessarily work. I've been using File|Close Tab in Firefox.

I think this will suffice for now.

Thanks again for all the help.

Bob


From the source comments:


 *  For use in association with jmol.sourceforge.net using -Msparshui startup option
 *     (see http://jmol.svn.sourceforge.net/viewvc/jmol/trunk/Jmol/src/com/sparshui)
 *
 *  NOTE: This driver will not work with the standard Sparsh-UI gesture server, which
 *        expects only a 17-byte message from the device. Here we send 29, which includes
 *        4 bytes for the int message data length and 8 bytes for the ULONLONG event time.
 *
 *  Modifications:
 *
 *  Version 0.2
 *
 *  -- consolidates touchInfos, lastTimes, and touchAlive as "TouchPoint" structure
 *  -- uses SystemTimeToFileTime to generate a "true" event time
 *  -- delivers event time as a 64-bit unsigned integer (ULONGLONG)
 *  -- ignores the NextWindow repetitions at a given location with slop (+/-1 pixel)
 *  -- delivers true moves as fast as they can come (about every 20 ms)
 *  -- times out only for "full death" events (all fingers lifted) after 75 ms
 *  -- automatically bails out if started with a server and later loses service
 *  --  (e.g. applet page closes)
 *  -- operates with or without server for testing
 *  -- not fully inspected for memory leaks or exceptions
 *
 * Version 0.3
 *
 *  -- can only deliver ID "0" or "1"
 *  -- use of bitsets to deliver deaths, then births, then moves
 *     and proper accounting for errors either due to premature deaths or,
 *     in certain circumstances, the NextWindow driver sending a move
 *     with no previous "down" message. This amounts to:
 *        -- canceling all currently active points
 *        -- creating a BIRTH for each of these points in addition to this "moved" point
 *        -- canceling the "move" operation
 *
 *  -- comments in Jmol script format for replaying at any speed within Jmol
 *  -- requires a script file data.spt that would, for example, include:
 *
      function setWidthAndHeight(w, h) {
         screenWidth = w
         screenHeight = h
      }
      function pt(id, index, state, time, x, y) {
          var c = (id == 0 ? "blue", "red")
          y = screenHeight - y
          draw ID @{"p" + id} [@x, @y] color @c
          delay 0.01
      }
 *
 *     thus allowing for playback of the transcript of a session.
 *    
 *
 *  -- uses port 5947 due to nonstandard SparshUI:
 *
 *   4  (int) -1  (1 point; negative to indicate that per-point message length follows)
 *   4  (int) 21  (21 bytes per point event will be sent)
 *
 *   4  (int) ID  (0 or 1)
 *   4  (float) x (0.0-1.0)
 *   4  (float) y (0.0-1.0)
 *   1  (char) state: 0 (down/BIRTH), 1(up/DEATH), 2(move)
 *   8  (long) time (ms since this driver started)
 *
 *   all ints, floats, and longs in network-endian order




--
Robert M. Hanson
Professor of Chemistry
St. Olaf College
1520 St. Olaf Ave.
Northfield, MN 55057
http://www.stolaf.edu/people/hansonr
phone: 507-786-3107


If nature does not answer first what we want,
it is better to take what answer we get.

-- Josiah Willard Gibbs, Lecture XXX, Monday, February 5, 1900