multitouch lags

[al...@paranoici.org]

Hi,

I'm tryning to implement the multitouch support for python-whiteboard
but I'm a bit confuse respect the callbacks behaviour.
See line 154 of cursor.py

def makeCallback(self):
def callback(id, q):

I have patched the code in order to pass the id for each callback.
The id is an integer from 1 to 4 each id at the end of the task will
drive a different cursor with xinput2. The id 1 will drive the stardard
master device,
when will appears id greater then 1 each one will drive a new master
device.

First of all ask Pere if can help me because of I don't know very well
python and this code.
I think that could be usefull an hashtable with the id to access to its
info
e.g.
- clocks
- filter
- timeouts

The second one is that the callbacks is inside a mutex. This could be
right but I have seen some
problems in this approach. If the callbacks computation time is highest
that the callbacks rating then
will arrive a new ir point (that I could process suddently) and I lost
time in order to way the
old callbacks. An other problem is that (and I have seen it) if I use
an old computer and draw a line in
the screen, I make a fast track with IR pen and then release the
button. I see that the mouse continue
to move the pointer after than I have released the IR pen switch.

I have not a solution on this topic but I think that if arrive a new
point might be neclected the old one in order to
give a grater reactivity to the tool. If I'll have some ideas will
share with you.

I attach modified files.

Pietro Pilolli

[al...@paranoici.org]

Sorry I have forgotten the attachments

Pietro Pilolli

[Pere Negre]

Hi Pietro.

I've seen your code.

I'm a little confused about how can I use the xinput2 functions. It will help me to understand it if you could show me some examples. What python libraries can I use? Where can I find some decent documentation? It's xinput2 supported on all modern linux distributions?

About the lags... I know. I think it's because python will be always slower than C. But at the same time, it's more portable and less complicated.

I think we can assume that this problem will go away as the time passes on...

--
Pere

[al...@paranoici.org]

On 23.09.2012 22:25, Pere Negre wrote:
> Hi Pietro.
>
> I've seen your code.
>

Good.

> I'm a little confused about how can I use the xinput2 functions.

In theory using the same function with the device id parameter. (see
XIWarpointer)

> It will help me to understand it if you could show me some examples.
> What
> python libraries can I use?

So do I'm confused, sincerly I have investigated a full day and I have
not found
a good python binding.

In order to use xinput2 we need of theese function:
- move the device n to point x, y
- send a press event to device n
- send a release event to device n
- create a fake input device
- destroy a fake input device

In c xlib the first function is XIWarpPointer and the second two can be
done
using a XSendEvent (seems that the event passed can be loaded with the
device id)

Now python-whiteboard code use some xtest function included in
python-xlib but
they have not the xinput2 version.

I have done a partial attempt in the code in order to see if is
possible call some C
code from python and I have some success.

I attach the code. See at line 66 and 110 I have reimplemented the
function move with XwarpPointer
(that could be updated with XIwarppointer), The mouse down is not
correct but it is an attempt
to use the C XsendEvent. Now I am blocked because I have not sufficient
skill in order to bind
c code in python.

Sincerly I do not know if it is the right way to proceed but at the
moment I have not found a complete python
library to do this think.

> Where can I find some decent documentation?

Anyway seems that does not exist a python binding in order to use
xinput2 function,
and I am agree with you that the python-xlib documentation is
incomplete (I think that this might be
the right place for the xinput2 functions).

> It's xinput2 supported on all modern linux
> distributions?

Yes it is! Xinput2 is a part of standard xorg. You can use multiple
mice now!
Read for example:
http://ao2.it/en/blog/2010/01/19/poor-mans-multi-touch-using-multiple-mice-xorg

I have tested it and I can use a pad and a table to move different
cursor.
I have also update Ardesia code in order to use gtk2, xinput2 and now I
am able to write with
multiple pen. I really like see Ardesia and other apps work with a
wiimote multi pointer driver!
I hope that we will able to end the task successfully.

>
> About the lags... I know. I think it's because python will be always
> slower than C. But at the same time, it's more portable and less
> complicated.
>
> I think we can assume that this problem will go away as the time
> passes on...

I don't know if the probem really is python. The wiimote has 100Hz
frequency this means that
if the library send you all the ir point you will process call back 100
times per second,
and the function is atomic the the callbacks will make a long ir dot to
process "queue".

I think that one strategy can be exit from previous callback when a new
callback is called.
I hope to be able to explain the concept; it is a sort of streaming, is
more important process the new data in real time
that process all the ir point. Think that the mouse frequency is only
40Hz and it seems precise.

An other strategy could be avoid the callback usage and put a scheduler
that read the ir point after a timeslice (e.g 40ms)
and see if the tool is more reactive and precise enough. This parameter
could be tuned manually or with some policy.

This aspect is strongly related to the smoothing and the trambling
problem. I think that is possible that processing less IR point
that the device could be appear more precise (because it does not read
all our hand tremors).
But it is theory an empirical attempt could be better. A fast patch in
order to test if it is better could be make a callback
and jump the other one:

callback=true


execute_callback(pos)
{
//process ir pos
.....
}

callback(pos)
{
if (callback)
{
execute_callback();
callback=false
}
else
{
callbacks=true
}
}

In this way we work at 50Hz similar to mouse framerate. This test might
be done
with a whiteboard and an old computer or a netbook in order to see if
it will
become more reactive.

Pietro Pilolli

[Pere Negre]

OK. I see.

You have given me a start on all of this. I will work on it soon. Right now I can't because I'm a teacher and school has just started. I have work to do with my students.

On weekends I will take a good look at all this information you've given me. I think we will be able to implement the multitouch capability. I'll get another IR pen too, to go on with the experimentation.

Thank you, as always, Pietro!

--
Pere

[al...@paranoici.org]

I have some update about multitouch.
I have found a very interesting code called tuiototouch that transform
tuio data in touch events
http://lii-enac.fr/en/architecture/linux-input/tuiototouch.html

Tuio http://www.tuio.org/ is a protocol that define blob events,
we have already the wiimote data and we might send the touch events.

In tuiototouch.py might be all the code in order to generate a touch
event

Pietro Pilolli

[al...@paranoici.org]

The library used to emit touch events is

https://github.com/tuomasjjrasanen/python-uinput
that seems good!

Pilolli Pietro