Suggestion for UI solution to lower cost and increase flexibility and security: GestIC

533 views
Skip to first unread message

David Harker

unread,
Apr 16, 2014, 10:34:43 AM4/16/14
to moolt...@googlegroups.com
Hi all

I'm wondering whether the GestIC 3D gesture controller has been considered as an input device? This is a non-contact, 3d gesture recognition chip which supports e.g. virtual "thumbwheels" out of the box. The chips are cheap in volume (much cheaper than gorilla glass, I'm sure!) and would allow firmware defined UIs. Could even support the user drawing a "3d shape" in the volume in front of the panel which is then interpolated into a longer PIN i.e. like a 3d version of the android lock screen.

I have not yet used this chip in a design but have researched it in detail and it looks good to me.

Microchip's MGC3130 is the world’s first electrical near field (E-field) 3D Tracking and Gesture Controller. Based on Microchip’s patented GestIC® technology, it allows users to interact with their devices using hand/finger position tracking and intuitive free-space gestures in real time. The MGC3130 is a unique solution that enables the next breakthrough in user interface design.


GestIC® Technology Benefits


  • Very low power design
  • From 0 (touch) to 15 cm detection range
  • No detection blind spots
  • Usage of thin, low-cost sensing electrodes
  • Electrodes hidden behind housing
  • No ambient influences (light/sound)
  • High sensitivity
  • 70-130 kHz carrier frequency – no RF interference

MGC3130 Key Features


  • Recognition of 3D hand gestures and x/y/z positional data
  • High resolution of up to 150 dpi
  • Integrated digital signal processing unit
  • Low noise analog front end
  • Frequency hopping against noise
  • Digital interface
  • On-chip Colibri gesture suite

Typical Applications


  • Notebooks/Keyboards/PC Peripherals
  • Mobile phones
  • Tablet computers
  • Electronic readers
  • Remote controls
  • Game controllers
  • Light Switches
  • Air condition control



mathieu...@gmail.com

unread,
Apr 16, 2014, 11:26:11 AM4/16/14
to David Harker, mooltipass
Hello David,

Compared the the AT42QT2120 we're currently using, this chip seems to need a lot more external components.
Moreover, given that the top PCB is quite small, I wonder what range we could achieve?
On a more general note I wonder what is the benefit of having a 3D interface for 3x the price of our current touch IC.
(A minor problem is that it is using 3.3v)

Let me know what you think,
Mathieu


--
You received this message because you are subscribed to the Google Groups "mooltipass" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mooltipass+...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

David Harker

unread,
Apr 16, 2014, 12:52:38 PM4/16/14
to moolt...@googlegroups.com, David Harker
Hi Mathieu

I'm not yet really familiar with all the ins and outs of this project so please forgive me if I'm going over already trodden ground.

This seemed like a great fit for the project but, thinking about it more, perhaps it would be better off as part of a future version/separately available addon. It would need a 4-layer board to keep the LEDs on the front PCB and really robust and detection of complicated 3D "squiggles" seems like it would require doing some vector arithmetic on the chip so might require a dedicated coprocessor/lots of program memory on main µC.

Detection area would be pretty much the whole front PCB in X and Y, have just checked through the datasheet and the electrode design guidelines and it seems that microchip are unwilling to commit themselves on Z sensitivity. As the claimed max is 15cm it seems reasonable to expect you could get at least 5.


(for quick ref:)


Identified the following benefits:
  1. Mitigate security problems due to physical fingerprints/smears on front panel
    While opening it like a safe would improve security, the overlaid smears would not precisely overlay each other so a "zig-zag" pattern allowing pin to be much more easily guessed could remain. This would be exacerbated by any tendency press harder (=larger contact patch) during finger direction changes. Arbitrary starting points would mitigate this to an extent but the relative position of the digits would still remain, so security would still be reduced to requiring [number of rotational degrees of freedom] * [radial measurement error of input vs. fingerprint "zig" or "zag" point] tries. This sounds like it might give an attacker a success rate which made it well worth attacking, or an even lower one of 1/~RadialDoF if measurements of multiple input smudges for the same pin could be captured or radial location could be captured accurately in one go. Also, humans aren't random and it might be confusing/annoying if such a requirement was implemented in firmware. 
  2. Reduce need for wear/wipe resistant material for front panel to allow device to be cleaned of fingerprint smudges (assuming for aesthetics rather than security)
    A lack of fingerprint grease on front panel will reduce need for cleaning, reduce required force/duration during cleaning and prevent differential discolouration due to grease being rubbed selectively more into any micro surface damage in frequently touched areas.
  3. Potential for higher security PIN entry in the face of "shoulder surfing" or e.g. hidden camera
    Tracked point (i.e. fingertip) for PIN entry moves in 3d not 2d making capture from a single viewpoint more difficult as perceived strokes closer to the visual axis of the observer/camera would appear shorter and the small scale of the measured volume and movements themselves would frustrate inferring this "lost dimension" from a single viewpoint. It would also be more difficult for a casual observer to interpret/remember, especially if a 3d shape "PIN" didn't require feedback via the screen (i.e. the numbers on the "combination lock" wheel).
  4. Sci-fi cool factor
    To attract more customers and
    Add lots of value as an Arduino platform
  5. Firmware defined user interface and flexibility of input gestures
    Use of various gestures would make UI more efficient and more fun. E.g.s to lock the device, to scroll through lists with a linear motion and maybe even operate a more efficient to use soft keyboard for text entry.
    Allows innovative new UI ideas to be implemented in software, tested out and deployed to existing users without the need for a PCB redesign.
David

mathieu...@gmail.com

unread,
Apr 16, 2014, 1:55:22 PM4/16/14
to David Harker, mooltipass
Hey David,

Thanks for taking the time to write such a detailed answer.

It would need a 4-layer board to keep the LEDs on the front PCB and really robust and detection of complicated 3D "squiggles" seems like it would require doing some vector arithmetic on the chip so might require a dedicated coprocessor/lots of program memory on main µC.
>> I completely agree with you. 

As the claimed max is 15cm it seems reasonable to expect you could get at least 5.
>> I'm also wondering what would be the influence of our "bottom PCB"

Mitigate security problems due to physical fingerprints/smears on front panel
>> That's awesome you took the time to read about the way we'll enter the password. We could improve this indeed

Reduce need for wear/wipe resistant material for front panel to allow device to be cleaned of fingerprint smudges
>> that's correct

Potential for higher security PIN entry in the face of "shoulder surfing" or e.g. hidden camera
>> While that's definitely an advantage, it's also a main drawback for non-informatics familiarized persons :(

Add lots of value as an Arduino platform
>> We haven't gotten many feedback on the Arduino compatibility thingy yet... what do you think about it? I hope the OLED screen will make users happy

Firmware defined user interface and flexibility of input gestures
>> Agreed but see 3). Moreover, we'd definitely need a more powerful IC.

As you can see I agree with several of the points you mentionned. The only drawbacks I can see are:
- more expensive
- requires a more powerful uC
- may restrain the final market to geek persons

Bjorn Wielens

unread,
Apr 16, 2014, 2:50:31 PM4/16/14
to mathieu...@gmail.com, David Harker, mooltipass
I'm curious how susceptible this would be to said shoulder surfing - if it was configured to collect many aspects of the gesture it could potentially be very hard for someone to imitate said gesture even if observed because the sensor can measure subtle nuances in the correct person that the impostor would not have.

At the same time though... you'd have to balance it out with reliably unlocking for the same person and figure out the flexibility to gesture variants.

All in all, interesting idea, but definitely needs a lot of changes and research to be implemented. It might be easier to go the shield route and instead fab an arduino shield with the sensor/processor and then wire that to the mooltipass. 

Bjorn.




From: "mathieu...@gmail.com" <mathieu...@gmail.com>
To: David Harker <dhha...@gmail.com>
Cc: mooltipass <moolt...@googlegroups.com>
Sent: Wednesday, April 16, 2014 2:55:22 PM
Subject: Re: Suggestion for UI solution to lower cost and increase flexibility and security: GestIC

David Harker

unread,
Apr 16, 2014, 2:58:31 PM4/16/14
to moolt...@googlegroups.com, David Harker
Hi Mathieu

No problem! I like the look of this project a lot, I hope I can contribute something useful even if I don't have the time to help with PCBs/firmware at the moment. 

Perhaps there's no getting around the cost issue, though perhaps it can be mitigated in part with a reduction in front panel cost and (in an ideal world at least) increased volume with increased demand due to awesomecool 3d input feature ;-)


If this thing was implemented as an alternative/addon front PCB instead of being a core part of the system then it seems that people who wanted it would understand and be willing to pay the small extra for the coprocessor. If fully integrated then I agree it isn't ideal without a high percentage of potential users who would specifically like this feature. 

As the claimed max is 15cm it seems reasonable to expect you could get at least 5.
>> I'm also wondering what would be the influence of our "bottom PCB"
Was thinking you'd have to include a ground plane as layer 3 (where 1 is closest to the user). This would hurt sensor performance but from the datasheet/electrode design docs it seems like it would mitigate any problems from anything behind the board. No idea what impact the slots and traces for the LEDs would have but they're small relative to the overall sensor layout so hopefully would get away with it. Seems like it would be a pretty common requirement for people implementing this the GestIC 

Mitigate security problems due to physical fingerprints/smears on front panel
>> That's awesome you took the time to read about the way we'll enter the password. We could improve this indeed
It's a tricky problem! For me I'd ideally like to move my finger in the same sequence from the same starting point each time I enter the PIN to reduce cognitive load and PIN entry time by taking advantage of my "muscle memory", but that automatically causes the fingerprints problem so personally I'd consider it not viable... here's a potential solution which has just sprung to mind using the existing hardware only (apologies if this is what you were going for already but not read anywhere about the random modifiers step):
  1. The user is about to enter their pin
  2. The screen shows a random digit from 0-9. They place their finger anywhere (though we can assume probably in the same place every time for most people) on the wheel. As they move their finger around the number goes up/down.
  3. The user releases their finger when the digit is correct.
  4. The screen now shows one asterisk (and another asterisk for each subsequently entered digit) and another random number 0-9.
  5. The user places their finger back on the wheel to enter the next digit. Hopefully (after a bit of practice at least) the fact that it's easier to place the finger back exactly where it lifted off from rather than returning it to some arbitrary "home" position plus the lack of muscle memory will mean that users naturally place the finger straight back down.
  6. Goto 1, OR
  7. User presses a button to confirm/cancel the PIN entry (or maybe just delete one digit/cancel if no digits or tap to delete, hold to cancel).
Seems like this would randomise the smudge marks a bit more without overly compromising the usability - what do you think? I guess the smudge patterns would still very much be vulnerable to interpretation but they wouldn't leak as badly so hopefully an attacker would need to capture an impractically large number of distinct smudge patters from the entry of the same PIN in order to shrink the search space usefully. An improvement might be to also randomise the relative direction of increment/decrement of the digit shown on screen (i.e. the digit currently being entered by the user) relative to the direction they spin the wheel. You could provide a hint on screen for this but doesn't sound too annoying that I'd turn it off without this. 


Potential for higher security PIN entry in the face of "shoulder surfing" or e.g. hidden camera
>> While that's definitely an advantage, it's also a main drawback for non-informatics familiarized persons :(
Perhaps, although the GestIC can be used in a "touch" mode which works exactly as the current interface or to emulate a keypad as well as anything using the 3d. It could ship in "I-can't-understand-the-concept-of-moving-my-finger-in-3d-because-that's-not-how-my-iPod-works"-mode with the option to enable "high security"- or "3d"- or whatever mode for geeks/hackers/young people. You could also use exactly the same UI paradigm as the currently but just implement it 1cm above the surface - that way it's still a 2d input conceptually but solves the fingerprint problem.

Add lots of value as an Arduino platform
>> We haven't gotten many feedback on the Arduino compatibility thingy yet... what do you think about it? I hope the OLED screen will make users happy
I think it's a great idea - while it does place some limits on your choice of chips (I guess you're pretty much bound to using one of the same chips are official Arduinos) it opens it up to a wider market. While I can see that a sleek looking Arduino-compatible with the OLED screen and touch interface would be an attractive platform for hobbyist projects it would have to be at a competitive price to all the other 'duinos out there to sell in volume - the OLED screen looks awesome, but there's plenty of awesome Arduino compatible screens on the market already, and ditto the touch interface. While reducing cost to compete might seem incompatible with a small cost rise from adding the feature, the 3d interface would be a "killer feature" and would be unique and exciting enough to be attractive at a higher price.


Firmware defined user interface and flexibility of input gestures
>> Agreed but see 3). Moreover, we'd definitely need a more powerful IC.
A bigger µC is an issue for cost and might also cause issues for Arduino compatibility?
Cost: That said, it seems like you might fit glue code on a small micro which served as a co-processor for the GestIC and provided a noob friendly interface over I2C or something so people wishing to hack it in "Arduino mode" could just send commands like "listen for a 3d pin and store an encoded version in a FIFO (for readout on interrupt)" or "capture a number from the virtual number pad and then interrupt" or "trigger an interrupt when a user's hand approaches the front panel". Would make it easy to hack and have a minimal memory footprint.
Non-technical users: On the flipside, this would make it really easy to test out different UI paradigms with a range of users and so you could refine it to be easier to use based on user feedback rather than having to respin the front PCB in order to test a new idea.
 

As you can see I agree with several of the points you mentionned. The only drawbacks I can see are:
- more expensive
- requires a more powerful uC
- may restrain the final market to geek persons

I've addressed these as best I can but fully agree with you that it isn't clear cut - there are tradeoffs either way but for me, at least, the 3d interface is unique and exciting enough to justify and increase in cost plus it would dramatically broaden the appeal to the hacker/maker crowd. Assuming it would add something like $10-20 to the cost to the consumer assuming 250% of additional BOM cost and the 4-layer board (and generally pulling numbers out of my arse) so I suppose what it comes down to is what proportion of the base cost this would be; if it's going to be pretty damn expensive compared to an Arduino anyway then I'd add it, if it's going to be as dirt cheap as possible then I'd save it for an addon or ditch the idea all together :-)

David Harker

unread,
Apr 16, 2014, 3:25:55 PM4/16/14
to moolt...@googlegroups.com, mathieu...@gmail.com, David Harker, Bjorn Wielens
Hi Bjorn

I don't think the thing is so repeatable/accurate, or at least encoding such a complex gesture reliably enough to avoid getting the PIN wrong and bricking the smartcard would be tricky. A potential solution might be recording a gesture as a series of relative "vectors" with the stroke length normalised in axis to the average of all inputted strokes. An encoded gesture might consist of "START // X+1|Y+0|Z-1 // X+0|Y+2|Z+0 // ... etc.". It could be bracketed by some sort of "synchronisation" gesture e.g. your finger always enters the sensed space from the top center along the Z axis (perhaps until the device beeps or gives some visual feedback that the finger is in the "home" position) and leaves the same way. Stroke lengths could be normalised to "short" and "long" (e.g. 1 and 2 above). This would mean each vector could be represented as 6^5 (7776) symbols so each stroke of the finger could fit into a 13 bits as well as hopefully being unambiguous enough to match reliably and supporting two lengths and 48 directions. A "PIN" with 6 strokes would then have 12230590464 possible combinations (34.51 bits of entropy if my crappy maths is right!).

D

David Harker

unread,
Apr 16, 2014, 3:31:42 PM4/16/14
to moolt...@googlegroups.com, mathieu...@gmail.com, David Harker, Bjorn Wielens
Thinking about it, the entropy would lower because of the "edges" of the sensed space, those numbers only apply to a volume that the user can't reach the edge of so actual entropy would depend on how small the user could draw their glyph compared to the sensed volume. 

mathieu...@gmail.com

unread,
Apr 17, 2014, 4:36:09 AM4/17/14
to David Harker, mooltipass, Bjorn Wielens
Hey everyone,

Bjorn,

At the same time though... you'd have to balance it out with reliably unlocking for the same person and figure out the flexibility to gesture variants. 
>> That'd be an interesting research topic :)

All in all, interesting idea, but definitely needs a lot of changes and research to be implemented. It might be easier to go the shield route and instead fab an arduino shield with the sensor/processor and then wire that to the mooltipass
>> Agreed


David,

No problem! I like the look of this project a lot, I hope I can contribute something useful even if I don't have the time to help with PCBs/firmware at the moment. 
>> Thanks!

If this thing was implemented as an alternative/addon front PCB instead of being a core part of the system then it seems that people who wanted it would understand and be willing to pay the small extra for the coprocessor. If fully integrated then I agree it isn't ideal without a high percentage of potential users who would specifically like this feature. 
>> Agreed. Though when you have some spare time you may be motivated to fork our current project to integrate this IC... It'd be quite nice :)

Was thinking you'd have to include a ground plane as layer 3 (where 1 is closest to the user). This would hurt sensor performance but from the datasheet/electrode design
>> Yep. On our current design there's no ground plane under the sensing footprints, just traces. From the check list in the AT02259: "There is no GND plane behind components, tracks and electrodes"

here's a potential solution which has just sprung to mind using the existing hardware only
>> That's awesome! Either me or another contributor will implement that... thanks!

A bigger µC is an issue for cost and might also cause issues for Arduino compatibility?
>> Your implementation idea is correct :)

for me, at least, the 3d interface is unique and exciting enough to justify and increase in cost plus it would dramatically broaden the appeal to the hacker/maker crowd.
>> I feel that you're motivated by that... though I also know many sceptical persons that will be very skeptical

I suppose what it comes down to is what proportion of the base cost this would be
>> This is how engineers think, but not how customers do ;). I'd really like the *cost* of the device to be around $50

Cheers,
Mathieu

David Harker

unread,
Apr 17, 2014, 12:43:39 PM4/17/14
to moolt...@googlegroups.com, David Harker, Bjorn Wielens
Hi Mathieu/Bjorn


Bjorn, re: implementation as a shield - wouldn't work as sensor PCB needs to be where the front PCB is in order to have any chance of working. I suppose you could add a secondary sensing interface that looks like one of those wireless Apple trackpads and clips on/sits in front of the device and is wired back to a shield; that'd have the advantage of a larger available sensing area but the inability to predict in advance what sort of surface it'll be placed on might hurt performance. The extra sensing area would be really useful though and make it easier to use.

Mathieu, re: my wheel interface - it occurs to me that the random digit would have to appear when the user places their finger on the wheel rather than before they do, otherwise there'd be a random un-entered digit hanging on the end of the PIN when they come to press the "OK" button. Seems like that would be confusing/worrying for users if it was silently discarded if OK was pressed before any wheel input.

Thanks for all the good discussion - looks like my idea was a non-starter after all - aiming for a $50 cost I wouldn't consider adding this thing in at this stage, either! In future I would be more than happy to design an alternative front panel PCB.

Just one question in that regard - I see there's a b2b interconnect for the front PCB to connect to the mainboard; does this carry raw signals from the touch electrodes or is it a data bus of some sort which would be suitable for interfacing to the GestIC? If it's the former of course it might make the task impossible but if it's I2C or a UART or something then it seems like it would be a pretty straightforward board to design. The software will be the tricky bit!

D

mathieu...@gmail.com

unread,
Apr 18, 2014, 7:17:50 AM4/18/14
to David Harker, mooltipass, Bjorn Wielens
Hey David,

Mathieu, re: my wheel interface - it occurs to me that the random digit would have to appear when the user places their finger on the wheel rather than before they do, otherwise there'd be a random un-entered digit hanging on the end of the PIN when they come to press the "OK" button. Seems like that would be confusing/worrying for users if it was silently discarded if OK was pressed before any wheel input.
>> good point

Just one question in that regard - I see there's a b2b interconnect for the front PCB to connect to the mainboard; does this carry raw signals from the touch electrodes or is it a data bus of some sort which would be suitable for interfacing to the GestIC?
>> the connector includes the I2C bus, one IO and the 5V/GND so it'd be quite easy to design an alternative input interface :)

Cheers,
Mathieu

Bjorn Wielens

unread,
Apr 18, 2014, 7:28:08 AM4/18/14
to David Harker, moolt...@googlegroups.com
Hi David,

Good point, it might not be practical as a final design but the shield route would make it easier to test things without a full case redesign.
Oliver's original design does have a base for the unit to sit on... perhaps replace the base piece with said shield(in an enclosure) and have the sensor boar protrude forward at 90° or sit parallel below the device?

Just throwing out some thoughts, feel free to discard if I'm just waffling :)

Bjorn.





From: David Harker <dhha...@gmail.com>
To: moolt...@googlegroups.com
Cc: David Harker <dhha...@gmail.com>; Bjorn Wielens <unia...@yahoo.ca>
Sent: Thursday, April 17, 2014 1:43:39 PM
Reply all
Reply to author
Forward
0 new messages