Emscripten mobile and keyboards

[Beuc]

Hi,

Mobile browsers are making progress with WebAssembly support and I see more interest in handling on-screen keyboards in web games.
(typically so your game's port doesn't get the mobile player stuck with a mere "whose your name" prompt).

I explored 2 main ideas to make the virtual keyboard pop up in SDL2 web applications in a generic way, typically on SDL_StartTextInput (as Android SDL2 games do) but browsers don't make it easy:
https://github.com/emscripten-ports/SDL2/issues/80

At this point I'm not even sure at what layer it would be best to implement virtual keyboard support (besides re-implementing it in each application).

What's your experience with mobile webapps and keyboard support?

Cheers!
Beuc

[Floh]

> What's your experience with mobile webapps and keyboard support?

It massively *sucks* :D

I explored a "solution" which involves a hidden HTML text field, focusing that text field to bring up the keyboard, and unfocusing it to hide the keyboard again. It kinda works, but it has all sorts of problems:

- the helper text field cannot be outside the visible area for "security reasons" or hidden, so you'll may see a blinking cursor somewhere overlayed over your WebGL canvas

- showing the keyboard must be initiated from inside a Javascript input handler, so you can't open it programmatically, but only on user input (such as the user tapping the screen)

- I remember some weird problems on different browsers or platforms of the keyboard either overlaying the page, or scrolling the page up, or even shrinking the WebGL canvas while the keyboard appears

- don't expect it to work for more then 6 six months after a browser update breaks something, or on any browser, basically each browser needs specific workarounds and hacks.

You can check out my attempts here:

https://floooh.github.io/tiny8bit/cpc.html

Tap the screen to bring up the keyboard. It seems to work on iOS, but it's broken on Android last I checked...

For those 8-bit emulators I came to the conclusion that it is better anyway to draw the keyboard myself, because mobile keywords are missing essential keys (such as the arrow keys, or an Escape key.

Cheers!

-Floh.

[Beuc]

Hi Floh and thanks for your feedback!

This is exactly what I attempted to do, indeed it doesn't sound reliable to try and reuse the browser's keyboard.

I made tests on Android with Chromium and Firefox, and I notice a few differences:

- No security issues with offscreen text input (using top:-100px) (iOS-specific issue?)

- No issues with triggering the focus/keyboard programmatically, e.g. from a setTimeout (iOS-specific issue?)

- In your Amstrad demo, I can't type any letter (the completion/composition system gets in the way AFAICS); I attempted to always clear the text input to compensate, but I got dup events in Firefox

More details at https://github.com/emscripten-ports/SDL2/issues/80

Arrows/Escape are a different issue, I'm not trying to emulate a full keyboard, but I'd like to support the SDL TextInput API.

Maybe just display the HTML text input and send a single input event with the full text line when pressing Enter...

Cheers!
Beuc

--
You received this message because you are subscribed to the Google Groups "emscripten-discuss" group.
To unsubscribe from this group and stop receiving emails from it, send an email to emscripten-disc...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/emscripten-discuss/4c799ad0-0232-459c-8672-140cfdd9a077o%40googlegroups.com.