Key and mouse system-wide listeners in Linux app on Chrome OS

[Ashwin Mittal]

I did some research on it and found that there is no way we can do that by monitoring events in ChromeOS apps through a Linux app, but still, I am not fully confident and sure about it so would appreciate any help on this.

I found that Sommelier can get input events as mentioned here: https://www.chromium.org/chromium-os/developer-library/guides/containers/containers-and-vms/ but wasn't able to get it working properly.

[Mike Frysinger]

my design, random apps running in containers cannot snoop on all input. they can only see inputs if they are the foreground window. 

-mike 

On Tue, Apr 2, 2024, 03:15 'Ashwin Mittal' via ChromiumOS Discussion <chromium-...@chromium.org> wrote:

I did some research on it and found that there is no way we can do that by monitoring events in ChromeOS apps through a Linux app, but still, I am not fully confident and sure about it so would appreciate any help on this.

I found that Sommelier can get input events as mentioned here: https://www.chromium.org/chromium-os/developer-library/guides/containers/containers-and-vms/ but wasn't able to get it working properly.

--
--
Chromium OS Discussion mailing list: chromium-...@chromium.org
View archives, change email options, or unsubscribe:
https://groups.google.com/a/chromium.org/group/chromium-os-discuss

[Ashwin Mittal]

If they are the foreground window, it means they can only monitor local key events, not global ones. Correct me if I am wrong.

- Ashwin

[Mike Frysinger]

what is it you're trying to do? 

-mike 

[Ashwin Mittal]

We make a snippet application where users can create templates or snippets of text and give them shortcuts. Then when the user types the shortcut (in any application) the shortcut is replaced with the text of the corresponding snippet.

For example, if a user has a snippet with contents "Thank you" with the shortcut "/ty" and they typed "/ty" in an application, the "/ty" should be replaced with "Thank you".

[Mike Frysinger]

this is not possible with Linux apps by design. a Chrome extension might be able to do it in many places (like web pages), but that's the best you could get.

-mike 

[Ashwin Mittal]

Thanks, Mike for the response. It is very helpful 🙂

Regards,

Ashwin

[Dren]

I've been paying attention to this feed for several reasons and just thought this an appropriate moment to interject. It does not go against Linux by design. Ultimately it comes down to multiple shells and authorities. To be entirely honest I think that a simple command and predefined snippets is a fantastic idea to grant access. it will be quite a project but ultimately I think that it will be entirely worth it. If I were going to undertake such an endeavor I would first gain a better understanding of the way DBus works on ChromeOS devices because the processes that are used to mask and rename different ports and interfaces so that they can only easily be seen by the intended recipients is a similar computing concept. 

I've been doing quite a lot of study of the shells used in ChromeOS and working with various distros to gain different types of hardware access recently. If you expand on the concept please keep me informed and perhaps I could be of assistance. I believe that ultimately we are working toward a breathing machine for all computing and making certain that we eliminate cross contaminations is important! I, personally, don't have the time for the undertaking but eventually it will have to be done so please let me know if you're going to jump at it or not. 

PS: I was going through system logs the other day and learned something and found a concerning entry. I learned that the MouseCursorContainer isn't set to be drawn into the 'UI Hierarchy: Layers' when it is first run which is why, I assume, therr is no mouse access in the v2 shell (but there theoretically could be).  There is nothing during the establishment if UI Views and when the system is then booting the 'UI hierarchy: Windows' it simply sends a signal of "ctrl+shift" to add the visual mouse to the keyboard.

rd in the ExoShellSurface. (not completely pertinent but just a new snippet I recently gained) The concerning entry... 

       within 2 seconds of the beginning of the chrome_system_log, after informing me that it is unable to verify the multidevice_setup client factory or the gpu_surface_factory, I'm informed that perfetto cannot be accessed and then notified that the mouse 'has been flagged as a suspected imposter mouse'.

PSS: I believe that these observations are a direct result of a few small xfce4 scripts that I had written to initially establish a second full distro (have something to do with the org,freedesktop.DBus) When I find the time I will interpret it all better. If I cannot figure it out I'll have to pose the issue to the community but in the meantime, if you have any idea

[input_ime_api_chromeos.cc(969)] The engine is not active. extension id: jkghodnilhceideoidjikpgommlajknk

---
To unsubscribe from this group and stop receiving emails from it, send an email to chromium-os-dis...@chromium.org.

[Mike Frysinger]

to be clear, when i say "by design", i mean "CrOS security design does not permit arbitrary programs running in the VM to sniff/inject arbitrary input for the entire CrOS session, most notably outside of the VM".  we have no plans to change this, and if anything, our plans are explicitly to not permit this.  everything running inside the VM is arbitrary untrusted code and we cannot allow it to escape and access user data.

there's also the wayland design which makes it much more difficult for random apps to show up and start grabbing/injecting input.  the X design made this trivial, but wayland, by its design, set out to make this much more difficult.  the compositor has to support this flow and allow specific apps to do it.

(my understanding of the wayland stack is not as strong, so i could be using the wrong terms, or mis-representing -- feel free to correct)

-mike