Dragonfly and Windows applications running as administrator

[Ryan Richter]

Okay, here's another question for the group that I've just been living with for a while. Is there a way to make Dragonfly scripts work while using a Windows application that is running as administrator?

I am on Windows 10, Dragon 15, latest version of "classic" dragonfly, NatLink 4.1victor, Python 2.7.15.

natlinkmain started from C:\NatLink\NatLink\MacroSystem\core:

  NatLink version: 4.1victor

  DNS version: 15

  Python version: 27

  Windows Version: 10

Thanks,

-Ryan

[Caspar]

AFAIK: not trivially, not yet. The gist of the problem is that the process causing the clicks/keypresses/etc needs to be running as an administrator itself, but Dragon refuses to operate if it runs as an administrator.

I read somewhere that some of the folks working on Caster have been experimenting with building a "proxy" which runs as an administrator, and Dragonfly's actions are modified or wrapped to call into that proxy - basically what Aenea does, except the windows version of Aenea supports only a limited subset of actions. Not sure what the status of that is. (I have a proof of concept along these lines, but haven't gotten around to cleaning it up and publishing it yet. Also I suspect that the approach taken by Talon's Draconity of putting the brains outside Dragon is better in the long run, as it provides a path forward through Py2's imminent deprecation, so I've been thinking it's better to focus on that.)

Come to think of it, an approach you could try is to not dump your grammar files into your natlink user dir, and instead run your python process separately then import natlink from your scripts. Theoretically this might work because natlink talks to Dragon using a COM interface, and I've accidentally run my grammars as Python scripts and seen them start or interact with Dragon (can't remember). I'd say Implementation and Acceptance of Natlink is a good paper to read if you want to try going down this route.

[Caspar]

I found a way to make dragonfly actions running inside Dragon interact with Admin applications; technique is at the bottom of the message if you're not interested in the exposition.

After I wrote my last message, I poked at this problem some more because something was bugging me: actions triggered by Dragon's built in grammar do work on applications running as Administrator. (Useful to know because you can "mouse click", "press enter", "type A", "close window" etc on them even when your grammars don't work.)

I had assumed this was because 2 of natspeak.exe's subprocesses (dgnuiasvr.exe and dgnuiasvr_x64.exe - both labeled as "Dragon NaturallySpeaking UI Automation Server") were responsible for actually sending keystrokes, and they were running as administrator. However, turns out they are not running in Elevated (per Task Manager -> Details, if you add the Elevated column) mode. Instead, there's another concept at play here: UI Privilege levels. Most processes (including natspeak.exe) run at medium privilege level, but dgnuiasvr.exe and dgnuiasvr_x64.exe run at high privilege levels (checked via Sysinternals Process Explorer, process Properties, Security tab, "mandatory label/XXXX" table entry), and that's why they can interact with elevated apps (otherwise windows silently drops the data of postmessage and sendmessage API calls from lower privilege processes to higher privilege processes - unless the higher privilege process whitelists those types of messages).

How come they get to run at a higher privilege level? Their application manifest (in the exe file) contains a uiAccess="true" section, and their code is signed with a valid signature from a certificate the system trusts, and they live in a "protected location" (program files) which only high privilege apps can write to; all 3 of those things need to be true for a process to get higher privileges.

natspeak.exe is already 2/3rds of the way there (in a secure location and code-signed by Nuance, but it has uiAccess="false" in its manifest), so we can make it run at a higher privilege level by following these steps.

1. Accept: if you continue, you do so at your own risk! This is definitely voiding your warranty, and I take no responsibility for any bad things that may happen.
2. Shut down Dragon - make sure natspeak.exe isn't running.
   
3. Make a backup of natspeak.exe (it normally lives at C:\Program Files (x86)\Nuance\NaturallySpeaking15\Program\natspeak.exe ) in case it all goes south.
   
4. Use a hex editor like HxD running as Administrator to open c:\temp\natspeak.exe , update uiAccess="false" to uiAccess="true" , and save the file.
   
5. That last step invalidated the signature on natspeak.exe by changing the file's checksum, so generate a new cert, install it, and sign natspeak.exe yourself: https://gist.github.com/caspark/29a38ebb47cc7b2315c33f71c01ec237
6. Start Dragon again; it should be able to interact with elevated apps now, despite not being elevated itself.
   

Troubleshooting:

• if you get a message of "A referral was returned by the server", the signature on the executable is probably wrong. Confirm that over at natspeak.exe's Properties >> Digital Signatures >> Details.
• Windows seems to cache exe manifests. Rebooting is supposed to clear the cache, but updating the last modified timestamp on the exe is supposedly the only reliable fix. HxD and the code signing steps both update the last modified timestamp, so hopefully you won't run into this, but this tripped me up for a while before I figured it out (I was using another tool to edit the manifest which wasn't updating the last modified timestamp, and that made my unaltered natspeak.exe backup stop working!).
• You can use Sysinternals Process Explorer to check whether or not an app is running with medium or high privileges.
  

Hope this is helpful to people - it had been bugging me for quite some time now!

[Caspar]

Oh, slight correction to previous instructions: ignore the "c:\temp\" part of "c:\temp\natspeak.exe" in step 4. I was originally writing this up as if we would copy natspeak.exe to a separate location first, then changed my mind and forgot to update that bit.

Other tips for step 4:

• if you're struggling to find the section to update in your hex editor, navigate to offset 005916F3 or use the search function to search for "uiAccess"
• "false" contains 1 fewer letters than "true"; if your hex editor doesn't let you delete characters, just add a space as padding after the closing double-quote to make up the difference. (The manifest is XML so a space after an attribute value is harmless.)
  

[Ryan Richter]

I can't believe I missed this. Thank you so much. I've just been tasked with something at work that requires using applications that must be run with administrative privileges and so I've been googling around. Lo and behold I find the answer to my original question!

It works great at home and can't wait to set it up at work tomorrow and get cooking.

Thank you again!

[Jamison Wells]

 A much simpler way is to run grammars on its own thread rather than in process with Natlink. Disable Natlink through it's GUI. Then start Dragon and run the following from an elevated PowerShell window.

`python -m dragonfly load --engine natlink _*.py --no-recobs-messages`

You will then be able to interact with programs with elevated privileges.

It's not perfect as really all of dragonfly does not and should not need elevated privileges to interact with programs with elevated privileges. Possibly only Key and Text needs to be implemented. For more granular permissions thing like the following would need to be completed https://github.com/chilimangoes/CasterUIAutomation_POC

[Caspar]

Oh excellent, thanks for writing that up! I stumbled on running dragonfly out of process when I read one of the original papers behind natlink, but since I had my approach working I never followed it up with any research to figure out how to do it. But this is much simpler than the elaborate process I wrote up, so I'll have to give it a shot :)

Re elevating dragonfly, it's probably a tad more dangerous than running Dragon with uiAccess=true in theory, but probably not meaningfully so in practice. And on the plus side, running the whole Python process elevated means it'd be easy to start or kill other elevated processes without resorting to horrible hacks.