Issue with sending keystrokes to another application from Kivy

[Sh Ty]

Hello. I'm new to programming, and I would like to ask for your help.

I want to make a Kivy app to send keystrokes to another application (in this case, Notepad).

I want it to work as follows:

1. User presses GUI button

2. The program looks for a running instance of Notepad

3. If notepad isn't already open, program opens Notepad

4. The program activates the Notepad window and sends keystrokes to it

When Notepad is already open and the GUI button is pressed, it works as expected.

However, when the button is pressed with no running instance of Notepad,

the Notepad window is activated but no keystrokes are sent.

When I run the non-GUI script ("python typerScript.py"), it works fine.

What am I doing wrong? This is driving me insane.

I would really appreciate all your kind help.

-----

Below are the codes:

typerApp.py

from kivy.app import App
from kivy.uix.floatlayout import FloatLayout
import typerScript


class MainScreen(FloatLayout):


    def type(self, text):
        typerScript.openNotepad()
        typerScript.sendKeys(text)


class TyperApp(App): # define Base Class of Kivy App


    def build(self):
        root_widget = MainScreen()
        return root_widget
 
# run program
if __name__ == '__main__':
    TyperApp().run()

typer.kv

#:kivy 1.11.1


<MainScreen>
    BoxLayout:
        orientation: 'vertical'


        TextInput:
            id: text_to_type
            text: 'hello world'


        Button:
            text: 'Type'
            on_release: root.type(text_to_type.text)

typerScript.py

import os
from pynput.keyboard import Key, Controller
import subprocess
import time
import win32com.client as comclt


keyboard = Controller()
wsh = comclt.Dispatch('WScript.Shell')


def openNotepad():


    # check if notepad is open
   
    print('checking for notepad')
    r = os.popen('tasklist').read()
   
    # open notepad if not opened
   
    if 'notepad.exe' not in r:
   
        print('opening notepad')
        subprocess.Popen('notepad', stdin = subprocess.PIPE, shell = True)
        time.sleep(0.5)


    print('notepad is open')
       
def sendKeys(text):


    # activate notepad


    print('activating notepad...')
    wsh.AppActivate('無題') # name of new Notepad window in Japanese OS
    time.sleep(0.5)
   
    # send keystrokes
   
    print('sending keystrokes...')
    keyboard.press(Key.f5)
    keyboard.release(Key.f5)
    wsh.SendKeys('{Enter}' + text + '{Enter}')
   
   
if __name__ == '__main__':


    text_to_type = 'hello world'
    openNotepad()
    sendKeys(text_to_type)

[Sh Ty]

By the way, I am using Python 3.7.7 on Windows 10, 64-bit.

[Elliot Garbus]

I have not tried to run your code but here are a few ideas.

 

While I don’t think this is related to your problem, you should change the name of the type() method.  It conflicts with the built-in type().

 

In sendKeys check for Notepad prior to activating. 

 

Replace your sleep calls using Clock callbacks.

See: https://kivy.org/doc/stable/api-kivy.clock.html?highlight=clock#module-kivy.clock

--
You received this message because you are subscribed to the Google Groups "Kivy users support" group.
To unsubscribe from this group and stop receiving emails from it, send an email to kivy-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/kivy-users/3df20bef-5c9e-4a58-a4ec-402c340231b7%40googlegroups.com.

 

[Robert Flatt]

time.sleep(0.5)

Does it work if you make this say 5 seconds?

If yes then Notepad just needs time to start.

But a sleep() in the UI thread is not a good idea, the whole UI will freeze, and Windows will tell you so.

In this case the start notepad functionality must be in a Thread with a callback on completion.

The callback executes the "#send the keystrokes" statements

To unsubscribe from this group and stop receiving emails from it, send an email to kivy-...@googlegroups.com.

[Sh Ty]

Thank you for all your help and advice.

It turned out that removing time.sleep after subprocess.Popen fixed the problem.

Not the best solution, I'm sure, but it somehow works.

This is beyond my current understanding, and if anyone could explain why, it would be greatly appreciated.

I still have so much to learn.

-----

Below are the modified codes:

typerApp.py

from kivy.app import App
from kivy.uix.floatlayout import FloatLayout
import typerScript


class MainScreen(FloatLayout):

    def openAndType(self, text):
        ##typerScript.openNotepad()     <- deleted

        typerScript.sendKeys(text)


class TyperApp(App): # define Base Class of Kivy App


    def build(self):
        root_widget = MainScreen()
        return root_widget
 
# run program
if __name__ == '__main__':
    TyperApp().run()

typer.kv

#:kivy 1.11.1


<MainScreen>
    BoxLayout:
        orientation: 'vertical'


        TextInput:
            id: text_to_type
            text: 'hello world'


        Button:
            text: 'Type'

            on_release: root.openAndType(text_to_type.text)

typerScript.py

import os
from pynput.keyboard import Key, Controller
import subprocess
import time
import win32com.client as comclt


keyboard = Controller()
wsh = comclt.Dispatch('WScript.Shell')


def openNotepad():


    # check if notepad is open
   
    print('checking for notepad')
    r = os.popen('tasklist').read()
   
    # open notepad if not opened
   
    if 'notepad.exe' not in r:
   
        print('opening notepad')
        subprocess.Popen('notepad', stdin = subprocess.PIPE, shell = True)

        ##time.sleep(0.5)          <- removed

   
    print('notepad is open')
       
def sendKeys(text):

    # open Notepad if not open
    openNotepad()                # <- added

   
    # activate notepad
    print('activating notepad...')
    wsh.AppActivate('無題') # name of new Notepad window in Japanese OS
    time.sleep(0.5)
   
    # send keystrokes
   
    print('sending keystrokes...')
    keyboard.press(Key.f5)
    keyboard.release(Key.f5)
    wsh.SendKeys('{Enter}' + text + '{Enter}')
   
   
if __name__ == '__main__':


    text_to_type = 'hello world'

    ##openNotepad()                <- removed
    sendKeys(text_to_type)

[Sh Ty]

I have now replaced sleep calls with Clock callbacks.

I'm not sure if I'm using it right, but at least it works!

Any further advice would be greatly appreciated. Thanks!

-----

Below are the modified codes:

typerApp.py

from kivy.app import App
from kivy.uix.floatlayout import FloatLayout
import typerScript

class MainScreen(FloatLayout):

    def openAndType(self, text):

        typerScript.text_to_type = text
        typerScript.openNotepad()

class TyperApp(App):

    def build(self):

        root_widget = MainScreen()
        return root_widget

if __name__ == '__main__':

    TyperApp().run()

typer.kv

#:kivy 1.11.1

<MainScreen>
    BoxLayout:
        orientation: 'vertical'

        TextInput:
            id: text_to_type
            text: 'hello world'

        Button:
            text: 'Type'

            on_release: root.openAndType(text_to_type.text)

typerScript.py

from kivy.clock import Clock

import os
from pynput.keyboard import Key, Controller
import subprocess
import time
import win32com.client as comclt

keyboard = Controller()
wsh = comclt.Dispatch('WScript.Shell')

text_to_type = ''

def openNotepad():

    # check if notepad is open
   
    print('checking for notepad')
    r = os.popen('tasklist').read()
   
    # open notepad if not opened
   
    if 'notepad.exe' not in r:
   
        print('opening notepad')
        subprocess.Popen('notepad', stdin = subprocess.PIPE, shell = True)

       
        if __name__ == '__main__':
            time.sleep(1)
            my_callback_1(0)
       
        else:
            Clock.schedule_once(my_callback_1, 1)
   
    else:

   
        print('notepad is open')

       
        if __name__ == '__main__':
            my_callback_1(0)
       
        else:
            Clock.schedule_once(my_callback_1, 0)

def my_callback_1(dt):

    print('Callback 1 called', dt)

   
    print('activating notepad...')
   
    wsh.AppActivate('無題') # name of new Notepad window in Japanese OS

    if __name__ == '__main__':
        time.sleep(1)
        my_callback_2(0)
       
    else:
        Clock.schedule_once(my_callback_2, 1)
       
def my_callback_2(dt):

    print('Callback 2 called', dt)
    sendKeys(text_to_type)
       
def sendKeys(text):

   
    # send keystrokes
   
    print('sending keystrokes...')
    keyboard.press(Key.f5)
    keyboard.release(Key.f5)
    wsh.SendKeys('{Enter}' + text + '{Enter}')
   
if __name__ == '__main__':

    text_to_type = 'hello world'
    openNotepad()

On Thursday, March 26, 2020 at 9:24:32 PM UTC+9, Sh Ty wrote:

[Robert Flatt]

Good work.

Though we don't need a Process and Shell here, a Thread will do the job

Sudo code:

import threading

def send_commands():
    # send keystrokes

def start_notepad(callback):

   wsh.AppActivate('無題') # name of new Notepad window in Japanese OS

   callback()  # run send_commands() after activate is done.

def open_notepad():      
      # get r

      if 'notepad.exe' not in r:

        threading.Thread(target=start_notepad,args=(send_commands)).start()  
      else:
         send_commands()


More generally spawing a shell is generally not portable, and a security risk. So it is a good habit to avoid doing this.

[Sh Ty]

Thank you for the kind advice.
I will keep it in mind from now on.