Mouse click relative to current window's dimensions?

[JC]

In AutoHotkey, it's possible to set pixels according to the current window's absolute dimensions, not the entire monitor's. Is this possible in AutoKey? I found the following info on StackOverflow:

mouse.click_relative_self(x, y, left/middle/right)

mouse.click_absolute(x, y, 1/2/3)

I'm assuming "absolute" here refers to the entire monitor. Is there a way to get mouse.click_window() going?

[Little Girl]

Hey there,

JC wrote:

>In AutoHotkey, it's possible to set pixels according to the current
>window's absolute dimensions, not the entire monitor's. Is this
>possible in AutoKey?

Here are some notes I took on mouse clicks that might help. Note that
there's one ??? in there, but the rest is solid and tested:

====================

Send a mouse click relative to the screen:

mouse.click_absolute(x, y, button)

* The x is the x-coordinate in pixels from the left on the horizontal
axis relative to the upper left corner of the screen.
* The y is the y-coordinate in pixels from the top on the vertical
axis relative to the upper left corner of the screen.
* The button is the kind of mouse button to emulate (1, 2, or 3 to
represent left, middle, or right, accepting up to 9 buttons for
fancy mice).

====================

Send a mouse click relative to the active window:

mouse.click_relative(x, y, button)

* The x is the x-coordinate in pixels from the left on the horizontal
axis relative to the upper left corner of the active window
* The y is the y-coordinate in pixels from the top on the vertical
axis relative to the upper left corner of the active window.
* The button is the kind of mouse button to emulate (1, 2, or 3 to
represent left, middle, or right, accepting up to 9 buttons for
fancy mice).

====================

Send a mouse-click relative to the mouse's current position:

mouse.click_relative_self(x, y, button)

* The x is the x-coordinate in pixels from the left on the horizontal
axis relative to the mouse's current position.
* The y is the y-coordinate in pixels from the top on the vertical
axis relative to the mouse's current position.
* The button is the kind of mouse button to emulate (1, 2, or 3 to
represent left, middle, or right, accepting up to 9 buttons for
fancy mice).

Example - perform a right-click at the mouse's current location:

mouse.click_relative_self(0, 0, 3)

Example - perform a middle-click at the mouse's current location:

mouse.click_relative_self(0, 0, 2)

Example - perform a left-click at the mouse's current location:

mouse.click_relative_self(0, 0, 1)

Example - perform a left-click 20 pixels to the right and 100 pixels
down from the mouse's current location:

mouse.click_relative_self(20, 100, 1)

Example - perform a left-click 20 pixels to the left(???) and 100
pixels down from the mouse's current location:

mouse.click_relative_self(-20, 100, 1)

--
Little Girl

There is no spoon.

[JC]

Sweet, thanks! And my bad, it was on Ask Ubuntu, not Stack Overflow.

[JC]

Okay, my next question is: is there a free tool we could use to quickly find XY dimensions for the given active window? I'm realizing AutoHotkey was quite spoiling of its users lol with the built-in WindowSpy...

On Friday, July 23, 2021 at 5:37:20 PM UTC-5 JC wrote:

Oh wait, nvm; you did that lol. That's it!

On Friday, July 23, 2021 at 5:37:00 PM UTC-5 JC wrote:

Hmm, I can't seem to edit my messages here. I think to go relatively left you'd need a negative value; in that case, -20.

[Little Girl]

Hey there,

JC wrote:

>Okay, my next question is: is there a free tool we could use to
>quickly find XY dimensions for the given active window? I'm
>realizing AutoHotkey was quite spoiling of its users lol with the
>built-in WindowSpy...

Yep. The AutoKey API documentation discusses the API calls you can
use on windows on its https://autokey.github.io/index.html page.

I cobbled this together and it works for me. It returns a tuple
containing the x-origin, y-origin, width, and height of the active
window in pixels and displays the result in a dialog. If it doesn't
work for you, there are things we can try:

```
winGeom = window.get_active_geometry()
dialog.info_dialog("Window geometry", "Active window geometry:\n\n'%s'" % winGeom)
```

[Little Girl]

Hey there,

JC wrote:

>Oh wait, nvm; you did that lol. That's it!

Heh. That takes care of the ??? in my notes. Thanks for testing.

[jos...@main.nc.us]

This looks like a wiki page to me. :)

Joe

> --
> You received this message because you are subscribed to the Google Groups
> "autokey-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to autokey-user...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/autokey-users/60fb4201.1c69fb81.f48f9.e469%40mx.google.com.
>

[Little Girl]

Hey there,

jos...@main.nc.us wrote:

>This looks like a wiki page to me. :)

Heh. That's not even all of it. My copy of those notes also contain
instructions for waiting for a mouse click and pressing and holding a
mouse button. Maybe I'll turn it all into a wiki page, although it's
already got mentions in the wiki and in the AutoKey documentation:

The mouse API is covered on the "API Examples" page in the wiki:

https://github.com/autokey/autokey/wiki/API-Examples#mouse

There are also details on this GitHub page under the mouse class:

https://autokey.github.io/

It might be better to add a few more details to the existing examples
on the wiki page above than to create a new one. What do you think?

[JC]

Wow, interesting! That script did work for its purposes, but sorry, I meant the dimensions of the mouse cursor inside the given active window.

1. I'm trying to get the mouse to auto-click the "Set" button (to set abbreviations in AutoKey) for the given phrase/window. I can't seem to figure out the button's X/Y dimensions; this is what I'm trying to pinpoint.
2. I did this only as a test but already found it really weird that mouse.click_relative(10, 10, 1), when used in the AutoKey window itself, just highlights "File" in the upper-left but doesn't actually drop down the File menu, nor does this function move the mouse cursor at all. Is there any other method that literally simulates a hand moving the cursor to the targeted spot and clicking, the way AutoHotkey does with MouseClick?
   

Thanks for your help thus far!

[jos...@main.nc.us]

Apparently, it's split into several places, so unifying it into a tutorial
or summary article would probably be good. It could include links to
anything you don't want to duplicate or significantly improve/expand.

It might be cool to at least mention some of the things that AutoKey can't
do that the standalone packages can. One that comes to mind (because I use
it a lot elsewhere) is dialogs that time out. I use them for information
pop-ups that don't require a response. They could also be used for
questions that have default answers.

Another one is custom buttons, some of which have custom icons. I have a
dialog that lets you use an optional feature or just proceed, so its
buttons are OK and Skip. You can also just specify one button for OK and
then no Cancel button is displayed.

This brings up a possible enhancement for AutoKey. We could add an extra
optional field or list to the dialog API calls that passes raw dialog
manager options strings for the dialog manager to interpret. This would
let you do anything the dialog manager can handle as long as there's an
internal way for AutoKey to pass the raw options to the dialog manager.

Cross posting this on Gitter.

Joe

> --
> You received this message because you are subscribed to the Google Groups
> "autokey-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to autokey-user...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/autokey-users/60fc74aa.1c69fb81.3ef37.78b5%40mx.google.com.
>

[Little Girl]

Hey there,

JC wrote:

>Wow, interesting! That script did work for its purposes, but sorry,
>I meant the dimensions of the mouse cursor inside the given active
>window.

This should do it. It's a script by Sam Sebastian that displays the
mouse coordinates in case you need them for anything. I'm not sure
where I got it from. Maybe in the Gitter chat or on Stack Exchange. I
can't currently find it in the wiki:

from Xlib import X, display
d = display.Display().screen().root.query_pointer() #get pointer
location x = str(d.root_x) # get x coord and convert to string
y = str(d.root_y) # get y coord and convert to string
dialog.info_dialog("(X, Y)", x+", "+y) #create an info dialog to
display the coordinates

> 1. I'm trying to get the mouse to auto-click the "Set" button (to

> set abbreviations in AutoKey) for the given phrase/window. I can't
> seem to figure out the button's X/Y dimensions; this is what I'm
> trying to pinpoint.

That script above should do it if you hover over the button while
triggering the script.

> 2. I did this only as a test but already found it really weird that

> mouse.click_relative(10, 10, 1), when used in the AutoKey window
> itself, just highlights "File" in the upper-left but doesn't
> actually drop down the File menu, nor does this function move the
> mouse cursor at all. Is there any other method that literally
> simulates a hand moving the cursor to the targeted spot and
> clicking, the way AutoHotkey does with MouseClick

> <https://www.autohotkey.com/docs/commands/MouseClick.htm>?

This "Autoclicker with toggle using Globals" script by Kreezxil might
be useful:

https://github.com/autokey/autokey/wiki/Contributed-Scripts-2#autoclicker-with-toggle-using-globals

====================

These ways of doing a press-and-hold (which isn't a default feature
of AutoKey) might also be useful:

Someone here rebound the mouse button as a work-around:

https://askubuntu.com/questions/1223158/how-do-i-hold-down-left-click-in-autokey

Someone here suggested that you would need more low-level tools than
AutoKey to accomplish this:

https://askubuntu.com/questions/1104340/autokey-question-how-can-i-hold-down-a-simulated-keypress-while-holding-down-th

Someone here remapped the Z key to the left mouse button as a
work-around:

https://github.com/autokey/autokey/issues/243

Another person on the same page suggested this as a work-around if
you have xdotool installed:

import subprocess
subprocess.run(["xdotool", "mousedown", "1"])
# Click the mouse button to turn it off.

====================

Last, but not least, AutoKey is great at pressing keys and sending
characters pretty much anywhere you like. Menus can be controlled by
key-presses, so you can get around this that way if you like and
even have the part of the script that presses the keys wait for the
mouse click before doing so:

# Wait for left-click before running your code:
mouse.wait_for_click(1)
# your code here

# Wait for middle-click before running your code:
mouse.wait_for_click(2)
# your code here

# Wait for left-click before running your code or run it when the
timer runs out if no left-click occurs:
mouse.wait_for_click(1, timeOut=3.0)
# your code here

>Thanks for your help thus far!

Any time. I enjoy messing around with this stuff.

[Little Girl]

Hey there,

jos...@main.nc.us wrote:

>Apparently, it's split into several places, so unifying it into a
>tutorial or summary article would probably be good. It could include
>links to anything you don't want to duplicate or significantly
>improve/expand.

Okay. I'll see what I can do and you're welcome to jump in and
improve it or add to it.

>It might be cool to at least mention some of the things that AutoKey
>can't do that the standalone packages can. One that comes to mind
>(because I use it a lot elsewhere) is dialogs that time out. I use
>them for information pop-ups that don't require a response. They
>could also be used for questions that have default answers.

Whoops. This is a thread about mouse buttons. I think this post
belonged in the "Can the GUI support checkboxes" thread, but I don't
know how to move it there, so I'm responding here.

>Another one is custom buttons, some of which have custom icons. I
>have a dialog that lets you use an optional feature or just proceed,
>so its buttons are OK and Skip. You can also just specify one button
>for OK and then no Cancel button is displayed.
>
>This brings up a possible enhancement for AutoKey. We could add an
>extra optional field or list to the dialog API calls that passes raw
>dialog manager options strings for the dialog manager to interpret.
>This would let you do anything the dialog manager can handle as long
>as there's an internal way for AutoKey to pass the raw options to
>the dialog manager.

I believe all of this is already possible. If you go to the
https://autokey.github.io/index.html page and click the
lib.scripting.GtkDialog link in the left pane, you can look over the
examples and see **kwargs as part of the code examples. Those are for
any additional arguments that you can optionally add to the dialog
call as comma-separated strings (note that they *must* be passed as
strings).

Here's an example dialog that uses two of the default API arguments:

dialog.info_dialog(title="foo", message="bar")

Here's an example dialog that uses two of the default API arguments
and the custom "width" argument:

dialog.info_dialog(title="foo", message="bar", width="400")

Here's an example dialog that uses two of the default API arguments
and the custom "height" and "width" arguments:

dialog.info_dialog(title="foo", message="bar", height="300",
width="400")

[JC]

Cool! So I'm assuming that any script that imports foo has also imported time, right?

[Little Girl]

Hey there,

JC wrote:

>Cool! So I'm assuming that any script that imports foo has also
>imported time, right?

Not in the above examples, but yes, if foo is importable (it must be
in the folder that you specify in the AutoKey settings or be where
your system keeps Python modules) and if foo imports the time module,
then any script that imports foo will have imported the time module,
too.

If you'd rather not specify and use a folder to store importable
scripts or if you'd rather not put them where your system keeps
Python modules (I don't bother doing either of those), you can use the
AutoKey API to run any AutoKey script from within another AutoKey
script, which is effectively the same thing.

As an example, the foo script could contain this line:

import time

The bar script could contain these lines to run the foo script, use
the time module by setting and activating a sleep delay, and then
print a message to the screen afterwards:

engine.run_script("foo")
time.sleep(5)
keyboard.send_keys("foo")

Note that I've had trouble getting this to work when the script I
want to run has more than one word in its name, but I've been using a
much older version of AutoKey that hasn't had some of its bugs ironed
out. Your mileage may vary. If you run into trouble, try using
single-word names for your helper scripts (the ones you intend to run
from other scripts).

[jos...@main.nc.us]

I know it was out of place. I got lazy.

Cool. I'm sure you're right. I'll have to try it out and make a few examples.

Joe

> --
> You received this message because you are subscribed to the Google Groups
> "autokey-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to autokey-user...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/autokey-users/60fd86b1.1c69fb81.067b.4a91%40mx.google.com.
>

[Little Girl]

Hey there,

jos...@main.nc.us wrote:

>I know it was out of place. I got lazy.

No worries. I joined you in talking about it here.

>Cool. I'm sure you're right. I'll have to try it out and make a few
>examples.

I'll be interested to hear how you get along with that. I took a
quick look at the Gitter chat and saw the examples you gave. I'm able
to successfully use timeout="10" in my dialogs, but I haven't been
able to get button="gtk-ok:0" to work, although that might be reliant
on a type of dialog that AutoKey doesn't offer (I'm pretty sure
AutoKey doesn't offer all of them) or a type that I wasn't testing
at the time. More testing will be happening.

[jos...@main.nc.us]

When you're trying new things like custom buttons, it would probably be
better to make sure they work for you outside of AutoKey first.

I don't think I had to do anything special to make them available, but
I've been using them for years and may have forgotten about some helper
package I had to install.

I use them from within yad - although I don't think that should make a
difference. Yad does use GTK, so I don't know if any of that stuff is
available from Qt if you're using that front end.

Joe

Here are my Button notes:

gtk-close Close
gtk-cancel Cancel
gtk-ok Ok
gtk-file File
gtk-print Print
gtk-print-preview Print Preview
gtk-quit Quit
gtk-remove Remove
gtk-stop Stop

Some of the rest don't have nice button text


rest are in gtk-demo

To get that,
sudo apt-get install gtk2.0-examples

Then run gtk-demo
go down to Stock Item and Icon Browser
and double click on the entry
Then clicking on any item shows it's text and icon

> --
> You received this message because you are subscribed to the Google Groups
> "autokey-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to autokey-user...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/autokey-users/60ff1655.1c69fb81.fc46c.37a0%40mx.google.com.
>

[jos...@main.nc.us]

I researched this a little bit.

dialog.info_dialog(title="Information", message="junk works", width="300")

bombs in autokey-qt, but works fine in autokey-gtk.

I had to put the 300 in quotes to avoid an exception.

So, you have to use options that the dialog manager likes - which means,
among other things, that you have to detect which front end is running to
know what options you can use.

Is there something like a man kdialog somewhere? I found tutorials, etc.,
but nothing that just lists dialogs and their available options.

I also saw something where the first two arguments to kdialog were the
width and height for the box, but with no explanations, etc.

I'm going to ask them.

I love KDE, but I hate their documentation!

Joe

> --
> You received this message because you are subscribed to the Google Groups
> "autokey-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to autokey-user...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/autokey-users/60ff1655.1c69fb81.fc46c.37a0%40mx.google.com.
>

[jos...@main.nc.us]

This is the answer I got:

Re: Is there something like a man page for kdialog? Topic is solved
Tue Jul 27, 2021 12:47 pm
kdialog --help is probably the closest.

But maybe :
https://man.cx/kdialog
https://develop.kde.org/deploy/kdialog/

claydoh, proud to be a member of KDE forums since 2008-Oct, and KDE user
since 2001

> https://groups.google.com/d/msgid/autokey-users/15bdb639bae93d3338278feb444b2ddf.squirrel%40main.nc.us.
>

[Little Girl]

Hey there,

jos...@main.nc.us wrote:

>I researched this a little bit.
>
>dialog.info_dialog(title="Information", message="junk works",
>width="300")
>
>bombs in autokey-qt, but works fine in autokey-gtk.

Yep. I don't know why. Take off the width option and it will work in
AutoKey (Qt):

dialog.info_dialog(title="foo", message="bar")

I tried using geometry to specify the size, but haven't managed it
yet. All I've been able to do is control its position with partial
geometry:

dialog.info_dialog(title="foo", message="bar", geometry="+300+400")

This version works just like the previous one and puts the dialog in
the same place on the screen, but doesn't affect its size:

dialog.info_dialog(title="foo", message="bar",

geometry="100x200+300+400")

All other efforts to use geometry (bad syntax: 100+200+300+400 or
+100+200+300+400 or 100x200x300x400 or x100x200x300x400) failed. You
might have better luck with some of your experiments.

I have to say that it would have been nice if the developers had put
real-world examples in for all of these API calls rather than just
giving us partial usage information to decipher.

If it helps any, I used the current LTS of Kubuntu and AutoKey (Qt)
version 0.95.10 for these tests.

>I had to put the 300 in quotes to avoid an exception.

Yes, always. AutoKey will only ever accept strings as options for
dialogs, so you'll have to get creative at times when testing
options that aren't documented that might just happen to work.

>So, you have to use options that the dialog manager likes - which
>means, among other things, that you have to detect which front end
>is running to know what options you can use.

Yes, that seems to be a bit of a problem.

We should probably clearly distinguish between them by being very
clear which front end each is for:

gtkdialog.info_dialog(title="foo", message="bar")

vs

kdialog.info_dialog(title="foo", message="bar")

In the meanwhile (or if that's not advisable), we'll certainly want
to comment our code to remind ourselves which front end a script with
dialogs is intended for or at least brace ourselves for occasional
failure as the result of choosing the wrong front end for a script.

>Is there something like a man kdialog somewhere? I found tutorials,
>etc., but nothing that just lists dialogs and their available
>options.

No man page, oddly enough, but kdialog --help is somewhat useful.
Then there's always my blog page, but I may have overlooked something
and it still needs me to finish the huge overhaul I'm doing on it.
It's mostly done, but not fully, so I've got a heavily-updated copy
on my drive, but the old one is still online. Perhaps this is the
kick in the pants I need to get that done.

>I also saw something where the first two arguments to kdialog were
>the width and height for the box, but with no explanations, etc.

Yep, that's geometry. It works like this in a regular KDialog:

--geometry 100x200+300+400

I believe that's:

--geometry[HEIGHT]x[WIDTH]+[X-OFFSET]+[Y-OFFSET]

But don't quote me on that. I didn't test it just now (I'm not
currently in KDE) and the width and height could be backwards.

You can also break it up to specify just the height and width or just
the x and y offsets:

--geometry 100x200
--geometry 300+400

The AutoKey option should accept this, but it ignores the height and
width and only honors the x and y offsets:

geometry="100x200+300+400"

If you then dare to add a width option to the command, the script
will silently fail.

>I'm going to ask them.

I'll be interested in their reply, especially if they say why they
never made it a man page.

>I love KDE, but I hate their documentation!

Same here, but creating it will be a big job.

[Little Girl]

Hey there,

jos...@main.nc.us wrote:

>https://man.cx/kdialog

Aha. That explains it. I had no idea there was a Qt version and a KDE
version of KDialog. Learn something new every day.

It seems that there is no geometry option in the Qt version of
KDialog. We have hit a wall.

>https://develop.kde.org/deploy/kdialog/

Interesting. They didn't demonstrate the use of geometry at all.

[jos...@main.nc.us]

If it was hard to program, it should be hard to understand! LOL

The geometry example was in the other link and was a couple of raw numbers
for box width and height with no flag, etc.

I haven't looked into it (and probably won't unless challenged), but it
should not be too hard to figure out which desktop library set =(Qt or
GTK) is in use at run time and choose the appropriate arguments/calling
sequence. I'm just not sure anybody needs to get that fancy.

Joe

>
> --
> Little Girl
>
> There is no spoon.
>

> --
> You received this message because you are subscribed to the Google Groups
> "autokey-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to autokey-user...@googlegroups.com.
> To view this discussion on the web visit

> https://groups.google.com/d/msgid/autokey-users/61046971.1c69fb81.59945.afec%40mx.google.com.
>

[Little Girl]

Hey there,

jos...@main.nc.us wrote:

>When you're trying new things like custom buttons, it would probably
>be better to make sure they work for you outside of AutoKey first.

Yeah, although the syntax is different from there to inside of
AutoKey.

>I don't think I had to do anything special to make them available,
>but I've been using them for years and may have forgotten about some
>helper package I had to install.

If you've got them working in AutoKey, I'd love to see a few of them.

>I use them from within yad - although I don't think that should make
>a difference. Yad does use GTK, so I don't know if any of that stuff
>is available from Qt if you're using that front end.

It's not, but I'll probably end up with some GTK files in my copy of
KDE since I have some GTK software I can't live without.

>Here are my Button notes:
>
>gtk-close Close
>gtk-cancel Cancel
>gtk-ok Ok
>gtk-file File
>gtk-print Print
>gtk-print-preview Print Preview
>gtk-quit Quit
>gtk-remove Remove
>gtk-stop Stop

Thanks. None of those will work in AutoKey because AutoKey sees them
as expressions because of the dash. It triggers an invalid syntax
warning.

>Some of the rest don't have nice button text

Yeah, I noticed that, too. I wonder why they didn't make everything
available to all of them.

[Little Girl]

Hey there,

jos...@main.nc.us wrote:

> Little Girl wrote:
>> jos...@main.nc.us wrote:

>>>https://man.cx/kdialog
>>
>> Aha. That explains it. I had no idea there was a Qt version and a
>> KDE version of KDialog. Learn something new every day.
>>
>> It seems that there is no geometry option in the Qt version of
>> KDialog. We have hit a wall.

>If it was hard to program, it should be hard to understand! LOL

That explains all the cryptic documentation out there.

>>>https://develop.kde.org/deploy/kdialog/
>>
>> Interesting. They didn't demonstrate the use of geometry at all.

>The geometry example was in the other link and was a couple of raw
>numbers for box width and height with no flag, etc.

Yep, and I was surprised that the developers didn't include at least
one example of it on their page, too.

>I haven't looked into it (and probably won't unless challenged), but
>it should not be too hard to figure out which desktop library set
>=(Qt or GTK) is in use at run time and choose the appropriate
>arguments/calling sequence. I'm just not sure anybody needs to get
>that fancy.

I started to and didn't get very far, but it should be doable.