DroidScript library

[GOOD]

is it possible to connect OpenCV, AccessibilityService, GL context? And how? 

[GOOD]

I want to make a full-fledged bot application for recognizing images on the screen and click on them

вторник, 10 мая 2022 г. в 10:22:28 UTC+3, GOOD:

[Alan Hendry]

HI,

If you search the web for opencv nodejs you may find nodejs modules that fit the bill.

There are notes on how to use DS and a nodejs module to read image data (exif)  http://droidscript.org/wiki/doku.php?id=plugins:notesonexif

And a video on using DS and a nodejs module to control a smart plug.

Regards, ah

[GOOD]

tell me how to implement image search and click on the found ones. I have no idea how to use the libraries and how to download them. can I have an example, please, and at the same time how to download them.

вторник, 10 мая 2022 г. в 14:05:15 UTC+3, hendr...@gmail.com:

[Alan Hendry]

HI,

The example on the wiki and the video include how to create the correct type of project/app in DS,

and download the library.  

The web pages (generally on npmjs) usually include examples/documentation. 

DS cameraview can do face detection, and there have been posts about face recognition

https://groups.google.com/g/androidscript/c/TAER5jK27jU/m/fTGCyiwXBAAJ

https://groups.google.com/g/androidscript/c/Hn2sLe4GEn8/m/8fp08ue7AAAJ

https://groups.google.com/g/androidscript/c/PnZ3FOWVNZc/m/XsdEv8o5BgAJ

https://groups.google.com/g/androidscript/c/ZqSg89tpxJ8/m/kAAB-qO8AQAJ

Regards, ah

[GOOD]

I don't need to recognize it on the camera, but on the phone screen-a significant difference

вторник, 10 мая 2022 г. в 17:27:03 UTC+3, hendr...@gmail.com:

[GOOD]

//Configure app to use NodeJS as the main scripting engine
//giving you the full power of Node directly in your app!
cfg.Node

//Configure for Material UI and light theme.
cfg.MUI, cfg.Light

//Make sure the required node modules are installed to ide.
//(This downloads modules from https://www.npmjs.com).
ide.AddModule( "exif" )

//Called when application is started.
function OnStart()
{
    //Set MUI primary color.
    app.InitializeUIKit( MUI.colors.teal.teal, "Light" )
    //Create a MUI card layout.
    lay = MUI.CreateLayout("Linear", "VCenter,FillXY")

path = app.RealPath(app.GetAppPath())+'/Img/world.jpeg'
//alert(path)
uri = app.Path2Uri(path)

var ExifImage = require('exif').ExifImage
 try {
    new ExifImage({ image : path }, function (error, exifData) {
        if (error)
            alert('Error: '+error.message);
        else
            alert(JSON.stringify(exifData));
    })
} catch (error) {
    alert('Error: ' + error.message);
}

    //Add main layout to app.
    app.AddLayout(lay)

}

I tried this code, but an error came out

here are links that may help you
https://www.npmjs.com/search?q=image%20search

вторник, 10 мая 2022 г. в 18:20:00 UTC+3, GOOD:

[GOOD]

https://www.npmjs.com/package/canvas

вторник, 10 мая 2022 г. в 22:29:01 UTC+3, GOOD:

[GOOD]

https://pastebin.com/1mh2ZAT4

вторник, 10 мая 2022 г. в 22:42:50 UTC+3, GOOD:

[GOOD]

I'm sorry, please. but it doesn't reach me. I can't understand, please explain. he still does not look for an image on the phone screen, even the permission does not require that the user would allow droidscript to control the screen and view information on it

вторник, 10 мая 2022 г. в 22:52:28 UTC+3, GOOD:

[Alan Hendry]

HI,

Photos contain additional data (date, time, place, camera, etc) that is known as exif data.

The example uses a nodejs module to extract the exif data from a photo.

Pretty sure the image you tried had no exif data.

This is just an example of how to get a nodejs package and use it.

DS itself doesn't include image recognition, but they may be a nodejs package out there that does what you need.

 Not sure what you want

" bot application for recognizing images on the screen"

" image search and click on the found ones"

https://www.npmjs.com/package/canvas  for drawing on the screen

https://pastebin.com/1mh2ZAT4 says "Java Screen Scanner"  (?)

https://www.npmjs.com/search?q=image%20search  seems to be the result of searching npmjs for image search 

195 packages including  using google/bing/duckduckgo to find images, subimage within image, etc

Regards, ah

[GOOD]

my plan is this: click the button in the DS application and, due to the downloaded library, it will start looking for an image on the screen and if it finds it, it clicks on it. but I don't understand how to download libraries and how to use them, it would be good if there were concrete examples of searching and clicking on the found images in DS, as well as the implementation of these libraries in DS. the image recognition consists of several parts: 1-screen screen (probably), 2-pixel search (correlations), 3-comparing a picture from a template with a screenshot, 4-getting the coordinates of the found image, 5-the ability to click

среда, 11 мая 2022 г. в 00:51:10 UTC+3, hendr...@gmail.com:

[GOOD]

please, you can search for an example and if you find it, then click on this image. I have no idea where to insert these codes from the site, maybe in a file with the node exif folder?

среда, 11 мая 2022 г. в 09:21:57 UTC+3, GOOD:

[Alan Hendry]

HI,

The ability to use nodejs (fairly easily) is pretty new in DS, so there are few examples.

There are loads of nodejs modules for lots of tasks.

Not sure where image 1 is coming from and where image 2 is coming from.

https://www.npmjs.com/search?q=image%20search%20google

shows modules that can search Google for images.

https://www.npmjs.com/search?q=reverse%20image

shows modules that can search for similar images. 

https://www.npmjs.com/search?q=subimage%20search 

shows 3 nodejs packages that can search for one (sub) image within another image

"Fuzzy" means that the images don't have to match exactly.

Regards, ah 

[GOOD]

1) an image is taken from the phone screen. 2)an image is taken from a folder of any path, for example, /sdcard. and then he searches for the image (2) on the phone screen

среда, 11 мая 2022 г. в 13:19:27 UTC+3, hendr...@gmail.com:

[GOOD]

Please, integrate ds

https://github.com/monai/node-imagesearch/blob/develop/lib/imagesearch.js

среда, 11 мая 2022 г. в 13:40:22 UTC+3, GOOD:

[Alan Hendry]

HI,

Do you mean a screen shot for (1) ?

Would the user choose image (2) on their phone, or would it try to search all images on the phone?

Regards, ah

[GOOD]

the function of downloading an image from a folder with files is performed. then the image search function is called on the entire screen of the android phone (most likely, to search from the phone screen, you will need to take a screenshot and save it to the buffer to compare with the template)

среда, 11 мая 2022 г. в 14:22:48 UTC+3, hendr...@gmail.com:

[GOOD]

this means SEARCHING FOR IMAGES in other (separate applications)

среда, 11 мая 2022 г. в 15:34:56 UTC+3, GOOD:

[GOOD]

Source: https://github.com/strarsis/node-opencv-templatematching-test/find/master

I edited all the files in DS that were there in the source code, and even the code, but it doesn't work. Not result. 

Download test2.spk: https://transfiles.ru/jy0i6

[GOOD]

+++ide.AddModule("opencv"); code in test2.spk

Not work image search 

четверг, 12 мая 2022 г. в 12:06:00 UTC+3, GOOD:

[Alan Hendry]

HI,

ide.AddModule( "exif" )  gets the nodejs module for exif, 

you need to change it to the nodejs module that you want  ide.AddModule( "opencv" )

The first time you run it it should prompt you for the installation of the package

(the package should then appear in your project/app in a subdirectory called node_modules)

Your code uses the RealPaths of path and path2 (for exif I had to use Path2Uri of the RealPath)

Regards, ah

[GOOD]

Not work. I understand, please. Help correct. My English is bad and translator. 

четверг, 12 мая 2022 г. в 17:08:09 UTC+3, hendr...@gmail.com:

[GOOD]

what other libraries does DS support besides cfg.Node?

четверг, 12 мая 2022 г. в 18:43:58 UTC+3, GOOD:

[GOOD]

these libraries are only downloaded, they don't even work. no one checks them, because they are updated and changed

четверг, 12 мая 2022 г. в 21:56:49 UTC+3, GOOD:

[GOOD]

How usage Python in DS? 

четверг, 12 мая 2022 г. в 23:16:08 UTC+3, GOOD:

[GOOD]

https://towardsdatascience.com/object-detection-on-python-using-template-matching-ab4243a0ca62

пятница, 13 мая 2022 г. в 12:35:40 UTC+3, GOOD:

[GOOD]

unfortunately, I don't see any prospects other than switching to the sdk. no help. few examples. not enough features

пятница, 13 мая 2022 г. в 12:41:08 UTC+3, GOOD:

[Alan Hendry]

HI,

You can write DS plugins using platforms like AIDE or Android Studio. 

Javascript libraries can be used in HTML projects/apps (and in WebViews in Native DS projects/apps).

As far I'm aware you'd have to put your Python code on a server and "talk" to it using DS.

If there are problems with nodejs packages then npm usualy has contact details for the developer. 

It looks like test.SPK does a readImage to get the image from variable path

I think you need to use a readImage to get the image from variable path2

(Not sure what the second parameter (3) in the match is for)

It may be easier to use something like https://www.npmjs.com/package/fast-template-matcher

which allows you to specify source and template filenames (may need to be realpaths and/or path2uri) 

Or perhaps https://www.npmjs.com/package/matches-subimage

Regards, ah

[GOOD]

thanks for your reply. I'll try it now. it's a pity that this is a theory and you could help me by working together.

пятница, 13 мая 2022 г. в 18:32:18 UTC+3, hendr...@gmail.com: