Please add Windows 10 Desktop Duplication API support

233 views
Skip to first unread message

Davide Perini

unread,
May 29, 2020, 8:42:01 AM5/29/20
to Java Native Access
### Provide complete information about the problem
1. Version of JNA and related jars = JNA 5.5.0
2. Version and vendor of the java virtual machine = JDK 11
3. Operating system = Windows 10
4. System architecture (CPU type, bitness of the JVM) = i7 5930K 6 cores
5. Complete description of the problem

Windows 8 introduces Desktop Duplication API, (improved with Windows 10) that allow for fast screen capturing without any lag.

Will you ever support this API?
Is there a way to capture a screenshot using JNA and Windows Desktop Duplication API?

Matthias Bläsing

unread,
May 29, 2020, 1:55:18 PM5/29/20
to jna-...@googlegroups.com
Hi Davide,

Am Freitag, den 29.05.2020, 05:42 -0700 schrieb Davide Perini:
> Windows 8 introduces Desktop Duplication API, (improved with Windows
> 10) that allow for fast screen capturing without any lag.
>
> Will you ever support this API?

JNA is a community project. It supports what people contributed. I came
to project because I needed COM access and decided to contribute
fixes/enhancements. Feature requests have the tendency to remain
unfixed.

So it can be turned around: If you bind it and contribute the bindings,
it will be supported :-)

Greetings

Matthias

Davide Perini

unread,
May 29, 2020, 2:17:08 PM5/29/20
to Java Native Access
Is this a confirmation that there is no support for Desktop Duplication API in the current state?

Thanks :)

Daniel B. Widdis

unread,
May 29, 2020, 2:21:34 PM5/29/20
to Java Native Access
Davide,

There is "support" in the case that the base JNA functionality will easily allow you to create an interface and map the functions.

Just because someone hasn't mapped a function yet doesn't mean it's "not supported".

On Fri, May 29, 2020 at 11:17 AM Davide Perini <sblan...@gmail.com> wrote:
Is this a confirmation that there is no support for Desktop Duplication API in the current state?

Thanks :)

--
You received this message because you are subscribed to the Google Groups "Java Native Access" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jna-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jna-users/d53241de-c682-43ae-9cd4-c3eadce0984d%40googlegroups.com.


--
Dan Widdis

Davide Perini

unread,
May 29, 2020, 7:30:26 PM5/29/20
to Java Native Access
I don't know JNA so well, it's not that easy for me at the moment :D


Il giorno venerdì 29 maggio 2020 20:21:34 UTC+2, Daniel B. Widdis ha scritto:
Davide,

There is "support" in the case that the base JNA functionality will easily allow you to create an interface and map the functions.

Just because someone hasn't mapped a function yet doesn't mean it's "not supported".

On Fri, May 29, 2020 at 11:17 AM Davide Perini <sblan...@gmail.com> wrote:
Is this a confirmation that there is no support for Desktop Duplication API in the current state?

Thanks :)

--
You received this message because you are subscribed to the Google Groups "Java Native Access" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jna-...@googlegroups.com.


--
Dan Widdis

Daniel B. Widdis

unread,
May 29, 2020, 8:05:19 PM5/29/20
to Java Native Access
This is a perfect time to learn JNA!   Mapping is straightforward.  Declare an interface for the DLL you want to load (in this case, DXGI) and load the library.

Create a class Dxgi.java:

public interface Dxgi extends Library {
    Dxgi INSTANCE = Native.load("Dxgi", Dxgi.class, W32APIOptions.DEFAULT_OPTIONS);
    
    // structure and function mappings here
}

Here's a function mapping for AcquireNextFrame:

HRESULT AcquireNextFrame(int TimeoutInMilliseconds, DXGI_OUTDUPL_FRAME_INFO pFrameInfo, PointerByReference ppDesktopResource);

That requires the DXGI_OUTDUPL_FRAME_INFO structure, map like this:

    @FieldOrder({ "LastPresentTime", "LastMouseUpdateTime", "AccumulatedFrames", "RectsCoalesced",
            "ProtectedContentMaskedOut", "PointerPosition", "TotalMetadatBufferSize", "PointerShapeBufferSize" })
    class DXGI_OUTDUPL_FRAME_INFO extends Structure {
        public LARGE_INTEGER LastPresentTime;
        public LARGE_INTEGER LastMouseUpdateTime;
        public int AccumulatedFrames;
        public boolean RectsCoalesced;
        public boolean ProtectedContentMaskedOut;
        public DXGI_OUTDUPL_POINTER_POSITION PointerPosition;
        public int TotalMetadataBufferSize;
        public int PointerShapeBufferSize;

        // may need constructor if you will create from a Pointer
    }

That requires a DXGI_OUTDUPL_POINTER_POSITION structure, map similarly, repeat nested mappings as necessary.

Look at almost any random class in the win32 platform mappings and you'll see ample examples:

Try and get started and feel free to ask here for help if you get stuck.


To unsubscribe from this group and stop receiving emails from it, send an email to jna-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jna-users/62ed9f37-22c5-4f1a-b8cd-94ceb2b740e1%40googlegroups.com.


--
Dan Widdis

Davide Perini

unread,
May 31, 2020, 1:21:49 PM5/31/20
to jna-...@googlegroups.com
Hi,
thanks for the kind answer, I really appreciate it.

I would really like to learn JNA and give my contribute but at the moment I need to be sincere, I'm not able to do it and I don't have time to learn it without some good docs that shows the basics with step by step examples for simple APIs.

At the moment, I don't even know how to compile the project :)
Is this possible to do it via IntelliJ? Is there some guide on how to build using IntelliJ?

I will continue to look into the project and probably I will understand how it works one day and how to contribute.

Thank you.
Davide

Daniel Widdis

unread,
May 31, 2020, 2:33:43 PM5/31/20
to jna-...@googlegroups.com

I’d recommend using Maven for dependency management.  Pick one of many online tutorials for setting up a Maven project in IntelliJ, and then add the JNA project’s artifacts as dependencies:

 

                                <dependencies>

                                                <dependency>

                                                                <groupId>net.java.dev.jna</groupId>

                                                                <artifactId>jna</artifactId>

                                                                <version>5.5.0</version>

                                                </dependency>

                                                <dependency>

                                                                <groupId>net.java.dev.jna</groupId>

                                                                <artifactId>jna-platform</artifactId>

                                                                <version>5.5.0</version>

                                                </dependency>

                                </dependencies>

 

Others may prefer ant or Gradle, which give more flexibility. For me, Maven is simpler.

 

From: <jna-...@googlegroups.com> on behalf of Davide Perini <perini...@dpsoftware.org>

Reply-To: <jna-...@googlegroups.com>

Date: Sunday, May 31, 2020 at 10:21 AM

To: <jna-...@googlegroups.com>

Subject: Re: Please add Windows 10 Desktop Duplication API support

 

Hi,

thanks for the kind answer, I really appreciate it.

 

I would really like to learn JNA and give my contribute but at the moment I need to be sincere, I'm not able to do it and I don't have time to learn it without some good docs that shows the basics with step by step examples for simple APIs.

 

At the moment, I don't even know how to compile the project :)

Is this possible to do it via IntelliJ? Is there some guide on how to build using IntelliJ?

 

I will continue to look into the project and probably I will understand how it works one day and how to contribute.

 

Thank you.

Davide

 

 

Il 30/05/2020 02.05, Daniel B. Widdis ha scritto:

 

This is a perfect time to learn JNA!   Mapping is straightforward.  Declare an interface for the DLL you want to load (in this case, DXGI) and load the library.

 

Create a class Dxgi.java:

 

public interface Dxgi extends Library {

    Dxgi INSTANCE = Native.load("Dxgi", Dxgi.class, W32APIOptions.DEFAULT_OPTIONS);

    

    // structure and function mappings here

}

 

 

HRESULT AcquireNextFrame(int TimeoutInMilliseconds, DXGI_OUTDUPL_FRAME_INFO pFrameInfo, PointerByReference ppDesktopResource);

 

 

    @FieldOrder({ "LastPresentTime", "LastMouseUpdateTime", "AccumulatedFrames", "RectsCoalesced",

            "ProtectedContentMaskedOut", "PointerPosition", "TotalMetadatBufferSize", "PointerShapeBufferSize" })

    class DXGI_OUTDUPL_FRAME_INFO extends Structure {

        public LARGE_INTEGER LastPresentTime;

        public LARGE_INTEGER LastMouseUpdateTime;

        public int AccumulatedFrames;

        public boolean RectsCoalesced;

        public boolean ProtectedContentMaskedOut;

        public DXGI_OUTDUPL_POINTER_POSITION PointerPosition;

        public int TotalMetadataBufferSize;

        public int PointerShapeBufferSize;

 

        // may need constructor if you will create from a Pointer

    }

 

That requires a https://docs.microsoft.com/en-us/windows/win32/api/dxgi1_2/ns-dxgi1_2-dxgi_outdupl_pointer_position structure, map similarly, repeat nested mappings as necessary.

 

Look at almost any random class in the win32 platform mappings and you'll see ample examples:

https://github.com/java-native-access/jna/tree/master/contrib/platform/src/com/sun/jna/platform/win32

 

Try and get started and feel free to ask here for help if you get stuck.

 

 

On Fri, May 29, 2020 at 4:30 PM Davide Perini <mailto:sblan...@gmail.com> wrote:

I don't know JNA so well, it's not that easy for me at the moment :D

 

Il giorno venerdì 29 maggio 2020 20:21:34 UTC+2, Daniel B. Widdis ha scritto:

Davide,

 

There is "support" in the case that the base JNA functionality will easily allow you to create an interface and map the functions.

 

Just because someone hasn't mapped a function yet doesn't mean it's "not supported".

 

On Fri, May 29, 2020 at 11:17 AM Davide Perini <sblan...@gmail.com> wrote:

Is this a confirmation that there is no support for Desktop Duplication API in the current state?

 

Thanks :)

 

--

You received this message because you are subscribed to the Google Groups "Java Native Access" group.

To unsubscribe from this group and stop receiving emails from it, send an email to jna-...@googlegroups.com.

 

 

 

--

Dan Widdis

--

You received this message because you are subscribed to the Google Groups "Java Native Access" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mailto:jna-users+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jna-users/62ed9f37-22c5-4f1a...@googlegroups.com?utm_medium=email&utm_source=footer.

 

 

 

--

Dan Widdis

--

You received this message because you are subscribed to the Google Groups "Java Native Access" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mailto:jna-users+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jna-users/CANb1Va6gZJ8AfvTMpmQY-GMs...@mail.gmail.com?utm_medium=email&utm_source=footer.

 

--

You received this message because you are subscribed to the Google Groups "Java Native Access" group.

To unsubscribe from this group and stop receiving emails from it, send an email to mailto:jna-users+...@googlegroups.com.

To view this discussion on the web visit https://groups.google.com/d/msgid/jna-users/a089023e-63ea-3158...@dpsoftware.org?utm_medium=email&utm_source=footer.

 

Davide Perini

unread,
May 31, 2020, 3:28:14 PM5/31/20
to jna-...@googlegroups.com
How can a JNA maven dependency help in building JNA?
If I git clone the hole JNA project
https://github.com/java-native-access/jna.git

what is the pom.xml file that I need to edit to add this dependency?
I'm trying to build and compile JNA with my new Dxgi.java class,
not to simply "use JNA".

Thanks,
Davide
To unsubscribe from this group and stop receiving emails from it, send an email to jna-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jna-users/74A86C45-37D5-481E-937B-3BF52728BD59%40gmail.com.

Matthias Bläsing

unread,
May 31, 2020, 3:35:40 PM5/31/20
to jna-...@googlegroups.com
Hi Davide,

Am Sonntag, den 31.05.2020, 21:28 +0200 schrieb Davide Perini:
> How can a JNA maven dependency help in building JNA?
> If I git clone the hole JNA project
> https://github.com/java-native-access/jna.git
>
> what is the pom.xml file that I need to edit to add this dependency?
> I'm trying to build and compile JNA with my new Dxgi.java class,
> not to simply "use JNA".
>

this is not needed. Take one step after the other. Building JNA is a
bit involved, building jna-platform (that is the project your class
could end up) is easier, but before doing that, the class can live
outside JNA project.

The gstreamer bindings are a sample. They are independend of the JNA
project, but use JNA to do the heavylifting of the C<->Java interface.

Daniels hint was totally correct and the easiest starting point. So:

* create a new project with a build system of your chosing (I also
prefer maven) and make sure the two artifacts:

https://search.maven.org/artifact/net.java.dev.jna/jna/5.5.0/jar
https://search.maven.org/artifact/net.java.dev.jna/jna-platform/5.5.0/jar

are included

* Add your binding class to this project
* Create a Main class and use that to test your binding

HTH

Matthias

Davide Perini

unread,
May 31, 2020, 3:42:24 PM5/31/20
to jna-...@googlegroups.com, Matthias Bläsing
Sorry if I don't understood before.
I will do it this way. Thank you all guys :)


In data 31 maggio 2020 21:35:41 Matthias Bläsing
<mbla...@doppel-helix.eu> ha scritto:
> --
> You received this message because you are subscribed to the Google Groups
> "Java Native Access" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to jna-users+...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jna-users/f82c64f91dec7f87f2c7ee83f4d2cc714f721f45.camel%40doppel-helix.eu.



Daniel Widdis

unread,
May 31, 2020, 3:52:57 PM5/31/20
to jna-...@googlegroups.com
Here's a good "hello world" sample project to get you started.

https://www.techbeamers.com/write-a-simple-jna-program-in-java/

Replace the "download" step 1 with the maven setup described, then write those two classes in your project. Those will call native versions of printf and scanf.
To view this discussion on the web visit https://groups.google.com/d/msgid/jna-users/1726c4191c0.2820.3c8a1e3a2388806d058c6e4a23c6ecab%40dpsoftware.org.


Davide Perini

unread,
May 31, 2020, 4:48:22 PM5/31/20
to jna-...@googlegroups.com
This project is so cool, can't wait to use it everywhere :)
Thanks for the help Daniel, I really appreciate your help.

I have followed your suggestion,
I have a maven project, I imported the jna deps, and I can call all the native API implemented in the lib.

Now I would like to implement my IDXGIOutputDuplication::AcquireNextFrame method

I have created my Dxgi.java
public interface Dxgi extends Library {

    Dxgi INSTANCE = Native.load("Dxgi", Dxgi.class, W32APIOptions.DEFAULT_OPTIONS);

    WinNT.HRESULT AcquireNextFrame(int TimeoutInMilliseconds, DXGI_OUTDUPL_FRAME_INFO pFrameInfo, PointerByReference ppDesktopResource);

    // structure and function mappings here

}

    my DXGI_OUTDUPL_FRAME_INFO.java
@AllArgsConstructor // using Lombok to generate all args constructor
@Structure.FieldOrder({ "LastPresentTime", "LastMouseUpdateTime", "AccumulatedFrames", "RectsCoalesced",
            "ProtectedContentMaskedOut", "PointerPosition", "TotalMetadatBufferSize", "PointerShapeBufferSize" })

class DXGI_OUTDUPL_FRAME_INFO extends 
Structure {

        public WinNT.LARGE_INTEGER LastPresentTime;

        public WinNT.LARGE_INTEGER LastMouseUpdateTime;

        public int AccumulatedFrames;

        public boolean RectsCoalesced;

        public boolean ProtectedContentMaskedOut;

        public DXGI_OUTDUPL_POINTER_POSITION PointerPosition;

        public int TotalMetadataBufferSize;

        public int PointerShapeBufferSize;

}

    and my DXGI_OUTDUPL_POINTER_POSITION.java
@Structure.FieldOrder({ "Position", "Visible" })

    class DXGI_OUTDUPL_POINTER_POSITION extends Structure {

    public Pointer Position;
    public boolean Visible;

}
now in my main.java I do something like this
Dxgi dxgi = Dxgi.INSTANCE;
DXGI_OUTDUPL_FRAME_INFO frameInfo = null;
PointerByReference ppDesktopResource = null;
dxgi.AcquireNextFrame(1,frameInfo,ppDesktopResource);
but I don't know how to initialize the second and the third argument of the AcquireNextFrame method.
Any suggestion on how to initialize that Structure?

The AcquireNextFrame seems to return a WinNT.HRESULT, but how can I use that result in Java?
That method should return an image of some sort, but how to "convert" that result in a Java readable format? ImageBuffer for example?

Thanks!
Davide

Daniel Widdis

unread,
May 31, 2020, 5:56:29 PM5/31/20
to jna-...@googlegroups.com

Great progress!

 

> but I don't know how to initialize the second and the third argument of the AcquireNextFrame method.
> Any suggestion on how to initialize that Structure?

 

This is where you have to read the Native API.  It tells you what to do with each argument:

  • Timeout in milliseconds: You can specify a short timeout so it will return even if it doesn’t succeed in the required timeframe. Your 1 ms is probably too short, 33ms would correspond to 30 FPS, so maybe split the difference with 15 or 20?
  • pFrameInfo: Receives the structure.  Don’t pass “null”, you need to give it memory to populate.  You can just create a structure with new DXGI_OUTDUPL_FRAME_INFO() and pass that. The function call will populate it with data.
  • ppDesktopResource: Receives a reference to a pointer. Same thing, you need a non-null pointer-sized memory field to populate. Initialize with new PointerByReference()

 

You’ll use the returned pointer value to instantiate an IDXGIResource interface.  Defining that is a little more complex but once you get this pointer, I’ll walk you through that part.  I’ve summarized the general steps in this StackOverflow answer but I’ll give more specific help for you here.

 

From: <jna-...@googlegroups.com> on behalf of Davide Perini <perini...@dpsoftware.org>
Reply-To: <jna-...@googlegroups.com>
Date: Sunday, May 31, 2020 at 1:48 PM
To: <jna-...@googlegroups.com>
Subject: Re: Please add Windows 10 Desktop Duplication API support

 

This project is so cool, can't wait to use it everywhere :)

Daniel Widdis

unread,
May 31, 2020, 6:10:26 PM5/31/20
to jna-...@googlegroups.com

> The AcquireNextFrame seems to return a WinNT.HRESULT, but how can I use that result in Java?

 

HRESULT just extends NativeLong.  Use its .intValue() method to extract an error code.  0 is success, other codes are explained here and those constant names are mapped in the WinErr class.

 

The COMUtils class has a method SUCCEEDED(HRESULT hr) which easily turns that into a boolean, and checkRC(HRESULT hr) which turns a failed result into a COMException.  There may be other utilities.

 

From: <jna-...@googlegroups.com> on behalf of Davide Perini <perini...@dpsoftware.org>
Reply-To: <jna-...@googlegroups.com>
Date: Sunday, May 31, 2020 at 1:48 PM
To: <jna-...@googlegroups.com>
Subject: Re: Please add Windows 10 Desktop Duplication API support

 

This project is so cool, can't wait to use it everywhere :)

Davide Perini

unread,
May 31, 2020, 7:47:39 PM5/31/20
to jna-...@googlegroups.com
Done it

Dxgi dxgi = Dxgi.INSTANCE;
PointerByReference ppDesktopResource = new PointerByReference();
DXGI_OUTDUPL_FRAME_INFO frameInfo = new DXGI_OUTDUPL_FRAME_INFO();
dxgi.AcquireNextFrame(33,frameInfo,ppDesktopResource);

but I have this exception on runtime:
Exception in thread "main" java.lang.UnsatisfiedLinkError: Error looking up function 'AcquireNextFrame': The specified procedure could not be found.

this is my Dxgi class

public interface Dxgi extends Library {

    Dxgi INSTANCE = Native.load("Dxgi", Dxgi.class, W32APIOptions.DEFAULT_OPTIONS);

    WinNT.HRESULT AcquireNextFrame(int TimeoutInMilliseconds, DXGI_OUTDUPL_FRAME_INFO pFrameInfo, PointerByReference ppDesktopResource);

    // structure and function mappings here

}

I haven't added any structure or function mapping, should I add something else to that class?

Thanks for the help, I really appreciate it.

Daniel Widdis

unread,
May 31, 2020, 8:18:45 PM5/31/20
to jna-...@googlegroups.com

Uh oh.

 

So, we’ve reached an instructive point in this exercise.  First, you’ve learned how to map functions to JNA.

 

However, JNA works via libffi which is a C interface, not C++.  In order to work with DLLs targeted to C++, those DLLs must export the functions to C. 

 

In many cases, particularly with the Windows APIs, this is the case.  However, in this particular case, it seems that this mapping isn’t possible directly.

 

I’ll step back and let brighter minds than mine give more commentary in case I’m missing something.

Davide Perini

unread,
Jun 2, 2020, 10:11:44 AM6/2/20
to jna-...@googlegroups.com
Reading on the internet this seems a "common problem for JNA users" and every people
who hit this wall does not find a solution, or at least they don't share it on the internet.

Pretty said that my tour on JNA is already finished.

Thanks Daniel, I have appreciated your help. :)

Kind Regards,
Davide

Daniel Widdis

unread,
Jun 2, 2020, 10:42:23 AM6/2/20
to jna-...@googlegroups.com

Actually there is a solution posted many places on the internet, and it’s to write a C wrapper function to expose the C++ functions.

 

I understand your frustration, but the fault lies with library developers who do not expose their APIs to C.  JNA is one of many cross-platform / cross-language programs relying on libffi and all suffer the same roadblock. 

Davide Perini

unread,
Jun 2, 2020, 7:19:41 PM6/2/20
to jna-...@googlegroups.com
So the idea of wrapping DXGI function is passed away xD
I need to capture 30 BufferedImage per second in Java.

I tried VLCJ but its snapshot function is worse than the  GDI32Util.getScreenshot(hwnd);
from JNA.

Is it possible to make GDI32Util.getScreenshot(hwnd) able to capture 30 frames per second?

With two threads running a loop like this:
while(true) {
  GDI32Util.getScreenshot(hwnd);
  System.gc();
}
that wrote the ImageBuffer on a LinkedBlockedQueue
I can capture at about 27FPS.
System.gc() seems to do an excellent jobs and the memory used is in check.

The only problem is that this method seems to create some sort of hiccups on my system.
If I try to play a video when the loops are running the video stutter like hell.

Is there a solution to avoid stuttering when using GDI32Util.getScreenshot(hwnd)???

What API does this method wrap exactly?

Thanks,
Davide

Daniel Widdis

unread,
Jun 2, 2020, 7:52:05 PM6/2/20
to jna-...@googlegroups.com

As Obi-Wan would say, “Use the Source, Luke.”  😊

 

Take a look at the GDI32Util getScreenshot method implementation.

 

Copy that as a starting point, look at every case of “new X()” and see if you can re-use objects instead of instantiating them and disposing of them, that should save you the heap issues and delays with garbage collection.  Are you compiling for JDK11 with faster GC?

 

I believe reusing objects was suggested to you in another email thread on this list, so check for that reply that had a lot more detail, I think.

 

The Memory object used as a buffer is a good starting point to re-use.  Keeping the DataBufferInt around and just setting its value instead of re-instantiating it is another.

Davide Perini

unread,
Jun 2, 2020, 8:50:29 PM6/2/20
to jna-...@googlegroups.com
Ok done it. Hiccups remains. I need to find an alternative API.

It seems that I have found a new API that could help me.


Daniel do you think that this API can be mapped with JNA?


Thank you for your great support.

Davide

Daniel Widdis

unread,
Jun 3, 2020, 12:04:15 AM6/3/20
to jna-...@googlegroups.com

Based on browsing the docs from the link you provided, it appears (I’m guessing) these are in the WinRT dll file, and this stackoverflow answer implies there’s a C API.

 

Note this is still COM which is another layer of complexity (you have to locate the header file, find the VTBL which lists the pointers to each function in order, count them, and then use a com invoking function based on that index) but it seems it would be possible.

Davide Perini

unread,
Jun 3, 2020, 4:21:55 AM6/3/20
to jna-...@googlegroups.com
Do you think that is something "doable" for a noob like me?
Can you give me some hints to start?

Thanks!

Neil C Smith

unread,
Jun 3, 2020, 5:22:45 AM6/3/20
to jna-...@googlegroups.com
On Wed, 3 Jun 2020, 00:19 Davide Perini, <perini...@dpsoftware.org> wrote:
> I tried VLCJ but its snapshot function is worse than the GDI32Util.getScreenshot(hwnd);
> from JNA.

Looking at the source at
https://github.com/caprica/vlcj/blob/master/src/main/java/uk/co/caprica/vlcj/player/base/SnapshotApi.java
if that's what you mean, I'm not surprised.

If you're going to criticise someone's library here, at least take the
time to read through the documentation and find out how to achieve
what you want to do. That certainly isn't the right way!

Anything with the word "snapshot" or "screenshot" in it is probably
not what you're looking for. Likewise, anything that returns a
BufferedImage or similar to you is almost certainly not what you want.

If you're using another library, you'll want to look for some sort of
callback function, if you end up deciding to map yourself, you still
probably want to provide to the rest of your application via a
callback too. You're not looking for snapshots, you're looking for a
video stream. A frame callback where the image / image data is only
valid for the duration of the callback allows for caching and reuse
under the covers, both in the Java and native code. Anything that is
returning a BufferedImage, or any image data (without a mechanism for
releasing back at the very least), has to do things that are the
opposite of what you require for this.

Note also that using one of the native video libraries may give you
the option of doing some / all of the scaling and colour calculations
you need outside of the JVM too - keeping the size of the data coming
into the JVM to as low as strictly necessary will help performance.

Best wishes,

Neil

Davide Perini

unread,
Jun 3, 2020, 3:54:26 PM6/3/20
to jna-...@googlegroups.com
Hey Neil,
I'm not too good in English and probably I haven't explained my self in
the right way.
Sorry if you felt like I was criticizing.

I don't want to criticize VLCJ :) I love the project and I use VLC since
the first releases.

GDI32Util.getScreenshot(hwnd) is not the right method for what I need, but I haven't said that JNA is not a good lib :)
I'm not that stupid yet to no be able to distinguish :)

I have searched in this examples too:
https://github.com/caprica/vlcj-examples/tree/master/src/main/java/uk/co/caprica/vlcj/test

I tried the snapshot one and I have seen that is not meant for my purpose, I then tried the screen recorder.
https://github.com/caprica/vlcj-examples/blob/master/src/main/java/uk/co/caprica/vlcj/test/screen/ScreenRecorder.java

That is cool but I wasn't able to get the awt.Color object of the pixel in position X,Y.
The resulting video is even "stuttering" even if I set a 30FPS output.
I have a six cores HEDT CPU i7 5930K with an RTX2080Ti, not a modern high end PC but not a bad too.

This is a JNA list, I don't know if it's correct to talk about VLCJ in this list.
I will open a thread in their "github" to continue this talk. (It seems that they accept questions on guthub issues)

Thanks
Davide

Davide Perini

unread,
Jun 4, 2020, 6:13:46 PM6/4/20
to jna-...@googlegroups.com
Hi Daniel,
as a good student (not too good) I am trying to understand things, rereading your answers.

I doubt that you are not right, but are you sure that the problem is that DXGI does not expose C functions?

Reading the dxgi.h sources I jump into the dxgi1_2.h header.
All the header is "correctly wrapped" into

extern "C"{
    // header's code
}

and all the functions have this ifdef
#if defined(__cplusplus) && !defined(CINTERFACE)

and an
#else     /* C style interface */

this should be the C declaration of the C function I need:

        HRESULT ( STDMETHODCALLTYPE *AcquireNextFrame )(
            IDXGIOutputDuplication * This,
            /* [annotation][in] */
            _In_  UINT TimeoutInMilliseconds,
            /* [annotation][out] */
            _Out_  DXGI_OUTDUPL_FRAME_INFO *pFrameInfo,
            /* [annotation][out] */
            _COM_Outptr_  IDXGIResource **ppDesktopResource);

I'm a noob, if I said inaccuracies please be patient :)

Thank you Daniel.



Il 02/06/2020 16.42, Daniel Widdis ha scritto:

Davide Perini

unread,
Jun 6, 2020, 11:10:51 AM6/6/20
to jna-...@googlegroups.com
I wrote to Daniel but the question is obviously opened to all users in the list :)

Isn't this the C wrapper that JNA needs?

Thanks,
Davide

Matthias Bläsing

unread,
Jun 7, 2020, 9:45:48 AM6/7/20
to jna-...@googlegroups.com
Hi Davide,

yes this looks like a workable API. Here is a C++ -> C -> Python
sample, that could help:

https://medium.com/steveindusteves/playing-games-with-python-9be869f7b189

If there is a plain C api defined in the headers, you can use/bind it
directly. From the looks this is indeed a COM api, but it might be
bindable without deep knowledge about that.

My advise:

- Try to find a _minimal_ C sample that captures a few frames/makes
minimal use of the API
- If that can't be found, try to find a sample, that uses libffi
- If that can't be found, try to find a sample that uses C++
- Try to adapt that to java

You need to get your feet wet and from experience I know, that it might
be painful and will take time.

Greetings

Matthias
> > > > > > https://groups.google.com/d/msgid/jna-users/1726c4191c0.2820.3c8a1e3a2388806d058c6e4a23c6ecab%40dpsoftware.org
> > > > > > .
> > > > > >
> > > > > >
> > > > >
> > > > >
> > > > > --
> > > > > You received this message because you are subscribed to the
> > > > > Google Groups "Java Native Access" group.
> > > > > To unsubscribe from this group and stop receiving emails from
> > > > > it, send an email to jna-users+...@googlegroups.com.
> > > > > To view this discussion on the web visit
> > > > > https://groups.google.com/d/msgid/jna-users/d7ce6d0a-5a5b-40e6-18e9-9e17f240491d%40dpsoftware.org
> > > > > .
> > > > >
> > > > >
> > > > >
> > > > > --
> > > > > You received this message because you are subscribed to the
> > > > > Google Groups "Java Native Access" group.
> > > > > To unsubscribe from this group and stop receiving emails from
> > > > > it, send an email to jna-users+...@googlegroups.com.
> > > > > To view this discussion on the web visit
> > > > > https://groups.google.com/d/msgid/jna-users/7A1FECB0-65D2-4668-A8C2-EE4E33749DDF%40gmail.com
> > > > > .
> > > >
> > > >
> > > > --
> > > > You received this message because you are subscribed to the
> > > > Google Groups "Java Native Access" group.
> > > > To unsubscribe from this group and stop receiving emails from
> > > > it, send an email to jna-users+...@googlegroups.com.
> > > > To view this discussion on the web visit
> > > > https://groups.google.com/d/msgid/jna-users/cd40cc31-1e43-c874-fe94-cf90687ef552%40dpsoftware.org
> > > > .
> > > >
> > > >
> > > > --
> > > > You received this message because you are subscribed to the
> > > > Google Groups "Java Native Access" group.
> > > > To unsubscribe from this group and stop receiving emails from
> > > > it, send an email to jna-users+...@googlegroups.com.
> > > > To view this discussion on the web visit
> > > > https://groups.google.com/d/msgid/jna-users/D6A4AA40-68FE-49DE-BF13-04393EA9B776%40gmail.com
> > > > .
> > >
> > >
> > > --
> > > You received this message because you are subscribed to the
> > > Google Groups "Java Native Access" group.
> > > To unsubscribe from this group and stop receiving emails from it,
> > > send an email to jna-users+...@googlegroups.com.
> > > To view this discussion on the web visit
> > > https://groups.google.com/d/msgid/jna-users/6e93552c-7acb-c7e5-11e3-ec567a5ad9b6%40dpsoftware.org
> > > .
> > >
> > > --
> > > You received this message because you are subscribed to the
> > > Google Groups "Java Native Access" group.
> > > To unsubscribe from this group and stop receiving emails from it,
> > > send an email to jna-users+...@googlegroups.com.
> > > To view this discussion on the web visit
> > > https://groups.google.com/d/msgid/jna-users/B6EACEFA-6BB1-428D-93FD-E6EBCE98DBA3%40gmail.com
> > > .
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "Java Native Access" group.
> > To unsubscribe from this group and stop receiving emails from it,
> > send an email to jna-users+...@googlegroups.com.
> > To view this discussion on the web visit
> > https://groups.google.com/d/msgid/jna-users/eff589ff-710f-0809-555e-52f9781312fa%40dpsoftware.org
> > .
>
> --
> You received this message because you are subscribed to the Google
> Groups "Java Native Access" group.
> To unsubscribe from this group and stop receiving emails from it,
> send an email to jna-users+...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/jna-users/b2923862-2c72-7479-55b9-4382dcd8cfd9%40dpsoftware.org
> .

Daniel B. Widdis

unread,
Jun 7, 2020, 12:10:56 PM6/7/20
to Java Native Access
Sorry for being slow to reply here, as I have been traveling.

I see the method does appear exposed in the API.  What other reason could prompt this error at runtime (with return types and args mapped appropriately, I think):

UnsatisfiedLinkError: Error looking up function 'AcquireNextFrame'



--
Dan Widdis

Matthias Bläsing

unread,
Jun 7, 2020, 12:33:55 PM6/7/20
to jna-...@googlegroups.com
Hi,

Am Sonntag, den 07.06.2020, 09:10 -0700 schrieb Daniel B. Widdis:
> I see the method does appear exposed in the API. What other reason
> could prompt this error at runtime (with return types and args mapped
> appropriately, I think):
>
> UnsatisfiedLinkError: Error looking up function 'AcquireNextFrame'

this is more subtle. AcquireNextFrame is not a plain function. It is a
method call. From the header files:

There is a helper makro:

#define IDXGIOutputDuplication_AcquireNextFrame(This,TimeoutInMilliseconds,pFrameInfo,ppDesktopResource) \
( (This)->lpVtbl -> AcquireNextFrame(This,TimeoutInMilliseconds,pFrameInfo,ppDesktopResource) )


And a structure definition (this is the C interface):

typedef struct IDXGIOutputDuplicationVtbl
{
/* ... */
HRESULT ( STDMETHODCALLTYPE *AcquireNextFrame )(
IDXGIOutputDuplication * This,
/* [annotation][in] */
_In_ UINT TimeoutInMilliseconds,
/* [annotation][out] */
_Out_ DXGI_OUTDUPL_FRAME_INFO *pFrameInfo,
/* [annotation][out] */
_COM_Outptr_ IDXGIResource **ppDesktopResource);
}

And that is wrapped into:

interface IDXGIOutput1
{
CONST_VTBL struct IDXGIOutput1Vtbl *lpVtbl;
};

My google foo tells me, that interface is defined to struct by MSVC.

The structure IDXGIOutput1 has a member lpVtable, which is a pointer to
a structure, that is a structure, with a member AcquireNextFrame, that
is a pointer to a function. That function takes as first parameter a
pointer to a IDXGIOutputDuplication (I assume, that the "interface" is
implemented by a concrete object).

This looks like the typical structure I would expect from a COM object.
COM objects are also basicly pointers to VTables holding function
references.

So there needs to be an entry function. The public entry function can
be found in dxgi.dll (DLL Export Viewer is one programm that can list
all defined functions in a dll).

I see CreateDXGIFactory, which looks like it fills a similar role to
CoCcreateInstance in "normal" COM as bound in JNA.

Maybe this helps a bit.

Greetings

Matthias

Daniel Widdis

unread,
Jun 7, 2020, 12:49:19 PM6/7/20
to jna-...@googlegroups.com
Matthias: Thank you, that was the pointer I needed. I had missed where it was a COM interface function.

Davide: search for IDXGIOutputDuplicationVtbl in the header file here. You'll find it at line 334.

You must create a COM object DXGIOutputDuplication. Extend Unknown. Take a look at JNA's Wbemcli class for a working example that you can copy the format of.

Observe how the VTBL functions are mapped using the _invokeNativeObject() method. The first parameter is the 0-indexed order of the function in the vtbl, so I think you need an index of 8. You'll use a wrapper method on your object to map the arguments to an array of objects to pass to that invoking function.

You will need to:
1. Create a class to represent the COM interface (optional) and the object implementing it (required). The interface starts with an I
--
You received this message because you are subscribed to the Google Groups "Java Native Access" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jna-users+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/jna-users/4d51ce573e8a10563e7efda5810aaeed5b7e0adb.camel%40doppel-helix.eu.


Davide Perini

unread,
Jun 7, 2020, 1:25:16 PM6/7/20
to jna-...@googlegroups.com
Thank you for the asnwer Matthias.

I have found some C++ examples, I'm able to compile them and get them work but I still don't understand
how to use JNA with those examples.

Should I create a C wrapper by my self that expose a wrappedFunction() and call it via JNA?
or can I call the API desktop duplication api directly using JNA?

the problem is that this:
public interface Dxgi extends Library {

    Dxgi INSTANCE = Native.load("Dxgi", Dxgi.class, W32APIOptions.DEFAULT_OPTIONS);

    public WinNT.HRESULT AcquireNextFrame(int TimeoutInMilliseconds, DXGI_OUTDUPL_FRAME_INFO pFrameInfo, PointerByReference ppDesktopResource);

}
return this

Exception in thread "main" java.lang.UnsatisfiedLinkError: Error looking up function 'AcquireNextFrame':

Does this means that I need to create a C wrapper to call the AcquireNextFrame function?

Thanks
Davide

Davide Perini

unread,
Jun 7, 2020, 1:28:50 PM6/7/20
to jna-...@googlegroups.com
Ok I will read all yours and Matthias answers and I will try to do it.

Thanks for your help guys, I appreciate it so much. :)

Matthias Bläsing

unread,
Jun 7, 2020, 2:47:18 PM6/7/20
to jna-...@googlegroups.com
Hi,

Am Sonntag, den 07.06.2020, 19:25 +0200 schrieb Davide Perini:
> Thank you for the asnwer Matthias.
>
> I have found some C++ examples, I'm able to compile them and get them
> work but I still don't understand
> how to use JNA with those examples.
>
> Should I create a C wrapper by my self that expose a
> wrappedFunction() and call it via JNA?
> or can I call the API desktop duplication api directly using JNA?
>
> the problem is that this:
> public interface Dxgi extends Library {
>
> Dxgi INSTANCE = Native.load("Dxgi", Dxgi.class, W32APIOptions.DEFAULT_OPTIONS);
>
> public WinNT.HRESULT AcquireNextFrame(int TimeoutInMilliseconds, DXGI_OUTDUPL_FRAME_INFO pFrameInfo, PointerByReference ppDesktopResource);
>
> }
> return this
> Exception in thread "main" java.lang.UnsatisfiedLinkError: Error
> looking up function 'AcquireNextFrame':
>
> Does this means that I need to create a C wrapper to call the
> AcquireNextFrame function?
>

please _read_ the already given answers. The way your are binding
expects the library to export a symbol (named function) with the name
AcquireNextFrame. This is not the case.

_Look_ at the dxgi.dll and see the exported functions and you won't
find it there.

If you don't understand what a vtable is or how it works, please read
about it and if you still have questions after that, ask specifics.

Please read first - if I try to explain that all here, I can do the
binding most probably myself in the same time.

Greetings

Matthias

Daniel Widdis

unread,
Jun 7, 2020, 2:59:50 PM6/7/20
to jna-...@googlegroups.com

No C wrapper for that function. You need a COM object per my previous answer.

 

You will create the COM object from a pointer that you will have to obtain using the API which generates that object… Matthias suggested one, you will have to search the API to find that, at which point you will:

 

  1. Map in a C interface the function that generates a (pointer to a) COM object
  2. Map in the COM object the functions that object exposes, using a function offset value you can find from the order in the vtbl.

 

Take a look at JNA WbemcliUtil and WbemCli class source code for similar types of things to what you will need to do.

Davide Perini

unread,
Jun 7, 2020, 3:38:12 PM6/7/20
to jna-...@googlegroups.com
I give up :)

I appreciate all your answers but things is getting too complicated for my purpose.

My next step will be to buy a good JNI book and start from it, understanding JNI will make me appreciate JNA more.

It seems easyer to write a C/C++ program that "does all the needed things", AcquireNextFrame and simply return an int array to be used in Java, but at this level I don't know if JNA is the right choice over JNI.

I appreciate all the time you spent with me but it's time for a good book.

Kind Regards and thank you guys.
Davide
Reply all
Reply to author
Forward
0 new messages