Re: gsoc26 – Interest in Windows GUI Agent (WDDM) Project
9 views
Skip to first unread message
Rafał Wojdyła
Mar 18, 2026, 9:30:32 AMMar 18
to qubes-devel, Qasim ahmad
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
On 3/15/26 12:15, Qasim ahmad wrote:
> Hello Rafał,
Hello!
> My name is Qasim Khawaja and I’m a Computer Science and Mathematics
> student at the University of British Columbia.
>
> I’m interested in the *GUI agent for Windows 8/10 project* for GSoC. I
> have experience working in C++ on Linux systems from my past
> internships, where I worked on debugging runtime issues and building
> components in larger codebases. I also enjoy low-level programming and
> learning how systems work under the hood.
>
> This project caught my attention because of the challenge around the
> *WDDM display driver model* and making a driver work without a physical
> GPU. I don’t have prior experience writing Windows display drivers yet,
> but I’m comfortable working in C/C++ and digging into complex systems
> code, and I’d be very interested in learning more about this area.
>
> I was wondering what the *current state of the WDDM driver work* is and
> whether there are specific parts of the codebase you’d recommend looking
> at before preparing a proposal.
Currently we have no usable WDDM code. I was researching the subject
several years ago but wasn't able to proceed then due to poor documentation
at the time and the complexity of the WDDM API. The documentation state
definitely improved since then, and there is some other prior work
to look at. A Display-only driver should be sufficient for our current
needs (emulated GPU).
I'll mention other potential avenues for improving the GUI agent:
1. Improve the performance of the current display capture solution by using
a ring buffer for frame data so the capture thread doesn't play event
ping-pong with the main thread.
2. Improve/rewrite the window tracking logic. Display capture is only
one part of the equation, we're also tracking all changes to visible
windows (display driver knows nothing about that). Currently this is
done via polling, which is really inefficient. IIRC the reason for that
was because notifications via windows hooks were unreliable: they need
DLL injection under the hood which is rather problematic (32bit vs 64bit
processes, different security contexts for things like the login
screen/winlogon etc).
3. Add a visual capture debug "log" capable of saving window snapshots
along with their metadata. Something like that would be really useful
for debugging the window tracking logic.
- --
Best regards,
Rafał Wojdyła
Invisible Things Lab
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCgAdFiEEVwOm02mHaIsSbeWChaL2sHYatbsFAmm6qIcACgkQhaL2sHYa
tbsLewgAp3e82aSfAdUi2hjkGptPFcjDru5pzuGSZPgMaZ+gq5RWHVuchAlqpqop
blsFQ7Jgv5Yxq3dZPMdRB8mxvS2gdRk51CmjgNTv0FNrTf8SATA6ZZFnNepcfMlR
nwjxHi/4uygGCNVgYKszcy0JtMeHllFC5JzqM1mgPfjQp2hwRfB0EQ/eQ35mgVBy
ozgwQSRn5CyUTnKqyeEPTeWx7gAVn04umPqU/iSMjHZXBuf86T7GOxTo60rlh7y5
+iJOHgwyeuovb9uUjV2xChc+K8u/h+6CMjDrHHlUb62QbvestQM9044rCPVQ3mdt
NE7ZLbeVm8v/k49DzEKTdi3g/MfDdg==
=Trz7
-----END PGP SIGNATURE-----
Hash: SHA512
On 3/15/26 12:15, Qasim ahmad wrote:
> Hello Rafał,
Hello!
> My name is Qasim Khawaja and I’m a Computer Science and Mathematics
> student at the University of British Columbia.
>
> I’m interested in the *GUI agent for Windows 8/10 project* for GSoC. I
> have experience working in C++ on Linux systems from my past
> internships, where I worked on debugging runtime issues and building
> components in larger codebases. I also enjoy low-level programming and
> learning how systems work under the hood.
>
> This project caught my attention because of the challenge around the
> *WDDM display driver model* and making a driver work without a physical
> GPU. I don’t have prior experience writing Windows display drivers yet,
> but I’m comfortable working in C/C++ and digging into complex systems
> code, and I’d be very interested in learning more about this area.
>
> I was wondering what the *current state of the WDDM driver work* is and
> whether there are specific parts of the codebase you’d recommend looking
> at before preparing a proposal.
Currently we have no usable WDDM code. I was researching the subject
several years ago but wasn't able to proceed then due to poor documentation
at the time and the complexity of the WDDM API. The documentation state
definitely improved since then, and there is some other prior work
to look at. A Display-only driver should be sufficient for our current
needs (emulated GPU).
I'll mention other potential avenues for improving the GUI agent:
1. Improve the performance of the current display capture solution by using
a ring buffer for frame data so the capture thread doesn't play event
ping-pong with the main thread.
2. Improve/rewrite the window tracking logic. Display capture is only
one part of the equation, we're also tracking all changes to visible
windows (display driver knows nothing about that). Currently this is
done via polling, which is really inefficient. IIRC the reason for that
was because notifications via windows hooks were unreliable: they need
DLL injection under the hood which is rather problematic (32bit vs 64bit
processes, different security contexts for things like the login
screen/winlogon etc).
3. Add a visual capture debug "log" capable of saving window snapshots
along with their metadata. Something like that would be really useful
for debugging the window tracking logic.
- --
Best regards,
Rafał Wojdyła
Invisible Things Lab
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCgAdFiEEVwOm02mHaIsSbeWChaL2sHYatbsFAmm6qIcACgkQhaL2sHYa
tbsLewgAp3e82aSfAdUi2hjkGptPFcjDru5pzuGSZPgMaZ+gq5RWHVuchAlqpqop
blsFQ7Jgv5Yxq3dZPMdRB8mxvS2gdRk51CmjgNTv0FNrTf8SATA6ZZFnNepcfMlR
nwjxHi/4uygGCNVgYKszcy0JtMeHllFC5JzqM1mgPfjQp2hwRfB0EQ/eQ35mgVBy
ozgwQSRn5CyUTnKqyeEPTeWx7gAVn04umPqU/iSMjHZXBuf86T7GOxTo60rlh7y5
+iJOHgwyeuovb9uUjV2xChc+K8u/h+6CMjDrHHlUb62QbvestQM9044rCPVQ3mdt
NE7ZLbeVm8v/k49DzEKTdi3g/MfDdg==
=Trz7
-----END PGP SIGNATURE-----
Reply all
Reply to author
Forward
0 new messages