Fl_gl_window on top of Fl_single_window...

[shayne....@comcast.net]

I have an FLTK app that uses an Fl_single_window to display video output from another application (e.g. it's embedding it's video output into this window).

I would like to place an Fl_gl_window on top of this window which has a transparent background and semi transparent polygons so that the underlying Fl_single_window is visible. I'm using the Fl_gl_window as an overlay on top of the Fl_single_window. Is such a thing possible in FLTK? What I've tried so far hasn't worked.

Any help would be appreciated...

[Ian MacArthur]

This is a tricky thing, and not something that fltk really supports.

In effect you have one render surface that is being handled by the GL context in the GPU, and one that is being handled by some other context in the GPU controlled by something else...

It *might* work, with some GPU cards and drivers, and not work with others.

Or it might never work at all.

And there’s probably not a lot fltk can do about it, since it is striving to present a reasonably uniform interface across disparate platforms, which limits our “opportunities" to address some more complex usages...

FWIW, if I were in your place, I’d want to look closely at whether you can persuade your video device to render into a GL surface of some sort (many of them can these days) and have your Fl_Gl_Window grab that and scribble whatever rendering you need over the top of it.

That might work...

[shayne....@comcast.net]

Ian...thanks for your reply.

I was hoping that fltk supported some sort of layered window concept but it sounds like it doesn't. I know that windows 7 has the concept of compositing but I doubt fltk exposes that.

I will try your last suggestion to see if that might present any possibilities for me to get this to work. As you suggested, perhaps some sort of frame buffer object approach might work.

I did try embedding the other application's video into the Fl_gl_window but things flickered like crazy due to the double buffering of the window.

[MacArthur, Ian (Selex ES, UK)]

> I was hoping that fltk supported some sort of layered window concept
> but it sounds like it doesn't.

No, not really. Fltk has always aimed more for the "fast and light" end of the game; compositing tends to be neither of these...

> I know that windows 7 has the concept of
> compositing but I doubt fltk exposes that.

I'm not even sure that compositing in the WM would help here. But you could certainly give it a try.

There was code posted in this list fairly recently, showing how to set a window alpha.
It used a bit of platform specific code for each of Win32, X11 and OSX to adjust the alpha of the window (once it was shown) so that it was relatively transparent to objects beneath it.
I played about with that code a bit, and it works pretty well, so may well be worth a try.

However... in my tests I was using "normal" windows, which the WM knows all about and is "responsible" for rendering the drawn regions.
I'm not sure whether the same trick would work to adjust the alpha of a region that was rendered by the GL context?
It might be "outside the control" of the WM I suppose?
It might even depend on how a given GPU driver integrates with the WM I guess?

Still, certainly worth a try... If it does work, that might be an easy win.

Look for a thread titled:

"How can I make a window transparent in OS X and Linux? (Windows already works)" dated around about late October / early November of 2014 and see if that's any use to you. Towards the end of the thread, there are samples of working code for each host, including Win7.
But whether it works for a GL context I can not say...

> I will try your last suggestion to see if that might present any
> possibilities for me to get this to work. As you suggested, perhaps
> some sort of frame buffer object approach might work.

Yes, I was imagining that if you can get it to render into an FBO, that might serve as a basis for drawing your other content on top and...

> I did try embedding the other application's video into the Fl_gl_window
> but things flickered like crazy due to the double buffering of the
> window.

Indeed; also, I guess it might be very sensitive to what frame rate your underlying window delivers.

It is a tricky one...



Selex ES Ltd
Registered Office: Sigma House, Christopher Martin Road, Basildon, Essex SS14 3EL
A company registered in England & Wales. Company no. 02426132
********************************************************************
This email and any attachments are confidential to the intended
recipient and may also be privileged. If you are not the intended
recipient please delete it from your system and notify the sender.
You should not copy it or use it for any purpose nor disclose or
distribute its contents to any other person.
********************************************************************

[Greg Ercolano]

On 05/11/15 09:34, MacArthur, Ian (Selex ES, UK) wrote:
>> I was hoping that fltk supported some sort of layered window concept
>> but it sounds like it doesn't.
>
> No, not really. Fltk has always aimed more for the "fast and light" end of the game; compositing tends to be neither of these...

I don't even know how FLTK could do this..
this sounds like it'd all be inside opengl or the window manager or both.

I think trying to find the opengl context that the video player is using
and 'scribble' into that sounds like a good approach.

Best approach: check the video player's library to see if they provide
a callback or some such that let's you include your own draw commands
into every frame of video. Surely any video library would provide some way
for the app to include overlay graphics; volume graphs, "STOP", "PLAY" overlays,
timecode/position info, etc etc.

>> I know that windows 7 has the concept of
>> compositing but I doubt fltk exposes that.

Perhaps overlay plane drawing might be another option.

[shayne....@comcast.net]

Greg and Ian,

Thanks much for your input. This gives me some ideas to explore. If I can get this to work, I will post my solution...

[shayne....@comcast.net]

Quick follow up question...

Is there any way I can do a bitblt from a Fl_single_window? That is, capture the pixels in that window. Does fltk provide any methods to do this?

[MacArthur, Ian (Selex ES, UK)]

Well... yes and no... It's likely not going to be what you want though...

You can use fl_read_image(...) to get the pixels for the current render context, and you end up with a byte array filled with the data, which you can then use to draw another view.

But, in many (most? all?) implementations this function literally reads back from the screen memory, so it is not just grabbing a snapshot of a backbuffer or something.

There are a few consequences to this, particularly if the window is mapped to the display;

- It can be slow in some cases as it needs to play nice with the display refresh etc.
- If the mouse cursor happens to be inside your window, it can get grabbed too...
- Sometimes, obscuration by other windows gets grabbed as well.

If your video grab can be persuaded to write to an Fl_Offscreen rather than an Fl_Window, that would probably be easier to deal with since it can run independent of the display refresh and can't be obscured by the mouse or other windows.

FWIW, following the earlier discussion, I tried poking at overlaying a "window with alpha" over another window, to see if that would help, and whilst it kinda worked the result was unsatisfactory, so I don't think that is worth pursuing either to be honest.

I'm still veering towards trying to get the video dumped into a GL FBO and working with that, but not knowing what your setup provides that suggestion is little better than speculation!

Cheers,
--
Ian

[shayne....@comcast.net]

Ian,

I tried using Fl_Offscreen instead of an Fl_Single_Window. However, in order for me to embed the other application's output into it, I need to specify a window ID to the other application so that it knows what window to render to. Right now, I'm getting the window ID by

winID = fl_xid(_flSingleWin);

Can I do something similar to get the window ID for the Fl_Offscreen object?

[MacArthur, Ian (Selex ES, UK)]

> I tried using Fl_Offscreen instead of an Fl_Single_Window. However, in
> order for me to embed the other application's output into it, I need to
> specify a window ID to the other application so that it knows what
> window to render to. Right now, I'm getting the window ID by

> winID = fl_xid(_flSingleWin);

> Can I do something similar to get the window ID for the Fl_Offscreen
> object?

Ah... No, that'll not work then; I think that Fl_Offscreen has no window ID than can be used... (I may be wrong about that though...)

Hmm, well, under the covers, fl_create_offscreen() will create a surface based on the current window. To do this it uses the MS CreateCompatibleBitmap() function, which returns a HBITMAP handle, but the docs then say "This bitmap can be selected into any memory device context that is compatible with the original device."

So, it might be possible to take the HBITMAP handle and just blindly cast it to what you want and see if that actually does anything useful.
Or something.

Worth a try at any rate. I mean what's the worst that can happen...

There's also an fl_makeDC(HBITMAP); function to convert a valid HBITMAP into a DC, though I doubt that helps.

You do know I;m just making this up as I go along, don't you? :-)


- or -

I wonder if it would work to create a "real" window, but positioned such that it was not on any screen, then use fl_read_image(...) to get the pixels from that, or something?
(Sorry, not a very helpful suggestion; Pass a few more straws to grasp at...)