Using GStreamer video with wxWidgets on Mac

[Andy Robinson]

Hello, I want to use GStreamer for displaying video in a control within
a window. In fact it's a subclass of wxControl.

I'm using the documented method which works fine on Linux and Windows,
which is to determine the native window id of the wxControl like this:

#ifdef __WXGTK__
GdkWindow *gdk_window = gtk_widget_get_window(m_wxwindow);
m_video_window_handle = GDK_WINDOW_XID(gdk_window);
#endif
#ifdef __WXMSW__
m_video_window_handle = (gulong)GetHWND();
#endif

And then in the sync callback:
gst_video_overlay_set_window_handle(overlay, m_video_window_handle);

As I say, this works on Windows & Linux. But on Mac it doesn't and I'm
wondering whether I am using the wrong id. I'm using this:

#ifdef __WXMAC__
NSView *wxw = GetPeer()->GetWXWidget();
m_video_window_handle = (gulong)wxw;
#endif

But the result is, no video at all. If I omit the call to
gst_video_overlay_set_window_handle then the video displays, but in a
new window of its own, which is not what I want.

- maybe GetPeer()->GetWXWidget() is not really a NSView*
- maybe it's a pointer to the wrong NSView
- maybe gst_video_overlay_set_window_handle doesn't want a NSView* but
something else instead.

Has anyone out there done this? Ideas welcome. Of course I will ask the
GStreamer people if I don't solve it here. But the question of what
exactly is returned by GetPeer()->GetWXWidget() is a wxWidgets question.

Thanks,
Andy Robinson, Seventh String Software, www.seventhstring.com

[Igor Korot]

Hi, Andy,

On Wed, Dec 16, 2015 at 8:01 AM, Andy Robinson <an...@seventhstring.com> wrote:
> Hello, I want to use GStreamer for displaying video in a control within a
> window. In fact it's a subclass of wxControl.
>
> I'm using the documented method which works fine on Linux and Windows, which
> is to determine the native window id of the wxControl like this:
>
> #ifdef __WXGTK__
> GdkWindow *gdk_window = gtk_widget_get_window(m_wxwindow);
> m_video_window_handle = GDK_WINDOW_XID(gdk_window);
> #endif
> #ifdef __WXMSW__
> m_video_window_handle = (gulong)GetHWND();
> #endif
>
> And then in the sync callback:
> gst_video_overlay_set_window_handle(overlay, m_video_window_handle);
>
> As I say, this works on Windows & Linux. But on Mac it doesn't and I'm
> wondering whether I am using the wrong id. I'm using this:
>
> #ifdef __WXMAC__
> NSView *wxw = GetPeer()->GetWXWidget();
> m_video_window_handle = (gulong)wxw;
> #endif

Are you looking for a window handle or a window itself here?

Thank you.

>
> But the result is, no video at all. If I omit the call to
> gst_video_overlay_set_window_handle then the video displays, but in a new
> window of its own, which is not what I want.
>
> - maybe GetPeer()->GetWXWidget() is not really a NSView*
> - maybe it's a pointer to the wrong NSView
> - maybe gst_video_overlay_set_window_handle doesn't want a NSView* but
> something else instead.
>
> Has anyone out there done this? Ideas welcome. Of course I will ask the
> GStreamer people if I don't solve it here. But the question of what exactly
> is returned by GetPeer()->GetWXWidget() is a wxWidgets question.
>
> Thanks,
> Andy Robinson, Seventh String Software, www.seventhstring.com
>

> --
> Please read http://www.wxwidgets.org/support/mlhowto.htm before posting.
>
> To unsubscribe, send email to wx-users+u...@googlegroups.com
> or visit http://groups.google.com/group/wx-users

[Andy Robinson]

Hi Igor,

On 16/12/15 13:25, Igor Korot wrote:
> Hi, Andy,
>
> On Wed, Dec 16, 2015 at 8:01 AM, Andy Robinson <an...@seventhstring.com> wrote:
>> #ifdef __WXMAC__
>> NSView *wxw = GetPeer()->GetWXWidget();
>> m_video_window_handle = (gulong)wxw;
>> #endif
>
> Are you looking for a window handle or a window itself here?

The problem is that I don't really know - GStreamer documentation is not
very clear.

So really I'm looking for suggestions for other things I might try - can
you suggest anything else I might try feeding to GStreamer, other than
wxControl::GetPeer()->GetWXWidget() ?

Regards,

[Andy Robinson]

And P.S. perhaps I should add that I am not a Mac expert at all. AFAIK I
am currently obtaining a NSView*, so if you could tell me how to obtain,
from a wxControl in Mac OS Cocoa, the "window handle or window" that you
mention, I'd be very grateful - then I'll try feeding them to GStreamer
and see what happens!

Regards,
Andy Robinson, Seventh String Software, www.seventhstring.com

[Andy Robinson]

I've just tried this:

NSWindow *NSWindow_from_NSView(NSView *nsv)
{return nsv ? [nsv window] : 0;}

followed by
NSView *nsv = GetPeer()->GetWXWidget();
NSWindow *nsw = NSWindow_from_NSView(nsv);
m_video_window_handle = (gulong)nsw;

but sadly it crashes.

Regards,
Andy Robinson, Seventh String Software, www.seventhstring.com

[Vadim Zeitlin]

On Wed, 16 Dec 2015 13:01:20 +0000 Andy Robinson wrote:

AR> As I say, this works on Windows & Linux. But on Mac it doesn't and I'm
AR> wondering whether I am using the wrong id. I'm using this:

I don't know, you need to find, in GStreamer documentation, source code or
IRC channel, what do you need to give it under OS X.

AR> - maybe GetPeer()->GetWXWidget() is not really a NSView*

It is (when using wxOSX/Cocoa).

AR> - maybe it's a pointer to the wrong NSView

For a plain wxWindow I don't think it can be anything else. But you could
print out its size or just look at it under a debugger to be sure.

AR> - maybe gst_video_overlay_set_window_handle doesn't want a NSView* but
AR> something else instead.

This would indeed be surprising, but surely it mustn't be so difficult to
find an answer to what exactly it needs. This seems safer than just giving
it random things until it accepts one of them.

AR> Has anyone out there done this? Ideas welcome. Of course I will ask the
AR> GStreamer people if I don't solve it here. But the question of what
AR> exactly is returned by GetPeer()->GetWXWidget() is a wxWidgets question.

It is definitely a NSView. BTW, you can use wxWindow::GetHandle() directly
to retrieve it, i.e. use the same code for MSW and OSX.

Regards,
VZ

--
TT-Solutions: wxWidgets consultancy and technical support
http://www.tt-solutions.com/

[Andy Robinson]

If anyone else was interested in this thread, I just thought I'd post
the answer (discovered with help from Sebastian Dröge in the GStreamer
group).

It seems that if you want to use GStreamer to display video on Mac then
GstVideoOverlay is *not* the way to do it. Instead:

See:
http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/tests/examples/avsamplesink
which should work as it is. Then switch it from avsamplebufferlayersink
to caopengllayersink (Sebastian recommends this choice) and it should
work the same.

You can then get the same effect in your wxWidgets app like this, using
caopengllayersink (this happens within a subclass of wxControl, which is
positioned at the place in the window where we want the video to appear):
CALayer *layer;
g_object_get(m_videosink, "layer", &layer, NULL);

NSView *nsv = GetPeer()->GetWXWidget();

[nsv setWantsLayer:YES];
[nsv setLayer:layer];

Thank you Vadim also for your help : you're a true wxWidgets hero.

Regards,

[Vadim Zeitlin]

On Sat, 16 Jan 2016 14:38:51 +0000 Andy Robinson wrote:

AR> If anyone else was interested in this thread, I just thought I'd post
AR> the answer (discovered with help from Sebastian Dröge in the GStreamer
AR> group).
AR>
AR> It seems that if you want to use GStreamer to display video on Mac then
AR> GstVideoOverlay is not the way to do it. Instead:
AR>
AR> See:
AR> http://cgit.freedesktop.org/gstreamer/gst-plugins-bad/tree/tests/examples/avsamplesink
AR> which should work as it is. Then switch it from avsamplebufferlayersink
AR> to caopengllayersink (Sebastian recommends this choice) and it should
AR> work the same.
AR>
AR> You can then get the same effect in your wxWidgets app like this, using
AR> caopengllayersink (this happens within a subclass of wxControl, which is
AR> positioned at the place in the window where we want the video to appear):
AR> CALayer *layer;
AR> g_object_get(m_videosink, "layer", &layer, NULL);
AR> NSView *nsv = GetPeer()->GetWXWidget();
AR> [nsv setWantsLayer:YES];
AR> [nsv setLayer:layer];

Glad to hear that you could have made this work! But now I wonder whether
we should provide a GStreamer-based wxMediaCtrl under Mac using this tip.
Or is the code in src/unix/mediactrl.cpp too GTK-specific?

AR> Thank you Vadim also for your help : you're a true wxWidgets hero.

I don't think I've really helped you with this question but thanks
nevertheless!

[Steve Cookson]

Hello Guys,

This is good to see.

I see also that some work has been done generally on the MediaCtrl
Gstreamer backend to move it to Gst 1.0 and to replace GConf. This is
also very welcome.

What I think would be wonderful, is if we could somehow move to a two
step creation of a MediaCtrl widget to allow the gradual creation of
Gstreamer methods for custom pipelines. Gstreamer is a very popular
media engine and it would be a nice boost for wxWidgets.

The other way would be to directly create a fully functional GStreamer
control, but that might be more work.

What do people think?

Regards

Steve.

[Stefano Mtangoo]

Hi,

On 1/16/2016 11:53 PM, Steve Cookson wrote:
> Hello Guys,
>
> This is good to see.
>
> I see also that some work has been done generally on the MediaCtrl
> Gstreamer backend to move it to Gst 1.0 and to replace GConf. This is
> also very welcome.
>
> What I think would be wonderful, is if we could somehow move to a two
> step creation of a MediaCtrl widget to allow the gradual creation of
> Gstreamer methods for custom pipelines. Gstreamer is a very popular
> media engine and it would be a nice boost for wxWidgets.
>
> The other way would be to directly create a fully functional GStreamer
> control, but that might be more work.

My opinion is wxMediaCtrl will be more strong if it happens to have
GStreamer backend and let users in Windows/Mac choose native backend
when they don't want GST dependencies.
The reasoning behind is GST is cross-platform just as wxWidgets is great
than native multimedia libraries/frameworks

[Joachim Bürmann]

Hello,

I downloaded the last wxWidgets sources from github yesterday and
compiled them against gtk3 with:

../configure --with-gtk3 --enable-unicode --enable-no_rtti --enable-debug

After sucessfully built and installed the libs, I build the widgets demo
in the sample/widgets folder.
The sample compiled without an error, but after starting the program I
saw that all the free space/background is black instead of the
system/theme color or transparency. See attached image.
(The same happens with the controls example).

I'm running an up-to-date Ubuntu 14.04 LTS with the standard Unity desktop.

After some google research I found out, that this is a known bug in the
wxWidgets trac.

http://trac.wxwidgets.org/ticket/10273

Is there any known solution/workaround or fix for this?

Thanks a lot and best regards

Joachim Bürmann

[Joachim Bürmann]

Addendum:

Building wxWidgets against gtk2 works as expected (see attached image).
So it seems an issue with
gtk3.

We (our company) considering to port our applications to wxWidgets 3
(currently we are using 2.8.11) and I would like to use gtk3 for the
Linux port. But I'm a little bit unsure, if gtk3 is already the right
choice for 'production' (I read a lot of pros and cons in the past
years, but it seems off-topic now).

If there are some arguments of or against building wxGTK with gtk3 I
would be glad to hear about experiences of other wxWidgets users in this
matter.

Anyway, thanks a lot for the great wxWidgets library. We are very happy
to use it over the last 15 years.

Best regards

Joachim

[Vadim Zeitlin]

On Sun, 24 Jan 2016 12:48:27 +0100 Joachim Bürmann wrote:

JB> After sucessfully built and installed the libs, I build the widgets demo
JB> in the sample/widgets folder.
JB> The sample compiled without an error, but after starting the program I
JB> saw that all the free space/background is black instead of the
JB> system/theme color or transparency. See attached image.
JB> (The same happens with the controls example).
JB>
JB> I'm running an up-to-date Ubuntu 14.04 LTS with the standard Unity desktop.

I'm afraid this is another Unity-only problem :-( At the very least, I
definitely don't see this with GTK+ 3 under Debian Jessie. Which version of
GTK+ 3 does 14.04 have and does it have any environment variables/whatever
that could be set to disable Ubuntu-specific "improvements" and make it
behave it in a more compatible way as used to be the case in the past with
e.g. overlay scrollbars or app menu?

JB> After some google research I found out, that this is a known bug in the
JB> wxWidgets trac.
JB>
JB> http://trac.wxwidgets.org/ticket/10273

I'm not sure why do you think it's the same problem, AFAICS it isn't. Not
that it really helps, of course.

JB> Is there any known solution/workaround or fix for this?

Right now I really have no idea where does the problem come from, sorry.