Only the first frame is captured by the DesktopCapturer on mac

1,362 views
Skip to first unread message

Olivier BERTHONNEAU

unread,
Jun 30, 2017, 6:26:21 AM6/30/17
to discuss-webrtc
Hi,

I have a C++ application on mac capturing the desktop using WebRTC's desktop capturer and sending it as video frames to a connected browser.
All the connection seems to work as no errors are reported in the logs.

The DesktopCapturer::CaptureFrame() method is called periodically from another thread and OnCaptureResult is called back accordingly with a std::unique_ptr<webrtc::DesktopFrame> as parameter.

Here is my implementation of OnCaptureResult:

void Capturer::OnCaptureResult(webrtc::DesktopCapturer::Result result,

                              std::unique_ptr<webrtc::DesktopFrame> desktopFrame)

{

     if (result != webrtc::DesktopCapturer::Result::SUCCESS)

   {

       return; // Obviously never called

   }


   int width = desktopFrame->size().width();

   int height = desktopFrame->size().height();

   rtc::scoped_refptr<webrtc::I420Buffer> res_i420_frame = webrtc::I420Buffer::Create(width, height);


   webrtc::ConvertToI420(webrtc::VideoType::kARGB,

                      desktopFrame->data(),

                      0, 0,

                      width, height,

                      0,

                      webrtc::kVideoRotation_0,

                      res_i420_frame);

       

   webrtc::VideoFrame frame = webrtc::VideoFrame(res_i420_frame, 0, 0, webrtc::kVideoRotation_0);

   this->OnFrame(frame, width, height);

}


This code basically re-encode the ARBG DesktopFrame into a I420 frame then feed it to OnFrame.

When my browser connects to my C++ peer, I can immediately see a mirror of my screen in the video tag.
However, only the first frame of the video is displayed, even though OnCaptureResult (and thus OnFrame) is called periodically.

I first suspected my video was simply not playing on the browser side. So I modified the above code to send a black frame on one every two frame.
As expected my "video" blinks showing that why video tag is indeed receiving data from my C++ app.

Is there something I'm missing ? Why is DesktopCapturer always returning the same frame even though I call DesktopCapturer::CaptureFrame() periodically ?

Olivier BERTHONNEAU

unread,
Jul 1, 2017, 7:02:46 AM7/1/17
to discuss-webrtc
I investigated further more. It looks like the CGDisplayStreamFrameAvailableHandler is never called, which means no dirty regions are ever reported back to the capturer.

I can see this because the handler defined in ScreenCapturerMac::RegisterRefreshAndMoveHandlers is never called. Therefor, after the first frame is sent, ScreenCapturerHelper::TakeInvalidRegion will always return an empty invalid region, leading to no DesktopRect ever being pulled back from the screen.

Looking at how the display stream is started we can see it uses CGDisplayStreamCreate (https://developer.apple.com/documentation/coregraphics/1455170-cgdisplaystreamcreate).
There is another function for creating a display stream called CGDisplayStreamCreateWithDispatchQueue (https://developer.apple.com/documentation/coregraphics/1454968-cgdisplaystreamcreatewithdispatc).

Both seem to achieve the same thing but using a different implementation.

However, for some reason, implementing the capturer with this last function works on my macOS Sierra 10.12.5.

I am not sure of the implications of switching those functions. Nor do I understand why the current code doesn't work on my setup.

Is there anyone aware of the DesktopCapturer implementation on mac who could help me before I file a bug report ? 

Alexandre GOUAILLARD

unread,
Jul 2, 2017, 4:00:35 PM7/2/17
to discuss...@googlegroups.com
hi olivier,

I can help.

Is your problem in webrtc standalone, or in chrome? (not the same bug tracker).

Alex.

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrtc+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/e7562ace-a08c-4fc5-bff0-f50b94cfdb53%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.



--
Alex. Gouaillard, PhD, PhD, MBA
------------------------------------------------------------------------------------
President - CoSMo Software Consulting, Singapore
------------------------------------------------------------------------------------

Olivier BERTHONNEAU

unread,
Jul 2, 2017, 5:46:01 PM7/2/17
to discuss-webrtc
Hi Alex,

Yes my problem is in WebRTC standalone.
It is a C++ app capturing frames on a mac and transmitting to a chrome client. The chrome peer is just a viewer and do no capture frames.

Olivier
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.

Alexandre GOUAILLARD

unread,
Jul 2, 2017, 6:32:51 PM7/2/17
to discuss...@googlegroups.com
then the bug tracker is here:

and the explanation about how to proceed for the content is here:

To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrtc+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/discuss-webrtc/e5f7a172-b9b1-46ea-a950-194d9d3979cf%40googlegroups.com.

For more options, visit https://groups.google.com/d/optout.

Xander Dumaine

unread,
Jul 3, 2017, 5:45:53 PM7/3/17
to discuss-webrtc
Just for a sanity check: does your viewer <video/> tag have autoplay=true? If not, it would only show the first frame and no updates until calling .play() on the element. I've seen this be the cause of the same symptoms for others a few times in the past.

Olivier BERTHONNEAU

unread,
Jul 4, 2017, 3:58:49 AM7/4/17
to discuss-webrtc
Hi Xander,

Yes, autoplay is enabled.
I can verify the encoding, transport and playback in the browser work if I modify the above code to insert a black frame one every two frame.

Olivier BERTHONNEAU

unread,
Jul 9, 2017, 11:59:27 AM7/9/17
to discuss-webrtc

Olivier BERTHONNEAU

unread,
Oct 18, 2017, 7:00:41 PM10/18/17
to discuss-webrtc
Ok so I found what happens thanks to people on the bug tracker.
Turns out the DesktopCapturer on Mac needs a Core Foundation run loop to process events on the capture thread.

So I just ran a little 

CFRunLoopRunInMode(kCFRunLoopDefaultMode, 0, true);


In my main loop and got it sorted out.

sajid....@cosmosoftware.io

unread,
Sep 5, 2018, 9:38:09 AM9/5/18
to discuss-webrtc
Hi Olivier,

I am trying to do the same but in my case, if I call CaptureFrame of the DesktopCapturer repeatedly with delay of less than 0.5 seconds, I get the following error:

#

# Fatal error in ../../modules/desktop_capture/screen_capturer_mac.mm, line 381

# last system error: 316

# Check failed: !queue_.current_frame() || !queue_.current_frame()->IsShared()

# 

#


So potentially, I am able to get 2 fps. Did you encounter any such problem?


Moreover I'd be interested to see the code using CFRunLoop.


Thanks!

Olivier BERTHONNEAU

unread,
Sep 14, 2018, 11:05:25 AM9/14/18
to discuss-webrtc
HI,

I haven't work on this matter for almost a year. Here are some informations I can think of that may be able to guide you.

I never encountered the fatal error you mentioned.
Last time my test program has been tested was on WebRTC 63.
I added this line in my main event loop to have it working. In my case it's in a different thread but I would try to call that before capturing a frame periodically. 

CFRunLoopRunInMode(kCFRunLoopDefaultMode, 1, false);

Hope this helps,


Olivier

Spencer Dixon

unread,
Nov 1, 2018, 4:54:55 PM11/1/18
to discuss-webrtc
Did you ever find a solution for this?  I'm running into the same issue :(

Jacques-Olivier Haché

unread,
Aug 19, 2019, 11:34:40 PM8/19/19
to discuss-webrtc
I successfully applied Olivier's solution.
This was very useful, thanks !

A couple of additional points: 
  • used CFRunLoopRunInMode(kCFRunLoopDefaultMode, 0, true); 
  • I placed the all right above my call to captureFrame();
  • must run on the same thread as the thread that set up the CFRunLoopAddSource. In my case, the worker thread.
  • I had to do one call per frame captured (hence above captureFrame();)
  • I don't advice setting the loop for 1 sec, as it would block the worker thread
Cheers,
JOH

The V

unread,
Oct 4, 2019, 4:24:19 AM10/4/19
to discuss...@googlegroups.com
Those instructions didn't change anything for me, still got the first frame only. Looks like I never get CGDisplayStream notifications from macOS (I'm talking about handlers set up in RegisterRefreshAndMoveHandlers), so for me the only solution was to invalidate the whole screen in every ScreenCapturerMac::CaptureFrame:
  helper_.InvalidateScreen(screen_pixel_bounds_.size());

No idea what I'm doing wrong. What I'm working on is not WebRTC solution from the ground up though, I just extract some VOIP/SS features from WebRTC code base.

--

---
You received this message because you are subscribed to the Google Groups "discuss-webrtc" group.
To unsubscribe from this group and stop receiving emails from it, send an email to discuss-webrt...@googlegroups.com.

Tien Nguyen

unread,
Jul 31, 2020, 4:37:34 AM7/31/20
to discuss-webrtc
If you call CapturerFrame() in a worker thread, try to use a dispatch_queue_t within CGDisplayStreamCreateWithDispatchQueue instead of using CGDisplayStreamCreate with 
CFRunLoop. Maybe it's works.


Vào 17:26:21 UTC+7 Thứ Sáu, ngày 30 tháng 6 năm 2017, Olivier BERTHONNEAU đã viết:
Reply all
Reply to author
Forward
0 new messages