Screencast in headless chromium

[Chao]

Hi All,

I'm trying to screencast the webpage rendered by headless chromium as the source of a VNC server, I tried chrome remote protocol method "Page.startScreencast " & "Page.captureScreenshot",

However, there are serveral issues I can't fix.

• When using "Page.startScreencast", the event only fired when the whole frame is reloaded, is there anyway to fire the event or place a hook whenever the screen changed?
• Since I use the screen change as the source of a VNC, all I need is the dirty rect region in every frame change, is there possible to place an os level hook to get the changed region?
• I want to set the new chrome target width and height, is "Emulation.setDeviceMetricsOverride" the right API to setup it?  I tried start headless_shell with "--window-size=1024,768", but seems didn't work.

Thanks for help.

[Eric Seckler]

On Thu, Jan 5, 2017 at 4:13 AM Chao <zur...@inlight.ai> wrote:

Hi All,

I'm trying to screencast the webpage rendered by headless chromium as the source of a VNC server, I tried chrome remote protocol method "Page.startScreencast " & "Page.captureScreenshot",

However, there are serveral issues I can't fix.

• When using "Page.startScreencast", the event only fired when the whole frame is reloaded, is there anyway to fire the event or place a hook whenever the screen changed?

AFAIK Page.startScreencast should fire Page.screencastFrame events for every frame produced. That said, I haven't actually tried using it. If there are no changes on the page, no new frames are produced, so you wouldn't get updates in such a case. Are you acknowledging each received frame using Page.screencastFrameAck?

• Since I use the screen change as the source of a VNC, all I need is the dirty rect region in every frame change, is there possible to place an os level hook to get the changed region?

I don't think this is easily possible. Page.screencastFrame seems to always encode the whole frame. We don't actually have OS-level windows in headless (we assume there's no display server), so the OS wouldn't observe any visible changes.

• I want to set the new chrome target width and height, is "Emulation.setDeviceMetricsOverride" the right API to setup it?  I tried start headless_shell with "--window-size=1024,768", but seems didn't work.

Emulation.setDeviceMetricsOverride affects layouting sizes, but not the screenshot/screencast size. For that, you can use Emulation.setVisibleSize. That said, --window-size works, too, but you have to provide the size as 1024x768. We should probably also support the comma-separated version, filed a bug for that.

 

Thanks for help.

--
You received this message because you are subscribed to the Google Groups "headless-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to headless-dev...@chromium.org.
To post to this group, send email to headle...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/headless-dev/f637a56f-1cfd-4145-90e0-98e4905a4808%40chromium.org.

[Chao]

Hi Eric,

Thanks for reply.

On Thursday, January 5, 2017 at 1:15:55 AM UTC-8, Eric Seckler wrote:

On Thu, Jan 5, 2017 at 4:13 AM Chao <zur...@inlight.ai> wrote:

Hi All,

I'm trying to screencast the webpage rendered by headless chromium as the source of a VNC server, I tried chrome remote protocol method "Page.startScreencast " & "Page.captureScreenshot",

However, there are serveral issues I can't fix.

• When using "Page.startScreencast", the event only fired when the whole frame is reloaded, is there anyway to fire the event or place a hook whenever the screen changed?

AFAIK Page.startScreencast should fire Page.screencastFrame events for every frame produced. That said, I haven't actually tried using it. If there are no changes on the page, no new frames are produced, so you wouldn't get updates in such a case. Are you acknowledging each received frame using Page.screencastFrameAck?

Yes, I missed this part, and after I send the Ack to chrome, the screencastFrame acts very well. : ) 

• Since I use the screen change as the source of a VNC, all I need is the dirty rect region in every frame change, is there possible to place an os level hook to get the changed region?

I don't think this is easily possible. Page.screencastFrame seems to always encode the whole frame. We don't actually have OS-level windows in headless (we assume there's no display server), so the OS wouldn't observe any visible changes.

 Still I'm trying to find where I can diff the screencast and calculate dirty region to send. Though there's no display server in headless chrome, can you tell me where is the best place I can put my diff hook when headless chrome encoding the whole frame?

• I want to set the new chrome target width and height, is "Emulation.setDeviceMetricsOverride" the right API to setup it?  I tried start headless_shell with "--window-size=1024,768", but seems didn't work.

Emulation.setDeviceMetricsOverride affects layouting sizes, but not the screenshot/screencast size. For that, you can use Emulation.setVisibleSize. That said, --window-size works, too, but you have to provide the size as 1024x768. We should probably also support the comma-separated version, filed a bug for that.

Thanks, it works. 

[Eric Seckler]

On Thu, Jan 5, 2017, 20:08 Chao <zur...@inlight.ai> wrote:

Hi Eric,

Thanks for reply.

On Thursday, January 5, 2017 at 1:15:55 AM UTC-8, Eric Seckler wrote:

On Thu, Jan 5, 2017 at 4:13 AM Chao <zur...@inlight.ai> wrote:

Hi All,

I'm trying to screencast the webpage rendered by headless chromium as the source of a VNC server, I tried chrome remote protocol method "Page.startScreencast " & "Page.captureScreenshot",

However, there are serveral issues I can't fix.

• When using "Page.startScreencast", the event only fired when the whole frame is reloaded, is there anyway to fire the event or place a hook whenever the screen changed?

AFAIK Page.startScreencast should fire Page.screencastFrame events for every frame produced. That said, I haven't actually tried using it. If there are no changes on the page, no new frames are produced, so you wouldn't get updates in such a case. Are you acknowledging each received frame using Page.screencastFrameAck?

Yes, I missed this part, and after I send the Ack to chrome, the screencastFrame acts very well. : ) 

Great to hear!

• Since I use the screen change as the source of a VNC, all I need is the dirty rect region in every frame change, is there possible to place an os level hook to get the changed region?

I don't think this is easily possible. Page.screencastFrame seems to always encode the whole frame. We don't actually have OS-level windows in headless (we assume there's no display server), so the OS wouldn't observe any visible changes.

 Still I'm trying to find where I can diff the screencast and calculate dirty region to send. Though there's no display server in headless chrome, can you tell me where is the best place I can put my diff hook when headless chrome encoding the whole frame?

I'm not entirely sure what sort of hook you're looking for. 

The relevant code that encodes the frames lives in content::protocol::PageHandler::ScreencastFrameCaptured. Modifying what this does is probably not convenient, though. Maybe you're better off doing the diff-comparison in your DevTools client when you receive a screencastFrame event?

• I want to set the new chrome target width and height, is "Emulation.setDeviceMetricsOverride" the right API to setup it?  I tried start headless_shell with "--window-size=1024,768", but seems didn't work.

Emulation.setDeviceMetricsOverride affects layouting sizes, but not the screenshot/screencast size. For that, you can use Emulation.setVisibleSize. That said, --window-size works, too, but you have to provide the size as 1024x768. We should probably also support the comma-separated version, filed a bug for that.

Thanks, it works. 

 

Thanks for help.

--
You received this message because you are subscribed to the Google Groups "headless-dev" group.

To unsubscribe from this group and stop receiving emails from it, send an email to headless-dev...@chromium.org.
To post to this group, send email to headle...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/headless-dev/f637a56f-1cfd-4145-90e0-98e4905a4808%40chromium.org.

--
You received this message because you are subscribed to the Google Groups "headless-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to headless-dev...@chromium.org.
To post to this group, send email to headle...@chromium.org.

To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/headless-dev/fb945960-136d-4b99-9f13-c142c556a224%40chromium.org.

[aba...@gmail.com]

Thanks Eric, the acking was necessary in order to stream the shots.