Re: [chromium-discuss] Re: Screencast frame doesn't trigger every frame as expected
402 views
Skip to first unread message
Eric Seckler
Apr 9, 2018, 2:35:59 AM4/9/18
to Cédric “Nyro” Nirousset, headless-dev
+headless-dev, bcc chromium-discuss.
Screencasts are best-effort, i.e. you'll get the frame that chromium managed to produce in the browser process. If your machine is slowish, this may well be a stale frame.
Page.captureScreenshot avoids this problem, but as you've discovered, is more expensive.
Alternatively, there's a more or less experimental rendering mode that ensures that each produced frame is up-to-date. It's exposed in the HeadlessExperimental domain right now, see this test for an example [1] and bit.ly/headless-rendering for some background reading.
On Sun, 8 Apr 2018, 15:08 Cédric Nirousset, <nyr...@gmail.com> wrote:
Here is the code for the 2 different node script--
Le dimanche 8 avril 2018 15:22:48 UTC+2, Cédric Nirousset a écrit :Hello,I'm not sure if it's a bug or if I can find a workaround, so I'm starting here.I'm using Chrome Canary 67 in MacOs, starting with this command :
expression: 'showFrame(' + currentFrame + ');',
awaitPromise: true
}).then(function(showRes) {
if (!showRes.result) {
throw new Error('Failed showFrame ' + currentFrame);
}
return saveScreenshot(client, width, height, currentFrame);
});For the screenshot, here is the code :
format: 'png',
clip: {
x: 0,
y: 0,
width: width,
height: height,
scale: 1
}
}).then(function(screenshot) {
return new Buffer(screenshot.data, 'base64');
}).then((buffer) => {
let outputFile = filePath + formatFrameNumber(currentFrame) + '.png';
return new Promise((resolve, reject) => {
fs.writeFile(outputFile, buffer, 'base64', async (err) => {
if (err) {
console.error(err);
reject();
} else {
console.log('Screenshot saved to "' + outputFile + '"');
resolve();
}
});
});
}).catch((err) => {
console.error('ERORO WRITE BUFFER '+err);
});For screencast, it's requested with this code just after the load:
console.log('screencastFrame');
lastScreencastData = data;
lastScreencastMetadata = metadata;
Page.screencastFrameAck({ sessionId });
});
await Page.startScreencast({
format: 'png'
});The last screencastFrame is saved in a global variable, and then used when saving is requested:
if (!lastScreencastData || !lastScreencastMetadata) {
throw new Error('screencast frame '+currentFrame+' is empty');
reject();
}
resolve(new Buffer(lastScreencastData, 'base64'));
}).then((buffer) => {
lastScreencastData = false;
let outputFile = filePath + formatFrameNumber(currentFrame) + '.png';
return new Promise((resolve, reject) => {
fs.writeFile(outputFile, buffer, 'base64', async (err) => {
if (err) {
console.error(err);
reject();
} else {
console.log('Screenshot saved to "' + outputFile + '"');
resolve();
}
});
});
}).catch((err) => {
console.error('ERORO WRITE BUFFER '+err);
});In some cases, there is error with "screencast frame XXX is empty".In some other cases, the resulting PNG is the same as the previous one.There is 2 screencastFrame which looks exactly the same.In the HTML, here is the code called by the node script:
frameDiv.innerHTML+= ' '+frame;
console.log('show frame '+frame);
return new Promise(function(resolve, reject) {
requestAnimationFrame(function() {
requestAnimationFrame(function() {
console.log('show frame END '+frame);
resolve({
result: true
});
});
});
});
};As you can see, I already add a double requestAnimationFrame, but it's still not sufficient for screencasting...The difference of timing between the 2 scripts :
- 8 seconds for screenshot
- 1 second for screencast
For screen casting, if I add a setTimeout in the HTML of 100ms, there is no frame missing or problem.But I'd like to avoid this problem in order to be as fast as possible...Could anybody already experience same problem and found any solutions?Thanks in advance.
--
Chromium Discussion mailing list: chromium...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-discuss
Eric Seckler
Apr 10, 2018, 2:45:17 AM4/10/18
to Cédric “Nyro” Nirousset, headless-dev
On Mon, 9 Apr 2018, 14:34 Cédric Nirousset, <nyr...@gmail.com> wrote:
Thanks Eric for your quick answer.After reading the linked documents and testing, I ended with this gotchas:1. As mentioned in the doc, enableBeginFrameControl on Target.createTarget() is not working on MacOs.
Moreover, this should be activated while creating the target in order to make beginFrame enabled and working.
Correct, only windows and Linux for now.
2. In this mode, requestAnimationFrame doesn't seem to be usable. I fixed it by executing this dirty code before starting doing anything related to my stuff in the HTML page:
expression: 'window.requestAnimationFrame = function(clb) {clb();};',
awaitPromise: true
});Am I missing something or is it intended?
rAF and animations will only execute when a beginFrame is sent. If you'd like them to execute more often than you're taking screenshots, you'll need to send some beginFrames without screenshot params (and optionally with noDisplayUpdate=true).
See also this class and its test for some ideas - it is combining beginFrames with virtual time and sends some animation-only BeginFrames. (If you don't need the determinism that virtual time provides, you can do something similar in your client with real time intervals.)
3. If you want to use the screenshot param of the beginFrame, it doesn't seem to work on the first one. Thus, before starting my frame loop, I start with an initial beginFrame to let the next one working with screenshotData.
As far as I can see, the linked test seems to indicate that is intended and/or the expected behavior.
If you use all of these flags, even screenshots on the first BeginFrame should work (use --enable-features=SurfaceSynchronization for the last one):
The commands in HeadlessExperimental are designed to be used with these flags, so may not behave very well without them.
4. For my first try, I didn't want to use the screenshot param for beginFrame and still use the screencast.
But when receiving the result of beginFrame, I have to wait and multiple callback in order to be sure the frame was painted.
Moreover, the screencastAck was always called AFTER the last beginframe result with hasDamage = false.
Yeah I'm not surprised that screencasts behave weirdly with beginFrame. They aren't really designed to work well together - beginFrame gives you a more fine-grained control so there isn't really a need for screencasts. (Screencasts frames can only be produced when a beginFrame is sent.)
Here is the code I used in this try (working)
return client.Runtime.evaluate({
expression: 'showFrame(' + currentFrame + ');',
awaitPromise: true
}).then(function(showRes) {
return waitNoDamageAndScreencast(client);
}).then(function() {
return saveLastScreencast(client, width, height, currentFrame);
}).then(function() {
currentFrame++;
if (currentFrame < totalFrameCount) {
return takeScreenshots(client, width, height, currentFrame, totalFrameCount);
}
return true;
});
};
const waitScreencast = async () => {
return new Promise((resolve, reject) => {
if (lastScreencastData) {
resolve(lastScreencastData);
} else {
console.log('wait for screencast');
return delay(10).then(waitScreencast).then(resolve)
}
});
};
const waitNoDamageAndScreencast = async (client) => {
return client.HeadlessExperimental.beginFrame()
.then(function({hasDamage}) {
if (!hasDamage) {
return true;
}
console.log('damaged');
return waitNoDamageAndScreencast(client);
}).then(waitScreencast);
};This worked, but as you can see, it's not very efficient because there is a loop of test and a delay of 10ms...As far as I can see and understand, there is no way to listen for a frame to be completely painted?
The result of beginFrame (with screenshot params and using the flags mentioned above) will include a screenshot of the fully-painted frame. It's important that you use those flags though, otherwise that's not guaranteed.
5. Regarding the timing, on my debian machine, I now have these result :
- Screenshot version: 4.117s (no image missing)
- initial screencast version: 0.512s (but missing frames, the script might be executed too fast regarding Chrome rendering)
- Screencast with waiting loop: 2.578s (no image missing)
- beginFrame using screenshot param: 2.392s (no image missing)
So it seems the usage of beginFrame with screenshot param is better than simple captureScreenshot, by a factor of almost 2.
I guess I will use this version for now.6. I read that the "hasDamage" might be removed in the future.
Why is that?
It's not really necessary since when there's a screenshot there's always damage (provided you use the flags). And if you set noUpdateDisplay=true if you don't need a screenshot, you're also guaranteed that there won't be any damage.
If you remove it, do you intend to add an event to listen when a frame have been completly painted?
As above - use the flags and this isn't an issue :)
Reply all
Reply to author
Forward
0 new messages