Fix Android ChromeDriver viewport screenshot dimensions [chromium/src : main]
Alex Rudenko (Gerrit)
Alex Rudenko added 1 comment![]( "Open in Gerrit")
Classic WebDriver top-level screenshots on Android could return PNG
dimensions based on the Android visible surface instead of the top-level
browsing context viewport. This caused Android-only WPT failures becauseJulio PiubelloI am not sure this is the right approach. Can we fix the underlying CDP behavior on Android? Have we tried fromSurface: false in https://chromedevtools.github.io/devtools-protocol/tot/Page/#method-captureScreenshot?
Alex RudenkoHi Alex! thanks for the review.
I checked the `fromSurface:false` and first concluded it was not a drop-in replacement because Chromium's PageHandler bypasses clip/emulation for that path and captures directly from the browser view. In Android WPT validation it returned a 1080x1545 PNG where classic WebDriver expected 2940x4206.
So this CL keeps the surface capture path and passes an explicit CSS viewport clip, matching the WebDriver's top-levle viewport screenshot contract.
I also updated the overview wording to avoid implying CDP is generally wrong here
Julio PiubelloI believe by default CDP should capture the viewport screenshot by default (from surface or not). Why is there a difference on Android?
@ale...@chromium.org could you please check? I think we should find a fix on the CDP level or at least better understand why the CDP command behaves differently? does Android have a different page handler implementation.
Alex RudenkoI did a deeper look CDP-level check and wrote my thoughts here: https://docs.google.com/document/d/1Jnk_ISpLkrLCDHJqfrkJ_9oVUxF71yVAxMtdGzeJ6ug/edit?usp=sharing
tldr; with no explicit clip, CDP copies the Android compositor surface. For pages without a mobile viewport meta tag, that surface size can differ from the classic WebDriver viewport size because Android page scale is involved.
Let me know if you want us to another approach or if the clip is the narrow fix for that contracte
Julio PiubelloThanks! I got a few questions:
1) Is there CDP emulation enabled or how does the device pixel ratio have the value of 3? is the native dpr of the emulator device?
2) When the screenshot is taken is the viewport actually resized in the emulator to get the scaled results?
3) which part of the WebDriver spec results in the expectation of 2940x4206?Perhaps you have captured screenshots for various combinations as images? I would be interested in taking a look.
Alex Rudenko1 - no CDP device emulation enabled in this probe. it is a normal session with only "goog:chromeOptions.androidPackage" set. I did not call setDeviceMetricsoverride and this CL does not add any device metrics override or resize call.
the value of 3 for devicePixelRatio is the emulator value reported by Chrome. The regenerated probe used the Android 35 x64 with wm size: 1080x1920 and wm density 480.
the page reported:
window.devicePixelRatio = 3
screen.width = 360
screen.height = 640
window.innerWidth = 980
window.innerHeight = 1459The screenshot command is not resizing the emulator viewport. the android-only change passes an explicit CSS viewport clip to Page.captureScreenshot. width: 980, height: 1459 and scale: 1
CDP then returns the image in physical pixels, chromeDriver does not manually multiply by 3
regarding question 3 the 2940x4206 from WPT failure is not hardcoded, the spec describes screenshot caputre for the current top-lvel browsing onctext viewport.
so a page reporting `innerWidth = 980`, `innerHeight = 1402`, and `devicePixelRatio = 3` expects: 980 * 3 = 2940
1402 * 3 = 4206In the regenerated artifact run, the page height was `1459`, so the expected
980 * 3 = 2940
1459 * 3 = 4377I generated the PNGs to show the behavior differences:
https://drive.google.com/drive/folders/1xVEdDrf0qAAVXIqT2odf0jPRvR4FWFu6?usp=sharingimage 1: no params, image 2: {fromSurface:false}, image 3: {clip: viewport}, WebDriver `/screenshot` with this CL
That is the reason this CL uses an explicit CSS viewport clip for Android
classic WebDriver screenshots instead of relying on default CDP no-clip
screenshot behavior or `fromSurface:false`.What to notice:
01-cdp-default-no-params 1080x1848
02 fromSurface:false: 1080x1608
03 explicit viewport clip: 2940x4377
04 WebDriver with this CL: 2940x4377image 3 is CDP with explicit viewport clip
images 3 and 4 have the same dimensions and are byte-for-byte identical, proving WebDriver now uses the same path as the explicit viewport clipImage 2 looks “right” because it captures the visible Android view. But this CL is fixing the classic WebDriver/WPT dimension contract, and the page reported:
window.innerWidth = 980
window.innerHeight = 1459
devicePixelRatio = 3so WPT expects:
980 * 3 = 2940
1459 * 3 = 4377That means fromSurface:false is still returning the smaller surface-sized result:
360 * 3 = 1080
536 * 3 = 1608Regaring the question 2: The confusing part is that CDP’s clipped screenshot path internally uses temporary DevTools screenshot emulation/resize machinery to produce a requested image size. But that is inside the browser capture implementation; it is not an emulator/device viewport resize, and it is not something ChromeDriver is doing via device emulation APIs.
prove three things:
1. Default CDP and fromSurface:false both produce view/surface-sized screenshots, not WPT-expected viewport-sized screenshots.
2. Explicit CDP viewport clip produces the expected physical dimensions.
3. WebDriver with this CL is byte-for-byte identical to explicit CDP viewport clip, so the CL is using the intended path.The remaning work is intentionally split out to keep the size of the CL small. My next CL will address classic element screenshot scale/clipping, which uses a different endpoint and ChromeDriver path from this top-level. and later another cl that only fixes the shadow-root negative-case error maping. Keeping these separate avois mixing top-level viewport screenshot parity, element clipping behavior and webdiver error semantics in one CL
Alex RudenkoThanks! so what is not clear to me why is there a difference in CDP behavior without clip and with clip (especially fromSurface:false). I would have expected the default behavior to use the viewport of the navigable already. Also, is the text overlapping in the screenshots intended? in CDP screenshots without clip I do not see overlaps.
Julio PiubelloMaybe a different angle of the same question: if we run the same CDP screenshot command on desktop (non-Android) without the explicit clip do we get innerWidth * devicePixelRatio x innerHeight * devicePixelRatio dimensions?
Alex RudenkoThanks for the insights, this was a real issue. The repeated/overlapping text was not intended.
I ended up paying too much attention to the dimension and didn't check properly the screenshots.
The difference is that CDP no-clip and explicit-clip use different paths. No-clip captures the current Android composited surface, so it returns the physical surface size and looks correct. Explicit clip enters the CDP clip/emulation path, which computes the requested output from the clip and device scale.
On Android, that path previously requested the larger WebDriver-sized output, but the Java-owned Android view was not actually resized. The captured bitmap stayed surface-sized, and PageHandler tiled it up to the requested size. That caused the repeated text.
I fixed this by passing the requested output size into the surface copy request, so the clipped surface capture produces the requested size directly instead of relying on tiling. ChromeDriver also now preserves the current `visualViewport.scale` in the clip, so the WebDriver screenshot has the expected dimensions without changing the Android visual viewport.
`fromSurface:false` avoids the surface/clip/emulation path, so it also avoids the tiling, but it returns Android view-sized output rather than the Classic WebDriver expected size.
For desktop: in the normal non-emulated case, no-clip CDP usually matches `innerWidth * devicePixelRatio` by `innerHeight * devicePixelRatio`, because the composited surface and viewport-in-device-pixels coincide. Android differs because the physical surface size and the layout viewport times DPR do not necessarily match.
I regenerated the artifacts: `03` is the corrected CDP visual-scale clip, `06` is WebDriver, and they are byte-identical. `04/05` are only diagnostic `clip.scale=1` variants; they are no longer tiled, but they still show the expected zoomed/cropped CDP behavior.
You can check them here:
https://drive.google.com/drive/folders/1sR7zie-XWsb1KO3AkkzZUb6uGWQIukVM?usp=sharing
I left the Android cross-origin iframe exact-byte case expected-failing; that appears to be a separate Android/OOPIF rendering comparison issue, not the repeated/tiled top-level screenshot bug.
So this CL narrows the Android expected failure from 3 iframe failures to 1.
Rhe failure reason changes. Before, cross_origin failed early on screenshot dimensions. After fixing dimensions, it reaches the final exact-byte comparison and fails there. That exact-byte OOPIF mismatch was previously masked by the dimension failure, but the subtest itself was already expected-failing on Android.
For desktop: in the normal non-emulated case, no-clip CDP usually matches innerWidth * devicePixelRatio by innerHeight * devicePixelRatio, because the composited surface and viewport-in-device-pixels coincide. Android differs because the physical surface size and the layout viewport times DPR do not necessarily match
To clarify I believe this is a bug. Although Android differs in implementation, we still expect CDP to do innerWidth * devicePixelRatio by innerHeight * devicePixelRatio there by default. Is there a chance to move the fix to CDP? We should not be doing the script evaluation in the main world to workaround the issue that affects most likely all CDP clients.
Related details
- Alex N. Jose
- Julio Piubello
Code-Coverage
Code-Owners
Code-Review
No-Unresolved-Comments
Review-Enforcement
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |
Julio Piubello (Gerrit)
Julio Piubello added 1 comment![]( "Open in Gerrit")
thanks for the discussion! I will turn this back to draft, and start over with a new point of view and investigation on our system, then push again after internal review. will write up my findings and create a new ticket to be discovered in gitstart
Related details
- Alex N. Jose
- Alex Rudenko
Code-Coverage
Code-Owners
Code-Review
No-Unresolved-Comments
Review-Enforcement
| Inspect html for hidden footers to help with email filtering. To unsubscribe visit settings. |