org.chromium.android_webview.test.AwSettingsTest#testLoadWithOverviewModeWithTwoViews fails on local

35 views
Skip to first unread message

Takashi Toyoshima

unread,
Aug 24, 2016, 4:22:58 AM8/24/16
to android-w...@chromium.org
Hi WebView experts,

My CL https://codereview.chromium.org/2257743002/ makes some WebView tests fail on trybot.

Here is a failed tests list
org.chromium.android_webview.test.AwSettingsTest#testLoadWithOverviewModeWithTwoViews
org.chromium.android_webview.test.AwSettingsTest#testLoadWithOverviewModeViewportTagWithTwoViews
org.chromium.android_webview.test.AwSettingsTest#testLoadWithOverviewModeWithTwoViews with {--webview-sandboxed-renderer}
org.chromium.android_webview.test.AwSettingsTest#testLoadWithOverviewModeViewportTagWithTwoViews with {--webview-sandboxed-renderer}

But, when I run these tests on my local machine, they fail even without my change.
Note: Local run fails with assertion failure, but it fails with TimeoutException(: waitForCallback timed out!) on trybots. So the failure reason is different. Thus, I can not reproduce and investigate what happens on the try.

Here is the step I did:

- ninja android_webview_test_apk
- ${out}/bin/run_android_webview_test_apk -vv -f org.chromium.android_webview.test.AwSettingsTest#testLoadWithOverviewModeWithTwoViews

Error log is here:
C   60.375s Main  ********************************************************************************
C   60.375s Main  Detailed Logs                                                  
C   60.375s Main  ********************************************************************************
C   60.375s Main  [FAIL] org.chromium.android_webview.test.AwSettingsTest#testLoadWithOverviewModeWithTwoViews:
C   60.375s Main  junit.framework.AssertionFailedError: expected:<1.0> but was:<0.25>
C   60.375s Main        at org.chromium.android_webview.test.AwSettingsTest$AwSettingsLoadWithOverviewModeTestHelper.doEnsureSettingHasValue(AwSettingsTest.java:1348)
C   60.375s Main        at org.chromium.android_webview.test.AwSettingsTest$AwSettingsLoadWithOverviewModeTestHelper.doEnsureSettingHasValue(AwSettingsTest.java:1298)
C   60.376s Main        at org.chromium.android_webview.test.AwSettingsTest$AwSettingsTestHelper.ensureSettingHasValue(AwSettingsTest.java:144)
C   60.376s Main        at org.chromium.android_webview.test.AwSettingsTest$AwSettingsTestHelper.ensureSettingHasInitialValue(AwSettingsTest.java:88)
C   60.376s Main        at org.chromium.android_webview.test.AwSettingsTest.runPerViewSettingsTest(AwSettingsTest.java:2988)
C   60.376s Main        at org.chromium.android_webview.test.AwSettingsTest.testLoadWithOverviewModeWithTwoViews(AwSettingsTest.java:2660)
C   60.376s Main        at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214)
C   60.376s Main        at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199)
C   60.376s Main        at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:192)
C   60.376s Main        at org.chromium.base.test.BaseTestResult.runParameterized(BaseTestResult.java:161)
C   60.376s Main        at org.chromium.base.test.BaseTestResult.run(BaseTestResult.java:124)
C   60.376s Main        at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191)
C   60.376s Main        at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176)
C   60.376s Main        at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:555)
C   60.376s Main        at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1879)
C   60.377s Main__                                                               
C   60.377s Main  [FAIL] org.chromium.android_webview.test.AwSettingsTest#testLoadWithOverviewModeWithTwoViews with {--webview-sandboxed-renderer}:
C   60.377s Main  junit.framework.AssertionFailedError: expected:<1.0> but was:<0.25>
C   60.377s Main        at org.chromium.android_webview.test.AwSettingsTest$AwSettingsLoadWithOverviewModeTestHelper.doEnsureSettingHasValue(AwSettingsTest.java:1348)
C   60.377s Main        at org.chromium.android_webview.test.AwSettingsTest$AwSettingsLoadWithOverviewModeTestHelper.doEnsureSettingHasValue(AwSettingsTest.java:1298)
C   60.377s Main        at org.chromium.android_webview.test.AwSettingsTest$AwSettingsTestHelper.ensureSettingHasValue(AwSettingsTest.java:144)
C   60.377s Main        at org.chromium.android_webview.test.AwSettingsTest$AwSettingsTestHelper.ensureSettingHasInitialValue(AwSettingsTest.java:88)
C   60.377s Main        at org.chromium.android_webview.test.AwSettingsTest.runPerViewSettingsTest(AwSettingsTest.java:2992)
C   60.377s Main        at org.chromium.android_webview.test.AwSettingsTest.testLoadWithOverviewModeWithTwoViews(AwSettingsTest.java:2660)
C   60.377s Main        at android.test.InstrumentationTestCase.runMethod(InstrumentationTestCase.java:214)
C   60.377s Main        at android.test.InstrumentationTestCase.runTest(InstrumentationTestCase.java:199)
C   60.377s Main        at android.test.ActivityInstrumentationTestCase2.runTest(ActivityInstrumentationTestCase2.java:192)
C   60.378s Main        at org.chromium.base.test.BaseTestResult.runParameterized(BaseTestResult.java:161)
C   60.378s Main        at org.chromium.base.test.BaseTestResult.run(BaseTestResult.java:124)
C   60.378s Main        at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191)
C   60.378s Main        at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176)
C   60.378s Main        at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:555)
C   60.378s Main        at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1879)
C   60.378s Main  ********************************************************************************
C   60.378s Main  Summary                                                        
C   60.378s Main  ********************************************************************************
C   60.379s Main  [==========] 2 tests ran.                                      
C   60.379s Main  [  PASSED  ] 0 tests.                                          
C   60.379s Main  [  FAILED  ] 2 tests, listed below:                            
C   60.379s Main  [  FAILED  ] org.chromium.android_webview.test.AwSettingsTest#testLoadWithOverviewModeWithTwoViews
C   60.379s Main  [  FAILED  ] org.chromium.android_webview.test.AwSettingsTest#testLoadWithOverviewModeWithTwoViews with {--webview-sandboxed-renderer}
C   60.379s Main__                                                               
C   60.379s Main  2 FAILED TESTS                                                 
C   60.379s Main  ********************************************************************************

If I remove assertions, it get to be flaky and sometime raise similar TimeoutException, and sometime pass.


--
Takashi Toyoshima
Software Engineer, Google

Hui Shu

unread,
Aug 24, 2016, 2:01:01 PM8/24/16
to Takashi Toyoshima, android-w...@chromium.org
It passed for me locally.

What device did you use? What Android version is it?

--
You received this message because you are subscribed to the Google Groups "android-webview-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-webview-dev+unsub...@chromium.org.
To post to this group, send email to android-webview-dev@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/android-webview-dev/CAFWCB1%3DHEyqoCNzDXihyBuKsWp1Km_RUCqicp-Bc5CkHOo6oiA%40mail.gmail.com.

Bo Liu

unread,
Aug 24, 2016, 11:23:30 PM8/24/16
to Hui Shu, Takashi Toyoshima, android-w...@chromium.org
Sorry I meant to look into this today, but didn't have time.

Unfortunately android tests are quriky. Most of the time when things don't reproduce locally, it's due to some difference in configuration. So to start, the bot that failed use nexus 5 (hammerhead) devices, running kitkat build KTU84P. Can you try running your test again in that setup and see if it reproduces?

To unsubscribe from this group and stop receiving emails from it, send an email to android-webview-dev+unsubscribe...@chromium.org.

--
You received this message because you are subscribed to the Google Groups "android-webview-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-webview-dev+unsub...@chromium.org.
To post to this group, send email to android-webview-dev@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/android-webview-dev/CAAuAVnmK%2BDkNbVJPoqNb-LQDvYe5t36tvPVEFhbWZK8b-n85gg%40mail.gmail.com.

Takashi Toyoshima

unread,
Aug 25, 2016, 3:23:27 AM8/25/16
to bo...@chromium.org, Hui Shu, android-w...@chromium.org
On Thu, Aug 25, 2016 at 12:22 PM, Bo Liu <bo...@chromium.org> wrote:
Sorry I meant to look into this today, but didn't have time.

Unfortunately android tests are quriky. Most of the time when things don't reproduce locally, it's due to some difference in configuration. So to start, the bot that failed use nexus 5 (hammerhead) devices, running kitkat build KTU84P. Can you try running your test again in that setup and see if it reproduces?

I ran on Pixel C + Android M, and Nexus 5X + Android N.
So, this seems to be the reason why I could not reproduce.
It's hard to setup a real Nexus 5 + KK now, but the test passes on SDK's emulatir with KK arm + Nexus 5 configuration. So, I would be able to investigate it somehow on this environment.

Thanks.
 

--
You received this message because you are subscribed to the Google Groups "android-webview-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to android-webview-dev+unsub...@chromium.org.
To post to this group, send email to android-webview-dev@chromium.org.

Takashi Toyoshima

unread,
Aug 25, 2016, 4:59:35 AM8/25/16
to bo...@chromium.org, Hui Shu, android-w...@chromium.org
KK x86 also looks working. This is a good news to work with an emulated device.
But, I need a kind of workaround to support KK x86 to run a test by catapult; https://codereview.chromium.org/2275253002/




--
Takashi Toyoshima
Software Engineer, Google

Bo Liu

unread,
Aug 25, 2016, 4:58:47 PM8/25/16
to Takashi Toyoshima, Hui Shu, android-w...@chromium.org
Looks like your PS2 has the fix for webview?

Takashi Toyoshima

unread,
Aug 26, 2016, 7:07:03 AM8/26/16
to bo...@chromium.org, Hui Shu, android-w...@chromium.org
On Fri, Aug 26, 2016 at 5:58 AM, Bo Liu <bo...@chromium.org> wrote:
Looks like your PS2 has the fix for webview?

Yeah, thanks to your helps, I understood what happened there, and had a fix for the WebView tests, though it made other tests fail :/
Reply all
Reply to author
Forward
0 new messages