CTS failures on ICS

[John C. Vernaleo]

I've been working with the CTS code for ICS recently and I'm encountering a number of test case failures that I don't understand.

I started by building both android and CTS based on the git branch android-cts-verifier-4.0.3_r1.  As far as I can tell, both build just fine.  I've repeated this test with both eng and userdebug builds.  I can flash my test hardware and I then follow the instructions for CTS (I've done it both wirelessly and over USB with no differences in the cases I care about).

Of the ~17K results, 45 fail which is pretty good.  I can explain away a few of them, but there are also some that seem to be timeout related that I don't understand.

The worst offenders seem to be in the package android.media.

From the results file, I get these results for many of the media local video tests:

testLocalVideo_3gp_H263_176x144_300kbps_25fps_AAC_Stereo_24kbps_11025Hz

fail

Test failed to run to completion. Reason: 'Failed to receive adb shell test output within 600000 ms. Test may have timed out, or adb connection to device became unresponsive'. Check device logcat for details

When I look at logcat, the output when these failing tests are running looks like:

I/AwesomePlayer(   93): after SEEK_VIDEO_ONLY we're late by 387.07 secs
I/AwesomePlayer(   93): we're much too late (387.09 secs), video skipping ahead
repeated many many times.

Eventually, after enough of these, the whole thing stops running the tests (although if I resume it, it carries on just fine after that).  Interestingly, this does not happen for all of the local video tests, just some of them.

Has anyone seen anything similar with tests timeouts like this in video?  It is especially annoying since each test takes 10 minutes (600000ms) to time out, slowing down my testing dramatically.

Thanks for any input.
John

[Guru Nagarajan]

Hello,

 You will see a timeout if the device does not respond or freezes. In this scenario the test harness wil print the message you have below. The test is a local video test i.e the file is copied to the device. The file below in context is a low resolution video and I am assuming your hardware is capable of playing it. I have not been able to replicate this, but have seen similar reports wherein a test failure or a crash can result in the test harness being in a state where it hangs. Can you provide a bug report of this issue? 

Thanks,

Guru 

--
You received this message because you are subscribed to the "Android Building" mailing list.
To post to this group, send email to android-...@googlegroups.com
To unsubscribe from this group, send email to
android-buildi...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/android-building?hl=en

[John C. Vernaleo]

I've posted a bug with whatever info I could come up with at:

https://code.google.com/p/android/issues/detail?id=29687

In the meantime, is there any way to shorten the timeout value so things will at least fail faster so I can speed up my testing?

Thanks.
John

On Friday, April 27, 2012 12:20:53 PM UTC-4, Guru Nagarajan wrote:

Hello,

 You will see a timeout if the device does not respond or freezes. In this scenario the test harness wil print the message you have below. The test is a local video test i.e the file is copied to the device. The file below in context is a low resolution video and I am assuming your hardware is capable of playing it. I have not been able to replicate this, but have seen similar reports wherein a test failure or a crash can result in the test harness being in a state where it hangs. Can you provide a bug report of this issue? 

Thanks,

Guru 

On Wed, Apr 25, 2012 at 4:01 PM, John C. Vernaleo <> wrote:

I've been working with the CTS code for ICS recently and I'm encountering a number of test case failures that I don't understand.

I started by building both android and CTS based on the git branch android-cts-verifier-4.0.3_r1.  As far as I can tell, both build just fine.  I've repeated this test with both eng and userdebug builds.  I can flash my test hardware and I then follow the instructions for CTS (I've done it both wirelessly and over USB with no differences in the cases I care about).

Of the ~17K results, 45 fail which is pretty good.  I can explain away a few of them, but there are also some that seem to be timeout related that I don't understand.

The worst offenders seem to be in the package android.media.

From the results file, I get these results for many of the media local video tests:

testLocalVideo_3gp_H263_176x144_300kbps_25fps_AAC_Stereo_24kbps_11025Hz

fail

Test failed to run to completion. Reason: 'Failed to receive adb shell test output within 600000 ms. Test may have timed out, or adb connection to device became unresponsive'. Check device logcat for details

When I look at logcat, the output when these failing tests are running looks like:

I/AwesomePlayer(   93): after SEEK_VIDEO_ONLY we're late by 387.07 secs
I/AwesomePlayer(   93): we're much too late (387.09 secs), video skipping ahead
repeated many many times.

Eventually, after enough of these, the whole thing stops running the tests (although if I resume it, it carries on just fine after that).  Interestingly, this does not happen for all of the local video tests, just some of them.

Has anyone seen anything similar with tests timeouts like this in video?  It is especially annoying since each test takes 10 minutes (600000ms) to time out, slowing down my testing dramatically.

Thanks for any input.
John

--
You received this message because you are subscribed to the "Android Building" mailing list.

To post to this group, send email to android-building@googlegroups.com

To unsubscribe from this group, send email to

android-building+unsubscribe@googlegroups.com