Running angle_end2end_tests on Android outside of Google?

[Chris Dalton]

Hello ANGLE team!

Trying to run the angle_end2end_tests on Android, test_runner.py fails while attempting to download adb, even after running "gcloud auth login":

py_utils.cloud_storage.CloudStorageError: Your credentials are invalid. For more help, see "gsutil help creds", or re-run the gsutil config command (see "gsutil help config").

Is there something I can do outside of Google to authenticate and get this script working?

Full output:

csmartdalton@tank:~/code/angle/out/android$ ../../build/android/test_runner.py gtest --output-directory . --suite angle_end2end_tests
E    1.284s Main  Unrecognized error occurred.
Traceback (most recent call last):
  File "../../build/android/test_runner.py", line 1207, in main
    return RunTestsCommand(args, result_sink_client)
  File "../../build/android/test_runner.py", line 822, in RunTestsCommand
    return RunTestsInPlatformMode(args, result_sink_client)
  File "../../build/android/test_runner.py", line 1002, in RunTestsInPlatformMode
    env = environment_factory.CreateEnvironment(
  File "/home/csmartdalton/code/angle/build/android/pylib/base/environment_factory.py", line 29, in CreateEnvironment
    return local_device_environment.LocalDeviceEnvironment(
  File "/home/csmartdalton/code/angle/build/android/pylib/local/device/local_device_environment.py", line 137, in __init__
    adb_dir = os.path.dirname(adb_wrapper.AdbWrapper.GetAdbPath())
  File "/home/csmartdalton/code/angle/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 301, in GetAdbPath
    return cls._adb_path.read()
  File "/home/csmartdalton/code/angle/third_party/catapult/devil/devil/utils/lazy/weak_constant.py", line 39, in read
    self._val = initializer_thread.GetReturnValue()
  File "/home/csmartdalton/code/angle/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 87, in GetReturnValue
    self.ReraiseIfException()
  File "/home/csmartdalton/code/angle/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 83, in ReraiseIfException
    raise self._exc_info[1]
  File "/home/csmartdalton/code/angle/third_party/catapult/devil/devil/utils/reraiser_thread.py", line 94, in run
    self._ret = self._func(*self._args, **self._kwargs)
  File "/home/csmartdalton/code/angle/third_party/catapult/devil/devil/android/sdk/adb_wrapper.py", line 89, in _FindAdb
    return devil_env.config.FetchPath('adb')
  File "/home/csmartdalton/code/angle/third_party/catapult/devil/devil/devil_env.py", line 175, in FetchPath
    return self._dm.FetchPath(dependency, GetPlatform(arch, device))
  File "/home/csmartdalton/code/angle/third_party/catapult/dependency_manager/dependency_manager/manager.py", line 130, in FetchPath
    path, _ = self.FetchPathWithVersion(dependency, platform)
  File "/home/csmartdalton/code/angle/third_party/catapult/dependency_manager/dependency_manager/manager.py", line 93, in FetchPathWithVersion
    path = dependency_info.GetRemotePath()
  File "/home/csmartdalton/code/angle/third_party/catapult/dependency_manager/dependency_manager/dependency_info.py", line 84, in GetRemotePath
    return self._cloud_storage_info.GetRemotePath()
  File "/home/csmartdalton/code/angle/third_party/catapult/dependency_manager/dependency_manager/cloud_storage_info.py", line 78, in GetRemotePath
    cloud_storage.GetIfHashChanged(
  File "/home/csmartdalton/code/angle/third_party/catapult/common/py_utils/py_utils/cloud_storage.py", line 531, in GetIfHashChanged
    _GetLocked(bucket, cs_path, download_path)
  File "/home/csmartdalton/code/angle/third_party/catapult/common/py_utils/py_utils/cloud_storage.py", line 444, in _GetLocked
    _RunCommand(['cp', url, partial_download_path.name])
  File "/home/csmartdalton/code/angle/third_party/catapult/common/py_utils/py_utils/cloud_storage.py", line 161, in _RunCommand
    raise GetErrorObjectForCloudStorageStderr(stderr.decode('utf-8'))
py_utils.cloud_storage.CloudStorageError: Your credentials are invalid. For more help, see "gsutil help creds", or re-run the gsutil config command (see "gsutil help config").

Thanks!

-Chris

[Jamie Madill]

Hey Chris,

We're working on an alternative wrapper script for running tests. Roman, is there a current way for a non-Googler to use your script to run end2end tests?

--
You received this message because you are subscribed to the Google Groups "angleproject" group.
To unsubscribe from this group and stop receiving emails from it, send an email to angleproject...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/angleproject/CABbYZgiiKkiZFCvYAQDw6F7KkxQxvoRd4Sm0zktZuzMx2k%3DVKQ%40mail.gmail.com.

[Roman Lavrov]

Hi Chris,

https://crrev.com/c/3691048 might be an option, example:

% autoninja -C out/Android angle_end2end_tests

% (cd out/Android; ../../src/tests/run_angle_android_test.py angle_end2end_tests --filter='*WebGL*')

It's very recent though so there be dragons.

Note that results can be device-dependent too. For instance, I noticed that those WebGL tests above pass on some of the devices but fail on others.

Could you share a bit more details about your use case? Do you need to run a specific test or is this for broad validation? If it's the latter, does uploading a change to Gerrit then CQ dry run work?

Thanks,

Roman.

[Chris Dalton]

Hi Roman,

Sorry for the suuper late delay! I'm working the ANGLE_shader_pixel_local_storage extension with the WebGL working group and am now at a point where I need to test locally on android.

Is  run_angle_android_test.py still the preferred option? When I run it I get:

"Exception: adb root failed"

Is it possible to run these tests without rooting your phone?

Thanks,

-Chris

[Roman Lavrov]

Hi Chris,

Good question! Running without root does appear to work at least on some tests I just tried.

Could you try commenting out _GetAdbRoot() from src/tests/py_utils/android_helper.py and see if you run into any issues?

[Ken Russell]

Aha. The root requirement's been problematic in other test harnesses as well. See the "dont-require-rooted-device" switch here:

https://chromium.googlesource.com/catapult/+/refs/heads/main/telemetry/docs/run_benchmarks_locally.md

Probably worth adding this as a command line option in ANGLE's scripts.

-Ken

To view this discussion on the web visit https://groups.google.com/d/msgid/angleproject/CAN6rRpZweBw%3DRNbfPaapYugyuS9bzs2QwyPQoMaNN5T4SbK4iw%40mail.gmail.com.

[Chris Dalton]

Thanks Ken and Roman,

Commenting out _GetAdbRoot() gets me further, but now my phone crashes trying to run the tests. Logcat below.

Have you ever seen a null dereference in ANativeWindow_setBuffersGeometry?

--------- beginning of crash
09-19 13:45:37.656 25849 25876 F libc    : Fatal signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x98 in tid 25876 (Thread-2), pid 25849 (st:test_process)
09-19 13:45:37.746 25880 25880 I crash_dump64: obtaining output fd from tombstoned, type: kDebuggerdTombstone
09-19 13:45:37.747   921   921 I tombstoned: received crash request for pid 25876
09-19 13:45:37.753 25880 25880 I crash_dump64: performing dump of process 25849 (target tid = 25876)
09-19 13:45:37.768 25880 25880 F DEBUG   : *** *** *** *** *** *** *** *** *** *** *** *** *** *** *** ***
09-19 13:45:37.769 25880 25880 F DEBUG   : Build fingerprint: 'google/crosshatch/crosshatch:11/RQ3A.210905.001/7511028:user/release-keys'
09-19 13:45:37.769 25880 25880 F DEBUG   : Revision: 'MP1.0'
09-19 13:45:37.769 25880 25880 F DEBUG   : ABI: 'arm64'
09-19 13:45:37.770 25880 25880 F DEBUG   : Timestamp: 2022-09-19 13:45:37-0600
09-19 13:45:37.770 25880 25880 F DEBUG   : pid: 25849, tid: 25876, name: Thread-2  >>> com.android.angle.test:test_process <<<
09-19 13:45:37.770 25880 25880 F DEBUG   : uid: 10255
09-19 13:45:37.770 25880 25880 F DEBUG   : signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x98
09-19 13:45:37.770 25880 25880 F DEBUG   : Cause: null pointer dereference
09-19 13:45:37.770 25880 25880 F DEBUG   :     x0  0000000000000000  x1  0000000000000080  x2  0000000000000080  x3  0000000000000000
09-19 13:45:37.770 25880 25880 F DEBUG   :     x4  000000756efb9b11  x5  000000756efb9a21  x6  655420454c474e41  x7  5345202d20737473
09-19 13:45:37.770 25880 25880 F DEBUG   :     x8  00000075eef53350  x9  0000000000000002  x10 0000000000000001  x11 0000000000000000
09-19 13:45:37.770 25880 25880 F DEBUG   :     x12 61636f4c6c657869  x13 656761726f74536c  x14 00000077c0d4dd22  x15 0000000029851c8f
09-19 13:45:37.770 25880 25880 F DEBUG   :     x16 00000074c3ef8438  x17 0000007526e5e078  x18 00000074c110a000  x19 000000756efb9770
09-19 13:45:37.770 25880 25880 F DEBUG   :     x20 00000074c3ef9800  x21 00000074c3ef9000  x22 00000074c3879000  x23 000000759f00dab0
09-19 13:45:37.770 25880 25880 F DEBUG   :     x24 00000074c3879000  x25 0000000000000000  x26 00000000000000c4  x27 00000000000000c3
09-19 13:45:37.770 25880 25880 F DEBUG   :     x28 0000000000000058  x29 00000074c16900d0
09-19 13:45:37.770 25880 25880 F DEBUG   :     lr  00000074c3ecaac8  sp  00000074c16900d0  pc  0000007526e5e088  pst 0000000080000000
09-19 13:45:37.772 25880 25880 F DEBUG   : backtrace:
09-19 13:45:37.772 25880 25880 F DEBUG   :       #00 pc 000000000001b088  /system/lib64/libandroid.so (ANativeWindow_setBuffersGeometry+16) (BuildId: 38312aef9bf39f6a76bef9f4e156a5a8)
09-19 13:45:37.772 25880 25880 F DEBUG   :       #01 pc 0000000000043ac4  /data/app/~~BhGtrv5HnpaGsWsAfiPrBg==/com.android.angle.test-Z9Vwb0hnaWXs0WIGN32ODQ==/lib/arm64/libangle_util.so (BuildId: 369d4b063a14b8ff)
09-19 13:45:38.066  3127  3151 D DeviceStateHelper: Audo mode: 0
09-19 13:45:38.067  3127  3151 D DeviceStateHelper: Audo mode: 0
09-19 13:45:38.543   921   921 E tombstoned: Tombstone written to: /data/tombstones/tombstone_09
09-19 13:45:38.545  1509 25883 W ActivityTaskManager:   Force finishing activity com.android.angle.test/.AngleUnitTestActivity

[Roman Lavrov]

I don't remember seeing this.

ANativeWindow_setBuffersGeometry is called from util/android/AndroidWindow.cpp, is it that sApp->window == nullptr or something else?..

Unless you're willing to try some tracing/debugging yourself, could you share your gn args, Android version and a command-line repro, ideally with --filter={single_test_name}?

[Chris Dalton]

The phone I was trying to run on was a little old and out of date.

With a newer, up to date phone, and commenting out the root step, run_angle_android_test.py with angle_end2end_tests works 🎉

Thanks for all the help!

[Roman Lavrov]

Great, thanks for the update Chris. I'll add some kind of flag to run without root shortly.

[Geoff Lang]

See also the android debugging doc for how to symbolize crashes or attach a debugger

To view this discussion on the web visit https://groups.google.com/d/msgid/angleproject/CAN6rRpaQQgEqpXP-cmOmns%2B%3Dh8JY6r%2BoW9Msq5P1DzjSRD2a%2BQ%40mail.gmail.com.

[Jamie Madill]

Roman, should we default root to off with your runner script? Maybe it's only needed for stuff like performance stabilization.

[Roman Lavrov]

We could try doing that, I'm just not aware of all the possible corner cases where we might need root, that's just something I didn't explore at all. The link that Ken posted above says:

"you can try running benchmarks with --compatibility-mode=dont-require-rooted-device switch, however this configuration may not be supported and you may run into errors"

Ideally if we want to actually support this we would run without root in some of our own use cases. Otherwise it'll just break at some point in future and we won't know.

[Jamie Madill]

I'd say remove all the root access completely to ensure we don't need it on the bots, then re-add it for use cases that may specifically need it like perf test stabilization. We want to require root as little as possible.

[Roman Lavrov]

https://anglebug.com/7681

[Chris Dalton]

Hello again,

I synced today and started getting errors from run_angle_android_test.py:

Traceback (most recent call last):

  File "/home/csmartdalton/code/angle/out/android/../../src/tests/run_angle_android_test.py", line 66, in <module>
    sys.exit(main())
  File "/home/csmartdalton/code/angle/out/android/../../src/tests/run_angle_android_test.py", line 48, in main
    tests = android_helper.ListTests(args.suite)
AttributeError: module 'android_helper' has no attribute 'ListTests'

Does this look familiar?

[Roman Lavrov]

Yeah oops, ListTests got lost in cleanup here: https://chromium-review.googlesource.com/c/angle/angle/+/3963651

You'd just need to revive the android_helper bit, I'll fix it soon.

[Chris Dalton]

Thanks Roman!

[Roman Lavrov]

np; should now be fixed by https://crrev.com/c/3975353

[Chris Dalton]

It works! I'm back in business. Thanks again Roman.