Where did the the Android crash log / tombstone go?

[Primiano Tucci]

Until not too long ago, crashes on unittest were spitting out a crash dump, even if unsymbolized.

Now I don't seem able to find them anymore.

Concrete case, this failure. The only useful signal I could find in the log is the stuff below. It seems that something is fetching the tombstone and uploading it somewhere (good). I cannot tell where though and how to read it. What am I missing?

I feel this is one of those cases where somebody will reply "you just have to click this link" and my reply will be "aaaaaaah thanks". 

I tried before sending this email and didn't manage to figure out where though.

I   33.659s run_tests_on_device(06b8018900622622)  [ RUN      ] CoordinationUnitImplTest.AddChild

I   33.659s run_tests_on_device(06b8018900622622)  [ CRASHED      ]

...

I   33.893s TimeoutThread-1-for-run_tests_on_device(06b8018900622622)  [host]> /b/swarm_slave/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 06b8018900622622 shell '( su -c sh -c '"'"'rm /data/tombstones/tombstone_00'"'"' );echo %$?'

I   33.926s run_tests_on_device(06b8018900622622)  tombstone_00 created on 2017-06-10 11:58:00, about this long ago: 7:00:05 Device: 06b8018900622622

I   33.926s run_tests_on_device(06b8018900622622)  Resolving...

I   34.642s run_tests_on_device(06b8018900622622)  Writing text to logdog stream, tombstones_20170610T115807_06b8018900622622

I   34.642s run_tests_on_device(06b8018900622622)  Getting logdog client.

I   34.643s TimeoutThread-1-for-run_tests_on_device(06088a760062351c)  [host]> /b/swarm_slave/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 06088a760062351c shell '( su -c sh -c '"'"'test -e /data/tombstones'"'"' );echo %$?'

I   34.675s TimeoutThread-1-for-run_tests_on_device(06088a760062351c)  [host]> /b/swarm_slave/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 06088a760062351c shell '( su -c sh -c '"'"'TZ=utc ls -a -l /data/tombstones/'"'"' );echo %$?'

W   34.707s run_tests_on_device(06088a760062351c)  No tombstones to clear.

I   34.709s TimeoutThread-1-for-run_tests_on_device(06088a760062351c)  [host]> /b/swarm_slave/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 06088a760062351c shell '( pm dump org.chromium.native_test | grep dataDir=; echo "PIPESTATUS: ${PIPESTATUS[@]}" );echo %$?'

I   35.876s run_tests_on_device(06088a760062351c)  flags:

I   35.878s TimeoutThread-1-for-run_tests_on_device(06088a760062351c)  [host]> /b/swarm_slave/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 06088a760062351c push /b/swarm_slave/w/itZ_DqC7/tmpZMkAhW /data/local/tmp/temp_file-dc1646874aab6

I   35.897s TimeoutThread-1-for-run_tests_on_device(06088a760062351c)  [host]> /b/swarm_slave/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 06088a760062351c shell '( ( p=org.chromium.native_test;am instrument -w -e "$p".NativeTestInstrumentationTestRunner.ShardNanoTimeout 120000000000 -e "$p".NativeTestInstrumentationTestRunner.NativeTestActivity "$p".NativeUnitTestActivity -e "$p".NativeTestInstrumentationTestRunner.TestList /data/local/tmp/temp_file-dc1646874aab6 -e "$p".NativeTestInstrumentationTestRunner.StdoutFile /storage/emulated/legacy/temp_file-ea9cb6d24783b.gtest_out "$p"/"$p".NativeTestInstrumentationTestRunner )>/data/local/tmp/temp_file-913be573fa447 );echo %$?'

I   37.055s TimeoutThread-1-for-run_tests_on_device(06088a760062351c)  [host]> /b/swarm_slave/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 06088a760062351c pull /data/local/tmp/temp_file-913be573fa447 /b/swarm_slave/w/itZ_DqC7/tmp7DYcKa/tmp_ReadFileWithPull

I   37.113s TimeoutThread-1-for-delete_temporary_file(06088a760062351c)  [host]> /b/swarm_slave/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 06088a760062351c shell 'rm -f /data/local/tmp/temp_file-913be573fa447'

I   37.119s TimeoutThread-1-for-run_tests_on_device(06088a760062351c)  [host]> /b/swarm_slave/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 06088a760062351c shell '( TZ=utc ls -a -l /storage/emulated/legacy/ );echo %$?'

I   37.186s run_tests_on_device(06088a760062351c)  /storage/emulated/legacy/temp_file-ea9cb6d24783b.gtest_out size on device: 5979

I   37.187s TimeoutThread-1-for-run_tests_on_device(06088a760062351c)  [host]> /b/swarm_slave/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 06088a760062351c shell '( TZ=utc ls -a -l /storage/emulated/legacy/ );echo %$?'

I   37.208s TimeoutThread-1-for-run_tests_on_device(06088a760062351c)  [host]> /b/swarm_slave/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 06088a760062351c shell '( cat /storage/emulated/legacy/temp_file-ea9cb6d24783b.gtest_out );echo %$?'

I   37.252s TimeoutThread-1-for-delete_temporary_file(06088a760062351c)  [host]> /b/swarm_slave/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 06088a760062351c shell 'rm -f /storage/emulated/legacy/temp_file-ea9cb6d24783b.gtest_out'

I   37.254s TimeoutThread-1-for-delete_temporary_file(06088a760062351c)  [host]> /b/swarm_slave/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 06088a760062351c shell 'rm -f /data/local/tmp/temp_file-dc1646874aab6'

I   37.265s TimeoutThread-1-for-delete_temporary_file(06088a760062351c)  [host]> /b/swarm_slave/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 06088a760062351c shell 'rm -f /data/data/org.chromium.native_test/temp_file-804d8dea8e4f.xml'

I   37.266s TimeoutThread-1-for-run_tests_on_device(06088a760062351c)  [host]> /b/swarm_slave/w/ir/third_party/android_tools/sdk/platform-tools/adb -s 06088a760062351c shell '( pm clear org.chromium.native_test );echo %$?'

I   37.880s run_tests_on_device(06088a760062351c)  >>ScopedMainEntryLogger

[John Budorick]

I was going to have a "you just click this link" bit, but it looks like we missed the merge spec for service_unittests here. (I'll send a CL to fix in a bit.)

In the meantime, your tombstone is here (and here, due to retries). It would typically be surfaced in the result_details link on the bot page. In this case, I pulled it from "shard #0 isolated out" -> output.json -> CTRL+F tombstones.

As I mentioned on a similar chromium-dev thread recently, hzl@ is working on symbolization.

--
You received this message because you are subscribed to the Google Groups "infra-dev" group.
To unsubscribe from this group and stop receiving emails from it, send an email to infra-dev+unsubscribe@chromium.org.
To post to this group, send email to infr...@chromium.org.
To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/infra-dev/CA%2ByH71fwHhTJ-gsmP0Jh%2BP09OonMu_-LDo9kXhiZ7UhRQweyaA%40mail.gmail.com.