CQ stats for the week of January 11
32 views
Skip to first unread message
Paweł Hajdan, Jr.
Jan 18, 2016, 9:23:16 AM1/18/16
to chromium-dev
Statistics for project chromium
excluding paths in the following set:
third_party/WebKit
from 2016-01-11 13:38:14.184620 till 2016-01-18 13:27:45.420720 (local time).
CQ users: 376 out of 381 total committers 98.69%
Committed 1084 out of 1117 commits 97.05%.
1157 issues (1428 patches) were tried by CQ, resulting in 1405 attempts.
962 patches (67.4% of tried patches, 68.5% of attempts) were committed by CQ,
False Rejections:
103 attempts (7.3% of 1405 attempts) were false rejections in 88 committed patches
62 attempts (4.4% of 1405 attempts) were infra false rejections in 58 committed patches
Patches which eventually land percentiles:
10: 0.1 hrs, 1 attempts, 0 tryjob retries, 0 global retry quota
25: 0.4 hrs, 1 attempts, 0 tryjob retries, 0 global retry quota
50: 0.8 hrs, 1 attempts, 0 tryjob retries, 0 global retry quota
75: 1.3 hrs, 1 attempts, 0 tryjob retries, 1 global retry quota
90: 1.9 hrs, 2 attempts, 1 tryjob retries, 3 global retry quota
95: 2.8 hrs, 2 attempts, 2 tryjob retries, 13 global retry quota
99: 4.3 hrs, 3 attempts, 3 tryjob retries, 56 global retry quota
max: 6.5 hrs, 5 attempts, 9 tryjob retries, 120 global retry quota
Per-day stats:
2016-01-11: 256 attempts; 50% 0.9; 90% 1.5; false rejections 0.6% (0.0% infra)
2016-01-12: 257 attempts; 50% 0.8; 90% 1.6; false rejections 0.4% (0.1% infra)
2016-01-13: 341 attempts; 50% 1.2; 90% 3.7; false rejections 3.5% (2.7% infra)
2016-01-14: 328 attempts; 50% 0.6; 90% 1.5; false rejections 0.7% (0.1% infra)
2016-01-15: 186 attempts; 50% 0.5; 90% 1.4; false rejections 0.4% (0.3% infra)
2016-01-16: 9 attempts; 50% 1.2; 90% 1.4; false rejections 0.0% (0.0% infra)
2016-01-17: 26 attempts; 50% 0.9; 90% 1.4; false rejections 0.0% (0.0% infra)
Slowest CLs:
Top flaky builders (which fail and succeed in the same patch):
Master Builder Flakes |Infra |Compile|Test |Invalid|Patch |Other
chromium.mac mac_chromium_rel_ng 157/1680 ( 9%)| 69%| 6%| 22%| 1%| 3%| 0%
blink linux_blink_rel 4/ 43 ( 9%)| 0%| 0%| 100%| 0%| 0%| 0%
chromium.linux linux_chromium_rel_ng 127/1676 ( 8%)| 42%| 2%| 9%| 1%| 2%| 45%
chromium.linux linux_chromium_asan_rel_ng 79/1589 ( 5%)| 63%| 6%| 20%| 0%| 5%| 5%
chromium.linux chromium_presubmit 72/1777 ( 4%)| 15%| 0%| 0%| 0%| 4%| 81%
chromium.linux linux_chromium_chromeos_rel_ng 63/1581 ( 4%)| 65%| 6%| 13%| 0%| 6%| 10%
chromium.android android_clang_dbg_recipe 63/1586 ( 4%)| 65%| 25%| 0%| 0%| 8%| 2%
chromium.mac ios_dbg_simulator_ninja 56/1574 ( 4%)| 71%| 0%| 0%| 0%| 7%| 21%
chromium.android android_chromium_gn_compile_dbg 54/1561 ( 3%)| 83%| 6%| 0%| 0%| 9%| 2%
chromium.android android_chromium_gn_compile_rel 51/1559 ( 3%)| 80%| 10%| 0%| 0%| 10%| 0%
chromium.linux linux_chromium_chromeos_ozone_rel_ng 51/1576 ( 3%)| 76%| 8%| 2%| 0%| 8%| 6%
chromium.android android_arm64_dbg_recipe 49/1554 ( 3%)| 80%| 10%| 0%| 0%| 10%| 0%
chromium.android cast_shell_android 48/1559 ( 3%)| 90%| 0%| 0%| 0%| 10%| 0%
chromium.android android_compile_dbg 42/1554 ( 3%)| 86%| 2%| 0%| 0%| 12%| 0%
chromium.android linux_android_rel_ng 34/1570 ( 2%)| 44%| 6%| 29%| 3%| 15%| 3%
chromium.mac ios_rel_device_ninja 31/1546 ( 2%)| 77%| 0%| 0%| 0%| 13%| 10%
chromium.win win_chromium_rel_ng 26/1539 ( 2%)| 35%| 12%| 31%| 8%| 15%| 0%
chromium.mac mac_chromium_10.10_rel_ng 25/1485 ( 2%)| 72%| 16%| 0%| 0%| 12%| 0%
chromium.linux linux_chromium_clobber_rel_ng 24/1533 ( 2%)| 38%| 12%| 0%| 0%| 17%| 33%
chromium.linux chromeos_x86-generic_chromium_compile_only_ng 23/1535 ( 1%)| 39%| 22%| 0%| 0%| 17%| 22%
chromium.mac mac_chromium_gn_rel 22/1553 ( 1%)| 41%| 41%| 0%| 0%| 18%| 0%
chromium.linux chromeos_amd64-generic_chromium_compile_only_ng 20/1528 ( 1%)| 35%| 25%| 0%| 0%| 20%| 20%
chromium.linux linux_chromium_compile_dbg_ng 18/1530 ( 1%)| 44%| 11%| 0%| 0%| 22%| 22%
chromium.linux linux_chromium_gn_chromeos_rel 17/1534 ( 1%)| 53%| 18%| 0%| 0%| 24%| 6%
chromium.linux linux_chromium_chromeos_compile_dbg_ng 15/1527 ( 1%)| 53%| 20%| 0%| 0%| 27%| 0%
chromium.linux chromeos_daisy_chromium_compile_only_ng 15/1531 ( 1%)| 47%| 27%| 0%| 0%| 27%| 0%
chromium.mac mac_chromium_compile_dbg_ng 15/1533 ( 1%)| 40%| 33%| 0%| 0%| 27%| 0%
chromium.linux cast_shell_linux 15/1550 ( 1%)| 73%| 0%| 0%| 0%| 27%| 0%
chromium.mac mac_chromium_asan_rel_ng 2/ 221 ( 1%)| 50%| 0%| 50%| 0%| 0%| 0%
chromium.linux linux_chromium_chromeos_asan_rel_ng 2/ 221 ( 1%)| 100%| 0%| 0%| 0%| 0%| 0%
chromium.win win_chromium_compile_dbg_ng 12/1524 ( 1%)| 33%| 33%| 0%| 0%| 33%| 0%
chromium.win win_chromium_x64_rel_ng 12/1528 ( 1%)| 33%| 17%| 8%| 8%| 33%| 0%
chromium.win win8_chromium_ng 8/1525 ( 1%)| 12%| 38%| 0%| 0%| 50%| 0%
chromium.linux linux_chromium_compile_dbg_32_ng 7/1519 ( 0%)| 0%| 43%| 0%| 0%| 57%| 0%
chromium.win win_clang_x64_dbg 0/ 1 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.mac mac_chromium_gn_dbg 0/ 2 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
webrtc win_drmemory_full 0/ 2 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
webrtc mac_asan 0/ 1 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
blink linux_blink_dbg 0/ 2 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
blink mac_blink_oilpan_compile_rel 0/ 1 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.android android_chromium_gn_rel 0/ 2 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
blink mac_blink_dbg 0/ 2 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.linux linux_site_isolation 0/ 5 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.win win_chromium_xp_rel_ng 0/ 1 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
blink win_blink_dbg 0/ 2 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
blink win_blink_oilpan_compile_rel 0/ 1 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.win win_chromium_vista_rel_ng 0/ 1 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.linux android_chromium_gn_compile_dbg 0/ 2 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.linux android_compile_dbg 0/ 2 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.win win_chromium_compile_dbg_ng_exp 0/ 214 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.mac mac_chromium_10.6_rel_ng 0/ 1 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.win win_drmemory 0/ 1 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.linux linux_chromium_msan_rel_ng 0/ 4 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
webrtc linux_asan 0/ 1 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
blink win_blink_oilpan_dbg 0/ 1 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.win win_clang_x64_rel 0/ 1 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
blink win_blink_rel 0/ 3 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.win win_chromium_compile_dbg 0/ 1 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.win win_chromium_syzyasan_rel 0/ 1 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
blink linux_blink_compile_dbg 0/ 1 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.linux linux_android_rel_ng 0/ 4 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
webrtc win_asan 0/ 2 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
blink linux_blink_oilpan_compile_rel 0/ 1 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.win win_clang_rel 0/ 1 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.linux linux_arm 0/ 4 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.linux linux_chromium_dbg_ng 0/ 5 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
blink mac_blink_rel 0/ 3 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.linux android_chromium_gn_compile_rel 0/ 2 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.linux linux_blink_oilpan_rel 0/ 216 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
blink mac_blink_oilpan_dbg 0/ 2 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
blink linux_blink_compile_rel 0/ 1 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.linux android_arm64_dbg_recipe 0/ 2 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.mac mac_chromium_dbg_ng 0/ 5 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.android linux_android_dbg_ng 0/ 6 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.linux cast_shell_android 0/ 2 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.win win_chromium_gn_x64_dbg 0/ 1 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.linux android_clang_dbg_recipe 0/ 2 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.linux linux_chromium_chromeos_dbg_ng 0/ 3 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
webrtc win_drmemory_light 0/ 2 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.win win_optional_gpu_tests_rel 0/ 2 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
libyuv linux_asan 0/ 0 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.win win_clang_dbg 0/ 2 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.win win_chromium_dbg_ng 0/ 2 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
blink blink_presubmit 0/ 1 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.win win10_chromium_rel_ng 0/ 4 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
chromium.linux linux_chromium_tsan_rel_ng 0/ 1 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
blink linux_blink_oilpan_dbg 0/ 2 ( 0%)| 0%| 0%| 0%| 0%| 0%| 0%
Paweł
John Abd-El-Malek
Jan 20, 2016, 10:25:30 AM1/20/16
to Paweł Hajdan Jr., chromium-dev
The false rejections section above says 7.3% were false rejections. But the daily stats are all < .6% except one day that is 3.5%. How do we reconcile these differences?
--
--
Chromium Developers mailing list: chromi...@chromium.org
View archives, change email options, or unsubscribe:
http://groups.google.com/a/chromium.org/group/chromium-dev
Paweł Hajdan, Jr.
Jan 21, 2016, 10:22:12 AM1/21/16
to John Abd-El-Malek, chromium-dev
I'm glad you asked about this.
A false rejection is a failed CQ attempt, where a later attempt for the same patch was successful.
While we can know immediately that given CQ attempt is a rejection (because it failed), we only know it's a false rejection when (if at all) the patch eventually lands. The delay between these two events might be arbitrarily long. Furthermore, there may be multiple rejections before the final successful attempt. Also, the metric is based on CQ attempts within time window requested for stats, i.e. not necessarily all attempts for given issue+patchset.
This means that a larger window (i.e. weekly) can contain more rejections (and more attempts in general), than a smaller one (daily). It also has a higher chance of containing a successful attempt, since it doesn't necessarily have to be on the same day as the previous attempts.
WDYT?
Paweł
John Abd-El-Malek
Jan 21, 2016, 7:02:11 PM1/21/16
to Paweł Hajdan, Jr., chromium-dev
Got it, that makes sense.
In that case, the daily flakiness stats are not that useful to look at?
Dirk Pranke
Jan 21, 2016, 7:16:30 PM1/21/16
to John Abd-El-Malek, Paweł Hajdan, Jr., chromium-dev
Does it make sense to calculate the flakiness rate different, such that we compute the number of CLs that land in a given window of time, and the number of those CLs that had flaked one or more times prior to landing? That way the numbers would be stable.
That would not measure the number of flaky jobs that haven't yet landed, but I'm not sure if that number is either all that interesting or actually much different over time from the number that do land.
-- Dirk
Paweł Hajdan, Jr.
Jan 22, 2016, 7:02:00 AM1/22/16
to Dirk Pranke, John Abd-El-Malek, chromium-dev
On Fri, Jan 22, 2016 at 1:00 AM, John Abd-El-Malek <j...@chromium.org> wrote:
Got it, that makes sense.In that case, the daily flakiness stats are not that useful to look at?
Not sure. That might be too pessimistic. In this case, the daily stats still show a spike of false rejection on the day there was an outage. We also have per-day cycle time stats. I hope this can be useful for you somehow.
On Fri, Jan 22, 2016 at 1:15 AM, Dirk Pranke <dpr...@chromium.org> wrote:
Does it make sense to calculate the flakiness rate different, such that we compute the number of CLs that land in a given window of time, and the number of those CLs that had flaked one or more times prior to landing? That way the numbers would be stable.
The numbers may actually become too stable with that change. In May 2015 I landed https://codereview.chromium.org/1158723003 which makes cq_stats only take into account CQ attempts within requested date range. The rationale was that otherwise an outage or significant outburst of flakiness would affect stats for at least a couple of days after the event (even after it was actually fixed), potentially masking other regressions, and not really useful to determine whether issues are over or not. In practice, with above change it's much easier to answer questions like that based on daily CQ stats. They became more "responsive" to the current state of CQ.
Of course we can give it a try and see how historical stats look like with such a change applied. WDYT?
That would not measure the number of flaky jobs that haven't yet landed, but I'm not sure if that number is either all that interesting or actually much different over time from the number that do land.
FWIW, is such a metric ("flaky jobs that haven't yet landed") computable? Before the change lands, we don't really know (in an automated way) if the rejection is real or a flake, right?
Paweł
Dirk Pranke
Jan 25, 2016, 9:28:30 PM1/25/16
to Paweł Hajdan, Jr., John Abd-El-Malek, chromium-dev
On Fri, Jan 22, 2016 at 4:00 AM, Paweł Hajdan, Jr. <phajd...@chromium.org> wrote:
On Fri, Jan 22, 2016 at 1:00 AM, John Abd-El-Malek <j...@chromium.org> wrote:Got it, that makes sense.In that case, the daily flakiness stats are not that useful to look at?Not sure. That might be too pessimistic. In this case, the daily stats still show a spike of false rejection on the day there was an outage. We also have per-day cycle time stats. I hope this can be useful for you somehow.On Fri, Jan 22, 2016 at 1:15 AM, Dirk Pranke <dpr...@chromium.org> wrote:Does it make sense to calculate the flakiness rate different, such that we compute the number of CLs that land in a given window of time, and the number of those CLs that had flaked one or more times prior to landing? That way the numbers would be stable.The numbers may actually become too stable with that change. In May 2015 I landed https://codereview.chromium.org/1158723003 which makes cq_stats only take into account CQ attempts within requested date range. The rationale was that otherwise an outage or significant outburst of flakiness would affect stats for at least a couple of days after the event (even after it was actually fixed), potentially masking other regressions, and not really useful to determine whether issues are over or not. In practice, with above change it's much easier to answer questions like that based on daily CQ stats. They became more "responsive" to the current state of CQ.
That's a fair point, but it seems like we should recover from most outages within a given day of the fix landing, and so I wouldn't expect the stats to skew too much, and so it seems like having the numbers be stable is perhaps a better tradeoff to me. But that's just my thought ...
Of course we can give it a try and see how historical stats look like with such a change applied. WDYT?That would not measure the number of flaky jobs that haven't yet landed, but I'm not sure if that number is either all that interesting or actually much different over time from the number that do land.FWIW, is such a metric ("flaky jobs that haven't yet landed") computable? Before the change lands, we don't really know (in an automated way) if the rejection is real or a flake, right?
Umm, true :). I'm not sure at this point if I was thinking of something else or what.
-- Dirk
Paweł Hajdan, Jr.
Feb 1, 2016, 10:09:48 AM2/1/16
to Dirk Pranke, John Abd-El-Malek, chromium-dev
John, actually you're right. There was a bug in the way false rejection rate was calculated in per-day stats (fixed in https://codereview.chromium.org/1656793002).
Here are the updated stats (note that slight differences in other numbers are caused by a different time window, i.e. at which hour the script was called; that's also something we could fix):
Statistics for project chromium
excluding paths in the following set:
third_party/WebKit
from 2016-01-11 23:07:04.428280 till 2016-01-18 22:36:05.646000 (local time).
CQ users: 356 out of 359 total committers 99.16%
Committed 1001 out of 1032 commits 97.00%.
1086 issues (1334 patches) were tried by CQ, resulting in 1314 attempts.
900 patches (67.5% of tried patches, 68.5% of attempts) were committed by CQ,
False Rejections:
99 attempts (7.5% of 1314 attempts) were false rejections in 85 committed patches
62 attempts (4.7% of 1314 attempts) were infra false rejections in 58 committed patches
Patches which eventually land percentiles:
10: 0.1 hrs, 1 attempts, 0 tryjob retries, 0 global retry quota
25: 0.4 hrs, 1 attempts, 0 tryjob retries, 0 global retry quota
50: 0.8 hrs, 1 attempts, 0 tryjob retries, 0 global retry quota
75: 1.3 hrs, 1 attempts, 0 tryjob retries, 1 global retry quota
90: 2.0 hrs, 2 attempts, 1 tryjob retries, 3 global retry quota
95: 2.9 hrs, 2 attempts, 2 tryjob retries, 13 global retry quota
99: 4.3 hrs, 3 attempts, 3 tryjob retries, 58 global retry quota
max: 6.5 hrs, 5 attempts, 9 tryjob retries, 120 global retry quota
Per-day stats:
2016-01-11: 271 attempts; 50% 0.7; 90% 1.5; false rejections 2.2% ( 0.0% infra)
2016-01-12: 234 attempts; 50% 0.8; 90% 1.4; false rejections 1.7% ( 0.4% infra)
2016-01-13: 402 attempts; 50% 1.2; 90% 2.8; false rejections 16.4% (13.2% infra)
2016-01-14: 286 attempts; 50% 0.5; 90% 1.3; false rejections 5.9% ( 2.1% infra)
2016-01-15: 63 attempts; 50% 0.7; 90% 1.6; false rejections 0.0% ( 0.0% infra)
2016-01-16: 6 attempts; 50% 0.7; 90% 1.3; false rejections 0.0% ( 0.0% infra)
2016-01-17: 51 attempts; 50% 0.9; 90% 1.5; false rejections 0.0% ( 0.0% infra)
Dirk, FYI this is the effect of undoing https://codereview.chromium.org/1158723003 locally (note e.g. slightly larger number of attempts).
Actually it may be a good idea to do this, since it can e.g. give us proper stats for CLs landed on Mondays.
Also note earlier (internal) thread about marking attempts during weekends as unsupported, i.e. not counting towards stats (https://goto.google.com/pmpnb).
Statistics for project chromium
excluding paths in the following set:
third_party/WebKit
from 2016-01-11 23:07:04.428280 till 2016-01-18 22:36:05.646000 (local time).
CQ users: 356 out of 359 total committers 99.16%
Committed 1001 out of 1032 commits 97.00%.
1086 issues (1334 patches) were tried by CQ, resulting in 1338 attempts.
900 patches (67.5% of tried patches, 67.3% of attempts) were committed by CQ,
False Rejections:
117 attempts (8.7% of 1338 attempts) were false rejections in 89 committed patches
76 attempts (5.7% of 1338 attempts) were infra false rejections in 59 committed patches
Patches which eventually land percentiles:
10: 0.1 hrs, 1 attempts, 0 tryjob retries, 0 global retry quota
25: 0.4 hrs, 1 attempts, 0 tryjob retries, 0 global retry quota
50: 0.8 hrs, 1 attempts, 0 tryjob retries, 0 global retry quota
75: 1.3 hrs, 1 attempts, 0 tryjob retries, 1 global retry quota
90: 2.2 hrs, 2 attempts, 1 tryjob retries, 3 global retry quota
95: 3.0 hrs, 2 attempts, 2 tryjob retries, 13 global retry quota
99: 4.5 hrs, 3 attempts, 3 tryjob retries, 58 global retry quota
max: 6.5 hrs, 15 attempts, 23 tryjob retries, 120 global retry quota
Per-day stats:
2016-01-11: 294 attempts; 50% 0.8; 90% 1.6; false rejections 8.2% ( 4.8% infra)
2016-01-12: 242 attempts; 50% 0.8; 90% 1.5; false rejections 2.1% ( 0.4% infra)
2016-01-13: 405 attempts; 50% 1.3; 90% 3.6; false rejections 16.5% (13.1% infra)
2016-01-14: 300 attempts; 50% 0.5; 90% 1.4; false rejections 6.7% ( 2.7% infra)
2016-01-15: 65 attempts; 50% 0.7; 90% 2.2; false rejections 0.0% ( 0.0% infra)
2016-01-16: 6 attempts; 50% 0.7; 90% 1.3; false rejections 0.0% ( 0.0% infra)
2016-01-17: 52 attempts; 50% 0.9; 90% 1.5; false rejections 1.9% ( 0.0% infra)
Paweł
Reply all
Reply to author
Forward
0 new messages